The AI Future of Information Security @ ADDO

DevSecOps hasn't worked out nearly as well as the original move of DevOps. In this talk, we'll explore why that is and, more importantly, show how the current changing dynamics in technology now favor a new era of security. By leveraging AI, we're seeing a radical transformation of tried-and-true security practices from code scanning and risk assessments to threat modeling and API defense.

This talk will cover how top performers are leveraging the latest models, prompting techniques, and research to radically change how infosec is contributing value to the organization. This won't be just a history lesson or a theory talk about AI and security, but instead, you'll leave with pragmatic tips, tools, and techniques for this next era.

This talk was presented at All Day DevOps (ADDO)

Transcript

1. The AI Future of Information Security
2. James Wickett Co-founder & CEO @ DryRun Security LinkedIn Learning Author Austin, TX @wickett
3. The cloud was to Ops as AI will be to Security
4. First, Agile Agile needed ops It took ops a decade (and a computing revolution)
5. Devs and Security didn't exactly get along
6. DevOps is the inevitable result of needing to do efficient operations in a [distributed computing and cloud] environment — Tom Limoncelli
7. Agile, IaC, Virtualization, and Cloud were all necessary pre- existing conditions
8. We didn't choose the devops life, the devops life chose us
9. Reality in 2009 in Ops Lots of event and time series data Outnumbered 10 to 1 by devs Complex Needs: network, cooling, compute, ... Limited dev experience
10. The First DevOpsDays
11. John Willis would drop into your conference and teach you IaC
12. Ops != Security
13. Current Reality in Security Lots of data (documents,reports,output) Chronic under-staffing (100/10/1) Complex Needs: scanning, audits, training, ... CI/CD, IaC, 10+ Languages
14. Clearly something is wrong. [...] We’re protecting the wrong things, and we’re hurting productivity in the process. — Steven Bellovin
15. The First DevSecOps Event
16. Dear Shift Left, We Never Knew You
17. Penalties of the Shift Left Added new gates (and complexity) Backlog bloat Devs decoding output Pipeline cluttering
18. What makes AppSec particularly difficult Noise to signal ratio Expert analysis needed Loads of documents Specificity and detail Lots of pattern-matching and rule-writing
19. How we imagine AppSec training
20. How it actually goes
21. How it ends up
22. AppSec Context Who wrote the code? What does the app do? What are app areas that are important? Are there any critical functions? Did the developer pass secure code training? Is the code brittle in certain areas?
23. Contextual Security Analysis Contextual Security Analysis uses all available context gathered as developers are writing code to make contextually aware assertions.
24. Security Context You Already Have Commit or PR Author Codepaths & Functions Dependencies Sectool findings Past problem areas
25. Factors of Contextual Security Analysis Surface - how the surface of the application changes Language - the language and framework the app is written in Intent - evaluates the person making the change, both in their patterns and their purpose Detection - the output from sectools to detect vulnerabilities Environment - the purpose of the app or service in the organization
26. Stuff AI Does Well Transforming Summarization Information extraction Generation
27. The cloud was to Ops as AI will be to Security
28. CSPM is the solution we deserve, but not the solution we ever wanted
29. Take these SAST results to Mount JIRA
30. Life with AI in Information Security Stop security burden for engineering Augment security to reduce their toil Insight for security into engineering
37. GPT-4 vs. SAST https://www.youtube.com/watch?v=ATN2bJizTFE&t=6s
38. https://github.com/latiotech/LAST
39. LAST Prompt You are an application security expert, skilled in explaining complex programming vulnerabilities with simplicity. You will receive the full code for an application. Your task is to review the code for security vulnerabilities and suggest improvements. Don't overly focus on one file, and instead provide the top security concerns based on what you think the entire application is doing. https://github.com/latiotech/LAST/blob/main/src/latio/ core.py
40. CO-STAR Prompting Context Output Specificity & Style Task & Tone Assumption & Audience Requirements & Response https://www.linkedin.com/pulse/ prompt-engineering-deep-dive- mastering-co-star-framework- mittal-xlqhe/
41. https://chat.openai.com/g/g-HTsfg2w2z-secgpt
42. https://github.com/mrwadams/stride-gpt
43. https://github.com/danielmiessler/fabric
44. # IDENTITY and PURPOSE You are an expert at summarizing pull requests to a given coding project. https://raw.githubusercontent.com/danielmiessler/ fabric/main/patterns/summarize_pull-requests/ system.md
45. # STEPS 1. Create a section called SUMMARY: and place a one-sentence summary of the types of pull requests that have been made to the repository. 2. Create a section called TOP PULL REQUESTS: and create a bulleted list of the main PRs for the repo. https://raw.githubusercontent.com/danielmiessler/ fabric/main/patterns/summarize_pull-requests/ system.md
46. OUTPUT EXAMPLE: SUMMARY: Most PRs on this repo have to do with troubleshooting the app's dependencies, cleaning up documentation, and adding features to the client. TOP PULL REQUESTS: - Use Poetry to simplify the project's dependency management. - Add a section that explains how to use the app's secondary API. - A request to add AI Agent endpoints that use CrewAI. - Etc. END EXAMPLE
47. The cloud was to Ops as AI will be to Security
48. Academic Research (with some controversy) LLM's can autonomously hack websites https://arxiv.org/pdf/ 2402.06664 LLM Agents can Autonomously Exploit One-day Vulnerabilities https://arxiv.org/pdf/2404.08144 The controversy https:// struct.github.io/ autoagents1_day.html
49. AI Sec Research (more promising) https://arxiv.org/pdf/2312.00024
50. Security's Data Documents Policies Procedures and Controls Architecture decisions Tool output Threat models Industry Standards
51. Security's Data Events Changes to all the previous categories Code changes Infra changes Industry news
52. The cloud was to Ops as AI will be to Security
53. Bonus because you stayed to the end! The DevOps Painter GPT https://chat.openai.com/g/g- OatLRsfo1-the-devops-painter
54. Contextual Security Analysis Guide dryrun.security/csa
55. Want to chat more about AI and Security? wickett@dryrun.security linkedin.com/in/wickett @wickett