The AI Future of AppSec

Talk delivered at BlackHat 2024 in Las Vegas.

Transcript

1. The AI Future of Application Security
3. James Wickett Co-founder & CEO @ DryRun Security LinkedIn Learning Author Austin, TX @wickett
4. The cloud was to Ops as AI will be to Security
5. First, Agile Agile needed ops It took ops a decade (and a computing revolution)
6. Devs and Security didn't exactly get along
7. DevOps is the inevitable result of needing to do efficient operations in a [distributed computing and cloud] environment — Tom Limoncelli
8. Agile, IaC, Virtualization, and Cloud were all necessary pre- existing conditions
9. We didn't choose the devops life, the devops life chose us
10. Reality in 2009 in Ops Lots of event and time series data Outnumbered 10 to 1 by devs Complex Needs: network, cooling, compute, ... Limited dev experience
11. The First DevOpsDays
12. DevOps leaders would drop into your conference and teach you IaC
13. Ops != Security
14. Current Reality in Security Lots of data (documents,reports,output) Chronic under-staffing (100/10/1) Complex Needs: scanning, audits, training, ... CI/CD, IaC, 10+ Languages
15. The rise of the xSPM is a symptom of security's data problem
16. Clearly something is wrong. [...] We’re protecting the wrong things, and we’re hurting productivity in the process. — Steven Bellovin
17. The First DevSecOps Event
18. Take these SAST results to Mount JIRA
19. Dear Shift Left, We Never Knew You
20. Penalties of the Shift Left Added new gates (and complexity) Backlog bloat Devs decoding output Pipeline cluttering
21. What makes AppSec particularly difficult Noise to signal ratio Expert analysis needed Loads of documents Specificity and detail Lots of pattern-matching and rule-writing
22. How we imagine AppSec training
23. How it actually goes
24. How it ends up
25. AppSec Context Who wrote the code? What does the app do? What are app areas that are important? Are there any critical functions? Did the developer pass secure code training? Is the code brittle in certain areas?
26. Contextual Security Analysis Contextual Security Analysis uses all available context gathered as developers are writing code to make contextually aware assertions.
27. Security Context You Already Have Commit or PR Author Codepaths & Functions Dependencies Sectool findings Past problem areas
28. Factors of Contextual Security Analysis Surface - how the surface of the application changes Language - the language and framework the app is written in Intent - evaluates the person making the change, both in their patterns and their purpose Detection - the output from sectools to detect vulnerabilities Environment - the purpose of the app or service in the organization
29. Stuff AI Does Well Transforming Summarization Information extraction Generation
30. The cloud was to Ops as AI will be to Security
31. Last year we created a functional attack with ChatGPT Created a working XSS payload Found the user's active session Made a lambda receiver to exfil tokens Emailed myself the active session token source: https://speakerdeck.com/wickett/context-over- control-delivering-security-value-to-the-team
32. GPT-4 vs. SAST https://www.youtube.com/watch?v=ATN2bJizTFE&t=6s
33. https://github.com/latiotech/LAST
34. LAST Prompt You are an application security expert, skilled in explaining complex programming vulnerabilities with simplicity. You will receive the full code for an application. Your task is to review the code for security vulnerabilities and suggest improvements. Don't overly focus on one file, and instead provide the top security concerns based on what you think the entire application is doing. https://github.com/latiotech/LAST/blob/main/src/latio/ core.py
35. CO-STAR Prompting Context Output Specificity & Style Task & Tone Assumption & Audience Requirements & Response https://www.linkedin.com/pulse/ prompt-engineering-deep-dive- mastering-co-star-framework- mittal-xlqhe/
36. https://chat.openai.com/g/g-HTsfg2w2z-secgpt
37. https://github.com/mrwadams/stride-gpt
38. https://github.com/danielmiessler/fabric
39. # IDENTITY and PURPOSE You are an expert at summarizing pull requests to a given coding project. https://raw.githubusercontent.com/danielmiessler/ fabric/main/patterns/summarize_pull-requests/ system.md
40. # STEPS 1. Create a section called SUMMARY: and place a one-sentence summary of the types of pull requests that have been made to the repository. 2. Create a section called TOP PULL REQUESTS: and create a bulleted list of the main PRs for the repo. https://raw.githubusercontent.com/danielmiessler/ fabric/main/patterns/summarize_pull-requests/ system.md
41. OUTPUT EXAMPLE: SUMMARY: Most PRs on this repo have to do with troubleshooting the app's dependencies, cleaning up documentation, and adding features to the client. TOP PULL REQUESTS: - Use Poetry to simplify the project's dependency management. - Add a section that explains how to use the app's secondary API. - A request to add AI Agent endpoints that use CrewAI. - Etc. END EXAMPLE
42. What we've been up to at DryRun Security
53. The cloud was to Ops as AI will be to Security
54. Academic Research (with some controversy) LLM's can autonomously hack websites https://arxiv.org/pdf/ 2402.06664 LLM Agents can Autonomously Exploit One-day Vulnerabilities https://arxiv.org/pdf/2404.08144 The controversy https:// struct.github.io/ autoagents1_day.html
55. AI Sec Research (more promising) https://arxiv.org/pdf/2312.00024
56. Security's Data Documents Policies Procedures and Controls Architecture decisions Tool output Threat models Industry Standards
57. Security's Data Events Changes to all the previous categories Code changes Infra changes Industry news
58. The cloud was to Ops as AI will be to Security
59. And you probably thought I was going to talk about xz-utils
60. Get the Free Contextual Security Analysis Guide dryrun.security/csa
61. Stay in Touch wickett@dryrun.security linkedin.com/in/wickett @wickett
62. Bonus because you stayed to the end! The DevOps Painter GPT https://chat.openai.com/g/g- OatLRsfo1-the-devops-painter