Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The Probable Future of AI and Security

The Probable Future of AI and Security

A talk at the 2024 API and Application Security Summit in Austin, Texas.

James Wickett

May 15, 2024

More Decks by James Wickett

Other Decks in Programming


  1. DevOps is the inevitable result of needing to do efficient

    operations in a [distributed computing and cloud] environment — Tom Limoncelli
  2. Reality in 2009 in Ops Lots of event and time

    series data Outnumbered 10 to 1 by devs Complex Needs: network, cooling, compute, ... Limited dev experience
  3. Current Reality in Security Lots of data (documents,reports,output) Chronic under-staffing

    (100/10/1) Complex Needs: scanning, audits, training, ... CI/CD, IaC, 10+ Languages
  4. Clearly something is wrong. [...] We’re protecting the wrong things,

    and we’re hurting productivity in the process. — Steven Bellovin
  5. Penalties of the Shift Left Added new gates (and complexity)

    Backlog bloat Devs decoding output Pipeline cluttering
  6. What makes AppSec particularly difficult Noise to signal ratio Expert

    analysis needed Loads of documents Specificity and detail
  7. AppSec Context Who wrote the code? What does the app

    do? What are app areas that are important? Are there any critical functions? Did the developer pass secure code training? Is the code brittle in certain areas?
  8. Contextual Security Analysis Contextual Security Analysis uses all available context

    gathered as developers are writing code to make contextually aware assertions.
  9. Security Context You Already Have Commit or PR Author Codepaths

    & Functions Dependencies Sectool findings Past problem areas
  10. Factors of Contextual Security Analysis Surface - how the surface

    of the application changes Language - the language and framework the app is written in Intent - evaluates the person making the change, both in their patterns and their purpose Detection - the output from sectools to detect vulnerabilities Environment - the purpose of the app or service in the organization
  11. Last year we created a functional attack with ChatGPT Created

    a working XSS payload Found the user's active session Made a lambda receiver to exfil tokens Emailed myself the active session token source: https://speakerdeck.com/wickett/context-over- control-delivering-security-value-to-the-team
  12. LAST Prompt You are an application security expert, skilled in

    explaining complex programming vulnerabilities with simplicity. You will receive the full code for an application. Your task is to review the code for security vulnerabilities and suggest improvements. Don't overly focus on one file, and instead provide the top security concerns based on what you think the entire application is doing. https://github.com/latiotech/LAST/blob/main/src/latio/ core.py
  13. CO-STAR Prompting Context Output Specificity & Style Task & Tone

    Assumption & Audience Requirements & Response https://www.linkedin.com/pulse/ prompt-engineering-deep-dive- mastering-co-star-framework- mittal-xlqhe/
  14. # IDENTITY and PURPOSE You are an expert at summarizing

    pull requests to a given coding project. https://raw.githubusercontent.com/danielmiessler/ fabric/main/patterns/summarize_pull-requests/ system.md
  15. # STEPS 1. Create a section called SUMMARY: and place

    a one-sentence summary of the types of pull requests that have been made to the repository. 2. Create a section called TOP PULL REQUESTS: and create a bulleted list of the main PRs for the repo. https://raw.githubusercontent.com/danielmiessler/ fabric/main/patterns/summarize_pull-requests/ system.md
  16. OUTPUT EXAMPLE: SUMMARY: Most PRs on this repo have to

    do with troubleshooting the app's dependencies, cleaning up documentation, and adding features to the client. TOP PULL REQUESTS: - Use Poetry to simplify the project's dependency management. - Add a section that explains how to use the app's secondary API. - A request to add AI Agent endpoints that use CrewAI. - Etc. END EXAMPLE
  17. Academic Research (with some controversy) LLM's can autonomously hack websites

    https://arxiv.org/pdf/ 2402.06664 LLM Agents can Autonomously Exploit One-day Vulnerabilities https://arxiv.org/pdf/2404.08144 The controversy https:// struct.github.io/ autoagents1_day.html
  18. Bonus because you stayed to the end! The DevOps Painter

    GPT https://chat.openai.com/g/g- OatLRsfo1-the-devops-painter