Upgrade to Pro — share decks privately, control downloads, hide ads and more …

The Good, The Bad and The Ugly of AI in Security

The Good, The Bad and The Ugly of AI in Security

Avatar for Olimpiu Pop

Olimpiu Pop

June 26, 2024

More Decks by Olimpiu Pop

Other Decks in Technology

Transcript

  1. 🤖The Good, The Bad and The Ugly of AI in

    Security DevoxxPL - June 2024, Olimpiu Pop
  2. Part I: Debunking your POV: “The Good, The Bad and

    The Ugly” History 🥱 + Challenges A (Gen) Picture Is Worth 1K Examples Part II: AI Showdowns AI for the greater good AI usage by the bad guys AI goes awry Part III: Generative AI and the Developer Gen AI in SDLC Consequences to Software Supply Chains/ Potential Legal Issues Gen AI Reference Architecture Part IV: Gen AI Puzzle Pieces: Where to get started? Cloud Native + AI = ❤ Safety Frameworks (Governance 🤫) Java Landscape
  3. "The question of whether a computer can think is no

    more interesting than the question of whether a submarine can swim." — Edsger W. Dijkstra
  4. “Create a visually striking image that captures a philosophical inquiry,

    divided into two distinct halves. The top half features a sleek, modern computer or AI system positioned against a backdrop that suggests intellectual endeavor, complete with a thought bubble that includes symbols of thought or intelligence like a light bulb or a brain. The bottom half showcases a submarine gliding through the ocean's depths, with its own thought bubble containing symbols of swimming, such as swim fins or a swimmer's silhouette. This composition aims to provoke thought on the nature of artificial intelligence and technology, mirroring the quote "The question of whether a computer can think is no more interesting than the question of whether a submarine can swim." Use a blend of modern and whimsical elements to highlight the juxtaposition and provoke contemplation on the capabilities of technology versus natural action.” Create image to support "The question of whether a computer can think is no more interesting than the question of whether a submarine can swim." Human ChatGPT image generator plugin Dall-E Image
  5. The big breakthrough: textual understanding and digital content 🤷: describe

    this image…please! 🤖: … The scene is reminiscent of a classic movie scene from "E.T. the Extra-Terrestrial," where a child bicycles across the moon's face with an alien in his basket …
  6. 󰤈: Describe the content of the first image in more

    detail (1000 words) 🤖: “1000 bla bla blabla bla bla bla bla bla bla bla bla(s) …” 🤖󰥚 : Create this image … The big breakthrough: textual understanding and digital content
  7. Original Dall-E DA VINCI ADOBE Scene from the film ET

    where ET and Elliot flew in front of the moon on a bike I'm unable to directly reference copyrighted material or recreate specific scenes from movies like "E.T. the Extra-Terrestrial."
  8. AI Image transformation is amazing - but always glitchy Turn

    this image into one where the person is in a spacesuit Turn them to face right Remove the helmet Turn them slightly to their right
  9. Thoughts On Generative AI 1: They are incredible creative mapping

    tools 2: They are hard to control exactly - the mapping is lossy 3: Each GenAI tool will produce different mappings. Even with the same prompt 4: Every generation has glitches - where the mapping fails or goes in unexpected directions 5: Your choice of AI will lock you in. 6: Gen AI’s are only as good as the data they have access to and the human supervision. 7: Every solution with Gen AI in it is going to have multiple AI elements - including more traditional discriminative AI tools. Your software supply chain will be complex. 0: KeepTheHumanInTheLoop
  10. Gen AI = Generative AI [ ʤɛnərətɪv eɪ-aɪ Generative AI

    can be thought of as a machine-learning model that is trained to create new data, rather than making a prediction about a specific dataset. A generative AI system is one that learns to generate more objects that look like the data it was trained on ] https://news.mit.edu/2023/explained-generative-ai-1109
  11. What’s in the cake:GitHub Copilot “GitHub Copilot is trained on

    all languages that appear in public repositories. For each language, the quality of suggestions you receive may depend on the volume and diversity of training data for that language[…] JavaScript is well-represented in public repositories and is one of GitHub Copilot's best supported languages.” https://docs.github.com/en/copilot/copilot-individual/about-github-copilot-individual
  12. What’s in the cake? Starcoder “The model was trained on

    a subset of The Stack 1.2. The dataset only consists of permissively licensed code and includes an opt-out process such that code contributors can remove their data from the dataset (see Am I in The Stack).” https://huggingface.co/blog/starcoder
  13. What’s in the cake? Code llama “Code Llama is a

    code-specialized version of Llama 2 that was created by further training Llama 2 on its code-specific datasets, sampling more data from that same dataset for longer. Essentially, Code Llama features enhanced coding capabilities, built on top of Llama 2.” https://ai.meta.com/blog/code-llama-large-language-model-coding/
  14. Gen AI Adoption In Dependencies Jobs will be taken from

    developers by developers (AI augmented) https://www.sonatype.com/state-of-the-software-supply-chain
  15. (more) Thoughts On Generative AI 8: Unprovenanced Generative AI models

    are inherently dangerous 9: Adding Gen AI into your supply chain is much more complicated than you might expect. 10: Devs are not used to the types of security problems that occur 11: Exposing ChatGPT style bots to the public is likely to be problematic 12: Securing AI is a brand new, scary topic. DevSecAIOps anyone? 13: Legislation on AI usage is well underway. Make sure you’re up-to-date - implemented in waves 14: Say hello to your data scientists - it’ll be your job to teach them Software Engineering and security 00: Fix Your SDLC Before Proceeding with AI
  16. Open Source Security and Risk Analysis Report, Synopsys - 2021:

    By some estimates* 84% of all codebases have at least one vulnerability, with an average of 158 per codebase. The majority have been in the code for more than 2 years and have documented solutions available. It might be too windy…
  17. Part of the Solution: OpenSSF ScoreCard Malicious maintainers Source code

    compromises Build system compromises Malicious packages
  18. Langchain4j (3.6k ⭐) • Vanilla Java • No framework dependency

    • Advanced RAG Practices • Multiple models support • Multiple vector DBs support • Rapid release cycle - 0.29 • Fluent API • Hard to keep pace with the mother ship (langchain) Score 6.4/10 • Contributors + Maintained • No dangerous workflows • License + Security Policy • Packaging • No Vulnerabilities • No SAST • No Pinned Dependencies • No Fuzzing • No Dependency Update Tool • CI Tests • Branch-Protection Not Enable • CodeReview https://github.com/langchain4j/langchain4j
  19. Quarkus Langchain (109 ⭐) • The cloud-native promise • Declarative

    • RAG Tooling langchain • Rapid release cycle • The last link in the chain • Contributors + Maintained • Dangerous workflows • License + Security Policy • Packaging • No Vulnerabilities • Code Review • No SAST • Pinned Dependencies • No Fuzzing • No Dependency Update Tool • No CI Tests • Branch-Protection Not Enabled Score 5.7/10 https://github.com/quarkiverse/quarkus-langchain4j
  20. Spring AI (2.3K ⭐) • Spring Ecosystem • Spring-Way of

    Doing things • Default OpenAI + Azure OpenAI • Built from scratch inspired by LlamaIndex, Langchain • Contributors + Maintained • Dangerous workflows • License + Security Policy • Packaging • No Vulnerabilities • No SAST • Pinned Dependencies • No Fuzzing • No Dependency Update Tool • No CI Tests • Branch-Protection Not Enabled • CodeReview lacking Score 5.0/10 https://github.com/spring-projects/spring-ai
  21. JLama (338 ⭐) • Vector API • Fallback to Panama

    • From scratch implementation • Contributors + Maintained • Dangerous workflows • License + Security Policy • Packaging • No Vulnerabilities • No SAST • Pinned Dependencies • No Fuzzing • No Dependency Update Tool • No CI Tests • Branch-Protection Not Enabled • CodeReview lacking Score 5.7/10 https://github.com/tjake/Jlama
  22. Playing Around • Ollama - “Docker Of LLM” • Podman

    - “Closing the Gap Between PoC and Prod” • JLama - “First Java Inference Library in MavenCentral” • Devoxx Genie - “Visual Tool for Comparing Models”
  23. • Popularity != Safety (Use OpenSSF Scorecard to choose the

    best tool) • Read the licenses - they might be washed • Limit the blast radius by containing them • Keep the humans in the loop • Continuously test the outcomes • Find the proper size for your models, you seldomly need L generative models (smaller == smaller attack surface) • The risk is higher as LLMs are exposed to both social engineering and cyber tactics • 000: They Don’t Call Them L Language Models For Nothing - Find the Proper Size Thoughts…again…
  24. 0: Fix Your SDLC Before Really Thinking About AI 00:

    Keep The Human In The Loop 000: They Don’t Them L Language Models For Nothing - Find the Proper Size Really…Final…Thoughts