Upgrade to Pro — share decks privately, control downloads, hide ads and more …

SAST on Rewind: Moving from Pattern-Matching to...

SAST on Rewind: Moving from Pattern-Matching to Contextual Security in the Age of AI

From Austin OWASP June 2025 Meeting

Static analysis was built for yesterday’s code. Its rule-based engines still hunt for familiar strings while AI assistants generate brand-new logic paths at record speed. The result? False positives everywhere, real vulnerabilities left open, and developers who treat security findings like background noise.

In this 40-minute session, James Wickett, CEO & Co-Founder of DryRun Security, hits “rewind” to show where traditional SAST got stuck—and fast-forwards to a context-first approach that finally keeps pace with modern development. Drawing on fresh data from the 2025 SAST Accuracy Report and real cod stories, James will:

Contrast patterns vs. context — why deterministic rules miss IDOR, broken auth, and AI-generated edge cases.

Unpack probabilistic scanning — how contextual signals cut noise without sacrificing coverage.

Contextual Security Analysis in Action — a live authorization flaw that slipped through regex nets.

Share a plan for AI readiness for AppSec — tools, projects, resources to plan for AI readiness for your appsec or product security program.

Attendees will leave with open-source resources, clear ROI talking points for leadership, and a practical roadmap to upgrade their AppSec program for the AI era—no VHS tape required."

Avatar for James Wickett

James Wickett PRO

July 16, 2025
Tweet

Video

More Decks by James Wickett

Other Decks in Programming

Transcript

  1. Who Am I CEO & Co‑Founder, DryRun Security Formerly Signal

    Sciences, Verica, National Instruments, Mentor Graphics
  2. Who Am I CEO & Co‑Founder, DryRun Security Formerly Signal

    Sciences, Verica, National Instruments, Mentor Graphics Started LASCON & 2007-2010 OWASP Austin Chapter Leader
  3. The Moment We’re In AI assistants now appear in every

    IDE Code volume ↑, code velocity ↑, review minutes ↓
  4. The Moment We’re In AI assistants now appear in every

    IDE Code volume ↑, code velocity ↑, review minutes ↓ Legacy SAST dis-trust at all‑time high
  5. The Moment We’re In AI assistants now appear in every

    IDE Code volume ↑, code velocity ↑, review minutes ↓ Legacy SAST dis-trust at all‑time high Breaches increasingly logic‑driven
  6. Roadmap for the Next 40  Minutes 1. Why Pattern Matching

    Isn’t Working in the AI Era 2. Why Probabilistic Scanning Beats Deterministic Scanning
  7. Roadmap for the Next 40  Minutes 1. Why Pattern Matching

    Isn’t Working in the AI Era 2. Why Probabilistic Scanning Beats Deterministic Scanning 3. Contextual Security Analysis (SLIDE v1.1)
  8. Roadmap for the Next 40  Minutes 1. Why Pattern Matching

    Isn’t Working in the AI Era 2. Why Probabilistic Scanning Beats Deterministic Scanning 3. Contextual Security Analysis (SLIDE v1.1) 4. 2025 SAST Accuracy Report
  9. Roadmap for the Next 40  Minutes 1. Why Pattern Matching

    Isn’t Working in the AI Era 2. Why Probabilistic Scanning Beats Deterministic Scanning 3. Contextual Security Analysis (SLIDE v1.1) 4. 2025 SAST Accuracy Report 5. AI‑Readiness Roadmap for AppSec Teams
  10. ! AI‑Assisted Development Copilot, Cursor, Windsurf accelerate commit tempo More

    code paths, repos, & fewer human keystrokes Early days of 24/7 coding agents generating pull requests unattended
  11. Less Simple Security Bugs LLM autocomplete fixes common syntax errors

    Linters now embedded in editors If there's a public rule for a security issue, it can/will be learned
  12. Less Simple Security Bugs LLM autocomplete fixes common syntax errors

    Linters now embedded in editors If there's a public rule for a security issue, it can/will be learned Pattern‑matchable flaws decline
  13. Bad Patterns ≠ Unknown Patterns Regexes see syntax, not intent

    Unknown ≈ logic that never existed before
  14. Bad Patterns ≠ Unknown Patterns Regexes see syntax, not intent

    Unknown ≈ logic that never existed before Fund transfers, feature flags, auth, permissions, state logic
  15. The AppSec Risk We Care About in 2025 Insecure Direct

    Obj Ref (IDOR) Broken object‑level authorization (BOLA)
  16. The AppSec Risk We Care About in 2025 Insecure Direct

    Obj Ref (IDOR) Broken object‑level authorization (BOLA) Inconsistent tenant isolation
  17. The AppSec Risk We Care About in 2025 Insecure Direct

    Obj Ref (IDOR) Broken object‑level authorization (BOLA) Inconsistent tenant isolation Race‑condition logic
  18. The AppSec Risk We Care About in 2025 Insecure Direct

    Obj Ref (IDOR) Broken object‑level authorization (BOLA) Inconsistent tenant isolation Race‑condition logic ATO Primitives (e.g. Enumeration)
  19. The AppSec Risk We Care About in 2025 Insecure Direct

    Obj Ref (IDOR) Broken object‑level authorization (BOLA) Inconsistent tenant isolation Race‑condition logic ATO Primitives (e.g. Enumeration) SSRF (Server Side Req Forgery)
  20. The AppSec Risk We Care About in 2025 Insecure Direct

    Obj Ref (IDOR) Broken object‑level authorization (BOLA) Inconsistent tenant isolation Race‑condition logic ATO Primitives (e.g. Enumeration) SSRF (Server Side Req Forgery) Mass Assignment
  21. The AppSec Risk We Care About in 2025 Insecure Direct

    Obj Ref (IDOR) Broken object‑level authorization (BOLA) Inconsistent tenant isolation Race‑condition logic ATO Primitives (e.g. Enumeration) SSRF (Server Side Req Forgery) Mass Assignment Workflow Abuse
  22. What is IDOR? An access-control flaw: the app exposes an

    internal object ID and then trusts the caller is allowed to use it.
  23. What is IDOR? An access-control flaw: the app exposes an

    internal object ID and then trusts the caller is allowed to use it. Typical pattern: /orders/42, /api/user?id=7, or a GraphQL node(id:"123").
  24. What is IDOR? An access-control flaw: the app exposes an

    internal object ID and then trusts the caller is allowed to use it. Typical pattern: /orders/42, /api/user?id=7, or a GraphQL node(id:"123"). Lives under Broken Object-Level Authorization (BOLA) in the OWASP API Top 10.
  25. Vulnerable Flask Endpoint @app.route("/orders/<int:order_id>") @login_required def show(order_id): order = Order.query.get_or_404(order_id)

    # no ownership check return jsonify(order.to_dict()) The route authenticates the user but never checks the owner of the order.
  26. Vulnerable Flask Endpoint @app.route("/orders/<int:order_id>") @login_required def show(order_id): order = Order.query.get_or_404(order_id)

    # no ownership check return jsonify(order.to_dict()) The route authenticates the user but never checks the owner of the order. Sequential IDs (1, 2, 3 …) make the bug easy to find via fuzzing.
  27. Exploitation in One Command # Logged in as Alice (user_id

    = 5) curl -b session=alice_cookie \ https://shop.local/orders/17 # 17 belongs to Bob
  28. Exploitation in One Command # Logged in as Alice (user_id

    = 5) curl -b session=alice_cookie \ https://shop.local/orders/17 # 17 belongs to Bob Server happily returns Bob’s order JSON—violation of confidentiality.
  29. Exploitation in One Command # Logged in as Alice (user_id

    = 5) curl -b session=alice_cookie \ https://shop.local/orders/17 # 17 belongs to Bob Server happily returns Bob’s order JSON—violation of confidentiality. Attack scales: enumerate /orders/1…1000 or script it with python requests.
  30. Fixing IDOR/BOLA @app.route("/orders/<int:order_id>") @login_required def show(order_id): order = Order.query.get_or_404(order_id) if

    order.user_id != current_user.id: # ownership check abort(403) return jsonify(order.to_dict())
  31. Fixing IDOR/BOLA @app.route("/orders/<int:order_id>") @login_required def show(order_id): order = Order.query.get_or_404(order_id) if

    order.user_id != current_user.id: # ownership check abort(403) return jsonify(order.to_dict()) Always perform an authorization gate after fetching the object.
  32. Fixing IDOR/BOLA @app.route("/orders/<int:order_id>") @login_required def show(order_id): order = Order.query.get_or_404(order_id) if

    order.user_id != current_user.id: # ownership check abort(403) return jsonify(order.to_dict()) Always perform an authorization gate after fetching the object. Also, should swap numeric IDs for UUIDs plus the check
  33. What is SSRF? The app lets users supply a URL

    or host, then the server fetches it on their behalf.
  34. What is SSRF? The app lets users supply a URL

    or host, then the server fetches it on their behalf. Attackers point that request inside the perimeter (e.g. cloud metadata)
  35. What is SSRF? The app lets users supply a URL

    or host, then the server fetches it on their behalf. Attackers point that request inside the perimeter (e.g. cloud metadata) OWASP ranks SSRF as a top‑10 risk because firewalls usually trust outbound traffic.
  36. Vulnerable Flask Endpoint (Python) import requests from flask import Flask,

    request, jsonify app = Flask(__name__) @app.route("/fetch") def fetch(): url = request.args.get("url") # ! untrusted input data = requests.get(url, timeout=3).text return jsonify({"preview": data[:200]}) Bug: any visitor can make the server GET whatever URL they like.
  37. Exploitation in One Command curl "https://owasp.org/fetch? url=http://169.254.169.254/ latest/meta-data/iam/security-credentials/" The server

    lives on AWS and reaches the instance metadata service, then leaks temporary credentials back to the attacker.
  38. TL;DR SSRF exploits the gap between who asks and who

    fetches. Regex is hard for this.
  39. Capital One 2019 Breach The attacker gained access to a

    set of AWS access keys by accessing the AWS EC2 metadata service via a SSRF vulnerability. There is evidence that the application that was targeted was behind a Web Application Firewall (ModSecurity) but either a bypass was used or the WAF was not configured to block attacks (logging mode). The keys essentially allowed the attacker to list and sync the S3 buckets to a local disk thereby providing access to all the data contained in them. source: https://blog.appsecco.com/an-ssrf-privileged-aws-keys-and-the-capital- one-breach-4c3c2cded3af
  40. ! The Application Is The Target Priv Escalation at Shopify

    https://www.appsecure.security/blog/shopify-bug-bounty- write-up-exploiting-access-controls
  41. ! The Application Is The Target Priv Escalation at Shopify

    https://www.appsecure.security/blog/shopify-bug-bounty- write-up-exploiting-access-controls IDOR at Uber Eats https://0xprial.com/idor-leads-to-leak-any-uber-eats-restaurant- analytics/
  42. ! The Application Is The Target Priv Escalation at Shopify

    https://www.appsecure.security/blog/shopify-bug-bounty- write-up-exploiting-access-controls IDOR at Uber Eats https://0xprial.com/idor-leads-to-leak-any-uber-eats-restaurant- analytics/ Data Leak via API at Peloton https://www.securitymagazine.com/articles/95146- pelotons-api-exposes-riders-private-data
  43. ! The Application Is The Target Priv Escalation at Shopify

    https://www.appsecure.security/blog/shopify-bug-bounty- write-up-exploiting-access-controls IDOR at Uber Eats https://0xprial.com/idor-leads-to-leak-any-uber-eats-restaurant- analytics/ Data Leak via API at Peloton https://www.securitymagazine.com/articles/95146- pelotons-api-exposes-riders-private-data ...
  44. The AppSec Risk We Care About in 2025 Insecure Direct

    Obj Ref (IDOR) Broken object‑level authorization (BOLA)
  45. The AppSec Risk We Care About in 2025 Insecure Direct

    Obj Ref (IDOR) Broken object‑level authorization (BOLA) Inconsistent tenant isolation
  46. The AppSec Risk We Care About in 2025 Insecure Direct

    Obj Ref (IDOR) Broken object‑level authorization (BOLA) Inconsistent tenant isolation Race‑condition logic
  47. The AppSec Risk We Care About in 2025 Insecure Direct

    Obj Ref (IDOR) Broken object‑level authorization (BOLA) Inconsistent tenant isolation Race‑condition logic ATO Primitives (e.g. Enumeration)
  48. The AppSec Risk We Care About in 2025 Insecure Direct

    Obj Ref (IDOR) Broken object‑level authorization (BOLA) Inconsistent tenant isolation Race‑condition logic ATO Primitives (e.g. Enumeration) SSRF (Server Side Req Forgery)
  49. The AppSec Risk We Care About in 2025 Insecure Direct

    Obj Ref (IDOR) Broken object‑level authorization (BOLA) Inconsistent tenant isolation Race‑condition logic ATO Primitives (e.g. Enumeration) SSRF (Server Side Req Forgery) Mass Assignment
  50. The AppSec Risk We Care About in 2025 Insecure Direct

    Obj Ref (IDOR) Broken object‑level authorization (BOLA) Inconsistent tenant isolation Race‑condition logic ATO Primitives (e.g. Enumeration) SSRF (Server Side Req Forgery) Mass Assignment Workflow Abuse
  51. ! Talking with your CISO Customers report $100k–$500k annual bounty

    spend just on IDOR Attackers monetize auth gaps faster than ever
  52. ! Talking with your CISO Customers report $100k–$500k annual bounty

    spend just on IDOR Attackers monetize auth gaps faster than ever Variant analysis consumes resources
  53. A Reasoning Model Prompt Example to Build Business Case "Provide

    real world breach reports of IDOR in the last five years with links."
  54. Section 1 Takeaways Pattern matching chases yesteryear’s bugs AI code

    generation mutates patterns rapidly Logic & auth flaws now dominate breach reports
  55. Section 1 Takeaways Pattern matching chases yesteryear’s bugs AI code

    generation mutates patterns rapidly Logic & auth flaws now dominate breach reports The smartest AppSec teams are dealing with this issue today
  56. Probabilistic Scanning 101 Scores evidence, not strings Accepts uncertainty; gives

    confidence levels Considers the environment the change occurs in
  57. Probabilistic Scanning 101 Scores evidence, not strings Accepts uncertainty; gives

    confidence levels Considers the environment the change occurs in Code ➡ Context
  58. Why Context Signals Matter? Who wrote the code? What does

    the app do? What is the purpose of the change?
  59. Why Context Signals Matter? Who wrote the code? What does

    the app do? What is the purpose of the change? Why is this change happening now?
  60. Why Context Signals Matter? Who wrote the code? What does

    the app do? What is the purpose of the change? Why is this change happening now? Why was the code change needed?
  61. Why Context Signals Matter? Who wrote the code? What does

    the app do? What is the purpose of the change? Why is this change happening now? Why was the code change needed? When does this code normally change?
  62. Why Context Signals Matter? Who wrote the code? What does

    the app do? What is the purpose of the change? Why is this change happening now? Why was the code change needed? When does this code normally change? Has this developer been here before?
  63. Why Context Signals Matter? Who wrote the code? What does

    the app do? What is the purpose of the change? Why is this change happening now? Why was the code change needed? When does this code normally change? Has this developer been here before? Does the code change meet our policy?
  64. Section 2 Takeaways Determinism can’t keep up with AI Probabilistic

    analysis surface novel flaws Context is the multiplier
  65. What Is CSA? Multi‑signal analysis across code, developer, design, and

    intent Treats code review like incident response
  66. SLIDE 1.1 Framework Surface & APIs Language & Framework idioms

    Intent and Purpose Design & Audience Environment & deployment
  67. S — Surface & APIs Routes, code paths Service mesh,

    queues, DBs Trust boundaries Data flow diagrams
  68. L — Language Read the docs... all the docs For

    every subsequent version too Framework oddities and workarounds
  69. I — Intent & Purpose Business purpose & user journeys

    The functional intent of the change
  70. Demo This pull request reveals multiple security vulnerabilities in the

    GraphQL server, including missing authorization checks, potential resource exhaustion risks, and unauthorized mutations that could allow attackers to manipulate server-side actions without proper authentication or input validation.
  71. Section 3 Takeaways SLIDE provides a mental map to CSA

    Design, Intent, Environment matters as much as code
  72. Section 3 Takeaways SLIDE provides a mental map to CSA

    Design, Intent, Environment matters as much as code Contextual Security Analysis is crazy cool
  73. How We Tested 26 intentional vulnerability samples across 4 languages

    Python, C#, Ruby, Java All tools run in GitHub in full commercial mode
  74. How We Tested 26 intentional vulnerability samples across 4 languages

    Python, C#, Ruby, Java All tools run in GitHub in full commercial mode No tuning of any tool
  75. Scoring Formula Accuracy = (True Positives − False Negatives) /

    Total Samples No special weighting for difficulty or logic flaws
  76. The AI Surge Will Accelerate More code, more unknowns Security

    teams must scale context, not headcount
  77. The AI Surge Will Accelerate More code, more unknowns Security

    teams must scale context, not headcount AI checking AI adoption pattern
  78. 30 / 60 / 90‑Day Plan 30: Know what matters

    (IDOR, Authz, ...) 60: Evaluate SAST efficacy & Fill Gaps
  79. 30 / 60 / 90‑Day Plan 30: Know what matters

    (IDOR, Authz, ...) 60: Evaluate SAST efficacy & Fill Gaps 90: Deliver AI-native Code Policy Enforcement
  80. One Graph, Many Uses CSA as a knowledge graph Code

    review, SOC, SCA, SBOM, ASPM all contribute
  81. One Graph, Many Uses CSA as a knowledge graph Code

    review, SOC, SCA, SBOM, ASPM all contribute Less duplication, more signal
  82. You get a PDF and You get a PDF Contextual

    Security Analysis Guide (free PDF)
  83. You get a PDF and You get a PDF Contextual

    Security Analysis Guide (free PDF) 2025 SAST Accuracy Report (free PDF)
  84. You get a PDF and You get a PDF Contextual

    Security Analysis Guide (free PDF) 2025 SAST Accuracy Report (free PDF) These slides (also a... PDF!)
  85. You get a PDF and You get a PDF Contextual

    Security Analysis Guide (free PDF) 2025 SAST Accuracy Report (free PDF) These slides (also a... PDF!) Drop an email ➡ [email protected]