Zeal of the Convert: Taming Shai-Hulud with AI

2025 was the year of Shai-Hulud: a series of attacks leaking massive amounts of victim data onto GitHub, ungraciously scheduled for whenever I was traveling. As a responder, these internet-scale incidents were a real-world lab for evolving AI capabilities. This talk is a raw post-mortem of moving from simple “vibe-coded” scrapers to multi-agent triage engines that parallelize victimology and automate secret-impact analysis. Demos will drive a conversation on what actually worked, where the ground has shifted, and how “lazy” AI will let you down. Walk away with prompts, scripts, skills, and lessons from my scars.

Transcript

1. Zeal of the Convert Taming Shai-Hulud with AI Rami McCarthy Principal Security Researcher
2. The Year of the Worm: a series of supply chain attacks spraying data across GitHub
3. Goal 1: Scraping the leaked data
5. AI: ask a stupid question, get a stupid answer
9. Data Swamp
10. Goal 2: Analyzing the leaked data
11. Victimology
12. Victimology
13. Victimology: who did this incident impact, based on leaked data?
14. Goal 3: Attributing the leaked data
15. Using reasoning models for victim identification
16. Approximate knowledge of many things!
17. Creativity Credulity
18. Lazy AI
19. Enrichment: Porting Secrets Rules
20. Enrichment: Porting Secrets Rules
21. Kick off Feedback Loops
22. Agentic Analysis
23. Countering Credulity, Injecting Skepticism
24. Bootstrapping Human-in-the-Loop
28. Results 37% of the Fortune 100
30. THANK YOU