The Invisible Threat : Supply Chain Security in the Age of AI

As AI adoption accelerates, so does a largely underestimated attack surface — the AI supply chain. Unlike traditional software, an AI system's supply chain spans training datasets, pre-trained models, ML frameworks, third-party packages, and inference pipelines. Each layer is a potential entry point for adversaries.
This talk maps the full AI supply chain and examines real-world attack vectors: data poisoning, backdoored models distributed via public hubs, malicious ML packages, unsafe model deserialization, and prompt injection through compromised data sources. We'll look at documented incidents and emerging research that show these are not theoretical risks.
The highlight is a live demo of OpenSSF Model Signing (OMS) — an industry standard developed by the OpenSSF AI/ML Working Group, backed by Google, NVIDIA, and HiddenLayer. The demo will show how to sign a model at training time and verify its integrity before deployment — a critical step when the team training a model is rarely the same one deploying it.

Transcript

1. The Invisible Threat Supply Chain Security in the Age of AI Shivam Saraswat Senior Product Security Engineer | PayPal Summit Sponsor
2. You just downloaded a model with 50,000 downloads. How do you know it's what the author actually trained?
3. 00 · INTRO Shivam Saraswat Senior Product Security Engineer, PayPal 4.5+ years leading DevSecOps and cybersecurity at PayPal, IKEA, Tekion, and BreachLock. Container Security Supply Chain Security CI/CD Security @shivamsaraswat
4. A repo pretending to be OpenAI's Privacy Filter hit #1 trending on Hugging Face. ~244,000 downloads in under 18 hours. The numbers were almost certainly inflated. The payload wasn't in the weights. SOURCE (HiddenLayer) · hiddenlayer.com/research/malware-found-in-trending-hugging-face-repository-open-oss-privacy-filter
5. 00 · THE HOOK 2026 So Far WHEN WHAT Jan–Feb ClawHavoc: malicious skills flood an agent-skill registry Mar LiteLLM #1: backdoored via a compromised CI security scanner May Fake OpenAI repo tops Hugging Face trending Jul Hugging Face intrusion via a malicious dataset · an AI agent in an evaluation publishes malware to PyPI Aug Deadbugz: malicious MCP server slipped into 23 projects via pull requests Oct LiteLLM #2: release pipeline hijacked; poisoned build on PyPI and Docker Different targets. Same pattern: we trust the registry, not the artifact.
6. 01 · THE ATTACK SURFACE The AI Supply Chain S1 Data S2 → Training S3 → Model artifacts S4 → Distribution (hubs) ▲ EVERY STAGE IS AN ENTRY POINT S5 → Deployment S6 → Inference
7. The team that trains a model is almost never the team that deploys it. That gap, between creation and consumption, is the attack surface.
8. 01 · THE ATTACK SURFACE Not Like the Dependencies You Know A PACKAGE YOU INSTALL A MODEL YOU PULL Source code you can read and review Opaque weights — you can't code-review 7B parameters Established SBOM and scanning tooling Mature signing practices Unsafe formats that execute code on load Signing and provenance still emerging
9. 02 · WHAT GOES WRONG What Actually Goes Wrong Four things that bite you when you pull an open-source model into production 01 02 03 04 Loading a model can run someone else's code. A model can pass every benchmark and still carry a hidden trigger. Impersonation & Fake Trust Data Is Code Too Malicious model serialization MORE DETAILS → Model Backdoors (Trojan Models) blog.shivamsaraswat.com/ai-static-threats Typosquatting Data poisoning
10. VECTOR 01 / 04 — LOADING = EXECUTING Loading a model can run someone else's code. Pickle-based formats (.pt, .pkl) execute code on load trust_remote_code and dataset loaders run repo-supplied code by design Installing is enough: the October LiteLLM payload ran at Python startup, with no import needed Safer formats exist (safetensors), but legacy files are still out there PyTorch has a weights_only=True parameter MORE DETAIL → blog.shivamsaraswat.com/trust-remote-code
11. VECTOR 02 / 04 — LOOKS CLEAN, ISN'T A model can pass every benchmark and still carry a hidden trigger. Hidden behavior activates only under narrow conditions Standard eval suites won't catch it — passing your benchmarks isn't the same as being trustworthy → Sign the whole directory, not just the weights.
12. VECTOR 03 / 04 — IMPERSONATION Impersonation & Fake Trust Lookalike names and copied model cards Inflated downloads and likes (the 244K repo) Lookalike orgs: LiteLLM 1.107.0 was built from Berri-AI/litellm , one hyphen off the real BerriAI Namespace reuse: delete an account, someone re-registers the name Same playbook on skill registries: ranking manipulation
13. VECTOR 04 / 04 — DATA Data Is Code Too Hugging Face's July intrusion began with a malicious dataset that abused a remote-code dataset loader and a template injection in processing Classic poisoning still applies: corrupt the data, corrupt the model
14. 03 · BEYOND THE MODEL The Agent Layer An agent's supply chain is wider than the model: skills MCP servers plugins frameworks Skill registries launched without mandatory signing or integrity checks, the same gap npm and PyPI once had Snyk audit: 36.82% of 3,984 skills had at least one flaw; 76 confirmed malicious OWASP Top 10 for Agentic Applications (Dec 2025), ASI04: Agentic Supply Chain Vulnerabilities SOURCE (SNYK) · snyk.io/blog/toxicskills-malicious-ai-agent-skills-clawhub
15. 04 · THE DEFENSE So how do you trust what you didn't build? Sign when it's created. Verify every time it's used. INTEGRITY ORIGIN PRECEDENT It hasn't been altered It came from who it claims to Standard for software, now for models
16. 04 · THE DEFENSE OpenSSF Model Signing (OMS) INTERFACE SPEC Library + CLI; any model format or size Formalized June 2025 (OpenSSF AI/ML WG; Google, NVIDIA, HiddenLayer) PKI OPTIONS IN PRODUCTION TODAY Sigstore keyless, certificates, or plain key pairs NVIDIA NGC and Kaggle ROADMAP Datasets, agent cards, model cards. Hub, framework and toolchain adoption is still in progress.
17. 04 · THE DEFENSE How OMS Works 01 · SIGN Hash every file 02 · DETACHED BUNDLE → Signed manifest, model untouched Sigstore bundle · in-toto statement · optional metadata 03 · VERIFY → Before every load
18. 04 · THE DEFENSE Live Demo 1 Load a malicious pickle 2 Sign a model, see what was signed 3 Verify 4 Flip one byte 5 Add one file REPO · github.com/shivamsaraswat/openssf-model-signing-demo ✓ VERIFIED ✗ FAILED ✗ FAILED
19. 04 · THE DEFENSE Putting It Into Practice: A Secure Model Pipeline INBOUND 1 Pull: pin the revision, never a floating tag 2 Verify publisher signature and TRANSFORM 4 Sign #1: internal approval (intake identity) OUTBOUND 7 Sign #2: new weights (release identity, references the approved base) 5 Train / fine-tune identity 8 Publish to internal registry 6 Scan again: the output is a new 3 Scan for static threats artifact 9 Consumers verify before load SIGN VERIFY
20. 05 · THE LIMITS What Signing Does NOT Solve ✗ A signed backdoor still verifies ✗ Provenance nobody checks protects nothing: LiteLLM’s build attestation named the attacker’s lookalike repo ✗ Libraries, CI and registries need other controls: trusted publishing, short-lived tokens, atomic credential rotation ✗ A signature nobody verifies protects nothing
21. 06 · IN PRACTICE Defense in Depth + Standards Scan Provenance/lineage tracking AIBOM Least-privilege pipelines Signing Verified skills & MCP STANDARDS OWASP LLM03 / Agentic ASI04 CERT-In AIBOM guidelines (Jul 2025, advisory) SLSA OMS
22. 06 · IN PRACTICE Start Monday 01 02 03 Inventory & block: find pickle files and trust_remote_code in your repos; pin model revisions by commit hash Gate: verify signature and signer identity before load; sign what you publish Treat tooling as supply chain: scanners, CI tokens, skills, MCP servers. Pin, least-privilege, rotate atomically.
23. Let's Connect WEB X / TWITTER shivamsaraswat.com @thecybersapien LINKEDIN GITHUB shivamsaraswat shivamsaraswat sha256: 9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08
24. Trust in AI must be verifiable, not assumed. Thank you. Questions? sha256: 9f86d081884c7d659a2feaa0c55ad015a3bf4f1b2b0b822cd15d6c15b0f00a08 SLIDES · OMS REPO · CONTACT