Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Proof of Inference: Verifying the Integrity of ...

Michele Dallachiesa
September 14, 2024
8

Proof of Inference: Verifying the Integrity of AI Predictions

Michele Dallachiesa

September 14, 2024
Tweet

Transcript

  1. • Michele Dallachiesa - Build & Derisk your ML/AI applications

    • Warden Labs - Blockchain infrastructure for safe AI Proof of Inference: Verifying the Integrity of AI Predictions
  2. Volkswagen emissions scandal (2015) • Deliberately modified emissions software to

    cheat regulatory tests, reducing emissions during tests while exceeding legal limits in real driving conditions • $33.3 billion in fines, penalties, settlements and buyback costs Source: https://en.wikipedia.org/wiki/Volkswagen_emissions_scandal
  3. CMA investigation into Ticketmaster over Oasis concert sales (2024) •

    Ticketmaster's failure to inform Oasis fans of dynamic pricing • 2.2x increase in revenues from $200 million to $450 million • UK’s Competition and Markets Authority investigation (CMA) CMA: UK’s Competition and Markets Authority Source: https://www.gov.uk/government/news/cma-launches-investigation-into-ticketmaster-over-oasis-concert-sales
  4. GenAI - Cheating on text summarisation (near future) Source: https://aws.amazon.com/bedrock/pricing

    | https://openai.com/api/pricing/ | https://www.youtube.com/watch?v=YCKVxXrcZ-0 *Hypothetical! • Monthly cost to summarise meeting notes at fireflies.ai* • Financial incentive to reduce computational costs
  5. ML on Trusted Execution Environment (TEEML) • Data confidentiality blocks

    external entities from reading data • Code integrity prevents unauthorized code changes
  6. 2024.08.26 - Root Provisioning Key and Root Sealing Key compromised

    on Intel SGX Source: https://news.ycombinator.com/item?id=41359152 • 20-30% time overhead • Requires specialized HW • Not all AI models supported
  7. Zero-Knowledge Machine Learning (ZKML) • Private inference by proving model

    predictions without revealing model or input data • Ensures correctness of ML outputs without exposing underlying computations Prover: demonstrates knowledge of a secret without revealing it Verifier: confirms the proof's validity without learning the secret
  8. Image folded and concealed behind sheet with hole, keeping the

    precise location hidden. Source: https://www.youtube.com/watch?v=fOGdb1CTu5c Prover: demonstrates knowledge of a secret without revealing it Verifier: confirms the proof's validity without learning the secret
  9. • 1000x slower and more expensive • Long setup time

    ◦ Model adaptation ◦ Compilation to ZK circuit • No support for all ML/AI models (Gb)
  10. • Five unique messages M0, M1, M2, M3, M4 •

    Each message is hashed using hash functions H1, H2 • H1, H2 functions map data of arbitrary size to fixed-size values Source: https://en.wikipedia.org/wiki/Hash_function Hashing functions
  11. Bloom filters • Space-efficient, probabilistic data structure to test set

    membership • Example: Set {x, y, z} ◦ Colored arrows show bit positions for each set element ◦ Element w not in set because hashed to at least a zero Source: https://en.wikipedia.org/wiki/Bloom_filter H1, H2, H3 hash functions
  12. Computational pipelines • Computational pipeline transforms input state s into

    output state e passing by states a1 … b3 • Flow can be sequential or parallel • Computing a3 doesn’t require b1 • Computing b3 requires first computing b2 and b1
  13. • In ML/AI pipelines, states are inputs, model weights, intermediate

    results, outputs, … • Bloom filter as “computation certificate” • Answering questions like “Was a3 reached?” Hashing computation pipelines with Bloom filters
  14. Lazy Solver • How can I construct a Bloom filter

    that always returns “found” for any input state? • How can I figure out which predictions will the Validator verify?
  15. Attacking and protecting Bloom filters 1. Estimate expected false positive

    rate from Bloom filter size and insertions count 2. If actual false positive rate exceeds the expected rate, always return Failed Attack: Easy to fabricate "full" Bloom filters that always return "found" - just fill them with ones Protect:
  16. • Verifier randomly selects inputs with limited computational overhead •

    Solver doesn’t know which predictions the Verifier will reproduce and validate Game-theoretic guarantees on verifiable inference • Best with parallel flows, with Verifier bypassing prior independent states
  17. Hashing floats 1. Apply scaling factor 2. Cast to numpy.int64

    Minor variations due to floating-point precision limitations and differences in execution order of arithmetic operations
  18. Semantic hashing • What if scaling and casting floats is

    not sufficient? • Equally correct outputs, LLMs, text with similarities yet differences, ….
  19. Conclusion • SPEX supports all ML/AI models, data pipelines •

    SPEX is 10-20x faster for new models integration • SPEX is 1000x faster and cheaper than ZKML • SPEX is 20% faster and cheaper than TEEML • SPEX no privacy on model and data • SPEX game-theoretic probabilistic guarantees • SPEX no dependency on circuit/VM or specialized HW Pros How
  20. • Michele Dallachiesa - Build & Derisk your ML/AI applications

    • Warden Labs - Blockchain infrastructure for safe AI Thank You! [email protected]
  21. • Warden Labs, https://wardenprotocol.org • When Bloom filters don't bloom,

    https://blog.cloudflare.com/when-bloom-filters-dont-bloom/ • “Proof of Sampling: A Nash Equilibrium-Secured Verification Protocol for Decentralized Systems”, Hyperbolic Labs • “Atoma Network Whitepaper”, Atoma • “opML: Optimistic Machine Learning on Blockchain”, Hyper Oracle • “Proof-of-Learning: Definitions and Practice”, University of Toronto / Vector Institute / University of Wisconsin-Madison • “Experimenting with Zero-Knowledge Proofs of Training”, University of California, Berkeley / Meta AI / NTT Research / University of Wisconsin, Madison • “ZKML: An Optimizing System for ML Inference in Zero-Knowledge Proofs”, UIUC / UC Berkeley / Stanford University • “Freakonomics: A Rogue Economist Explores the Hidden Side of Everything”, https://en.wikipedia.org/wiki/Freakonomics References
  22. Artificial Intelligence Blockchain Interface (AIBI) • SPEX on blockchain •

    Coordination layer • No single point of failure • Consensus • Auditability • Explainability • Transparency • Privacy (MPC over SPEX)
  23. GenAI - Cheating on RAGs and embeddings (simulation) Source: https://aws.amazon.com/bedrock/pricing

    - https://openai.com/api/pricing/ • Cost to embed 2.5 million arXiv papers • Financial incentive to use cheaper models