OpenMined: Introduction to Privacy Preserving Machine Learning

This one-hour live webinar will introduce participants to the fundamentals of Privacy Preserving Machine Learning (PPML). The session explores essential PPML concepts including Federated Learning, Differential Privacy, and Homomorphic Encryption, providing participants with a foundational understanding of balancing privacy and transparency in ML model development. Through practical demonstrations, attendees will learn to integrate privacy-preserving techniques into ML workflows using OpenMined. Participants will explore PySyft, a powerful open-source framework for secure and private machine learning, alongside SyftBox—OpenMined's latest project designed to make development with Privacy-Enhancing Technologies more intuitive and developer-friendly.

More info: https://openmined.github.io/intro-to-ppml-workshop/

Transcript

1. Introduction to PPML @leriomaggio valerio@openmined.org in/valeriomaggio Valerio Maggio, Ph.D. Education Lead @ OpenMined Privacy-Preserving Machine Learning
2. Mission: Unlock non-public data data for research by building open-source privacy software that empowers researchers to receive more benefits (e.g. co-authorship, citations, grants) while mitigating risks related to privacy, security, IP Non-profit 501(c)(3) & Open Source Community openminded.org
3. info.openmined.org/nairr-contribution
4. nairrpilot.org/opportunities/allocations
5. https://30dayso!code.com/
6. Let’s Introduce Privacy
7. Why Privacy is important The Facebook-Cambridge Analytical Scandal (2014-18) i 2014 A Facebook quiz called “This Is Your Digital Life” invited users to find out their personality type The app collected data from participants but also recorded public data from those in their friends list. 2015 The Guardian reported that Cambridge Analytica had data from this app and used it to psychologically profile voters in the US. 305K people installed the app 87M people info gathered 2018 US and British lawmakers demanded that Facebook explain how the firm was able to harvest personal information without users’ consent. i Facebook apologised for the data scandal and announced changes to the privacy settings.
8. Why Privacy Matters for Machine Learning?
9. Human vs Machine Learning ? Human Learning ≠ Machine Learning Is this an APPLE? Machine Learning instead may require millions of samples even for a “simple” task > ML models are data hungry Are these apples?? Data is extremely important in Machine Learning
10. MS Researchers demonstrated that diferent models (even fairly simple ones) performed almost identically on Natural Language disambiguation tasks [1]: Scaling to very very large corpora for natural language disambiguation, Banko M., Brill E., ACL '01: Proceedings of the 39th Annual ACL Meeting doi.org/10.3115/1073012.1073017 [1] Data >> Model ? See Also: “The Unreasonable Efectiveness of Data”, Halevy, Norvig, and Pereira, IEEE Intelligent Systems, 2009 Human Learning ≠ Machine Learning Data is extremely important in Machine Learning Human vs Machine Learning ?
11. Neural Network Architectures (not-exhaustive) Timeline MNIST CIFAR100 Focus on Models Focus on Data +
12. Data Availability Not every dataset can be open ISSUE #1
13. Develop new automated methods to detect breast cancer tumours SAMPLE RESEARCH PROJECT
14. To get started ? • Look for any available resource (dataset/benchmark/paper) Develop new automated methods to detect breast cancer tumours
15. breast imaging dataset
17. Feb. 2023 May. 2023 doi.org/10.1038/s41597-023-02100-7 doi.org/10.1148/ryai.220047 using doi.org/10.1038/s43856-024-00446-6 Feb. 2024 3.4M scans 5K scans
18. 3.4M scans 60K scans (~2%?) Permission for external collaboration by research and industry partners are reviewed on a case-by-case basis […] we have released 20% of the dataset […] allowing researchers to review the structure and content of EMBED […] 360K scans (~10%?) EMBED License […] Emory grants you permission to view and use EMBED for personal, non-commercial research. You may not otherwise copy, reproduce, retransmit, distribute, publish, commercially exploit or otherwise transfer any material.
19. 5K scans (4 views) - The LICENSEE will not attempt to identify any individual or institution referenced in PhysioNet restricted data. […] - The LICENSEE will not share access [.…] […] - The LICENSEE will use the data for […] scientific research and no other.
20. 5K scans (4 views) - The LICENSEE will not attempt to identify any individual or institution referenced in PhysioNet restricted data. […] - The LICENSEE will not share access [.…] […] - The LICENSEE will use the data for […] scientific research and no other. • Not all data is available • 60K from original 3.4M • Research cannot immediately start • Additional Admin required • License to agree and abide to • No (real) control over data use/misuse
21. Data Availability Not every dataset can be open ISSUE #1
22. Data Availability Not every dataset can be open ISSUE #1 ISSUE #1.1 We’re trying to move data where computation lives (i.e. the Copy) problem
23. Data Availability Not every dataset can be open ISSUE #1 ISSUE #1.1 We’re trying to move data where computation lives (i.e. the Copy) problem ISSUE #1.2 There is NO control over data use/misuse
24. Machine Learning Models are not always safe / reliable! ISSUE #2
25. venturebeat.com/ai/why-anthropic-and-openai-are-obsessed-with-securing-llm-model-weights/
26. youtube.com/watch?v=ySZyJipdH9Q
27. Threats and Attacks for ML systems 2008 De-Anonymisation (re-identification) 2011 
 Reconstruction Attack 2013 Parameter Inference Attack 2015 Model Inversion Attacks 2017 Membership Inference Attacks ML Model MLaaS White Box Attacks Black Box Attacks Perimeter • privacy has to be implemented systematically without using arbitrary mechanisms. • ML applications are prone to different privacy and security threats.
28. Model Vulnerabilities Adversarial Examples
29. Model Vulnerabilities Deep Learning Memoization blog.kjamistan.com/deep-learning-memorization-and-why-you-should-care.html
30. Model Vulnerabilities Deep Learning Memoization https://arxiv.org/html/2406.03880v1
31. Model Stealing Model Inversion Attacks
32. Model Stealing Model Inversion Attacks
33. leriomaggio/ppml-tutorial/2-ml-models-attacks/ Model Stealing Model Inversion Attacks 3-MIA-Reconstruction.ipynb
34. Introducing PETs Privacy Enhancing Technologies
35. PETs? HOMOMORPHIC ENCRYPTION K-ANONYMIZATION SECURE ENCLAVES FUNCTIONAL ENCRYPTION ZERO-KNOWLEDGE PROOFS SYNTHETIC DATA DIFFERENTIAL PRIVACY FEDERATED LEARNING SECURE MULTI-PARTY COMPUTATION
36. PETs and PPML • PETs have the huge potential to become the Data Science paradigm of the future • Joint effort of ML & Security & Math & Open Source Communities • Privacy-Preserving Machine Learning (PPML) refers to PETs integrated within Machine learning workflows.
37. What are PETs? HOMOMORPHIC ENCRYPTION K-ANONYMIZATION SECURE ENCLAVES FUNCTIONAL ENCRYPTION ZERO-KNOWLEDGE PROOFS SYNTHETIC DATA DIFFERENTIAL PRIVACY FEDERATED LEARNING SECURE MULTI-PARTY COMPUTATION Input Privacy Output Privacy ZERO-KNOWLEDGE PROOFS CRYPTOGRAPHIC SIGNATURES TRUST OVER IP INFRA Input Verification ACTIVE SECURITY SECURE ENCLAVES Output Verification
38. PETs, PPML, and the “Data vs Privacy” Dilemma AI models are data hungry: • The more the data, the better the model • Push for High-quality and Curated* Open Datasets * More on the Curated possible meanings in the next slide! High-sensitive data: we need to keep data safe from both intentional and accidental leakage Data &| Models are kept in silos! Privacy Enhancing Technology 
 (PETs)
39. Utility vs Privacy Trade-off The real challenge is balancing privacy and performance in ML applications so that we can better utilise the data while ensuring the privacy of the individuals. Image Credits: Johannes Stutz (@dafflowjoe) No PETs PETs
40. PETs and OpenMined Vision
41. PETS MAKE IT POSSIBLE TO: answer a question using data you cannot see.
42. PETS MAKE IT POSSIBLE TO: answer a question using data you cannot see. HOMOMORPHIC ENCRYPTION K-ANONYMIZATION SECURE ENCLAVES FUNCTIONAL ENCRYPTION ZERO-KNOWLEDGE PROOFS SYNTHETIC DATA DIFFERENTIAL PRIVACY FEDERATED LEARNING SECURE MULTI-PARTY COMPUTATION This is the ability that matters! These are just algorithms! In Another Country In Another Org In Another Dept.
43. PETs must be combined to realise their potential.
44. Remotely study data on another organisation server Org With Data Researcher OUR VISION / SOLUTION Can answer a “specific” question without seeing nor copying the data Retains governance over the information and never shares a copy of the data safeguarding privacy Systematic way! HOW?
45. Structured Transparency Framework arxiv.org/abs/2012.08347
46. Structured Transparency Framework Information Hierarchy Stakeholder Access Public Internal Private General Team Admin Governance Goal: make transparency systematic and sustainable rather than ad hoc, creating trust while protecting sensitive information.
47. Structured Transparency Framework Input Policy Data Validation Access Control Input Filtering Processing Policy Data Transform Compliance Check Audit Logging Output Policy Data Filtering Privacy Check Format Control Structured Transparency Policy Framework
48. github.com/OpenMined/PySyft pip install syft
49. PySyft: Step-by-step Researcher Univ. Wisconsin Data Owner
50. 1.1 Upload the Dataset into the Datasite 1.2 Creates an account for an external researcher 1.3 Univ. Wisconsin Datasite Data Owner … goes and has a coffee… …(or tea) Step 1: University launches the Datasite
51. 1.1 Upload the Dataset into the Datasite 1.2 Creates an account for an external researcher 1.3 Univ. Wisconsin Datasite Data Owner … goes and has a coffee… …(or tea) Step 1: University launches the Datasite
52. 1.1 Upload the Dataset into the Datasite 1.2 Creates an account for an external researcher 1.3 Univ. Wisconsin Datasite Data Owner … goes and has a coffee… …(or tea) Step 1: University launches the Datasite
53. 1.1 Upload the Dataset into the Datasite 1.2 Creates an account for an external researcher 1.3 Univ. Wisconsin Datasite Data Owner … goes and has a coffee… …(or tea) Step 1: University launches the Datasite
54. Univ. Wisconsin Datasite Data Owner rachel@datascience.inst Dr. Rachel Science ************* 1.1 Upload the Dataset into the Datasite 1.2 Creates an account for an external researcher 1.3 … goes and has a coffee… …(or tea) Step 1: University launches the Datasite
55. bye! * Univ. Wisconsin Datasite Owen, Data Owner Step 1: University launches the Datasite 1.1 Upload the Dataset into the Datasite 1.2 Creates an account for an external researcher 1.3 … goes and has a coffee… …(or tea)
56. Step 2: External Researcher proposes a project * Univ. Wisconsin Datasite Rachel, Data Scientist Research Code Project Proposal
57. 2.1 Login to Domain Server Rachel, Data Scientist 2.2 Prepare Code using public mock data Univ. Wisconsin Datasite Step 2: External Researcher proposes a project 2.3 Submit Research Project
58. 2.1 Login to Domain Server Rachel, Data Scientist 2.2 Prepare Code using public mock data Univ. Wisconsin Datasite Step 2: External Researcher proposes a project 2.3 Submit Research Project
59. Open-access
 Mock Data Securely-hosted
 Real Data 2.1 Login to Domain Server 2.2 Prepare Code using public mock data 2.3 Submit Research Project Step 2: External Researcher proposes a project Rachel, Data Scientist This is the Structured Transparency + PETs part
60. Open-access
 Mock Data Securely-hosted
 Real Data 2.1 Login to Domain Server 2.2 Prepare Code using public mock data 2.3 Submit Research Project Step 2: External Researcher proposes a project Rachel, Data Scientist This is the Structured Transparency + PETs part
61. Open-access
 Mock Data Securely-hosted
 Real Data 2.1 Login to Domain Server 2.2 Prepare Code using public mock data 2.3 Submit Research Project Step 2: External Researcher proposes a project Rachel, Data Scientist This is the Structured Transparency + PETs part
62. 2.1 Login to Domain Server 2.2 Prepare Code using public mock data 2.3 Submit Research Project Rachel, Data Scientist Univ. Wisconsin Datasite Q Q Q Step 2: External Researcher proposes a project This is the ST+PETs part Di" priv
63. 2.1 Login to Domain Server 2.2 Prepare Code using public mock data 2.3 Submit Research Project Rachel, Data Scientist Univ. Wisconsin Datasite Q Q Q Step 2: External Researcher proposes a project This is the ST+PETs part Di" priv
64. 2.1 Login to Domain Server 2.2 Prepare Code using public mock data 2.3 Submit Research Project Rachel, Data Scientist Univ. Wisconsin Datasite Q Q Q Step 2: External Researcher proposes a project This is the ST+PETs part Di" priv
65. 2.1 Login to Domain Server 2.2 Prepare Code using public mock data 2.3 Submit Research Project Rachel, Data Scientist Univ. Wisconsin Datasite Q Q Q Step 2: External Researcher proposes a project
66. 2.1 Login to Domain Server 2.2 Prepare Code using public mock data 2.3 Submit Research Project Rachel, Data Scientist Univ. Wisconsin Datasite Q Q Q Step 2: External Researcher proposes a project
67. 2.1 Login to Domain Server 2.2 Prepare Code using public mock data 2.3 Submit Research Project Rachel, Data Scientist Univ. Wisconsin Datasite Q Q Q Step 2: External Researcher proposes a project
68. PySyft function & Structured Transparency
69. Repeated Calls (e.g. ML training)
70. Q Q Q … 010101010110101 01010… … 010101010110 10101010… Step 3: Admin reviews code Owen, Data Owner Rachel, Data Scientist * A A A Univ. Wisconsin Datasite
71. 3.1 Reviews incoming code 3.3 ORGANISATION ADMIN CODE REVIEW 10010101010101010101 10101011010100101010 01000001110101010101 01011101101010101011 Submits the results 3.2 Executes the audit code … 0101010101101 0101010… * A A A Univ. Wisconsin Datasite Data Owner Step 3: Admin reviews code
72. 3.1 Reviews incoming code 3.3 ORGANISATION ADMIN CODE REVIEW 10010101010101010101 10101011010100101010 01000001110101010101 01011101101010101011 Submits the results 3.2 Executes the audit code … 0101010101101 0101010… * A A A Univ. Wisconsin Datasite Data Owner Step 3: Admin reviews code
73. 3.1 Reviews incoming code 3.3 ORGANISATION ADMIN CODE REVIEW 10010101010101010101 10101011010100101010 01000001110101010101 01011101101010101011 Submits the results 3.2 Executes the audit code … 0101010101101 0101010… * A A A Univ. Wisconsin Datasite Data Owner Step 3: Admin reviews code
74. 3.1 Reviews incoming code 3.3 ORGANISATION ADMIN CODE REVIEW 10010101010101010101 10101011010100101010 01000001110101010101 01011101101010101011 Submits the results … 0101010101101 0101010… * A A A Univ. Wisconsin Datasite Data Owner 3.2 Executes the code Step 3: Admin reviews code
75. 3.1 Reviews incoming code * A A A Univ. Wisconsin Datasite Data Owner 3.2 Executes the code Step 3: Admin reviews code APPROVED! 10010101010 10101010110 10101101010 01010100100 3.3 Submits the results
76. A A A Step 4: External Researcher downloads answers Rachel, Data Scientist Univ. Wisconsin Datasite *
77. A A A Step 4: External Researcher downloads answers Rachel, Data Scientist Univ. Wisconsin Datasite *
78. docs.openmined.org
80. OUTPUT PRIVACY Data Anonymisation
81. Privacy-Preserving Data Data Anonymisation Techniques: e.g. k-anonimity • (From Wikipedia) In the context of k-anonymization problems, a database is a table with n rows and m columns. Each row of the table represents a record relating to a specific member of a population and the entries in the various rows need not be unique. The values in the various columns are the values of attributes associated with the members of the population. Privacy as a property of Data Dataset !K-Anonymised Dataset Algorithm #1 Algorithm #2 Algorithm #k Data Sharing https://github.com/leriomaggio/privacy-preserving-data-science
82. Privacy-Preserving Data Privacy as a property of Data Patients Medical Records (Sanitised) History of Medical Prescriptions Patients bought meds from which pharmacy Pharmacies visited Roughly infer ZIP codes and residency (even without address info: e.g. most visited pharmacy) Correlation between medications and disease
83. Data Privacy Issues Source: https://venturebeat.com/2020/04/07/2020-census-data-may-not-be-as-anonymous-as-expected/ […] (we) show how these methods can be used in practice to de-anonymize the Netflix Prize dataset, a 500,000-record public dataset. Linking Attack
84. Introducing OUTPUT PRIVACY #2 Inspired from: Differential Privacy on PyTorch | PyTorch Developer Day 2020 youtu.be/l6ffbl2CBnq0 Differential Privacy
86. Source: pinterest.com/ agirlandaglobe/
90. Differential Privacy A more formal definition of Differential Privacy Like k-anonimity, di!erential privacy is a formal notion of privacy (i.e. it's possible to prove that a data release has the property). Unlike k-Anonymity however, di!erential privacy is a property of algorithms, and not a property of data.
91. Privacy Budget A more formal definition of Differential Privacy
92. The Laplace Mechanism A more formal definition of Differential Privacy
93. The Laplace Mechanism A more formal definition of Differential Privacy
94. Properties of Differential Privacy Methods Differential Privacy
95. Learning from Aggregates Introducing OPACUS Diferential Privacy within the ML Pipeline leriomaggio/ppml-tutorial/3-diferential-privacy • Aggregate Count on the Data • Computing Mean • (Complex) Train ML model Diferential Privacy within the ML Pipeline
96. leriomaggio/ppml-tutorial/3-diferential-privacy/ Model Inversion Attack after training with Differential Privacy 6-MIA-Reconstruction-OPACUS.ipynb
97. INPUT PRIVACY Federated Learning & Homomorphic Encryption
98. Introducing: Federated Learning
99. Federated Learning & Encryption
100. Federated Learning & Homomorphic Encryption What is Homomorphic Encryption? • Encryption method allowing computations on encrypted data • Result matches encryption of computation on unencrypted data • E(x) ⊕ E(y) = E(x + y) 1. Partially Homomorphic (PHE) ◦ Single operation (addition OR multiplication) ◦ Examples: RSA, Paillier 2. Fully Homomorphic (FHE) ◦ Multiple operations (both addition AND multiplication) ◦ Enables arbitrary computations ◦ Higher computational overhead Types
101. Homomorphic Encryption Example leriomaggio/ppml-tutorial/4-remote-data-science/ 2-Homomorphic-Encryption.ipynb
102. openmined.github.io/syft-heart-disease-tutorial
103. SyftBox + Introducing SyftBox https://syftbox-documentation.openmined.org/ A LIVE Network for Federated Learning
104. curl -LsSf https://syftbox.openmined.org/install.sh | sh Introducing SyftBox
105. curl -LsSf https://syftbox.openmined.org/install.sh | sh Introducing SyftBox SyftBox Client SyftBox Datasites Folder
106. syftbox.openmined.org/datasites/aggregator@openmined.org/syft_stats.html
107. FL Workflow with SyftBox
108. https://syftbox-documentation.openmined.org/ FL Workflow with SyftBox syftbox client ____ __ _ ____ / ___| _ _ / _| |_| __ ) _____ __ \___ \| | | | |_| __| _ \ / _ \ \/ / ___) | |_| | _| |_| |_) | (_) > < |____/ \__, |_| \__|____/ \___/_/\_\ |___/ Installing uv Installing SyftBox (with managed Python 3.12) Installation completed! Start the client now? [y/n] y Starting SyftBox client... 2024-12-17 18:36:42.845 | INFO | syftbox.client.client2:run_client:308 - Client metadata { "client_config": { "data_dir": "/Users/leriomaggio/SyftBox", "server_url": "https://syftbox.openmined.org/", "client_url": "http://127.0.0.1:8080/", "email": "valerio@openmined.org", "token": "6096369892341579219", "access_token": “eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJlbWFpbCI6InZhbGVyaW9Ab3Blbm1pbmVkLm9yZyaertfbd” }, "server_syftbox_version": null, "client_syftbox_version": "0.2.11", "python_version": "3.12.7 (main, Oct 1 2024, 02:05:46) [Clang 16.0.0 (clang-1600.0.26.3)]", "platform": "macOS-15.1.1-arm64-arm-64bit", "timestamp": "2024-12-17T18:36:42.845623Z", "env": { "DISABLE_ICONS": false, "CLIENT_CONFIG_PATH": "/Users/leriomaggio/.syftbox/config.json" } } Join the Live SyftBox Network
109. https://syftbox-documentation.openmined.org/ git clone https://github.com/OpenMined/fl_aggregator FL Workflow with SyftBox
110. https://syftbox-documentation.openmined.org/ git clone https://github.com/OpenMined/fl_aggregator FL Workflow with SyftBox fl_aggregator API
111. https://syftbox-documentation.openmined.org/ FL Workflow with SyftBox fl_aggregator $HOME/SyftBox/ apis datasites private API
112. https://syftbox-documentation.openmined.org/ FL Workflow with SyftBox fl_aggregator $HOME/SyftBox/ apis datasites private API
113. https://syftbox-documentation.openmined.org/ FL Workflow with SyftBox fl_aggregator API main.py run.sh SyftBox Datasites " # $ Ana Bob John
114. https://syftbox-documentation.openmined.org/ FL Workflow with SyftBox fl_aggregator API main.py run.sh " # $ Ana Bob John git clone https://github.com/OpenMined/fl_client $HOME/SyftBox/ apis datasites
115. https://syftbox-documentation.openmined.org/ FL Workflow with SyftBox fl_aggregator API main.py run.sh " # $ Ana Bob John git clone https://github.com/OpenMined/fl_client fl_client API $HOME/SyftBox/ apis datasites
116. https://syftbox-documentation.openmined.org/ FL Workflow with SyftBox fl_aggregator API main.py run.sh " # $ fl_client API /Ana/SyftBox/ apis datasites fl_client API /Bob/SyftBox/ apis datasites fl_client API /John/SyftBox/ apis datasites datasites datasites datasites
117. https://syftbox-documentation.openmined.org/ FL Workflow with SyftBox fl_aggregator API main.py run.sh " # $ /Ana/SyftBox/datasites/ana@mail.com/ /Bob/SyftBox/datasites/bob@jumbomail.com/ /Bob/SyftBox/datasites/john@bmail.com/ Local Model Local Model Local Model Global Model Accuracy
118. https://syftbox-documentation.openmined.org/ FL Workflow with SyftBox fl_aggregator API main.py run.sh " # $ /Ana/SyftBox/datasites/ana@mail.com/ /Bob/SyftBox/datasites/bob@jumbomail.com/ /Bob/SyftBox/datasites/john@bmail.com/ Global Model Accuracy
119. https://syftbox-documentation.openmined.org/ FL Workflow with SyftBox fl_aggregator API main.py run.sh " # $ /Ana/SyftBox/datasites/ana@mail.com/ /Bob/SyftBox/datasites/bob@jumbomail.com/ /Bob/SyftBox/datasites/john@bmail.com/ Global Model Accuracy
120. https://syftbox-documentation.openmined.org/ FL Workflow with SyftBox fl_aggregator API main.py run.sh " # $ /Ana/SyftBox/datasites/ana@mail.com/ /Bob/SyftBox/datasites/bob@jumbomail.com/ /Bob/SyftBox/datasites/john@bmail.com/ Global Model Accuracy
121. https://syftbox-documentation.openmined.org/ FL Workflow with SyftBox fl_aggregator API main.py run.sh " # $ /Ana/SyftBox/datasites/ana@mail.com/ /Bob/SyftBox/datasites/bob@jumbomail.com/ /Bob/SyftBox/datasites/john@bmail.com/ Global Model Accuracy
122. https://syftbox-documentation.openmined.org/ FL Workflow with SyftBox fl_aggregator API main.py run.sh " # $ /Ana/SyftBox/datasites/ana@mail.com/ /Bob/SyftBox/datasites/bob@jumbomail.com/ /Bob/SyftBox/datasites/john@bmail.com/ Global Model Accuracy
123. Thank You! Valerio Maggio @leriomaggio valerio@openmined.org in/valeriomaggio Join the Community bit.ly/feeedback-form ❤ your Feedback