pyconIT.pdf

Transcript

2. How many of you use Google Gboard?
3. Do you know how Gboard models are trained?
4. 👋 Luca Corbucci 👨💻 Ph.D. Student @ University of Pisa 🎤 Podcaster @ PointerPodcast 🦸 Co-Organizer @ Superhero Valley and Pisa.dev 🌏 lucacorbucci.me
5. 👋 Luca Corbucci 👨💻 Ph.D. Student @ University of Pisa 🎤 Podcaster @ PointerPodcast 🦸 Co-Organizer @ Superhero Valley and Pisa.dev 🌏 lucacorbucci.me
6. How classic ML works?
7. HOW CLASSIC ML WORKS COLLECT TRAINING DATA
8. HOW CLASSIC ML WORKS TRAIN A MODEL COLLECT TRAINING DATA
9. HOW CLASSIC ML WORKS TRAIN A MODEL MAKE PREDICTIONS COLLECT TRAINING DATA WHAT’S THE COLOR OF THE CAT?
11. TRAIN THE MODEL WITH THE RECEIVED DATA
12. PRO AND CONS
13. PRO AND CONS ✅ EASIER TO TRAIN & DEBUG
14. PRO AND CONS ✅ EASIER TO TRAIN & DEBUG ✅ MORE MATURE ECOSYSTEM
15. PRO AND CONS ✅ EASIER TO TRAIN & DEBUG ✅ MORE MATURE ECOSYSTEM ✅ GLOBAL VIEW ON THE DATASET
16. PRO AND CONS ✅ EASIER TO TRAIN & DEBUG ✅ MORE MATURE ECOSYSTEM ✅ GLOBAL VIEW ON THE DATASET ⚠ PRIVACY AND TRUST RISKS
17. PRO AND CONS ✅ EASIER TO TRAIN & DEBUG ✅ MORE MATURE ECOSYSTEM ✅ GLOBAL VIEW ON THE DATASET ⚠ GDPR AND SIMIlAR LAWS CAN LIMIT DATA SHARING ⚠ PRIVACY AND TRUST RISKS
18. PRO AND CONS ✅ EASIER TO TRAIN & DEBUG ✅ MORE MATURE ECOSYSTEM ✅ GLOBAL VIEW ON THE DATASET ⚠ SCALABILITY LIMITS ⚠ GDPR AND SIMIlAR LAWS CAN LIMIT DATA SHARING ⚠ PRIVACY AND TRUST RISKS
19. FEDERATED LEARNING
20. FEDERATED LEARNING THE IDEA WHY NOW? MORE COMPUTE AT THE EDGE RISE IN PRIVACY CONCERNS BETTER TOOLS TO IMPLEMENT FL * * * AGGREGATE THE SHARED MODELS * * MOVE THE MODELS, NOT THE DATA.
21. TRAIN A MODEL ON MY DATA TRAIN A MODEL ON MY DATA TRAIN A MODEL ON MY DATA TRAIN A MODEL ON MY DATA TRAIN A MODEL ON MY DATA TRAIN A MODEL ON MY DATA
22. TRAIN A MODEL ON MY DATA TRAIN A MODEL ON MY DATA TRAIN A MODEL ON MY DATA TRAIN A MODEL ON MY DATA TRAIN A MODEL ON MY DATA TRAIN A MODEL ON MY DATA
27. THE MODELS ARE AGGREGATED INTO A SINGLE MODEL
28. THE MODELS ARE AGGREGATED INTO A SINGLE MODEL
29. THE MODELS ARE AGGREGATED INTO A SINGLE MODEL USUALLY, THIS IS JUST AN AVERAGE OF THE WEIGHTS OF THE MODELS
30. Do you know how Gboard models are trained?
35. FL FRAMEWORKS
36. FL FRAMEWORKS AND COUNTLESS DIY IMPLEMENTATIONS ON GITHUB
37. FL FRAMEWORKS
38. WHY FLOWER?
39. WHY FLOWER?
40. WHY FLOWER?
41. WHY FLOWER?
42. WHY FLOWER? +
43. WHY FLOWER? +
44. WHY FLOWER? +
45. WHY FLOWER? +
46. WHY FLOWER? + SIMULATION DEPLOY
47. ARCHITECTURE SERVER CLIENT CLIENT CLIENT
48. ARCHITECTURE SERVER SERVER APP SUPERLINK CLIENT SUPERNODE CLIENT APP CLIENT SUPERNODE CLIENT APP CLIENT SUPERNODE CLIENT APP
49. DEMO!
50. RECAP Centralized Learning Federated Learning Data location All data is collected in one central server Data stays on the local device Privacy Higher risk Raw data never leaves device Regulations Difficult with sensitive data Ideal for sensitive or regulated environments Entry barrier Low (lots of tools, guides) Lowering
51. REAL-WORLD CONSIDERATIONS ⚡ Network Latency 🐢 Stragglers 🎲 Non-IID Data
52. RESOURCES TO GO FURTHER
53. RESOURCES TO GO FURTHER
54. CONCLUSIONS AS INTERESTS IN DATA PRIVACY GROWS, FL IS BECOMING INCREASINGLY IMPORTANT *
55. CONCLUSIONS AS INTERESTS IN DATA PRIVACY GROWS, FL IS BECOMING INCREASINGLY IMPORTANT * * EVEN LLM CAN BE TRAINED THIS WAY!
56. CONCLUSIONS
57. CONCLUSIONS AS INTERESTS IN DATA PRIVACY GROWS, FL IS BECOMING INCREASINGLY IMPORTANT * THE FUTURE OF AI IS FEDERATED! * * EVEN LLM CAN BE TRAINED THIS WAY!
58. Thank you! Questions? 👨💻 Ph.D. Student @ University of Pisa 🎤 Podcaster @ PointerPodcast 🦸 Co-Organizer @ Superhero Valley and Pisa.dev 🌏 lucacorbucci.me