Federated Learning using Deep Learning

Federated learning is a family of Machine Learning algorithms that has the core idea: a connected network exists in which there is a central server node. Each of the nodes creates data – that has to be used for training as well as for prediction. Each of the nodes trains a local model and only that model is shared with the server, not the data.
In this talk, We talk about how to build deep learning models using federated learning that is truly privacy-preserving. We will show how to build custom algorithms and loss functions.
Key Takeaways:
Introduction to Federated Learning
Decentralized Training
Encryption
Differential Privacy
Federated Learning – Notebook
Introduction
Custom algorithm and loss function

Transcript

1. BARGAVA SUBRAMANIAN TUHIN SHARMA Federated Learning
2. AGENDA 1 2 3 4 FEDERATED LEARNING TOOLS & CHALLENGES DEMO (notebook) CURRENT LEARNING PARADIGM
3. Centralized Machine Learning AI/ML
4. BUT We live in a Smart Connected World
5. AI/ML Cost of Data Transfer Bandwidth Feasibility Challenges
6. Challenges AI/ML Data Privacy Compliance Awareness / Trust
7. USE CASES Anomaly Detection Cybersecurity Preventive Maintenance Sales & Marketing
8. GOAL Find anomalies across a stream of time series events Sensor Data Network Data
9. DATA FLOW
10. NETWORK SETUP
11. HOW FEDERATED LEARNING SOLVES THIS? DECENTRALIZED LEARNING SECURE COMPUTING PRESERVE PRIVACY
12. Decentralized Learning Federated Construction Pre-Trained Model
13. Decentralized Learning Decentralized Training Data Data Data Data
14. Decentralized Learning Aggregation
15. Decentralized Learning Rinse - Repeat
16. PROBLEM!
17. Inference Attack Down: W t+1 Up: W n t
18. Inference Attack Differential Privacy Clip Gradients Noise Average
19. Model Poisoning
20. Model Poisoning Sybil Detection Adaptive Learning per Client Adversarial Network
21. Secure Computing Train on Encrypted Data
22. Types of Federated Learning SINGLE PARTY MULTI PARTY
23. SINGLE PARTY
24. MULTI PARTY VERTICAL FL HORIZONTAL FL
25. TOOLS TENSORFLOW PySyft Privacy Federated
26. TOOLS TENSORFLOW Lite Deployment PyTorch Mobile
27. TOOLS TENSORFLOW Lite Deployment PyTorch Mobile Pruning Quantization Graph --> C++
28. Challenges Not IID Model Convergence Time Limited Deployment Choices
29. DEMO
30. BENEFITS • Better model accuracy • Lower latency • Lesser power consumption • Lesser network load • Privacy • Can be used across organizations* • Can be used immediately*
31. BENEFITS • Better model accuracy • Lower latency • Lesser power consumption • Lesser network load • Privacy • Can be used across organizations* • Can be used immediately*
32. • https://github.com/tuhinsharma121/federated-ml/ • https://www.tensorflow.org/federated • https://github.com/tensorflow/privacy • https://github.com/uTensor/uTensor • https://github.com/OpenMined/PySyft • https://pytorch.org/mobile/home/ Resources
33. THANKS! BARGAVA SUBRAMANIAN @bargava TUHIN SHARMA @tuhinsharma121