Technical Debt in Machine Learning

Presentation regarding the paper: https://ai.google/research/pubs/pub43146

Transcript

1. Technical Debt in Machine Learning
2. whoiam QR Code
3. Why Machine Learning is so wonderful...
4. Why Machine Learning is so wonderful...
5. But everything has a price...
6. ML - Overview Inputs / Features Machine Learning Algorithm / Model Output
7. ML - Examples Personal Information Decision Tree / Random Forest Positive Number Address, Age, Education, Taxes, Last Name… Each observation is a row A single number between R$954 and R$1.79 trillion
8. ML - Examples Image Neural Network Probability Is it a cat or a dog? Matrix 32x32x3 where 3 is the RGA representation Each observation is a matrix 3D Probability to be a cat
9. ML - Training (supervised learning) Inputs / Features Machine Learning Algorithm / Model Output ???
10. ML - Training (supervised learning) Inputs / Features Output Which algorithm is appropriated for this problem? How I find the best parameters? What am I trying to optimize? What is my cost / loss? How should I evaluate it?
11. ML - Training (supervised learning) Inputs / Features Output Linear Algebra Statistics Calculus, convex and non convex optimization Business
12. ML - Training (supervised learning) Inputs / Features Train / Find best parameters and hyperparameters
13. ML - Training (supervised learning) Inputs / Features + Labels Labels: *Observed “outputs” from historical data* Train / Find best parameters and hyperparameters
14. ML - Training (supervised learning) Inputs / Features + Labels Labels: *Observed “outputs” from historical data* Train / Find best parameters and hyperparameters Evaluation Deploy API call Library, package Job
15. ML - Training (supervised learning) Inputs / Features + Labels Labels: *Observed “outputs” from historical data* Train / Find best parameters and hyperparameters Evaluation Deploy Remember it! API call Library, package Job
16. ML - Training (supervised learning) Inputs / Features + Labels Labels: *Observed “outputs” from historical data* Train / Find best parameters and hyperparameters Evaluation Deploy Remember it! Getting and cleaning data take about 80% of working time (not sexy) Most of companies do not have a mature data warehouse to support it API call Library, package Job
17. ML - Batch and Real Time Batch - You can wait to get the answer (5 min, 1 day...) Real Time - You can’t more than one second
18. ML - Batch and Real Time Batch - You can wait to get the answer (5 min, 1 day...) - It is closer to a job Real Time - You can’t more than one second - It is closer to a service
19. ML - Batch and Real Time Batch - You can wait to get the answer (5 min, 1 day...) - It is closer to a job - It is possible to wait until an information be available Real Time - You can’t more than one second - It is closer to a service - Has to give a answer even if some information is missing
20. ML - Batch and Real Time Batch - You can wait to get the answer (5 min, 1 day...) - It is closer to a job - It is possible to wait until an information be available - It can take more information in account Real Time - You can’t more than one second - It is closer to a service - Has to give a answer even if some information is missing - Information is limited
21. ML - Batch and Real Time Batch - You can wait to get the answer (5 min, 1 day...) - It is closer to a job - It is possible to wait until an information be available - It can take more information in account - Model can be more complex (more time to run) Real Time - You can’t more than one second - It is closer to a service - Has to give a answer even if some information is missing - Information is limited - Model have to be simpler
22. ML - Batch and Real Time Batch - You can wait to get the answer (5 min, 1 day...) - It is closer to a job - It is possible to wait until an information be available - It can take more information in account - Model can be more complex (more time to run) - Deployment is usually easier Real Time - You can’t more than one second - It is closer to a service - Has to give a answer even if some information is missing - Information is limited - Model have to be simpler - Deployment is usually harder
23. ML - Batch and Real Time Batch - You can wait to get the answer (5 min, 1 day...) - It is closer to a job - It is possible to wait until an information be available - It can take more information in account - Model can be more complex (more time to run) - Deployment is usually easier - Does not affect at lot your current infra Real Time - You can’t more than one second - It is closer to a service - Has to give a answer even if some information is missing - Information is limited - Model have to be simpler - Deployment is usually harder - Affects at lot your current infra Cool project to deploy Cool project in C - YOLO
24. Technical Debt - What is it?
25. Technical Debt - What is it?
26. Technical Debt - What is it?
27. Technical Debt - System Level Spaghetti
28. Technical Debt - System Level Spaghetti
29. Technical Debt - System Level Spaghetti … due to “engineering” and “research” roles (who train is not the person who deploy)
30. Technical Debt - System Level Spaghetti … due to “engineering” and “research” roles (who train is not the person who deploy)
31. Technical Debt - Dependencies
32. Technical Debt - Dependencies
33. Technical Debt - Dependencies
34. Technical Debt - Dependencies
35. Technical Debt - External World
36. Technical Debt - External World
37. Technical Debt - External World
38. Technical Debt - Complex Models
39. Technical Debt - Complex Models
40. Technical Debt - Complex Models
41. Privacy and Ethics - - - -
42. Privacy and Ethics - - - - - - - - -
43. Hard Decisions...
44. Summary
45. Summary
46. Summary
47. Summary
48. Summary
49. Summary
50. References are in the slides as links! Thank you ;) Questions?