Just Deploy It! How to Ship Your ML Model to Production

Once upon a time in the kingdom of Artificial Intelligence, there were data scientists who worked hard on their complex researches and ML models development. But then the day of atonement came - time to deploy the models to production. And at first, it seemed that everything they’ve been building so hard started to fall apart slowly piece by piece. But as data scientist read more manuscripts they’ve discovered there are many ways to ship your model using the open-source library with not much effort to be applied.
This talk will uncover the ways to ship the model to production - automated as well as potential custom solutions and also tools and libraries which can ease this process.

Transcript

1. Just deploy it! How to ship your ML model to production. M A R I A N N A D I A C H U K 1
2. @dark_matter88_ A B O U T M E • Data scientist based in Kyiv (Ukraine) • Developed and deployed multiple ensemble models • Leaded a small but proud team of 2 scientists and 1 data engineer • Supports use of software engineering best practices in data science 2
3. @dark_matter88_ A G E N D A 1. Things to focus on before deployment 2. How to deploy your model from Jupyter Notebooks 3. How to build a custom solution 3
4. @dark_matter88_ O N C E U P O N A T I M E T H E R E W E R E D ATA S C I E N T I S T S D O I N G T H E I R M A G I C R E S E A R C H … 4
5. @dark_matter88_ O N C E U P O N A T I M E T H E R E W E R E D ATA S C I E N T I S T S D O I N G T H E I R M A G I C R E S E A R C H … 5
6. @dark_matter88_ W H AT T O S TA RT F R O M ? ? ? ? ? 6
7. @dark_matter88_ 7 D ATA S C I E N T I S T
8. @dark_matter88_ 8 quality of predictions insights cleanliness of data D ATA S C I E N T I S T
9. @dark_matter88_ 9 quality of predictions insights cleanliness of data S O F T WA R E E N G I N E E R D ATA S C I E N T I S T
10. @dark_matter88_ 10 quality of predictions insights cleanliness of data quality of code user acceptance test coverage maintainability… S O F T WA R E E N G I N E E R D ATA S C I E N T I S T
11. @dark_matter88_ W H AT D O W E H AV E ? 11
12. @dark_matter88_ W H AT D O W E H AV E ? Model 12
13. @dark_matter88_ W H AT D O W E H AV E ? Model Features 13
14. @dark_matter88_ W H AT D O W E H AV E ? Model Features 14
15. @dark_matter88_ W H AT D O W E H AV E ? Model Features Raw data 15
16. @dark_matter88_ W H AT D O W E H AV E ? Model Features Raw data 16
17. @dark_matter88_ W H AT D O W E H AV E ? Model Features Raw data Pipeline 17
18. @dark_matter88_ W H AT D O W E H AV E ? 18
19. @dark_matter88_ W H AT D O W E H AV E ? 19 Research environment
20. @dark_matter88_ W H AT D O W E H AV E ? 20 System with its characteristics Research environment
21. @dark_matter88_ D E P L O Y M E N T D E S I G N D E P E N D S O N … • Type of production data 21
22. @dark_matter88_ D E P L O Y M E N T D E S I G N D E P E N D S O N … • Type of production data 22 • Architecture of production system
23. @dark_matter88_ A R C H I T E C T U R E O F P R O D U C T I O N S Y S T E M part of monolith system M L 23
24. @dark_matter88_ A R C H I T E C T U R E O F P R O D U C T I O N S Y S T E M part of monolith system M L microservice M L 24
25. @dark_matter88_ A R C H I T E C T U R E O F P R O D U C T I O N S Y S T E M part of monolith system M L microservice M L M L module 25
26. @dark_matter88_ • Type of production data • Architecture of production system • Infrastructure of production system 26 D E P L O Y M E N T D E S I G N D E P E N D S O N …
27. @dark_matter88_ 27 D E P L O Y M E N T D E S I G N D E P E N D S O N …
28. @dark_matter88_ Y O U C A N D E P L O Y Y O U R M O D E L … 28 - as a part of the system
29. @dark_matter88_ Y O U C A N D E P L O Y Y O U R M O D E L … 29 - from - as a part of the system
30. @dark_matter88_ Y O U C A N D E P L O Y Y O U R M O D E L … - moving your pipeline to frontend 30 - from - as a part of the system
31. @dark_matter88_ Y O U C A N D E P L O Y Y O U R M O D E L … 31 - from - as a part of the system
32. @dark_matter88_ D E P L O Y I N G F R O M 32
33. @dark_matter88_ D E P L O Y I N G F R O M I S N O T R E A L LY G O O D ? 33
34. @dark_matter88_ D E P L O Y I N G F R O M IMHO I S N O T R E A L LY G O O D ? 34
35. @dark_matter88_ D E P L O Y I N G F R O M IMHO I S N O T R E A L LY G O O D ? 35 • Non-linear workflow
36. @dark_matter88_ D E P L O Y I N G F R O M IMHO I S N O T R E A L LY G O O D ? 36 • Hard to test • Non-linear workflow
37. @dark_matter88_ D E P L O Y I N G F R O M IMHO I S N O T R E A L LY G O O D ? 37 • Hard to test • Limits and complicates support • Non-linear workflow
38. @dark_matter88_ 38 D E P L O Y I N G F R O M J U P Y T E R N O T E B O O K S
39. @dark_matter88_ 39 D E P L O Y I N G F R O M J U P Y T E R N O T E B O O K S
40. @dark_matter88_ D E P L O Y I N G F R O M J U P Y T E R N O T E B O O K S 40
41. @dark_matter88_ D E P L O Y I N G F R O M J U P Y T E R N O T E B O O K S 41
42. @dark_matter88_ D E P L O Y I N G F R O M J U P Y T E R N O T E B O O K S 42
43. @dark_matter88_ D E P L O Y I N G F R O M J U P Y T E R N O T E B O O K S 43
44. @dark_matter88_ 44 D E P L O Y I N G F R O M J U P Y T E R N O T E B O O K S
45. @dark_matter88_ 45 D E P L O Y I N G F R O M J U P Y T E R N O T E B O O K S
46. @dark_matter88_ 46 D E P L O Y I N G F R O M J U P Y T E R N O T E B O O K S
47. @dark_matter88_ A P I S Y S T E M request response 47
48. @dark_matter88_ C U S T O M D E P L O Y M E N T A P I 1 48
49. @dark_matter88_ C U S T O M D E P L O Y M E N T A P I 1 2 49
50. @dark_matter88_ C U S T O M D E P L O Y M E N T A P I 3 get prediction prediction 50
51. @dark_matter88_ C U S T O M D E P L O Y M E N T A P I S Y S T E M request response 0 51
52. @dark_matter88_ C U S T O M D E P L O Y M E N T 52
53. @dark_matter88_ C U S T O M D E P L O Y M E N T - lightweight - more freedom in modules design - more adjustments - a lot of built-in features 53
54. @dark_matter88_ C U S T O M D E P L O Y M E N T - lightweight - more freedom in modules design - more adjustments - a lot of built-in features 54
55. @dark_matter88_ C U S T O M D E P L O Y M E N T - lightweight - more freedom in modules design - more adjustments - a lot of built-in features 55
56. @dark_matter88_ C U S T O M D E P L O Y M E N T 2 56
57. @dark_matter88_ C U S T O M D E P L O Y M E N T 57 0 Feature_1 1 Feature_2 2 Feature_3 3 Feature_4 4 Feature_5 5 Feature_6 6 Feature_7
58. @dark_matter88_ C U S T O M D E P L O Y M E N T 58 0 Feature_1 1 Feature_2 2 Feature_3 3 Feature_4 4 Feature_5 5 Feature_6 6 Feature_7 model 1 model 2
59. @dark_matter88_ C U S T O M D E P L O Y M E N T 59
60. @dark_matter88_ M O N I T O R I N G & S C A L I N G 60
61. @dark_matter88_ M O N I T O R I N G & S C A L I N G 61
62. @dark_matter88_ M O N I T O R I N G & S C A L I N G 62
63. @dark_matter88_ J U S T D E P L O Y I T ! 63
64. @dark_matter88_ J U S T D E P L O Y I T ! 64
65. @dark_matter88_ T H A N K Y O U F O R Y O U AT T E N T I O N 65
66. @dark_matter88_ C O N TA C T M E marianna-diachuk @dark_matter88_ @mariaannadiachuk DarkMatter88 66
67. @dark_matter88_ Q & A S E S S I O N 67 ? ? ?