How to deploy machine learning models in the Cloud

Developing and experimenting with machine learning models in Python is easy and well supported by robust and agile libraries such as scikit-learn, although efficiently deploying multi-model systems at scale is still a challenge in the data science field.

This talk will focus on the main issues related to deploying machine learning models and how to make scikit-learn production-ready with minimal operational efforts, by means of Cloud Computing services, in particular Amazon Web Services.

Transcript

1. How  to  deploy  Machine  Learning   models  in  the  Cloud Alex  Casalboni PYCON  SETTE   @  Firenze clda.co/pycon7-alex
2. About  Me PYCON  SETTE  @  FIRENZE @alex_casalboni clda.co/pycon7-alex Bachelor  in  Computer  Science Master  in  Sound  &  Music  Engineering 6+  years  of  Web  Development 3+  years  of  Python
3. Have  you  ever  met  a  Data  Scien@st? PYCON  SETTE  @  FIRENZE Bob Bob  leads  a  skilled  team  of  data  scienCsts Bob  has  built  a  great  ML  model Bob  loves  Python Bob  likes  user-­‐centered  ML clda.co/pycon7-alex
4. How  did  Bob  build  his  model  in  Python? PYCON  SETTE  @  FIRENZE clda.co/pycon7-code clda.co/pycon7-alex Bob
5. PYCON  SETTE  @  FIRENZE GPU  availability distributed  compuCng infrastructure-­‐aware  code 1.  Bob  needs  Elas@city clda.co/pycon7-alex And  why  is  deploying  a  challenge? Bob
6. PYCON  SETTE  @  FIRENZE MulC-­‐model  architecture A/B  tesCng  &  versioning RESTful  API 2.  Bob  needs  plenty  of  Features Auth  layer clda.co/pycon7-alex And  why  is  deploying  a  challenge? Bob
7. PYCON  SETTE  @  FIRENZE infrastructure  maintenance conCnuous  delivery  &  deployments automaCon,  performance  &  monitoring 3.  Data  Scien@sts  ain’t  DevOps ≠ clda.co/pycon7-alex And  why  is  deploying  a  challenge? Data   ScienCst DevOps
8. PYCON  SETTE  @  FIRENZE model  design,  training  &  tesCng code  reviews    &  debugging data  exploraCon 4.  DevOps  ain’t  Data  Scien@sts ≠ clda.co/pycon7-alex And  why  is  deploying  a  challenge? Data   ScienCst DevOps
9. Deployment  Strategies PYCON  SETTE  @  FIRENZE 1.  Web-­‐app  controller clda.co/pycon7-alex how  will  Bob  update  his  model(s)? same  website  auth  layer? shared  (limited)  uWSGI  processes? It’s  disarmingly  simple,  but… Bob
10. Deployment  Strategies PYCON  SETTE  @  FIRENZE 2.  Fleet  of  servers clda.co/pycon7-alex same  problems  as  before Bob  has  many  more  machines  to  maintain sCll  no  elasCcity  (over-­‐provisioning) Bigger  capacity  and  no  code  changes,  but… Bob Bob2 Bob3 Load  Balancing
11. Deployment  Strategies PYCON  SETTE  @  FIRENZE 3.  Auto  Scaling clda.co/pycon7-alex <Bob> sCll  shared  resources?  containers? even  bigger  lack  of  ownership what  about  caching,  versioning  and  auth? Bob  achieved  elasCcity  and  availability  ,  but… AWS  ELB  +  Auto  Scaling   (or  maybe  ElasCc  Beanstalk?)
12. Deployment  Strategies PYCON  SETTE  @  FIRENZE 4.  Serverless clda.co/pycon7-alex + Amazon  API   Gateway AWS   Lambda Happy   Bob Infinite  scalability,  global  CDN  and  caching Transparent  elasCcity  (and  PAYG) RESTful  &  auth  layer  +  versioning  (for  free!) Bob  doesn’t  need  to  be  a  DevOps  anymore!
13. The  Serverless  Cloud PYCON  SETTE  @  FIRENZE clda.co/pycon7-alex There is always a server somewhere,
 you just don't have to worry about it :)
14. Does  Bob’s  code  change  a  lot? PYCON  SETTE  @  FIRENZE clda.co/pycon7-alex Basically,  no  change! Bob
15. How  does  Bob  keep  the  model  updated? PYCON  SETTE  @  FIRENZE clda.co/pycon7-alex Bob Hint:  use  gzip!
16. PYCON  SETTE  @  FIRENZE clda.co/pycon7-lambda clda.co/pycon7-alex
17. Serverless  Cloud  Domina@on PYCON  SETTE  @  FIRENZE Amazon  Web  Services  (Lambda) Microso`  Azure  (FuncCons) Google  Cloud  Plaborm  (FuncCons) IBM  Bluemix  (OpenWhisk) hook.io Cloud  PlaSorms Hybrid  &  Microservices stackhut.com webtask.io iron.io clda.co/pycon7-alex
18. Thank  you  :) jobs.cloudacademy.com P.S.  we  are  hiring!