How to deploy machine learning models in the Cloud

How to deploy machine learning models in the Cloud

Developing and experimenting with machine learning models in Python is easy and well supported by robust and agile libraries such as scikit-learn, although efficiently deploying multi-model systems at scale is still a challenge in the data science field.

This talk will focus on the main issues related to deploying machine learning models and how to make scikit-learn production-ready with minimal operational efforts, by means of Cloud Computing services, in particular Amazon Web Services.


Alex Casalboni

April 15, 2016


  1. How  to  deploy  Machine  Learning   models  in  the  Cloud

    Alex  Casalboni PYCON  SETTE   @  Firenze
  2. About  Me PYCON  SETTE  @  FIRENZE @alex_casalboni Bachelor  in

     Computer  Science Master  in  Sound  &  Music  Engineering 6+  years  of  Web  Development 3+  years  of  Python
  3. Have  you  ever  met  a  Data  Scien@st? PYCON  SETTE  @

     FIRENZE Bob Bob  leads  a  skilled  team  of  data  scienCsts Bob  has  built  a  great  ML  model Bob  loves  Python Bob  likes  user-­‐centered  ML
  4. How  did  Bob  build  his  model  in  Python? PYCON  SETTE

     @  FIRENZE Bob
  5. PYCON  SETTE  @  FIRENZE GPU  availability distributed  compuCng infrastructure-­‐aware  code

    1.  Bob  needs  Elas@city And  why  is  deploying  a  challenge? Bob
  6. PYCON  SETTE  @  FIRENZE MulC-­‐model  architecture A/B  tesCng  &  versioning

    RESTful  API 2.  Bob  needs  plenty  of  Features Auth  layer And  why  is  deploying  a  challenge? Bob
  7. PYCON  SETTE  @  FIRENZE infrastructure  maintenance conCnuous  delivery  &  deployments

    automaCon,  performance  &  monitoring 3.  Data  Scien@sts  ain’t  DevOps ≠ And  why  is  deploying  a  challenge? Data   ScienCst DevOps
  8. PYCON  SETTE  @  FIRENZE model  design,  training  &  tesCng code

     reviews    &  debugging data  exploraCon 4.  DevOps  ain’t  Data  Scien@sts ≠ And  why  is  deploying  a  challenge? Data   ScienCst DevOps
  9. Deployment  Strategies PYCON  SETTE  @  FIRENZE 1.  Web-­‐app  controller

    how  will  Bob  update  his  model(s)? same  website  auth  layer? shared  (limited)  uWSGI  processes? It’s  disarmingly  simple,  but… Bob
  10. Deployment  Strategies PYCON  SETTE  @  FIRENZE 2.  Fleet  of  servers same  problems  as  before Bob  has  many  more  machines  to  maintain sCll  no  elasCcity  (over-­‐provisioning) Bigger  capacity  and  no  code  changes,  but… Bob Bob2 Bob3 Load  Balancing
  11. Deployment  Strategies PYCON  SETTE  @  FIRENZE 3.  Auto  Scaling

    <Bob> sCll  shared  resources?  containers? even  bigger  lack  of  ownership what  about  caching,  versioning  and  auth? Bob  achieved  elasCcity  and  availability  ,  but… AWS  ELB  +  Auto  Scaling   (or  maybe  ElasCc  Beanstalk?)
  12. Deployment  Strategies PYCON  SETTE  @  FIRENZE 4.  Serverless +

    Amazon  API   Gateway AWS   Lambda Happy   Bob Infinite  scalability,  global  CDN  and  caching Transparent  elasCcity  (and  PAYG) RESTful  &  auth  layer  +  versioning  (for  free!) Bob  doesn’t  need  to  be  a  DevOps  anymore!
  13. The  Serverless  Cloud PYCON  SETTE  @  FIRENZE There is

    always a server somewhere,
 you just don't have to worry about it :)
  14. Does  Bob’s  code  change  a  lot? PYCON  SETTE  @  FIRENZE Basically,  no  change! Bob
  15. How  does  Bob  keep  the  model  updated? PYCON  SETTE  @

     FIRENZE Bob Hint:  use  gzip!

  17. Serverless  Cloud  Domina@on PYCON  SETTE  @  FIRENZE Amazon  Web  Services

     (Lambda) Microso`  Azure  (FuncCons) Google  Cloud  Plaborm  (FuncCons) IBM  Bluemix  (OpenWhisk) Cloud  PlaSorms Hybrid  &  Microservices
  18. Thank  you  :) P.S.  we  are  hiring!