Serverless and Machine Learning

Serverless and Machine Learning

Using Servlerless with Fn Project for Machine Learning with H2O or Tensorflow.

643cd45dcfa73b072018046e39ed36d1?s=128

Frank Munz

March 21, 2018
Tweet

Transcript

  1. 2.

    • Founded munz & more in 2007 • > 15

    years Middleware, Multi Cloud, and Distributed Computing • Consulting, Conferences and High-End Training • Wrote two WebLogic and a Cloud book Dr. Frank Munz @frankmunz
  2. 3.

    Agenda • Cloud • The Serverless Journey + FaaS •

    Machine Learning • Q&A @frankmunz
  3. 4.
  4. 7.

    Pay Per Use • ~1 US cent/h (micro instance), with

    HA & geo-distribution included • You own a massively parallel, distributed and highly available supercomputer with linear costs: 1 instance for 100 hours $= 100 instances for 1h @frankmunz
  5. 11.

    FaaS: AWS Lambda • AWS announced Lambda in 2014 •

    Lambda is Function as a Service (FaaS) • Serverless: Abstract away all notion of infrastructure – Automated elasticity – True pay per invocation • Other serverless examples: Messaging, DBs, blob storage -> Future of cloud @frankmunz
  6. 12.

    AWS Lambda AWS SAM Local: test and locally run Lambda

    event: event passed to function context: runtime context callback: optional return (or null) @frankmunz
  7. 15.

    Cost Savings: Case Studies Expedia(2016): – 2.3 billions calls –

    200k hours / month – $ 550 a month Postlight – API GW, Lambda, Serverless – Costs dropped 2 orders of magnitude down to $ 370 https://www.youtube.com/watch?v=gT9x9LnU_rE https://trackchanges.postlight.com/serving-39-million-requests-for-370-month-or-how-we-reduced-our-hosting-costs-by-two-orders-of-edc30a9a88cd http://serverlesscalc.com/ @frankmunz
  8. 19.

    FaaS OS Frameworks Overlap in functionality -> expect consolidation Who

    will survive? • Integration: K8s, Traefik, Zipkin, Prometheus, Kafka etc. • Adoption, adoption, adoption • FaaS as PaaS implementation? https://github.com/faas-lane/ FaaS-Lane/tree/master/candidates @frankmunz
  9. 20.

    Fn Project • Apache 2, Open Source since Java One

    2017 • Fn Server (micro API GW), LB, Flow • Function / Container duality • Docker is only dependency -> polyglot • Helm chart for Kubernetes @frankmunz
  10. 22.
  11. 25.

    ML Steps 1. Gather / prepare data 2. Choose ML

    model 3. Training (80% data) 4. Evaluation 5. Hyperparameter tuning 6. Use model for prediction
  12. 27.
  13. 28.

    Export ML Model • Host on-premises / cloud as FaaS

    • Formats: – As POJO (H2O) – Predictive Model Markup Language (PMML), e.g. pmml package in R – Exported model + Model server (TensorFlow) @frankmunz
  14. 29.
  15. 33.

    Serverless + Machine Learning • Learn about Severless and FaaS

    • Start ML with top down approach – Zeppelin / Jupyter notebooks – Keep maths for later • Consider hosting your prediction model as FaaS @frankmunz
  16. 34.

    Serverless + Machine Learning • Fn Project gives you func

    + container • A future Fn cloud service should give you ... – True pay per use / never pay for idle – Automated scaling – Integration with other Oracle Cloud services – Standards based • Check Oracle‘s new capabilities with ML and notebooks • Soon? serverless training and prediction of ML models @frankmunz
  17. 35.