Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Serverless and Machine Learning

Serverless and Machine Learning

Using Servlerless with Fn Project for Machine Learning with H2O or Tensorflow.

Frank Munz

March 21, 2018
Tweet

More Decks by Frank Munz

Other Decks in Technology

Transcript

  1. • Founded munz & more in 2007 • > 15

    years Middleware, Multi Cloud, and Distributed Computing • Consulting, Conferences and High-End Training • Wrote two WebLogic and a Cloud book Dr. Frank Munz @frankmunz
  2. Agenda • Cloud • The Serverless Journey + FaaS •

    Machine Learning • Q&A @frankmunz
  3. Pay Per Use • ~1 US cent/h (micro instance), with

    HA & geo-distribution included • You own a massively parallel, distributed and highly available supercomputer with linear costs: 1 instance for 100 hours $= 100 instances for 1h @frankmunz
  4. FaaS: AWS Lambda • AWS announced Lambda in 2014 •

    Lambda is Function as a Service (FaaS) • Serverless: Abstract away all notion of infrastructure – Automated elasticity – True pay per invocation • Other serverless examples: Messaging, DBs, blob storage -> Future of cloud @frankmunz
  5. AWS Lambda AWS SAM Local: test and locally run Lambda

    event: event passed to function context: runtime context callback: optional return (or null) @frankmunz
  6. Cost Savings: Case Studies Expedia(2016): – 2.3 billions calls –

    200k hours / month – $ 550 a month Postlight – API GW, Lambda, Serverless – Costs dropped 2 orders of magnitude down to $ 370 https://www.youtube.com/watch?v=gT9x9LnU_rE https://trackchanges.postlight.com/serving-39-million-requests-for-370-month-or-how-we-reduced-our-hosting-costs-by-two-orders-of-edc30a9a88cd http://serverlesscalc.com/ @frankmunz
  7. FaaS OS Frameworks Overlap in functionality -> expect consolidation Who

    will survive? • Integration: K8s, Traefik, Zipkin, Prometheus, Kafka etc. • Adoption, adoption, adoption • FaaS as PaaS implementation? https://github.com/faas-lane/ FaaS-Lane/tree/master/candidates @frankmunz
  8. Fn Project • Apache 2, Open Source since Java One

    2017 • Fn Server (micro API GW), LB, Flow • Function / Container duality • Docker is only dependency -> polyglot • Helm chart for Kubernetes @frankmunz
  9. ML Steps 1. Gather / prepare data 2. Choose ML

    model 3. Training (80% data) 4. Evaluation 5. Hyperparameter tuning 6. Use model for prediction
  10. Export ML Model • Host on-premises / cloud as FaaS

    • Formats: – As POJO (H2O) – Predictive Model Markup Language (PMML), e.g. pmml package in R – Exported model + Model server (TensorFlow) @frankmunz
  11. Serverless + Machine Learning • Learn about Severless and FaaS

    • Start ML with top down approach – Zeppelin / Jupyter notebooks – Keep maths for later • Consider hosting your prediction model as FaaS @frankmunz
  12. Serverless + Machine Learning • Fn Project gives you func

    + container • A future Fn cloud service should give you ... – True pay per use / never pay for idle – Automated scaling – Integration with other Oracle Cloud services – Standards based • Check Oracle‘s new capabilities with ML and notebooks • Soon? serverless training and prediction of ML models @frankmunz