Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Serverless and Machine Learning

Serverless and Machine Learning

Using Servlerless with Fn Project for Machine Learning with H2O or Tensorflow.

643cd45dcfa73b072018046e39ed36d1?s=128

Frank Munz

March 21, 2018
Tweet

More Decks by Frank Munz

Other Decks in Technology

Transcript

  1. munz & more Serverless Computing & Machine Learning March 2018

    Dr. Frank Munz
  2. • Founded munz & more in 2007 • > 15

    years Middleware, Multi Cloud, and Distributed Computing • Consulting, Conferences and High-End Training • Wrote two WebLogic and a Cloud book Dr. Frank Munz @frankmunz
  3. Agenda • Cloud • The Serverless Journey + FaaS •

    Machine Learning • Q&A @frankmunz
  4. Clouds!

  5. API @frankmunz

  6. Elasticity

  7. Pay Per Use • ~1 US cent/h (micro instance), with

    HA & geo-distribution included • You own a massively parallel, distributed and highly available supercomputer with linear costs: 1 instance for 100 hours $= 100 instances for 1h @frankmunz
  8. Cloud Computing API Elasticity Pay per use Fully Programmable Data

    Center
  9. Serverless

  10. AWS Lambda

  11. FaaS: AWS Lambda • AWS announced Lambda in 2014 •

    Lambda is Function as a Service (FaaS) • Serverless: Abstract away all notion of infrastructure – Automated elasticity – True pay per invocation • Other serverless examples: Messaging, DBs, blob storage -> Future of cloud @frankmunz
  12. AWS Lambda AWS SAM Local: test and locally run Lambda

    event: event passed to function context: runtime context callback: optional return (or null) @frankmunz
  13. Create Lambda S3 @frankmunz

  14. Example: REST Request

  15. Cost Savings: Case Studies Expedia(2016): – 2.3 billions calls –

    200k hours / month – $ 550 a month Postlight – API GW, Lambda, Serverless – Costs dropped 2 orders of magnitude down to $ 370 https://www.youtube.com/watch?v=gT9x9LnU_rE https://trackchanges.postlight.com/serving-39-million-requests-for-370-month-or-how-we-reduced-our-hosting-costs-by-two-orders-of-edc30a9a88cd http://serverlesscalc.com/ @frankmunz
  16. Frameworks

  17. Serverless Frameworks @frankmunz AWS Chalice Portability Complexity Standards

  18. OS FaaS Platforms

  19. FaaS OS Frameworks Overlap in functionality -> expect consolidation Who

    will survive? • Integration: K8s, Traefik, Zipkin, Prometheus, Kafka etc. • Adoption, adoption, adoption • FaaS as PaaS implementation? https://github.com/faas-lane/ FaaS-Lane/tree/master/candidates @frankmunz
  20. Fn Project • Apache 2, Open Source since Java One

    2017 • Fn Server (micro API GW), LB, Flow • Function / Container duality • Docker is only dependency -> polyglot • Helm chart for Kubernetes @frankmunz
  21. Fn Flow Java 8 CompletableFutures API

  22. Fn Demo?

  23. Machine Learning

  24. Stanford Course: Andrew Ng

  25. ML Steps 1. Gather / prepare data 2. Choose ML

    model 3. Training (80% data) 4. Evaluation 5. Hyperparameter tuning 6. Use model for prediction
  26. @frankmunz

  27. None
  28. Export ML Model • Host on-premises / cloud as FaaS

    • Formats: – As POJO (H2O) – Predictive Model Markup Language (PMML), e.g. pmml package in R – Exported model + Model server (TensorFlow) @frankmunz
  29. None
  30. Recommendation Engine Mock Demo? https://github.com/fmunz/fn-recommend

  31. Demo: FaaS Bot?

  32. Conclusion

  33. Serverless + Machine Learning • Learn about Severless and FaaS

    • Start ML with top down approach – Zeppelin / Jupyter notebooks – Keep maths for later • Consider hosting your prediction model as FaaS @frankmunz
  34. Serverless + Machine Learning • Fn Project gives you func

    + container • A future Fn cloud service should give you ... – True pay per use / never pay for idle – Automated scaling – Integration with other Oracle Cloud services – Standards based • Check Oracle‘s new capabilities with ML and notebooks • Soon? serverless training and prediction of ML models @frankmunz
  35. None
  36. munzandmore.com/blog @frankmunz munzandmore.com/youtube

  37. Q&A Discussion