Serverless and Machine Learning

Using Servlerless with Fn Project for Machine Learning with H2O or Tensorflow.

Transcript

1. munz & more Serverless Computing & Machine Learning March 2018 Dr. Frank Munz
2. • Founded munz & more in 2007 • > 15 years Middleware, Multi Cloud, and Distributed Computing • Consulting, Conferences and High-End Training • Wrote two WebLogic and a Cloud book Dr. Frank Munz @frankmunz
3. Agenda • Cloud • The Serverless Journey + FaaS • Machine Learning • Q&A @frankmunz
4. Clouds!
5. API @frankmunz
6. Elasticity
7. Pay Per Use • ~1 US cent/h (micro instance), with HA & geo-distribution included • You own a massively parallel, distributed and highly available supercomputer with linear costs: 1 instance for 100 hours $= 100 instances for 1h @frankmunz
8. Cloud Computing API Elasticity Pay per use Fully Programmable Data Center
9. Serverless
10. AWS Lambda
11. FaaS: AWS Lambda • AWS announced Lambda in 2014 • Lambda is Function as a Service (FaaS) • Serverless: Abstract away all notion of infrastructure – Automated elasticity – True pay per invocation • Other serverless examples: Messaging, DBs, blob storage -> Future of cloud @frankmunz
12. AWS Lambda AWS SAM Local: test and locally run Lambda event: event passed to function context: runtime context callback: optional return (or null) @frankmunz
13. Create Lambda S3 @frankmunz
14. Example: REST Request
15. Cost Savings: Case Studies Expedia(2016): – 2.3 billions calls – 200k hours / month – $ 550 a month Postlight – API GW, Lambda, Serverless – Costs dropped 2 orders of magnitude down to $ 370 https://www.youtube.com/watch?v=gT9x9LnU_rE https://trackchanges.postlight.com/serving-39-million-requests-for-370-month-or-how-we-reduced-our-hosting-costs-by-two-orders-of-edc30a9a88cd http://serverlesscalc.com/ @frankmunz
16. Frameworks
17. Serverless Frameworks @frankmunz AWS Chalice Portability Complexity Standards
18. OS FaaS Platforms
19. FaaS OS Frameworks Overlap in functionality -> expect consolidation Who will survive? • Integration: K8s, Traefik, Zipkin, Prometheus, Kafka etc. • Adoption, adoption, adoption • FaaS as PaaS implementation? https://github.com/faas-lane/ FaaS-Lane/tree/master/candidates @frankmunz
20. Fn Project • Apache 2, Open Source since Java One 2017 • Fn Server (micro API GW), LB, Flow • Function / Container duality • Docker is only dependency -> polyglot • Helm chart for Kubernetes @frankmunz
21. Fn Flow Java 8 CompletableFutures API
22. Fn Demo?
23. Machine Learning
24. Stanford Course: Andrew Ng
25. ML Steps 1. Gather / prepare data 2. Choose ML model 3. Training (80% data) 4. Evaluation 5. Hyperparameter tuning 6. Use model for prediction
26. @frankmunz
28. Export ML Model • Host on-premises / cloud as FaaS • Formats: – As POJO (H2O) – Predictive Model Markup Language (PMML), e.g. pmml package in R – Exported model + Model server (TensorFlow) @frankmunz
30. Recommendation Engine Mock Demo? https://github.com/fmunz/fn-recommend
31. Demo: FaaS Bot?
32. Conclusion
33. Serverless + Machine Learning • Learn about Severless and FaaS • Start ML with top down approach – Zeppelin / Jupyter notebooks – Keep maths for later • Consider hosting your prediction model as FaaS @frankmunz
34. Serverless + Machine Learning • Fn Project gives you func + container • A future Fn cloud service should give you ... – True pay per use / never pay for idle – Automated scaling – Integration with other Oracle Cloud services – Standards based • Check Oracle‘s new capabilities with ML and notebooks • Soon? serverless training and prediction of ML models @frankmunz
36. munzandmore.com/blog @frankmunz munzandmore.com/youtube
37. Q&A Discussion