munz & more Serverless Computing & Machine Learning March 2018 Dr. Frank Munz

• Founded munz & more in 2007 • > 15 years Middleware, Multi Cloud, and Distributed Computing • Consulting, Conferences and High-End Training • Wrote two WebLogic and a Cloud book Dr. Frank Munz @frankmunz

Agenda • Cloud • The Serverless Journey + FaaS • Machine Learning • Q&A @frankmunz

Clouds!

API @frankmunz

Elasticity

Pay Per Use • ~1 US cent/h (micro instance), with HA & geo-distribution included • You own a massively parallel, distributed and highly available supercomputer with linear costs: 1 instance for 100 hours $= 100 instances for 1h @frankmunz

Cloud Computing API Elasticity Pay per use Fully Programmable Data Center

Serverless

AWS Lambda

FaaS: AWS Lambda • AWS announced Lambda in 2014 • Lambda is Function as a Service (FaaS) • Serverless: Abstract away all notion of infrastructure – Automated elasticity – True pay per invocation • Other serverless examples: Messaging, DBs, blob storage -> Future of cloud @frankmunz

AWS Lambda AWS SAM Local: test and locally run Lambda event: event passed to function context: runtime context callback: optional return (or null) @frankmunz

Create Lambda S3 @frankmunz

Example: REST Request

Cost Savings: Case Studies Expedia(2016): – 2.3 billions calls – 200k hours / month – $ 550 a month Postlight – API GW, Lambda, Serverless – Costs dropped 2 orders of magnitude down to $ 370 https://www.youtube.com/watch?v=gT9x9LnU_rE https://trackchanges.postlight.com/serving-39-million-requests-for-370-month-or-how-we-reduced-our-hosting-costs-by-two-orders-of-edc30a9a88cd http://serverlesscalc.com/ @frankmunz

Frameworks

Serverless Frameworks @frankmunz AWS Chalice Portability Complexity Standards

OS FaaS Platforms

FaaS OS Frameworks Overlap in functionality -> expect consolidation Who will survive? • Integration: K8s, Traefik, Zipkin, Prometheus, Kafka etc. • Adoption, adoption, adoption • FaaS as PaaS implementation? https://github.com/faas-lane/ FaaS-Lane/tree/master/candidates @frankmunz

Fn Project • Apache 2, Open Source since Java One 2017 • Fn Server (micro API GW), LB, Flow • Function / Container duality • Docker is only dependency -> polyglot • Helm chart for Kubernetes @frankmunz

Fn Flow Java 8 CompletableFutures API

Fn Demo?

Machine Learning

Stanford Course: Andrew Ng

ML Steps 1. Gather / prepare data 2. Choose ML model 3. Training (80% data) 4. Evaluation 5. Hyperparameter tuning 6. Use model for prediction

@frankmunz

No content

Export ML Model • Host on-premises / cloud as FaaS • Formats: – As POJO (H2O) – Predictive Model Markup Language (PMML), e.g. pmml package in R – Exported model + Model server (TensorFlow) @frankmunz

No content

Recommendation Engine Mock Demo? https://github.com/fmunz/fn-recommend

Demo: FaaS Bot?

Conclusion

Serverless + Machine Learning • Learn about Severless and FaaS • Start ML with top down approach – Zeppelin / Jupyter notebooks – Keep maths for later • Consider hosting your prediction model as FaaS @frankmunz

Serverless + Machine Learning • Fn Project gives you func + container • A future Fn cloud service should give you ... – True pay per use / never pay for idle – Automated scaling – Integration with other Oracle Cloud services – Standards based • Check Oracle‘s new capabilities with ML and notebooks • Soon? serverless training and prediction of ML models @frankmunz

No content

munzandmore.com/blog @frankmunz munzandmore.com/youtube

Q&A Discussion