Serverless microservices using AWS Lambda and Groovy

In this talk we'll explore how two revolutionary ideas come together with AWS Lambda: microservice architectures, and serverless event-driven code.

You will see how we can break a Grails monolithic app into microservices with live code, implemented in Groovy and based on AWS Lambda and Amazon DynamoDB. No servers, no instances, and no containers were harmed in the making of this session.

If you love to focus on building your applications and pushing new features instead of managing/scaling infrastructure, this talk is for you!

Demo source code: https://github.com/benorama/skillsmatter-demo

Transcript

1. SERVERLESS MICROSERVICES USING AWS LAMBDA AND GROOVY SkillMatters Groovy & Grails 2015 agora pulse 1 Benoit Hediard @benorama
2. GOING SERVERLESS ON AWS Part 1 2 VOTE APP Demo MONOLITH TO MICROSERVICES Part 2 HELLO WORLD Demo
3. About Me 3 Grails plugin author: AWS SDK, Facebook SDK, Raven, Segment… Entrepreneur since 2000 Passionate about: •  Lean/DevOps •  Cloud/AWS •  Java/Grails •  UX/UI •  Facebook/Twitter Platforms A Social Media Management Platform for Brands Benoit Hédiard Co-founder & CTO @benorama @benorama @benorama
4. 4 LEAN STARTUP Context 1 200+ customers in 150+ countries Multiple releases per week Tech rebel team = 5 devs No Ops
5. 2 revolutions lead by startups 5 Monolithic Serverless Choose your side… Startups Enterprises Infrastructure Architecture Microservices Bare metal
6. ON PREMISE INFRASTRUCTURE Pre-2006 Bare metal servers 6
7. 7 CLOUD REVOLUTION AWS Elastic Compute Cloud (EC2) + Simple Storage Service (S3)
8. 8 GOING SERVERLESS ON AWS Part 1
9. Compute EC2 Storage & Content Delivery S3 Application Services SQS AWS - 2006 9
10. Compute Storage & Content Delivery Application Services S3 API Gateway AppStream CloudSearch Elastic Transcoder SES SWF EC2 Lambda EC2 Container Service Elastic Beanstalk Elastic Load Balancing CloudFront EFS Glacier Storage Gateway Snowball Networking VPC Direct Connect Route 53 Analytics EMR Data Pipeline Kinesis Machine Learning QuickSight Elasticsearch Service Developer Tools Management Tools Security & Identity CodeCommit CodeDeploy CodePipeline CloudWatch Cloud Formation CloudTrail Config OpsWorks Service Catalog Identity & Access Management Directory Service Trusted Advisor Cloud HSM Key Management Service Web App Firewall Mobile Services Device Farm Mobile Analytics Cognito SNS Enterprise Applications WorkSpaces WorkDocs WorkMail IOT IoT Hubs Mobile Hub Database RDS DynamoDB ElastiCache RedShift Simple DB Database Migration Service AWS - 2015 SQS 10
11. On premise infrastructure - 2000 11 Applica:ons Data Run:me Middleware O/S Virtualiza:on Servers Storage Networking You manage 100% Bare metal
12. Infrastructure as a Service (IaaS) - 2006 12 Applica:ons Data Run:me Middleware O/S Virtualiza:on Servers Storage Networking You manage
13. Platform as a Service (PaaS) - 2011 13 Applica:ons Data Run:me Middleware O/S Virtualiza:on Servers Storage Networking You manage 90% Serverless!
14. Evolution of compute 14 Infrastructure EC2 Instance Apps running on servers EC2 Container Service Container Services running in containers Lambda Functions running in the cloud
15. What is AWS Lambda? 15
16. Servers scaling and pricing 16 2 EC2 instances 1 EC2 instance 1. Need to keep a minimum of server instances alive 2. Make sure server software stays up to date 3. Can’t scale up or down in a granular way
17. Lambda scaling and pricing 17 Lambda 1. No server to manage 2. Automated granular unlimited scaling 2. Do no pay for idle time
18. Polyglot services, Lambda supports: AWS Lambda 18 Event-driven services, Lambda reacts to: Limitations: 300s max duration, 100 concurrent execution, 50MB max package size Oct 2015 Nov 2014 Jun 2015 CloudWatch S3 API Gateway SNS DynamoDB Kinesis Jun 2015 Alarm responder Image processing Real time message handling DB triggers Real time streaming processing REST endpoints
19. HELLO WORLD Demo 19
20. 20 FROM MONOLITH TO MICROSERVICES Part 2
21. Quote 21 Source: http://martinfowler.com/bliki/MonolithFirst.html “Almost all the successful microservice stories have started with a monolith that got too big and was broken up” – Martin Fowler, Monolith first approach, 2015
22. Paul Graham’s Startup Curve 22 TROUGH OF SORROW J L Scale Initial enthusiasm Reality sets in Experimenting & pivoting Hockey stick Starts working Minimum Viable Product From search mode Move fast, break things Monolith Product Market Fit To execution mode Grow fast, scale things Microservices
23. 0. Layered monolith architecture 23 Domain Services Controllers Views Browser Simple to ✔ Develop ✔ Deploy (single war) ✔ Scale Issues (when the app grows) ✗ Large code base ✗ Heavy app ✗ No granular deployment ✗ No granular scaling ✗ Dev team size ✗ Single tech stack Perfect for MVP! Minimum Viable Product to validate your business hypothesis Lasagna-oriented architecture
24. 1. Back / Front boundary 24 Domain Services API Mobile client Web client Back / Front boundary Single page app (Angular, React, etc) Documented APIs
25. 2. Data boundaries 25 Domain Services API Mobile client Web client Domain SQL or NoSQL (depending on scaling/transaction requirements) Business data boundary
26. 3. Service boundaries 26 Services API Services API Domain Mobile client Web client Domain Business context boundary
27. 4. Microservices architecture 27 Services API Domain Services API Domain Services API Domain Web client Mobile client Drawbacks ✗ Service boundaries hard to defined ✗ Distributed system ✗ Deployment complexity (DevOps required) ✗ Operational overhead Benefits ✔ Small code bases ✔ Lightweight independent apps ✔ Granular scaling ✔ Independent cross- functional teams ✔ Independent tech stack Ravioli-driven architecture Perfect for growth!
28. Which microservices framework/lib? 28 ✗ Heavyweight ✗ Server required ✔ Fastest way to build an app ✔ Plugin ecosystem Minimum package size 50MB to 60MB
29. Which microservices framework/lib? 29 ✔ Lightweight ✗ Server required ✗ Plumbing required ✔ Very fast/scalable Minimum package size 11MB to 16MB
30. Minimum package size 4KB or 4MB Which microservices framework/lib? 30 ✔ Lightweight ✔ Serverless ✗ Vendor lock-in ✗ Limitations (duration, size…) AWS Lambda (java) (groovy)
31. 31 VOTE APP Demo
32. Demo app 32 http://localhost:8080 Bare metal monolith
33. 1. API CONTROLLER + SERVICE 33 Microservice: QuizVoteAPIController Data: QuizVote
34. 2. REST API GATEWAY 34 Microservice: QuizVoteAPIController Data: QuizVote
35. 3. DB EVENT PROCESSOR 35 Microservice: QuizVoteDBEventProcessor Microservice: QuizVoteAPIController Data: QuizVote QuizInsight
36. 4. FINAL ARCHITECTURE 36 Microservice: QuizVoteDBEventProcessor Static resources: VoteApp (html, js, png, …) Microservice: QuizVoteAPIController Data: QuizVote QuizInsight CDN DNS
37. Demo app 37 http://starwars.benorama.com 100% Serverless microservices!
38. 38 VICTORY!!! Bare metal servers defeated 100% serverless microservices Responsive Resilient Elastic Message-driven 100% managed by AWS
39. 39 THANK YOU The end @benorama @benorama Demo source code: https://github.com/benorama/skillsmatter-demo