Serverless microservices using AWS Lambda and Groovy

In this talk we'll explore how two revolutionary ideas come together with AWS Lambda: microservice architectures, and serverless event-driven code.

You will see how we can break a Grails monolithic app into microservices with live code, implemented in Groovy and based on AWS Lambda and Amazon DynamoDB. No servers, no instances, and no containers were harmed in the making of this session.

If you love to focus on building your applications and pushing new features instead of managing/scaling infrastructure, this talk is for you!

Demo source code: https://github.com/benorama/skillsmatter-demo

Transcript

1. SERVERLESS MICROSERVICES
   USING AWS LAMBDA AND GROOVY
   SkillMatters Groovy & Grails 2015
   agora pulse
   1
   Benoit Hediard
   @benorama
   

   View Slide

2. GOING SERVERLESS ON AWS
   Part 1
   2
   VOTE APP
   Demo
   MONOLITH TO MICROSERVICES
   Part 2
   HELLO WORLD
   Demo
   

   View Slide

3. About Me
   3
   Grails plugin author: AWS SDK, Facebook SDK, Raven, Segment…
   Entrepreneur since 2000
   Passionate about:
   •  Lean/DevOps
   •  Cloud/AWS
   •  Java/Grails
   •  UX/UI
   •  Facebook/Twitter Platforms
   A Social Media Management
   Platform for Brands
   Benoit Hédiard
   Co-founder & CTO
   @benorama
   @benorama
   @benorama
   

   View Slide

4. 4
   LEAN STARTUP
   Context
   1 200+ customers
   in 150+ countries
   Multiple releases per week
   Tech rebel team = 5 devs
   No Ops
   

   View Slide

5. 2 revolutions lead by startups
   5
   Monolithic
   Serverless
   Choose your side…
   Startups
   Enterprises
   Infrastructure
   Architecture
   Microservices
   Bare metal
   

   View Slide

6. ON PREMISE INFRASTRUCTURE
   Pre-2006
   Bare metal servers
   6
   

   View Slide

7. 7
   CLOUD REVOLUTION
   AWS Elastic Compute Cloud (EC2) + Simple Storage Service (S3)
   

   View Slide

8. 8
   GOING SERVERLESS ON AWS
   Part 1
   

   View Slide

9. Compute
   EC2
   Storage & Content Delivery
   S3
   Application Services
   SQS
   AWS - 2006
   9
   

   View Slide

10. Compute
    Storage & Content Delivery Application Services
    S3
    API Gateway AppStream CloudSearch
    Elastic
    Transcoder
    SES SWF
    EC2 Lambda
    EC2 Container
    Service
    Elastic
    Beanstalk
    Elastic Load
    Balancing
    CloudFront EFS Glacier
    Storage
    Gateway
    Snowball
    Networking
    VPC
    Direct
    Connect
    Route 53
    Analytics
    EMR Data Pipeline Kinesis
    Machine
    Learning
    QuickSight
    Elasticsearch
    Service
    Developer Tools Management Tools Security & Identity
    CodeCommit CodeDeploy CodePipeline CloudWatch
    Cloud
    Formation
    CloudTrail Config OpsWorks
    Service
    Catalog
    Identity &
    Access
    Management
    Directory
    Service
    Trusted
    Advisor
    Cloud HSM
    Key
    Management
    Service
    Web App
    Firewall
    Mobile Services
    Device Farm
    Mobile
    Analytics
    Cognito SNS
    Enterprise Applications
    WorkSpaces WorkDocs WorkMail
    IOT
    IoT
    Hubs
    Mobile Hub
    Database
    RDS DynamoDB ElastiCache RedShift Simple DB
    Database
    Migration
    Service
    AWS - 2015
    SQS
    10
    

    View Slide

11. On premise infrastructure - 2000
    11
    Applica:ons
    Data
    Run:me
    Middleware
    O/S
    Virtualiza:on
    Servers
    Storage
    Networking
    You manage
    100% Bare metal
    

    View Slide

12. Infrastructure as a Service (IaaS) - 2006
    12
    Applica:ons
    Data
    Run:me
    Middleware
    O/S
    Virtualiza:on
    Servers
    Storage
    Networking
    You manage
    

    View Slide

13. Platform as a Service (PaaS) - 2011
    13
    Applica:ons
    Data
    Run:me
    Middleware
    O/S
    Virtualiza:on
    Servers
    Storage
    Networking
    You manage
    90% Serverless!
    

    View Slide

14. Evolution of compute
    14
    Infrastructure
    EC2
    Instance
    Apps
    running on servers
    EC2 Container
    Service
    Container
    Services
    running in containers
    Lambda
    Functions
    running in the cloud
    

    View Slide

15. What is AWS Lambda?
    15
    

    View Slide

16. Servers scaling and pricing
    16
    2 EC2
    instances
    1 EC2
    instance
    1. Need to keep a minimum
    of server instances alive
    2. Make sure server software
    stays up to date
    3. Can’t scale up or down
    in a granular way
    

    View Slide

17. Lambda scaling and pricing
    17
    Lambda
    1. No server to
    manage
    2. Automated
    granular
    unlimited scaling
    2. Do no pay for
    idle time
    

    View Slide

18. Polyglot services, Lambda supports:
    AWS Lambda
    18
    Event-driven services, Lambda reacts to:
    Limitations: 300s max duration, 100 concurrent execution, 50MB max package size
    Oct 2015
    Nov 2014 Jun 2015
    CloudWatch S3 API
    Gateway
    SNS DynamoDB Kinesis
    Jun 2015
    Alarm
    responder
    Image
    processing
    Real time
    message
    handling
    DB triggers Real time
    streaming
    processing
    REST
    endpoints
    

    View Slide

19. HELLO WORLD
    Demo
    19
    

    View Slide

20. 20
    FROM MONOLITH TO
    MICROSERVICES
    Part 2
    

    View Slide

21. Quote
    21
    Source: http://martinfowler.com/bliki/MonolithFirst.html
    “Almost all the successful microservice stories
    have started with a monolith that got too big
    and was broken up”
    – Martin Fowler, Monolith first approach, 2015
    

    View Slide

22. Paul Graham’s Startup Curve
    22
    TROUGH OF SORROW
    J
    L
    Scale
    Initial enthusiasm
    Reality sets in
    Experimenting & pivoting
    Hockey stick
    Starts working
    Minimum Viable Product
    From search mode
    Move fast, break things
    Monolith
    Product Market Fit
    To execution mode
    Grow fast, scale things
    Microservices
    

    View Slide

23. 0. Layered monolith architecture
    23
    Domain
    Services
    Controllers
    Views
    Browser
    Simple to
    ✔ Develop
    ✔ Deploy (single war)
    ✔ Scale
    Issues (when the app grows)
    ✗ Large code base
    ✗ Heavy app
    ✗ No granular
    deployment
    ✗ No granular scaling
    ✗ Dev team size
    ✗ Single tech stack
    Perfect for
    MVP!
    Minimum Viable Product
    to validate your business
    hypothesis
    Lasagna-oriented
    architecture
    

    View Slide

24. 1. Back / Front boundary
    24
    Domain
    Services
    API
    Mobile
    client
    Web client
    Back / Front
    boundary
    Single page app
    (Angular, React, etc)
    Documented APIs
    

    View Slide

25. 2. Data boundaries
    25
    Domain
    Services
    API
    Mobile
    client
    Web client
    Domain
    SQL or NoSQL
    (depending on
    scaling/transaction
    requirements)
    Business data
    boundary
    

    View Slide

26. 3. Service boundaries
    26
    Services
    API
    Services
    API
    Domain
    Mobile
    client
    Web client
    Domain
    Business context
    boundary
    

    View Slide

27. 4. Microservices architecture
    27
    Services
    API
    Domain
    Services
    API
    Domain
    Services
    API
    Domain
    Web client
    Mobile
    client
    Drawbacks
    ✗ Service boundaries
    hard to defined
    ✗ Distributed system
    ✗ Deployment
    complexity (DevOps
    required)
    ✗ Operational
    overhead
    Benefits
    ✔ Small code bases
    ✔ Lightweight
    independent apps
    ✔ Granular scaling
    ✔ Independent cross-
    functional teams
    ✔ Independent tech
    stack
    Ravioli-driven
    architecture
    Perfect for
    growth!
    

    View Slide

28. Which microservices framework/lib?
    28
    ✗ Heavyweight
    ✗ Server required
    ✔ Fastest way to
    build an app
    ✔ Plugin ecosystem
    Minimum package size
    50MB to 60MB
    

    View Slide

29. Which microservices framework/lib?
    29
    ✔ Lightweight
    ✗ Server required
    ✗ Plumbing
    required
    ✔ Very fast/scalable
    Minimum package size
    11MB to 16MB
    

    View Slide

30. Minimum package size
    4KB or 4MB
    Which microservices framework/lib?
    30
    ✔ Lightweight
    ✔ Serverless
    ✗ Vendor lock-in
    ✗ Limitations
    (duration, size…)
    AWS
    Lambda
    (java) (groovy)
    

    View Slide

31. 31
    VOTE APP
    Demo
    

    View Slide

32. Demo app
    32
    http://localhost:8080
    Bare metal monolith
    

    View Slide

33. 1. API CONTROLLER + SERVICE
    33
    Microservice:
    QuizVoteAPIController
    Data:
    QuizVote
    

    View Slide

34. 2. REST API GATEWAY
    34
    Microservice:
    QuizVoteAPIController
    Data:
    QuizVote
    

    View Slide

35. 3. DB EVENT PROCESSOR
    35
    Microservice:
    QuizVoteDBEventProcessor
    Microservice:
    QuizVoteAPIController
    Data:
    QuizVote
    QuizInsight
    

    View Slide

36. 4. FINAL ARCHITECTURE
    36
    Microservice:
    QuizVoteDBEventProcessor
    Static resources:
    VoteApp
    (html, js, png, …)
    Microservice:
    QuizVoteAPIController
    Data:
    QuizVote
    QuizInsight
    CDN
    DNS
    

    View Slide

37. Demo app
    37
    http://starwars.benorama.com 100% Serverless microservices!
    

    View Slide

38. 38
    VICTORY!!!
    Bare metal servers defeated
    100% serverless microservices
    Responsive
    Resilient
    Elastic
    Message-driven
    100% managed by AWS
    

    View Slide

39. 39
    THANK YOU
    The end @benorama
    @benorama
    Demo source code: https://github.com/benorama/skillsmatter-demo
    

    View Slide