How LEGO.com accelerates innovation with serverless

AWS re:Invent 2020

Mostly presented by Sheen Brisals, AWS Serverless Hero, Senior Engineering Manager, The LEGO Group.

After experiencing scaling issues on Black Friday, the LEGO team fully refactored its monolith to serverless microservices on AWS. In this session, LEGO Senior Engineering Manager and AWS Serverless Hero Sheen Brisals shows you how LEGO.com rebuilt a monolith into an event-driven architecture with Amazon EventBridge and AWS Step Functions at its core. He also discusses how designing with a serverless-first mindset can provide surprising benefits. Come learn about successful architectural patterns and see how LEGO.com manages and grows its serverless team to deliver more in less time at the scale of millions of customers.

Transcript

1. © 2020, Amazon Web Services, Inc. or its affiliates. All rights reserved.
   How LEGO.com accelerates
   innovation with serverless
   Sheen Brisals
   AWS Serverless Hero
   Senior Engineering Manager
   The LEGO Group
   Danilo Poccia
   Chief Evangelist (EMEA)
   AWS
   S V S 3 0 2
   

   View Slide

2. © 2020, Amazon Web Services, Inc. or its affiliates. All rights reserved.
   Last year at re:Invent
   Accelerating with AWS Step Functions
   Accelerating with Amazon EventBridge
   Serverless team growth
   Business value acceleration
   Agenda
   

   View Slide

3. Last year at re:Invent
   © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
   The serverless journey of
   shop.LEGO.com
   S V S 3 2 0
   Sheen Brisals
   Senior Application Engineer
   The LEGO Group
   Danilo Poccia
   Principal Evangelist
   Amazon Web Services
   A journey through patterns
   Use case Pattern
   ✓ Add item to shopping basket Atomic request-response API
   ✓ Status polling for long-running processes CQRS with status cache
   ✓ Voucher codes generation and notification Email notification with signed URL
   ✓ User identity lookup in different systems API authorizer with identity lookup
   ✓ On-demand customer data migration Publish-subscribe sync
   ✓ Product catalog import and update Event-driven data pipeline with buffering
   ✓ API-driven data ingestion Codeless data ingestion
   ✓ Unique order number generation Codeless sequence generator
   ✓ Website migration with URL changes URL redirects cached by CDN
   ✓ Keeping website sitemaps updated Scheduled workflow
   ✓ Checkout event processing Hub-and-spoke event bus
   Pattern – Codeless data ingestion
   Kinesis
   Data Firehose
   API
   Gateway
   S3
   bucket
   Event
   producer
   • API to stream directly
   • Less point of failures
   • Native integration
   • Less compute costs
   • Less code to maintain
   • Fully managed and scalable
   Transformation
   and validation
   Backup
   Fan-out
   function
   Processing
   functions
   Trigger
   Async
   Errors
   Pattern – Hub-and-spoke event bus
   {
   "version": "0",
   "id": "6a7e8feb-b491-4cf7-a9f1-bf3703467718",
   "detail-type": "State change Notification",
   "source": "service-order-submit-dev",
   "account": "123456789012",
   "time": "2019-08-29T12:10:21Z",
   "region": "eu-central-1",
   "resources": ["arn:aws:events:event-bus/checkout-bus"],
   "detail": {
   "event": {
   "meta_data": {
   "site_id": "LEGO Shop",
   "type": "CHECKOUT",
   "subtype": "ORDER",
   "status": "COMPLETE"
   },
   "data": {
   "order_number": "T123456789",
   "customer_id": "bf3703467718-29T12-6a7e8feb"
   }
   }
   }
   }
   Standard syntax
   across multiple
   services
   Custom for each
   service
   

   View Slide

4. Accelerating
   with serverless
   

   View Slide

5. © 2020, Amazon Web Services, Inc. or its affiliates. All rights reserved.
   Accelerating with
   AWS Step Functions
   

   View Slide

6. Catalog feed generation in parallel
   U S I N G D Y N A M I C P A R A L L E L I S M
   Timer Process Product
   catalogs
   Publish
   Social
   media
   

   View Slide

7. Dynamic parallelism
   AWS Step
   Functions
   Trigger
   rule
   Amazon CloudWatch
   Events
   

   View Slide

8. Dynamic parallelism
   {
   "CreateFeedsForCountries": {
   "Type": "Map",
   "ItemsPath": "$.countryList",
   "MaxConcurrency": 5,
   "Iterator": {
   "StartAt": "CreateCountryFeed",
   "States": {
   "CreateCountryFeed": {
   "Type": "Task",
   "Resource": “feed-create",
   "End": true
   }
   }
   }
   }
   }
   

   View Slide

9. Failure message replay with state machine
   Recommendations
   engine
   Product
   feeds
   Failed
   product feeds
   Check
   status
   Process
   failed feeds
   

   View Slide

10. Failure
    event
    Failed feeds
    DLQ
    Failure message replay with state machine
    Product
    recommendation
    feed generator
    Product feed
    store
    Product feed
    dispatcher
    Recommendations
    API
    Alarm channel
    notifier
    Resumption
    event
    Failure
    event
    Failure
    messages
    Resumption
    event
    Reprocess failed feeds
    (AWS Step Functions)
    Service state
    Event bus
    (Amazon EventBridge)
    

    View Slide

11. In a way, this is implementing the
    circuit breaker pattern using AWS
    Step Functions
    Failure message replay with state machine
    

    View Slide

12. © 2020, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Accelerating with
    Amazon EventBridge
    

    View Slide

13. Back-in-stock notification
    Email store
    LEGO.com Notify customer Stock checker
    Customer
    

    View Slide

14. Back-in-stock notification
    Email
    registration
    Product
    status check
    Customer
    notification
    Email
    dispatch
    Email
    feedback
    Business
    insights
    Registration
    request
    Registration
    response
    Registration
    request
    Registration
    response
    Product
    status
    Product
    available
    Amazon
    SES
    Email
    feedback
    events
    Email
    feedback
    events
    Status events
    Amazon
    EventBridge
    

    View Slide

15. Choreographing back-in-stock notification
    A D V A N T A G E S O F C H O R E O G R A P H Y V S . O R C H E S T R A T I O N
    Registration
    request
    Registration
    API
    Registration
    request
    Product status
    check
    Product
    available
    Product
    available
    Notification
    orchestrator
    Fetch emails
    Email dispatch
    queue
    Email
    dispatcher
    Feedback
    events
    Feedback
    events
    Request
    handler
    Store
    registrations
    Amazon
    SES
    Email store Amazon
    EventBridge
    

    View Slide

16. Choreographing back-in-stock notification
    Registration
    request
    Registration
    API
    Email store
    Registration
    request
    Product status
    check
    Product
    available
    Product
    available
    Notification
    orchestrator
    Fetch emails
    Email dispatch
    queue
    Email
    dispatcher
    Feedback
    events
    Feedback
    events
    Amazon
    SES
    Amazon
    EventBridge
    

    View Slide

17. Orchestrating the notification process
    Product
    available
    

    View Slide

18. Email feedback event stream with batching
    Amazon
    SES
    Amazon S3
    bucket
    Amazon Kinesis
    Data Firehose
    AWS Lambda
    function
    Feedback
    event
    Buffering
    events
    Storing
    events
    Grouping
    events
    Amazon EventBridge
    event bus
    Batched
    feedback
    event
    Success
    events
    Error
    events
    Reject, Bounce, Complaint,
    Send, Open, Click
    Event with an
    array of custom
    data
    Business
    insights
    Error
    feedback
    handling
    A single EventBridge event contains multiple feedback events
    

    View Slide

19. Email feedback event stream with batching
    Amazon
    SES
    S3 bucket
    Kinesis
    Data Firehose
    Lambda
    function
    Buffering
    events
    Storing
    events
    Business
    insights
    Error
    feedback
    handling
    Success
    events
    Error
    events
    Efficient data ingestion pattern
    Patterns help to accelerate your serverless adoption
    Batched
    feedback
    event
    Feedback
    event
    Grouping
    events
    EventBridge
    event bus
    

    View Slide

20. https://bit.ly/2FZ1b1a
    

    View Slide

21. Order placement flow
    Checkout
    Submit
    order
    Payment
    Authorize
    payment
    Order
    Process
    order
    Order
    submitted
    Order
    submitted
    Order
    canceled
    Payment
    authorized
    Payment
    canceled
    Payment
    authorized
    Order
    confirmed
    Order
    canceled
    Order
    confirmed
    Shipping
    Orders
    to SAP
    Amazon
    EventBridge
    

    View Slide

22. Orchestration inside order processing
    

    View Slide

23. • Use choreography to coordinate different microservices
    § For example, with Amazon EventBridge
    • Use orchestration inside microservices, or across different
    microservices if you need a distributed transaction
    § For example, using AWS Step Functions
    Choreography vs. orchestration
    

    View Slide

24. © 2020, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Serverless team growth
    

    View Slide

25. Start of our serverless journey
    Serverless team Front-end team
    • Small focus team
    • Build skills and expertise
    • Expand incrementally
    

    View Slide

26. Feature-driven and full-stack squads
    payments
    rewards platform
    content
    insights
    new
    products checkout
    • Feature squads
    • Full stack
    • Distributed skills
    

    View Slide

27. • Feature squads with growing serverless knowledge
    • Engineers become architecture-aware
    • Engineers are part of the DevOps process
    • Adapting best practices, including Serverless Lens
    • Service ownership – we build, we run!
    • Sharing knowledge with the community
    

    View Slide

28. © 2020, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Business value acceleration
    

    View Slide

29. Scalability gives business value too
    

    View Slide

30. • Faster feature development
    • Stakeholder trust in delivering value
    • Increased operational visibility
    • Scalable, serverless platform that serves more customers
    Business value acceleration
    

    View Slide

31. Thank you!
    © 2020, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Sheen Brisals
    Twitter @sheenbrisals
    Danilo Poccia
    Twitter @danilop
    

    View Slide