Upgrade to Pro — share decks privately, control downloads, hide ads and more …

How LEGO.com accelerates innovation with serverless

How LEGO.com accelerates innovation with serverless

AWS re:Invent 2020

Mostly presented by Sheen Brisals, AWS Serverless Hero, Senior Engineering Manager, The LEGO Group.

After experiencing scaling issues on Black Friday, the LEGO team fully refactored its monolith to serverless microservices on AWS. In this session, LEGO Senior Engineering Manager and AWS Serverless Hero Sheen Brisals shows you how LEGO.com rebuilt a monolith into an event-driven architecture with Amazon EventBridge and AWS Step Functions at its core. He also discusses how designing with a serverless-first mindset can provide surprising benefits. Come learn about successful architectural patterns and see how LEGO.com manages and grows its serverless team to deliver more in less time at the scale of millions of customers.

Danilo Poccia

February 08, 2021
Tweet

More Decks by Danilo Poccia

Other Decks in Programming

Transcript

  1. © 2020, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    How LEGO.com accelerates
    innovation with serverless
    Sheen Brisals
    AWS Serverless Hero
    Senior Engineering Manager
    The LEGO Group
    Danilo Poccia
    Chief Evangelist (EMEA)
    AWS
    S V S 3 0 2

    View Slide

  2. © 2020, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Last year at re:Invent
    Accelerating with AWS Step Functions
    Accelerating with Amazon EventBridge
    Serverless team growth
    Business value acceleration
    Agenda

    View Slide

  3. Last year at re:Invent
    © 2019, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    The serverless journey of
    shop.LEGO.com
    S V S 3 2 0
    Sheen Brisals
    Senior Application Engineer
    The LEGO Group
    Danilo Poccia
    Principal Evangelist
    Amazon Web Services
    A journey through patterns
    Use case Pattern
    ✓ Add item to shopping basket Atomic request-response API
    ✓ Status polling for long-running processes CQRS with status cache
    ✓ Voucher codes generation and notification Email notification with signed URL
    ✓ User identity lookup in different systems API authorizer with identity lookup
    ✓ On-demand customer data migration Publish-subscribe sync
    ✓ Product catalog import and update Event-driven data pipeline with buffering
    ✓ API-driven data ingestion Codeless data ingestion
    ✓ Unique order number generation Codeless sequence generator
    ✓ Website migration with URL changes URL redirects cached by CDN
    ✓ Keeping website sitemaps updated Scheduled workflow
    ✓ Checkout event processing Hub-and-spoke event bus
    Pattern – Codeless data ingestion
    Kinesis
    Data Firehose
    API
    Gateway
    S3
    bucket
    Event
    producer
    • API to stream directly
    • Less point of failures
    • Native integration
    • Less compute costs
    • Less code to maintain
    • Fully managed and scalable
    Transformation
    and validation
    Backup
    Fan-out
    function
    Processing
    functions
    Trigger
    Async
    Errors
    Pattern – Hub-and-spoke event bus
    {
    "version": "0",
    "id": "6a7e8feb-b491-4cf7-a9f1-bf3703467718",
    "detail-type": "State change Notification",
    "source": "service-order-submit-dev",
    "account": "123456789012",
    "time": "2019-08-29T12:10:21Z",
    "region": "eu-central-1",
    "resources": ["arn:aws:events:event-bus/checkout-bus"],
    "detail": {
    "event": {
    "meta_data": {
    "site_id": "LEGO Shop",
    "type": "CHECKOUT",
    "subtype": "ORDER",
    "status": "COMPLETE"
    },
    "data": {
    "order_number": "T123456789",
    "customer_id": "bf3703467718-29T12-6a7e8feb"
    }
    }
    }
    }
    Standard syntax
    across multiple
    services
    Custom for each
    service

    View Slide

  4. Accelerating
    with serverless

    View Slide

  5. © 2020, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Accelerating with
    AWS Step Functions

    View Slide

  6. Catalog feed generation in parallel
    U S I N G D Y N A M I C P A R A L L E L I S M
    Timer Process Product
    catalogs
    Publish
    Social
    media

    View Slide

  7. Dynamic parallelism
    AWS Step
    Functions
    Trigger
    rule
    Amazon CloudWatch
    Events

    View Slide

  8. Dynamic parallelism
    {
    "CreateFeedsForCountries": {
    "Type": "Map",
    "ItemsPath": "$.countryList",
    "MaxConcurrency": 5,
    "Iterator": {
    "StartAt": "CreateCountryFeed",
    "States": {
    "CreateCountryFeed": {
    "Type": "Task",
    "Resource": “feed-create",
    "End": true
    }
    }
    }
    }
    }

    View Slide

  9. Failure message replay with state machine
    Recommendations
    engine
    Product
    feeds
    Failed
    product feeds
    Check
    status
    Process
    failed feeds

    View Slide

  10. Failure
    event
    Failed feeds
    DLQ
    Failure message replay with state machine
    Product
    recommendation
    feed generator
    Product feed
    store
    Product feed
    dispatcher
    Recommendations
    API
    Alarm channel
    notifier
    Resumption
    event
    Failure
    event
    Failure
    messages
    Resumption
    event
    Reprocess failed feeds
    (AWS Step Functions)
    Service state
    Event bus
    (Amazon EventBridge)

    View Slide

  11. In a way, this is implementing the
    circuit breaker pattern using AWS
    Step Functions
    Failure message replay with state machine

    View Slide

  12. © 2020, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Accelerating with
    Amazon EventBridge

    View Slide

  13. Back-in-stock notification
    Email store
    LEGO.com Notify customer Stock checker
    Customer

    View Slide

  14. Back-in-stock notification
    Email
    registration
    Product
    status check
    Customer
    notification
    Email
    dispatch
    Email
    feedback
    Business
    insights
    Registration
    request
    Registration
    response
    Registration
    request
    Registration
    response
    Product
    status
    Product
    available
    Amazon
    SES
    Email
    feedback
    events
    Email
    feedback
    events
    Status events
    Amazon
    EventBridge

    View Slide

  15. Choreographing back-in-stock notification
    A D V A N T A G E S O F C H O R E O G R A P H Y V S . O R C H E S T R A T I O N
    Registration
    request
    Registration
    API
    Registration
    request
    Product status
    check
    Product
    available
    Product
    available
    Notification
    orchestrator
    Fetch emails
    Email dispatch
    queue
    Email
    dispatcher
    Feedback
    events
    Feedback
    events
    Request
    handler
    Store
    registrations
    Amazon
    SES
    Email store Amazon
    EventBridge

    View Slide

  16. Choreographing back-in-stock notification
    Registration
    request
    Registration
    API
    Email store
    Registration
    request
    Product status
    check
    Product
    available
    Product
    available
    Notification
    orchestrator
    Fetch emails
    Email dispatch
    queue
    Email
    dispatcher
    Feedback
    events
    Feedback
    events
    Amazon
    SES
    Amazon
    EventBridge

    View Slide

  17. Orchestrating the notification process
    Product
    available

    View Slide

  18. Email feedback event stream with batching
    Amazon
    SES
    Amazon S3
    bucket
    Amazon Kinesis
    Data Firehose
    AWS Lambda
    function
    Feedback
    event
    Buffering
    events
    Storing
    events
    Grouping
    events
    Amazon EventBridge
    event bus
    Batched
    feedback
    event
    Success
    events
    Error
    events
    Reject, Bounce, Complaint,
    Send, Open, Click
    Event with an
    array of custom
    data
    Business
    insights
    Error
    feedback
    handling
    A single EventBridge event contains multiple feedback events

    View Slide

  19. Email feedback event stream with batching
    Amazon
    SES
    S3 bucket
    Kinesis
    Data Firehose
    Lambda
    function
    Buffering
    events
    Storing
    events
    Business
    insights
    Error
    feedback
    handling
    Success
    events
    Error
    events
    Efficient data ingestion pattern
    Patterns help to accelerate your serverless adoption
    Batched
    feedback
    event
    Feedback
    event
    Grouping
    events
    EventBridge
    event bus

    View Slide

  20. https://bit.ly/2FZ1b1a

    View Slide

  21. Order placement flow
    Checkout
    Submit
    order
    Payment
    Authorize
    payment
    Order
    Process
    order
    Order
    submitted
    Order
    submitted
    Order
    canceled
    Payment
    authorized
    Payment
    canceled
    Payment
    authorized
    Order
    confirmed
    Order
    canceled
    Order
    confirmed
    Shipping
    Orders
    to SAP
    Amazon
    EventBridge

    View Slide

  22. Orchestration inside order processing

    View Slide

  23. • Use choreography to coordinate different microservices
    § For example, with Amazon EventBridge
    • Use orchestration inside microservices, or across different
    microservices if you need a distributed transaction
    § For example, using AWS Step Functions
    Choreography vs. orchestration

    View Slide

  24. © 2020, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Serverless team growth

    View Slide

  25. Start of our serverless journey
    Serverless team Front-end team
    • Small focus team
    • Build skills and expertise
    • Expand incrementally

    View Slide

  26. Feature-driven and full-stack squads
    payments
    rewards platform
    content
    insights
    new
    products checkout
    • Feature squads
    • Full stack
    • Distributed skills

    View Slide

  27. • Feature squads with growing serverless knowledge
    • Engineers become architecture-aware
    • Engineers are part of the DevOps process
    • Adapting best practices, including Serverless Lens
    • Service ownership – we build, we run!
    • Sharing knowledge with the community

    View Slide

  28. © 2020, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Business value acceleration

    View Slide

  29. Scalability gives business value too

    View Slide

  30. • Faster feature development
    • Stakeholder trust in delivering value
    • Increased operational visibility
    • Scalable, serverless platform that serves more customers
    Business value acceleration

    View Slide

  31. Thank you!
    © 2020, Amazon Web Services, Inc. or its affiliates. All rights reserved.
    Sheen Brisals
    Twitter @sheenbrisals
    Danilo Poccia
    Twitter @danilop

    View Slide