Upgrade to Pro — share decks privately, control downloads, hide ads and more …

How LEGO.com accelerates innovation with serverless

How LEGO.com accelerates innovation with serverless

AWS re:Invent 2020

Mostly presented by Sheen Brisals, AWS Serverless Hero, Senior Engineering Manager, The LEGO Group.

After experiencing scaling issues on Black Friday, the LEGO team fully refactored its monolith to serverless microservices on AWS. In this session, LEGO Senior Engineering Manager and AWS Serverless Hero Sheen Brisals shows you how LEGO.com rebuilt a monolith into an event-driven architecture with Amazon EventBridge and AWS Step Functions at its core. He also discusses how designing with a serverless-first mindset can provide surprising benefits. Come learn about successful architectural patterns and see how LEGO.com manages and grows its serverless team to deliver more in less time at the scale of millions of customers.

7c9b8b368924556d8642bdaed3ded1f5?s=128

Danilo Poccia

February 08, 2021
Tweet

Transcript

  1. © 2020, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. How LEGO.com accelerates innovation with serverless Sheen Brisals AWS Serverless Hero Senior Engineering Manager The LEGO Group Danilo Poccia Chief Evangelist (EMEA) AWS S V S 3 0 2
  2. © 2020, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Last year at re:Invent Accelerating with AWS Step Functions Accelerating with Amazon EventBridge Serverless team growth Business value acceleration Agenda
  3. Last year at re:Invent © 2019, Amazon Web Services, Inc.

    or its affiliates. All rights reserved. The serverless journey of shop.LEGO.com S V S 3 2 0 Sheen Brisals Senior Application Engineer The LEGO Group Danilo Poccia Principal Evangelist Amazon Web Services A journey through patterns Use case Pattern ✓ Add item to shopping basket Atomic request-response API ✓ Status polling for long-running processes CQRS with status cache ✓ Voucher codes generation and notification Email notification with signed URL ✓ User identity lookup in different systems API authorizer with identity lookup ✓ On-demand customer data migration Publish-subscribe sync ✓ Product catalog import and update Event-driven data pipeline with buffering ✓ API-driven data ingestion Codeless data ingestion ✓ Unique order number generation Codeless sequence generator ✓ Website migration with URL changes URL redirects cached by CDN ✓ Keeping website sitemaps updated Scheduled workflow ✓ Checkout event processing Hub-and-spoke event bus Pattern – Codeless data ingestion Kinesis Data Firehose API Gateway S3 bucket Event producer • API to stream directly • Less point of failures • Native integration • Less compute costs • Less code to maintain • Fully managed and scalable Transformation and validation Backup Fan-out function Processing functions Trigger Async Errors Pattern – Hub-and-spoke event bus { "version": "0", "id": "6a7e8feb-b491-4cf7-a9f1-bf3703467718", "detail-type": "State change Notification", "source": "service-order-submit-dev", "account": "123456789012", "time": "2019-08-29T12:10:21Z", "region": "eu-central-1", "resources": ["arn:aws:events:event-bus/checkout-bus"], "detail": { "event": { "meta_data": { "site_id": "LEGO Shop", "type": "CHECKOUT", "subtype": "ORDER", "status": "COMPLETE" }, "data": { "order_number": "T123456789", "customer_id": "bf3703467718-29T12-6a7e8feb" } } } } Standard syntax across multiple services Custom for each service
  4. Accelerating with serverless

  5. © 2020, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Accelerating with AWS Step Functions
  6. Catalog feed generation in parallel U S I N G

    D Y N A M I C P A R A L L E L I S M Timer Process Product catalogs Publish Social media
  7. Dynamic parallelism AWS Step Functions Trigger rule Amazon CloudWatch Events

  8. Dynamic parallelism { "CreateFeedsForCountries": { "Type": "Map", "ItemsPath": "$.countryList", "MaxConcurrency":

    5, "Iterator": { "StartAt": "CreateCountryFeed", "States": { "CreateCountryFeed": { "Type": "Task", "Resource": “feed-create", "End": true } } } } }
  9. Failure message replay with state machine Recommendations engine Product feeds

    Failed product feeds Check status Process failed feeds
  10. Failure event Failed feeds DLQ Failure message replay with state

    machine Product recommendation feed generator Product feed store Product feed dispatcher Recommendations API Alarm channel notifier Resumption event Failure event Failure messages Resumption event Reprocess failed feeds (AWS Step Functions) Service state Event bus (Amazon EventBridge)
  11. In a way, this is implementing the circuit breaker pattern

    using AWS Step Functions Failure message replay with state machine
  12. © 2020, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Accelerating with Amazon EventBridge
  13. Back-in-stock notification Email store LEGO.com Notify customer Stock checker Customer

  14. Back-in-stock notification Email registration Product status check Customer notification Email

    dispatch Email feedback Business insights Registration request Registration response Registration request Registration response Product status Product available Amazon SES Email feedback events Email feedback events Status events Amazon EventBridge
  15. Choreographing back-in-stock notification A D V A N T A

    G E S O F C H O R E O G R A P H Y V S . O R C H E S T R A T I O N Registration request Registration API Registration request Product status check Product available Product available Notification orchestrator Fetch emails Email dispatch queue Email dispatcher Feedback events Feedback events Request handler Store registrations Amazon SES Email store Amazon EventBridge
  16. Choreographing back-in-stock notification Registration request Registration API Email store Registration

    request Product status check Product available Product available Notification orchestrator Fetch emails Email dispatch queue Email dispatcher Feedback events Feedback events Amazon SES Amazon EventBridge
  17. Orchestrating the notification process Product available

  18. Email feedback event stream with batching Amazon SES Amazon S3

    bucket Amazon Kinesis Data Firehose AWS Lambda function Feedback event Buffering events Storing events Grouping events Amazon EventBridge event bus Batched feedback event Success events Error events Reject, Bounce, Complaint, Send, Open, Click Event with an array of custom data Business insights Error feedback handling A single EventBridge event contains multiple feedback events
  19. Email feedback event stream with batching Amazon SES S3 bucket

    Kinesis Data Firehose Lambda function Buffering events Storing events Business insights Error feedback handling Success events Error events Efficient data ingestion pattern Patterns help to accelerate your serverless adoption Batched feedback event Feedback event Grouping events EventBridge event bus
  20. https://bit.ly/2FZ1b1a

  21. Order placement flow Checkout Submit order Payment Authorize payment Order

    Process order Order submitted Order submitted Order canceled Payment authorized Payment canceled Payment authorized Order confirmed Order canceled Order confirmed Shipping Orders to SAP Amazon EventBridge
  22. Orchestration inside order processing

  23. • Use choreography to coordinate different microservices § For example,

    with Amazon EventBridge • Use orchestration inside microservices, or across different microservices if you need a distributed transaction § For example, using AWS Step Functions Choreography vs. orchestration
  24. © 2020, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Serverless team growth
  25. Start of our serverless journey Serverless team Front-end team •

    Small focus team • Build skills and expertise • Expand incrementally
  26. Feature-driven and full-stack squads payments rewards platform content insights new

    products checkout • Feature squads • Full stack • Distributed skills
  27. • Feature squads with growing serverless knowledge • Engineers become

    architecture-aware • Engineers are part of the DevOps process • Adapting best practices, including Serverless Lens • Service ownership – we build, we run! • Sharing knowledge with the community
  28. © 2020, Amazon Web Services, Inc. or its affiliates. All

    rights reserved. Business value acceleration
  29. Scalability gives business value too

  30. • Faster feature development • Stakeholder trust in delivering value

    • Increased operational visibility • Scalable, serverless platform that serves more customers Business value acceleration
  31. Thank you! © 2020, Amazon Web Services, Inc. or its

    affiliates. All rights reserved. Sheen Brisals Twitter @sheenbrisals Danilo Poccia Twitter @danilop