Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Build event-driven architectures effortlessly with AWS

F71c764a1edb53e705519a0aa3b66395?s=47 Lalit Kale
September 25, 2021

Build event-driven architectures effortlessly with AWS

Build event-driven architectures effortlessly with AWS
Session Abstract:
Many customers are choosing to build event-driven application architectures – those in which subscriber or target services automatically perform work in response to events triggered by publisher or source services. Event driven architectures enable development teams to operate more independently so they can release new features faster, while also making their applications more scalable. In this session, we will discuss how we can build event driven architectures with AWS Lambda and AWS Eventbridge along with widely used patterns.

About Speaker:
Lalit Kale is a Principal Engineer at Verizon Connect. Lalit has 16 years of experience in the software industry. His interests include large scale distributed systems and scalability of platforms.

He is a published author of a book on Microservices ("Building Microservices with .NET Core") published by Packt. He has been recognized as an AWS Community Builder by AWS for his community contributions. You can follow him on LinkedIn (https://www.linkedin.com/in/lalitkale/)

Recorded Session: https://www.youtube.com/watch?v=tDwZLiENDSM

F71c764a1edb53e705519a0aa3b66395?s=128

Lalit Kale

September 25, 2021
Tweet

Other Decks in Programming

Transcript

  1. Build Event-Driven Architectures with AWS Lalit Kale Principal Software Engineer

    - Verizon Connect Effortlessly
  2. Lalit Kale • Principal Software Engineer, Verizon Connect • Dublin,

    Ireland • First AWS Use: 2006 https://www.linkedin.com/in/lalitkale/ @atechiethought
  3. Agenda • What is Event Driven Architectures • AWS Serverless

    Application Integration Services ◦ AWS Lambda ◦ Selecting Right Messaging service: SQS, SNS and Eventridge ◦ Application Integration Patterns support • Amazon EventBridge • Use Cases
  4. Current Landscape Large Systems (couple of years ago) • Few

    Kb of internet speed • Couple of giant single core servers • Seconds of response time • Hours of offline maintenance • Data size in Gigabytes Large Systems (Today & Future) • GBs scale internet speed • Thousands of multicore servers • Milliseconds of response time • Users expect 100% uptime • Data size in Petabytes and Zetabytes
  5. Reactive Systems

  6. The big idea is "messaging" - Alan Kay src: https://wiki.c2.com/?AlanKayOnMessaging

  7. "If your application is cloud-native or, large scale or, distributed,

    and doesn't include a messaging component, that is probably a bug." - Tim Bray Former Sr. Principal Engineer, AWS src: https://wiki.c2.com/?AlanKayOnMessaging
  8. Communication Synchronous Communication Asynchronous Communication Face To Face Video Conferencing

    Email
  9. Types of Communication Synchronous Communication Asynchronous Communication

  10. Types of Messages Commands • "I Intent to …" •

    Can be rejected by receiver • Represented as verbs in present tense, e.g. "create_customer" • Source system is aware of side effects that command processing will have Events • Something has happened • Immutable—cannot change the past • Represented as verbs in the past tense, e.g. “customer_created” • Source system has no expectations on how an event is processed
  11. Event Driven Architecture Event driven architecture is a distributed asynchronous

    integration pattern to create highly scalable and loosely coupled reactive applications Events drives the data processing
  12. Advantages of Event driven architecture • Loose coupling between components/services

    • Ability to scale individual components • Processing components can be developed independent of each other • Fault Tolerance and better resiliency • Improved Availability • Evolutionary architecture • Business Agility
  13. Challenges with Event Driven Architectures • Consistency Model mindshift -

    Eventual Consistency • Observability of Event Pipelines • Immutable Consumers • Error Handling • Event Replay Scenarios
  14. Challenges with Event Driven Architectures •

  15. Event Driven Architecture Event driven architecture is a distributed asynchronous

    integration pattern to create highly scalable and loosely coupled reactive applications
  16. Enterprise Integration Patterns • Point to Point channel • Pub-Sub

    pattern • Message Router • Event Bus
  17. Event Driven Architectures with AWS Serverless

  18. Serverless Story is incomplete without...

  19. AWS Lambda Execution Models

  20. AWS Lambda Destinations

  21. AWS Serverless Application Integration Services

  22. Service Selection Criteria Scale & Concurrency Controls Durability Persistence Consumption

    Retries Pricing
  23. AWS Serverless Application Integration Services Amazon SQS Durable and scalable

    Buffer and Batching support Comprehensive security point to point channel Amazon SNS Performance at Scale Enterprise Ready, Deliver to endpoints like email, SMS, Https, Push Notifications Fan-Out, Pub-Sub Amazon EventBridge Serverless event bus for AWS Services, your own apps and SaaS providers Event Bus
  24. Failures and Retries Amazon SQS Amazon SNS Amazon EventBridge Delete

    Message. Redrive policy for failed messages. Retry policy for different protocols Event Retention and Replay Retry upto max. 14 days Retry upto 23 days Retry upto 24 hours Dead Letter Queue Dead Letter Queue Dead Letter Queue Exactly once with FIFO Exactly Once with FIFO Topic
  25. Amazon EventBridge • Serverless, Pay as you go • 90+

    AWS Services as sources • 17+ AWS services as targets • Native integrations with SaaS Providers • $1 per million events put into the bus • No additional cost of delivery • Cross Account, Cross Region
  26. Amazon EventBridge

  27. Amazon EventBridge - Events and Rules { "version": "0", "id":

    "3e3c153a-8339-4e30-8c35-687ebef853fe", "detail-type": "EC2 Instance Launch Successful", "source": "aws.autoscaling", "account": "123456789012", "time": "2015-11-11T21:31:47Z", "region": "us-east-1", "resources": [], "detail": { "eventVersion": "", "responseElements": null } } Example Event Example Rule { "source": ["aws.autoscaling"] }
  28. • Source of truth for sharing schema • Explicitly published

    and auto-discovered • Integrations for IDEs like Jetbrains and VS Code • Language Bindings available for Java, Python and Typescript Amazon EventBridge - Schema Registry and Discovery
  29. Amazon EventBridge - Event Archive and Replay

  30. When Amazon EventBridge is not best fit? • Ordering Guarantees

    • Event Deduplication • Push Delivery • Ultra Low latency • Non-JSON Transport
  31. Event Modelling CloudEvents - A specification for describing event data

    in a common way • Consistent schema across all your events • Many Language SDK available - Java, GoLang, Python, Rust etc • Portable Across AWS, GCP, Azure • CNCF Serverless Group supported • https://cloudevents.io/
  32. Event Driven Architectures - Infrastructure as Code Support • https://serverlessland.com/

    • https://cdkpatterns.com • Use patterns to quickly build integrations using AWS SAM and CDK templates. • 70+ Templates • You can contribute to it!
  33. EventBridge - Real Time Threat Mitigation

  34. Loan Broker Use case src: https://www.enterpriseintegrationpatterns.com/patterns/messaging/ComposedMessagingExample.html

  35. • Receive the consumer's loan quote request • Obtain credit

    score and history from credit agency • Determine the most appropriate banks to contact • Send a request to each selected bank • Collect responses from each selected bank • Determine the best response • Pass the result back to the consumer Loan Broker Use case - Processing Flow src: https://www.enterpriseintegrationpatterns.com/patterns/messaging/ComposedMessagingExample.html
  36. Loan Broker Use case src: https://www.enterpriseintegrationpatterns.com/patterns/messaging/ComposedMessagingExample.html

  37. Summary • event driven architectures are essential complexity for large

    scale decoupled applications • Awareness of enterprise integration patterns will ease out the journey through event driven architectures • For choosing the aws service for your scenarios, compare scale, durability, persistence and consumption models, retries and pricing. • Adopting AWS serverless services like EventBridge ease out complex implementations like content filtering and event replays and cross account, cross region scenarios. • You can compose solution by combining multiple patterns and services • Utilize IaC templates available in open source or build your own.
  38. Further Reading • http://www.sosp.org/2001/papers/welsh.pdf • https://www.enterpriseintegrationpatterns.com/ • https://aws.amazon.com/blogs/compute/introducing-cross-region-event-routing-with-amazon-eventbridge/ • https://aws.amazon.com/blogs/compute/building-well-architected-serverless-applications-introduction/

    • https://serverlessland.com/patterns
  39. Q & A

  40. Thank You!

  41. Extras

  42. Amazon EventBridge - Pricing AWS Service Events Free Custom Events

    $1/ million events published Third Party Events $1/ million events published Cross Account Events $1/ million cross-account events sent Schema Discovery $0.10 / million events ingested Archive Processing $0.12 per GB Storage Month $0.0025 per GB
  43. SQS, SNS, EventBridge Amazon SQS Amazon SNS Amazon EventBridge Messaging

    1 to 1 1 to Many 1 to Many Event Type Single Event Type Single Event Type Multiple Event Type Route Messages Single Single Multiple Rules Filter Messages No Only to Message Body Full Message Allow transformation? No No Yes Cross Account Yes Yes Yes Message Latency Low Low Moderate Throughput High High Moderate