4 Patterns to Jumpstart your Event-Driven Architecture Journey Hans-Peter Grahsl Developer 🥑 Advocate 1

Who am I ? 2 Developer 🥑 Advocate @ Red Hat based in Graz, Austria 󰎈 open-source enthusiast 5x Confluent Community Catalyst MongoDB Champion since 2020 speaker at dev & tech conferences @[email protected] Hans-Peter Grahsl “I’m passionate about event-driven architectures, distributed stream processing systems and data engineering.”

Agenda 3 What we’ll discuss today ▸ Claim Check Pattern ▸ Content Enricher Pattern ▸ Message Translator Pattern ▸ Outbox Pattern

EDA Terminology 4 … is a way of designing loosely coupled services that respond (in near real-time) based on the sending and receiving of events. 🤔 Event-Driven Architecture (EDA) ?

EDA Terminology 5 Multiple flavours https://martinfowler.com/articles/201701-event-driven.html 1. Event Notification 2. Event-Carried State Transfer 3. Event Sourcing 4. CQRS (selectively combined with the above)

EDA Terminology 6 This talk’s meaning of Event-Driven Architecture https://martinfowler.com/articles/201701-event-driven.html 1. Event Notification 2. Event-Carried State Transfer 3. Event Sourcing 4. CQRS (selectively combined with the above) this talk’s meaning

It’s all about events … 7 EVENTS EVENTS EVERYWHERE

8 One seeks bullets of silver … … but only finds trade-offs.

Agenda 9 Let’s explore the patterns… ▸ Claim Check Pattern ▸ Content Enricher Pattern ▸ Message Translator Pattern ▸ Outbox Pattern

Claim Check Pattern 10 ▸ inherently binary data graphics, documents, audio, video, … ▸ embedded data (e.g. Base64 encoded) in otherwise textual formats (json, xml, …) unnaturally large / unreasonably heavy event payloads Challenge

Claim Check Pattern 11 ▸ offload large (binary) payload into separate storage system ▸ replace actual payload with reference to claim check ▸ use claim check to fetch data when needed at consumer side externalize large event payloads … or parts thereof Claim Check Pattern

Claim Check Pattern 12

Claim Check Pattern 13

Claim Check Pattern 14

Claim Check Pattern 15

Claim Check Pattern 16

Claim Check Pattern 17

18 Claim Check Pattern

Agenda 19 Let’s explore the patterns… ▸ Claim Check Pattern ▸ Content Enricher Pattern ▸ Message Translator Pattern ▸ Outbox Pattern

Content Enricher Pattern 20 ▸ payloads are lacking additional details ▸ certain consumers need to know more ▸ example: very compact notification events unnaturally small / unreasonably light event payloads Challenge

Content Enricher Pattern 21 ▸ add missing data e.g. include contextual information ▸ do lookups / perform joins against reference data ▸ take reference data to augment original payload increase the utility of sparse event payloads Content Enricher Pattern

Content Enricher Pattern 22

Content Enricher Pattern 23

Content Enricher Pattern 24

Content Enricher Pattern 25

Content Enricher Pattern 26

27 Content Enricher Pattern

Agenda 28 Let’s explore the patterns… ▸ Claim Check Pattern ▸ Content Enricher Pattern ▸ Message Translator Pattern ▸ Outbox Pattern

Message Translator Pattern 29 ▸ examples: ･ integrate legacy systems ･ work with proprietary data formats ▸ minimize or at least reduce the contract-related coupling incompatible or inconvenient event payloads Challenge

Message Translator Pattern 30 ▸ perform a wide range of payload modifications: ･ basic field type conversions ･ rename, redact, or mask fields ･ switch format / protocol ･ … ▸ can also be used to build an anti-corruption layer establish compatibility between disparate systems Message Translator Pattern

Message Translator Pattern 31

Message Translator Pattern 32

Message Translator Pattern 33

Message Translator Pattern 34

Message Translator Pattern 35

Message Translator Pattern 36

37 Message Translator Pattern

Agenda 38 Let’s explore the patterns… ▸ Claim Check Pattern ▸ Content Enricher Pattern ▸ Message Translator Pattern ▸ Outbox Pattern

Outbox Pattern 39 ▸ XA transactions ? potentially unreasonable choice or sometimes not available at all ▸ example: (micro)service updates its own database and sends event(s) to messaging/event streaming infra consistently update data in 2 different systems Challenge

Outbox Pattern 40 ▸ perform single database transaction spanning: ･ update(s) to affected DB tables (“domain data”) ･ update(s) to outbox DB table (“event data”) ▸ setup separate CDC pipeline for outbox table ･ transfer event data to messaging/event streaming infra ▸ consume CDC stream on-demand in near real-time avoid the dual-writes issue Outbox Pattern

Outbox Pattern 41

Outbox Pattern 42

Outbox Pattern 43

Outbox Pattern 44

Outbox Pattern 45

46 Outbox Pattern

▸ no silver bullets for event design / communication → only trade-offs ▸ patterns help to deal with implications and cope with trade-offs Wrap Up 47 Summary of discussed EDA patterns Claim Check Pattern Content Enricher Pattern Message Translator Pattern Outbox Pattern

Demo Repository 48 bit.ly/eda-patterns

Upcoming Talk 49 From edge to cloud, creating data pipelines using open-source with Strimzi on Kubernetes. Date: Wednesday, September 27 Time: 4:30 pm Location: Breakout Room 7 Speakers: Chris Cranford, Carles Arnal There will be a raffle with gifts at the end of the session!

linkedin.com/showcase/red-hat-developer youtube.com/RedHatDevelopers facebook.com/RedHatDeveloper twitter.com/rhdevelopers 50 Red Hat is the world’s leading provider of enterprise open source software solutions. Award-winning support, training, and consulting services make Red Hat a trusted adviser to the Fortune 500. Thank you ! Questions ?

51 https://unsplash.com/photos/DRzYMtae-vA https://pixabay.com/illustrations/night-sky-stars-moon-wolf-garou-4726445/ https://unsplash.com/photos/HTIduwcMMfQ https://unsplash.com/photos/w9rYOScXf6E https://unsplash.com/photos/iar-afB0QQw https://unsplash.com/photos/q65bNe9fW-w https://pixabay.com/photos/video-production-video-movie-film-4223885/ https://pixabay.com/photos/feather-bird-plumage-flying-4380861/ https://unsplash.com/photos/MceA9kSze0U https://pixabay.com/photos/travel-adapter-plug-charging-plug-3320763/ https://unsplash.com/photos/g97bviyFhvQ https://pixabay.com/illustrations/post-office-mailbox-old-yellow-2438804/ https://pixabay.com/photos/post-office-letter-box-mail-box-2455580/ https://pixabay.com/photos/ecommerce-shopping-credit-card-2607114/ in order of appearance Photo Credits