Slide 1

Slide 1 text

Image © Nathalie https://flic.kr/p/21Ghf2g (CC BY 2.0) Keep Your Cache Always Fresh With Debezium! Gunnar Morling Software Engineer, Red Hat @gunnarmorling

Slide 2

Slide 2 text

#Debezium @gunnarmorling The Challenge Multi-site Application With Shared Database

Slide 3

Slide 3 text

#Debezium @gunnarmorling The Challenge Multi-site Application With Shared Database

Slide 4

Slide 4 text

#Debezium @gunnarmorling The Challenge Multi-site Application With Shared Database

Slide 5

Slide 5 text

#Debezium @gunnarmorling The Challenge ● Deployments in different geographies, single DB ● 90% read requests ● Complex queries Multi-site Application With Shared Database

Slide 6

Slide 6 text

#Debezium @gunnarmorling … Multi-site application with shared system-of-record database … With local, denormalized read views (CQRS) ... Automatically kept in sync after writes Today’s Mission 🤔 Explore How to Build a…

Slide 7

Slide 7 text

#Debezium @gunnarmorling ● Software engineer at Red Hat ○ Debezium ○ Quarkus ● kcctl 🧸, JfrUnit, ModiTect, MapStruct ● Spec Lead for Bean Validation 2.0 ● Java Champion Gunnar Morling

Slide 8

Slide 8 text

#Debezium @gunnarmorling The Idea Caching to the Rescue!

Slide 9

Slide 9 text

#Debezium @gunnarmorling The Idea Caching to the Rescue!

Slide 10

Slide 10 text

https://flic.kr/p/PFDvkY Public Domain, Angelo Brathot

Slide 11

Slide 11 text

#Debezium @gunnarmorling Infinispan 100% Open-source In-Memory Distributed Data Store Interoperability Resilient Fault Tolerant Data Clustered Processing Query ACID Tx

Slide 12

Slide 12 text

#Debezium @gunnarmorling App 1 Data App 2 Data 1, Maria 2, Jenny 2, Jenny 3, Juan App 3 Data 1, Maria 3, Juan Infinispan Deployment Distributed Cache

Slide 13

Slide 13 text

#Debezium @gunnarmorling App 1 Data App 2 Data 1, Maria 2, Jenny 2, Jenny 3, Juan App 3 Data 1, Maria 3, Juan Put 4 Will Put 4 Will Infinispan Deployment Distributed Cache

Slide 14

Slide 14 text

#Debezium @gunnarmorling App 1 Data App 2 Data 1, Maria 2, Jenny 4, Will 2, Jenny 3, Juan 4, Will App 3 Data 1, Maria 3, Juan Infinispan Deployment Distributed Cache

Slide 15

Slide 15 text

#Debezium @gunnarmorling

Slide 16

Slide 16 text

#Debezium @gunnarmorling Infinispan Client/Server Service 1 Service 2 Binary (hot rod) REST ... Service 3 Data Infinispan Cluster Data Data

Slide 17

Slide 17 text

#Debezium @gunnarmorling Infinispan Cross-Site Replication AWS (LON) GCP (NYC) Load Balancer APP APP Service APP APP Service Shared State Shared State Shared State Shared State Data Data Data NYC Data LON RELAY2

Slide 18

Slide 18 text

#Debezium @gunnarmorling The Question How To Keep The Cache In Sync?

Slide 19

Slide 19 text

#Debezium @gunnarmorling Dual Writes Easy to Get Wrong

Slide 20

Slide 20 text

https://flic.kr/p/PFDvkY Public Domain, Angelo Br

Slide 21

Slide 21 text

#Debezium @gunnarmorling Debezium in a Nutshell Open-Source Change Data Capture ● A CDC Platform ○ Based on transaction logs ○ Snapshotting, filtering, etc. ○ Outbox support ○ Web-based UI ● Fully open-source, very active community ● Large production deployments

Slide 22

Slide 22 text

#Debezium @gunnarmorling Debezium Becoming the De-Facto CDC Standard https://debezium.io/blog/2021/09/22/deep-dive-into-a-debezium-community-connector-scylla-cdc-source-connector/

Slide 23

Slide 23 text

#Debezium @gunnarmorling Debezium: Deployment Alternatives Embedded Engine and Debezium Server

Slide 24

Slide 24 text

#Debezium @gunnarmorling Solution Overview Capturing Changes From the Database

Slide 25

Slide 25 text

https://flic.kr/p/PFDvkY Public Domain, Angelo Brathot Demo

Slide 26

Slide 26 text

No content

Slide 27

Slide 27 text

#Debezium @gunnarmorling ● Fast start-up, low memory consumption ● Developer joy ● Imperative and Reactive ● Best-of-breed libraries ● Run via HotSpot and GraalVM native binaries Quarkus - Supersonic Subatomic Java A Stack for Building Cloud-native Apps

Slide 28

Slide 28 text

https://flic.kr/p/PFDvkY Public Domain, Angelo Brathot Demo

Slide 29

Slide 29 text

#Debezium @gunnarmorling Eventual Consistency Dealing With Stale Data ● Cached views updated asynchronously ● How to ensure read your own writes semantics? ● How to prevent taking action based on stale data? 🤔

Slide 30

Slide 30 text

#Debezium @gunnarmorling Eventual Consistency Read Your Own Writes

Slide 31

Slide 31 text

#Debezium @gunnarmorling Eventual Consistency Read Your Own Writes

Slide 32

Slide 32 text

#Debezium @gunnarmorling Eventual Consistency Read Your Own Writes

Slide 33

Slide 33 text

#Debezium @gunnarmorling Eventual Consistency Read Your Own Writes

Slide 34

Slide 34 text

#Debezium @gunnarmorling Eventual Consistency Detecting Writes Derived From Stale State

Slide 35

Slide 35 text

Wrap-Up

Slide 36

Slide 36 text

#Debezium @gunnarmorling … Multi-site application with shared system-of-record database ✅ … With local, denormalized read views (CQRS) ✅ ... Automatically kept in sync after writes ✅ Today’s Mission 🤩 Explore How to Build a…

Slide 37

Slide 37 text

#Debezium @gunnarmorling Discussion What Have We Gained? ● Pro: ○ Lower latencies ○ Reduced load on primary database ○ Increased availability ● Con: ○ Increased complexity

Slide 38

Slide 38 text

#Debezium @gunnarmorling ● Incremental snapshotting ● Postgres logical decoding messages ● Multi-DB support (SQL Server) ● Debezium Server sinks ● MongoDB change streams support ● Debezium UI ● Debezium 2.0 What’s New in Debezium?

Slide 39

Slide 39 text

#Debezium @gunnarmorling ● Infinispan: @infinispan | https://infinispan.org/ ● Debezium: @debezium | https://debezium.io/ ● Demo: https://github.com/debezium/debezium-examples/ → distributed-caching ● kcctl 🧸: https://github.com/kcctl/kcctl/ Learn More

Slide 40

Slide 40 text

#Debezium @gunnarmorling Q & A [email protected] @gunnarmorling 📧 Thank You!

Slide 41

Slide 41 text

No content