Slide 1

Slide 1 text

Building Event Driven Systems at Scale with Azure Cosmos DB Øredev, Malmö November 6th, 2024

Slide 2

Slide 2 text

We‘re all good! I N TR ODU C TI O N !

Slide 3

Slide 3 text

We‘re all good! I N TR ODU C TI O N !

Slide 4

Slide 4 text

We‘re all good! I N TR ODU C TI O N !

Slide 5

Slide 5 text

We‘re all good! I N TR ODU C TI O N !

Slide 6

Slide 6 text

> whoami Shahab Ganji Software Architecture Software Transformation .NET and C# enthusiast Embracing Change Telling dad jokes (Proudly) Code Artisan Lead Coding Architect MA IN F OC US ON T RI VI A @shahab-ganji @shahabganji @shahabganji

Slide 7

Slide 7 text

What is an Event Driven Architecture? I N TR ODU C TI O N Has three main components Software components execute in response to events Uses events to communicate Promotes loose coupling

Slide 8

Slide 8 text

Type of Events W H A T I S A N EV EN T D RI VE N A R C HI T EC T U RE ? ENTITY EVENT KEYED EVENT U NK EYED EV ENT Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet. An entity is a unique thing and is keyed on the unique id of the thing Describes the properties and state of the entity at a given point in time Contains a key, but does not represent an entity Used for partitioning the stream of events to guarantee data locality within a single partition of an event stream Describes an event as a singular statement of a fact

Slide 9

Slide 9 text

EVENT SOURCING Captures every change to the state Provides full audit trail Easier handling of complex transactions Replay what has happened in the system Related Patterns W H A T I S A N EV EN T D RI VE N A R C HI T EC T U RE ? CQR S Separate read and write models Enables optimized performance and scalability https://lostechies.com/jimmybogard/2012/08/22/busting-some-cqrs-myths

Slide 10

Slide 10 text

Event Streams W H A T I S A N EV EN T D RI VE N A R C HI T EC T U RE Forward to EH Snapshot Email Service Inventory Service

Slide 11

Slide 11 text

Schema free, NoSQL Cloud Solution A Z U R E C OS MO S DB Globally Distributed Horizontally Scalable Provisioned throughput Multi model database

Slide 12

Slide 12 text

3 Dimensions of scaling S C A L A BI LI TY https://microservices.io/articles/scalecube.html DATABASE PER APPLICATION Y axis – Functional Decomposition Scale by splitting different things 1 REPLICATION x axis – horizonal decomposition Scale by cloning 2 Sharding z axis – data partitioning Scale by splitting similar things 3

Slide 13

Slide 13 text

S C A L A BI LI TY Sharding Application A single logical database Nodes have different data Cluster of databases Application Application

Slide 14

Slide 14 text

S C A L A BI LI TY Sharding – Database Level PHYSICAL INSTANCES LOGICAL DATABASE

Slide 15

Slide 15 text

S C A L A BI LI TY Sharding – Database Level PHYSICAL INSTANCES LOGICAL DATABASE Application READ/WRITE READ/WRITE

Slide 16

Slide 16 text

Advantages S C A L A BI LI TY Each Server deals with a subset of data Improves transaction scalability Fault Isolation Cache Utilization Reduces Memory & I/O usage

Slide 17

Slide 17 text

Disadvantages S C A L A BI LI TY Increased application complexity Design a Partition Schema Re-partitioning Improper Traffic Distribution Performance Issues with Queries – Cross Partition

Slide 18

Slide 18 text

Containers, Partitions, Request Units A Z U R E C OS MO S DB 2 MB 20 GB 50 GB 50 GB LOGICAL PARTITION PHYSICAL PARTITION 10K RUs 10K RUs

Slide 19

Slide 19 text

Transaction Scope A Z U R E C OS MO S DB LOGICAL PARTITION TRANSACTION BATCH /partition-key: 123 LOGICAL PARTITION

Slide 20

Slide 20 text

Transaction Scope A Z U R E C OS MO S DB LOGICAL PARTITION TRANSACTION BATCH /partition-key: 123 LOGICAL PARTITION

Slide 21

Slide 21 text

Dig deeper! A Z U R E C OS MO S DB WEST EUROPE LEADER FOLLOWER FOLLOWER FOLLOWER

Slide 22

Slide 22 text

CAP theorem A Z U R E C OS MO S DB CONSISTENCY AVIALABILTY PARTITION TOLERANCE

Slide 23

Slide 23 text

Globally Distributed A Z U R E C OS MO S DB

Slide 24

Slide 24 text

Globally Distributed A Z U R E C OS MO S DB WEST EUROPE LEADER FOLLOWER FOLLOWER FORWARDER SOUTH CENTRAL US LEADER FORWARDER FOLLOWER FOLLOWER

Slide 25

Slide 25 text

Strong A Z U R E C OS MO S DB CONSISTENCY LEVELS

Slide 26

Slide 26 text

Bounded Staleness A Z U R E C OS MO S DB CONSISTENCY LEVELS

Slide 27

Slide 27 text

Session A Z U R E C OS MO S DB CONSISTENCY LEVELS

Slide 28

Slide 28 text

Consistent Prefix A Z U R E C OS MO S DB CONSISTENCY LEVELS

Slide 29

Slide 29 text

Eventual A Z U R E C OS MO S DB CONSISTENCY LEVELS

Slide 30

Slide 30 text

Change Feed A Z U R E C OS MO S DB

Slide 31

Slide 31 text

Change Feed A Z U R E C OS MO S DB PHYSICAL PARTITION 1 LOGICAL DATABASE 1 1 1 1 1 1 2 2 2 2 1 1 3 3 2 2 2 2 PHYSICAL PARTITION 2

Slide 32

Slide 32 text

Change Feed A Z U R E C OS MO S DB FeedIterator iteratorForPartitionKey = _container.GetChangeFeedIterator( ChangeFeedStartFrom.Beginning(FeedRange.FromPartitionKey(new PartitionKey("stream-id"))), ChangeFeedMode.LatestVersion);

Slide 33

Slide 33 text

Change Feed A Z U R E C OS MO S DB

Slide 34

Slide 34 text

Summary BU I LD I N G EV EN T D RI VE N A R C HI T EC T U RE W IT H A Z U RE C O S MOS DB E DA AZURE COS MOS DB Global distribution Guaranteed Performance and SLAs Change Feed Automatic Indexing Multi-model and Multi-API support High volume of events Real-time processing Scalability is a primary concern Requires immediate reaction Separate read and write models Audit log Tracking state changes are critical C QR S E V E N T S OU R CI N G

Slide 35

Slide 35 text

Get in touch Shahab aka. Saeed Ganji Code Artisan!

Slide 36

Slide 36 text

Q&A