🔮 Event Sourcing in Azure Which database to use?

🖖 Hello! @spontoreau Sylvain Pontoreau 🎪 Co-organizer 📕 Book writer Microsoft 󰳕 Work with

📖 Time for self-promotion

🕺 Why I am talking about this topic ? I have been diving deep in it for a long time!

❓ What is Event Sourcing

💁 The Event Sourcing pattern “Every change to the state of an application is captured in an event” https://martinfowler.com/eaaDev/EventSourcing.html

✨ Event sourcing benefits Rebuild Temporal queries Replay

🎭 Event sourcing use cases Audit Debug Consistency

🗄 Event Store Sequence of events Event immutability Single point of Truth

📨 Event Stream Sequence of events = Event Stream Event Stream Event Event Event Event Event Event Event Event Event Timeline

📨 Event Stream Load from stream Event Event Event Event Event Event Event Event Event Load Reduce State

📨 Event Stream Append event Event Event Event Event Event Event Event Event Event Append Event

📨 Event Stream Subscribe Event Event Event Event Event Event Event Event Event Append Event Subscribe

📰 Read operations Load Reduce State A little bit expensive for read operations… … and not really great for querying :(

📽 Projections Using subscribe to build a queryable representation of the application state (aka Projections) Subscribe Projections Projector ℹ A good database for event sourcing must have a data streaming mechanism

✋ Keep that in mind App DB Projections Append Subscribe App Projections Append Publish ✅ ❌ DB ❗ Even if you’re using a Message Broker or an Event Bus

⚠ Eventually consistent CAP Theorem (Eric Brewer) Consistency Availability Partition tolerance All system nodes needs to manipulate exactly the same data at the same time Requests must succeed Node failure must not create a complete system failure A distributed system can only guarantee 2 constraints at the same time, not 3…

🔃 CQRS pattern What is CQRS? • A pattern designed by Greg Young • Command Query Responsibility Segregation • Read != Write (different needs) Query A read operation Command Validation Business logic Persistence CQRS suit perfectly well with Event Sourcing!

🔃 CQRS pattern GetBalance Query Deposite Command GetAllOperation Query Withdraw Command … … Dispatcher Read DB Dispatcher QueryHandler CommandHandler Event Store Projection Projection Aggregate State Events Behaviors

☁ Azure Flexibility Hyper scale Out of the box

🗃 PostgreSQL - Table CREATE TABLE "EVENT_STREAM" ( "streamId" VARCHAR NOT NULL, "createdAt" VARCHAR NOT NULL, "type" VARCHAR NOT NULL, "version" INTEGER NOT NULL, "payload" JSONB, PRIMARY KEY ("streamId", "createdAt", "version") ); CREATE INDEX ON "EVENT_STREAM" ("streamId");

🗃 PostgreSQL - Notify CREATE FUNCTION notify_new_event() RETURNS TRIGGER AS $trigger$ BEGIN PERFORM pg_notify('new_event', row_to_json(NEW)::text); RETURN NULL; END; $trigger$ LANGUAGE plpgsql VOLATILE COST 100;

🗃 PostgreSQL - Trigger CREATE TRIGGER notify_new_event AFTER INSERT ON "EVENT_STREAM" FOR EACH ROW EXECUTE PROCEDURE notify_new_event();

🗃 PostgreSQL - Rules CREATE FUNCTION THROW_WHEN_TYRING_TO_UPDATE_EVENT_STREAM() RETURNS VOID AS $$ BEGIN RAISE EXCEPTION 'Event Stream can not be updated'; END; $$ LANGUAGE plpgsql VOLATILE; CREATE RULE "PREVENT_DELETE_ON_EVENT_STREAM" AS ON DELETE TO "EVENT_STREAM" DO INSTEAD SELECT THROW_WHEN_TYRING_TO_UPDATE_EVENT_STREAM(); CREATE RULE "PREVENT_UPDATE_ON_EVENT_STREAM" AS ON UPDATE TO "EVENT_STREAM" DO INSTEAD SELECT THROW_WHEN_TYRING_TO_UPDATE_EVENT_STREAM();

🗃 PostgreSQL - Load events SELECT "type", "payload", "version" FROM "EVENT_STREAM" WHERE "streamId" = $1 ORDER BY "createdAt", "version"

🗃 PostgreSQL - Append events INSERT INTO "EVENT_STREAM" ( "streamId", "createdAt", "type", "version", "payload" ) VALUES ($1, $2, $3, ( SELECT COUNT("streamId") + 1 FROM "EVENT_STREAM" WHERE "streamId" = $5 ), $4 )

🗃 PostgreSQL - Conclusion ✅ ❌ • Simple to implement • One impl for all languages • On premise if needed • DB Tuning • You must know how to do it • Distributed • No acknowledgement

🗃 SQL Server The same thing but with “Service Broker” 😅

🌐 Cosmos DB Change feed for the win! Change Feed Event Stream Data

🌐 Cosmos DB Table, Trigger, Notify ➡ Nothing to do !!!

🌐 Cosmos DB - Load stream const endpoint = "https://your-account.documents.azure.com"; const key = ""; const client = new CosmosClient({ endpoint, key }); const container = client.database("EventStore").container("EventStream"); const querySpec: SqlQuerySpec = { query: `SELECT eventStream.eventType, eventStream.payload, eventStream.version, FROM eventStream WHERE eventStream.streamId = @streamId ORDER BY eventStream.createdAt, eventStream.version`, parameters: [{ name: "@streamId", value: "" } ] }; const { resources } = await container.items.query(querySpec).fetchAll();

🌐 Cosmos DB - Append events const endpoint = "https://your-account.documents.azure.com"; const key = ""; const client = new CosmosClient({ endpoint, key }); const container = client.database("EventStore").container("EventStream"); await container.items.create({ streamId: "", createdAt: "", version: , payload: { //… } });

🌐 Cosmos DB - Conclusion ✅ ❌ • Straightforward • Serverless • Schemaless • No rules • Version computed in code

🌀 Event Store DB Install the SDK and enjoy 🥳

🌀 Event Store DB - Load stream const client = new EventStoreDBClient({ endpoint: "", }); const streamName = ""; const events = client.readStream(streamName, { fromRevision: START, direction: FORWARDS, maxCount: 10, });

🌀 Event Store DB - Append events const client = new EventStoreDBClient({ endpoint: "", }); const streamName = ""; const event = jsonEvent({ type: "", data: { //… }, }); await client.appendToStream(streamName, [event]);

🌀 Event Store DB ✅ • Out of the box • State of the art • Built in projection • On premise if needed • Manage outside of Azure… • Not too much customization • Only gRPC client ❌

Conclusion

Questions❓

🔮 Thanks!