Rethinking "State Management"

Rethinking State Management currentState = foldl(pastEvents, initialState) @tamizhvendan http://blog.tamizhvendan.in www.ajira.tech

Rethinking State ??

F# CafeApp Courtesy: http://cqrs.nu/

Old School Backend Data Store

Old School Backend Data Store Imperative Data Modeling!

{ "tables" : [{"id" : 1, "status" : "available"}] }
{ "tables" : [{"id" : 1, "status" : "waiting_for_order"}] } { "tables" : [{"id" : 1, "status" : "in_service"}] }

A Different Perspective

t0 t1 Open Tab

t0 t1 Open Tab t2 Place Order

t0 {"status" : "available"}

t0 {"status" : "available"} t1 {"status" : "waiting_for_order"}

t0 {"status" : "available"} t1 {"status" : "waiting_for_order"} t2 {"status"
: "in_service"}

STATE is Transient

Closed Tab

Closed Tab Open Tab

Closed Tab Opened Tab Open Tab

Closed Tab Opened Tab Open Tab Place Order

Closed Tab Placed Order Opened Tab Open Tab Place Order

Serve Drink

Serve Drink Prepare Food

Closed Tab Placed Order Opened Tab Order InProgress Open Tab
Place Order Serve Drink Prepare Food

Serve Drink Closed Tab Placed Order Opened Tab Order InProgress
Open Tab Place Order Serve Drink Prepare Food

Serve Drink Prepare Food Closed Tab Placed Order Opened Tab
Order InProgress Open Tab Place Order Serve Drink Prepare Food

Serve Drink Prepare Food Serve Food Closed Tab Placed Order
Opened Tab Order InProgress Open Tab Place Order Serve Drink Prepare Food

Opened Tab Order InProgress Served Order Open Tab Place Order Serve Drink Prepare Food

Opened Tab Order InProgress Served Order Open Tab Place Order Serve Drink Prepare Food Close Tab

State X

State X Command

State X State Y Command

State X State Y Command // State -> Command ->
State let evolve state command = let newState = // ... newState

State X State Y Command // State -> Command ->
State let evolve state command = let newState = // ... newState let state = ??

Closed Tab

Closed Tab initial state

Closed Tab Open Tab initial state

Closed Tab Opened Tab Open Tab initial state

Closed Tab Opened Tab Open Tab initial state Tab Opened

event

event // State -> Command -> Event list let execute state command = let events = // ... events

Closed Tab Placed Order Opened Tab Order InProgress Served Order

Tab Opened

Tab Opened Order Placed

Tab Opened Order Placed Drink Served Food Prepared

Tab Opened Order Placed Drink Served Food Prepared Drink Served Food Prepared Food Served

Tab Opened Order Placed Drink Served Food Prepared Drink Served Food Prepared Food Served Order Served

Tab Opened Order Placed Drink Served Food Prepared Drink Served Food Prepared Food Served Tab Closed Order Served

Events Are Facts Closed Tab Opened Tab Tab Opened event
current state new state

Events Are Facts It’s like a Bank Statement! Closed Tab
Opened Tab Tab Opened event current state new state

// State -> Event -> State let apply state event
= let newState = // ... newState Events Are Facts It’s like a Bank Statement! Closed Tab Opened Tab Tab Opened event current state new state

Tab Opened Order Placed Drink Served Food Prepared Drink Served Food Prepared Food Served Tab Closed Order Served

Tab Opened Closed Tab Placed Order Opened Tab Order InProgress
Served Order Tab Opened Order Placed Drink Served Food Prepared Drink Served Food Prepared Food Served Tab Closed Order Served

Tab Opened Order Placed Closed Tab Placed Order Opened Tab
Order InProgress Served Order Tab Opened Order Placed Drink Served Food Prepared Drink Served Food Prepared Food Served Tab Closed Order Served

Tab Opened Order Placed Drink Served Closed Tab Placed Order
Opened Tab Order InProgress Served Order Tab Opened Order Placed Drink Served Food Prepared Drink Served Food Prepared Food Served Tab Closed Order Served

Tab Opened Order Placed Drink Served Food Prepared Closed Tab
Placed Order Opened Tab Order InProgress Served Order Tab Opened Order Placed Drink Served Food Prepared Drink Served Food Prepared Food Served Tab Closed Order Served

Tab Opened Order Placed Drink Served Food Prepared Food Served
Closed Tab Placed Order Opened Tab Order InProgress Served Order Tab Opened Order Placed Drink Served Food Prepared Drink Served Food Prepared Food Served Tab Closed Order Served

Order Served Closed Tab Placed Order Opened Tab Order InProgress Served Order Tab Opened Order Placed Drink Served Food Prepared Drink Served Food Prepared Food Served Tab Closed Order Served

Order Served Tab Closed Closed Tab Placed Order Opened Tab Order InProgress Served Order Tab Opened Order Placed Drink Served Food Prepared Drink Served Food Prepared Food Served Tab Closed Order Served

Order Served Tab Closed It’s append only! Closed Tab Placed Order Opened Tab Order InProgress Served Order Tab Opened Order Placed Drink Served Food Prepared Drink Served Food Prepared Food Served Tab Closed Order Served

past events Closed Tab initial state // State -> Event
-> State let apply state event = let newState = // ... newState Tab Opened Order Placed Drink Served Food Prepared Food Served Order Served Tab Closed

Closed Tab Tab Opened Opened Tab past events Closed Tab
initial state // State -> Event -> State let apply state event = let newState = // ... newState Tab Opened Order Placed Drink Served Food Prepared Food Served Order Served Tab Closed

initial state Opened Tab Order Placed Placed Order // State -> Event -> State let apply state event = let newState = // ... newState Tab Opened Order Placed Drink Served Food Prepared Food Served Order Served Tab Closed

initial state Opened Tab Order Placed Placed Order Placed Order Drink Served Order InProgress // State -> Event -> State let apply state event = let newState = // ... newState Tab Opened Order Placed Drink Served Food Prepared Food Served Order Served Tab Closed

initial state Opened Tab Order Placed Placed Order Placed Order Drink Served Order InProgress Food Prepared Order InProgress Order InProgress // State -> Event -> State let apply state event = let newState = // ... newState Tab Opened Order Placed Drink Served Food Prepared Food Served Order Served Tab Closed

initial state Opened Tab Order Placed Placed Order Placed Order Drink Served Order InProgress Food Prepared Order InProgress Order InProgress Food Served Order InProgress Order InProgress // State -> Event -> State let apply state event = let newState = // ... newState Tab Opened Order Placed Drink Served Food Prepared Food Served Order Served Tab Closed

initial state Opened Tab Order Placed Placed Order Placed Order Drink Served Order InProgress Food Prepared Order InProgress Order InProgress Food Served Order InProgress Order InProgress Order Served Served Order Order InProgress // State -> Event -> State let apply state event = let newState = // ... newState Tab Opened Order Placed Drink Served Food Prepared Food Served Order Served Tab Closed

initial state Opened Tab Order Placed Placed Order Placed Order Drink Served Order InProgress Food Prepared Order InProgress Order InProgress Food Served Order InProgress Order InProgress Order Served Served Order Order InProgress Tab Closed Closed Tab Served Order // State -> Event -> State let apply state event = let newState = // ... newState Tab Opened Order Placed Drink Served Food Prepared Food Served Order Served Tab Closed

initial state Opened Tab Order Placed Placed Order Placed Order Drink Served Order InProgress Food Prepared Order InProgress Order InProgress Food Served Order InProgress Order InProgress Order Served Served Order Order InProgress Tab Closed Closed Tab Served Order // State -> Event list -> State let computeState initialState pastEvents = List.fold apply initialState pastEvents // State -> Event -> State let apply state event = let newState = // ... newState Tab Opened Order Placed Drink Served Food Prepared Food Served Order Served Tab Closed

initial state Opened Tab Order Placed Placed Order Placed Order Drink Served Order InProgress Food Prepared Order InProgress Order InProgress Food Served Order InProgress Order InProgress Order Served Served Order Order InProgress Tab Closed Closed Tab Served Order // State -> Event list -> State let computeState initialState pastEvents = List.fold apply initialState pastEvents // State -> Event -> State let apply state event = let newState = // ... newState Performance ?? Tab Opened Order Placed Drink Served Food Prepared Food Served Order Served Tab Closed

// State -> Command -> State let evolve state command
= let newState = // ... newState

= let newState = // ... newState // State -> Command -> Event list let execute state command = let events = // ... events // State -> Event list -> State let computeState initialState pastEvents = List.fold apply initialState pastEvents // State -> Event -> State let apply state event = let newState = // ... newState

= let newState = // ... newState // State -> Command -> Event list let execute state command = let events = // ... events // State -> Event list -> State let computeState initialState pastEvents = List.fold apply initialState pastEvents // State -> Event -> State let apply state event = let newState = // ... newState // Event list -> Command -> Event list let evolve pastEvents command = let state = computeState TabClosed pastEvents execute state command

= let newState = // ... newState // Event list -> Command -> Event list let evolve pastEvents command = let state = computeState TabClosed pastEvents execute state command

= let newState = // ... newState // Event list -> Command -> Event list let evolve pastEvents command = let state = computeState TabClosed pastEvents execute state command We started with this

= let newState = // ... newState // Event list -> Command -> Event list let evolve pastEvents command = let state = computeState TabClosed pastEvents execute state command We started with this and evolved to this!

= let newState = // ... newState // Event list -> Command -> Event list let evolve pastEvents command = let state = computeState TabClosed pastEvents execute state command Current state is a left-fold of past events Event Sourcing - Greg Young We started with this and evolved to this!

= let newState = // ... newState // Event list -> Command -> Event list let evolve pastEvents command = let state = computeState TabClosed pastEvents execute state command currentState = foldl(pastEvents, initialState) Current state is a left-fold of past events Event Sourcing - Greg Young We started with this and evolved to this!

// Event list -> Command -> Event list let evolve
= // ... Handling Commands

= // ... // Command -> int let getTableId command = // ... // int -> Event list let getPastEvents tableId = // ... Handling Commands

= // ... // Command -> int let getTableId command = // ... // int -> Event list let getPastEvents tableId = // ... // Command -> Event list let handleCommand command = let pastEvents = getTableId command |> getPastEvents evolve pastEvents command Handling Commands

// Command -> Event list let handleCommand command = let
pastEvents = getTableId command |> getPastEvents evolve pastEvents command // HttpRequest -> HttpResponse let commandHandler httpRequest = let command = // get Command from HTTP request let newEvents = handleCommand command saveEvents newEvents // return OK response Handling Commands

Serve Food Open Tab Place Order Serve Drink Prepare Food
Close Tab Commands

Table Status Food(s) To Prepare Item(s) To Serve Amount To
Pay

Table Status Food(s) To Prepare Item(s) To Serve Amount To
Pay Queries!!

Old School Backend Data Store

Old School Backend Data Store Imperative Data Modeling!

{ "tables" : [{"id" : 1, "status" : "waiting_for_order"}] } { "tables" : [{"id" : 1, "status" : "in_service"}] } Data Model

{ "tables" : [{"id" : 1, "status" : "waiting_for_order"}] } { "tables" : [{"id" : 1, "status" : "in_service"}] } Data Model Mutate during Commands

{ "tables" : [{"id" : 1, "status" : "waiting_for_order"}] } { "tables" : [{"id" : 1, "status" : "in_service"}] } Data Model Mutate during Commands Fetch during Queries

Single Responsibility Commands Queries Command Query Responsibility Separation

Event Projection Closed Tab Opened Tab Open Tab Tab Opened

Publish

Publish Message Queue Tab Opened Kafka, RabbitMQ, etc.,

Publish Listen Message Queue Tab Opened Kafka, RabbitMQ, etc.,

Publish ReadModel Projection Client Listen Message Queue Tab Opened Kafka, RabbitMQ, etc.,

Publish ReadModel Projection Client Listen Populate Message Queue Tab Opened Kafka, RabbitMQ, etc.,

Publish ReadModel Projection Client Listen Populate Query API Server Message Queue Tab Opened Kafka, RabbitMQ, etc.,

Publish ReadModel Projection Client Listen Populate Query API Server Read Message Queue Tab Opened Kafka, RabbitMQ, etc.,

Publish ReadModel Projection Client Listen Populate Query API Server Read Pull Message Queue Tab Opened Kafka, RabbitMQ, etc.,

Publish ReadModel Projection Client Listen Populate Query API Server Read Listen Pull Message Queue Tab Opened Kafka, RabbitMQ, etc.,

Publish ReadModel Projection Client Listen Populate Query API Server Read WebSocket Client Listen Pull Message Queue Tab Opened Kafka, RabbitMQ, etc.,

Publish ReadModel Projection Client Listen Populate Query API Server Read WebSocket Client Listen Push Pull Message Queue Tab Opened Kafka, RabbitMQ, etc.,

Publish ReadModel Projection Client Listen Populate Query API Server Read WebSocket Client Listen Push Pull Listen Message Queue Tab Opened Kafka, RabbitMQ, etc.,

Publish ReadModel Projection Client Listen Populate Query API Server Read WebSocket Client Listen Push Pull Data Analytics Client Listen Message Queue Tab Opened Kafka, RabbitMQ, etc.,

Publish ReadModel Projection Client Listen Populate Query API Server Read WebSocket Client Listen Push Pull Data Analytics Client Listen Populate Message Queue Tab Opened Kafka, RabbitMQ, etc.,

Publish ReadModel Projection Client Listen Populate Query API Server Read WebSocket Client Listen Push Pull Data Analytics Client Listen Populate Message Queue Tab Opened Kafka, RabbitMQ, etc., Uni directional Data flow (Flux?)

Read Model Projection

Read Model Projection { "tables" : [{"id" : 1, "status"
: "available"}] }

: "available"}] } Tab Opened

: "available"}] } Tab Opened { "tables" : [{"id" : 1, "status" : "waiting_for_order"}] }

: "available"}] } Tab Opened { "tables" : [{"id" : 1, "status" : "waiting_for_order"}] } Order Placed

: "available"}] } Tab Opened { "tables" : [{"id" : 1, "status" : "waiting_for_order"}] } Order Placed { "tables" : [{"id" : 1, "status" : "in_service"}] }

: "available"}] } Tab Opened { "tables" : [{"id" : 1, "status" : "waiting_for_order"}] } Order Placed { "tables" : [{"id" : 1, "status" : "in_service"}] } Order Served

Error Handling

Error Handling User may post an invalid command Fetching Events
may fail Saving Events may fail

Error Handling User may post an invalid command Fetching Events
may fail Saving Events may fail type Result<'T,'E> = | Success of 'T | Error of 'E

User may post an invalid command // Event list ->
Command -> Event list let evolve = // ... // Command -> Event list let handleCommand command = let pastEvents = getTableId command |> getPastEvents evolve pastEvents command // State -> Command -> Event list let execute state command = let events = // ... events

Command -> Event list let evolve = // ... // Command -> Event list let handleCommand command = let pastEvents = getTableId command |> getPastEvents evolve pastEvents command // State -> Command -> Event list let execute state command = let events = // ... events // State -> Command -> Result<Event list,string> let execute state command = match command with | OpenTab -> match state with | ClosedTab -> Success [TabOpened] | _ -> Error "Tab already opened" // ...

Command -> Event list let evolve = // ... // Command -> Event list let handleCommand command = let pastEvents = getTableId command |> getPastEvents evolve pastEvents command // State -> Command -> Event list let execute state command = let events = // ... events // State -> Command -> Result<Event list,string> let execute state command = match command with | OpenTab -> match state with | ClosedTab -> Success [TabOpened] | _ -> Error "Tab already opened" // ... // Event list -> Command -> Result<Event list, string> let evolve = // ...

Command -> Event list let evolve = // ... // Command -> Event list let handleCommand command = let pastEvents = getTableId command |> getPastEvents evolve pastEvents command // State -> Command -> Event list let execute state command = let events = // ... events // State -> Command -> Result<Event list,string> let execute state command = match command with | OpenTab -> match state with | ClosedTab -> Success [TabOpened] | _ -> Error "Tab already opened" // ... // Event list -> Command -> Result<Event list, string> let evolve = // ... // Command -> Result<Event list, string> let handleCommand command = // ...

User may post an invalid command // HttpRequest -> HttpResponse
let commandHandler httpRequest = let command = // get Command from HTTP request let newEvents = handleCommand command saveEvents newEvents // return OK response

let commandHandler httpRequest = let command = // get Command from HTTP request let newEvents = handleCommand command saveEvents newEvents // return OK response // HttpRequest -> HttpResponse let commandHandler httpRequest = let command = //... match handleCommand command with | Success newEvents -> saveEvents newEvents // return OK response | Error message -> // return BAD_REQUEST

let commandHandler httpRequest = let command = // get Command from HTTP request let newEvents = handleCommand command saveEvents newEvents // return OK response // HttpRequest -> HttpResponse let commandHandler httpRequest = let command = //... match handleCommand command with | Success newEvents -> saveEvents newEvents // return OK response | Error message -> // return BAD_REQUEST Fetching Events may fail Saving Events may fail

let commandHandler httpRequest = let command = // get Command from HTTP request let newEvents = handleCommand command saveEvents newEvents // return OK response // HttpRequest -> HttpResponse let commandHandler httpRequest = let command = //... match handleCommand command with | Success newEvents -> saveEvents newEvents // return OK response | Error message -> // return BAD_REQUEST Fetching Events may fail Saving Events may fail https://fsharpforfunandprofit.com/rop/

Performance Tab Opened Order Placed Drink Served Food Prepared Food
Served Order Served Tab Closed

Served Order Served Tab Closed Do I need to fetch all the events to compute the state every time?

Served Order Served Tab Closed Do I need to fetch all the events to compute the state every time? Need not to be!

Served Order Served Tab Closed Do I need to fetch all the events to compute the state every time? Need not to be! Snapshots

Parallel Update

Parallel Update Optimistic Locking

Parallel Update Optimistic Locking Tab Opened 0 Order Placed 1
Drink Served 2 Food Prepared 3 Food Served 4

It’s not a “Silver Bullet” Thinks To Consider

It’s not a “Silver Bullet” Thinks To Consider CQRS and
Event Sourcing are not top-level Architectures

It’s not a “Silver Bullet” Thinks To Consider A whole
system based on Event Sourcing is an Anti-Pattern CQRS and Event Sourcing are not top-level Architectures

- ThoughtWorks Tech Radar 2012 F# is excellent at concisely
expressing business and domain logic - http://fsharp.org/testimonials/

bit.ly/fsapplied https://github.com/tamizhvendan/CafeApp http://fsharp.org

Rethinking "State Management"

Rethinking "State Management"

Transcript