Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Life Beyond Relational Database

Life Beyond Relational Database

Introduction to event sourcing in Haskell, in the context of Capital Match

Arnaud Bailly

March 10, 2016

More Decks by Arnaud Bailly

Other Decks in Technology


  1. Who are we? Capital Match is the leading plaform in

    Singapore for peer‐ to‐peer lending to SMEs Backend system developed in Haskell, frontend in Clojurescript/Om since 2014 Core Development team of 3 + 1: Amar, Arnaud, Guo Liang, Zhou Yu
  2. What's good with Relational Model? Really great for querying  → 

    SQL Rocks! Conceptually simple to understand: Everything is a Table Ubiquitous
  3. What's wrong with Relational Model? Writes/updates are complex Impedance Mismatch:

    Lot of data is more tree‐ish or graph‐ ish One single Database for everything  →  SPOF Mutable State
  4. State vs. Transitions RDBMS stores the state of the model

    at some point in me... ... But we are also interested in the transi ons ... ... And state can always be reconstructed from a sequence of transi ons. 1
  5. The Event Sourcing Model Event Sourcing ensures that all changes

    to applica on state are stored as a sequence of events. Not just can we query these events, we can also use the event log to reconstruct past states, and as a founda on to automa cally adjust the state to cope with retroac ve changes. Mar n Fowler
  6. Events makes it easier to... Audit current state and what

    lead to it Implement generic undo/redo mechanism Run simula ons with different hypothesis over live data Cope with data format migra ons Handle poten ally conflic ng changes 2 3
  7. Events Drive Business Events are what makes a model dynamic:

    What affects it, how it reacts to outside world... Provide founda on for techniques  →  Be er business models, Ubiquitous language Lead to technique for "requirements" elicita on and business domain modelling Domain Driven Design Event Storming 4
  8. Pure Business Models Each model delimits a Bounded Context: It

    is responsible for a single cohesive part of the domain Models are pure immutable data structures Dis nguish Commands from Events
  9. Pure Business Models (2) Commands compute Event from State Events

    modify model a c t : : C o m m a n d - > M o d e l - > E v e n t a p p l y : : E v e n t - > M o d e l - > M o d e l
  10. E ectful Services Services are used to orchestrate interac on

    between one or more business models and the outside world Services are func ons opera ng across several contexts They can be synchronous or asynchronous (we use mostly synchronous) There are no distributed transac ons: Service has to cope with failures from each context 5
  11. E ectful Services (2) We have a monad to express

    effects and sequencing on each context: W e b S t a t e M g is a "global" Model which can be accessed concurrently   →  protected in STM l is local data, contextual to a single service execu on m is underlying monad, usually I O n e w t y p e W e b S t a t e M g l m a = W e b S t a t e M { r u n W e b M : : T V a r g - > l - > m a }
  12. Events Storage d a t a S t o r

    e d E v e n t s = S t o r e d E v e n t { e v e n t V e r s i o n : : E v e n t V e r s i o n , e v e n t T y p e : : E v e n t T y p e s , e v e n t D a t e : : D a t e , e v e n t U s e r : : U s e r I d , e v e n t R e q u e s t : : E n c o d e d H e x , e v e n t S H A 1 : : E n c o d e d H e x , e v e n t : : B y t e S t r i n g }
  13. Events Storage (2) We use a simple Append‐only file store,

    wri ng serialized events (mostly JSON) packed with metadata Each event has a (monotonically increasing) version which is used for proper deserializa on Events carry useful informa on for troubleshoo ng and audi ng: User who ini ated the request, request id itself, SHA1 represen ng version of appplica on Events Store serializes concurrent writes
  14. Demo Anatomy of a complete business model Web layer w/

    servant Service layer (w/ Free monads...) Business model Migra on code Standalone service Using Haskell scripts for opera onal queries and updates
  15. Implement Better CQRS Separate Read Model from Write Model Write

    Model: Append‐only linear data store per context, very fast, minimize locking/write me Read model: Op mized for specific querying, may be rela onal if needed in order to make it more user‐friendly
  16. Make models resilient Resilience of models  →  Replica on Use

    to maintain strong consistency of models: Haskell Started implementa on of prac cal cluster based on Ra , called Ra several implementa ons in raptr
  17. Make models secure Turn event stream into a source of

    truth  →  Blockchain and beyond... Juno: over Ra cluster Uses cryptographically signed events to ensure history cannot be tampered with Turns journal into a "legally binding ledger"? 6 Smart contracts
  18. 1. Assuming state is determinis c of course 2. May

    require inver ble events 3. That's the way RDBMS handle transac onal isola on: Record a log of all opera ons on data then reconcile when transac ons are commi ed 4. I never know how many l s modelling takes... 5. Synchronicity is a property of the business domain, e.g. depends on what client expects from the service and whether or not he wants to "pay" for synchronous confirma on 6. Blockchain is all rage in the FinTech ecosystem those days, although early implementa on like Bitcoins or Dogecoins failed to deliver all their promises. ↩ ↩ ↩ ↩ ↩ ↩