Event Sourcing in Tatami

How we event source at Broadlume using Ruby / Hanami using Google Cloud PubSub

Transcript

1. Event Sourcing in Tatami
2. Some terms Aggregates: take the current state, verify the business rules for the particular operation (e.g. if there is no invoice with the same number) and apply the business logic that returns the new state Eventual Consistency: the idea that, in a distributed system, all the different parts of the system will eventually return all the same values.
3. Events + PubSub Many actions in Tatami are event-based, and responded to in real time We use Google Cloud Pub-Sub as the event bus Subscribers are auto-scaled based on number of outstanding messages in the queue (horizontal pod auto-scaling + prometheus = )
4. Hanami.events.broadcast('lead.submitted', lead_info, ordering_key: lead_info[:id]) Hanami.events.broadcast('lead.updated', lead_info, ordering_key: lead_info[:id]) class LeadTagger < Subscriber subscribe_to 'lead.submitted', deadline: 60 subscribe_to 'lead.updated', deadline: 60 # send it def call(lead_info); end end class LeadWebhookDispatcher < Subscriber subscribe_to 'lead.submitted', deadline: 60 subscribe_to 'lead.updated', deadline: 60 # send it def call(lead_info); end end
5. Event Store We also store the raw, sequenced events in the DB, and have consumers/subscribers which build the aggregated state in plain old tables.
6. You can replay the events: idempotency and all of that jazz, although it’s rarely needed since Tatami is the sole user most of the time. The “clean architecture” in Tatami makes this straightforward.
7. DHQ / Event Sourcing for Users For making event sourcing useful to DHQ users, we also have structured/typed approach to event sourcing, so can make it look pretty in the UI. Think “activity log.” It makes use of DB triggers, some materialized views, and POROs to do it.
8. The majority of the DHQ app is modeled in an event sourcing pattern: phone calls are events chats are events a form fill is an event reviews are events comments, assignments, uploads, etc are stored as events, too! The other things, like inferred prospects, flow out of that “event stream.” The purists would call them projections.
9. In a sense, it’s nicer than using an opinionated framework which does “event sourcing,” since the mental model is easier to understand and more natural, since we are talking about concrete things that happen in the real world. A developer doesn’t have to try to explain to a reactor or aggregate to a PM, or vice versa. Maybe we’ll call it “organic” event sourcing, USDA approved!
10. Future Move to a common event bus a la Kafka more easily pluggable for other teams built in schema registry materialized event views with ksql built in syncing using offsets infinitely #webscale