Event Sourcing: A New Dimension in Software Design

Event Sourcing: A New Dimension in Software Design

In this session I will introduce you to a novel way of designing information systems - Event Sourcing. This pattern has been applied for ages in other industries, but only recently it was rediscovered in software design. Although the idea is actually pretty simple , it states that all changes to application state should be modeled as a sequence of events, its benefits are numerous: * It fits like a glove building scalable, highly concurrent, distributed systems * Provides a truly transactional audit log of everything that happened in the system * Allows rewinding application state to any point in time for analysis, and even retroactive debugging * And most importantly - although it is a technical pattern, it provides a business value that can be transformed into a competitive advantage The focus of my talk will be the Event Sourcing pattern, but I’ll also briefly describe CQRS - an architecture that goes hand in hand with Event Sourcing. This knowledge will allow you to employ this powerful pattern in your next project.

B90ab53ba7cf16ed1a4bb679cc6751d7?s=128

Vladik Khononov

November 17, 2016
Tweet

Transcript

  1. None
  2. 15 – 16 November, Sofia ISTACon.org Event Sourcing By Vladik

    Khononov A New Dimension in Software Design
  3. 15 – 16 November, Sofia ISTACon.org

  4. 15 – 16 November, Sofia ISTACon.org

  5. 15 – 16 November, Sofia ISTACon.org

  6. 15 – 16 November, Sofia ISTACon.org Agenda • How it

    happened • Why it happened • How event sourcing solves the problem • Why event sourcing benefits the company
  7. 15 – 16 November, Sofia ISTACon.org Call Center Management

  8. 15 – 16 November, Sofia ISTACon.org LEAD • Id •

    Name • Email • Phone • Status (Available / Follow-up / Closed / Converted)
  9. 15 – 16 November, Sofia ISTACon.org Name Email Phone Status

    Stefan Ivanov stefan@ivanov.com 03-27243457 Available Viktor Kovachev viktor@kovachev.com 08-8332491 Available Martin Mateev martin@mateev.com 04-8537112 Available Lilyana Yankov lilyana@yankov.com 04-6092212 Available
  10. 15 – 16 November, Sofia ISTACon.org Name Email Phone Status

    Stefan Ivanov stefan@ivanov.com 03-27243457 Available Viktor Kovachev viktor@kovachev.com 08-8332491 Available Martin Mateev martin@mateev.com 04-8537112 Available Lilyana Yankov lilyana@yankov.com 04-6092212 Available
  11. 15 – 16 November, Sofia ISTACon.org

  12. 15 – 16 November, Sofia ISTACon.org public class Lead {

    public long Id { get; set; } public string Name { get; set; } public string Email { get; set; } public string Phone { get; set; } public LeadStatus Status { get; set; } public DateTime? FollowupOn { get; set; } }
  13. 15 – 16 November, Sofia ISTACon.org

  14. 15 – 16 November, Sofia ISTACon.org SELECT * FROM leads

    WHERE [name] LIKE ‘%name%’ OR [phone] LIKE ‘%phone%’ OR [email] LIKE ‘%email%’;
  15. WTF??? ISTACon.org

  16. 15 – 16 November, Sofia ISTACon.org Lead #1410 Name Viktor

    Radkov Email viktor@radkov.com Phone 09-9801298 Status Available
  17. 15 – 16 November, Sofia ISTACon.org

  18. 15 – 16 November, Sofia ISTACon.org PM

  19. 15 – 16 November, Sofia ISTACon.org …1 month later

  20. 15 – 16 November, Sofia ISTACon.org

  21. 15 – 16 November, Sofia ISTACon.org PM

  22. 15 – 16 November, Sofia ISTACon.org …2 months later

  23. 15 – 16 November, Sofia ISTACon.org

  24. 15 – 16 November, Sofia ISTACon.org PM

  25. 15 – 16 November, Sofia ISTACon.org Search on stale data

    Status change dates Audit log for BI
  26. 15 – 16 November, Sofia ISTACon.org PM

  27. 15 – 16 November, Sofia ISTACon.org PM

  28. 15 – 16 November, Sofia ISTACon.org public class Lead {

    public long Id { get; set; } public string Name { get; set; } public string Email { get; set; } public string Phone { get; set; } public LeadStatus Status { get; set; } public DateTime? FollowupOn { get; set; } }
  29. 15 – 16 November, Sofia ISTACon.org Name Email Phone Status

    Follow-up Stefan Ivanov stefan@ivanov.com 03-27243457 Available
  30. 15 – 16 November, Sofia ISTACon.org Name Email Phone Status

    Follow-up Date Stefan Ivanov stefan@ivanov.com 050-8139904 Follow-up 23/11/2016
  31. 15 – 16 November, Sofia ISTACon.org Name Email Phone Status

    Follow-up Date Stefan Ivanov stefan@ivanov.com 050-8139904 Converted
  32. 15 – 16 November, Sofia ISTACon.org Name Email Phone Status

    Follow-up Date Stefan Ivanov stefan@ivanov.com 050-8139904 Converted
  33. 15 – 16 November, Sofia ISTACon.org

  34. 15 – 16 November, Sofia ISTACon.org Event Sourcing

  35. 15 – 16 November, Sofia ISTACon.org Changes = First class

    citizens = Events
  36. 15 – 16 November, Sofia ISTACon.org Lead Events • Lead

    Was Initialized (Id) • Status Changed (NewStatus) • Name Changed (NewName) • Contact Information Changed (NewEmail, NewPhone) • Followup Set (Date)
  37. 15 – 16 November, Sofia ISTACon.org Lead: Stefan Ivanov •

    Lead Was Initialized (1410) • Status Changed (Available) • Contact Information Changed (stefan@ivanov.com, 03-27243457) • Status Changed (Followup) • Followup Set (23/11/2016) • Contact Information Changed (stefan@ivanov.com, 050-8139904) • Status Changed (Converted)
  38. 15 – 16 November, Sofia ISTACon.org public class Lead {

    public long Id { get; set; } public string Name { get; set; } public string Email { get; set; } public string Phone { get; set; } public LeadStatus Status { get; set; } public DateTime? FollowupOn { get; set; } public void Apply(LeadWasInitialized event) {…} public void Apply(StatusChanged event) {…} public void Apply(NameChanged event) {…} public void Apply(ContactInfoChanged event) {…} public void Apply(FollowupSet event) {…} }
  39. 15 – 16 November, Sofia ISTACon.org public class Lead {

    public long Id { get; set; } public string Name { get; set; } public string Email { get; set; } public string Phone { get; set; } public LeadStatus Status { get; set; } public DateTime? FollowupOn { get; set; } public void Apply(LeadWasInitialized event) {…} public void Apply(StatusChanged event) {…} public void Apply(NameChanged event) {…} public void Apply(ContactInfoChanged event) {…} public void Apply(FollowupSet event) {…} } Apply(StatusChanged event) { this.Status = event.NewStatus; }
  40. 15 – 16 November, Sofia ISTACon.org public class Lead {

    public long Id { get; set; } public string Name { get; set; } public string Email { get; set; } public string Phone { get; set; } public LeadStatus Status { get; set; } public DateTime? FollowupOn { get; set; } public void Apply(LeadWasInitialized event) {…} public void Apply(StatusChanged event) {…} public void Apply(NameChanged event) {…} public void Apply(ContactInfoChanged event) {…} public void Apply(FollowupSet event) {…} } Apply(NameChanged event) { this.Name = event.NewName; }
  41. 15 – 16 November, Sofia ISTACon.org public class Lead {

    public long Id { get; set; } public string Name { get; set; } public string Email { get; set; } public string Phone { get; set; } public LeadStatus Status { get; set; } public DateTime? FollowupOn { get; set; } public void Apply(LeadWasInitialized event) {…} public void Apply(StatusChanged event) {…} public void Apply(NameChanged event) {…} public void Apply(ContactInfoChanged event) {…} public void Apply(FollowupSet event) {…} } Apply(ContactInfoChanged event) { this.Phone = event.NewPhone; this.Email = event.NewEmail; }
  42. 15 – 16 November, Sofia ISTACon.org public class Lead {

    public long Id { get; set; } public string Name { get; set; } public string Email { get; set; } public string Phone { get; set; } public LeadStatus Status { get; set; } public DateTime? FollowupOn { get; set; } public void Apply(LeadWasInitialized event) {…} public void Apply(StatusChanged event) {…} public void Apply(NameChanged event) {…} public void Apply(ContactInfoChanged event) {…} public void Apply(FollowupSet event) {…} } Apply(FollowupSet event) { this.FollowUpOn = event.Date; }
  43. 15 – 16 November, Sofia ISTACon.org Lead: Stefan Ivanov Lead

    Was Initialized (1410) Status Changed (Available) Contact Information Changed (stefan@ivanov.com,03-27243457) Status Changed (Converted) Status Changed (Followup) Followup Set (23/11/2016) Contact Information Changed (stefan@ivanov.com, 050-8139904)
  44. 15 – 16 November, Sofia ISTACon.org Name Email Phone Status

    Follow-up Date Stefan Ivanov stefan@ivanov.com 050-8139904 Converted
  45. 15 – 16 November, Sofia ISTACon.org Lead: Stefan Ivanov Lead

    Was Initialized (1410) Status Changed (Available) Contact Information Changed (stefan@ivanov.com,03-27243457) Status Changed (Converted) Status Changed (Followup) Followup Set (23/11/2016) Contact Information Changed (stefan@ivanov.com, 050-8139904)
  46. 15 – 16 November, Sofia ISTACon.org Lead: Stefan Ivanov Lead

    Was Initialized (1410) Status Changed (Available) Contact Information Changed (stefan@ivanov.com,03-27243457)
  47. 15 – 16 November, Sofia ISTACon.org Lead: Stefan Ivanov Lead

    Was Initialized (1410) Status Changed (Available) Contact Information Changed (stefan@ivanov.com,03-27243457) Status Changed (Followup) Followup Set (23/11/2016) Contact Information Changed (stefan@ivanov.com, 050-8139904)
  48. 15 – 16 November, Sofia ISTACon.org Lead: Stefan Ivanov Lead

    Was Initialized (1410) Status Changed (Available) Contact Information Changed (stefan@ivanov.com,03-27243457) Status Changed (Converted) Status Changed (Followup) Followup Set (23/11/2016) Contact Information Changed (stefan@ivanov.com, 050-8139904)
  49. 15 – 16 November, Sofia ISTACon.org public class Lead {

    public long Id { get; set; } public string Name { get; set; } public string Email { get; set; } public string Phone { get; set; } public LeadStatus Status { get; set; } public DateTime? FollowupOn { get; set; } }
  50. 15 – 16 November, Sofia ISTACon.org Lead: Stefan Ivanov Lead

    Was Initialized (1410) Status Changed (Available) Contact Information Changed (stefan@ivanov.com,03-27243457) Status Changed (Converted) Status Changed (Followup) Followup Set (23/11/2016) Contact Information Changed (stefan@ivanov.com, 050-8139904)
  51. 15 – 16 November, Sofia ISTACon.org public class LeadSearch {

    public long Id { get; private set; } … public List<string> Emails; public List<string> Phones; … public void Apply(ContactInfoChanged event) {…} … } Apply(ContactInfoChanged event) { this.Phones.Append(event.NewPhone); this.Emails.Append(event.NewEmail); }
  52. 15 – 16 November, Sofia ISTACon.org public class LeadStatusChangesModel {

    public long Id { get; set; } public List<StatusLog> StatusChangesLog; … public void Apply(StatusChanged event) {…} … } Apply(StatusChanged event) { this.StatusChangesLog.Append( new StatusLog(event.NewStatus, DateTime.Now); );
  53. 15 – 16 November, Sofia ISTACon.org public class LeadStatusChangesModel {

    public long Id { get; set; } public List<StatusLog> StatusChangesLog; … } public class Lead { public long Id { get; set; } public string Name { get; set; } public string Email { get; set; } public string Phone { get; set; } public LeadStatus Status { get; set; } public DateTime? FollowupOn { get; set; } … } public class LeadSearch { public long Id { get; private set; } public List<string> Emails; public List<string> Phones; … }
  54. 15 – 16 November, Sofia ISTACon.org Lead: Stefan Ivanov Lead

    Was Initialized (1410) Status Changed (Available) Contact Information Changed (stefan@ivanov.com,03-27243457) Status Changed (Converted) Status Changed (Followup) Followup Set (23/11/2016) Contact Information Changed (stefan@ivanov.com, 050-8139904)
  55. 15 – 16 November, Sofia ISTACon.org Events = Source of

    Truth Event Sourcing
  56. 15 – 16 November, Sofia ISTACon.org

  57. 15 – 16 November, Sofia ISTACon.org

  58. 15 – 16 November, Sofia ISTACon.org

  59. 15 – 16 November, Sofia ISTACon.org Storage: Event Store

  60. 15 – 16 November, Sofia ISTACon.org Event Store API: The

    Good News Append(Entity Id + New events) GetEvents(Entity Id) Event1,
 Event2,
 Event3, ….
  61. 15 – 16 November, Sofia ISTACon.org Key (Entity Id) Value

    (Events list) Lead #1410 1. LeadWasInitialized(1410) 2. StatusChanged(Available) 3. NameChanged(“Martin Mateev”) 4. StatusChanged(Converted) Lead #1406 1. LeadWasInitialized(1406) 2. StatusChanged(Available’’’) 3. NameChanged(“Lilyana Yankov”)
  62. 15 – 16 November, Sofia ISTACon.org Event Store API: The

    Good News Append(Entity Id + New events) GetEvents(Entity Id) Event1,
 Event2,
 Event3, ….
  63. 15 – 16 November, Sofia ISTACon.org Event Store API: The

    Bad News Append(Entity Id + New events) GetEvents(Entity Id) Event1,
 Event2,
 Event3, ….
  64. 15 – 16 November, Sofia ISTACon.org CQRS Command Query Responsibility

    Segregation
  65. 15 – 16 November, Sofia ISTACon.org CQRS • Command -

    write data • Query - read data • A use case can be either Command or Query. Never both
  66. 15 – 16 November, Sofia ISTACon.org Queries Read Model Read

    DB Commands Write Model Writes DB Projection engine
  67. 15 – 16 November, Sofia ISTACon.org Event Sourced Model (Write)

    Lead #1410 1. LeadWasInitialized(1410) 2. StatusChanged(Available) 3. NameChanged(“Martin Mateev”) 4. StatusChanged(Closed) … 1000. StatusChanged(FollowUp) 1001. FollowupSet(16/11/2017) Read Model Id 1410 Name Martin Mateev Status Followup FollowupOn (Empty) Email … Phone …
  68. 15 – 16 November, Sofia ISTACon.org Queries Read Model Read

    DB Commands Write Model Writes DB Projection engine
  69. 15 – 16 November, Sofia ISTACon.org Event Sourced Model (Write)

    Lead #1410 1. LeadWasInitialized(1410) 2. StatusChanged(Available) 3. NameChanged(“Martin Mateev”) 4. StatusChanged(Closed) 5. StatusChanged(Available) 6. NameChanged(“Viktor Rumenov”) Read Model Id 1410 Phone … Status Available FollowupOn (Empty) Email … Name Martin Mateev
  70. 15 – 16 November, Sofia ISTACon.org public class Lead {

    public long Id { get; set; } public string Name { get; set; } public string Email { get; set; } public string Phone { get; set; } public LeadStatus Status { get; set; } public DateTime? FollowupOn { get; set; } public void Apply(LeadWasInitialized event) {…} public void Apply(StatusChanged event) {…} public void Apply(NameChanged event) {…} public void Apply(ContactInfoChanged event) {…} public void Apply(FollowupSet event) {…} } Apply(NameChanged event) { this.Name = event.NewName; }
  71. 15 – 16 November, Sofia ISTACon.org Queries Read Model Read

    DB Commands Event Sourcing Event Store Projection engine
  72. 15 – 16 November, Sofia ISTACon.org Queries Read Model Read

    DB Commands Write Model Writes DB Projection engine Projection • RDBMS • Documents • Graphs • Files
  73. 15 – 16 November, Sofia ISTACon.org Queries Read Model Commands

    Write Model Writes DB Projection engine Projection • RDBMS • Documents • Graphs • Files • Multiple DBs Read DBs
  74. 15 – 16 November, Sofia ISTACon.org Queries Read Model Commands

    Write Model Writes DB Projection engine Read DBs Concurrency • Pessimistic • Optimistic • Optimistic on steroids
  75. 15 – 16 November, Sofia ISTACon.org SetFollowupCommand + StatusChangeEvent =>

    Collision ChangeNameCommand + StatusChangeEvent => OK
  76. 15 – 16 November, Sofia ISTACon.org Event Sourcing + CQRS

    • Flexible business domain modeling • Insane scalability and availability • Rock solid infrastructure • …look great on your C.V.
  77. 15 – 16 November, Sofia ISTACon.org

  78. 15 – 16 November, Sofia ISTACon.org CQRS: When? Event Sourcing

    ‎CQRS Non-functional benefits
  79. 15 – 16 November, Sofia ISTACon.org Event Sourcing CQRS

  80. 15 – 16 November, Sofia ISTACon.org Event Sourcing: When? ?

  81. 15 – 16 November, Sofia ISTACon.org Subdomains Generic, Supporting, Core

  82. 15 – 16 November, Sofia ISTACon.org

  83. 15 – 16 November, Sofia ISTACon.org Subdomains Generic, Supporting, Core

  84. 15 – 16 November, Sofia ISTACon.org CREATIVE CATALOG CAMPAIGN PUBLISHING

    BILLING USERS MANAGEMENT LEAD MANAGEMENT COMMISSIONS CALCULATION DESK MANAGEMENT VOIP MANAGEMENT EMAIL CAMPAIGNS
  85. 15 – 16 November, Sofia ISTACon.org Generic Subdomains

  86. 15 – 16 November, Sofia ISTACon.org CREATIVE CATALOG CAMPAIGN PUBLISHING

    BILLING USERS MANAGEMENT LEAD MANAGEMENT COMMISSIONS CALCULATION DESK MANAGEMENT VOIP MANAGEMENT EMAIL CAMPAIGN
  87. 15 – 16 November, Sofia ISTACon.org CREATIVE CATALOG CAMPAIGN PUBLISHING

    BILLING USERS MANAGEMENT LEAD MANAGEMENT COMMISSIONS CALCULATION DESK MANAGEMENT VOIP MANAGEMENT EMAIL CAMPAIGN
  88. 15 – 16 November, Sofia ISTACon.org Supporting Subdomains

  89. 15 – 16 November, Sofia ISTACon.org CREATIVE CATALOG CAMPAIGN PUBLISHING

    BILLING USERS MANAGEMENT LEAD MANAGEMENT COMMISSIONS CALCULATION DESK MANAGEMENT VOIP MANAGEMENT EMAIL CAMPAIGN
  90. 15 – 16 November, Sofia ISTACon.org CREATIVE CATALOG CAMPAIGN PUBLISHING

    BILLING USERS MANAGEMENT LEAD MANAGEMENT COMMISSIONS CALCULATION DESK MANAGEMENT VOIP MANAGEMENT EMAIL CAMPAIGN
  91. 15 – 16 November, Sofia ISTACon.org Core Domains

  92. 15 – 16 November, Sofia ISTACon.org CREATIVE CATALOG CAMPAIGN PUBLISHING

    BILLING USERS MANAGEMENT LEAD MANAGEMENT COMMISSIONS CALCULATION DESK MANAGEMENT VOIP MANAGEMENT EMAIL CAMPAIGN
  93. 15 – 16 November, Sofia ISTACon.org Event Sourcing: When? Core

    business domains
  94. 15 – 16 November, Sofia ISTACon.org Before you try this

    at home
  95. 15 – 16 November, Sofia ISTACon.org Available Event Stores •

    http://GetEventStore.com • NEventStore • Akka Persistence • Elastic Event Store (Coming soon)
  96. 15 – 16 November, Sofia ISTACon.org

  97. 15 – 16 November, Sofia ISTACon.org €100 Discount for attendees

    of ISTA 2016 https://ti.to/webengineers/ddd17/discount/istacon2016
  98. 15 – 16 November, Sofia ISTACon.org Domain-Driven Design • How

    to identify subdomains? • How to define business entities? • How do define events? • How to talk to business experts?
  99. 15 – 16 November, Sofia ISTACon.org

  100. 15 – 16 November, Sofia ISTACon.org Thank you! @vladikk vladikk.com

    vladikkhononov vladik@khononov.com