Event Sourcing: A New Dimension in Software Design

In this session I will introduce you to a novel way of designing information systems - Event Sourcing. This pattern has been applied for ages in other industries, but only recently it was rediscovered in software design. Although the idea is actually pretty simple , it states that all changes to application state should be modeled as a sequence of events, its benefits are numerous: * It fits like a glove building scalable, highly concurrent, distributed systems * Provides a truly transactional audit log of everything that happened in the system * Allows rewinding application state to any point in time for analysis, and even retroactive debugging * And most importantly - although it is a technical pattern, it provides a business value that can be transformed into a competitive advantage The focus of my talk will be the Event Sourcing pattern, but I’ll also briefly describe CQRS - an architecture that goes hand in hand with Event Sourcing. This knowledge will allow you to employ this powerful pattern in your next project.

Transcript

2. 15 – 16 November, Sofia ISTACon.org Event Sourcing By Vladik Khononov A New Dimension in Software Design
3. 15 – 16 November, Sofia ISTACon.org
4. 15 – 16 November, Sofia ISTACon.org
5. 15 – 16 November, Sofia ISTACon.org
6. 15 – 16 November, Sofia ISTACon.org Agenda • How it happened • Why it happened • How event sourcing solves the problem • Why event sourcing benefits the company
7. 15 – 16 November, Sofia ISTACon.org Call Center Management
8. 15 – 16 November, Sofia ISTACon.org LEAD • Id • Name • Email • Phone • Status (Available / Follow-up / Closed / Converted)
9. 15 – 16 November, Sofia ISTACon.org Name Email Phone Status Stefan Ivanov stefan@ivanov.com 03-27243457 Available Viktor Kovachev viktor@kovachev.com 08-8332491 Available Martin Mateev martin@mateev.com 04-8537112 Available Lilyana Yankov lilyana@yankov.com 04-6092212 Available
10. 15 – 16 November, Sofia ISTACon.org Name Email Phone Status Stefan Ivanov stefan@ivanov.com 03-27243457 Available Viktor Kovachev viktor@kovachev.com 08-8332491 Available Martin Mateev martin@mateev.com 04-8537112 Available Lilyana Yankov lilyana@yankov.com 04-6092212 Available
11. 15 – 16 November, Sofia ISTACon.org
12. 15 – 16 November, Sofia ISTACon.org public class Lead { public long Id { get; set; } public string Name { get; set; } public string Email { get; set; } public string Phone { get; set; } public LeadStatus Status { get; set; } public DateTime? FollowupOn { get; set; } }
13. 15 – 16 November, Sofia ISTACon.org
14. 15 – 16 November, Sofia ISTACon.org SELECT * FROM leads WHERE [name] LIKE ‘%name%’ OR [phone] LIKE ‘%phone%’ OR [email] LIKE ‘%email%’;
15. WTF??? ISTACon.org
16. 15 – 16 November, Sofia ISTACon.org Lead #1410 Name Viktor Radkov Email viktor@radkov.com Phone 09-9801298 Status Available
17. 15 – 16 November, Sofia ISTACon.org
18. 15 – 16 November, Sofia ISTACon.org PM
19. 15 – 16 November, Sofia ISTACon.org …1 month later
20. 15 – 16 November, Sofia ISTACon.org
21. 15 – 16 November, Sofia ISTACon.org PM
22. 15 – 16 November, Sofia ISTACon.org …2 months later
23. 15 – 16 November, Sofia ISTACon.org
24. 15 – 16 November, Sofia ISTACon.org PM
25. 15 – 16 November, Sofia ISTACon.org Search on stale data Status change dates Audit log for BI
26. 15 – 16 November, Sofia ISTACon.org PM
27. 15 – 16 November, Sofia ISTACon.org PM
28. 15 – 16 November, Sofia ISTACon.org public class Lead { public long Id { get; set; } public string Name { get; set; } public string Email { get; set; } public string Phone { get; set; } public LeadStatus Status { get; set; } public DateTime? FollowupOn { get; set; } }
29. 15 – 16 November, Sofia ISTACon.org Name Email Phone Status Follow-up Stefan Ivanov stefan@ivanov.com 03-27243457 Available
30. 15 – 16 November, Sofia ISTACon.org Name Email Phone Status Follow-up Date Stefan Ivanov stefan@ivanov.com 050-8139904 Follow-up 23/11/2016
31. 15 – 16 November, Sofia ISTACon.org Name Email Phone Status Follow-up Date Stefan Ivanov stefan@ivanov.com 050-8139904 Converted
32. 15 – 16 November, Sofia ISTACon.org Name Email Phone Status Follow-up Date Stefan Ivanov stefan@ivanov.com 050-8139904 Converted
33. 15 – 16 November, Sofia ISTACon.org
34. 15 – 16 November, Sofia ISTACon.org Event Sourcing
35. 15 – 16 November, Sofia ISTACon.org Changes = First class citizens = Events
36. 15 – 16 November, Sofia ISTACon.org Lead Events • Lead Was Initialized (Id) • Status Changed (NewStatus) • Name Changed (NewName) • Contact Information Changed (NewEmail, NewPhone) • Followup Set (Date)
37. 15 – 16 November, Sofia ISTACon.org Lead: Stefan Ivanov • Lead Was Initialized (1410) • Status Changed (Available) • Contact Information Changed (stefan@ivanov.com, 03-27243457) • Status Changed (Followup) • Followup Set (23/11/2016) • Contact Information Changed (stefan@ivanov.com, 050-8139904) • Status Changed (Converted)
38. 15 – 16 November, Sofia ISTACon.org public class Lead { public long Id { get; set; } public string Name { get; set; } public string Email { get; set; } public string Phone { get; set; } public LeadStatus Status { get; set; } public DateTime? FollowupOn { get; set; } public void Apply(LeadWasInitialized event) {…} public void Apply(StatusChanged event) {…} public void Apply(NameChanged event) {…} public void Apply(ContactInfoChanged event) {…} public void Apply(FollowupSet event) {…} }
39. 15 – 16 November, Sofia ISTACon.org public class Lead { public long Id { get; set; } public string Name { get; set; } public string Email { get; set; } public string Phone { get; set; } public LeadStatus Status { get; set; } public DateTime? FollowupOn { get; set; } public void Apply(LeadWasInitialized event) {…} public void Apply(StatusChanged event) {…} public void Apply(NameChanged event) {…} public void Apply(ContactInfoChanged event) {…} public void Apply(FollowupSet event) {…} } Apply(StatusChanged event) { this.Status = event.NewStatus; }
40. 15 – 16 November, Sofia ISTACon.org public class Lead { public long Id { get; set; } public string Name { get; set; } public string Email { get; set; } public string Phone { get; set; } public LeadStatus Status { get; set; } public DateTime? FollowupOn { get; set; } public void Apply(LeadWasInitialized event) {…} public void Apply(StatusChanged event) {…} public void Apply(NameChanged event) {…} public void Apply(ContactInfoChanged event) {…} public void Apply(FollowupSet event) {…} } Apply(NameChanged event) { this.Name = event.NewName; }
41. 15 – 16 November, Sofia ISTACon.org public class Lead { public long Id { get; set; } public string Name { get; set; } public string Email { get; set; } public string Phone { get; set; } public LeadStatus Status { get; set; } public DateTime? FollowupOn { get; set; } public void Apply(LeadWasInitialized event) {…} public void Apply(StatusChanged event) {…} public void Apply(NameChanged event) {…} public void Apply(ContactInfoChanged event) {…} public void Apply(FollowupSet event) {…} } Apply(ContactInfoChanged event) { this.Phone = event.NewPhone; this.Email = event.NewEmail; }
42. 15 – 16 November, Sofia ISTACon.org public class Lead { public long Id { get; set; } public string Name { get; set; } public string Email { get; set; } public string Phone { get; set; } public LeadStatus Status { get; set; } public DateTime? FollowupOn { get; set; } public void Apply(LeadWasInitialized event) {…} public void Apply(StatusChanged event) {…} public void Apply(NameChanged event) {…} public void Apply(ContactInfoChanged event) {…} public void Apply(FollowupSet event) {…} } Apply(FollowupSet event) { this.FollowUpOn = event.Date; }
43. 15 – 16 November, Sofia ISTACon.org Lead: Stefan Ivanov Lead Was Initialized (1410) Status Changed (Available) Contact Information Changed (stefan@ivanov.com,03-27243457) Status Changed (Converted) Status Changed (Followup) Followup Set (23/11/2016) Contact Information Changed (stefan@ivanov.com, 050-8139904)
44. 15 – 16 November, Sofia ISTACon.org Name Email Phone Status Follow-up Date Stefan Ivanov stefan@ivanov.com 050-8139904 Converted
45. 15 – 16 November, Sofia ISTACon.org Lead: Stefan Ivanov Lead Was Initialized (1410) Status Changed (Available) Contact Information Changed (stefan@ivanov.com,03-27243457) Status Changed (Converted) Status Changed (Followup) Followup Set (23/11/2016) Contact Information Changed (stefan@ivanov.com, 050-8139904)
46. 15 – 16 November, Sofia ISTACon.org Lead: Stefan Ivanov Lead Was Initialized (1410) Status Changed (Available) Contact Information Changed (stefan@ivanov.com,03-27243457)
47. 15 – 16 November, Sofia ISTACon.org Lead: Stefan Ivanov Lead Was Initialized (1410) Status Changed (Available) Contact Information Changed (stefan@ivanov.com,03-27243457) Status Changed (Followup) Followup Set (23/11/2016) Contact Information Changed (stefan@ivanov.com, 050-8139904)
48. 15 – 16 November, Sofia ISTACon.org Lead: Stefan Ivanov Lead Was Initialized (1410) Status Changed (Available) Contact Information Changed (stefan@ivanov.com,03-27243457) Status Changed (Converted) Status Changed (Followup) Followup Set (23/11/2016) Contact Information Changed (stefan@ivanov.com, 050-8139904)
49. 15 – 16 November, Sofia ISTACon.org public class Lead { public long Id { get; set; } public string Name { get; set; } public string Email { get; set; } public string Phone { get; set; } public LeadStatus Status { get; set; } public DateTime? FollowupOn { get; set; } }
50. 15 – 16 November, Sofia ISTACon.org Lead: Stefan Ivanov Lead Was Initialized (1410) Status Changed (Available) Contact Information Changed (stefan@ivanov.com,03-27243457) Status Changed (Converted) Status Changed (Followup) Followup Set (23/11/2016) Contact Information Changed (stefan@ivanov.com, 050-8139904)
51. 15 – 16 November, Sofia ISTACon.org public class LeadSearch { public long Id { get; private set; } … public List<string> Emails; public List<string> Phones; … public void Apply(ContactInfoChanged event) {…} … } Apply(ContactInfoChanged event) { this.Phones.Append(event.NewPhone); this.Emails.Append(event.NewEmail); }
52. 15 – 16 November, Sofia ISTACon.org public class LeadStatusChangesModel { public long Id { get; set; } public List<StatusLog> StatusChangesLog; … public void Apply(StatusChanged event) {…} … } Apply(StatusChanged event) { this.StatusChangesLog.Append( new StatusLog(event.NewStatus, DateTime.Now); );
53. 15 – 16 November, Sofia ISTACon.org public class LeadStatusChangesModel { public long Id { get; set; } public List<StatusLog> StatusChangesLog; … } public class Lead { public long Id { get; set; } public string Name { get; set; } public string Email { get; set; } public string Phone { get; set; } public LeadStatus Status { get; set; } public DateTime? FollowupOn { get; set; } … } public class LeadSearch { public long Id { get; private set; } public List<string> Emails; public List<string> Phones; … }
54. 15 – 16 November, Sofia ISTACon.org Lead: Stefan Ivanov Lead Was Initialized (1410) Status Changed (Available) Contact Information Changed (stefan@ivanov.com,03-27243457) Status Changed (Converted) Status Changed (Followup) Followup Set (23/11/2016) Contact Information Changed (stefan@ivanov.com, 050-8139904)
55. 15 – 16 November, Sofia ISTACon.org Events = Source of Truth Event Sourcing
56. 15 – 16 November, Sofia ISTACon.org
57. 15 – 16 November, Sofia ISTACon.org
58. 15 – 16 November, Sofia ISTACon.org
59. 15 – 16 November, Sofia ISTACon.org Storage: Event Store
60. 15 – 16 November, Sofia ISTACon.org Event Store API: The Good News Append(Entity Id + New events) GetEvents(Entity Id) Event1,
 Event2,
 Event3, ….
61. 15 – 16 November, Sofia ISTACon.org Key (Entity Id) Value (Events list) Lead #1410 1. LeadWasInitialized(1410) 2. StatusChanged(Available) 3. NameChanged(“Martin Mateev”) 4. StatusChanged(Converted) Lead #1406 1. LeadWasInitialized(1406) 2. StatusChanged(Available’’’) 3. NameChanged(“Lilyana Yankov”)
62. 15 – 16 November, Sofia ISTACon.org Event Store API: The Good News Append(Entity Id + New events) GetEvents(Entity Id) Event1,
 Event2,
 Event3, ….
63. 15 – 16 November, Sofia ISTACon.org Event Store API: The Bad News Append(Entity Id + New events) GetEvents(Entity Id) Event1,
 Event2,
 Event3, ….
64. 15 – 16 November, Sofia ISTACon.org CQRS Command Query Responsibility Segregation
65. 15 – 16 November, Sofia ISTACon.org CQRS • Command - write data • Query - read data • A use case can be either Command or Query. Never both
66. 15 – 16 November, Sofia ISTACon.org Queries Read Model Read DB Commands Write Model Writes DB Projection engine
67. 15 – 16 November, Sofia ISTACon.org Event Sourced Model (Write) Lead #1410 1. LeadWasInitialized(1410) 2. StatusChanged(Available) 3. NameChanged(“Martin Mateev”) 4. StatusChanged(Closed) … 1000. StatusChanged(FollowUp) 1001. FollowupSet(16/11/2017) Read Model Id 1410 Name Martin Mateev Status Followup FollowupOn (Empty) Email … Phone …
68. 15 – 16 November, Sofia ISTACon.org Queries Read Model Read DB Commands Write Model Writes DB Projection engine
69. 15 – 16 November, Sofia ISTACon.org Event Sourced Model (Write) Lead #1410 1. LeadWasInitialized(1410) 2. StatusChanged(Available) 3. NameChanged(“Martin Mateev”) 4. StatusChanged(Closed) 5. StatusChanged(Available) 6. NameChanged(“Viktor Rumenov”) Read Model Id 1410 Phone … Status Available FollowupOn (Empty) Email … Name Martin Mateev
70. 15 – 16 November, Sofia ISTACon.org public class Lead { public long Id { get; set; } public string Name { get; set; } public string Email { get; set; } public string Phone { get; set; } public LeadStatus Status { get; set; } public DateTime? FollowupOn { get; set; } public void Apply(LeadWasInitialized event) {…} public void Apply(StatusChanged event) {…} public void Apply(NameChanged event) {…} public void Apply(ContactInfoChanged event) {…} public void Apply(FollowupSet event) {…} } Apply(NameChanged event) { this.Name = event.NewName; }
71. 15 – 16 November, Sofia ISTACon.org Queries Read Model Read DB Commands Event Sourcing Event Store Projection engine
72. 15 – 16 November, Sofia ISTACon.org Queries Read Model Read DB Commands Write Model Writes DB Projection engine Projection • RDBMS • Documents • Graphs • Files
73. 15 – 16 November, Sofia ISTACon.org Queries Read Model Commands Write Model Writes DB Projection engine Projection • RDBMS • Documents • Graphs • Files • Multiple DBs Read DBs
74. 15 – 16 November, Sofia ISTACon.org Queries Read Model Commands Write Model Writes DB Projection engine Read DBs Concurrency • Pessimistic • Optimistic • Optimistic on steroids
75. 15 – 16 November, Sofia ISTACon.org SetFollowupCommand + StatusChangeEvent => Collision ChangeNameCommand + StatusChangeEvent => OK
76. 15 – 16 November, Sofia ISTACon.org Event Sourcing + CQRS • Flexible business domain modeling • Insane scalability and availability • Rock solid infrastructure • …look great on your C.V.
77. 15 – 16 November, Sofia ISTACon.org
78. 15 – 16 November, Sofia ISTACon.org CQRS: When? Event Sourcing ‎
    CQRS Non-functional benefits
79. 15 – 16 November, Sofia ISTACon.org Event Sourcing CQRS
80. 15 – 16 November, Sofia ISTACon.org Event Sourcing: When? ?
81. 15 – 16 November, Sofia ISTACon.org Subdomains Generic, Supporting, Core
82. 15 – 16 November, Sofia ISTACon.org
83. 15 – 16 November, Sofia ISTACon.org Subdomains Generic, Supporting, Core
84. 15 – 16 November, Sofia ISTACon.org CREATIVE CATALOG CAMPAIGN PUBLISHING BILLING USERS MANAGEMENT LEAD MANAGEMENT COMMISSIONS CALCULATION DESK MANAGEMENT VOIP MANAGEMENT EMAIL CAMPAIGNS
85. 15 – 16 November, Sofia ISTACon.org Generic Subdomains
86. 15 – 16 November, Sofia ISTACon.org CREATIVE CATALOG CAMPAIGN PUBLISHING BILLING USERS MANAGEMENT LEAD MANAGEMENT COMMISSIONS CALCULATION DESK MANAGEMENT VOIP MANAGEMENT EMAIL CAMPAIGN
87. 15 – 16 November, Sofia ISTACon.org CREATIVE CATALOG CAMPAIGN PUBLISHING BILLING USERS MANAGEMENT LEAD MANAGEMENT COMMISSIONS CALCULATION DESK MANAGEMENT VOIP MANAGEMENT EMAIL CAMPAIGN
88. 15 – 16 November, Sofia ISTACon.org Supporting Subdomains
89. 15 – 16 November, Sofia ISTACon.org CREATIVE CATALOG CAMPAIGN PUBLISHING BILLING USERS MANAGEMENT LEAD MANAGEMENT COMMISSIONS CALCULATION DESK MANAGEMENT VOIP MANAGEMENT EMAIL CAMPAIGN
90. 15 – 16 November, Sofia ISTACon.org CREATIVE CATALOG CAMPAIGN PUBLISHING BILLING USERS MANAGEMENT LEAD MANAGEMENT COMMISSIONS CALCULATION DESK MANAGEMENT VOIP MANAGEMENT EMAIL CAMPAIGN
91. 15 – 16 November, Sofia ISTACon.org Core Domains
92. 15 – 16 November, Sofia ISTACon.org CREATIVE CATALOG CAMPAIGN PUBLISHING BILLING USERS MANAGEMENT LEAD MANAGEMENT COMMISSIONS CALCULATION DESK MANAGEMENT VOIP MANAGEMENT EMAIL CAMPAIGN
93. 15 – 16 November, Sofia ISTACon.org Event Sourcing: When? Core business domains
94. 15 – 16 November, Sofia ISTACon.org Before you try this at home
95. 15 – 16 November, Sofia ISTACon.org Available Event Stores • http://GetEventStore.com • NEventStore • Akka Persistence • Elastic Event Store (Coming soon)
96. 15 – 16 November, Sofia ISTACon.org
97. 15 – 16 November, Sofia ISTACon.org €100 Discount for attendees of ISTA 2016 https://ti.to/webengineers/ddd17/discount/istacon2016
98. 15 – 16 November, Sofia ISTACon.org Domain-Driven Design • How to identify subdomains? • How to define business entities? • How do define events? • How to talk to business experts?
99. 15 – 16 November, Sofia ISTACon.org
100. 15 – 16 November, Sofia ISTACon.org Thank you! @vladikk vladikk.com vladikkhononov vladik@khononov.com