Event Sourcing Workshop @ Software Architecture Summit 2016

Unter Event Sourcing versteht man einen Architekturstil in dem Änderungen am Zustand der verwalteten Daten als eine Sequenz von Events festgehalten werden. Durch diese Herangehensweise können jederzeit Snapshots des Datenzustands erstellt und abgefragt werden. Des Weiteren ermöglicht uns das Persistieren von Events eine Optimierung der lesenden Zugriffe durch eine denormalisierung des „Lese-Modells“ der Daten. Letzerer Aspekt ist aktuell insbesondere durch den Architekturansatz CQRS in aller Munde. Im Rahmen des Workshops wird Michael Plöd eine Einführung in das Thema Event Sourcing geben und gängige Patterns rund um das Thema vorstellen. Des Weiteren werden Themen wie Rollback von Events (Kompensation-Services), Möglichkeiten der Erstellung von Daten Snapshots und Änderungen durch externe Systeme vorgestellt. Der Workshop wird neben Schaubildern zahlreiche praktische Live-Coding Bestandteile (Java / Groovy / Spring) beinhalten.

Transcript

1. Workshop Event Sourcing
   Michael Plöd | innoQ
   

   View Slide

2. Michael Plöd
   Principal Consultant bei innoQ
   @bitboss
   

   View Slide

3. IncidentSOAPEndpoint
   IncidentBusinessService
   IncidentDAO
   Incident
   Business
   Model
   Client
   Incident

   DTO
   Incident

   View

   Model
   RDBMS
   Incident

   ER-Model
   Network
   Network
   Let’s review the
   classical old
   school N-Tier
   architecture
   

   View Slide

4. Characteristics
   

   View Slide

5. 1
   We read and write data
   through the same layers
   IncidentSOAPEndpoint
   IncidentBusinessService
   IncidentDAO
   Incident
   Business Model
   Client
   Incident

   DTO
   Incident

   View

   Model
   RDBMS
   Incident

   ER-Model
   Network
   Network
   READ
   WRITE
   

   View Slide

6. 2
   We use the same model

   for read and write
   access
   IncidentSOAPEndpoint
   IncidentBusinessService
   IncidentDAO
   Incident
   Business Model
   Client
   Incident

   DTO
   Incident

   View

   Model
   RDBMS
   Incident

   ER-Model
   Network
   Network
   

   View Slide

7. 3
   We use coarse grained
   deployment units that
   combine read and write
   code
   IncidentSOAPEndpoint
   IncidentBusinessService
   IncidentDAO
   Client
   RDBMS
   

   View Slide

8. 4
   We change datasets
   directly
   IncidentRestController
   IncidentBusinessService
   IncidentDAO
   Incident
   ID USER_ID DATE TEXT
   1 23423 11.03.2014 Mouse is broken
   2 67454 12.03.2014 EMail not working
   3 93729 12.03.2014 Office license
   … … … …
   

   View Slide

9. We’ve done that
   for ages already and
   it’s proven
   

   View Slide

10. Many applications will
    run smooth and fine
    with this kind of
    approach
    

    View Slide

11. However there are
    drawbacks to this kind
    of architecture
    

    View Slide

12. 1 The data model is a compromise
    2 You can’t scale read and write independently
    3 No data history, no snapshots, no replay
    4 Tendency to a monolithic approach
    

    View Slide

13. Event Sourcing is an
    architectural pattern in
    which the state of the
    application is being
    determined by a
    sequence of events
    

    View Slide

14. Building Blocks
    Applications
    Event
    Queue
    Applications issues
    events to a queue
    Event
    Handler
    Event
    Store
    Events are stored in
    the event store
    The Event handler is
    processing the events
    

    View Slide

15. The sequence of events in the queue
    is called event stream
    t
    now
    Event
    Event
    Event
    Event
    Event
    

    View Slide

16. IncidentCreatedEvent

    

    incidentNumber: 1

    userNumber: 23423
    timestamp: 11.03.2014 12:23:23

    text: „Mouse broken“
    status: „open“
    Event Stream Example
    IncidentTextChangedEvent

    

    incidentNumber: 1

    text: „Left button of mouse broken“
    IncidentClosedEvent

    

    incidentNumber: 1

    solution: „Mouse replaced“
    status: „closed“
    

    View Slide

17. An event is something 

    that happened in the past
    t
    now
    Event
    Event
    Event
    Event
    Event
    

    View Slide

18. The names of the events are part
    of the

    Ubiquitous Language
    D D D
    

    View Slide

19. ShipmentDeliveredEvent

    CustomerVerifiedEvent
    CartCheckedOutEvent
    CreateCustomerEvent

    WillSaveItemEvent

    DoStuffEvent
    

    View Slide

20. Code Example
    public class CustomerVerifiedEvent {
    private String eventId;
    private Date eventDate;
    private CustomerNumber customerNumber;
    private String comment;
    public CustomerVerifiedEvent(CustomerNumber custNum, 

    String comment) {

    this.customerNumber = cusNum;
    this.comment = comment;
    this.eventDate = new Date();
    }
    }
    

    View Slide

21. Scope your events based on

    Aggregates
    D D D
    

    View Slide

22. An Event is always immutable
    !
    

    View Slide

23. There is no deletion of events
    !
    

    View Slide

24. A delete is
    just another
    event
    IncidentCreatedEvent

    

    incidentNumber: 1

    userNumber: 23423
    timestamp: 11.03.2014 12:23:23

    text: „Mouse is broken defekt“
    status: „open“
    IncidentChangedEvent

    

    incidentNumber: 1

    text: „Maus ist Kaputt“
    IncidentRemovedEvent

    

    incidentNumber: 1

    

    View Slide

25. The event bus is usually
    implemented by a message
    broker
    t
    now
    Event
    Event
    Event
    Event
    Event
    

    View Slide

26. Let’s reuse the ESB
    from the failed SOA
    project
    

    View Slide

27. NO
    

    View Slide

28. NO
    

    View Slide

29. NO
    

    View Slide

30. !
    Prefer dumb pipes
    with smart
    endpoints as a
    suitable message
    broker architecture
    

    View Slide

31. 1 Complete rebuild is possible
    2 Temporal Queries
    3 Event Replay
    

    View Slide

32. Well known examples
    =

    Version Control Systems

    or
    Database Transaction Logs
    

    View Slide

33. The Event
    Store has a
    very high
    business value
    

    View Slide

34. Aren’t there performance
    issues attached to this kind of
    data storage?
    

    View Slide

35. YES!
    

    View Slide

36. Application
    State
    Think about application state
    Application
    Event
    Queue
    Event
    Handler
    Event
    Store
    Application
    State
    The application queries the
    pre-processed application
    state
    

    View Slide

37. CQRS
    

    View Slide

38. Command
    Query
    Responsibility
    Separation
    

    View Slide

39. IncidentSOAPEndpoint
    IncidentBusinessService
    IncidentDAO
    Incident
    Business
    Model
    Client
    Incident

    DTO
    Incident

    View

    Model
    RDBMS
    Incident

    ER-Model
    Network
    Network
    

    View Slide

40. IncidentQueryEndpoint
    IncidentQueryService
    IncidentQueryDAO
    RDBMS
    Network
    IncidentCommandEndpoint
    IncidentCommandService
    IncidentCommandDAO
    Basically the idea behind CQRS is simple
    

    View Slide

41. Code Example
    

    View Slide

42. Classic Interface
    public interface IncidentManagementService {
    Incident saveIncident(Incident i);
    void updateIncident(Incident i);
    List retrieveBySeverity(Severity s);
    Incident retriveById(Long id);
    }
    

    View Slide

43. CQRS-ified Interfaces
    public interface IncidentManagementQueryService {
    List retrieveBySeverity(Severity s);
    Incident retriveById(Long id);
    }
    public interface IncidentManagementCommandService {
    Incident saveIncident(Incident i);
    void updateIncident(Incident i);
    }
    

    View Slide

44. Event Store
    EventHandler Events
    Events
    Event Sourcing & CQRS
    IncidentCommandEndpoint
    IncidentCommandService
    IncidentCommandDAO
    IncidentQueryEndpoint
    IncidentQueryService
    IncidentQueryDAO
    Read Storage
    Events
    READ
    

    View Slide

45. 1 Individual scalability and deployment
    options
    2 Technological freedom of choice for
    command, query and event handler
    code
    3 Excellent Fit for Bounded Context
    (Domain Driven Design)
    

    View Slide

46. Event Sourcing and CQRS
    are interesting architectural
    options. However there are
    various challanges, that have
    to be taken care of
    

    View Slide

47. 1 Consistency
    2 Validation
    3 Parallel Updates
    

    View Slide

48. YES
    

    View Slide

49. !Systems based on
    CQRS and Event
    Sourcing are
    mostly eventually
    consistent
    

    View Slide

50. Event Store
    EventHandler Events
    Events
    Eventual Consistency
    IncidentCommandEndpoint
    IncidentCommandService
    IncidentCommandDAO
    IncidentQueryEndpoint
    IncidentQueryService
    IncidentQueryDAO
    Read Storage
    Events
    READ
    

    View Slide

51. BUT
    

    View Slide

52. ! You can build a 

    fully consistent
    system which
    follows Event
    Sourcing
    principles
    

    View Slide

53. EventHandler
    Events
    Events
    Full Consistency
    IncidentCommandEndpoint
    IncidentCommandService
    IncidentCommandDAO
    IncidentQueryEndpoint
    IncidentQueryService
    IncidentQueryDAO
    Read Storage
    Events
    Select * from
    Event Store
    

    View Slide

54. Your business domain drives the level
    of consistency not technology

    Deeper Insight
    D D D
    

    View Slide

55. Increased (but still eventual)
    consistency
    EventHandler
    Events
    Events
    IncidentCommandEndpoint
    IncidentCommandService
    IncidentCommandDAO
    IncidentQueryEndpoint
    IncidentQueryService
    IncidentQueryDAO
    Read Storage
    Events
    Select * from
    Event Store
    async
    

    View Slide

56. ! There is no
    standard solution
    

    View Slide

57. 1 Consistency
    2 Validation
    3 Parallel Updates
    

    View Slide

58. Example Domain
    User

    Guid id
    String email
    String password
    RegisterUserCommand ChangeEmailCommand
    UserRegisteredEvent

    

    Guid id
    Date timestamp
    String email
    String password
    EmailChangedEvent

    

    Guid userId
    Date timestamp
    String email
    

    View Slide

59. We process 2
    million+ registrations per
    day. A user can change her
    email address. However the
    emails address must be
    unique
    

    View Slide

60. ? How high is the
    probability that a
    validation fails
    Which data is required
    for the validation
    Where is the required
    data stored
    

    View Slide

61. $What is the business
    impact of a failed
    validation that is not
    recognized due to
    eventual consistency
    and how high is the
    probability of failure
    

    View Slide

62. Your business domain drives the level
    of consistency

    Deeper Insight
    D D D
    

    View Slide

63. 1 Validate from Event Store
    2 Validate from Read Store
    3 Perform Validation in Event
    Handler
    

    View Slide

64. Never validate
    from the event
    store
    

    View Slide

65. 1 Consistency
    2 Validation
    3 Parallel Updates
    

    View Slide

66. Example Domain
    User

    Guid id
    String email
    String password
    RegisterUserCommand ChangeEmailCommand
    UserRegisteredEvent

    

    Guid id
    Date timestamp
    String email
    String password
    EmailChangedEvent

    

    Guid userId
    Date timestamp
    String email
    

    View Slide

67. What happens when Alice
    and Bob share an account and
    both update the email address at
    the same time
    

    View Slide

68. ?What would we do
    in a
    „classic old school
    architecture“
    

    View Slide

69. UserRestController
    UserBusinessService
    UserDAO
    User
    ID EMAIL PASSWORD
    … … …
    2341 alice_bob@xyz.com lksjdaslkdjas
    … … …
    Update
    Pessimistic or optimistic locking
    

    View Slide

70. Your business domain drives the
    locking quality

    Deeper Insight
    D D D
    

    View Slide

71. ! Pessimistic
    locking on a
    data-level will
    hardly work in
    event sourcing
    architectures
    

    View Slide

72. EventHandler
    Events
    Events
    Where to „pessimistically“ lock?
    Read Storage
    Events
    Event Store
    UserRestController
    UserBusinessService
    UserDAO
    User
    Commands
    Consider a business lock
    with a UserLockedEvent
    

    View Slide

73. ? Do you

    REALLY
    need a full lock
    Most „classic
    architecture“ applications
    are already running fine
    with optimistic locks
    

    View Slide

74. Introduce a version field for the domain
    entity
    User

    Guid id
    Long version
    String email
    String password
    RegisterUserCommand ChangeEmailCommand
    UserRegisteredEvent

    

    Guid id
    Date timestamp
    String email
    String password
    EmailChangedEvent

    

    Guid userId
    Date timestamp
    String email
    Long version
    

    View Slide

75. Each writing event
    increases the version
    UserRegisteredEvent {guid: 12, version: 0,

    email: alicebob@xyz.com, password: werwe2343}
    EmailChangedEvent
    version: 0
    EmailChangedEvent
    version: 1
    EmailChangedEvent
    version: 1
    {guid: 12, version: 1,

    email: alice_bob@xyz.com, password: werwe2343}
    {guid: 12, version: 2,

    email: alice@xyz.com, password: werwe2343}
    EmailChangeFailedEvent
    

    View Slide

76. Also here:

    you should be as
    consistent as
    your domain
    requires
    

    View Slide

77. Thank You!
    Michael Plöd - innoQ
    

    Twitter: @bitboss
    Slides: https://speakerdeck.com/mploed
    

    View Slide