TechDays 2017 - Event Sourcing after launch how to evolve your event store along with your application

Event sourcing can be a solution for certain problems in your application, but it also comes with its own set of challenges.
One of those is the evolution of your event schema. How should you handle changes to events?
We found many approaches while doing our research (we’ve published earlier results in our paper “The Dark Side of Event Sourcing”) and we gained a lot of experienced from building a full-size ERP application using CQRS and event sourcing.
In this talk we will show what kind of approaches we’ve found and what we use to deal with our specific challenges. We will also share early research results from our interviews with over 20 different companies using event sourcing.
Every request for advice can be answered with ‘it depends’, and this talk will not solve that. But we will tell you what your options are, what we think works, and what many others have done.

Transcript

1. michiel.overeem@afas.nl
   @michielovereem
   

   View Slide

2. 400+ employees (4 locations)
   10.500 customers (companies)
   AFAS Software
   

   View Slide

3. HRM, CRM, finance, order
   management, project
   management, workflow, ...
   AFAS Profit
   

   View Slide

4. View Slide

5. accountId balance owner
   1234567 0 Michiel
   Current state Event sourcing
   Event
   BankAccountCreated
   (accountId: 1234567, balance: 0, owner: Michiel)
   

   View Slide

6. accountId balance owner
   1234567 100 Michiel
   Current state Event sourcing
   Event
   BankAccountCreated
   (accountId: 1234567, balance: 0, owner: Michiel)
   DepositPerformed
   (accountId : 1234567, amount: 100, balance: 100)
   

   View Slide

7. Event
   BankAccountCreated
   (accountId: 1234567, balance: 0, owner: Michiel)
   DepositPerformed
   (accountId : 1234567, amount: 100, balance: 100)
   WithdrawalPerformed
   (accountId: 1234567, amount: 50, balance: 50)
   accountId balance owner
   1234567 50 Michiel
   Current state Event sourcing
   

   View Slide

8. Event
   BankAccountCreated
   (accountId: 1234567, balance: 0, owner: Michiel)
   DepositPerformed
   (accountId : 1234567, amount: 100, balance: 100)
   WithdrawalPerformed
   (accountId: 1234567, amount: 50, balance: 50)
   OwnerChanged
   (accountId: 1234567, newOwner: Marten)
   accountId balance owner
   1234567 50 Marten
   Current state Event sourcing
   

   View Slide

9. View Slide

10. Event
    ShoppingCartCreated
    (cartid: 1234567, user: Michiel)
    cartid articles user
    1234567 0 Michiel
    Current state Event sourcing
    

    View Slide

11. Event
    ShoppingCartCreated
    (cartid: 1234567, user: Michiel)
    ArticleAddedToCart(cartid: 1234567, article: Lumia
    950)
    cartid articles user
    1234567 1 Michiel
    Current state Event sourcing
    cartid article
    1234567 Lumia 950
    

    View Slide

12. cartid articles user
    1234567 0 Michiel
    Current state Event sourcing
    cartid article
    Event
    ShoppingCartCreated
    (cartid: 1234567, user: Michiel)
    ArticleAddedToCart(cartid: 1234567, article: Lumia
    950)
    ArticleRemovedFromCart(cartid: 1234567, article:
    Lumia 950)
    

    View Slide

13. cartid articles user
    1234567 1 Michiel
    Current state Event sourcing
    cartid article
    1234567 iPhone X
    Event
    ShoppingCartCreated
    (cartid: 1234567, user: Michiel)
    ArticleAddedToCart(cartid: 1234567, article: Lumia
    950)
    ArticleRemovedFromCart(cartid: 1234567, article:
    Lumia 950)
    ArticleAddedToCart(cartid: 1234567, article: iPhone
    X)
    

    View Slide

14. cartid articles user
    1234567 1 Michiel
    Current state Event sourcing
    cartid article
    1234567 iPhone X
    Event
    ShoppingCartCreated
    (cartid: 1234567, user: Michiel)
    ArticleAddedToCart(cartid: 1234567, article: Lumia
    950)
    ArticleRemovedFromCart(cartid: 1234567, article:
    Lumia 950)
    ArticleAddedToCart(cartid: 1234567, article: iPhone
    X)
    

    View Slide

15. View Slide

16. View Slide

17. View Slide

18. View Slide

19. View Slide

20. Copyright Nasa Goddard
    But there is a
    dark side…
    How can we
    handle changes?
    What is the best way
    for our context?
    

    View Slide

21. A stream of events
    

    View Slide

22. View Slide

23. Multiple streams
    

    View Slide

24. {
    “ArticleAddedToCart”: {
    “cartid”: 1234567,
    “article”: “Lumia 950”
    }
    }
    

    View Slide

25. View Slide

26. Webshop EventStore
    Customer Stream
    ShoppingCart Stream
    CustomerSignedUp { … }
    PriviligeUpgraded { … }
    ShoppingCartCreated { … }
    ChangedQuantityOfArticle { … }
    ArticleAddedToChart { … }
    

    View Slide

27. View Slide

28. Customer Stream
    CustomerSignedUp { …, newsletter: }
    PriviligeUpgraded { … }
    

    View Slide

29. Customer Stream
    CustomerSignedUp { … }
    PriviligeUpgraded { … }
    OptInOnNewsLetter { … }
    

    View Slide

30. Newsletter Stream
    CreateNewsletter { … }
    PublishNewsletter { … }
    

    View Slide

31. Customer Stream
    CustomerSignedUp { …, firstName: ,
    surName: }
    PriviligeUpgraded { … }
    CustomerSignedUp { …, fullName: }
    

    View Slide

32. Customer Stream
    CustomerSignedUp { … }
    PriviligeUpgraded { … }
    OptInOnNewsLetter { … }
    CustomerSignedUp { …, newsletter: }
    

    View Slide

33. Customer Stream
    CustomerSignedUp { … }
    PriviligeUpgraded { … }
    OptInOnNewsLetter { … }
    Customer Stream
    CustomerSignedUp { … }
    PriviligeUpgraded { … }
    NewsletterSubscription Stream
    OptInOnNewsLetter { … }
    

    View Slide

34. Store level operations
    Add / remove streams
    Merge streams / split / … streams
    Stream level operations
    Add / remove events
    Merge events / split / … events
    Event level operations
    Add / update / remove attributes
    Merge attributes / split / … attributes
    

    View Slide

35. View Slide

36. Copyright Stephen Cairns
    

    View Slide

37. Client
    

    View Slide

38. Client
    

    View Slide

39. Client
    

    View Slide

40. View Slide

41. Customer Stream
    CustomerSignedUp { … }
    PriviligeUpgraded { … }
    CustomerSignedUpV2 { …, newsletter:
    }
    

    View Slide

42. Customer Stream
    CustomerSignedUp { … }
    PriviligeUpgraded { … }
    

    View Slide

43. Client
    Upcaster
    

    View Slide

44. Client
    Upcaster
    

    View Slide

45. Updater
    Not for the faint of heart!
    

    View Slide

46. Updater
    

    View Slide

47. Updater
    

    View Slide

48. Multiple versions
    Weak schema
    Upcasters
    Lazy transformation
    In place transformation
    Copy and transformation
    Functional
    suitability
    +/-
    -
    +/-
    +/-
    +
    +
    

    View Slide

49. Multiple versions
    Weak schema
    Upcasters
    Lazy transformation
    In place transformation
    Copy and transformation
    Maintainability
    -
    +/-
    +/-
    +/-
    +
    +
    

    View Slide

50. Multiple versions
    Weak schema
    Upcasters
    Lazy transformation
    In place transformation
    Copy and transformation
    Performance
    efficiency
    +
    +
    +
    +/-
    +/-
    -
    

    View Slide

51. Multiple versions
    Weak schema
    Upcasters
    Lazy transformation
    In place transformation
    Copy and transformation
    Reliability
    +
    +
    +
    -
    -
    +
    

    View Slide

52. View Slide

53. View Slide

54. View Slide

55. View Slide

56. View Slide

57. View Slide

58. View Slide

59. View Slide

60. View Slide

61. michiel.overeem@afas.nl
    @michielovereem
    https://dev.afas.nl
    

    View Slide