Tweet
Tweet

Slide 1

Slide 1 text

Copyright Nasa Goddard Michiel Overeem @michielovereem

Slide 2

Slide 2 text

AFAS Software 400+ employees 10.000+ organizations

Slide 3

Slide 3 text

AFAS Profit HRM, CRM, finance, order management, project management

Slide 4

Slide 4 text

Model-Driven Generate a customized application with standardized software

Slide 5

Slide 5 text

Management Zero downtime upgrades

Slide 6

Slide 6 text

daptable odel-based and ser-specific oftware cosystems A M U S E

Slide 7

Slide 7 text

Copyright Nasa Goddard

Slide 8

Slide 8 text

How can an event sourced system be upgraded efficiently when the (implicit) event schema changes?

Slide 9

Slide 9 text

Sources

Slide 10

Slide 10 text

Structure of an Event Store Store A (distributed) collection of streams. An ordered list of events Stream An ordered list of events. Event A name, attributes with values, and metadata.

Slide 11

Slide 11 text

The schema is implicit and hidden! Which streams are known in the system? Which events are allowed in which streams? Which attributes are allowed in which events?

Slide 12

Slide 12 text

Webshop EventStore Customer Stream ShoppingCart Stream CustomerSignedUp { … } UpgradedCustomerToPremiumService { … } ShoppingCartCreated { … } ArticleRemovedFromChart { … } ArticleAddedToChart { … }

Slide 13

Slide 13 text

Change will happen “An software system written to perform some real-world activity must be continually adapted or it becomes progressively less satisfactory” Lehman’s law of continuing change

Slide 14

Slide 14 text

Add attribute to event Customer Stream CustomerSignedUp { … } UpgradedCustomerToPremiumService { … }

Slide 15

Slide 15 text

Add attribute to event Customer Stream CustomerSignedUp { …, newsletter: } UpgradedCustomerToPremiumService { … }

Slide 16

Slide 16 text

Customer Stream CustomerSignedUp { … } UpgradedCustomerToPremiumService { … } Add event to stream

Slide 17

Slide 17 text

Customer Stream CustomerSignedUp { … } UpgradedCustomerToPremiumService { … } OptInOnNewsLetter { … } Add event to stream

Slide 18

Slide 18 text

Newsletter Stream CreateNewsletter { … } PublishNewsletter { … } Add stream to store

Slide 19

Slide 19 text

Customer Stream CustomerSignedUp { …, firstName: , surName: } UpgradedCustomerToPremiumService { … } CustomerSignedUp { …, fullName: } Split attribute

Slide 20

Slide 20 text

Customer Stream CustomerSignedUp { …, firstName: , surName: } UpgradedCustomerToPremiumService { … } Split attribute

Slide 21

Slide 21 text

Customer Stream CustomerSignedUp { … } UpgradedCustomerToPremiumService { … } CustomerSignedUp { …, newsletter: } Split events

Slide 22

Slide 22 text

Customer Stream CustomerSignedUp { … } UpgradedCustomerToPremiumService { … } OptInOnNewsLetter { … } Split events

Slide 23

Slide 23 text

Customer Stream CustomerSignedUp { … } UpgradedCustomerToPremiumService { … } OptInOnNewsLetter { … } CustomerSignedUp { … } UpgradedCustomerToPremiumService { … } Split streams

Slide 24

Slide 24 text

CustomerSignedUp { … } UpgradedCustomerToPremiumService { … } Customer Stream CustomerSignedUp { … } UpgradedCustomerToPremiumService { … } NewsletterSubscription Stream OptInOnNewsLetter { … } Split streams

Slide 25

Slide 25 text

Store level operations Add / remove streams Merge streams / split / … streams Stream level operations Add / remove events Merge events / split / … events Event level operations Add / update / remove attributes Merge attributes / split / … attributes The operations

Slide 26

Slide 26 text

How can we execute these operations?

Slide 27

Slide 27 text

Always extend Customer Stream CustomerSignedUp { … } UpgradedCustomerToPremiumService { … } CustomerSignedUp_V2 { … }

Slide 28

Slide 28 text

Our codebase would only grow. Blue Cross Blue Shield Tower under construction in Chicago, by Photogonal, wikipedia.org

Slide 29

Slide 29 text

CustomerSignedUp { “customerId”: …, “fullName”: … } Weak schema & serialization CustomerSignedUp { “customerId”: …, “firstName”: …, “surName”: … }

Slide 30

Slide 30 text

CustomerSignedUp { “customerId”: …, … } Weak schema & serialization

Slide 31

Slide 31 text

If-then-else explosion? pexels.com

Slide 32

Slide 32 text

Upcasting

Slide 33

Slide 33 text

Upcasters Upcasting

Slide 34

Slide 34 text

CustomerSignedUp { “customerId”: …, “fullName”: … } Upcasting CustomerSignedUp { “customerId”: …, “firstName”: …, “surName”: … } CustomerSignedUp { “customerId”: …, “firstName”: …, “surName”: …, “newsletter”: … } Upcaster Upcaster

Slide 35

Slide 35 text

Upcasters Upcasting

Slide 36

Slide 36 text

Runtime overhead? pexels.com

Slide 37

Slide 37 text

Think about the order of upgrading components!

Slide 38

Slide 38 text

No content

Slide 39

Slide 39 text

No content

Slide 40

Slide 40 text

No content

Slide 41

Slide 41 text

No content

Slide 42

Slide 42 text

No content

Slide 43

Slide 43 text

Do we need immutability? Copyright Stephen Cairns

Slide 44

Slide 44 text

Data Information

Slide 45

Slide 45 text

Upcasters Lazy transformation

Slide 46

Slide 46 text

Upcasters Lazy transformation

Slide 47

Slide 47 text

When will this be done?

Slide 48

Slide 48 text

In place transformation

Slide 49

Slide 49 text

In place transformation

Slide 50

Slide 50 text

Not for the faint of heart?

Slide 51

Slide 51 text

Copy and transform

Slide 52

Slide 52 text

Copy and transform

Slide 53

Slide 53 text

Copy and transform

Slide 54

Slide 54 text

Slow?

Slide 55

Slide 55 text

Store A (distributed) collection of streams. An ordered list of events Store level Stream level Event level Add / remove streams Merge streams / split / … streams Should we perform all operations at runtime?

Slide 56

Slide 56 text

Let’s put all these operations together!

Slide 57

Slide 57 text

Event operations Stream operations Store operations simple Store operations complex Always extend Weak schema Upcasting Lazy transformation In place transformation Copy and transform

Slide 58

Slide 58 text

Always extend Weak schema Upcasting Lazy transformation In place transformation Copy and transformation Functional suitability +/- - +/- +/- + +

Slide 59

Slide 59 text

Always extend Weak schema Upcasting Lazy transformation In place transformation Copy and transformation Maintainability - +/- +/- +/- + +

Slide 60

Slide 60 text

Always extend Weak schema Upcasting Lazy transformation In place transformation Copy and transformation Performance efficiency + + + +/- +/- -

Slide 61

Slide 61 text

Always extend Weak schema Upcasting Lazy transformation In place transformation Copy and transformation Reliability + + + - - +

Slide 62

Slide 62 text

Event operations Stream operations Store operations simple Store operations complex Always extend Weak schema Upcasting Lazy transformation In place transformation Copy and transform

Slide 63

Slide 63 text

Can we execute this upgrade with zero downtime? Copyright SpaceX

Slide 64

Slide 64 text

Event operations Stream operations Store operations simple Store operations complex Always extend Weak schema Upcasting Lazy transformation In place transformation Copy and transform These techniques are executed within the application These techniques need a separate stage

Slide 65

Slide 65 text

Application deployments strategies

Slide 66

Slide 66 text

Big Flip

Slide 67

Slide 67 text

Big Flip

Slide 68

Slide 68 text

Big Flip

Slide 69

Slide 69 text

Big Flip

Slide 70

Slide 70 text

Big Flip

Slide 71

Slide 71 text

Big Flip

Slide 72

Slide 72 text

Big Flip Blue / Green

Slide 73

Slide 73 text

Big Flip Blue / Green

Slide 74

Slide 74 text

Big Flip Blue / Green

Slide 75

Slide 75 text

Big Flip Blue / Green

Slide 76

Slide 76 text

Big Flip Blue / Green

Slide 77

Slide 77 text

Big Flip Blue / Green Rolling Upgrade

Slide 78

Slide 78 text

Big Flip Blue / Green Rolling Upgrade

Slide 79

Slide 79 text

Big Flip Blue / Green Rolling Upgrade

Slide 80

Slide 80 text

Event operations Stream operations Store operations simple Store operations complex Always extend Weak schema Upcasting Lazy transformation In place transformation Copy and transform Big Flip Blue / Green Rolling Upgrade

Slide 81

Slide 81 text

Database deployments strategies

Slide 82

Slide 82 text

Blue / Green

Slide 83

Slide 83 text

Blue / Green

Slide 84

Slide 84 text

Blue / Green

Slide 85

Slide 85 text

Blue / Green

Slide 86

Slide 86 text

Blue / Green Expand & Contract

Slide 87

Slide 87 text

Blue / Green Expand & Contract

Slide 88

Slide 88 text

Blue / Green Expand & Contract

Slide 89

Slide 89 text

Event operations Stream operations Store operations simple Store operations complex Always extend Weak schema Upcasting Lazy transformation In place transformation Copy and transform Big Flip Blue / Green Rolling Upgrade Expand & Contract Blue / Green

Slide 90

Slide 90 text

Event operations Stream operations Store operations simple Store operations complex In place transformation Copy and transform Big Flip Blue / Green Rolling Upgrade Expand & Contract Blue / Green

Slide 91

Slide 91 text

Event operations Stream operations Store operations simple Store operations complex In place transformation Copy and transform Big Flip Blue / Green Rolling Upgrade Expand & Contract Blue / Green

Slide 92

Slide 92 text

Event operations Stream operations Store operations simple Store operations complex Always extend Weak schema Upcasting Lazy transformation In place transformation Copy and transform Big Flip Blue / Green Rolling Upgrade Expand & Contract Blue / Green Mix & Match!

Slide 93

Slide 93 text

What about the projections?

Slide 94

Slide 94 text

No content

Slide 95

Slide 95 text

How to keep your clients in sync?

Slide 96

Slide 96 text

It depends. Your context. Your solution.

Slide 97

Slide 97 text

Throw away with a reason It is an option!

Slide 98

Slide 98 text

Time can be of help Expirations?

Slide 99

Slide 99 text

You might not be Google Small enough to replay?

Slide 100

Slide 100 text

Domain design helps Design and test before production

Slide 101

Slide 101 text

Read our paper for more en And more is coming!

Slide 102

Slide 102 text

Recommended!

Slide 103

Slide 103 text

Copyright Nasa Goddard Michiel Overeem @michielovereem