Emergent Distributed Architectures

Slide 1

Slide 1 text

Microservices and Data Pipelines — Sean Braithwaite @brapse · Peter Bourgon @peterbourgon · Velocity London 2017 Emergent Distributed Architectures:

Slide 2

Slide 2 text

Microservices and Data Pipelines — Sean Braithwaite @brapse · Peter Bourgon @peterbourgon · Velocity London 2017 Emergent Distributed Architectures:

Slide 3

Slide 3 text

@brapse, @peterbourgon Emergent Distributed Architectures Architectural decisions are expensive.

Slide 4

Slide 4 text

@brapse, @peterbourgon Emergent Distributed Architectures We want to answer not only how, but why and when.

Slide 5

Slide 5 text

Organization as organism.

Slide 6

Slide 6 text

Cells as parallel actors.

Slide 7

Slide 7 text

Teams as cells.

Slide 8

Slide 8 text

@brapse, @peterbourgon Emergent Distributed Architectures Teams enable parallelism.  Teams must consider global interests to avoid harmful local maxima.

Slide 9

Slide 9 text

@brapse, @peterbourgon Emergent Distributed Architectures

Slide 10

Slide 10 text

@brapse, @peterbourgon Emergent Distributed Architectures

Slide 11

Slide 11 text

@brapse, @peterbourgon Emergent Distributed Architectures Bounded contexts  reduce external coördination costs & make internal work more eﬃcient.

Slide 12

Slide 12 text

@brapse, @peterbourgon Emergent Distributed Architectures

Slide 13

Slide 13 text

@brapse, @peterbourgon Emergent Distributed Architectures

Slide 14

Slide 14 text

@brapse, @peterbourgon Emergent Distributed Architectures The microservices pattern.

Slide 15

Slide 15 text

@brapse, @peterbourgon Emergent Distributed Architectures We can be overeager applying  apparently successful patterns.

Slide 16

Slide 16 text

@brapse, @peterbourgon Emergent Distributed Architectures Microservices cause more problems  than they solve.

Slide 17

Slide 17 text

@brapse, @peterbourgon Emergent Distributed Architectures But microservices codify and reify their teams' responsibilities.

Slide 18

Slide 18 text

@brapse, @peterbourgon Emergent Distributed Architectures Well-deﬁned teams and services maximize product velocity.

Slide 19

Slide 19 text

@brapse, @peterbourgon Emergent Distributed Architectures Microservices actually solve communication problems.

Slide 20

Slide 20 text

@brapse, @peterbourgon Emergent Distributed Architectures Microservices are an immune response to velocity gridlock.

Slide 21

Slide 21 text

@brapse, @peterbourgon Emergent Distributed Architectures The data pipelines pattern.

Slide 22

Slide 22 text

@brapse, @peterbourgon Emergent Distributed Architectures Data pipelines connect domains.

Slide 23

Slide 23 text

@brapse, @peterbourgon Emergent Distributed Architectures Variance in access patterns reveal problems and opportunities.

Slide 24

Slide 24 text

@brapse, @peterbourgon Emergent Distributed Architectures Variance in access patterns reveal problems and opportunities.

Slide 25

Slide 25 text

@brapse, @peterbourgon Emergent Distributed Architectures Domain experts should own ETLs.

Slide 26

Slide 26 text

@brapse, @peterbourgon Emergent Distributed Architectures Understanding resources for design.

Slide 27

Slide 27 text

@brapse, @peterbourgon Emergent Distributed Architectures Incidental coördination as failure.

Slide 28

Slide 28 text

@brapse, @peterbourgon Emergent Distributed Architectures Collaborative eﬀorts rely on trust.

Slide 29

Slide 29 text

@brapse, @peterbourgon Emergent Distributed Architectures Domains need integrity to grow.

Slide 30

Slide 30 text

@brapse, @peterbourgon Emergent Distributed Architectures Good protocols maximize velocity.

Slide 31

Slide 31 text

@brapse, @peterbourgon Emergent Distributed Architectures Know when and how much to architect.

Slide 32

Slide 32 text

@brapse, @peterbourgon Emergent Distributed Architectures Architectural decisions:   When? Why? How?

Slide 33

Slide 33 text

Organization as organism.

Slide 34

Slide 34 text

@brapse, @peterbourgon Emergent Distributed Architectures Microservices:  Why? —To solve velocity gridlock. When? —It’s obviously necessary, and there are resources for automation.

Slide 35

Slide 35 text

@brapse, @peterbourgon Emergent Distributed Architectures Microservices:  How? —Services with bounded contexts. How? —Autonomous, “vertical” teams.

Slide 36

Slide 36 text

@brapse, @peterbourgon Emergent Distributed Architectures Data pipelines:  Why? —To enable collaboration between   domain experts. When? —Domain expertise varies, and  coördination is expensive.

Slide 37

Slide 37 text

@brapse, @peterbourgon Emergent Distributed Architectures Data pipelines:  How? —Explicit dependency structures. How? —Autonomous, “vertical” teams.   

Slide 38

Slide 38 text

@brapse, @peterbourgon Emergent Distributed Architectures Explicit boundaries maximize velocity.

Slide 39

Slide 39 text

Microservices and Data Pipelines — Sean Braithwaite @brapse · Peter Bourgon @peterbourgon · Velocity London 2017 Emergent Distributed Architectures: