Emergent Distributed Architectures

Peter Bourgon and Sean Braithwaite offer an overview of microservices and data pipelines, explaining how both systems reflect the organizations and people that build them (in adherence to Conway’s law) and can be well understood in terms of their relationship to change and time. You'll learn the virtues and vices of each architecture and get enough context to apply them coherently.

Transcript

1. Microservices and Data Pipelines — Sean Braithwaite @brapse · Peter Bourgon @peterbourgon · Velocity London 2017 Emergent Distributed Architectures:
2. Microservices and Data Pipelines — Sean Braithwaite @brapse · Peter Bourgon @peterbourgon · Velocity London 2017 Emergent Distributed Architectures:
3. @brapse, @peterbourgon Emergent Distributed Architectures Architectural decisions are expensive.
4. @brapse, @peterbourgon Emergent Distributed Architectures We want to answer not only how, but why and when.
5. Organization as organism.
6. Cells as parallel actors.
7. Teams as cells.
8. @brapse, @peterbourgon Emergent Distributed Architectures Teams enable parallelism.
 Teams must consider global interests to avoid harmful local maxima.
9. @brapse, @peterbourgon Emergent Distributed Architectures
10. @brapse, @peterbourgon Emergent Distributed Architectures
11. @brapse, @peterbourgon Emergent Distributed Architectures Bounded contexts
 reduce external coördination costs & make internal work more efficient.
12. @brapse, @peterbourgon Emergent Distributed Architectures
13. @brapse, @peterbourgon Emergent Distributed Architectures
14. @brapse, @peterbourgon Emergent Distributed Architectures The microservices pattern.
15. @brapse, @peterbourgon Emergent Distributed Architectures We can be overeager applying
 apparently successful patterns.
16. @brapse, @peterbourgon Emergent Distributed Architectures Microservices cause more problems
 than they solve.
17. @brapse, @peterbourgon Emergent Distributed Architectures But microservices codify and reify their teams' responsibilities.
18. @brapse, @peterbourgon Emergent Distributed Architectures Well-defined teams and services maximize product velocity.
19. @brapse, @peterbourgon Emergent Distributed Architectures Microservices actually solve communication problems.
20. @brapse, @peterbourgon Emergent Distributed Architectures Microservices are an immune response to velocity gridlock.
21. @brapse, @peterbourgon Emergent Distributed Architectures The data pipelines pattern.
22. @brapse, @peterbourgon Emergent Distributed Architectures Data pipelines connect domains.
23. @brapse, @peterbourgon Emergent Distributed Architectures Variance in access patterns reveal problems and opportunities.
24. @brapse, @peterbourgon Emergent Distributed Architectures Variance in access patterns reveal problems and opportunities.
25. @brapse, @peterbourgon Emergent Distributed Architectures Domain experts should own ETLs.
26. @brapse, @peterbourgon Emergent Distributed Architectures Understanding resources for design.
27. @brapse, @peterbourgon Emergent Distributed Architectures Incidental coördination as failure.
28. @brapse, @peterbourgon Emergent Distributed Architectures Collaborative efforts rely on trust.
29. @brapse, @peterbourgon Emergent Distributed Architectures Domains need integrity to grow.
30. @brapse, @peterbourgon Emergent Distributed Architectures Good protocols maximize velocity.
31. @brapse, @peterbourgon Emergent Distributed Architectures Know when and how much to architect.
32. @brapse, @peterbourgon Emergent Distributed Architectures Architectural decisions: 
 When? Why? How?
33. Organization as organism.
34. @brapse, @peterbourgon Emergent Distributed Architectures Microservices:
 Why? —To solve velocity gridlock. When? —It’s obviously necessary, and there are resources for automation.
35. @brapse, @peterbourgon Emergent Distributed Architectures Microservices:
 How? —Services with bounded contexts. How? —Autonomous, “vertical” teams.
36. @brapse, @peterbourgon Emergent Distributed Architectures Data pipelines:
 Why? —To enable collaboration between 
 domain experts. When? —Domain expertise varies, and
 coördination is expensive.
37. @brapse, @peterbourgon Emergent Distributed Architectures Data pipelines:
 How? —Explicit dependency structures. How? —Autonomous, “vertical” teams.
 

38. @brapse, @peterbourgon Emergent Distributed Architectures Explicit boundaries maximize velocity.
39. Microservices and Data Pipelines — Sean Braithwaite @brapse · Peter Bourgon @peterbourgon · Velocity London 2017 Emergent Distributed Architectures: