Underestimated costs of microservice architectures

Slide 1

Slide 1 text

8th Light, Inc. Colin Jones @trptcolin https://8thlight.com Underestimated costs of microservice architectures

Slide 2

Slide 2 text

Microservices

Slide 3

Slide 3 text

Happy!

Slide 4

Slide 4 text

☠ Sad ☠

Slide 5

Slide 5 text

No content

Slide 6

Slide 6 text

No content

Slide 7

Slide 7 text

No content

Slide 8

Slide 8 text

No content

Slide 9

Slide 9 text

Avoid microservices?

Slide 10

Slide 10 text

Avoid microservices?

Slide 11

Slide 11 text

On other hand But to gain any benefit from microservice thinking, you have to understand what it is, how to do it, and why you should usually do something else. - Martin Fowler

Slide 12

Slide 12 text

Hype Cycle

Slide 13

Slide 13 text

Accusations

Slide 14

Slide 14 text

We underestimate costs

Slide 15

Slide 15 text

Benefits

Slide 16

Slide 16 text

independent deployability

Slide 17

Slide 17 text

independent scalability independent deployability

Slide 18

Slide 18 text

fault tolerance independent deployability independent scalability

Slide 19

Slide 19 text

avoid dependency hell independent deployability independent scalability fault tolerance

Slide 20

Slide 20 text

architectural boundaries independent deployability independent scalability fault tolerance avoid dependency hell

Slide 21

Slide 21 text

small team ownership independent deployability independent scalability fault tolerance avoid dependency hell architectural boundaries

Slide 22

Slide 22 text

eliminate legacy code independent deployability independent scalability fault tolerance avoid dependency hell architectural boundaries small team ownership

Slide 23

Slide 23 text

eliminate legacy code independent deployability independent scalability fault tolerance avoid dependency hell architectural boundaries small team ownership microservices!

Slide 24

Slide 24 text

Costs & Mitigations

Slide 25

Slide 25 text

Well-understood costs

Slide 26

Slide 26 text

Latency

Slide 27

Slide 27 text

Latency Mitigations • Cache responses • Batch calls together • Coarse-grained service API

Slide 28

Slide 28 text

Additional infrastructure Latency

Slide 29

Slide 29 text

Additional infrastructure Mitigations • Containers (e.g. Docker) • Infrastructure automation & configuration management • Virtual machines / cloud • Auto-scaling (metered cost) • Serverless

Slide 30

Slide 30 text

Understanding != Paying

Slide 31

Slide 31 text

Underestimated Costs

Slide 32

Slide 32 text

Data consistency Additional infrastructure Latency

Slide 33

Slide 33 text

Data consistency Orders Service Main App Main DB

Slide 34

Slide 34 text

Data consistency Orders Service Main App Orders DB Main DB

Slide 35

Slide 35 text

Data consistency Mitigations • Design for eventual consistency • Canonical source for data (aka “system of record” / “source of truth”) and derived data • Backend sync processes • Service teams co-own ETL for analytics/ business intelligence / data warehouse

Slide 36

Slide 36 text

Data consistency

Slide 37

Slide 37 text

Failure modes Data consistency Additional infrastructure Latency

Slide 38

Slide 38 text

Failure modes A B

Slide 39

Slide 39 text

Failure modes A B ???

Slide 40

Slide 40 text

Failure modes A B

Slide 41

Slide 41 text

Failure modes A B

Slide 42

Slide 42 text

Failure modes A B ???

Slide 43

Slide 43 text

Failure modes Mitigations • Use retries (with backoff; cap the max time) • Read the remote end to see if it succeeded • Use fallbacks for read timeouts • Use circuit breaker to limit cascading failures • Use bulkheads to protect independent modules

Slide 44

Slide 44 text

Failure modes

Slide 45

Slide 45 text

Development & testing Failure modes Data consistency Additional infrastructure Latency

Slide 46

Slide 46 text

Development & testing

Slide 47

Slide 47 text

from "Testing Microservice the Sane Way" by Cindy Sridharan Development & testing

Slide 48

Slide 48 text

Mitigations • Expand testing mindset to staging/production observability efforts • Integrate only a few services / API checking and rely more on unit tests • Unify on lightweight runtimes and single database technology [only delays the problem; conflicts with team ownership] • Test against external environments with services set up [risks test pollution] • Orchestrate new isolated infrastructure for each test run Development & testing

Slide 49

Slide 49 text

Observability Failure modes Development & testing Data consistency Additional infrastructure Latency

Slide 50

Slide 50 text

Observability

Slide 51

Slide 51 text

Observability Mitigations • Log aggregation with correlation IDs • Error reporting / alerting [generally on symptoms, not causes] • Distributed tracing tools • Monitoring tools / dashboards • Fancier 3rd-party observability tools

Slide 52

Slide 52 text

Tunnel vision Failure modes Development & testing Observability Data consistency Additional infrastructure Latency

Slide 53

Slide 53 text

Tunnel vision

Slide 54

Slide 54 text

Mitigations • Measure business metrics, not team velocity • Make sure team / service incentives are aligned with the company’s • Rotations / team exchanges / dynamic re- teaming • Cross-org communication Tunnel vision

Slide 55

Slide 55 text

Implicit connection data Failure modes Development & testing Observability Tunnel vision Data consistency Additional infrastructure Latency

Slide 56

Slide 56 text

Implicit connection data

Slide 57

Slide 57 text

Mitigations • Well known API contracts / specifications (e.g. JSON Schema, Swagger, Protocol Buffers, Thrift) • Centralized/standardized repository to track service metadata for service discovery • Put all services in one codebase (monorepo) for easier searchability • Custom tooling based on log aggregation / monitoring Implicit connection data

Slide 58

Slide 58 text

Inter-team priority conflicts Failure modes Development & testing Observability Tunnel vision Implicit connection data Data consistency Additional infrastructure Latency

Slide 59

Slide 59 text

Inter-team priority conflicts Them Us

Slide 60

Slide 60 text

Inter-team priority conflicts Them Consumer B Us (Consumer A) Consumer C Consumer C

Slide 61

Slide 61 text

Mitigations • Make our case really well to the service team, or management • Add staff on heavily-used microservices • Split heavily-used services further • Contribute to their project (aka “internal open- source”) • Rebuild a similar service with the changes we need Inter-team priority conflicts

Slide 62

Slide 62 text

Hard to change across boundaries Failure modes Development & testing Observability Tunnel vision Inter-team priority conflicts Implicit connection data Data consistency Additional infrastructure Latency

Slide 63

Slide 63 text

Hard to change across boundaries

Slide 64

Slide 64 text

Hard to change across boundaries

Slide 65

Slide 65 text

Mitigations • Be deliberate about the choice to Extract Microservice • Version your API? [controversial] • If versioning / breaking: Have a well-defined way to communicate breaking changes / deadlines • Sticking with the same runtime (e.g. JVM) makes Inline Microservice possible • Cross-org communication Hard to change across boundaries

Slide 66

Slide 66 text

Mitigations • Skill and culture of backwards compatibility (SemVer, Postel’s Law) • Don’t make breaking changes • Well known API contracts / specifications (e.g. JSON Schema, Swagger, Protocol Buffers, Thrift) • Consumer-driven contract tests in CI • See also “Connection data is implicit” mitigations Hard to change across boundaries

Slide 67

Slide 67 text

Failure modes Development & testing Observability Tunnel vision Inter-team priority conflicts Implicit connection data Data consistency Additional infrastructure Latency Hard to change across boundaries microservices!

Slide 68

Slide 68 text

Alternatives

Slide 69

Slide 69 text

Milliservices? Centiservices?

Slide 70

Slide 70 text

Modules / encapsulation

Slide 71

Slide 71 text

Modules / encapsulation

Slide 72

Slide 72 text

Modules / encapsulation

Slide 73

Slide 73 text

Modules / encapsulation

Slide 74

Slide 74 text

Recommendations

Slide 75

Slide 75 text

List problems. Then solutions. Then pros and cons.

Slide 76

Slide 76 text

Don’t believe the hype.

Slide 77

Slide 77 text

Be ready to pay the costs.

Slide 78

Slide 78 text

Make sure you’re getting the benefits.

Slide 79

Slide 79 text

Make the change easy. Then make the change.

Slide 80

Slide 80 text

Make the change easy. Then make the change. (maybe)

Slide 81

Slide 81 text

Learn more

Slide 82

Slide 82 text

Learn more

Slide 83

Slide 83 text

Learn more •Ben Christensen. “Don’t Build a Distributed Monolith”: https:// www.microservices.com/talks/dont-build-a-distributed-monolith/ •Michael Feathers. “Microservices and the Failure of Encapsulation”: https:// michaelfeathers.silvrback.com/microservices-and-the-failure-of-encapsulaton •Michael Feathers. Working Effectively with Legacy Code: https:// www.amazon.com/Working-Effectively-Legacy-Michael-Feathers/dp/ 0131177052 •Martin Fowler. Enterprise Application Architecture: https:// www.martinfowler.com/books/eaa.html •Martin Fowler. “MicroservicePremium”: https://www.martinfowler.com/ bliki/MicroservicePremium.html •Martin Fowler. “Microservice Prerequisites”: https://www.martinfowler.com/ bliki/MicroservicePrerequisites.html •Martin Fowler. “Microservices Resource Guide”: https:// www.martinfowler.com/microservices/

Slide 84

Slide 84 text

Learn more •Susan Fowler. Production-Ready Microservices: http://shop.oreilly.com/product/ 0636920053675.do •John Gall. The Systems Bible: https://www.amazon.com/Systems-Bible-Beginners- Guide-Large/dp/0961825170 •David Heinemeier Hansson. “The Majestic Monolith”: https://m.signalvnoise.com/ the-majestic-monolith-29166d022228 •Rich Hickey. “Hammock Driven Development”: https://www.youtube.com/watch? v=f84n5oFoZBc •Gregor Hohpe and Bobby Woolf. Enterprise Integration Patterns: http:// www.enterpriseintegrationpatterns.com/ •Mike Knepper. “The Hidden Costs of Leaving a Monolith”: https://8thlight.com/ blog/mike-knepper/2016/01/20/hidden-costs-of-leaving-a-monolith.html •Dan Manges. “The Modular Monolith: Rails Architecture”: https://medium.com/ @dan_manges/the-modular-monolith-rails-architecture-fb1023826fc4 •Sam Newman. Building Microservices: https://samnewman.io/books/ building_microservices/

Slide 85

Slide 85 text

Learn more •Michael Nygard. “The Entity Microservice Antipattern”: http:// www.michaelnygard.com/blog/2017/12/the-entity-service-antipattern/ •Michael Nygard. Release It!, 2nd edition: https://pragprog.com/book/mnee2/ release-it-second-edition •Ozan Onay. “You are not Google”: https://blog.bradfieldcs.com/you-are-not- google-84912cf44afb •Arnon Rotem-Gal-Oz. “Fallacies of Distributed Computing Explained”: http:// www.rgoarchitects.com/Files/fallacies.pdf •Cindy Sridharan. “Testing Microservices, the Sane Way”: https://medium.com/ @copyconstruct/testing-microservices-the-sane-way-9bb31d158c16 •Cindy Sridharan. “Testing in Production, the safe way”: https://medium.com/ @copyconstruct/testing-in-production-the-safe-way-18ca102d0ef1 •Jim Waldo, Geoff Wyant, Ann Wolrath, and Sam Kendall. “A Note on Distributed Computing”: http://web.cs.wpi.edu/~cs3013/a11/Papers/ Waldo_NoteOnDistributedComputing.pdf

Slide 86

Slide 86 text

8th Light, Inc. Colin Jones @trptcolin https://8thlight.com Thank you!