Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Microservices: A retrospective

Microservices: A retrospective

A talk given at Reversim Summit 2019 in Tel-Aviv, Israel.

SOA has been around for decades, and its latest iteration - microservices - for a while now. Just five years ago microservices were hip, dominating the agenda at conferences; now we almost take them for granted. With microservice-focused conference talks losing steam, the time is ripe to consider what we, as an industry, can learn from the microservice disruption; more importantly, we have a golden opportunity to consider what we have yet to learn!

In this talk I'll analyse the lessons hammered into us as we've adopted microservices, and take a hard look at the challenges I believe we still struggle with as an industry. If you're not doing microservices you'll leave the room knowing where the pains will come from; if you are, you will leave with ideas and leads on mitigating these pains.

0014decc65763e66f22891be724b5afa?s=128

Tomer Gabel

June 16, 2019
Tweet

More Decks by Tomer Gabel

Other Decks in Programming

Transcript

  1. Microservices: A Retrospective Tomer Gabel @ Reversim Summit, 2019

  2. Agenda 1. Why oh why 2. Lessons learned 3. Lessons

    not learned 4. The big reveal Image: Today's Menu by Hypnotica Studios Infinite via Flickr (CC BY 2.0)
  3. Technology malcontent Who am I? Engineer @ Organizer Reversim Summit

    alumnus x7
  4. 1. WHY OH WHY Image: 40+216 Faces by bark via

    Flickr (CC BY 2.0)
  5. Why do Microservices? • Developer velocity – Independent releases –

    Limited scope of work – Stronger decoupling • All thanks to bounded context Image: Playing on the Merry go round by Charles Knowles via Flickr (CC BY 2.0)
  6. Why do Microservices? • Scalability – Independent scaling – Independent

    storage – Limited scope is easier to optimize • All thanks to bounded context Image: Skyscraper Under Clear Day Sky via Pixabay (Pixels license)
  7. • Resilience – Strict error boundaries – Partial failures •

    Secondary benefits – Polyglot – Easier to test (maybe?) • Yup: also thanks to bounded context Why do Microservices? Image: Building collapse in Łódź, Sienkiewicza & Tuwima Streets, November 2015 by Zorro2212 (CC BY-SA 4.0)
  8. ORGANIZATIONAL SCALING Nah. It’s all about

  9. Let me explain • A successful organization grows – More

    products – More teams – More visibility – More liability – More responsibility! Image: Evolution Logo Wikimedia by Johanna Pung (CC BY-SA 3.0)
  10. Let me explain • But as you grow… • Products

    become interdependent – And so do teams – Incurring a synchronization cost • “… so what”, you say? Image: SVG Graph illustrating Amdahl's law by Daniels220 (CC BY-SA 3.0)
  11. Let me explain • Four metrics of high- performing† organizations:

    – Lead Time – Deployment Frequency – MTTR – Change Fail Percentage • All negatively affected by synchronization! † Per “Accelerate” by Forsgren, Humble and Kim (IT Revolution Press, 2018)
  12. Let me explain • So we’re shooting for: – Minimizing

    synchronization – Maximizing independence • Microservices are a great fit! Image: Aerial view of Watsonville, California, USA by Robert Campbell (CC BY-SA 3.0)
  13. 2. LESSONS LEARNED

  14. Small is Good • Less code = less bugs •

    Smaller interfaces – Easier to reason about – Easier to evolve – (hard to keep small) • Results in lower coupling
  15. Polyglot Ahoy Company # Languages Google 6 Facebook 9 Twitter

    4 Amazon 3 Netflix ~3 Spotify 4 • The great promise of microservices • Multiple stacks! • You ain’t gonna need it • You ain’t gonna want it • But it sure feels nice
  16. Conway Knows “Organizations which design systems [...] are constrained to

    produce designs which are copies of the communication structures of these organizations.” -- Melvin E. Conway
  17. Conway Knows • Common danger signs: – One data store

    owned by multiple services – Single system owned by multiple teams • If you’re violating SRP… – Reorganize your services – … or your teams! Image: 03053904 by Jerry Norbury via Flickr (CC BY-ND 2.0)
  18. Operations Matter • Developer velocity = ship fast and strong

    • No bottlenecks allowed! – Global release – Centralized ops – Manual deployment – Static topology
  19. Operations Matter • Developer velocity = ship fast and strong

    • No bottlenecks allowed! – Global release ⇨ Many small, rapid iterations – Centralized ops – Manual deployment – Static topology
  20. Operations Matter • Developer velocity = ship fast and strong

    • No bottlenecks allowed! – Global release ⇨ Many small, rapid iterations – Centralized ops ⇨ Distributed ownership – Manual deployment – Static topology
  21. Operations Matter • Developer velocity = ship fast and strong

    • No bottlenecks allowed! – Global release ⇨ Many small, rapid iterations – Centralized ops ⇨ Distributed ownership – Manual deployment ⇨ Full automation – Static topology
  22. Operations Matter • Developer velocity = ship fast and strong

    • No bottlenecks allowed! – Global release ⇨ Many small, rapid iterations – Centralized ops ⇨ Distributed ownership – Manual deployment ⇨ Full automation – Static topology ⇨ Ephemeral, autoscaled I approve this message
  23. Operations Matter • Automation is key – Infrastructure – Deployment

    – Monitoring – Remediation – Provisioning • Automation empowers your teams!
  24. 3. LESSONS WE HAVEN’T LEARNED Image: “If at first you

    don't succeed...” by Adam Fagen via Flickr (CC NC-SA 2.0)
  25. The Laws of Physics We wouldn’t design an airplane …

    while ignoring this guy Images: Landing at Madrid-Barajas airport by José A. Montes (CC BY 2.0) and Portrait of Sir Isaac Newton, oil on canvas (public domain)
  26. The Laws of Physics Why, then, do we ignore them

    … when designing these? Images: Leslie Lamport (from personal site), Eric Brewer at TNW Conference (CC BY-SA 4.0), and A diagram of the proposed system architecture by EpochFail (CC BY-SA 3.0)
  27. The Laws of Physics • We are all building distributed

    systems • Concurrency is key • Disregarding CAP means… – You will end up with inconsistent data – You will not scale – You will lose data Image: The Scream by Edvard Munch (public domain)
  28. Observability • Distributed systems are, by nature… – Disaggregate –

    Hard to reason about • We are still deficient – In tooling – In methodology Image via PXHere (CC0 public domain)
  29. Observability • Tooling is getting better! – Tracing (Jaeger, Zipkin)

    – Metrics (Grafana, Prometheus) – Log aggregation (ELK et al) • … but that’s not enough
  30. Observability • When you can’t debug in isolation… • You

    have to know: – What to log – What to count – What to monitor – How to make sense of it all!
  31. 4. THE BIG REVEAL Image: The Unveiling by nate.stevens via

    Flickr (CC BY-NC-ND 2.0)
  32. Quick Recap • We aim to: – Minimize synchronization –

    Maximize independence • We’re struggling with: – Safety – Scalability – Observability Image: Retrospective - Good by Denis De Mesmaeker via Flickr (CC BY-NC-ND 2.0)
  33. Events • They are your friends • Events make everything

    simpler • No, really. Much simpler • Let’s explore how Image: Ledger by Shea Smith via Flickr (CC BY 2.0)
  34. Events • Modeling interactions with events: – Enforces strong context

    boundaries – Lets you scale services independently – Lets you observe the system in motion – Increases system reliability Client User Service Onboarding Service Search Service Analytics Event Bus RPC
  35. Events • Persisting event streams (event sourcing): – Observability is

    built in! – (as is auditing) – Lets you scale use cases independently (CQRS) • Precludes full consistency – This is a good thingTM Opened Account Deposited $100 Withdrawn $25 Deposited $12 Balance: $87
  36. Events • Modeling workflows in terms of events… – Enables

    sagas – A pattern for cross- domain consistency – A viable alternative to transactions! – (in most cases) Sagas, Garcia-Molina and Salem, SIGMOD 1987
  37. SWEAT THE PRINCIPLES Not the implementation details.

  38. Commoditization • A lot has changed • We have tools

    now! – gRPC, Thrift, Avro – Kafka – Docker, Kubernetes – Spinnaker • No need to roll your own (less so, anyway)! Image via opensource.com (CC BY-SA)
  39. Expertise • Invest in studying: – CAP tradeoffs – Domain

    modelling – Event storming – Event sourcing/CQRS – Sagas • You’ll be better off! Image: Back to School by Phil Roeder via Flickr (CC BY 2.0)
  40. Further Reading Design Microservice Architectures the Right Way Michael Bryzek

    An Abriged Guide to Event Sourcing Tomer Gabel
  41. QUESTIONS? Thank you for listening tomer@tomergabel.com @tomerg On GitHub: https://github.com/holograph

    This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.