Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Cheating Gall's Law

Cheating Gall's Law

How the npm registry rewrite cheated Gall's Law and split a monolith into a set of microservices. Delivered at WebRebels 2015.

C J Silverio

May 22, 2015
Tweet

More Decks by C J Silverio

Other Decks in Programming

Transcript

  1. Cheating Gall's Law

    View full-size slide

  2. C J Silverio
    director of engineering,
    @ceejbot

    View full-size slide

  3. Cheating Gall's Law
    How we split a
    monolith
    and lived to tell the tale

    View full-size slide

  4. Gall's Law
    A complex system that works is
    invariably found to have evolved
    from a simple system that worked.

    View full-size slide

  5. monolith
    everything in one process

    View full-size slide

  6. npm's monolith:
    embedded in couchdb

    View full-size slide

  7. monoliths
    work just fine

    View full-size slide

  8. whatever it takes
    to build a system
    that satisfies your users

    View full-size slide

  9. success!
    now scale it.

    View full-size slide

  10. scaling monoliths
    many copies of the full thing

    View full-size slide

  11. Exponential growth of node
    resulted in exponential growth
    of the npm registry

    View full-size slide

  12. exponential monoliths
    were going to be expensive

    View full-size slide

  13. splitting the
    monolith

    View full-size slide

  14. yay microservices?

    View full-size slide

  15. just rewrite everything!
    what's the problem?

    View full-size slide

  16. Your monolith is complex.
    A split system is more complex.

    View full-size slide

  17. what did that Gall guy say about
    complex working systems?

    View full-size slide

  18. Gall's Law in full
    “A complex system that works is invariably found to
    have evolved from a simple system that worked. A
    complex system designed from scratch never works and
    cannot be patched up to make it work. You have to
    start over with a working simple system.”

    View full-size slide

  19. Systemantics:
    How Systems Really Work
    and How They Fail

    View full-size slide

  20. "A simple system
    may or may not work."

    View full-size slide

  21. "If a system is working,
    leave it alone.
    Don't change anything."

    View full-size slide

  22. Gall's Law
    warns us about
    that rewrite

    View full-size slide

  23. how do you split
    a monolith
    successfully?

    View full-size slide

  24. Let's cheat.

    View full-size slide

  25. Q: How do you cheat?
    A: By not rewriting the whole thing.

    View full-size slide

  26. slice off a part of the system
    into a module
    with a clearly-defined interface

    View full-size slide

  27. then write a
    second implementation
    of that interface

    View full-size slide

  28. send requests to that
    second implementation
    with a proxy

    View full-size slide

  29. now let's make our proxy smart
    Replace Varnish
    with a node service

    View full-size slide

  30. first step
    just send everything through

    View full-size slide

  31. record what the proxy does
    how do people use your service?
    measure performance

    View full-size slide

  32. second step
    divide & conquer

    View full-size slide

  33. beat Gall's Law with
    modularity
    aka information hiding

    View full-size slide

  34. each service is a module
    a simple, testable system
    when viewed in isolation

    View full-size slide

  35. best part:
    now you can change
    everything

    View full-size slide

  36. your platform
    your database

    View full-size slide

  37. why stop there?

    View full-size slide

  38. Each piece is a
    simple system

    View full-size slide

  39. more scaling dials to turn
    finer-grained perf data

    View full-size slide

  40. isolating tasks
    let us debug
    even without splitting the monolith

    View full-size slide

  41. the system overall is complex
    but loosely coupled

    View full-size slide

  42. you have a working system
    every step of the way

    View full-size slide

  43. a proxy service lets you
    flip back & forth
    between implementations

    View full-size slide

  44. the pitfalls

    View full-size slide

  45. now you've got a
    distributed system

    View full-size slide

  46. @architectclippy says
    “I see you have a poorly structured monolith. Would
    you like me to convert it into a poorly structured
    set of microservices?”

    View full-size slide

  47. Second system syndrome
    "this time we'll do it right"

    View full-size slide

  48. Generalize only when forced

    View full-size slide

  49. let's recap!
    how do you survive a rewrite?

    View full-size slide

  50. simple working service first
    scale it later

    View full-size slide

  51. respect Gall:
    rewrite in pieces

    View full-size slide

  52. use a proxy
    to divide & conquer

    View full-size slide

  53. Be bold
    you can change your system

    View full-size slide

  54. Nobody noticed
    when we changed
    the entire npm registry

    View full-size slide

  55. npm loves you
    npm install -g npm@latest

    View full-size slide