Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Breaking down the monolith - devone

Breaking down the monolith - devone

The story of how we broke our Node.js monolith into 20+ services in just a couple of weeks. The talk will focus on what we have learned during the journey and what technologies we use. This is the tale of how we did this rather than what microservices are in general. Attendees are going gain a better understanding on how a microservice architecture effects a company's structure and the daily work. The talk also highlights the key concepts and challenges of the microservice migration with providing practical solutions as well.


Peter Marton

June 01, 2017

More Decks by Peter Marton

Other Decks in Programming


  1. Breaking Down The Monolith By Peter Marton

  2. - CTO, Co-founder of RisingStack - @slashdotpeter on Twitter -

    https://blog.risingstack.com $ whoami
  3. - Helping companies to succeed with Node.js: - Training, Consulting,

    Support - Professional services - Trace: Node.js monitoring and debugging tool
  4. - Why we moved: monolith, microservices - How we moved:

    services, teams and principles - Evolutionary design - Reliability and fault tolerance - Debugging and monitoring Agenda
  5. Monolith Database

  6. - Easy to ship, as all the parts move together

    - Easy to create a developer environment - Easy to monitor and debug (we have stack traces) - Perfect for MVP and smaller applications Monolith advantages
  7. - Larger teams work on the same code base -

    Distributed teams can be challenging (in location / time) - Hard to keep a good modular structure - Cannot deploy small changes separately - Cannot scale features separately Monolith drawbacks
  8. Monolith Database

  9. Microservices Database Database

  10. - Each element of functionality is a separate service -

    Split by features - Collection of loosely coupled services - Distributed system with remote calls Microservices
  11. - Improves modularity - Smaller scope, easier to understand -

    Smaller codebase: easy to refactor or throw away - Services can be individually designed, developed, tested, and deployed (self-contained) Microservices advantages 1/2
  12. - Services can be scaled separately - Allows technology diversity

    - Can improve reliability - Autonomous teams Microservices advantages 2/2
  13. - Architectural complexity: distributed system - Eventual consistency - Operating

    complexity - Monitoring and debugging complexity - Not the most performant architectural pattern Microservices drawbacks
  14. Complexity: before we moved

  15. Complexity: today

  16. - Loosely coupled: minimal dependence on other services - High

    cohesion: closely related functionality should stay together - Single bounded context: encapsulates internal details of a domain Don't be misled by the "micro" prefix. “Micro”services
  17. - Avoid distributed transactions - Do not create service until:

    - You don’t understand the domain very well - When it’s not necessary - To prevent multiple data migrations Avoid: distributed monolith
  18. microservice architecture is not a silver bullet

  19. Moving to microservices

  20. Moving to microservices

  21. Microservices require organizational change

  22. “organizations which design systems ... are constrained to produce designs

    which are copies of the communication structures of these organizations” -  M. Conway Conway’s law
  23. - UI Team - Backend team - Database team Organizing

    around Business Capabilities
  24. Organizing around Business Capabilities Image: https://martinfowler.com

  25. - Cross functional teams - Full range of skills: From

    UX to PM - Each of the teams have: backend, database, ops etc. - Mini products inside the company - Teams ows the full lifecycle Organizing around features
  26. Cross functional teams Image: https://martinfowler.com

  27. - Distributed teams (different space / time zone) - Smaller

    teams are usually more efficient - More focused and specialized people - Separate smaller releases are possible Possible business reasons to move
  28. Why we moved?

  29. - Very different challenges: - time series metrics, distributed tracing

    - Mission critical features: - alerting Our product has:
  30. - We wanted to have focused teams - Develop and

    ship features separately - Segregate failures - Improve reliability We wanted to separate these features
  31. How we moved?

  32. - Highly aligned, loosely coupled teams - Principles helps in

    alignment - Principles should be created together - Teams must follow it Service principles: why?
  33. - Automation is a key in building and operating microservices

    - Setup a new service with CI jobs and monitoring - Easy and fast to deploy and rollback - Testing, Database migration etc. - Reliability automations: auto scaling, rate limiters etc. Service principles: automate everything
  34. - Services should own their data - Separate databases per

    services - Allow polyglot persistence (different database technologies) Service principles: data ownership
  35. - Minimize the depth of the service call-graph - Minimize

    complexity - Increases performance Service principles: complexity
  36. - Only high level utility code pieces - Common libraries

    and frameworks company wide: - High level framework - Libraries for: logging, caching, service communication etc. - Via private npm: scoped packages Service principles: code sharing
  37. - We don’t want to version services - We do

    version endpoints - We create backward compatible endpoints - Use feature flippers / toggles (dark launches) Service principles: versioning
  38. - Interface for other teams - Good and available documentation

    - Update docs and code together Good to start here: https://github.com/Yelp/service-principles Service principles: documentation
  39. Evolutionary design

  40. - Rewriting your application can be expensive and dangerous -

    Need to ship features in the meantime - Organizational change takes time Evolutionary design: why
  41. Proxy

  42. - Client(s) specific things: - Authentication: Cookie headers, JWT token

    etc. - Protocols: HTTP, WebSocket etc. - Response format: JSON, XML etc. - Combining resources: from multiple services API Gateway
  43. API Gateway Img: http://bits.citrusbyte.com/microservices

  44. - We moved from a PaaS - Services were communication

    via public internet - Trusted sources (services) - Request signing between services Security: request signing
  45. Security: request signing

  46. - Request signing had significant CPU overhead - Added ~10ms

    to each request - Moved to private networking - One entry point: API Gateway Private networking
  47. One entry point: API Gateway

  48. Fault tolerance

  49. Services fail separately

  50. Graceful service quality degradation Service B Service A Service D

    C Service B Service A Service D C
  51. Services fail separately (in theory)

  52. Failure Service B Service A Service D C Service B

    Service A Service D C
  53. Resources should be cached Cache

  54. - On service level by our communication library - Automatically

    via response headers (with HTTP): - "max-age" and "stale-if-error" - Multi store caching with stale and failover cache - https://github.com/RisingStack/cache Caching
  55. - Services fail separately - Critical resources should be cached

    - Rate limiters and load shedders - Reliability automations: auto scaling, deploy strategies - Architectural patterns and messaging queues can help Fault tolerance
  56. Fault tolerant data collection - CQRS (Command Query Responsibility Segregation)

    read (sync) write (async)
  57. Fault tolerant data collecting Queue size increasing during issue Queue

    size decreasing after issue resolved
  58. Infrastructure

  59. - Fast private network - Quick deploys and rollbacks (with

    immutability) - Zero downtime deployment - Flexible resources, cheap instances Infrastructure: wanted
  60. - Started with PaaS - Good at beginning: rollback, zero

    downtime deployment etc. - Slow and public networking - Expensive and non flexible instance types - Evaluated: more PaaS, AWS EB, Deis (v1) Infrastructure: what we had
  61. - We moved to Kubernetes (GCP managed) - Container orchestration

    (Docker/rkt) - Graceful shutdown - Rolling deployment - Self healing applications - Horizontal autoscaling Infrastructure: Kubernetes
  62. Monitoring and debugging

  63. Microservices producing lot’s of data - Logs, Errors - Metrics

    per service - Transactions - Logical connections - Side effects
  64. Needle in the haystack

  65. microservice monitoring is impossible for humans

  66. - Google Dapper white paper - Transaction ID, Correlation ID

    - Commercial: Trace by RisingStack, Dynatrace, AWS X-Ray - OSS: Jaeger, Zipkin - Instrumentation standard: OpenTracing Distributed tracing
  67. Distributed tracing: Trace by RisingStack

  68. Distributed tracing: Jaeger

  69. Case study: Network delay

  70. Case study: Network delay 3.8ms 564.8ms 95th response time: Call

    depth: 1 service 2 services
  71. Case study: Network delay network delay

  72. network delay is evil in microservices

  73. - Use private network - Keep your applications close to

    each other - Limit the lengths of service call chains - Cache everything Network delay is evil
  74. - https://blog.risingstack.com/tag/node-js-at-scale - https://microserviceweekly.com/ - https://risingstack.com/trainings What’s next?

  75. Be confident with Node.js Thanks! Breaking Down The Monolith by

    Peter Marton, @slashdotpeter