Breaking down the monolith - devone

The story of how we broke our Node.js monolith into 20+ services in just a couple of weeks. The talk will focus on what we have learned during the journey and what technologies we use. This is the tale of how we did this rather than what microservices are in general. Attendees are going gain a better understanding on how a microservice architecture effects a company's structure and the daily work. The talk also highlights the key concepts and challenges of the microservice migration with providing practical solutions as well.

https://devone.at/

Transcript

1. Breaking Down The Monolith By Peter Marton
2. - CTO, Co-founder of RisingStack - @slashdotpeter on Twitter - https://blog.risingstack.com $ whoami
3. - Helping companies to succeed with Node.js: - Training, Consulting, Support - Professional services - Trace: Node.js monitoring and debugging tool
4. - Why we moved: monolith, microservices - How we moved: services, teams and principles - Evolutionary design - Reliability and fault tolerance - Debugging and monitoring Agenda
5. Monolith Database
6. - Easy to ship, as all the parts move together - Easy to create a developer environment - Easy to monitor and debug (we have stack traces) - Perfect for MVP and smaller applications Monolith advantages
7. - Larger teams work on the same code base - Distributed teams can be challenging (in location / time) - Hard to keep a good modular structure - Cannot deploy small changes separately - Cannot scale features separately Monolith drawbacks
8. Monolith Database
9. Microservices Database Database
10. - Each element of functionality is a separate service - Split by features - Collection of loosely coupled services - Distributed system with remote calls Microservices
11. - Improves modularity - Smaller scope, easier to understand - Smaller codebase: easy to refactor or throw away - Services can be individually designed, developed, tested, and deployed (self-contained) Microservices advantages 1/2
12. - Services can be scaled separately - Allows technology diversity - Can improve reliability - Autonomous teams Microservices advantages 2/2
13. - Architectural complexity: distributed system - Eventual consistency - Operating complexity - Monitoring and debugging complexity - Not the most performant architectural pattern Microservices drawbacks
14. Complexity: before we moved
15. Complexity: today
16. - Loosely coupled: minimal dependence on other services - High cohesion: closely related functionality should stay together - Single bounded context: encapsulates internal details of a domain Don't be misled by the "micro" prefix. “Micro”services
17. - Avoid distributed transactions - Do not create service until: - You don’t understand the domain very well - When it’s not necessary - To prevent multiple data migrations Avoid: distributed monolith
18. microservice architecture is not a silver bullet
19. Moving to microservices
20. Moving to microservices
21. Microservices require organizational change
22. “organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations” -  M. Conway Conway’s law
23. - UI Team - Backend team - Database team Organizing around Business Capabilities
24. Organizing around Business Capabilities Image: https://martinfowler.com
25. - Cross functional teams - Full range of skills: From UX to PM - Each of the teams have: backend, database, ops etc. - Mini products inside the company - Teams ows the full lifecycle Organizing around features
26. Cross functional teams Image: https://martinfowler.com
27. - Distributed teams (different space / time zone) - Smaller teams are usually more efficient - More focused and specialized people - Separate smaller releases are possible Possible business reasons to move
28. Why we moved?
29. - Very different challenges: - time series metrics, distributed tracing - Mission critical features: - alerting Our product has:
30. - We wanted to have focused teams - Develop and ship features separately - Segregate failures - Improve reliability We wanted to separate these features
31. How we moved?
32. - Highly aligned, loosely coupled teams - Principles helps in alignment - Principles should be created together - Teams must follow it Service principles: why?
33. - Automation is a key in building and operating microservices - Setup a new service with CI jobs and monitoring - Easy and fast to deploy and rollback - Testing, Database migration etc. - Reliability automations: auto scaling, rate limiters etc. Service principles: automate everything
34. - Services should own their data - Separate databases per services - Allow polyglot persistence (different database technologies) Service principles: data ownership
35. - Minimize the depth of the service call-graph - Minimize complexity - Increases performance Service principles: complexity
36. - Only high level utility code pieces - Common libraries and frameworks company wide: - High level framework - Libraries for: logging, caching, service communication etc. - Via private npm: scoped packages Service principles: code sharing
37. - We don’t want to version services - We do version endpoints - We create backward compatible endpoints - Use feature flippers / toggles (dark launches) Service principles: versioning
38. - Interface for other teams - Good and available documentation - Update docs and code together Good to start here: https://github.com/Yelp/service-principles Service principles: documentation
39. Evolutionary design
40. - Rewriting your application can be expensive and dangerous - Need to ship features in the meantime - Organizational change takes time Evolutionary design: why
41. Proxy
42. - Client(s) specific things: - Authentication: Cookie headers, JWT token etc. - Protocols: HTTP, WebSocket etc. - Response format: JSON, XML etc. - Combining resources: from multiple services API Gateway
43. API Gateway Img: http://bits.citrusbyte.com/microservices
44. - We moved from a PaaS - Services were communication via public internet - Trusted sources (services) - Request signing between services Security: request signing
45. Security: request signing
46. - Request signing had significant CPU overhead - Added ~10ms to each request - Moved to private networking - One entry point: API Gateway Private networking
47. One entry point: API Gateway
48. Fault tolerance
49. Services fail separately
50. Graceful service quality degradation Service B Service A Service D C Service B Service A Service D C
51. Services fail separately (in theory)
52. Failure Service B Service A Service D C Service B Service A Service D C
53. Resources should be cached Cache
54. - On service level by our communication library - Automatically via response headers (with HTTP): - "max-age" and "stale-if-error" - Multi store caching with stale and failover cache - https://github.com/RisingStack/cache Caching
55. - Services fail separately - Critical resources should be cached - Rate limiters and load shedders - Reliability automations: auto scaling, deploy strategies - Architectural patterns and messaging queues can help Fault tolerance
56. Fault tolerant data collection - CQRS (Command Query Responsibility Segregation) read (sync) write (async)
57. Fault tolerant data collecting Queue size increasing during issue Queue size decreasing after issue resolved
58. Infrastructure
59. - Fast private network - Quick deploys and rollbacks (with immutability) - Zero downtime deployment - Flexible resources, cheap instances Infrastructure: wanted
60. - Started with PaaS - Good at beginning: rollback, zero downtime deployment etc. - Slow and public networking - Expensive and non flexible instance types - Evaluated: more PaaS, AWS EB, Deis (v1) Infrastructure: what we had
61. - We moved to Kubernetes (GCP managed) - Container orchestration (Docker/rkt) - Graceful shutdown - Rolling deployment - Self healing applications - Horizontal autoscaling Infrastructure: Kubernetes
62. Monitoring and debugging
63. Microservices producing lot’s of data - Logs, Errors - Metrics per service - Transactions - Logical connections - Side effects
64. Needle in the haystack
65. microservice monitoring is impossible for humans
66. - Google Dapper white paper - Transaction ID, Correlation ID - Commercial: Trace by RisingStack, Dynatrace, AWS X-Ray - OSS: Jaeger, Zipkin - Instrumentation standard: OpenTracing Distributed tracing
67. Distributed tracing: Trace by RisingStack
68. Distributed tracing: Jaeger
69. Case study: Network delay
70. Case study: Network delay 3.8ms 564.8ms 95th response time: Call depth: 1 service 2 services
71. Case study: Network delay network delay
72. network delay is evil in microservices
73. - Use private network - Keep your applications close to each other - Limit the lengths of service call chains - Cache everything Network delay is evil
74. - https://blog.risingstack.com/tag/node-js-at-scale - https://microserviceweekly.com/ - https://risingstack.com/trainings What’s next?
75. Be confident with Node.js Thanks! Breaking Down The Monolith by Peter Marton, @slashdotpeter