Breaking down the monolith - devone

The story of how we broke our Node.js monolith into 20+ services in just a couple of weeks. The talk will focus on what we have learned during the journey and what technologies we use. This is the tale of how we did this rather than what microservices are in general. Attendees are going gain a better understanding on how a microservice architecture effects a company's structure and the daily work. The talk also highlights the key concepts and challenges of the microservice migration with providing practical solutions as well.

https://devone.at/

Transcript

1. Breaking Down The Monolith
   By Peter Marton
   

   View Slide

2. - CTO, Co-founder of RisingStack
   - @slashdotpeter on Twitter
   - https://blog.risingstack.com
   $ whoami
   

   View Slide

3. - Helping companies to succeed with Node.js:
   - Training, Consulting, Support
   - Professional services
   - Trace: Node.js monitoring and debugging tool
   

   View Slide

4. - Why we moved: monolith, microservices
   - How we moved: services, teams and principles
   - Evolutionary design
   - Reliability and fault tolerance
   - Debugging and monitoring
   Agenda
   

   View Slide

5. Monolith
   Database
   

   View Slide

6. - Easy to ship, as all the parts move together
   - Easy to create a developer environment
   - Easy to monitor and debug (we have stack traces)
   - Perfect for MVP and smaller applications
   Monolith advantages
   

   View Slide

7. - Larger teams work on the same code base
   - Distributed teams can be challenging (in location / time)
   - Hard to keep a good modular structure
   - Cannot deploy small changes separately
   - Cannot scale features separately
   Monolith drawbacks
   

   View Slide

8. Monolith
   Database
   

   View Slide

9. Microservices
   Database Database
   

   View Slide

10. - Each element of functionality is a separate service
    - Split by features
    - Collection of loosely coupled services
    - Distributed system with remote calls
    Microservices
    

    View Slide

11. - Improves modularity
    - Smaller scope, easier to understand
    - Smaller codebase: easy to refactor or throw away
    - Services can be individually designed, developed, tested, and
    deployed (self-contained)
    Microservices advantages 1/2
    

    View Slide

12. - Services can be scaled separately
    - Allows technology diversity
    - Can improve reliability
    - Autonomous teams
    Microservices advantages 2/2
    

    View Slide

13. - Architectural complexity: distributed system
    - Eventual consistency
    - Operating complexity
    - Monitoring and debugging complexity
    - Not the most performant architectural pattern
    Microservices drawbacks
    

    View Slide

14. Complexity: before we moved
    

    View Slide

15. Complexity: today
    

    View Slide

16. - Loosely coupled: minimal dependence on other services
    - High cohesion: closely related functionality should stay together
    - Single bounded context: encapsulates internal details of a domain
    Don't be misled by the "micro" prefix.
    “Micro”services
    

    View Slide

17. - Avoid distributed transactions
    - Do not create service until:
    - You don’t understand the domain very well
    - When it’s not necessary
    - To prevent multiple data migrations
    Avoid: distributed monolith
    

    View Slide

18. microservice architecture is
    not a silver bullet
    

    View Slide

19. Moving to microservices
    

    View Slide

20. Moving to microservices
    

    View Slide

21. Microservices require
    organizational change
    

    View Slide

22. “organizations which design systems ... are constrained to
    produce designs which are copies of the communication
    structures of these organizations” -  M. Conway
    Conway’s law
    

    View Slide

23. - UI Team
    - Backend team
    - Database team
    Organizing around Business Capabilities
    

    View Slide

24. Organizing around Business Capabilities
    Image: https://martinfowler.com
    

    View Slide

25. - Cross functional teams
    - Full range of skills: From UX to PM
    - Each of the teams have: backend, database, ops etc.
    - Mini products inside the company
    - Teams ows the full lifecycle
    Organizing around features
    

    View Slide

26. Cross functional teams
    Image: https://martinfowler.com
    

    View Slide

27. - Distributed teams (different space / time zone)
    - Smaller teams are usually more efficient
    - More focused and specialized people
    - Separate smaller releases are possible
    Possible business reasons to move
    

    View Slide

28. Why we moved?
    

    View Slide

29. - Very different challenges:
    - time series metrics, distributed tracing
    - Mission critical features:
    - alerting
    Our product has:
    

    View Slide

30. - We wanted to have focused teams
    - Develop and ship features separately
    - Segregate failures
    - Improve reliability
    We wanted to separate these features
    

    View Slide

31. How we moved?
    

    View Slide

32. - Highly aligned, loosely coupled teams
    - Principles helps in alignment
    - Principles should be created together
    - Teams must follow it
    Service principles: why?
    

    View Slide

33. - Automation is a key in building and operating microservices
    - Setup a new service with CI jobs and monitoring
    - Easy and fast to deploy and rollback
    - Testing, Database migration etc.
    - Reliability automations: auto scaling, rate limiters etc.
    Service principles: automate everything
    

    View Slide

34. - Services should own their data
    - Separate databases per services
    - Allow polyglot persistence (different database technologies)
    Service principles: data ownership
    

    View Slide

35. - Minimize the depth of the service call-graph
    - Minimize complexity
    - Increases performance
    Service principles: complexity
    

    View Slide

36. - Only high level utility code pieces
    - Common libraries and frameworks company wide:
    - High level framework
    - Libraries for: logging, caching, service communication etc.
    - Via private npm: scoped packages
    Service principles: code sharing
    

    View Slide

37. - We don’t want to version services
    - We do version endpoints
    - We create backward compatible endpoints
    - Use feature flippers / toggles (dark launches)
    Service principles: versioning
    

    View Slide

38. - Interface for other teams
    - Good and available documentation
    - Update docs and code together
    Good to start here: https://github.com/Yelp/service-principles
    Service principles: documentation
    

    View Slide

39. Evolutionary design
    

    View Slide

40. - Rewriting your application can be expensive and dangerous
    - Need to ship features in the meantime
    - Organizational change takes time
    Evolutionary design: why
    

    View Slide

41. Proxy
    

    View Slide

42. - Client(s) specific things:
    - Authentication: Cookie headers, JWT token etc.
    - Protocols: HTTP, WebSocket etc.
    - Response format: JSON, XML etc.
    - Combining resources: from multiple services
    API Gateway
    

    View Slide

43. API Gateway
    Img: http://bits.citrusbyte.com/microservices
    

    View Slide

44. - We moved from a PaaS
    - Services were communication via public internet
    - Trusted sources (services)
    - Request signing between services
    Security: request signing
    

    View Slide

45. Security: request signing
    

    View Slide

46. - Request signing had significant CPU overhead
    - Added ~10ms to each request
    - Moved to private networking
    - One entry point: API Gateway
    Private networking
    

    View Slide

47. One entry point: API Gateway
    

    View Slide

48. Fault tolerance
    

    View Slide

49. Services fail separately
    

    View Slide

50. Graceful service quality degradation
    Service B
    Service A
    Service D
    C
    Service B
    Service A
    Service D
    C
    

    View Slide

51. Services fail separately (in theory)
    

    View Slide

52. Failure
    Service B
    Service A
    Service D
    C
    Service B
    Service A
    Service D
    C
    

    View Slide

53. Resources should be cached
    Cache
    

    View Slide

54. - On service level by our communication library
    - Automatically via response headers (with HTTP):
    - "max-age" and "stale-if-error"
    - Multi store caching with stale and failover cache
    - https://github.com/RisingStack/cache
    Caching
    

    View Slide

55. - Services fail separately
    - Critical resources should be cached
    - Rate limiters and load shedders
    - Reliability automations: auto scaling, deploy strategies
    - Architectural patterns and messaging queues can help
    Fault tolerance
    

    View Slide

56. Fault tolerant data collection - CQRS
    (Command Query Responsibility Segregation)
    read (sync)
    write (async)
    

    View Slide

57. Fault tolerant data collecting
    Queue size increasing during issue
    Queue size decreasing after issue resolved
    

    View Slide

58. Infrastructure
    

    View Slide

59. - Fast private network
    - Quick deploys and rollbacks (with immutability)
    - Zero downtime deployment
    - Flexible resources, cheap instances
    Infrastructure: wanted
    

    View Slide

60. - Started with PaaS
    - Good at beginning: rollback, zero downtime deployment etc.
    - Slow and public networking
    - Expensive and non flexible instance types
    - Evaluated: more PaaS, AWS EB, Deis (v1)
    Infrastructure: what we had
    

    View Slide

61. - We moved to Kubernetes (GCP managed)
    - Container orchestration (Docker/rkt)
    - Graceful shutdown
    - Rolling deployment
    - Self healing applications
    - Horizontal autoscaling
    Infrastructure: Kubernetes
    

    View Slide

62. Monitoring and debugging
    

    View Slide

63. Microservices producing lot’s of data
    - Logs, Errors
    - Metrics per service
    - Transactions
    - Logical connections
    - Side effects
    

    View Slide

64. Needle in the haystack
    

    View Slide

65. microservice monitoring is
    impossible for humans
    

    View Slide

66. - Google Dapper white paper
    - Transaction ID, Correlation ID
    - Commercial: Trace by RisingStack, Dynatrace, AWS X-Ray
    - OSS: Jaeger, Zipkin
    - Instrumentation standard: OpenTracing
    Distributed tracing
    

    View Slide

67. Distributed tracing: Trace by RisingStack
    

    View Slide

68. Distributed tracing: Jaeger
    

    View Slide

69. Case study: Network delay
    

    View Slide

70. Case study: Network delay
    3.8ms 564.8ms
    95th response time:
    Call depth: 1 service 2 services
    

    View Slide

71. Case study: Network delay
    network delay
    

    View Slide

72. network delay is evil
    in microservices
    

    View Slide

73. - Use private network
    - Keep your applications close to each other
    - Limit the lengths of service call chains
    - Cache everything
    Network delay is evil
    

    View Slide

74. - https://blog.risingstack.com/tag/node-js-at-scale
    - https://microserviceweekly.com/
    - https://risingstack.com/trainings
    What’s next?
    

    View Slide

75. Be confident with Node.js
    Thanks!
    Breaking Down The Monolith
    by Peter Marton, @slashdotpeter
    

    View Slide