Monolith to Microservices: A small teams journey

At Nomanini we built our first payments platform on Google Cloud when it was still only App Engine. As our product grew we built an ever-more-complex monolith. As our start-up pivoted we needed more flexibility, and wanted to experiment with newer technologies. With only three developers we made the bold decision to migrate our platform from an App Engine monolith to microservices running on Docker and Kubernetes. In this frank discussion I talk about the reasons why we made this move, the pitfalls we experienced, the benefits of the move, and why we love Kubernetes on Google Cloud.

Video: https://youtu.be/1sLg689YZtg

Transcript

1. Dale Humby, Nomanini @dalehumby South Africa Monolith to Microservices A small teams journey
3. Affordable access to payments for everyone, everywhere
6. $ 250+ $ 200 $ 150+ < $ 40 2011 2018 Cost of Android phones has radically decreased We could not scale hardware
7. Phone + Printer ≃ $ 100
9. Monolith to Microservices Location South Africa
10. tl;dr Monolith Microservices
11. Monolith Business Logic View layer Front end developers Back end developers DBA Feature development
12. Monolith Pro • Simplicity • Consistency • Ease of cross boundary refactoring Con • With Trunk based dev: Blocked sync 6 devs • Single language • Package entire project and deploy, even if small change • Scale by deploying entire project (heavy weight VMs)
13. Microservices API Business Logic Full stack team Feature development Manages State Stateless
14. Microservices Pro • Scale out independently • Experimentation • Cognitive load • Fast, independent release cycle • Seperate teams Con • Comms: Typically TCP/IP (http, graphql, rpc) instead of IPC • Debug ◦ Async ◦ Architecture complex, messaging ◦ Trace across services ◦ Visibility in to status • Dependency: large refactors, backward compat overhead, scheduling across teams • Infrastructure: Duplication of monitoring, CI/CD pipelines
15. Monolith and Microservices Initially easy Exponentially harder Steep learning curve Pays off over time
16. CI/CD Maturity Model Continuous Integration Continuous Delivery Continuous Deployment Manual everything
17. CI/CD Maturity Model Continuous Integration Continuous Delivery Continuous Deployment Manual everything Automated tests Build artifacts
18. CI/CD Maturity Model Continuous Integration Continuous Delivery Continuous Deployment Manual everything Automated tests Build artifacts Always deployable Readiness feedback Push-button deploy DevOps culture
19. CI/CD Maturity Model Continuous Integration Continuous Delivery Continuous Deployment Manual everything Automated tests Build artifacts Always deployable Readiness feedback Push-button deploy DevOps culture Auto deploy to prod Monitor, Alerting Canary releases
20. How do I start?
21. How do I start?
22. How do I start?
23. How big should microservices be? • 2 pizza team • Hold in head • Bounded contexts • Rewrite in 2 weeks • 12 people, 1 service • 6 people, 6 services • Does 1 thing well • As small as possible, and no smaller • Independently replaceable, deployable, upgradable • Heterogeneous tech
24. Too Big • Painful deployment • Hard to change • “Why does this depend on something I’ve never heard of?” • Large teams • 2-3 integration technologies (RPC, REST, Database)
25. Too Big • Painful deployment • Hard to change • “Why does this depend on something I’ve never heard of?” • Large teams • 2-3 integration technologies (RPC, REST, Database) How to fix? • Find the seams • Coupling, split on dependencies • Functional decomposition, data partition • Split, add API service
26. Too small • Integration issues • Inconvenient coupling, especially in deployments • Exposes representations, instead of behaviour
27. Too small • Integration issues • Inconvenient coupling, especially in deployments • Exposes representations, instead of behaviour How to fix? • Redraw domain boundaries • Combine services • Hide CRUD, add business oriented events • Service aggregator/orchestrator in front
28. Just right But first... • Mature DevOps • No siloed teams
29. Just right But first... • Mature DevOps • No siloed teams Business domain • High cohesion • Low coupling • Single responsibility
30. Just right But first... • Mature DevOps • No siloed teams Business domain Organisational culture • High cohesion • Low coupling • Single responsibility
31. "Organizations which design systems... are constrained to produce designs which are copies of the communication structures of these organizations." - Melvin Conway
32. Google Cloud Platform Google Cloud Platform
33. Continuum of services Most flexible Most responsibility More prescriptive, opinionated Just works
34. Cloud Functions App Engine Kubernetes Engine Google Compute Engine
35. Cloud Functions App Engine Kubernetes Engine Google Compute Engine
36. Cloud Functions App Engine Kubernetes Engine Google Compute Engine
37. Developed as only practical way to manage Google-scale compute Everything at Google runs in a container Containers at Google 37 Launch over 2 Billion containers per week.
38. Kubernetes
39. Kubernetes {...}
40. Kubernetes {...}
41. Kubernetes {...} Microservice
42. Kubernetes Microservice Proxy
43. Kubernetes Microservice Proxy
44. Kubernetes Microservice Proxy Pod
45. Kubernetes Microservice Proxy Pod Node
46. Kubernetes Node Pod
47. Kubernetes Node Microservice Pod
48. Kubernetes Node Microservice Proxy Pod
49. Kubernetes Node Microservice Proxy Pod Microservice Proxy Pod
50. Kubernetes Node 1 µS 1 µS 1 µS 4 µS 3 Node 2 µS 1 µS 3 µS 2 Node 3 µs 1 µs 1 µS 2
51. Kubernetes in Single Zone Node 1 µS 1 µS 1 µS 4 µS 3 Node 2 µS 1 µS 3 µS 2 Node 3 µS 1 µS 1 µS 2
52. Kubernetes in High Availability Node 1 µS 1 µS 1 µS 4 µS 3 Node 2 µS 1 µS 3 µS 2 Node 3 µS 1 µS 1 µS 2
53. Node 3 Kubernetes in High Availability Node 1 µS 1 µS 1 µS 3 µS 1 µS 3 µS 2 µS 1 µS 1 µS 2 µS 4
54. Node 3 Kubernetes in High Availability Node 1 µS 1 µS 1 µS 3 µS 1 µS 3 µS 2 µS 1 µS 1 µS 2
55. Node 3 Kubernetes in High Availability Node 1 µS 1 µS 1 µS 3 µS 1 µS 3 µS 4 µS 1 µS 1 µS 2 µS 2
56. Node 3 Kubernetes + Preemptible VM’s = 70% cost saving Node 1 µS 1 µS 1 µS 3 µS 3 µS 1 µS 1 µS 2 µS 2
57. Cloud Functions App Engine Kubernetes Engine Google Compute Engine
58. Best practices • Auto-scaling • Async, Eventual consistency • Timeouts, Circuit breakers to fail fast • Bulk-heads • Avoiding Cascading failures
59. Cloud Functions App Engine Kubernetes Engine Google Compute Engine
60. Firebase Cloud Functions
61. Monolith or Microservices ???? Location South Africa
62. CI time knowledge Learning curve Automation, testing
63. CI CD time knowledge Learning curve Always ready to release Automation, testing
64. CI CD Microservices time knowledge Learning curve Kubernetes: Control plane Always ready to release Automation, testing
65. CI CD Microservices Visibility time knowledge Learning curve Kubernetes: Control plane Istio: Microservice mesh Always ready to release Automation, testing
66. Google Cloud Platform Lessons learnt
67. Google Cloud Platform Lessons learnt Monolith • Increasingly difficult as team, product complexity scale • Not easy to change to another tech stack (DB’s, language, libraries)
68. Google Cloud Platform Lessons learnt Monolith • Increasingly difficult as team, product complexity scale • Not easy to change to another tech stack (DB’s, language, libraries) Microservices • Went too small: CRUD services… now consolidating • Really needed a strong CI/CD culture, even then challenging • Refactoring while live : Changing engine while aeroplane is flying • Distributed systems are challenging • Kubernetes ecosystem good, big learning curves, need larger team
69. Google Cloud Platform Lessons learnt Monolith • Increasingly difficult as team, product complexity scale • Not easy to change to another tech stack (DB’s, language, libraries) Microservices • Went too small: CRUD services… now consolidating • Really needed a strong CI/CD culture, even then challenging • Refactoring while live : Changing engine while aeroplane is flying • Distributed systems are challenging • Kubernetes ecosystem good, big learning curves, need larger team Not removing complexity – Just moving it
70. GDE Thank you dale@nomanini.com
71. Q&A South Africa