Legacy Monolith to Microservices

I will talk about the chaotic and vibrant world of microservices at Dubizzle, how we verticalized different sections of Dubizzle and how we architected our systems such that stability to one vertical doesn't affect another?

Agenda:
- Meet Dubizzle Monolith
- Why we decided to move to Microservice architecture?
- Mistakes we made while moving to Microservice and how we tackled them
- Challenges we faced when we moved to Microservices and how we tackled them
- Dubizzle's Strategy towards improving observability in Microservices
- What advantages did we get when we moved to Microservices?

Transcript

1. Smit Thakar (@mr_geek_smit) | Software Engineer Legacy monolith to microservices
2. • Software Engineer at Dubizzle • Founder of Zypher • Open source organizations I have been part of ◦ Gluster ◦ Ngui • Former Chapter Lead of GDG Gandhinagar • Github: github.com/smitthakkar96 • Linkedin: linkedin.com/in/smitthakkar96 • https://zypher.tech README.md
3. 16 Million Monthly Visits 3.5 Million MAUs 20 ads placed per minute
5. “ Empower teams to sustainably deliver value
6. 2005 2010 2011 2017
7. Source: http://web.archive.org/web/20051214234319/http://www.dubizzle.com:80/
8. 2005 2010 2011 2017
10. 2005 2010 2011 2017
11. Source: http://web.archive.org/web/20110530101439/http://www.dubizzle.com:80/
12. if site_profile.country_site.id == 4: # UAE facebook_url = 'http://www.facebook.com/dubizzle' elif site_profile.country_site.id == 7: # Egypt facebook_url = 'http://www.facebook.com/DubizzleEgypt' elif site_profile.country_site.id == 10: # KSA facebook_url = 'http://www.facebook.com/DubizzleKSA' elif site_profile.country_site.id == 17: # Kuwait facebook_url = 'http://www.facebook.com/pages/Dubizzle-Kuwait/133537483367816' elif site_profile.country_site.id == 19: # Qatar facebook_url = 'http://www.facebook.com/pages/Dubizzle-Qatar/144285568954101' elif site_profile.country_site.id == 21: # Bahrain facebook_url = 'http://www.facebook.com/pages/Dubizzle-Bahrain/176168439062742' elif site_profile.country_site.id == 23: # Oman facebook_url = 'http://www.facebook.com/pages/Dubizzle-Oman/133593556694431' elif site_profile.country_site.id == 25: # Syria facebook_url = 'http://www.facebook.com/DubizzleSyria' elif site_profile.country_site.id == 27: # Jordan facebook_url = 'http://www.facebook.com/DubizzleJordan' elif site_profile.country_site.id == 29: # Lebanon facebook_url = 'http://www.facebook.com/DubizzleLebanon' elif site_profile.country_site.id == 31: # Algeria facebook_url = 'http://www.facebook.com/dubizzle' elif site_profile.country_site.id == 33: # Libya facebook_url = 'http://www.facebook.com/dubizzle' elif site_profile.country_site.id == 37: # Morocco facebook_url = 'http://www.facebook.com/pages/Dubizzle-Morocco/112728338792888'
14. Source: http://docs.railsbridge.org/job-board/crud_and_resourceful_routing
16. Presentation Layer Logic Layer Data Layer Feature X
17. Monolith Ads Users Images Packages Payments Mobile API CRM Wrapper Chat Service
19. 2005 2010 2011 2017
20. “ https://twitter.com/malk_zameth/status/654710109214371841
21. The Guiding Principles ★ Low Coupling, High Cohesion ★ No service can depend on another for uptime, functionality or data availability ★ One transaction can span at most one service ★ No vertical can impact the stability of another ★ Bounded contexts are defined by business realms not CRUD APIs ★ Choreography over orchestration ★ Events are a first class citizen ★ Analytics and Logs are the only source of debugging information ★ Rate limits, data privacy, authentication and authorization are mandatory for all services ★ Every service must be auto-deployable independently, after passing unit, functional, load and security tests
22. Transactions Payments Products Ads Users Ads Users Report Ad Favourites Moderation Ad creation context Ad consumption context ★ Low Coupling, High Cohesion
23. Message Bus ★ No service can depend on another for uptime, functionality or data availability user_created db Bus events handlers Send emails job ad_created Chat Notification Service ad_published db Bus events handlers Send alert job Ad Notification Service user_created db Bus events handlers ad_updated Listings Service REST API ad_published ad_created Monolith user_created ad_created ad_updated
24. ★ One transaction can span at most one service
25. “Any organization that designs a system (defined broadly) will … produce a design whose structure is a copy of the organization's communication structure.” - Melvin Conway Source: Bonkers World ★ No vertical can impact the stability of another “If you have four groups working on a compiler, you’ll get a 4-pass compiler” - Eric S Raymond
26. ★ Bounded contexts are defined by business realms not CRUD APIs Monolith Ads Users Images Packages Payments Mobile API CRM Wrapper Chat Service vs Property Creation Service Property Search Service Property Ads Service
27. today
28. Property Creation Context Property Consumption Context Horizontal Services Gateway Services Motors Creation Context Motors Consumption Context Jobs Creation Context Jobs Consumption Context Message Bus
29. Observability
30. ▫ We use newrelic for application performance monitoring, Error reporting, analytics and alerts ▫ Newrelic offers us an ability to do distributed tracing where synchronous transactions are happening ▫ New relic alerts are routed to pagerduty which raises escalation to the owners of the respective components APM and Error reporting
31. ▫ All the application and system logs are propagated to elasticsearch via syslog and logstash. ▫ Some alerts are also configured on top of the logs we get and are send to the owners of the component via pagerduty and Slack Logging
32. Infrastructure monitoring SRE Team (on call) Devs (oncall)
33. Crons and Background tasks
34. Queue Alarms
35. Cron Jobs
36. Meet telemetry
37. Measure timelag for messages with telemetry
38. Measure execution time for messages with telemetry
39. Monitor the size of messages with telemetry
40. Monitor throughput with telemetry
41. Monitor crons using telemetry
42. Telemetry also provides us low level APIs to record application specific metrics so that we can create dashboards with those metrics using telemetry. Application specific metrics
43. Record Point
44. Record Point
45. Bounded Context Message Bus Eventual Consistency Event-based architecture Support Tooling Developer Training
46. Questions?
48. We are hiring! • Python Engineers • Full Stack Engineers • Mobile Engineers Let’s get in touch on smit.thakkar@dubizzle.com