Transforming from monolith to microservices to scale a growing organization

The timing and decision to migrate from a monolith to microservices can dramatically affect your business's ability to scale down the road. This talk will discuss considerations to take into account when evaluating the tradeoffs between maintaining a monolith or migrating to a micro-services based architecture.

Transcript

1. Transforming from monolith to microservices to scale a growing organization By Genevieve Krzeminski - Senior Engineering Manager @ Quizlet
2. What do you want to get from this talk? • How to implement, architect, or design microservices • See how it has been done • How to migrate from a monolith to microservices • Networking! • Came to learn or gain “knowledge” • Learn “the secrets to the universe”
3. Genevieve Krzeminski, Sr. Eng Manger @Quizlet At Quizlet, we’re helping students/teachers practice and master whatever they are learning - world wide! My team is focused on spreading Quizlet use worldwide - and I’m hiring! Contact: genevieve@quizlet.com @astraknots @tealeafreaderco
4. Conway’s Law “Conway's law is an adage named after computer programmer Melvin Conway, who introduced the idea in 1967. It states that organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations.” -Wikipedia
5. Monolith to Microservices(ish) How HomeAdvisor’s development & services transformed as the company grew by 200% (over about 5 years)
6. ...in the beginning, there was a monolith
7. The monolithic landscape... Development People • 2 groups of developers/qa: “Internal” and “External” (30 ppl) Code • Monolithic webapps Product • 2 primary Consumer products (web) • CRM (supporting Sales, Customer/Business Care) • Service Professional website Goals • Grow development organization • Expand product offerings • Speed up delivery • Scale to support more customers
8. Emerging pain points Fragile codebase Frustrated developers • Changes to one part break something unexpected • Legacy code lacking unit tests • No automated tests Slowing dev/qa Environmental challenges • Builds taking longer • Code conflicts increasing • Manual QA regression exceeding capacity Sluggish feature development Technology setbacks • Core logic buried in the database • Unreadable, legacy code • Large refactors needed
9. Solution? Microservices! • Separate code streams for development - less code clashing • Build brand new features faster • Scale deployments • Draw more distinct lines of ownership • …. And it will make us seem really cool!
10. Ok, but how do we break apart the monolith?
11. Identify Identify high level domain objects Organize Organize common code into libraries, start to pull apart Graph Try to understand the relationship between domain objects Refactor Isolating those domain objects in the code Copy/Clean Duplicate main code library into copies for each webapp; clear out dead code
12. The code Webapp Core library Core library Webapp CRM Webapp Consumer Webapp Service Professional Consumer Webapp Service Professional Webapp CRM
13. The teams Internal CRM External Web Consumer Web Pro Internal CRM External Consumer, Pro Feature New
15. Encapsulate Move business logic out of db stored procs into java layer Extend Implement new, extensible framework for expanding product offerings Experiment Rewrite core, legacy backend and frontend code. Evolve AB test to safely transition from the old to the new code without slowing down Expedite Fast iterations on now isolated business logic code
17. Ideal, standalone, microservices • True “micro” service • Standalone • Brand new code • A service the webapp could reach out to • Utilizing Elasticsearch and Kafka
18. Mini-monoliths • Own a large chunk of the domain, or parts of multiple domains • Still tied to functionality in main webapp • Accessing the database, Elastic Search • Dependent on webapp and other microservices
20. Compounding Challenges Large domains How to break them across microservice boundaries? • When the user flow and lifecycle of these domain objects should span multiple microservices Dependencies How to Identify upstream and downstream dependencies? • Knowing which chain of services your service depended on Configuration How to coordinate and communicate with other teams’ services? • Keeping all of the configuration setting straight and properly wired
21. Solution Microservice archetype template • Stable sandbox environments • Service Discovery ◦ Komodo ▪ Health checks • Zipkin - route tracing • Log aggregation - Sumo And...
22. Organizational Structure • Part of an organization where the tech team grew from 30 to ~200 (developers/qa) • Team structure evolved from “internal” and “external” groups to 20+ AGILE delivery teams of 4-10 members
23. “Product” Organization • Began with 2 primary customer program options -> Grew to 40+ variations • Internal CRM enabled Sales, Consumer and Professional Care • Consumer product variations increased from 3 primary use cases to 10+
24. More teams, more problems/pain Communication • How do teams communicate and coordinate changes? • Optimizing the efficiency of meetings Division • How do we divide and align teams? ◦ With products? ◦ With projects? Stability vs Change • Increasing amounts of “re-orgs” to address
25. credit:RedHat ish
26. “Monolithic” Sweater Raglan sweater • Knitted top down, all in one piece • Can change design as you go • No assembly required • Limited decisions about how it is constructed or the end shape
27. “Microservices” Sweater • Knitted in pieces and seamed together • More options for how it is constructed • Takes more effort to ensure the pattern (and tension) is consistent across all pieces • Requires thoughtful assembly • Parts constructed in isolation but must come together as a whole
28. Seams of Integrity • Communication between parts is an art form (some guidance or methodology, but situation dependent) • Need to have high integrity (holds it together, otherwise would fall apart) • It’s more work
29. Extras & Embellishments • 3rd party systems • Microservices written in different languages or with extremely distinct and truly isolated functionality ◦ Still have to mesh with the whole
30. A last little plug... If you enjoyed my talk today, consider looking for the talk I am going to submit to Denver Startup Week this year, and voting for it! • Knitting to coding (title will probably change….) Follow me: @tealeafreaderco or @astraknots
31. Questions?