2019 Agile Australia - Strangling the Monolith

When to rewrite, containerize, refactor, or use a "data-driven" parallel run approach, along with a case study.

Transcript

1. Strangle The Monolith A Data Driven Approach Amjad Sidqi, Associate Director | Pivotal Labs David Julia, Director | Pivotal Labs
2. HARD TO CHANGE
3. The Situation The Data Driven Strangler How to Get Started
4. The Situation The Data Driven Strangler How to Get Started
5. The Scenario Additional business features Cost Estimator for medical procedures Financial Penalties for inaccurate estimations
6. Existing Architecture 3rd Party Web UI Monolith Source System 1 Source System 2 Source System 3 … Source System N
7. Peeking Inside the Monolith Main Member-Facing Web UI Account Customization Member Store Secure Messages ...etc. Member Liability Estimator SOAP Service Component Source System 1 Source System 2 Source System 3 … Source System N
8. Peeking Inside the Monolith Main Member-Facing Web UI Account Customization Member Store Secure Messages ...etc. Member Liability Estimator SOAP Service Component Source System 1 Source System 2 Source System 3 … Source System N
9. Commit to the rewrite: A new API that returns the same results & supports “tiered” networks
10. Initial approach The expert leads the way
11. The Strangler Pattern: An Iterative Rewrite Benefits of a rewrite with reduced risk, faster time to value Does require investment in the approach. Strangler Fig Hollow Inside of Strangler Fig
12. Uncertainty Complex flows create anxiety Fundamental assumptions were wrong
13. Pssst… This isn’t working
14. Strangler is great for decomposition. BUT We couldn’t know what logic to build in our new services.
15. The Data Driven Strangler The Situation How to Get Started
16. Enter The Data Driven Strangler + = ☺
17. Pass-through & Log (in prod) 3rd Party Web UI Monolith Source System 1 Source System 2 Source System 3 Project X Collect Request/Response Data 3rd Party Web UI 1 week
18. It was like turning on the lights
19. Log Both Results & Default 3rd Party Web UI Monolith Source System 1 Source System 2 Source System 3 Project X Collect Request/Response Data for Both Defaulting → No Risk of Bad Result 3rd Party Web UI Monolith Source System 1 Source System 2 Source System 3 Project X Calculation Module 2 weeks
20. Log The Deltas! Automate Analysis Web App UI Monolith Source System 1 Source System 2 Source System 3 Project X Calculation Module 3 weeks
21. We optimized for near real time feedback loops
22. Let’s focus on what matters ...
23. % Error Cases ✖ Avg. $ Diff ✖ Avg. Requests/Day = Possible Financial Impact/Day
24. Starting to strangle stable cases Started to turn off path to old system for some cases Web App UI Monolith Project X Calculation Module Source System 1 Source System 2 Source System 3 5 weeks
25. Possible Financial Impact/Day < Cost to Maintain Legacy When Then...
27. Shut down the Legacy Calculation Path Project X Only call into our new calculation module We’ve now strangled a large part of the monolith! 3rd Party Web UI Source System 1 Source System 2 Source System 3 Project X Calculation Module 13 weeks
28. This made us feel great!
29. The Data Driven Strangler The Situation How to Get Started
30. Did any of that sound familiar? Are you thinking of a rewrite?
31. Is legacy technology holding you back?
32. Data-Driven Strangler is Not a Silver Bullet
33. Options 1. Rewrite from scratch 2. Buy off the shelf 3. Do nothing 4. Containerize 5. Strangler Pattern
34. Build vs Buy → Build & Buy Is it core to your business? Somewhere you want to differentiate? Will the buy option require a lot of customization-- building logic into the system? Often, the best option is both: Build the differentiating parts, “buy” commodity components (eg don’t build your own SendGrid, don’t build Stripe, don’t build your own cloud platform).
35. When to ‘Do nothing’? • No delivery pressures • Low strategic importance • Stable enough if not touched • Opex costs under control
36. What about Containerizing? Doesn’t actually solve your problem Fragmented business rules Painful deployment process Slow to augment Technical Debt Hard to test
37. When should you rewrite? Maturity/Traction of product • Original product was way off the mark, didn’t achieve goals (eg no user adoption).
38. When should you rewrite? Maturity/Traction of product • Original product was way off the mark, didn’t achieve goals (eg no user adoption). • Original product does not have traction
39. When should you rewrite? Maturity/Traction of product • Original product was way off the mark, didn’t achieve goals (eg no user adoption). • Original product does not have traction • Significant deviation from original intent of product, going after a new market
40. When should you rewrite? Maturity/Traction of product • Original product was way off the mark, didn’t achieve goals (eg no user adoption). • Original product does not have traction • Significant deviation from original intent of product, going after a new market • Technology holding you back (Mainframe, Visual Basic overly-customized SFDC or AEM)
41. When should you rewrite? Maturity/Traction of product • Original product was way off the mark, didn’t achieve goals (eg no user adoption). • Original product does not have traction • Significant deviation from original intent of product, going after a new market • Technology holding you back (Mainframe, Visual Basic overly-customized SFDC or AEM) • You can redefine the business process around the new system.
42. When to use the Strangler Pattern • Well established product with significant user base • A significant risk to revenue streams • Lots of necessary complexity in your existing product (eg complex regulatory compliance rules) • You don’t know the business rules in the existing system
43. Learnings/Takeaways • This sounds technical but don’t compromise User Centred Design • An opportunity to remove complexity • Get laser focused on what really matters 80:20 • Don’t rebuild like for like • When rewriting take an iterative approach
44. How do you do this in your organization? Start Small Put together a business case around a subset of the capabilities that will deliver value over a matter of months, not years. Frame it as a “no regrets” move with near term benefits. Quantify Outcomes Establish a baseline and measure against it (dev cycle time is good, but cost/revenue/acquisition metrics are even better) Use one win to build momentum for the next By starting small, you can prove out the process and build support to keep going. Once you have a first win, a technical foundation, and understanding of the system, you can “double down” and scale the effort.
45. With the support of Pivotal!
46. email: djulia@pivotal.io Twitter: @DavidJulia email: asidqi@pivotal.io We Love Feedback What would you like to hear more about? What questions do you still have? And… We are hiring Get in Touch!