Splitting the Monolith

The story of a monolithic application that has been modified into a distributed system.

Transcript

1. Splitting the Monolith Enrique Zamudio @chochosmx An unfinished journey towards microservices
2. Who? • Working as a programmer since 1994 • Working with Java since 2000 • Java Champion in 2015 • Working on this project since 2017
3. The monolith • PHP-based system • Mostly web-based • Incoming calls are endpoints • cronjobs are methods in the same codebase, run from CLI • Redis as DB • Running on traditional servers
4. • Redis as DB The monolith
5. Trade Engine Emails Notifications Deposits (crypto/fiat) Placing orders Withdrawals Confirm withdrawals/deposits Admin
6. Web server Web server Web server Web server cron server
7. Current situation • PHP is hard to maintain and test • Current architecture not very scalable • Particularly with manual setup • Anticipated growth
8. The project • Migrate data to RRRDBMSEUR
9. The project • Migrate data to RDBMS • PostgreSQL
10. The project • Migrate data to RDBMS • PostgreSQL • Migrate logic to services in Java (the split) • Put those services in containers • Manage those containers
11. Migrating data user:1 = { name:John, last:Doe, email:john@doe.com, active:1, suspended:0 } user_id first_name last_name email status 1 John Doe john@doe.co m active 䡾
12. Migrating data user_model.php UserModel.java UserModelImpl.java SELECT * FROM user WHERE ... UPDATE user SET status=? WHERE...
13. Migrating data user_model.php user-service.jar
14. Migrating data user_model.php user-service.jar
15. Migrating data user_model.php user-service.jar Protocol Buffers
16. Migrating data user_model.php trade_model.php deposit_model.php order_model.php user-service.jar trade-service.jar deposit-service.jar order-service.jar
17. Migrating data user_model.php trade_model.php deposit_model.php order_model.php db-service.jar
18. Migrating data user_model.php trade_model.php deposit_model.php order_model.php db-service.jar
19. Why? • Moving to PostgreSQL is a priority • ACID • Querying • Low impact on PHP codebase
20. Trade Engine Emails Notifications Deposits (crypto/fiat) Placing orders Withdrawals Confirm withdrawals/deposits Admin
21. Emails Notifications Withdrawals Confirm withdrawals/deposits DB Proxy Deposits (crypto/fiat) Trade Engine Placing orders Admin
22. Unexpected growth 2017 • 18K users in January • 200K users in November 0 50 100 150 200 January May June November
23. Trade Engine Placing orders Login Admin
25. Trade Engine Placing orders Login Admin Order queue
26. Web server Web server Web server Web server cron server
27. Web server cron server DB Proxy Notifications Mailer Banks
28. Microservices!
29. Microservices! ✘
30. Distributed system • Dedicated services calling each other • Kubernetes helps a lot • Redundancy, deployment, no downtime on upgrades, etc • Monitoring is trickier – observability • RDBMS is a SPoF • Tradeoffs are worth it • DB proxy service is another SPoF, but it's temporary (hopefully)
31. What's next? • Finish migrating data to PostgreSQL • Leave Redis just for queues and caching • Start migrating higher-level logic to Java • This eliminates low-level DB calls from PHP • Which eliminates the need for models in PHP • Which eliminates the need for the DB Proxy service • Services talking directly to the database
34. ¿?