Gall's Law applied to Distributed System - Ruby Conf Brazil 2021

https://online.rubyconf.com.br/

Gall’s Law states that any complex system that works, evolved from a simple system that also worked. I'll show you practical examples of how to start small and let complexity grow as needed. Sharing how we identified the hints for improvement, and the techniques and tools that supported it.

Transcript

1. Gall’s law applied to distributed systems by Fabricio Buzeto
2. In love with code since 2002 Entrepreneur since 2005 Researcher since 2008 Startuping since 2011 CTO @ bxblue Fabricio Buzeto a.k.a Fabs
3. 📜 Gall’s Law
4. “ A complex system that works is invariably found to have evolved from a simple system that worked. “ Gall’s Law John Gall - 1975 “Systemantics: How Systems Really Work and How They Fail.”
5. How complex systems evolve? © The Pokémon Company™
6. How complex systems evolve? © 2019 - 2021 Qbliviens
7. “ A complex system that works is invariably found to have evolved from a simple system that worked. “ Gall’s Law John Gall - 1975 “Systemantics: How Systems Really Work and How They Fail.”
8. 1. Any living cell with 2 or 3 neighbors survives 2. Any dead cell with 3 living neighbors comes to life 3.Any remaining living cell dies Game of Life John Conway - 1970
9. Simple rules can lead to complex behaviors
10. “ A simple system may or may not work. “ Gall’s Law Continuing John Gall - 1975 “Systemantics: How Systems Really Work and How They Fail.”
11. What “works” means?
12. What’s “working” A software that works, is a software that fulfil its purpose What a working software does?
13. What’s Purpose For what? For what purpose? - Business ← - Politics - Learning
14. Who needs it? - The User ← - The Sponsor - The Company - The Team Whos is the Client? Who we need this?
15. “ A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system. “ Gall’s Law Continuing John Gall - 1975 “Systemantics: How Systems Really Work and How They Fail.”
16. 🌱 Starting simple
18. bxblue the first years Would users want to transfer their balance?
19. bxblue the first years Unbounce Google Docs
20. bxblue the first years Unbounce Google Docs Rails
21. bxblue the first years Unbounce Pipedrive Rails Google Spreadsheets
22. bxblue the first years MongoDB Pipedrive Rails
23. bxblue the first years Unbounce Google Docs Unbounce Google Docs Rails Unbounce Pipedrive Rails Google Spreadsheet s MongoDB Pipedrive Rails Would users want to transfer their balance? How do loans are sold? How can we convert better? How can we control it properly?
24. The monolith Why? The Utah monolith - November 2020
25. Simple to Develop Simple to Test Simple to Deploy Simple to Reuse Simple to Scale* The monolith Simple like that Chris Richardson https://microservices.io/patterns/monolithic.html
26. Hard to Scale: - Tests - Team - Deploy - Stack - Changes The monolith Drawbacks Chris Richardson https://microservices.io/patterns/monolithic.html
27. bxblue as scale +15 MongoDB Pipedrive Rails
28. bxblue as scale +15 MongoDB Pipedrive Rails Sidekiq
29. Automate your tools - Tests - Code Quality - Deploy - Monitoring Don’t trust yourself Trust the machine
30. Automate your tasks Tests Code Quality Deploy Unit Tests Commons Integration Coverage Linter Code Quality CI/CD Monitoring Source Control Cloud Pipeline Errors Servers Logs Speed Security Journey
31. Automate your tasks Tests Unit Tests Commons Integration Speed Ensure features works as expected Tests contexts are reusable and repeatable Ensure features work together Ensure feedback is contant
32. Code Quality Automate your tasks Coverage Linter Code Quality Security Ensure tests cover enough of the codebase Let the machine ensure style Let the machine ensure simple rules Let the machine ensure simple breaches
33. Deploy CI/CD Source Control Cloud Pipeline Automate your tasks Ensure it works before deploying Good code is produced in groups Deploy cadence leads to faster feedback
34. Monitoring Errors Servers Logs Journey Automate your tasks Be aware of where, when and what is breaking Be aware of how your infrastructure is reacting Have the proper means to investigate issues Follow your users in order to help them
35. Automate your tasks - bxblue Tests Code Quality Deploy rspec factorybot VCR simplecov rubocop reek LayerCI Monitoring Github Heroku Sentry New Relic Logentries Knapsack breakman Posthog
36. bxblue as scale Rails User Journey Content marketing Landing Pages ETL Event handling Data Analysis ...
37. 🤓 How to de-complicate things?
38. They run on multiple servers. They manage data. Distributed Systems What does it mean for a system to be distributed? Unmesh Joshi - 2020 https://martinfowler.com/articles/patterns-of-distributed-systems/
39. Simple, self-contained, loosely coupled, single focused, services Microservices Miniservices as well Chris Richardson https://microservices.io/patterns/monolithic.html
40. Large, self-contained, Monolith Supported by, small single focused, problem specific, services The Citadel Monolith Town DHH - 2020 https://m.signalvnoise.com/the-majestic-monolith-can-become-the-citadel/
41. Simple, self-contained, context-focused, multi-purpose services Macroservices Hungry Microservices Nitesh Gupta - 2020 https://dzone.com/articles/micro-service-mini-service-and-macro-service
42. bxblue the later years +15 MongoDB Pipedrive Rails Sidekiq
43. Rails bxblue the later years +15 MongoDB Pipedrive Sidekiq
44. Rails bxblue the later years +15 MongoDB Pipedrive Sidekiq backoffice MongoDB Sidekiq 5
45. Rails bxblue the later years +15 MongoDB Pipedrive Sidekiq backoffice MongoDB Sidekiq 5 anti-fraud MongoDB Sidekiq 7
46. Rails bxblue the later years +10 MongoDB Pipedrive Sidekiq
47. 1. Almost all the successful microservice stories have started with a monolith that got too big and was broken up 2. Almost all the cases where I've heard of a system that was built as a microservice system from scratch, it has ended up in serious trouble. Monolith First Martin Fowler - 2015 https://martinfowler.com/bliki/MonolithFirst.html
48. 1. A more common approach is to start with a monolith and gradually peel off microservices at the edges. 2. Another route is to start with just a couple of coarse-grained services, larger than those you expect to end up with. Sacrificial Architecture Often the best code you can write now is code you'll discard in a couple of years time Martin Fowler - 2014 https://martinfowler.com/bliki/SacrificialArchitecture.html
49. The Stonehenge
50. Simple, self-sufficient, context-focused, service-enabled, applications Stonehenge One step further onto Macro Services
51. Self-sufficient applications They work by themselves. Stonehenge One step further onto Macro Services
52. Context-focused applications They handle a single context very well Stonehenge One step further onto Macro Services
53. Service-enabled applications They are available to integrate and scale with others Stonehenge One step further onto Macro Services
54. Every application has an inherent amount of complexity that cannot be removed or hidden Complexity has to go somewhere Law of conservation of complexity Larry Tesler ~ 1980 http://www.designingforinteraction.com/tesler.html
55. ⚠ Warning
56. A distributed monolith, due to poorly designed services, suffer from inefficient reuse, low cohesion, high coupling, and weak service contracts. With the Distributed Monolith Be Careful José Neto - 2021 https://medium.com/quintoandar-tech-blog/how-we-planned-fo r-monolith-decomposition-at-quintoandar-363f160b4427
57. 30% of files 25% of classes 5%~10% of methods Don’t let the zombies bite you Dead Code Simone Romano and Giuseppe Scanniello - 2018 http://www.cs.wm.edu/~denys/pubs/TSE'18-DeadCode.pdf
58. 70% of Companies will fail until 20 months after last fundraising* * for seed round or crowdfunded companies death rate is 97% Dead Products CBInsights - 2020 https://www.cbinsights.com/research/startup-failure-post-mortem/
59. 📝 Summing up
60. A simple system may or may not work. A complex system designed from scratch never works and cannot be patched up to make it work. Gall’s Law A complex system that works is invariably found to have evolved from a simple system that worked. John Gall - 1975 “Systemantics: How Systems Really Work and How They Fail.”
61. Automate your tools - Tests - Code Quality - Deploy - Monitoring Don’t trust yourself Trust the machine
62. “The simplest solution is usually the best” It’s all Distributed Systems Decision making is hard Things will change Why Stonegenge? friar William of Ockham 1287–1347 Occam’s razor
63. by Fabricio Buzeto about.buzeto.com @nukdf bxblue.com.br/vagas Gall’s law applied to distributed systems 󰢏 Thanks
64. bxblue.com.br/vagas We’re hiring