DevOpsDays Cuba 2017: Evolving Architecture And Organization Together - Lessons From Google And EBay

Author: Randy Shoup
Summary: The “right” architecture and organization depends on the size and scale of your company. The only constant is change, and what works for 5 engineers does not work for 5000. Based upon lessons from Google and eBay, learn how to evolve both technology and organization together successfully.

This presentation is based on many hard-won lessons by the speaker, who led large-scale engineering teams at Google and eBay, but also co-founded a tiny startup and tried (unsuccessfully) to apply the same techniques. This session hopes to help others from making the same mistakes by introducing the concept of “Minimal Viable Architecture”. It outlines the common architectural evolution of a company or project through the search, execution, and scaling phases, and discusses the appropriate technologies, disciplines, and organizational structures at each phase.

Transcript

1. Evolving Architecture and Organization Together Lessons from Google and eBay Randy Shoup @randyshoup linkedin.com/in/randyshoup
2. Scalable Software Development Organization Technology Culture Process
3. Scalable Software Development Small Teams Microservices DevOps Continuous Delivery
4. Scalable Software Development Small Teams Microservices DevOps Continuous Delivery
5. Conway’s Law • Organization determines architecture o Design of a system will be a reflection of the communication paths within the organization • We can engineer the system we want by engineering the organization
6. Small “Service” Teams • Amazon “2 Pizza” Teams o No team should be larger than can be fed by 2 large pizzas o Typically 4-6 people • Aligned to Business Domains o Clear, well-defined area of responsibility o Single service or set of related services o Minimal, well-defined “interface” @randyshoup linkedin.com/in/randyshoup
7. Full-Stack Teams • All disciplines required for the team’s function o Design o Development o Quality and Performance o Maintenance o Operations • Depend on other teams for supporting services, libraries, and tools @randyshoup linkedin.com/in/randyshoup
8. “DevOps is a reorg” -- Adrian Cockcroft, Netflix
9. Scalable Software Development Small Teams Microservices DevOps Continuous Delivery
10. “Tell us how you did things at Google and eBay.” “Sure, I’ll tell you, but you have to promise not to do them! [… yet]”
11. Architecture Evolution • eBay • 5th generation today • Monolithic Perl à Monolithic C++ à Java à microservices • Twitter • 3rd generation today • Monolithic Rails à JS / Rails / Scala à microservices • Amazon • Nth generation today • Monolithic Perl / C++ à Java / Scala à microservices
12. No one starts with microservices … Past a certain scale, everyone ends up with microservices
13. Getting to rearchitect a system is a sign of success, not failure.
14. If you don’t end up regretting your early technology decisions, you probably over- engineered.
15. Microservices • Single-purpose • Simple, well-defined interface • Modular and independent A C D E B
16. Microservices • Single-purpose • Simple, well-defined interface • Modular and independent • Isolated persistence (!) A C D E B
17. Microservices Each unit is simple Independent scaling and performance Independent testing and deployment Can optimally tune performance (caching, replication, etc.) Pros Many cooperating units Many small repos Requires more sophisticated tooling and dependency management Network latencies Cons
18. Why Rearchitect? • Velocity o Teams step on each others’ toes, and can no longer develop independently o Difficult for new engineers to be productive • Scaling o Vertical scaling of the monolith no longer works o Parts of the system need to scale independently of others
19. Why Rearchitect? • Deployment o Parts of the system need to deploy independently of others o Monolithic release is too slow, too complicated, too risky
20. “The only thing a Big Bang migration guarantees is a big *Bang*.” -- Martin Fowler
21. Carving up the Monolith • Look for (or create) a “seam” in the monolith o This is often the hardest part (!) • Wall it off behind an interface • Write automated tests around the interface • Replace implementation with an independent component • è Rinse and Repeat
22. Scalable Software Development Small Teams Microservices DevOps Continuous Delivery
23. You Build It, You Run It. -- Werner Vogels
24. Ownership • End-to-end Ownership o Team owns service from design to deployment to retirement • Responsible for o Features o Quality o Performance o Operations o Maintenance
25. Cross-Functional Collaboration • Open communication o Individuals encouraged to work directly with each other o Prefer informal cooperation over formal channels • Best decisions made through partnership o Agreement on goals and priorities makes it easier to agree on tactics o Given common context, well-meaning people will generally agree o “Disagree and Commit”
26. None of us is as smart as all of us. -- Japanese proverb, as quoted by Bob Taylor
27. Scalable Software Development Small Teams Microservices DevOps Continuous Delivery
28. Continuous Integration • Small incremental changes o Faster feedback loop o Easy to understand, code-review, test, roll back • Larger changes are much riskier o Risk of code change is nonlinear in the size of the change o Ex. Initial memcache service submission • Main code branch always shippable o Increased rate of change while reducing risk of change
29. Continuous Delivery • More solid systems o Release smaller units of work o Faster to repair, easier to diagnose o Small change to roll back or roll forward • Enables experimentation o Small experiments and rapid iteration are cheap • Enabled by o PaaS, containers, cloud
30. Scalable Software Development Small Teams Microservices DevOps Continuous Delivery
31. ¡Muchas Gracias! • @randyshoup • linkedin.com/in/randyshoup