Michael DeHaan - Keynote - Speaking for the Dead: Is "Waterfall" and "Monolithic" Actually Good?

Transcript

1. Speaking for the Dead: Is "Waterfall" and "Monolithic" Actually Good? MICHAEL DEHAAN @LASERLLAMA - 9.6.2016
2. PRELUDE u In Early 2012, I created a project called Ansible. u Named after Ender’s Game u Orson Scott Card wrote “Speaker for the Dead” as a sequel
3. PRELUDE u An honest, instructional, and empathetic eulogy. u I began writing software full time at IBM in 2001 u I’m also not dead. u We’ve declared Waterfall and Monolithic Architecture dead. u Let’s talk about that.
4. TODAY u “Waterfall” vs “Agile” u “Monolithic” vs “Microservices” u Labels, spin, and assumptions u An appreciation of the very recent past
5. First: A Theory u We get bored at work. u We tend to like a challenge. u We hastily invent new things, that are not always better. u New people come into the field, and haven’t had the experience of older ways. u We get caught up in spin and “grandfalloons” (see Cat’s Cradle). u We assume there is a true way. There is no true way.
6. Labels Apply Spin Good Connotation Bad Connotation Affordable Care Act “Obamacare” USA PATRIOT ACT Agile Waterfall Microservices Monolithic Big Design Up Front Enterprisey
7. TRUE NAMES u “Ansible” was originally coined by Ursula LeGuinn. u In Earthsea, she writes of “True Names” u We must know things as they really are u “Waterfall” and “Monolithic” – do we know them, or just their given names?
8. “Waterfall” and “Iterative Development” u In the early 2000s, waterfall was the norm. u Requirements – Design – Implementation - Test u Engineering was engineering – greater responsibility, less hand holding. u Plenty of time to design solutions and research.
9. Waterfall and Iterative Development (Cont) u Marketing Requirements Documents u Engineering Requirements Documents u Questions, comments, documented mutual agreement u Status meetings usually once a week u Accurate view of strengths of various team members u Accurate view of dependencies between tasks, task order
10. The Gantt Chart Why don’t we do these anymore?
11. Agile (especially Scrum) u “Big Design Up Front” is labelled “bad”. Is it? u Engineers are viewed as interchangeable. u Estimates are encouraged to not be time based u Scope is more unpredictable. u Overhead and status-focus is increased u Practices/Games make work feel like Kindergarten - “fist of five”, “planning poker” u Crunch time becomes constant
12. Even In Lightweight Agile Systems u Task inter-relationships ignored u Design time is shortened u Estimates are forced down u Requirements are more vague u No mutual sign-off. u Testing can suffer.
13. A Middle Ground u Do share design u Do have an accessible product owner, but a good one u Know hard requirements vs soft requirements. u Lightweight agile and Iterative Development can be closer than we think u Test driven development and test automation = super important.
14. What’s So Bad About A Monolith?
15. Monolithic Application Architecture Load Balancer app app app Message bus tasks tasks tasks
16. Microservices
17. Professor Plum with the Metadata Service in us-east-1
18. Example Worldwide Microservices Shop u ~150 services u Many thousands of Jenkins builds per day u 8 Amazon regions (dev, stage, many prod regions) u ~1000s of EC2 instances u Could be modelled in ~50 EC2 instances in monolithic models u Jenkins configurations drift u Security enforcement drifts u Code sharing suboptimal u “DevOps” writing lots of custom code to manage it.
19. Inefficiency Kills Polar Bears
20. Lessons Learned From Monoliths u Increased ability to develop on one laptop u Interfaces can benefit from static typing vs design-by-contract u Very small numbers of different server/image types in production u Local function calls are much faster than RPC
21. Lessons Learned From Monoliths (Cont) u Not many instances sitting around with 5% utilization u No need for “containers” to get around above utilization problem u Greatly simplified debugging u Easy code sharing u Very simple version upgrades
22. Microservices u Was created as a technical solution to a people problem u Design by contract is effective u but increases test, ops, and debugging demands u Versioning increases need for communication and development formality u Avoids controversial design debates by … avoiding design u Potentially reduces code sharing, increases potential for NIH u Creates an infinite sea of new management challenges.
23. Hmm. Metal VM Container Cloud SW Container Container App Not Simpler.
24. Lessons Learned From Microservices u Design by contract enables: u Language/tech choice u Documenting of interfaces u An API focus u Better decoupling from UI
25. Conclusions u Labels cloud thought. u Monolithic architectures ARE more efficient. u Agile can be less efficient. u Microservices attempts to solve a people problem by introducing new technical and people problems. u There’s something to learn from everything. u Try to separate the labels from the true meanings of things.