Go Faster - Fred George - Agile SG 2016

Presented in Agile Singapore 2016 conference

Transcript

1. Go Faster Fred George fredgeorge@acm.org @fgeorge52 Copyright © 2015 by Fred George. All rights reserved. 1
2. Fred George Copyright © 2015 by Fred George. All rights reserved. 2 • Technologist • Computer networks – 70’s • Token Ring LAN – 80’s • GUI’s – late 80’s • OO – late 80’s • Agile – late 90’s • MicroServices – mid-2000’s • Programmer • Since 1968 (Basic) • 65,000 hours experience • 70+ languages • Computer Science Degree 1973 • Manager • 17 years IBM • Business degree, MIT Sloan School 1986 • Product Owner, IBM • VP, ThoughtWorks • Co-founder, Outpace (Silicon Valley) • Senior Advisor to 3 tech companies
3. Fred George Copyright © 2015 by Fred George. All rights reserved. 3 Consulting Roles • Change Agent • Disruptor • “Hand grenade I am throwing into development” • CTO describing Fred to his Vice President
4. Industry “Hot” Topics Copyright © 2015 by Fred George. All rights reserved. 4 MicroServices Cloud Docker Dev/Ops Full-stack developer Agile Lean startup Cassandra Event bus MVP (minimum viable product Programmer anarchy No-SQL One Hacker Way
5. Why: Competition is coming Copyright © 2015 by Fred George. All rights reserved. 5 • Enablement through technologies • Cloud computing • Ubiquitous, high bandwidth • Languages (and supporting frameworks) • Recognition of business opportunities • Silicon Valley innovators as role models • Accelerating business needs • Few inhibitors for global competitors • Reduction of entry barriers for niche competitors • Micro-marketing to specific customer niches
6. Why: New Problem Domains: The Cynefin Framework Copyright © 2015 by Fred George. All rights reserved. 6 Simple Complicated Complex Chaotic Cause a Effect Cause aaa Effect Effect? Cause? Cause ? Effect Disorder
7. How Fast Can You Go? Copyright © 2015 by Fred George. All rights reserved. 7
8. Iteration Length in Agile Age Copyright © 2015 by Fred George. All rights reserved. 8 1 mo. 2015 2000 2005 2010 3 wks. 2 wks. 1 wk. 1 day XP XP XP XP SCRUM* *2-3 months
9. Project Delivery Cycles Copyright © 2015 by Fred George. All rights reserved. 9 5 yrs. 1980 1990 2000 1 mon. 1 week 1 day 2010 1 year Project Duration (log) Waterfall Waterfall Agile Agile A2 A2 Waterfall w OO
10. How Fast Can You Go? Copyright © 2015 by Fred George. All rights reserved. 10 Deployment to Production Every 3.5 minutes
11. Inhibitors Copyright © 2015 by Fred George. All rights reserved. 11
12. •Cloud exploitation •Specialized databases •New programming languages •Open source frameworks (users and creators) •Continuous releases “Valley Tech” Copyright © 2015 by Fred George. All rights reserved. 12
13. Hardware Lead Times Copyright © 2015 by Fred George. All rights reserved. 13 6 mo. 1990 1995 2005 30 min. 1 week 1 day 2015 1 mo. 2000 2010 5 min. 5 sec. Order lead times (log) Data Center Virtual Machines Docker Commercial Cloud Capacity planning? Dedicated ops team?
14. Copyright © 2015 by Fred George. All rights reserved. 14 10 1 1000 100 100,000 1 10 100 1000 10,000 Size (loc) Number of Services MicroServices Rails / Java Zone SOA Workshop Forward Netflix Services Impact
15. 0MQ 0MQ 0MQ 0MQ 0MQ 0MQ Copyright © 2015 by Fred George. All rights reserved. 15 Kafka Service Emerging: Event Bus
16. New Architecture Patterns Copyright © 2015 by Fred George. All rights reserved. 16 Bus Service Service Service Need Need Need Sol’n’s Sol’n •Variants easy •Graceful degradation ChooseS ol’n
17. Incremental Applications Copyright © 2015 by Fred George. All rights reserved. 17 Event Bus Rental offers Car rent Legacy server Brand offers Location offers Segmentation Membership CarRent.com Login Sally A B
18. Databases: Holy Grail or Ball-and-Chain? • Fortune 100 view: • Entity-oriented; consistency essential • As few as possible • MicroService view: • Shift to Event Bus, plus • DB per MicroService (if persistence needed) • Poly-glot (various NoSQL, SQL) + event buses • Few (10%) writable; even fewer transactional Copyright © 2015 by Fred George. All rights reserved. 18
19. Copyright © 2015 by Fred George. All rights reserved. 19 Rental offers Car rent Legacy server Brand offers Location offers Segmentation Membership CarRent.com Login Sally A B Rental offers Event publishing Solution collecting 300ms responder Redis
20. Copyright © 2015 by Fred George. All rights reserved. 20 Rental offers Car rent Legacy server Brand offers Location offers Segmentation Membership CarRent.com Login Sally A B Membership enrichment ETL Key-value
21. Open Source: Netflix Copyright © 2015 by Fred George. All rights reserved. 21
22. Open Source: Docker Copyright © 2015 by Fred George. All rights reserved. 22
23. Disruptive Technology: Functional Languages Copyright © 2015 by Fred George. All rights reserved. 23 •Middleware exploiting: • Kafka written in Scala • RabbitMQ written in Erlang •Case study: MailOnline page rendering • Legacy system: 130K loc Java/JSP’s • Replacement: 4,000 loc Clojure
24. Mitigate Process Inhibitors Copyright © 2015 by Fred George. All rights reserved. 24
25. Understand Your Problem Copyright © 2015 by Fred George. All rights reserved. 25 Complex Simple Complicated Chaotic Cause a Effect Cause aaa Effect Effect? Cause? Cause ? Effect Disorder
26. Copyright © 2015 by Fred George. All rights reserved. 26 Requirements Trying ideas “Experimentation drives Innovation” New Vision of “What” to Build
27. Copyright © 2015 by Fred George. All rights reserved. 27 Task Story Feature Project Initiative Co-Credit: Greg Reiser, ThoughtWorks Re-think Interactions
28. • Bad metrics: • Lines of code • Development stages and milestones met •Business success metrics: • Sales, clicks, registrations, customer retention, etc. Measure What Matters Copyright © 2015 by Fred George. All rights reserved. 28
29. Mitigate Organization Inhibitors Copyright © 2015 by Fred George. All rights reserved. 29
30. Over-Specialization Copyright © 2015 by Fred George. All rights reserved. •Theory: • Specialist are more productive •Practice: • Overhead of communication is under-estimated • Unbalanced workload creates delays 30
31. Specialization Institutionalized with Titles Copyright © 2015 by Fred George. All rights reserved. 31 Case Study: •50 IT professionals •25+ titles •0 people understanding projects
32. Solution: Fix the Titles Copyright © 2015 by Fred George. All rights reserved. 32 Step 1: Define competence in key technologies Strategic Technologies • Ruby • Java • iOS • Android • DB (SQL and non-SQL) • Testing • OO Design • Node.js • HTML/CSS/JavaScript • …
33. New Titles for New Projects Copyright © 2015 by Fred George. All rights reserved. 33 Not competent in a key technology Competent in a key technology Expert in a key technology Competent in 5-7 key technologies Expert in 3-5 key technologies
34. “Fix the Furniture”* Copyright © 2015 by Fred George. All rights reserved. 34 *Kent Beck, XP Explained
35. Non-dedicated Leadership Copyright © 2015 by Fred George. All rights reserved. 35
36. Bring Work to the Team… Copyright © 2015 by Fred George. All rights reserved. 36 Not musical chairs for each small project
37. Summary Principles of MicroServices Copyright © 2015 by Fred George. All rights reserved. 37 • Very, very small • Team size of one to develop/maintain • Loosely coupled (including flow) • Multiple versions acceptable (encouraged?) • Self-monitoring of each service • Publish interesting “stuff” (w/o explicit requirements) • “Application” seems to be a poor conceptualization
38. New Architecture Paradigms Copyright © 2015 by Fred George. All rights reserved. 38
39. Copyright © 2015 by Fred George. All rights reserved. 39 MicroServices Cloud Docker Dev/Ops Full-stack developer Agile Lean startup Cassandra Event bus MVP (minimum viable product Programmer anarchy No-SQL Managerless process
40. Copyright © 2015 by Fred George. All rights reserved. 40 MicroServices Cloud Docker Dev/Ops Full-stack developer Agile Lean startup Cassandra Event bus MVP (minimum viable product Programmer anarchy No-SQL Managerless process
41. Project Delivery Cycles Copyright © 2015 by Fred George. All rights reserved. 41 NRK NRK 5 yrs. 1980 1990 2000 1 mon. 1 week 1 day 2010 1 year Project Duration (log) Waterfall Waterfall Agile Agile Waterfall w OO
42. Acknowledgements Copyright © 2015 by Fred George. All rights reserved. 42 •Neil Hutchinson – Founder of Forward • Adopting a “Fail Fast” culture •Carl Gaywood • Former programmer who wanted to Go Faster •Paul Ingles and Mike Jones • Implementers of “Experimentation drives innovation”
43. More Information… •Google: • MicroService Architecture for videos • MicroXchg 2015/6 conference and others • MicroServices Meetups in Berlin • Programmer Anarchy for managerless teams • Docker for latest container deployment Copyright © 2015 by Fred George. All rights reserved. 43
44. Go Faster Fred George fredgeorge@acm.org @fgeorge52 Copyright © 2015 by Fred George. All rights reserved. 44