Rate of Change, Microservices and Platforms - A tale of devops coevolution

#devopsdays London talk all about the coevolution of devops, microservices and the need for platforms for managing technology.

Lots of squish people and organisational change bits along with some science.

Transcript

1. (without introducing more risk) Rate of Change, Microservices and Platforms Puppet Gareth Rushgrove A tale of devops coevolution
2. (without introducing more risk) Gareth Rushgrove @garethr
3. (without introducing more risk) Gareth Rushgrove
4. (without introducing more risk) So you want to go faster? Some starting assumptions
5. (without introducing more risk) What if I told you It’s possible to go faster and to reduce risk at the same time? Gareth Rushgrove
6. (without introducing more risk) You don’t need to be Google or Amazon or Facebook Gareth Rushgrove
7. (without introducing more risk) What do I need to do? Gareth Rushgrove
8. (without introducing more risk) - Devops and microservices - The effect of communication overhead - The influence of teams on software adoption Gareth Rushgrove
9. (without introducing more risk) We should adopt devops! The rush for closer collaboration
10. (without introducing more risk) 30x Gareth Rushgrove More frequent deployments Faster lead times than their peers 200x 2015 State of DevOps Report
11. (without introducing more risk) 60x Gareth Rushgrove Change success rate Faster mean time to recover 168x 2015 State of DevOps Report
12. (without introducing more risk) Gareth Rushgrove Regular Releases Reduce Risk
13. (without introducing more risk) Siloed teams Long cycle times Poor visibility Manual processes Gareth Rushgrove Cross-functional teams Short cycle times Fast feedback Automated processes
14. (without introducing more risk) Gareth Rushgrove
15. (without introducing more risk) Gareth Rushgrove devopstopologies.com
16. (without introducing more risk) Gareth Rushgrove From http://web.devopstopologies.com/ by Matthew Skelton Dev Ops
17. (without introducing more risk) Gareth Rushgrove Dev Ops From http://web.devopstopologies.com/ by Matthew Skelton
18. (without introducing more risk) Gareth Rushgrove Dev Ops From http://web.devopstopologies.com/ by Matthew Skelton
19. (without introducing more risk) Gareth Rushgrove Dev Devops Ops From http://web.devopstopologies.com/ by Matthew Skelton
20. (without introducing more risk) Bringing operational considerations into development Gareth Rushgrove
21. (without introducing more risk) Building empowered, cross- functional teams Gareth Rushgrove
22. (without introducing more risk) We should adopt microservices! The rush to smaller services (and teams)
23. (without introducing more risk) Gareth Rushgrove
24. (without introducing more risk) Gareth Rushgrove
25. (without introducing more risk) There are two essential strategies to manage [growing software]: a team can keep everything together (create a monolith) or a team can divide a project into smaller pieces (create microservices). Gareth Rushgrove https://blog.heroku.com/archives/2015/1/20/why_microservices_matter
26. (without introducing more risk) Microservices are as much about people as technology Gareth Rushgrove
27. (without introducing more risk) Paraphrasing There are two essential strategies to manage a growing team: keep everyone together or divide into smaller sub-teams Gareth Rushgrove Gareth paraphrasing https://blog.heroku.com/archives/2015/1/20/why_microservices_matter
28. (without introducing more risk) Gareth Rushgrove Software
29. (without introducing more risk) Gareth Rushgrove Monolith
30. (without introducing more risk) Gareth Rushgrove Microservice
31. (without introducing more risk) Gareth Rushgrove Microservice
32. (without introducing more risk) The resulting need for a platform Gareth Rushgrove
33. (without introducing more risk) Using standards to drive the adoption of platforms Gareth Rushgrove
34. (without introducing more risk) Gareth Rushgrove 12factor.net
35. (without introducing more risk) Gareth Rushgrove Codebase Dependencies Config Backing services Build, release, run Processes Port binding Concurrency Disposability Dev/prod parity Logs Admin processes
36. (without introducing more risk) Or using platforms to drive the adoption of standards? Gareth Rushgrove
37. (without introducing more risk) Problems with microservices - Significant operations overhead - Substantial ops skills required - Duplication of effort - Distributed system complexity Gareth Rushgrove http://highscalability.com/blog/2014/4/8/microservices-not-a-free-lunch.html
38. (without introducing more risk) How to address the operational challenges? Gareth Rushgrove
39. (without introducing more risk) We should adopt devops! Déjà vu all over again
40. (without introducing more risk) We’ve adopted devops, we’re going faster, deploying once a week rather than once a quarter Gareth Rushgrove
41. (without introducing more risk) We’ve adopted microservices, we have 10x more services now Gareth Rushgrove
42. (without introducing more risk) Does that mean 100x more change approval meetings? Gareth Rushgrove
43. (without introducing more risk) Communication overhead Bring the research
44. (without introducing more risk) Gareth Rushgrove
45. (without introducing more risk) Conway’s Law Any organization that designs a system (defined more broadly here than just information systems) will inevitably produce a design whose structure is a copy of the organization's communication structure Gareth Rushgrove
46. (without introducing more risk) Gareth Rushgrove
47. (without introducing more risk) By organizing the stimulus input simultaneously into several dimensions and successively into a sequence of chunks, we manage to break (or at least stretch) this informational bottleneck. Gareth Rushgrove The Magical Number Seven, Plus or Minus Two: Some Limits on our Capacity for Processing Information, 1956
48. (without introducing more risk) Gareth Rushgrove
49. (without introducing more risk) The need for coordination drives the need for communication among team members. Gareth Rushgrove Findings from Communication Overhead: The Hidden Cost of Team Cognition, 2004
50. (without introducing more risk) Reduction in communication overhead can be created in multiple ways, such as by restructuring the team to reduce the need for coordination… Gareth Rushgrove Findings from Communication Overhead: The Hidden Cost of Team Cognition, 2004
51. (without introducing more risk) Reduction in communication overhead can be created in multiple ways, such as by restructuring the team to reduce the need for coordination… Gareth Rushgrove Microservices?
52. (without introducing more risk) …or by activities that increase mutual awareness and shared mental models among team members Gareth Rushgrove Findings from Communication Overhead: The Hidden Cost of Team Cognition, 2004
53. (without introducing more risk) …or by activities that increase mutual awareness and shared mental models among team members Gareth Rushgrove Devops?
54. (without introducing more risk) Adopting software The changing vendor relationship
55. (without introducing more risk) Software adoption is a factor of your organisational structure Gareth Rushgrove
56. (without introducing more risk) Is your organisational structure an influence on the software you adopt? Gareth Rushgrove
57. (without introducing more risk) Melvin Conway If you have four groups working on a compiler, you’ll get a 4-pass compiler Gareth Rushgrove
58. (without introducing more risk) Paraphrasing If you have four separate business units with their own budgets you’ll have 4 contracts with <vendor> Gareth Rushgrove
59. (without introducing more risk) Gareth Rushgrove Developers as the new kingmakers
60. (without introducing more risk) Developer driven software adoption Gareth Rushgrove
61. (without introducing more risk) Developer evangelism Gareth Rushgrove
62. (without introducing more risk) Change the software you use by changing your organisation Gareth Rushgrove
63. (without introducing more risk) Change your organisation by changing the software you use Gareth Rushgrove
64. (without introducing more risk) Staffing And the influence on technology adoption
65. (without introducing more risk) Gareth Rushgrove Software is eating the world
66. (without introducing more risk) Organisations increasingly prioritising speed over efficiency Gareth Rushgrove
67. (without introducing more risk) Explosion of new languages Go, Rust, Clojure, Scala, Elixir, Elm, Swift, TypeScript… Gareth Rushgrove
68. (without introducing more risk) Explosion of new platforms Cloud Foundry, Docker, Rancher, DC/OS, Kubernetes, Apcera… Gareth Rushgrove
69. (without introducing more risk) Gareth Rushgrove The Full Stack Developer http://xkcd.com/435/
70. (without introducing more risk) Driven by need for speed? Pick the right tool for the job Gareth Rushgrove
71. (without introducing more risk) Driven by competitive market? Hiring pressure for skilled technologists wanting new toys Gareth Rushgrove
72. (without introducing more risk) More services, many small bets, lower risk, less focus on one strategic technology Gareth Rushgrove
73. (without introducing more risk) Cause or effect? Gareth Rushgrove
74. (without introducing more risk) Conclusions Wrapping everything up with more science
75. (without introducing more risk) Sociotechnical Systems suggests that all human systems include both a "technical system" and a "social system" Gareth Rushgrove https://en.wikipedia.org/wiki/Coevolution#Technological_coevolution
76. (without introducing more risk) It is possible for the system to optimize on the technology, giving priority to technical solutions and compelling the social system to adapt to it… Gareth Rushgrove https://en.wikipedia.org/wiki/Coevolution#Technological_coevolution
77. (without introducing more risk) … or to optimize on the social system, giving priority to existing social patterns and procedures and compelling the technology to fill in what gaps remain. Gareth Rushgrove https://en.wikipedia.org/wiki/Coevolution#Technological_coevolution
78. (without introducing more risk) In practice, both solutions are generally suboptimal in terms of outcomes. Gareth Rushgrove https://en.wikipedia.org/wiki/Coevolution#Technological_coevolution
79. (without introducing more risk) Better outcomes are usually obtained by a reciprocal process of joint optimization, through which both the technical system and the social system change to some degree in response to each other Gareth Rushgrove https://en.wikipedia.org/wiki/Coevolution#Technological_coevolution
80. (without introducing more risk) Devops, microservices and the quest for platforms are interdependent examples of coevolution Gareth Rushgrove
81. (without introducing more risk) Only by understanding this coevolution can you successfully evolve your organisation to take advantage Gareth Rushgrove
82. (without introducing more risk) Questions? And thanks for listening