Cultivating a Microservice Culture via Tooling

https://skillsmatter.com/skillscasts/12773-cultivating-a-microservice-culture-via-tooling

In this talk Tom will share with you a guided end-to-end tour of the microservice engineering experience at Holiday Extras, which gets used and loved daily by 80 engineers spread over 20 teams. You'll be peeking under the hood at how new services are built, how code gets shared, how CI and build processes operate, how engineers observe their infrastructure and how chatops is transforming the way engineers work. You should leave inspired to go away and build awesome tooling to remove barriers to entry and drive microservice adoption within your own organizations.

Transcript

1. join.holidayextras.co.uk @tomvance94 CULTIVATING A MICROSERVICE CULTURE VIA TOOLING
2. join.holidayextras.co.uk @tomvance94 Tom Vance Holiday Extras Confirmation Team join.holidayextras.co.uk Software Engineer @tomvance94 @tomvance94
3. join.holidayextras.co.uk @tomvance94 A small bit of history
4. join.holidayextras.co.uk @tomvance94 This is nothing unique
5. join.holidayextras.co.uk @tomvance94 Tooling at all stages of development
6. join.holidayextras.co.uk @tomvance94 Core teams remove themselves from process
7. join.holidayextras.co.uk @tomvance94 Removes some freedoms but helps junior engineers
8. join.holidayextras.co.uk @tomvance94 What is “dockyard”?
9. join.holidayextras.co.uk @tomvance94 join.holidayextras.co.uk Creating a new service...
10. join.holidayextras.co.uk @tomvance94 It takes 2 weeks to get to production
11. join.holidayextras.co.uk @tomvance94 Lots of people involved • Infrastructure • IT (DNS and Security) • Engineers
12. join.holidayextras.co.uk @tomvance94 Everyone does it differently
13. join.holidayextras.co.uk @tomvance94 How did we solve this with tooling?
14. join.holidayextras.co.uk @tomvance94 $ dockyard-create service-name
15. join.holidayextras.co.uk @tomvance94 Github Repos
16. join.holidayextras.co.uk @tomvance94 CI Deployment Pipelines
17. join.holidayextras.co.uk @tomvance94 Staging & Production Envs
18. join.holidayextras.co.uk @tomvance94 • Github Repo • Hello World Template • Alerting Dashboards • Logging Dashboards • Databases Provisioned • Caching Provisioned • CI Pipelines Dockyard-create (creation automation) • Secrets Allocated • Staging Built • Production Built • DNS configured • ChatOps Integrated
19. join.holidayextras.co.uk @tomvance94 < 5mins yeh... it’s really that fast!
20. join.holidayextras.co.uk @tomvance94 “I want to test this on my machine...”
21. join.holidayextras.co.uk @tomvance94 $ dockyard-local start
22. join.holidayextras.co.uk @tomvance94 Runs Docker Locally
23. join.holidayextras.co.uk @tomvance94 Runs your service, and all dependent services.
24. join.holidayextras.co.uk @tomvance94 Distributing Common Tooling.
25. join.holidayextras.co.uk @tomvance94 100 Engineers all need the same tools
26. join.holidayextras.co.uk @tomvance94 The same engineers need to be kept up to date
27. join.holidayextras.co.uk @tomvance94 We chose NPM.
28. join.holidayextras.co.uk @tomvance94 npm install -g github:holidayextras/dockyard-tools
29. join.holidayextras.co.uk @tomvance94 Ships executable binaries
30. join.holidayextras.co.uk @tomvance94 Automation creates consistency and eases development.
31. join.holidayextras.co.uk @tomvance94 join.holidayextras.co.uk Getting into the code...
32. join.holidayextras.co.uk @tomvance94 All Services Need
33. join.holidayextras.co.uk @tomvance94 Metrics Logging Debugging Lifecycle
34. join.holidayextras.co.uk @tomvance94 Logging
35. join.holidayextras.co.uk @tomvance94 console.log(“something went wrong”)
36. join.holidayextras.co.uk @tomvance94 logger.info(new Error“something went wrong”))
37. join.holidayextras.co.uk @tomvance94 logError(“something went wrong”)
38. join.holidayextras.co.uk @tomvance94 logger.info(new Error“something went wrong”))
39. join.holidayextras.co.uk @tomvance94 Databases & Caches
40. join.holidayextras.co.uk @tomvance94 Three teams took Three different approaches
41. join.holidayextras.co.uk @tomvance94 They each took one week.
42. join.holidayextras.co.uk @tomvance94 We wasted 2 of these weeks
43. join.holidayextras.co.uk @tomvance94 The Node Toolbox
44. join.holidayextras.co.uk @tomvance94 join.holidayextras.co.uk
45. join.holidayextras.co.uk @tomvance94
46. join.holidayextras.co.uk @tomvance94
47. join.holidayextras.co.uk @tomvance94
48. join.holidayextras.co.uk @tomvance94
49. join.holidayextras.co.uk @tomvance94
50. join.holidayextras.co.uk @tomvance94 “My service needs more resources”
51. join.holidayextras.co.uk @tomvance94 Memory, CPU and Scaling
52. join.holidayextras.co.uk @tomvance94
53. join.holidayextras.co.uk @tomvance94 Abstraction removes complexity
54. join.holidayextras.co.uk @tomvance94 join.holidayextras.co.uk So it’s built, but what now...?
55. join.holidayextras.co.uk @tomvance94 Managing Service Dependencies.
56. join.holidayextras.co.uk @tomvance94 With over 200 services, each with 10 dependencies
57. join.holidayextras.co.uk @tomvance94 Welcome Renovate Bot
58. join.holidayextras.co.uk @tomvance94
59. join.holidayextras.co.uk @tomvance94 Managing Builds and Deployments.
60. join.holidayextras.co.uk @tomvance94 At scale CI is expensive
61. join.holidayextras.co.uk @tomvance94 Builds weren’t quick enough
62. join.holidayextras.co.uk @tomvance94 Developers had to wait in queues due lack of capacity
63. join.holidayextras.co.uk @tomvance94 So...We built our own CI
64. join.holidayextras.co.uk @tomvance94
65. join.holidayextras.co.uk @tomvance94
66. join.holidayextras.co.uk @tomvance94 When something goes wrong.
67. join.holidayextras.co.uk @tomvance94 Issues raised via the call center
68. join.holidayextras.co.uk @tomvance94 Spend time attempting to reproduce in house
69. join.holidayextras.co.uk @tomvance94 Trying to identify which system is at fault
70. join.holidayextras.co.uk @tomvance94 Logs / Sumo Logic
71. join.holidayextras.co.uk @tomvance94 Metrics
72. join.holidayextras.co.uk @tomvance94
73. join.holidayextras.co.uk @tomvance94 Alerts In Slack
74. join.holidayextras.co.uk @tomvance94 Slack at Holiday Extras
75. join.holidayextras.co.uk @tomvance94 Used everyday by everyone
76. join.holidayextras.co.uk @tomvance94 Teams have 3 channels
77. join.holidayextras.co.uk @tomvance94 Public Facing Channel
78. join.holidayextras.co.uk @tomvance94 Private Channel
79. join.holidayextras.co.uk @tomvance94 Ack Channel
80. join.holidayextras.co.uk @tomvance94 ChatOps and Real Time Notifications
81. join.holidayextras.co.uk @tomvance94
82. join.holidayextras.co.uk @tomvance94
83. join.holidayextras.co.uk @tomvance94
84. join.holidayextras.co.uk @tomvance94
85. join.holidayextras.co.uk @tomvance94
86. join.holidayextras.co.uk @tomvance94 Github, Privacy and GDPR
87. join.holidayextras.co.uk @tomvance94 Lots of services and lots of secrets
88. join.holidayextras.co.uk @tomvance94 $ @AccessBot request
89. join.holidayextras.co.uk @tomvance94 Allows temporary 24hr Access
90. join.holidayextras.co.uk @tomvance94 Automation, Abstraction and Acknowledgments
91. join.holidayextras.co.uk @tomvance94 Fin. Questions?