WTF is Cloud Native: Enabling Engineering Productivity at the Financial Times

The Financial Times has embraced autonomy in our development teams, and as a result we move pretty fast - over 30,000 releases in a year from a development team of around 250. Because we have a wide variety of technical architectures, processes and team structures, we need to make sure that all those autonomous teams build in security, fault detection, and cost efficiency.

This is the responsibility of the Engineering Enablement group at the FT: teams who provide tooling and services for common capabilities like DNS, content delivery, cloud provisioning, observability, etc.; manage relationships with vendors; and provide insights and oversight to all our development teams.

To do this effectively, we have adopted principles and practices that centre on making sure teams don’t need to wait for us to do things, and we have stayed constantly focused on what our customer teams need from us.

In these slides, I discuss our approach and share what you can learn from our experiences working in this area for the last three years.

Transcript

1. Enabling Engineering Productivity at the Financial Times Sarah Wells Technical Director for Engineering Enablement @sarahjwells
2. Enabling Engineering Productivity at the Financial Times Sarah Wells Technical Director for Engineering Enablement @sarahjwells
3. @sarahjwells Changes to production at the FT
4. It’s only an experiment if it can fail “Experiments: the Good, the Bad and the Beautiful” by Linda Rising
5. @sarahjwells • Delivery lead time • Deployment frequency • Change fail rate • Time to restore service
6. @sarahjwells To do well on these metrics, teams need to be able to get on with things
7. @sarahjwells “Full stack” stops somewhere!
8. @sarahjwells Enter Engineering Enablement Our aim is to standardise, simplify and advise, supporting the FT's product development teams so that they can deliver value quickly, securely and scalably.
9. @sarahjwells In Team Topologies terms, we are platform and enabling teams
10. @sarahjwells
11. Pave the road Guardrails not fences
12. A cautionary tale…
13. @sarahjwells FT Platform impact on server provisioning time
14. @sarahjwells You needed sudo access to deploy code for the first time to a newly provisioned server
15. @sarahjwells Autonomous teams can choose something else http://matt.chadburn.co.uk/notes/teams-as-services.html
16. @sarahjwells Teams chose other options
17. @sarahjwells Teams chose other options: Heroku Containers
18. @sarahjwells Internal teams are service providers now
19. @sarahjwells
20. Guardrails not fences
21. @sarahjwells
22. @sarahjwells People shouldn’t need to read the guardrails
23. @sarahjwells You need to do this to spin up AWS resources
24. @sarahjwells
25. @sarahjwells
26. @sarahjwells
27. @sarahjwells Evolving the guardrails
28. @sarahjwells Our Tech Governance Group Influenced by https://philcalcado.com/2018/11/19/a_structured_rfc_process.html
29. Pave the road
30. @sarahjwells The golden path: An opinionated and supported way of doing things https://engineering.atspotify.com/2020/08/17/ how-we-use-golden-paths-to-solve- fragmentation-in-our-software-ecosystem/
31. @sarahjwells “Supported”
32. Principles for building the golden path
33. @sarahjwells Valuable Has obvious value to engineers Should we provide this capability?
34. @sarahjwells Make sure someone is signed up to use the thing you are building
35. @sarahjwells Owned and supported It won’t disappear under people Transparent usage and cost insights It’s clear who is using it and how much their bill would be Can people rely on it?
36. @sarahjwells
37. @sarahjwells Discoverable Engineers can find out it exists Documented Step by step guides, explainers, reference docs all exist Self service You can solve your problem yourself Can people use this without costly co-ordination?
38. @sarahjwells The Tech Hub
39. @sarahjwells
40. @sarahjwells “Lots of manual PR approvals, which delays time to release. E.g. approvals for DNS repo”
41. @sarahjwells Only added linesOnly removed lines Modified but line count the same 220 (47%) 71 (15%) 141 (30%) 117 (25%) approved without any comments
42. @sarahjwells • Automatically approve simple stuff • Send DMARC/DKIM/MX changes to Cyber Security team for approval • Check for common mistakes • Look for modifications that are benign and auto-approve • Use biz-ops to find a suitable team member that can peer review
43. @sarahjwells https://medium.com/ft-product-technology/cybernetic-meadows- how-a-bot-helps-engineers-at-the-ft-24636f6268e3
44. @sarahjwells Easy to use Documentation guides new users - an ‘on-ramp’ Consistent developer experience If you’ve used other capabilities, this should be recognisably similar Will people get stuck?
45. @sarahjwells
46. @sarahjwells Independent yet composable You can use it on its own or combine it with other capabilities Automation friendly APIs, SDKs, CLIs Can people use it in ways we didn’t expect?
47. @sarahjwells
48. @sarahjwells Safe to use Sensible defaults, small blast radius Secure and compliant Security issues are fixed for you, capabilities comply with our policies Reliable Suitable levels of availability, scalability and performance Does it guide people to do the right thing?
49. @sarahjwells The Biz Ops graph
50. @sarahjwells Runbooks are extracted to S3
51. @sarahjwells Valuable Owned and supported Transparent usage and cost insights Discoverable Documented Self service Easy to use Consistent developer experience Independent yet composable Automation friendly Safe to use Secure and compliant Reliable
52. @sarahjwells Charity Majors: https://charity.wtf/2018/12/02/software- sprawl-the-golden-path-and-scaling-teams- with-agency/ Galo Navarro: https://srvaroa.github.io/paas/infrastructure/pl atform/kubernetes/cloud/2020/01/02/ talk-how-to-build-a-paas-for-1500- engineers.html https://backstage.io/
53. Thank you @sarahjwells