Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Enabling Engineering Productivity at the Financial Times

A288fb976fc633cde90a2bc19bf2b5a6?s=47 Sarah Wells
November 03, 2021

Enabling Engineering Productivity at the Financial Times

The Financial Times has embraced autonomy in our development teams, and as a result we move pretty fast - over 30,000 releases in a year from a development team of around 250. Because we have a wide variety of technical architectures, processes and team structures, we need to make sure that all those autonomous teams build in security, fault detection, and cost efficiency.

This is the responsibility of the Engineering Enablement group at the FT: teams who provide tooling and services for common capabilities like DNS, content delivery, cloud provisioning, observability, etc.; manage relationships with vendors; and provide insights and oversight to all our development teams.

To do this effectively, we have adopted principles and practices that centre on making sure teams don’t need to wait for us to do things, and we have stayed constantly focused on what our customer teams need from us.

In these slides, I discuss our approach and share what you can learn from our experiences working in this area for the last three years.

A288fb976fc633cde90a2bc19bf2b5a6?s=128

Sarah Wells

November 03, 2021
Tweet

More Decks by Sarah Wells

Other Decks in Technology

Transcript

  1. Enabling Engineering Productivity at the Financial Times Sarah Wells Technical

    Director for Engineering Enablement @sarahjwells
  2. Enabling Engineering Productivity at the Financial Times Sarah Wells Technical

    Director for Engineering Enablement @sarahjwells
  3. Changes to production at the FT

  4. It’s only an experiment if it can fail “Experiments: the

    Good, the Bad and the Beautiful” by Linda Rising
  5. • Delivery lead time • Deployment frequency • Change fail

    rate • Time to restore service
  6. “Full stack” stops somewhere!

  7. Enter Engineering Enablement Our aim is to standardise, simplify and

    advise, supporting the FT's product development teams so that they can deliver value quickly, securely and scalably.
  8. In Team Topologies terms, we are platform and enabling teams

  9. None
  10. Pave the road Guardrails not fences

  11. A cautionary tale…

  12. FT Platform impact on server provisioning time

  13. You needed sudo access to deploy code for the first

    time to a newly provisioned server
  14. Autonomous teams can choose something else http:/ /matt.chadburn.co.uk/notes/teams-as- services.html

  15. Teams chose other options: • Heroku • Containers

  16. Internal teams are service providers now

  17. Guardrails not fences

  18. None
  19. People shouldn’t need to read the guardrails

  20. You need to do this to spin up AWS resources

  21. None
  22. None
  23. None
  24. Evolving the guardrails

  25. Our Tech Governance Group

  26. Pave the road

  27. None
  28. The golden path: An opinionated and supported way of doing

    things
  29. “Supported”

  30. Principles for building the golden path

  31. Valuable Has obvious value to engineers Should we provide this

    capability?
  32. Make sure someone is signed up to use the thing

    you are building
  33. Owned and supported It won’t disappear under people Transparent usage

    and cost insights It’s clear who is using it and how much their bill would be Can people rely on it?
  34. None
  35. Discoverable Engineers can find out it exists Documented Step by

    step guides, explainers, reference docs all exist Self service You can solve your problem yourself Can people use this without costly co-ordination?
  36. The Tech Hub

  37. “Lots of manual PR approvals, which delays time to release.

    E.g. approvals for DNS repo”
  38. Only added lines Only removed lines Modified but line count

    the same 220 (47%) 71 (15%) 141 (30%) 117 (25%) approved without any comments
  39. • Automatically approve simple stuff • Send DMARC/DKIM/MX changes to

    Cyber Security team for approval • Check for common mistakes • Look for modifications that are benign and auto- approve • Use biz-ops to find a suitable team member that can peer review
  40. Easy to use Documentation guides new users - an ‘on-

    ramp’ Consistent developer experience If you’ve used other capabilities, this should be recognisably similar Will people get stuck?
  41. None
  42. Independent yet composable You can use it on its own

    or combine it with other capabilities Automation friendly APIs, SDKs, CLIs Can people use it in ways we didn’t expect?
  43. None
  44. Safe to use Sensible defaults, small blast radius Secure and

    compliant Security issues are fixed for you, capabilities comply with our policies Reliable Suitable levels of availability, scalability and performance Does it guide people to do the right thing?
  45. The Biz Ops graph

  46. Runbooks are extracted to S3

  47. Valuable Owned and supported Transparent usage and cost insights Discoverable

    Documented Self service Easy to use Consistent developer experience Independent yet composable Automation friendly Safe to use Secure and compliant Reliable
  48. Charity Majors: https:/ /charity.wtf/ 2018/12/02/software-sprawl-the-golden- path-and-scaling-teams-with-agency/ Galo Navarro: https:/ /srvaroa.github.io/paas/

    infrastructure/platform/kubernetes/cloud/ 2020/01/02/talk-how-to-build-a-paas- for-1500-engineers.html https:/ /backstage.io/
  49. Thank you @sarahjwells