Pro Yearly is on sale from $80 to $50! »

If Government Can Do It...

If Government Can Do It...

Presented at GoTo Aarhus 2013

The Government Digital Service is a new unit of the UK Cabinet Office tasked with all things digital. It's made up of designers, developers, systems administrators, product managers and Government policy specialists - all building, running and helping to design large online government services. Our tools include agile, multidisciplinary teams with the ability to make fast decisions; close collaboration between developers, operations and information security specialists; and modern infrastructure and software as a service products. We look a lot like a startup.

This will be a technical talk with lots of culture thrown in. We'll discuss how we shipped GOV.UK, including:

* how we built our teams and technology to allow us to make changes to the live service daily
* how we worked closely with security and auditing colleagues from day one, not just as an afterthought
* how we moved infrastructure supplier quickly using modern configuration management tools and APIs
* how we're now trying to spread some of these practices and approaches around a super large organisation
* details of lots of the mistakes we made along the way

98234c645fe8c935edc0fec0186d28b8?s=128

Gareth Rushgrove

October 02, 2013
Tweet

Transcript

  1. GDS Gareth Rushgrove If Government can do it... Technical change

    in large organisations
  2. Who (Who is this person?) GDS Gareth Rushgrove

  3. GDS Gareth Rushgrove Gareth Rushgrove Technical Architect Government Digital Service

    @garethr
  4. @garethr GDS Gareth Rushgrove

  5. GDS Gareth Rushgrove

  6. GDS Gareth Rushgrove

  7. GDS Gareth Rushgrove Last code I wrote

  8. GDS Gareth Rushgrove

  9. GDS Gareth Rushgrove

  10. GDS Gareth Rushgrove

  11. Background (A startup in Government) GDS Gareth Rushgrove

  12. GDS Gareth Rushgrove

  13. GDS Gareth Rushgrove October 2010

  14. GDS Gareth Rushgrove June 2011

  15. GDS Gareth Rushgrove I joined. Team of about 10 people

    August 2011
  16. GDS Gareth Rushgrove January 2012

  17. GDS (Government Digital Service) GDS Gareth Rushgrove

  18. GDS Gareth Rushgrove

  19. GDS Gareth Rushgrove October 2012

  20. GDS Gareth Rushgrove Tools for citizens

  21. GDS Gareth Rushgrove Tools for internal users

  22. GDS Gareth Rushgrove Award winning

  23. GDS Gareth Rushgrove Organisation of about 250 people October 2013

  24. GDS Gareth Rushgrove

  25. What (What is this talk about again?) GDS Gareth Rushgrove

  26. GDS Gareth Rushgrove 1 Adopting agile in a large organisation

  27. GDS Gareth Rushgrove The importance of rapid releases 2

  28. GDS Gareth Rushgrove Why auditors love infrastructure as code 3

  29. GDS Gareth Rushgrove 4 How being open makes everything better

  30. Work in an agile manner (Scaling agile) GDS Gareth Rushgrove

    If government can
  31. GDS Gareth Rushgrove

  32. GDS Gareth Rushgrove http://www.flickr.com/photos/psd/9120523574

  33. GDS Gareth Rushgrove http://www.flickr.com/photos/psd/9104280608

  34. GDS Gareth Rushgrove http://www.flickr.com/photos/psd/9109319242

  35. GDS Gareth Rushgrove An example project

  36. GDS Gareth Rushgrove From one team of about 14

  37. GDS Gareth Rushgrove Product owner Delivery manager Developer Developer Developer

    Developer Developer Content designer Content designer Content designer Designer Developer Developer Developer
  38. GDS Gareth Rushgrove To 140 people in 14 teams

  39. GDS Gareth Rushgrove Product owner elivery manager Developer Developer Developer

    Developer Developer Product owner Delivery manager Designer Developer Developer Content designer Developer Content designer Business analyst Product owner Delivery manager Developer Developer Developer Developer Developer Business analyst Product owner Delivery manager Developer Developer Developer Developer Developer Product owner Delivery manager Designer Developer Content designer Content designer Content designer Content designer Business analyst Content designer
  40. GDS Gareth Rushgrove All in about a year

  41. GDS Gareth Rushgrove Web operations team just another product team

  42. GDS Gareth Rushgrove Product owner elivery manager Developer Developer Developer

    Developer Developer Product owner Delivery manager Designer Developer Developer Content designer Developer Content designer Business analyst Product owner Delivery manager Developer Developer Developer Developer Developer Business analyst Product owner Delivery manager Developer Developer Developer Developer Developer Product owner Delivery manager Designer Developer Content designer Content designer Content designer Content designer Business analyst Content designer Product owner Delivery manager Web operations Web operations Web operations Web operations Web operations Web operations Web operations
  43. GDS Gareth Rushgrove With web operations people in other teams

    when required
  44. GDS Gareth Rushgrove Product owner elivery manager Developer Developer Developer

    Developer Developer Product owner Delivery manager Designer Developer Developer Content designer Developer Content designer Business analyst Product owner Delivery manager Developer Developer Developer Developer Developer Business analyst Product owner Delivery manager Developer Developer Developer Developer Developer Product owner Delivery manager Designer Developer Content designer Content designer Content designer Content designer Business analyst Content designer Web operations Web operations
  45. GDS Gareth Rushgrove And with developers also working on operations

    tasks
  46. GDS Gareth Rushgrove Someone in charge

  47. GDS Gareth Rushgrove Product owner livery manager Developer Developer Developer

    Developer Developer Product owner Delivery manager Designer Developer Developer Content designer Developer Content designer Business analyst Product owner Delivery manager Developer Developer Developer Developer Developer Business analyst Product owner Delivery manager Developer Developer Developer Developer Developer Product owner Delivery manager Designer Developer Content designer Content designer Content designer Content designer Business analyst Content designer Programme owner Programme owner
  48. GDS Gareth Rushgrove Information assurance, legal, procurement, programme support

  49. GDS Gareth Rushgrove Product owner Delivery manager Developer Developer Developer

    Developer Developer Product owner Delivery manager Designer Developer Developer Content designer Developer Content designer Business analyst Product owner Delivery manager Developer Developer Developer Developer Developer Business analyst Product owner Delivery manager Developer Developer Developer Developer Developer Product owner Delivery manage Designer Developer Content designe Content designe Content designe Content designe Business analys Content designe Programme support Programme support Programme suppo Programme support ogramme support Programme support Programme support Programme sup Programme support gramme support
  50. GDS Gareth Rushgrove Governance

  51. Product owner elivery manager Developer Developer Developer Developer Developer Product

    owner Delivery manager Designer Developer Developer Content designer Developer Content designer Business analyst Product owner Delivery manager Developer Developer Developer Developer Developer Business analyst Product owner Delivery manager Developer Developer Developer Developer Developer Product owner Delivery manager Designer Developer Content designer Content designer Content designer Content designer Business analyst Programme owner Board member Board member Board member Board member Board member Board member Board member Board member GDS Gareth Rushgrove Board member Board member Board member Board member Board member Board member Board member Board member
  52. GDS Gareth Rushgrove To an organisation of 250 people, multiple

    programmes and a portfolio management team
  53. GDS Gareth Rushgrove Product owner Delivery manager Developer Developer Developer

    Developer Developer Product owner Delivery manager Designer Developer Developer Content designer Developer Content designer Business analyst Product owner Delivery manager Developer Developer Developer Developer Developer Business analyst Product owner Delivery manager Developer Developer Developer Developer Developer Product owner Delivery manager Designer Developer Content designer Content designer Content designer Content designer Business analyst Content designer Programme owner Board member Board member Board member Board member Board member Board member Board member Board member Product owner Delivery manager Developer Developer Developer Developer Developer Product owner Delivery manager Designer Developer Developer Content designer Developer Content designer Business analyst Product owner Delivery manager Developer Developer Developer Developer Developer Business analyst Product owner Delivery manager Developer Developer Developer Developer Developer Product owner Delivery manager Designer Developer Content designer Content designer Content designer Content designer Business analyst Content designer Programme owner Programme support Programme support Programme support Programme support Programme support Board member Board member Board member Board member Board member Board member Board member Board member Content designer Content designer Developer Content designer Content designer Content designer Portfolio manager Portfolio manager Portfolio manager Portfolio board Portfolio board Portfolio board Portfolio board Portfolio board Portfolio board Portfolio board
  54. GDS Gareth Rushgrove Now working out how to scale that

    across multiple organisations
  55. GDS Gareth Rushgrove Scaling agile

  56. GDS Gareth Rushgrove Things that worked

  57. GDS Gareth Rushgrove Avoiding dependencies

  58. GDS Gareth Rushgrove Understand interfaces

  59. GDS Gareth Rushgrove People AND software

  60. GDS Gareth Rushgrove Multi-disciplinary teams

  61. GDS Gareth Rushgrove Many small services

  62. GDS Gareth Rushgrove Ability to throw things away

  63. GDS Gareth Rushgrove Trust

  64. Release quickly (Change control is fun) GDS Gareth Rushgrove If

    government can
  65. GDS Gareth Rushgrove http://www.flickr.com/photos/psd/9099796942

  66. GDS Gareth Rushgrove

  67. GDS Gareth Rushgrove Average about 6 releases a day over

    first 6 months
  68. GDS Gareth Rushgrove We changed less software on the day

    of launch than probably any day before or since
  69. GDS Gareth Rushgrove One click deploy

  70. GDS Gareth Rushgrove One place to deploy

  71. GDS Gareth Rushgrove Zero click deploy for early parts of

    pipeline
  72. GDS Gareth Rushgrove One click deploy with suitable processes and

    access controls
  73. GDS Gareth Rushgrove Talk about operations

  74. GDS Gareth Rushgrove 1. Transition planning and support 2. Change

    management 3. Asset and configuration management 4. Release and deployment management 5. Service validation and testing 6. Change evaluation 7. Knowledge management
  75. GDS Gareth Rushgrove Share language

  76. GDS Gareth Rushgrove Talk about risk

  77. GDS Gareth Rushgrove

  78. GDS Gareth Rushgrove Regular releases reduce risk

  79. GDS Gareth Rushgrove Because everyone understands practice

  80. Treat everything as code (The importance of process) GDS Gareth

    Rushgrove If government can
  81. GDS Gareth Rushgrove Configuration management

  82. GDS Gareth Rushgrove

  83. GDS Gareth Rushgrove package { 'apache2': ensure => latest, }

    service { 'apache2': ensure => running, provider => upstart, require => Package['apache2'] } Infrastructure as code
  84. class govuk::apps::calendars( $port = 3011 ) { govuk::app { 'calendars':

    app_type => 'rack', port => $port, health_check_path => ‘/bank-holidays’, } } GDS Gareth Rushgrove Higher level abstractions
  85. GDS Gareth Rushgrove Infrastructure not just configuration

  86. { "role": "client", "platform": "production", "class": "backend", "zone": "backend", "vm_name":

    "backend-1", "ip": "xx.x.x.x" } GDS Gareth Rushgrove Virtual machines
  87. require 'rubygems' require 'nat' nat do snat :interface => "Client

    Data", :original => { :ip => "10.0.0.0/xx" }, :translated => { :ip => "xx.xx.xx.xx" }, :desc => "Outbound internet traffic" dnat :interface => "Client Data", :original => { :ip => "xx.xx.xx.xx", :port => 22 }, :translated => { :ip => "10.0.0.xx", :port => 22 }, :desc => "jumpbox-1 SSH" dnat :interface => "Client Data", :original => { :ip => "xx.xx.xx.xx", :port => 80 },, :translated => { :ip => "10.0.0.xx", :port => 80 }, :desc => "jenkins, logging, monitoring HTTP" GDS Gareth Rushgrove Virtual networks
  88. require 'rubygems' require 'firewall' firewall do # internal rules rule

    "ssh access to jumpbox1" do source :ip => "Any" destination :ip => "xx.xx.xx.xx", :port => 22 end rule "http to backend applications" do source :ip => "Any" destination :ip => "xx.xx.xx.xx", :port => 80 end rule "https to backend applications" do GDS Gareth Rushgrove Virtual firewalls
  89. @normal Scenario: check quick answers load When I visit "/vat-rates"

    Then I should see "VAT rates" GDS Gareth Rushgrove Smoke tests and monitoring
  90. @high Scenario: check quick answers load When I visit "/vat-rates"

    Then I should see "VAT rates" GDS Gareth Rushgrove Change who responds to issue
  91. GDS Gareth Rushgrove Reuse audited processes

  92. GDS Gareth Rushgrove Code review, pairing, source control, release pipeline,

    access controls
  93. GDS Gareth Rushgrove Avoiding spreadsheets as a source of truth

  94. GDS Gareth Rushgrove Developers want visibility of deploys

  95. GDS Gareth Rushgrove Organisations want auditability of deploys

  96. GDS Gareth Rushgrove App showing deploys

  97. GDS Gareth Rushgrove CMDB

  98. Work in the open (The benefits of openness) GDS Gareth

    Rushgrove If government can
  99. GDS Gareth Rushgrove http://www.flickr.com/photos/psd/9122642253

  100. GDS Gareth Rushgrove Blogging

  101. GDS Gareth Rushgrove

  102. GDS Gareth Rushgrove Coding in the open

  103. GDS Gareth Rushgrove Publishing tools

  104. GDS Gareth Rushgrove Frontend applications

  105. GDS Gareth Rushgrove Developer environment

  106. GDS Gareth Rushgrove Handy utilities

  107. GDS Gareth Rushgrove Published content

  108. GDS Gareth Rushgrove Testing in the open

  109. GDS Gareth Rushgrove Project backlogs

  110. GDS Gareth Rushgrove Performance data

  111. GDS Gareth Rushgrove

  112. GDS Gareth Rushgrove

  113. GDS Gareth Rushgrove

  114. GDS Gareth Rushgrove

  115. GDS Gareth Rushgrove Openness inside your organisation

  116. GDS Gareth Rushgrove http://www.flickr.com/photos/psd/8756580339 Lots of information on dashboards

  117. GDS Gareth Rushgrove What did we get from being open?

  118. GDS Gareth Rushgrove Good will

  119. GDS Gareth Rushgrove Validation

  120. GDS Gareth Rushgrove Contributions

  121. GDS Gareth Rushgrove Hiring

  122. In Summary (Just remember this) GDS Gareth Rushgrove

  123. GDS Gareth Rushgrove Devops is about the organisation

  124. GDS Gareth Rushgrove The processes in an organisation are based

    on what is possible
  125. GDS Gareth Rushgrove What is possible is changing all the

    time
  126. Find out more (Lots of reading for everyone) GDS Gareth

    Rushgrove
  127. GDS Gareth Rushgrove

  128. GDS Gareth Rushgrove Even contains a definition of devops

  129. Questions? (and thanks for listening) GDS Gareth Rushgrove

  130. GDS Gareth Rushgrove Gareth Rushgrove Technical Architect Government Digital Service

    @garethr