If Government Can Do It...

Presented at GoTo Aarhus 2013

The Government Digital Service is a new unit of the UK Cabinet Office tasked with all things digital. It's made up of designers, developers, systems administrators, product managers and Government policy specialists - all building, running and helping to design large online government services. Our tools include agile, multidisciplinary teams with the ability to make fast decisions; close collaboration between developers, operations and information security specialists; and modern infrastructure and software as a service products. We look a lot like a startup.

This will be a technical talk with lots of culture thrown in. We'll discuss how we shipped GOV.UK, including:

* how we built our teams and technology to allow us to make changes to the live service daily
* how we worked closely with security and auditing colleagues from day one, not just as an afterthought
* how we moved infrastructure supplier quickly using modern configuration management tools and APIs
* how we're now trying to spread some of these practices and approaches around a super large organisation
* details of lots of the mistakes we made along the way

Transcript

1. GDS Gareth Rushgrove If Government can do it... Technical change in large organisations
2. Who (Who is this person?) GDS Gareth Rushgrove
3. GDS Gareth Rushgrove Gareth Rushgrove Technical Architect Government Digital Service @garethr
4. @garethr GDS Gareth Rushgrove
5. GDS Gareth Rushgrove
6. GDS Gareth Rushgrove
7. GDS Gareth Rushgrove Last code I wrote
8. GDS Gareth Rushgrove
9. GDS Gareth Rushgrove
10. GDS Gareth Rushgrove
11. Background (A startup in Government) GDS Gareth Rushgrove
12. GDS Gareth Rushgrove
13. GDS Gareth Rushgrove October 2010
14. GDS Gareth Rushgrove June 2011
15. GDS Gareth Rushgrove I joined. Team of about 10 people August 2011
16. GDS Gareth Rushgrove January 2012
17. GDS (Government Digital Service) GDS Gareth Rushgrove
18. GDS Gareth Rushgrove
19. GDS Gareth Rushgrove October 2012
20. GDS Gareth Rushgrove Tools for citizens
21. GDS Gareth Rushgrove Tools for internal users
22. GDS Gareth Rushgrove Award winning
23. GDS Gareth Rushgrove Organisation of about 250 people October 2013
24. GDS Gareth Rushgrove
25. What (What is this talk about again?) GDS Gareth Rushgrove
26. GDS Gareth Rushgrove 1 Adopting agile in a large organisation
27. GDS Gareth Rushgrove The importance of rapid releases 2
28. GDS Gareth Rushgrove Why auditors love infrastructure as code 3
29. GDS Gareth Rushgrove 4 How being open makes everything better
30. Work in an agile manner (Scaling agile) GDS Gareth Rushgrove If government can
31. GDS Gareth Rushgrove
32. GDS Gareth Rushgrove http://www.flickr.com/photos/psd/9120523574
33. GDS Gareth Rushgrove http://www.flickr.com/photos/psd/9104280608
34. GDS Gareth Rushgrove http://www.flickr.com/photos/psd/9109319242
35. GDS Gareth Rushgrove An example project
36. GDS Gareth Rushgrove From one team of about 14
37. GDS Gareth Rushgrove Product owner Delivery manager Developer Developer Developer Developer Developer Content designer Content designer Content designer Designer Developer Developer Developer
38. GDS Gareth Rushgrove To 140 people in 14 teams
39. GDS Gareth Rushgrove Product owner elivery manager Developer Developer Developer Developer Developer Product owner Delivery manager Designer Developer Developer Content designer Developer Content designer Business analyst Product owner Delivery manager Developer Developer Developer Developer Developer Business analyst Product owner Delivery manager Developer Developer Developer Developer Developer Product owner Delivery manager Designer Developer Content designer Content designer Content designer Content designer Business analyst Content designer
40. GDS Gareth Rushgrove All in about a year
41. GDS Gareth Rushgrove Web operations team just another product team
42. GDS Gareth Rushgrove Product owner elivery manager Developer Developer Developer Developer Developer Product owner Delivery manager Designer Developer Developer Content designer Developer Content designer Business analyst Product owner Delivery manager Developer Developer Developer Developer Developer Business analyst Product owner Delivery manager Developer Developer Developer Developer Developer Product owner Delivery manager Designer Developer Content designer Content designer Content designer Content designer Business analyst Content designer Product owner Delivery manager Web operations Web operations Web operations Web operations Web operations Web operations Web operations
43. GDS Gareth Rushgrove With web operations people in other teams when required
44. GDS Gareth Rushgrove Product owner elivery manager Developer Developer Developer Developer Developer Product owner Delivery manager Designer Developer Developer Content designer Developer Content designer Business analyst Product owner Delivery manager Developer Developer Developer Developer Developer Business analyst Product owner Delivery manager Developer Developer Developer Developer Developer Product owner Delivery manager Designer Developer Content designer Content designer Content designer Content designer Business analyst Content designer Web operations Web operations
45. GDS Gareth Rushgrove And with developers also working on operations tasks
46. GDS Gareth Rushgrove Someone in charge
47. GDS Gareth Rushgrove Product owner livery manager Developer Developer Developer Developer Developer Product owner Delivery manager Designer Developer Developer Content designer Developer Content designer Business analyst Product owner Delivery manager Developer Developer Developer Developer Developer Business analyst Product owner Delivery manager Developer Developer Developer Developer Developer Product owner Delivery manager Designer Developer Content designer Content designer Content designer Content designer Business analyst Content designer Programme owner Programme owner
48. GDS Gareth Rushgrove Information assurance, legal, procurement, programme support
49. GDS Gareth Rushgrove Product owner Delivery manager Developer Developer Developer Developer Developer Product owner Delivery manager Designer Developer Developer Content designer Developer Content designer Business analyst Product owner Delivery manager Developer Developer Developer Developer Developer Business analyst Product owner Delivery manager Developer Developer Developer Developer Developer Product owner Delivery manage Designer Developer Content designe Content designe Content designe Content designe Business analys Content designe Programme support Programme support Programme suppo Programme support ogramme support Programme support Programme support Programme sup Programme support gramme support
50. GDS Gareth Rushgrove Governance
51. Product owner elivery manager Developer Developer Developer Developer Developer Product owner Delivery manager Designer Developer Developer Content designer Developer Content designer Business analyst Product owner Delivery manager Developer Developer Developer Developer Developer Business analyst Product owner Delivery manager Developer Developer Developer Developer Developer Product owner Delivery manager Designer Developer Content designer Content designer Content designer Content designer Business analyst Programme owner Board member Board member Board member Board member Board member Board member Board member Board member GDS Gareth Rushgrove Board member Board member Board member Board member Board member Board member Board member Board member
52. GDS Gareth Rushgrove To an organisation of 250 people, multiple programmes and a portfolio management team
53. GDS Gareth Rushgrove Product owner Delivery manager Developer Developer Developer Developer Developer Product owner Delivery manager Designer Developer Developer Content designer Developer Content designer Business analyst Product owner Delivery manager Developer Developer Developer Developer Developer Business analyst Product owner Delivery manager Developer Developer Developer Developer Developer Product owner Delivery manager Designer Developer Content designer Content designer Content designer Content designer Business analyst Content designer Programme owner Board member Board member Board member Board member Board member Board member Board member Board member Product owner Delivery manager Developer Developer Developer Developer Developer Product owner Delivery manager Designer Developer Developer Content designer Developer Content designer Business analyst Product owner Delivery manager Developer Developer Developer Developer Developer Business analyst Product owner Delivery manager Developer Developer Developer Developer Developer Product owner Delivery manager Designer Developer Content designer Content designer Content designer Content designer Business analyst Content designer Programme owner Programme support Programme support Programme support Programme support Programme support Board member Board member Board member Board member Board member Board member Board member Board member Content designer Content designer Developer Content designer Content designer Content designer Portfolio manager Portfolio manager Portfolio manager Portfolio board Portfolio board Portfolio board Portfolio board Portfolio board Portfolio board Portfolio board
54. GDS Gareth Rushgrove Now working out how to scale that across multiple organisations
55. GDS Gareth Rushgrove Scaling agile
56. GDS Gareth Rushgrove Things that worked
57. GDS Gareth Rushgrove Avoiding dependencies
58. GDS Gareth Rushgrove Understand interfaces
59. GDS Gareth Rushgrove People AND software
60. GDS Gareth Rushgrove Multi-disciplinary teams
61. GDS Gareth Rushgrove Many small services
62. GDS Gareth Rushgrove Ability to throw things away
63. GDS Gareth Rushgrove Trust
64. Release quickly (Change control is fun) GDS Gareth Rushgrove If government can
65. GDS Gareth Rushgrove http://www.flickr.com/photos/psd/9099796942
66. GDS Gareth Rushgrove
67. GDS Gareth Rushgrove Average about 6 releases a day over first 6 months
68. GDS Gareth Rushgrove We changed less software on the day of launch than probably any day before or since
69. GDS Gareth Rushgrove One click deploy
70. GDS Gareth Rushgrove One place to deploy
71. GDS Gareth Rushgrove Zero click deploy for early parts of pipeline
72. GDS Gareth Rushgrove One click deploy with suitable processes and access controls
73. GDS Gareth Rushgrove Talk about operations
74. GDS Gareth Rushgrove 1. Transition planning and support 2. Change management 3. Asset and configuration management 4. Release and deployment management 5. Service validation and testing 6. Change evaluation 7. Knowledge management
75. GDS Gareth Rushgrove Share language
76. GDS Gareth Rushgrove Talk about risk
77. GDS Gareth Rushgrove
78. GDS Gareth Rushgrove Regular releases reduce risk
79. GDS Gareth Rushgrove Because everyone understands practice
80. Treat everything as code (The importance of process) GDS Gareth Rushgrove If government can
81. GDS Gareth Rushgrove Configuration management
82. GDS Gareth Rushgrove
83. GDS Gareth Rushgrove package { 'apache2': ensure => latest, } service { 'apache2': ensure => running, provider => upstart, require => Package['apache2'] } Infrastructure as code
84. class govuk::apps::calendars( $port = 3011 ) { govuk::app { 'calendars': app_type => 'rack', port => $port, health_check_path => ‘/bank-holidays’, } } GDS Gareth Rushgrove Higher level abstractions
85. GDS Gareth Rushgrove Infrastructure not just configuration
86. { "role": "client", "platform": "production", "class": "backend", "zone": "backend", "vm_name": "backend-1", "ip": "xx.x.x.x" } GDS Gareth Rushgrove Virtual machines
87. require 'rubygems' require 'nat' nat do snat :interface => "Client Data", :original => { :ip => "10.0.0.0/xx" }, :translated => { :ip => "xx.xx.xx.xx" }, :desc => "Outbound internet traffic" dnat :interface => "Client Data", :original => { :ip => "xx.xx.xx.xx", :port => 22 }, :translated => { :ip => "10.0.0.xx", :port => 22 }, :desc => "jumpbox-1 SSH" dnat :interface => "Client Data", :original => { :ip => "xx.xx.xx.xx", :port => 80 },, :translated => { :ip => "10.0.0.xx", :port => 80 }, :desc => "jenkins, logging, monitoring HTTP" GDS Gareth Rushgrove Virtual networks
88. require 'rubygems' require 'firewall' firewall do # internal rules rule "ssh access to jumpbox1" do source :ip => "Any" destination :ip => "xx.xx.xx.xx", :port => 22 end rule "http to backend applications" do source :ip => "Any" destination :ip => "xx.xx.xx.xx", :port => 80 end rule "https to backend applications" do GDS Gareth Rushgrove Virtual firewalls
89. @normal Scenario: check quick answers load When I visit "/vat-rates" Then I should see "VAT rates" GDS Gareth Rushgrove Smoke tests and monitoring
90. @high Scenario: check quick answers load When I visit "/vat-rates" Then I should see "VAT rates" GDS Gareth Rushgrove Change who responds to issue
91. GDS Gareth Rushgrove Reuse audited processes
92. GDS Gareth Rushgrove Code review, pairing, source control, release pipeline, access controls
93. GDS Gareth Rushgrove Avoiding spreadsheets as a source of truth
94. GDS Gareth Rushgrove Developers want visibility of deploys
95. GDS Gareth Rushgrove Organisations want auditability of deploys
96. GDS Gareth Rushgrove App showing deploys
97. GDS Gareth Rushgrove CMDB
98. Work in the open (The benefits of openness) GDS Gareth Rushgrove If government can
99. GDS Gareth Rushgrove http://www.flickr.com/photos/psd/9122642253
100. GDS Gareth Rushgrove Blogging
101. GDS Gareth Rushgrove
102. GDS Gareth Rushgrove Coding in the open
103. GDS Gareth Rushgrove Publishing tools
104. GDS Gareth Rushgrove Frontend applications
105. GDS Gareth Rushgrove Developer environment
106. GDS Gareth Rushgrove Handy utilities
107. GDS Gareth Rushgrove Published content
108. GDS Gareth Rushgrove Testing in the open
109. GDS Gareth Rushgrove Project backlogs
110. GDS Gareth Rushgrove Performance data
111. GDS Gareth Rushgrove
112. GDS Gareth Rushgrove
113. GDS Gareth Rushgrove
114. GDS Gareth Rushgrove
115. GDS Gareth Rushgrove Openness inside your organisation
116. GDS Gareth Rushgrove http://www.flickr.com/photos/psd/8756580339 Lots of information on dashboards
117. GDS Gareth Rushgrove What did we get from being open?
118. GDS Gareth Rushgrove Good will
119. GDS Gareth Rushgrove Validation
120. GDS Gareth Rushgrove Contributions
121. GDS Gareth Rushgrove Hiring
122. In Summary (Just remember this) GDS Gareth Rushgrove
123. GDS Gareth Rushgrove Devops is about the organisation
124. GDS Gareth Rushgrove The processes in an organisation are based on what is possible
125. GDS Gareth Rushgrove What is possible is changing all the time
126. Find out more (Lots of reading for everyone) GDS Gareth Rushgrove
127. GDS Gareth Rushgrove
128. GDS Gareth Rushgrove Even contains a definition of devops
129. Questions? (and thanks for listening) GDS Gareth Rushgrove
130. GDS Gareth Rushgrove Gareth Rushgrove Technical Architect Government Digital Service @garethr