Upgrade to Pro — share decks privately, control downloads, hide ads and more …

If Government Can Do It...

If Government Can Do It...

Presented at GoTo Aarhus 2013

The Government Digital Service is a new unit of the UK Cabinet Office tasked with all things digital. It's made up of designers, developers, systems administrators, product managers and Government policy specialists - all building, running and helping to design large online government services. Our tools include agile, multidisciplinary teams with the ability to make fast decisions; close collaboration between developers, operations and information security specialists; and modern infrastructure and software as a service products. We look a lot like a startup.

This will be a technical talk with lots of culture thrown in. We'll discuss how we shipped GOV.UK, including:

* how we built our teams and technology to allow us to make changes to the live service daily
* how we worked closely with security and auditing colleagues from day one, not just as an afterthought
* how we moved infrastructure supplier quickly using modern configuration management tools and APIs
* how we're now trying to spread some of these practices and approaches around a super large organisation
* details of lots of the mistakes we made along the way

Gareth Rushgrove

October 02, 2013
Tweet

More Decks by Gareth Rushgrove

Other Decks in Business

Transcript

  1. GDS
    Gareth Rushgrove
    If Government
    can do it...
    Technical change in large
    organisations

    View Slide

  2. Who
    (Who is this person?)
    GDS
    Gareth Rushgrove

    View Slide

  3. GDS
    Gareth Rushgrove
    Gareth Rushgrove
    Technical Architect
    Government Digital Service
    @garethr

    View Slide

  4. @garethr
    GDS
    Gareth Rushgrove

    View Slide

  5. GDS
    Gareth Rushgrove

    View Slide

  6. GDS
    Gareth Rushgrove

    View Slide

  7. GDS
    Gareth Rushgrove
    Last code
    I wrote

    View Slide

  8. GDS
    Gareth Rushgrove

    View Slide

  9. GDS
    Gareth Rushgrove

    View Slide

  10. GDS
    Gareth Rushgrove

    View Slide

  11. Background
    (A startup in Government)
    GDS
    Gareth Rushgrove

    View Slide

  12. GDS
    Gareth Rushgrove

    View Slide

  13. GDS
    Gareth Rushgrove
    October 2010

    View Slide

  14. GDS
    Gareth Rushgrove
    June 2011

    View Slide

  15. GDS
    Gareth Rushgrove
    I joined. Team of
    about 10 people
    August 2011

    View Slide

  16. GDS
    Gareth Rushgrove
    January 2012

    View Slide

  17. GDS
    (Government Digital Service)
    GDS
    Gareth Rushgrove

    View Slide

  18. GDS
    Gareth Rushgrove

    View Slide

  19. GDS
    Gareth Rushgrove
    October 2012

    View Slide

  20. GDS
    Gareth Rushgrove
    Tools for
    citizens

    View Slide

  21. GDS
    Gareth Rushgrove
    Tools for
    internal users

    View Slide

  22. GDS
    Gareth Rushgrove
    Award
    winning

    View Slide

  23. GDS
    Gareth Rushgrove
    Organisation of about
    250 people
    October 2013

    View Slide

  24. GDS
    Gareth Rushgrove

    View Slide

  25. What
    (What is this talk about again?)
    GDS
    Gareth Rushgrove

    View Slide

  26. GDS
    Gareth Rushgrove
    1
    Adopting agile in a
    large organisation

    View Slide

  27. GDS
    Gareth Rushgrove
    The importance of
    rapid releases
    2

    View Slide

  28. GDS
    Gareth Rushgrove
    Why auditors love
    infrastructure as code
    3

    View Slide

  29. GDS
    Gareth Rushgrove
    4
    How being open
    makes everything
    better

    View Slide

  30. Work in an
    agile manner
    (Scaling agile)
    GDS
    Gareth Rushgrove
    If government can

    View Slide

  31. GDS
    Gareth Rushgrove

    View Slide

  32. GDS
    Gareth Rushgrove
    http://www.flickr.com/photos/psd/9120523574

    View Slide

  33. GDS
    Gareth Rushgrove
    http://www.flickr.com/photos/psd/9104280608

    View Slide

  34. GDS
    Gareth Rushgrove
    http://www.flickr.com/photos/psd/9109319242

    View Slide

  35. GDS
    Gareth Rushgrove
    An example
    project

    View Slide

  36. GDS
    Gareth Rushgrove
    From one team of
    about 14

    View Slide

  37. GDS
    Gareth Rushgrove
    Product owner
    Delivery manager
    Developer
    Developer
    Developer
    Developer
    Developer
    Content designer
    Content designer
    Content designer
    Designer
    Developer
    Developer
    Developer

    View Slide

  38. GDS
    Gareth Rushgrove
    To 140 people in 14
    teams

    View Slide

  39. GDS
    Gareth Rushgrove
    Product owner
    elivery manager
    Developer
    Developer
    Developer
    Developer
    Developer
    Product owner
    Delivery manager
    Designer
    Developer
    Developer
    Content designer
    Developer
    Content designer
    Business analyst
    Product owner
    Delivery manager
    Developer
    Developer
    Developer
    Developer
    Developer
    Business analyst
    Product owner
    Delivery manager
    Developer
    Developer
    Developer
    Developer
    Developer
    Product owner
    Delivery manager
    Designer
    Developer
    Content designer
    Content designer
    Content designer
    Content designer
    Business analyst
    Content designer

    View Slide

  40. GDS
    Gareth Rushgrove
    All in about a year

    View Slide

  41. GDS
    Gareth Rushgrove
    Web operations team
    just another product
    team

    View Slide

  42. GDS
    Gareth Rushgrove
    Product owner
    elivery manager
    Developer
    Developer
    Developer
    Developer
    Developer
    Product owner
    Delivery manager
    Designer
    Developer
    Developer
    Content designer
    Developer
    Content designer
    Business analyst
    Product owner
    Delivery manager
    Developer
    Developer
    Developer
    Developer
    Developer
    Business analyst
    Product owner
    Delivery manager
    Developer
    Developer
    Developer
    Developer
    Developer
    Product owner
    Delivery manager
    Designer
    Developer
    Content designer
    Content designer
    Content designer
    Content designer
    Business analyst
    Content designer
    Product owner
    Delivery manager
    Web operations
    Web operations
    Web operations
    Web operations
    Web operations
    Web operations
    Web operations

    View Slide

  43. GDS
    Gareth Rushgrove
    With web operations
    people in other teams
    when required

    View Slide

  44. GDS
    Gareth Rushgrove
    Product owner
    elivery manager
    Developer
    Developer
    Developer
    Developer
    Developer
    Product owner
    Delivery manager
    Designer
    Developer
    Developer
    Content designer
    Developer
    Content designer
    Business analyst
    Product owner
    Delivery manager
    Developer
    Developer
    Developer
    Developer
    Developer
    Business analyst
    Product owner
    Delivery manager
    Developer
    Developer
    Developer
    Developer
    Developer
    Product owner
    Delivery manager
    Designer
    Developer
    Content designer
    Content designer
    Content designer
    Content designer
    Business analyst
    Content designer
    Web operations
    Web operations

    View Slide

  45. GDS
    Gareth Rushgrove
    And with developers
    also working on
    operations tasks

    View Slide

  46. GDS
    Gareth Rushgrove
    Someone in charge

    View Slide

  47. GDS
    Gareth Rushgrove
    Product owner
    livery manager
    Developer
    Developer
    Developer
    Developer
    Developer
    Product owner
    Delivery manager
    Designer
    Developer
    Developer
    Content designer
    Developer
    Content designer
    Business analyst
    Product owner
    Delivery manager
    Developer
    Developer
    Developer
    Developer
    Developer
    Business analyst
    Product owner
    Delivery manager
    Developer
    Developer
    Developer
    Developer
    Developer
    Product owner
    Delivery manager
    Designer
    Developer
    Content designer
    Content designer
    Content designer
    Content designer
    Business analyst
    Content designer
    Programme owner
    Programme owner

    View Slide

  48. GDS
    Gareth Rushgrove
    Information
    assurance, legal,
    procurement,
    programme support

    View Slide

  49. GDS
    Gareth Rushgrove
    Product owner
    Delivery manager
    Developer
    Developer
    Developer
    Developer
    Developer
    Product owner
    Delivery manager
    Designer
    Developer
    Developer
    Content designer
    Developer
    Content designer
    Business analyst
    Product owner
    Delivery manager
    Developer
    Developer
    Developer
    Developer
    Developer
    Business analyst
    Product owner
    Delivery manager
    Developer
    Developer
    Developer
    Developer
    Developer
    Product owner
    Delivery manage
    Designer
    Developer
    Content designe
    Content designe
    Content designe
    Content designe
    Business analys
    Content designe
    Programme support Programme support Programme suppo
    Programme support
    ogramme support Programme support Programme support Programme sup
    Programme support
    gramme support

    View Slide

  50. GDS
    Gareth Rushgrove
    Governance

    View Slide

  51. Product owner
    elivery manager
    Developer
    Developer
    Developer
    Developer
    Developer
    Product owner
    Delivery manager
    Designer
    Developer
    Developer
    Content designer
    Developer
    Content designer
    Business analyst
    Product owner
    Delivery manager
    Developer
    Developer
    Developer
    Developer
    Developer
    Business analyst
    Product owner
    Delivery manager
    Developer
    Developer
    Developer
    Developer
    Developer
    Product owner
    Delivery manager
    Designer
    Developer
    Content designer
    Content designer
    Content designer
    Content designer
    Business analyst
    Programme owner
    Board member Board member Board member Board member
    Board member Board member Board member Board member
    GDS
    Gareth Rushgrove
    Board member Board member Board member Board member
    Board member Board member Board member Board member

    View Slide

  52. GDS
    Gareth Rushgrove
    To an organisation of
    250 people, multiple
    programmes and a
    portfolio management
    team

    View Slide

  53. GDS
    Gareth Rushgrove
    Product owner
    Delivery manager
    Developer
    Developer
    Developer
    Developer
    Developer
    Product owner
    Delivery manager
    Designer
    Developer
    Developer
    Content designer
    Developer
    Content designer
    Business analyst
    Product owner
    Delivery manager
    Developer
    Developer
    Developer
    Developer
    Developer
    Business analyst
    Product owner
    Delivery manager
    Developer
    Developer
    Developer
    Developer
    Developer
    Product owner
    Delivery manager
    Designer
    Developer
    Content designer
    Content designer
    Content designer
    Content designer
    Business analyst
    Content designer
    Programme owner
    Board member Board member Board member Board member
    Board member Board member Board member Board member
    Product owner
    Delivery manager
    Developer
    Developer
    Developer
    Developer
    Developer
    Product owner
    Delivery manager
    Designer
    Developer
    Developer
    Content designer
    Developer
    Content designer
    Business analyst
    Product owner
    Delivery manager
    Developer
    Developer
    Developer
    Developer
    Developer
    Business analyst
    Product owner
    Delivery manager
    Developer
    Developer
    Developer
    Developer
    Developer
    Product owner
    Delivery manager
    Designer
    Developer
    Content designer
    Content designer
    Content designer
    Content designer
    Business analyst
    Content designer
    Programme owner
    Programme support Programme support Programme support
    Programme support
    Programme support
    Board member Board member Board member Board member
    Board member Board member Board member Board member
    Content designer
    Content designer
    Developer Content designer
    Content designer
    Content designer
    Portfolio manager
    Portfolio manager
    Portfolio manager
    Portfolio board
    Portfolio board
    Portfolio board
    Portfolio board
    Portfolio board
    Portfolio board
    Portfolio board

    View Slide

  54. GDS
    Gareth Rushgrove
    Now working out how
    to scale that across
    multiple organisations

    View Slide

  55. GDS
    Gareth Rushgrove
    Scaling
    agile

    View Slide

  56. GDS
    Gareth Rushgrove
    Things that worked

    View Slide

  57. GDS
    Gareth Rushgrove
    Avoiding
    dependencies

    View Slide

  58. GDS
    Gareth Rushgrove
    Understand
    interfaces

    View Slide

  59. GDS
    Gareth Rushgrove
    People AND software

    View Slide

  60. GDS
    Gareth Rushgrove
    Multi-disciplinary
    teams

    View Slide

  61. GDS
    Gareth Rushgrove
    Many small services

    View Slide

  62. GDS
    Gareth Rushgrove
    Ability to throw
    things away

    View Slide

  63. GDS
    Gareth Rushgrove
    Trust

    View Slide

  64. Release quickly
    (Change control is fun)
    GDS
    Gareth Rushgrove
    If government can

    View Slide

  65. GDS
    Gareth Rushgrove
    http://www.flickr.com/photos/psd/9099796942

    View Slide

  66. GDS
    Gareth Rushgrove

    View Slide

  67. GDS
    Gareth Rushgrove
    Average about 6
    releases a day over
    first 6 months

    View Slide

  68. GDS
    Gareth Rushgrove
    We changed less
    software on the day
    of launch than
    probably any day
    before or since

    View Slide

  69. GDS
    Gareth Rushgrove
    One click deploy

    View Slide

  70. GDS
    Gareth Rushgrove
    One place
    to deploy

    View Slide

  71. GDS
    Gareth Rushgrove
    Zero click deploy for
    early parts of pipeline

    View Slide

  72. GDS
    Gareth Rushgrove
    One click deploy with
    suitable processes
    and access controls

    View Slide

  73. GDS
    Gareth Rushgrove
    Talk about operations

    View Slide

  74. GDS
    Gareth Rushgrove
    1. Transition planning and support
    2. Change management
    3. Asset and configuration management
    4. Release and deployment management
    5. Service validation and testing
    6. Change evaluation
    7. Knowledge management

    View Slide

  75. GDS
    Gareth Rushgrove
    Share
    language

    View Slide

  76. GDS
    Gareth Rushgrove
    Talk about risk

    View Slide

  77. GDS
    Gareth Rushgrove

    View Slide

  78. GDS
    Gareth Rushgrove
    Regular
    releases
    reduce risk

    View Slide

  79. GDS
    Gareth Rushgrove
    Because everyone
    understands practice

    View Slide

  80. Treat everything
    as code
    (The importance of process)
    GDS
    Gareth Rushgrove
    If government can

    View Slide

  81. GDS
    Gareth Rushgrove
    Configuration
    management

    View Slide

  82. GDS
    Gareth Rushgrove

    View Slide

  83. GDS
    Gareth Rushgrove
    package { 'apache2':
    ensure => latest,
    }
    service { 'apache2':
    ensure => running,
    provider => upstart,
    require => Package['apache2']
    }
    Infrastructure
    as code

    View Slide

  84. class govuk::apps::calendars( $port = 3011 ) {
    govuk::app { 'calendars':
    app_type => 'rack',
    port => $port,
    health_check_path => ‘/bank-holidays’,
    }
    }
    GDS
    Gareth Rushgrove
    Higher level
    abstractions

    View Slide

  85. GDS
    Gareth Rushgrove
    Infrastructure not just
    configuration

    View Slide

  86. {
    "role": "client",
    "platform": "production",
    "class": "backend",
    "zone": "backend",
    "vm_name": "backend-1",
    "ip": "xx.x.x.x"
    }
    GDS
    Gareth Rushgrove
    Virtual
    machines

    View Slide

  87. require 'rubygems'
    require 'nat'
    nat do
    snat :interface => "Client Data",
    :original => { :ip => "10.0.0.0/xx" },
    :translated => { :ip => "xx.xx.xx.xx" },
    :desc => "Outbound internet traffic"
    dnat :interface => "Client Data",
    :original => { :ip => "xx.xx.xx.xx", :port => 22 },
    :translated => { :ip => "10.0.0.xx", :port => 22 },
    :desc => "jumpbox-1 SSH"
    dnat :interface => "Client Data",
    :original => { :ip => "xx.xx.xx.xx", :port => 80 },,
    :translated => { :ip => "10.0.0.xx", :port => 80 },
    :desc => "jenkins, logging, monitoring HTTP"
    GDS
    Gareth Rushgrove
    Virtual
    networks

    View Slide

  88. require 'rubygems'
    require 'firewall'
    firewall do
    # internal rules
    rule "ssh access to jumpbox1" do
    source :ip => "Any"
    destination :ip => "xx.xx.xx.xx", :port => 22
    end
    rule "http to backend applications" do
    source :ip => "Any"
    destination :ip => "xx.xx.xx.xx", :port => 80
    end
    rule "https to backend applications" do
    GDS
    Gareth Rushgrove
    Virtual
    firewalls

    View Slide

  89. @normal
    Scenario: check quick answers load
    When I visit "/vat-rates"
    Then I should see "VAT rates"
    GDS
    Gareth Rushgrove
    Smoke tests and
    monitoring

    View Slide

  90. @high
    Scenario: check quick answers load
    When I visit "/vat-rates"
    Then I should see "VAT rates"
    GDS
    Gareth Rushgrove
    Change who
    responds to issue

    View Slide

  91. GDS
    Gareth Rushgrove
    Reuse audited
    processes

    View Slide

  92. GDS
    Gareth Rushgrove
    Code review, pairing,
    source control,
    release pipeline,
    access controls

    View Slide

  93. GDS
    Gareth Rushgrove
    Avoiding
    spreadsheets as a
    source of truth

    View Slide

  94. GDS
    Gareth Rushgrove
    Developers want
    visibility of deploys

    View Slide

  95. GDS
    Gareth Rushgrove
    Organisations want
    auditability of deploys

    View Slide

  96. GDS
    Gareth Rushgrove
    App showing
    deploys

    View Slide

  97. GDS
    Gareth Rushgrove
    CMDB

    View Slide

  98. Work in the open
    (The benefits of openness)
    GDS
    Gareth Rushgrove
    If government can

    View Slide

  99. GDS
    Gareth Rushgrove
    http://www.flickr.com/photos/psd/9122642253

    View Slide

  100. GDS
    Gareth Rushgrove
    Blogging

    View Slide

  101. GDS
    Gareth Rushgrove

    View Slide

  102. GDS
    Gareth Rushgrove
    Coding in
    the open

    View Slide

  103. GDS
    Gareth Rushgrove
    Publishing
    tools

    View Slide

  104. GDS
    Gareth Rushgrove
    Frontend
    applications

    View Slide

  105. GDS
    Gareth Rushgrove
    Developer
    environment

    View Slide

  106. GDS
    Gareth Rushgrove
    Handy
    utilities

    View Slide

  107. GDS
    Gareth Rushgrove
    Published
    content

    View Slide

  108. GDS
    Gareth Rushgrove
    Testing in
    the open

    View Slide

  109. GDS
    Gareth Rushgrove
    Project
    backlogs

    View Slide

  110. GDS
    Gareth Rushgrove
    Performance
    data

    View Slide

  111. GDS
    Gareth Rushgrove

    View Slide

  112. GDS
    Gareth Rushgrove

    View Slide

  113. GDS
    Gareth Rushgrove

    View Slide

  114. GDS
    Gareth Rushgrove

    View Slide

  115. GDS
    Gareth Rushgrove
    Openness inside
    your organisation

    View Slide

  116. GDS
    Gareth Rushgrove
    http://www.flickr.com/photos/psd/8756580339
    Lots of
    information on
    dashboards

    View Slide

  117. GDS
    Gareth Rushgrove
    What did we get from
    being open?

    View Slide

  118. GDS
    Gareth Rushgrove
    Good will

    View Slide

  119. GDS
    Gareth Rushgrove
    Validation

    View Slide

  120. GDS
    Gareth Rushgrove
    Contributions

    View Slide

  121. GDS
    Gareth Rushgrove
    Hiring

    View Slide

  122. In Summary
    (Just remember this)
    GDS
    Gareth Rushgrove

    View Slide

  123. GDS
    Gareth Rushgrove
    Devops is about the
    organisation

    View Slide

  124. GDS
    Gareth Rushgrove
    The processes in an
    organisation are
    based on what is
    possible

    View Slide

  125. GDS
    Gareth Rushgrove
    What is possible is
    changing all the time

    View Slide

  126. Find out more
    (Lots of reading for everyone)
    GDS
    Gareth Rushgrove

    View Slide

  127. GDS
    Gareth Rushgrove

    View Slide

  128. GDS
    Gareth Rushgrove
    Even contains
    a definition of
    devops

    View Slide

  129. Questions?
    (and thanks for listening)
    GDS
    Gareth Rushgrove

    View Slide

  130. GDS
    Gareth Rushgrove
    Gareth Rushgrove
    Technical Architect
    Government Digital Service
    @garethr

    View Slide