Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Open source behind the firewall

Ben Balter
July 07, 2015
310

Open source behind the firewall

What government can learn from startups and the open source community

Ben Balter

July 07, 2015
Tweet

Transcript

  1. !
    Open source behind the firewall
    What government can learn from startups and the open source community
    @benbalter
    government.github.com
    [email protected]

    View Slide

  2. !
    1. How government works
    2. How open source projects work
    3. How gov. can work like open source

    View Slide

  3. !
    How you work
    is as important as
    what you work on

    View Slide

  4. !
    But first, I’d like to talk about
    trash cans

    View Slide

  5. !
    But first, I’d like to talk about
    rubbish bins

    View Slide

  6. !
    For any given problem, 

    there are always two solutions

    View Slide

  7. !
    The easy way and 

    the government way

    View Slide

  8. !
    When outputs can’t be measured,
    government institutes process

    View Slide

  9. !
    A lot of process.

    View Slide

  10. !
    " on the DC Metro

    View Slide

  11. View Slide

  12. View Slide

  13. View Slide

  14. !
    " on the London Underground

    View Slide

  15. View Slide

  16. View Slide

  17. !
    Transparency solves for process

    View Slide

  18. !
    How government typically works

    View Slide

  19. !
    Few (if any) in-house developers

    View Slide

  20. !
    Same procurement process for 

    aircraft carriers and websites

    View Slide

  21. !
    Siloed IP, Siloed efforts

    View Slide

  22. !
    Closed source, but still somehow,
    too many cooks in the kitchen

    View Slide

  23. !
    Program managers as 

    human issue trackers

    View Slide

  24. !
    Non-purpose built tools
    (Excel + Outlook)

    View Slide

  25. Ye Olde workflow

    View Slide

  26. !
    Open Source ≠ Published Source

    View Slide

  27. !
    How Open Source projects work

    View Slide

  28. !
    If you remember one thing about
    open source’s secret sauce…

    View Slide

  29. !
    Prefer systems that naturally
    capture and expose process

    View Slide

  30. !
    Who made what change when
    (without the human blocker)

    View Slide

  31. !
    Open source is the story of 

    Wikipedia v. Encyclopedia Britannica

    View Slide

  32. !
    Rarely in the same place 

    at the same time

    View Slide

  33. !
    Rarely working on the same thing

    at the same time

    View Slide

  34. !
    Better results than 

    heavyweight alternatives

    View Slide

  35. !
    The constraints of open source

    View Slide

  36. !
    ! Electronic
    High fidelity mediums expose process

    View Slide

  37. !
    # Transparent
    Communicate decisions in realtime, and forever

    View Slide

  38. !
    $ Asynchronous
    Focus workflow on code, not meetings

    View Slide

  39. !
    % Informal
    Adopt cultures, not polices

    View Slide

  40. !
    What makes a healthy 

    open source project?

    View Slide

  41. ‣Code in the open
    ‣Shared, public issue tracker
    ‣Extensive test suite run on each proposed change
    & Code

    View Slide

  42. ‣Proposed improvements discussed in the open
    ‣Few synchronous meetings 

    (open and memorialized when necessary)
    ‣All stakeholders have opportunity to contribute
    ' Discussions

    View Slide

  43. !
    What government can 

    learn from open source
    (even if the code’s never public)

    View Slide

  44. !
    Friction (n)

    Time between “I want to contribute” and “I have”

    View Slide

  45. !
    1. Minimize friction

    View Slide

  46. !
    in Gov Speak

    View Slide

  47. !
    Maximize the substance-process ratio

    View Slide

  48. !
    ( Scripts to rule them all
    $ script/test
    Bootstrapping
    $ boxen giants
    $ cd ~/github/giants
    $ script/bootstrap
    Running
    $ script/server
    Testing
    githubengineering.com/scripts-to-rule-them-all

    View Slide

  49. !
    2. Prefer cultural constraints to
    administrative or technical constraints

    View Slide

  50. !
    Non-blocking is better than blocking

    View Slide

  51. !
    3. Optimize for 

    developer happiness

    View Slide

  52. !
    You can have open source without
    executive oversight

    View Slide

  53. !
    You can have open source without
    policy guidance

    View Slide

  54. !
    You can’t have open source
    without developers

    View Slide

  55. !
    You can’t have open source
    without code

    View Slide

  56. !
    What does that look like practically?

    View Slide

  57. !
    ) Email

    View Slide

  58. !
    At GitHub we use GitHub to build GitHub

    View Slide

  59. GitHub uses GitHub for
    Code
    Legal
    HR
    Blog posts
    Documentation
    Policies
    Sales
    Ops

    View Slide

  60. zachholman.com/posts/github-communication

    View Slide

  61. !
    “Meetings pull you from actual work
    in order to talk about doing work”

    View Slide

  62. View Slide

  63. ‣ We use 185 rooms at the moment.
    ‣ We generated 29,168 lines of text on February 4.
    ‣ Of that, Hubot accounted for 13,462 lines.
    ‣ 468 images were pasted in.
    ‣ We deployed various apps and services 544 times.

    View Slide

  64. !
    Why chat?

    View Slide

  65. !
    Chat is inherently asynchronous

    View Slide

  66. !
    Tapping someone on the shoulder
    is inherently being a jerk

    View Slide

  67. !
    Scheduling a meeting is
    inherently evil

    View Slide

  68. !
    Chat has transcripts

    View Slide

  69. !
    Chat exposes process

    View Slide

  70. * Meet Hubot

    View Slide

  71. !
    + Continuous Integration (CI)

    View Slide

  72. + Continuous Integration (CI)

    View Slide

  73. , Build Status

    View Slide

  74. ‣ ~1,500 test suits
    ‣ ~19,00 tests
    ‣ ~80,000 assertions
    ‣ x 2 environments (.com, Enterprise)
    ‣ Tests > 20 seconds are considered a failure
    ‣ ~ 90 seconds to complete
    - On each push…

    View Slide

  75. !
    /Deploy

    View Slide

  76. !
    Nothing gets deployed
    unless tests are green

    View Slide

  77. ⚡ Doing it live

    View Slide

  78. Meanwhile, back at the pull request…

    View Slide

  79. !
    githubengineering.com/deploying-branches-to-github-com

    View Slide

  80. 0 ChatOps

    View Slide

  81. " Shared Command Line
    •View cameras in our datacenter
    • Search error logs
    • Manage and trigger alerts
    • Perform server maintenance
    • View load balancer status
    • Spin up cloud instances
    • Provision servers
    • Adjust firewall ports
    • Activate DDoS Mitigation
    • Restart processes
    • Configure DNS
    • Deploy to production

    View Slide

  82. !
    Who’s doing this?

    View Slide

  83. !
    20,000+ government users

    View Slide

  84. !
    1,000+ government organizations

    View Slide

  85. !
    100+ UK government orgs

    View Slide

  86. !
    What about the private sector?

    View Slide

  87. Startups
    twitter.github.io
    yelp.github.io

    View Slide

  88. netflix.github.io
    adobe.github.io

    View Slide

  89. Tech Giants
    sap.github.io ibm.github.io microsoft.github.io

    View Slide

  90. !
    To summarize…

    View Slide

  91. !
    1. Transparency can save you 

    from the tyranny of process

    View Slide

  92. !
    2. Prefer systems that inherently

    capture and expose process

    View Slide

  93. !
    3. Minimize friction

    View Slide

  94. !
    Open source behind the firewall
    What government can learn from startups and the open source community
    @benbalter
    government.github.com
    [email protected]

    View Slide

  95. !
    Appendix

    View Slide

  96. A typical deployment flow
    Development Staging Production

    View Slide

  97. GitHub's 1 flow
    Development Staging Production
    Development
    Development Staging
    Staging

    View Slide

  98. 2 Branch Labs

    View Slide

  99. 3 Queuing

    View Slide

  100. ‣ Trash can — https://www.flickr.com/photos/dcmetroblogger/
    191721958/
    ‣ Rubbish Bin - http://anonw.com/2013/02/08/more-bins-are-
    appearing-on-the-underground/

    View Slide