Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Boxen: PuppetCamp SF 2013

Boxen: PuppetCamp SF 2013

A talk about Boxen and related bits of Puppet thoughts.

Will Farrington

April 09, 2013
Tweet

More Decks by Will Farrington

Other Decks in Programming

Transcript


  1. ops and
    friction

    View full-size slide

  2. why should i
    care about this?

    View full-size slide

  3. once upon a shitty time

    View full-size slide

  4. Time to get set up.
    I want to hack on stuff!
    once upon a shitty time

    View full-size slide

  5. Okay, install XCode...
    once upon a shitty time

    View full-size slide

  6. and rbenv...
    once upon a shitty time

    View full-size slide

  7. and a bunch of rubies...
    once upon a shitty time

    View full-size slide

  8. and pow...
    once upon a shitty time

    View full-size slide

  9. and nodejs...
    once upon a shitty time

    View full-size slide

  10. and bower...
    once upon a shitty time

    View full-size slide

  11. and bundler...
    once upon a shitty time

    View full-size slide

  12. and mysql...
    once upon a shitty time

    View full-size slide

  13. and postgresql...
    once upon a shitty time

    View full-size slide

  14. probably okay without
    mongodb...
    once upon a shitty time

    View full-size slide

  15. definitely need
    elasticsearch though...
    once upon a shitty time

    View full-size slide

  16. huh... well I can't
    connect to the db!
    once upon a shitty time

    View full-size slide

  17. and now ruby is
    segfaulting?!
    once upon a shitty time

    View full-size slide

  18. wat
    once upon a shitty time

    View full-size slide

  19. I just want to
    ship cool stuff...

    View full-size slide

  20. USS COOL STUFF

    View full-size slide

  21. I thought I wrote
    an install guide
    that people could
    just walk through.
    I wonder what
    broke...
    Ops Person

    View full-size slide

  22. A SOLUTION IS BORN

    View full-size slide

  23. once upon an awesome time

    View full-size slide

  24. Time to get set up.
    I want to hack on stuff!
    once upon an awesome time

    View full-size slide

  25. Okay, I go to
    https://boxen.company.com
    once upon an awesome time

    View full-size slide

  26. It says I should install
    Xcode real quick. Sure.
    once upon an awesome time

    View full-size slide

  27. Now I copy and paste this
    one line into the Terminal.
    once upon an awesome time

    View full-size slide

  28. Oh, hey! It's doing stuff.
    I guess I'll grab a coffee.
    once upon an awesome time

    View full-size slide

  29. I swear this is a
    different coffee cup
    from before

    View full-size slide

  30. Rad! It says it's done and
    to open a new Terminal.
    once upon an awesome time

    View full-size slide

  31. Holy automated awesome,
    Batman! I'm ready to work!
    once upon an awesome time

    View full-size slide

  32. so like what is
    boxen actually?

    View full-size slide

  33. wtf is a devops

    View full-size slide

  34. homebrew, git, gcc, .dev dns,
    nginx, rbenv, ruby-build, a bunch
    of ruby versions, nodenv, a bunch
    of nodejs versions, heroku
    toolchain, hub, mysql, postgresql,
    elasticsearch, riak, java, and a
    whole lot more available

    View full-size slide

  35. what do i do with
    boxen?

    View full-size slide

  36. MANAGE PROJECTS

    View full-size slide

  37. Hmm. Let's hack on
    github/github!

    View full-size slide

  38. The README says to run
    `boxen github`

    View full-size slide

  39. Okay, it's doing stuff...
    I think?
    The Puppet logging
    format is hella
    confusing to most
    people, seriously.

    View full-size slide

  40. What just happened?

    View full-size slide

  41. 1. Clone github/github
    2. Make sure dpkg is installed
    3. Make sure icu4c is installed
    4. Make sure libgithub is installed
    5. Make sure md5sha1sum is installed
    6. Make sure solr is installed
    7. Make sure mysql is installed and running
    8. Make sure elasticsearch is installed and running
    9. Make sure memcached is installed and running
    10. Make sure the dev and tests DBs are created
    11. Make sure python is installed
    12. Make sure redis is installed and running
    13. Make sure Ruby 1.9.3 is installed
    14. Make sure ~/github/github is set to use 1.9.3
    15. Notify the user of any new environment variables Boxen provides
    and let them know to restart their shell
    16. Make sure github/pages-jekyll is cloned
    17. Make sure github/pages-jekyll is set to use 1.9.3
    18. Make sure environment config for github/pages-jekyll is setup

    View full-size slide

  42. 1. Clone github/github
    2. Make sure dpkg is installed
    3. Make sure icu4c is installed
    4. Make sure libgithub is installed
    5. Make sure md5sha1sum is installed
    6. Make sure solr is installed
    7. Make sure mysql is installed and running
    8. Make sure elasticsearch is installed and running
    9. Make sure memcached is installed and running
    10. Make sure the dev and tests DBs are created
    11. Make sure python is installed
    12. Make sure redis is installed and running
    13. Make sure Ruby 1.9.3 is installed
    14. Make sure ~/github/github is set to use 1.9.3
    15. Notify the user of any new environment variables Boxen provides
    and let them know to restart their shell
    16. Make sure github/pages-jekyll is cloned
    17. Make sure github/pages-jekyll is set to use 1.9.3
    18. Make sure environment config for github/pages-jekyll is setup
    i had
    to
    decrease
    the
    font size
    by
    24pt just to
    fit it all on
    one
    slide
    and this is the annotated version

    View full-size slide

  43. okay, well, what about the code to
    make this happen?

    View full-size slide

  44. class projects::github {
    include dpkg
    include icu4c
    include libgithub
    include projects::pages-jekyll
    boxen::project { 'github':
    elasticsearch => true,
    memcached => true,
    mysql => [
    'github_enterprise',
    'github_development',
    'github_test'
    ],
    nginx => 'projects/github/nginx.conf.erb',
    python => true,
    redis => true,
    ruby => '1.9.3',
    source => 'github/github'
    }
    package { 'md5sha1sum': }
    }

    View full-size slide

  45. what about your "average" app?

    View full-size slide

  46. class projects::battle_station {
    boxen::project { 'battle_station':
    dotenv => true,
    mysql => true,
    nginx => true,
    redis => true,
    ruby => '2.0.0',
    source => 'github/battle_station'
    }
    }

    View full-size slide

  47. MANAGE EVERYTHING
    ELSE TOO

    View full-size slide

  48. That's cool and all, but
    what about me?

    View full-size slide

  49. I need to live on the bleeding
    edge version of `sl`.

    View full-size slide

  50. And my screensaver needs
    100% more lasers.

    View full-size slide

  51. Hmm. Some documentation!
    "Personal manifests?"

    View full-size slide

  52. Oh, this looks kind of neat.
    Maybe I'll do it.

    View full-size slide

  53. class people::wfarr {
    include emacs
    include zsh
    include projects::battle_station
    }

    View full-size slide

  54. class people::wfarr {
    git::config::global {
    'alias.st': value => 'status' ;
    'alias.ci': value => 'commit' ;
    'user.name': value => 'wfarr' ;
    }
    }

    View full-size slide

  55. class people::wfarr {
    repository {
    "/Users/${::boxen_user}/.emacs.d":
    source => 'wfarr/.emacs.d'
    }
    }

    View full-size slide

  56. class people::wfarr {
    boxen::osx_defaults {
    'Fix bluetooth audio streaming because OSX is dumb':
    ensure => present,
    domain => 'com.apple.BluetoothAudioAgent',
    key => shellquote('Apple Bitpool Min (editable)'),
    value => 50,
    user => $::boxen_user ;
    }
    }

    View full-size slide

  57. $ boxen
    # run it

    View full-size slide

  58. $ boxen --projects
    # list projects Boxen knows about

    View full-size slide

  59. $ boxen
    # run it but include in the
    catalog automatically

    View full-size slide

  60. $ boxen --debug
    # run it but include in the
    catalog automatically

    View full-size slide

  61. UPDATE EASILY

    View full-size slide

  62. $ boxen
    Boxen is up-to-date.

    View full-size slide

  63. $ boxen
    Boxen has a dirty tree, won't auto-
    update!

    View full-size slide

  64. $ boxen
    Boxen has unpushed commits on
    master, won't auto-update!

    View full-size slide

  65. $ boxen
    Boxen on a non-master branch 'new-
    project', won't auto-update!

    View full-size slide

  66. FAIL GRACEFULLY

    View full-size slide

  67. $ boxen
    Error: ZOMG at /opt/boxen/repo/
    modules/people/manifests/wfarr.pp:8
    on node scruffy.gateway.github.lan
    Sorry! Creating an issue on github/
    totally-a-madeup-repo-name.

    View full-size slide

  68. CODEZ
    who failed

    View full-size slide

  69. CODEZ
    how long ago

    View full-size slide

  70. CODEZ machine

    View full-size slide

  71. CODEZ
    os version

    View full-size slide

  72. CODEZ
    compare view

    View full-size slide

  73. CODEZ
    git status

    View full-size slide

  74. CODEZ
    full command

    View full-size slide

  75. CODEZ
    full log

    View full-size slide

  76. i fix the error and run `boxen`

    View full-size slide

  77. via issues api

    View full-size slide

  78. # config/boxen.rb
    ENV['BOXEN_REPO_NAME'] = \
    'github/boxen'
    ENV['BOXEN_ISSUES_ENABLED'] = 'yes'

    View full-size slide

  79. This is pretty freaking rad!

    View full-size slide

  80. It's just code.
    Maybe I can write this too...

    View full-size slide

  81. 
    
    
    
    
    
    
    
    
    
    
    
    
    66 GitHubbers (~42%)
    have contributed
    personal manifests or
    project manifests since
    October

    View full-size slide

  82. 4 Supportocats
    5 Designers
    44 Devs
    
    
    
    
    
    
    
    
    
    7 Ops
    3 Trainers
    1 Education Liaison
    1 Enterprise Salesperson
    1 Security Bloke

    View full-size slide

  83. really using it

    View full-size slide

  84. EMBRACE AND EXTEND

    View full-size slide

  85. create higher level abstractions

    View full-size slide

  86. class projects::battle_station {
    boxen::project { 'battle_station':
    dotenv => true,
    mysql => true,
    nginx => true,
    redis => true,
    ruby => '2.0.0',
    source => 'github/battle_station'
    }
    }
    This is the same for 80% of cases

    View full-size slide

  87. let's optimize for our use-case

    View full-size slide

  88. define github::project(
    $source = "github/${title}"
    # a whole bunch of other params
    ) {
    boxen::project { $name:
    source => $source,
    # that bunch of other params
    }
    }
    The sanest default, but easy to override

    View full-size slide

  89. maybe all of our ops team should
    get certain things by default

    View full-size slide

  90. class github::environment {
    include_github_team_environments()
    }

    View full-size slide

  91. class github::environment::ops {
    include
    github::environment::ops::vagrant
    include projects::battle_station
    include projects::octostatus
    include projects::puppet
    }

    View full-size slide

  92. START SIMPLE

    View full-size slide

  93. write new modules inside your
    repo first and extract later

    View full-size slide

  94. ~/github/boxen » ls -1 modules
    cmake
    ctags
    github
    libgithub
    libmicrohttpd
    nodejs
    people
    projects
    strap

    View full-size slide

  95. ~/github/boxen » ls -1 modules
    cmake
    ctags
    github
    libgithub
    libmicrohttpd
    nodejs
    people
    projects
    strap
    WIP v2.0.0 nodejs module

    View full-size slide

  96. start abstractions in Puppet

    View full-size slide

  97. refactor into Ruby later if
    performance demands it

    View full-size slide

  98. define nodejs::version(
    $ensure = present,
    $compile = false,
    $version = $title,
    ) {
    require nodejs
    case $ensure {
    present: { # exec to install }
    absent: { # exec or file to rm }
    }
    }

    View full-size slide

  99. Time:
    Filebucket: 0.00
    Vagrant plugin: 0.00
    Nodejs: 4.37
    Sudoers: 0.00
    File line: 0.00
    Ini setting: 0.01
    Group: 0.10
    Repository: 0.12
    File: 0.66
    Service: 1.54
    Package: 112.76
    Total: 130.29
    Last run: 1365003143
    Rbenv gem: 2.23
    Exec: 3.36
    Config retrieval: 7.27

    View full-size slide

  100. if it's consistently slowing down
    the run, then optimize

    View full-size slide

  101. define nodejs::version(
    $ensure = present,
    $compile = false,
    $version = $title,
    ) {
    require nodejs
    nodejs { $version:
    ensure => $ensure,
    compile => $compile,
    }
    }
    This impl. happens to fork less

    View full-size slide

  102. DO WHATEVER WORKS

    View full-size slide

  103. practicality beats purity /
    perfect is the enemy of good enough

    View full-size slide

  104. WHY ISN'T THIS JUST A
    BUNCH OF SCRIPTS?

    View full-size slide

  105. idempotent behavior
    is kind of a big deal

    View full-size slide

  106. shell scripts get the job done if you
    live in a wonderful vacuum where
    constraints never change and the
    dependencies between resources
    are forever static

    View full-size slide

  107. the real world really
    doesn't care one bit

    View full-size slide

  108. telling people
    "if it breaks, just reinstall"
    is a copout

    View full-size slide

  109. if people expect to your software
    to behave a certain way, and you
    have the power to automate that
    behavior and then don't,
    you're being a jerk to your users
    user experience common sense 101

    View full-size slide

  110. WHY ISN'T THIS IN CHEF?

    View full-size slide

  111. so glad people don't ask this one
    at puppet conferences!

    View full-size slide

  112. transcend tools;
    write beautiful porcelain

    View full-size slide

  113. WHAT ABOUT SECURITY
    UPDATES?

    View full-size slide


  114. THE DARK AGES OF
    SECURITY INTERACTION

    View full-size slide


  115. THE DARK AGES OF
    SECURITY INTERACTION
    Java browser plugins are
    literally going to beat you
    up, kick your puppy, and eat
    all of your Doritos! zomg!

    View full-size slide

  116. lol are you kidding me?
    I have important codez to write.
    I'll do it later.
    Now be gone, demon!
    HINT: They're never
    going to do it.

    View full-size slide


  117. THE awesome AGES OF
    SECURITY INTERACTION

    View full-size slide


  118. THE awesome AGES OF
    SECURITY INTERACTION
    Hey folks! I just had to push
    an update to Boxen to disable
    Java plugins in browsers.
    Sorry about that! Here's a PR
    with more context: .

    View full-size slide


  119. THE awesome AGES OF
    SECURITY INTERACTION
    Oh snap - audit season!
    Let me just review our
    code to see if we're in the
    clear and secure.

    View full-size slide

  120. what if i have
    problems?

    View full-size slide

  121. pull requests

    View full-size slide

  122. irc.freenode.net

    View full-size slide

  123. be chuck yeager

    View full-size slide

  124. Savannah
    Atlanta

    View full-size slide

  125. Savannah
    Atlanta

    View full-size slide

  126. Savannah
    Atlanta

    View full-size slide

  127. Savannah
    Atlanta

    View full-size slide

  128. Savannah
    Atlanta

    View full-size slide

  129. "So, we're uh in the air. We made
    it. That's uh good news,
    obviously."

    View full-size slide

  130. "Flight attendants, if you could
    just hang on to uh... whatever..."

    View full-size slide

  131. The point at which I
    realize I'm going to
    die
    The point at which I
    think the worst is
    over

    View full-size slide


  132. wfarr
    speakerdeck.com/wfarr/
    boxen-puppetcamp-sf-2013

    View full-size slide