Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Boxen: MWRC

Boxen: MWRC

A talk about Boxen and other related treats

Will Farrington

April 03, 2013
Tweet

More Decks by Will Farrington

Other Decks in Programming

Transcript

  1. boxen

    View Slide


  2. wfarr

    View Slide


  3. ops and
    friction

    View Slide

  4. boxen

    View Slide

  5. why should i
    care about this?

    View Slide

  6. STORY TIME

    View Slide

  7. once upon a shitty time

    View Slide

  8. Time to get set up.
    I want to hack on stuff!
    once upon a shitty time

    View Slide

  9. Okay, install XCode...
    once upon a shitty time

    View Slide

  10. and rbenv...
    once upon a shitty time

    View Slide

  11. and a bunch of rubies...
    once upon a shitty time

    View Slide

  12. and pow...
    once upon a shitty time

    View Slide

  13. and nodejs...
    once upon a shitty time

    View Slide

  14. and bower...
    once upon a shitty time

    View Slide

  15. and bundler...
    once upon a shitty time

    View Slide

  16. and mysql...
    once upon a shitty time

    View Slide

  17. and postgresql...
    once upon a shitty time

    View Slide

  18. probably okay without
    mongodb...
    once upon a shitty time

    View Slide

  19. definitely need
    elasticsearch though...
    once upon a shitty time

    View Slide

  20. huh... well I can't
    connect to the db!
    once upon a shitty time

    View Slide

  21. and now ruby is
    segfaulting?!
    once upon a shitty time

    View Slide

  22. wat
    once upon a shitty time

    View Slide

  23. View Slide

  24. View Slide

  25. View Slide

  26. I just want to
    ship cool stuff...

    View Slide

  27. USS COOL STUFF

    View Slide

  28. I thought I wrote
    an install guide
    that people could
    just walk through.
    I wonder what
    broke...
    Ops Person

    View Slide

  29. View Slide

  30. A SOLUTION IS BORN

    View Slide

  31. View Slide

  32. View Slide

  33. View Slide

  34. once upon an awesome time

    View Slide

  35. Time to get set up.
    I want to hack on stuff!
    once upon an awesome time

    View Slide

  36. Okay, I go to
    https://boxen.company.com
    once upon an awesome time

    View Slide

  37. It says I should install
    Xcode real quick. Sure.
    once upon an awesome time

    View Slide

  38. Now I copy and paste this
    one line into the Terminal.
    once upon an awesome time

    View Slide

  39. Oh, hey! It's doing stuff.
    I guess I'll grab a coffee.
    once upon an awesome time

    View Slide

  40. I swear this is a
    different coffee cup
    from before

    View Slide

  41. Rad! It says it's done and
    to open a new Terminal.
    once upon an awesome time

    View Slide

  42. Holy automated awesome,
    Batman! I'm ready to work!
    once upon an awesome time

    View Slide

  43. so like what is
    boxen actually?

    View Slide



  44. View Slide


  45. stdlib

    View Slide

  46. View Slide

  47. wtf is a devops

    View Slide

  48. View Slide

  49. View Slide

  50. View Slide

  51. homebrew, git, gcc, .dev dns,
    nginx, rbenv, ruby-build, a bunch
    of ruby versions, nodenv, a bunch
    of nodejs versions, heroku
    toolchain, hub, mysql, postgresql,
    elasticsearch, riak, java, and a
    whole lot more available

    View Slide

  52. what do i do with
    boxen?

    View Slide

  53. MANAGE PROJECTS

    View Slide

  54. View Slide

  55. Hmm. Let's hack on
    github/github!

    View Slide

  56. The README says to run
    `boxen github`

    View Slide

  57. Okay, it's doing stuff...
    I think?
    The Puppet logging
    format is hella
    confusing to most
    people, seriously.

    View Slide

  58. What just happened?

    View Slide

  59. 1. Clone github/github
    2. Make sure dpkg is installed
    3. Make sure icu4c is installed
    4. Make sure libgithub is installed
    5. Make sure md5sha1sum is installed
    6. Make sure solr is installed
    7. Make sure mysql is installed and running
    8. Make sure elasticsearch is installed and running
    9. Make sure memcached is installed and running
    10. Make sure the dev and tests DBs are created
    11. Make sure python is installed
    12. Make sure redis is installed and running
    13. Make sure Ruby 1.9.3 is installed
    14. Make sure ~/github/github is set to use 1.9.3
    15. Notify the user of any new environment variables Boxen provides
    and let them know to restart their shell
    16. Make sure github/pages-jekyll is cloned
    17. Make sure github/pages-jekyll is set to use 1.9.3
    18. Make sure environment config for github/pages-jekyll is setup

    View Slide

  60. 1. Clone github/github
    2. Make sure dpkg is installed
    3. Make sure icu4c is installed
    4. Make sure libgithub is installed
    5. Make sure md5sha1sum is installed
    6. Make sure solr is installed
    7. Make sure mysql is installed and running
    8. Make sure elasticsearch is installed and running
    9. Make sure memcached is installed and running
    10. Make sure the dev and tests DBs are created
    11. Make sure python is installed
    12. Make sure redis is installed and running
    13. Make sure Ruby 1.9.3 is installed
    14. Make sure ~/github/github is set to use 1.9.3
    15. Notify the user of any new environment variables Boxen provides
    and let them know to restart their shell
    16. Make sure github/pages-jekyll is cloned
    17. Make sure github/pages-jekyll is set to use 1.9.3
    18. Make sure environment config for github/pages-jekyll is setup
    i had
    to
    decrease
    the
    font size
    by
    24pt just to
    fit it all on
    one
    slide
    and this is the annotated version

    View Slide

  61. okay, well, what about the code to
    make this happen?

    View Slide

  62. class projects::github {
    include dpkg
    include icu4c
    include libgithub
    include projects::pages-jekyll
    boxen::project { 'github':
    elasticsearch => true,
    memcached => true,
    mysql => [
    'github_enterprise',
    'github_development',
    'github_test'
    ],
    nginx => 'projects/github/nginx.conf.erb',
    python => true,
    redis => true,
    ruby => '1.9.3',
    source => 'github/github'
    }
    package { 'md5sha1sum': }
    }

    View Slide

  63. what about your "average" app?

    View Slide

  64. class projects::battle_station {
    boxen::project { 'battle_station':
    dotenv => true,
    mysql => true,
    nginx => true,
    redis => true,
    ruby => '2.0.0',
    source => 'github/battle_station'
    }
    }

    View Slide

  65. MANAGE EVERYTHING
    ELSE TOO

    View Slide

  66. View Slide

  67. That's cool and all, but
    what about me?

    View Slide

  68. I need to live on the bleeding
    edge version of `sl`.

    View Slide

  69. And my screensaver needs
    100% more lasers.

    View Slide

  70. Hmm. Some documentation!
    "Personal manifests?"

    View Slide

  71. Oh, this looks kind of neat.
    Maybe I'll do it.

    View Slide

  72. class people::wfarr {
    include emacs
    include zsh
    include projects::battle_station
    }

    View Slide

  73. class people::wfarr {
    git::config::global {
    'alias.st': value => 'status' ;
    'alias.ci': value => 'commit' ;
    'user.name': value => 'wfarr' ;
    }
    }

    View Slide

  74. class people::wfarr {
    repository {
    "/Users/${::boxen_user}/.emacs.d":
    source => 'wfarr/.emacs.d'
    }
    }

    View Slide

  75. class people::wfarr {
    boxen::osx_defaults {
    'Fix bluetooth audio streaming because OSX is dumb':
    ensure => present,
    domain => 'com.apple.BluetoothAudioAgent',
    key => shellquote('Apple Bitpool Min (editable)'),
    value => 50,
    user => $::boxen_user ;
    }
    }

    View Slide

  76. CLI

    View Slide

  77. $ boxen
    # run it

    View Slide

  78. $ boxen --projects
    # list projects Boxen knows about

    View Slide

  79. $ boxen
    # run it but include in the
    catalog automatically

    View Slide

  80. $ boxen --debug
    # run it but include in the
    catalog automatically

    View Slide

  81. UPDATE EASILY

    View Slide

  82. $ boxen
    Boxen is up-to-date.

    View Slide

  83. $ boxen
    Boxen has a dirty tree, won't auto-
    update!

    View Slide

  84. $ boxen
    Boxen has unpushed commits on
    master, won't auto-update!

    View Slide

  85. $ boxen
    Boxen on a non-master branch 'new-
    project', won't auto-update!

    View Slide

  86. FAIL GRACEFULLY

    View Slide

  87. $ boxen
    Error: ZOMG at /opt/boxen/repo/
    modules/people/manifests/wfarr.pp:8
    on node scruffy.gateway.github.lan
    Sorry! Creating an issue on github/
    totally-a-madeup-repo-name.

    View Slide

  88. CODEZ

    View Slide

  89. CODEZ
    who failed

    View Slide

  90. CODEZ
    how long ago

    View Slide

  91. CODEZ machine

    View Slide

  92. CODEZ
    os version

    View Slide

  93. CODEZ shell

    View Slide

  94. CODEZ
    ref

    View Slide

  95. CODEZ
    compare view

    View Slide

  96. CODEZ
    git status

    View Slide

  97. CODEZ
    full command

    View Slide

  98. CODEZ
    full log

    View Slide

  99. i fix the error and run `boxen`

    View Slide

  100. CODEZ

    View Slide

  101. via issues api

    View Slide

  102. # config/boxen.rb
    ENV['BOXEN_REPO_NAME'] = \
    'github/boxen'
    ENV['BOXEN_ISSUES_ENABLED'] = 'yes'

    View Slide

  103. This is pretty freaking rad!

    View Slide

  104. It's just code.
    Maybe I can write this too...

    View Slide

  105. 
    
    
    
    
    
    
    
    
    
    
    
    
    64 GitHubbers have
    contributed personal
    manifests or project
    manifests since October.
    That's 40.5% if we include
    the 5 new Hubbernauts
    that started Monday.

    View Slide

  106. 4 Supportocats
    5 Designers
    43 Devs
    
    
    
    
    
    
    
    
    7 Ops
    3 Trainers
    1 Education Liaison
    1 Enterprise Salesperson

    View Slide

  107. View Slide

  108. View Slide

  109. really using it

    View Slide

  110. EMBRACE AND EXTEND

    View Slide

  111. create higher level abstractions

    View Slide

  112. class projects::battle_station {
    boxen::project { 'battle_station':
    dotenv => true,
    mysql => true,
    nginx => true,
    redis => true,
    ruby => '2.0.0',
    source => 'github/battle_station'
    }
    }
    This is the same for 80% of cases

    View Slide

  113. let's optimize for our use-case

    View Slide

  114. define github::project(
    $source = "github/${title}"
    # a whole bunch of other params
    ) {
    boxen::project { $name:
    source => $source,
    # that bunch of other params
    }
    }
    The sanest default, but easy to override

    View Slide

  115. maybe all of our ops team should
    get certain things by default

    View Slide

  116. class github::environment {
    include_github_team_environments()
    }

    View Slide

  117. class github::environment::ops {
    include
    github::environment::ops::vagrant
    include projects::battle_station
    include projects::octostatus
    include projects::puppet
    }

    View Slide

  118. View Slide

  119. START SIMPLE

    View Slide

  120. write new modules inside your
    repo first and extract later

    View Slide

  121. ~/github/boxen » ls -1 modules
    cmake
    ctags
    github
    libgithub
    libmicrohttpd
    nodejs
    people
    projects
    strap

    View Slide

  122. ~/github/boxen » ls -1 modules
    cmake
    ctags
    github
    libgithub
    libmicrohttpd
    nodejs
    people
    projects
    strap
    WIP v2.0.0 nodejs module

    View Slide

  123. start abstractions in Puppet

    View Slide

  124. refactor into Ruby later if
    performance demands it

    View Slide

  125. define nodejs::version(
    $ensure = present,
    $compile = false,
    $version = $title,
    ) {
    require nodejs
    case $ensure {
    present: { # exec to install }
    absent: { # exec or file to rm }
    }
    }

    View Slide

  126. Time:
    Filebucket: 0.00
    Vagrant plugin: 0.00
    Nodejs: 4.37
    Sudoers: 0.00
    File line: 0.00
    Ini setting: 0.01
    Group: 0.10
    Repository: 0.12
    File: 0.66
    Service: 1.54
    Package: 112.76
    Total: 130.29
    Last run: 1365003143
    Rbenv gem: 2.23
    Exec: 3.36
    Config retrieval: 7.27

    View Slide

  127. if it's consistently slowing down
    the run, then optimize

    View Slide

  128. define nodejs::version(
    $ensure = present,
    $compile = false,
    $version = $title,
    ) {
    require nodejs
    nodejs { $version:
    ensure => $ensure,
    compile => $compile,
    }
    }
    This impl. happens to fork less

    View Slide

  129. DO WHATEVER WORKS

    View Slide

  130. practicality beats purity

    View Slide

  131. but...

    View Slide

  132. WHY ISN'T THIS JUST A
    BUNCH OF SCRIPTS?

    View Slide

  133. idempotent behavior
    is kind of a big deal

    View Slide

  134. shell scripts get the job done if you
    live in a wonderful vacuum where
    constraints never change and the
    dependencies between resources
    are forever static

    View Slide

  135. the real world really
    doesn't care one bit

    View Slide

  136. telling people
    "if it breaks, just reinstall"
    is a copout

    View Slide

  137. if people expect to your software
    to behave a certain way, and you
    have the power to automate that
    behavior and then don't,
    you're being a jerk to your users
    user experience common sense 101

    View Slide

  138. WHY ISN'T THIS IN CHEF?

    View Slide

  139. chef is pretty cool

    View Slide

  140. but it's in puppet now

    View Slide

  141. the reasons don't much matter
    because a rewrite would be silly

    View Slide

  142. transcend tools;
    write beautiful porcelain

    View Slide

  143. View Slide

  144. WHAT ABOUT SECURITY
    UPDATES?

    View Slide


  145. THE DARK AGES OF
    SECURITY INTERACTION

    View Slide


  146. THE DARK AGES OF
    SECURITY INTERACTION
    Java browser plugins are
    literally going to beat you
    up, kick your puppy, and eat
    all of your Doritos! zomg!

    View Slide

  147. lol are you kidding me?
    I have important codez to write.
    I'll do it later.
    Now be gone, demon!
    HINT: They're never
    going to do it.

    View Slide


  148. View Slide


  149. View Slide

  150. no surprises

    View Slide


  151. THE awesome AGES OF
    SECURITY INTERACTION

    View Slide


  152. THE awesome AGES OF
    SECURITY INTERACTION
    Hey folks! I just had to push
    an update to Boxen to disable
    Java plugins in browsers.
    Sorry about that! Here's a PR
    with more context: .

    View Slide


  153. THE awesome AGES OF
    SECURITY INTERACTION
    Oh snap - audit season!
    Let me just review our
    code to see if we're in the
    clear and secure.

    View Slide

  154. what if i have
    problems?

    View Slide

  155. file issues

    View Slide

  156. irc.freenode.net

    View Slide

  157. haters anonymous

    View Slide

  158. View Slide

  159. GitHub sucks!

    View Slide

  160. ignore the haters;
    build cool software

    View Slide

  161. View Slide


  162. wfarr
    speakerdeck.com/wfarr/boxen-mwrc

    View Slide

  163. CODEZ
    drinkup TONIGHT
    little america sports bar @ 8:30

    View Slide

  164. thanks

    View Slide