Boxen: PuppetCamp SF 2013

Boxen: PuppetCamp SF 2013

A talk about Boxen and related bits of Puppet thoughts.

Cd839cc361ffa996be0cc8259f3d7555?s=128

Will Farrington

April 09, 2013
Tweet

Transcript

  1. boxen

  2.  wfarr

  3.  ops and friction

  4. boxen

  5. why should i care about this?

  6. STORY TIME

  7. once upon a shitty time

  8. Time to get set up. I want to hack on

    stuff! once upon a shitty time
  9. Okay, install XCode... once upon a shitty time

  10. and rbenv... once upon a shitty time

  11. and a bunch of rubies... once upon a shitty time

  12. and pow... once upon a shitty time

  13. and nodejs... once upon a shitty time

  14. and bower... once upon a shitty time

  15. and bundler... once upon a shitty time

  16. and mysql... once upon a shitty time

  17. and postgresql... once upon a shitty time

  18. probably okay without mongodb... once upon a shitty time

  19. definitely need elasticsearch though... once upon a shitty time

  20. huh... well I can't connect to the db! once upon

    a shitty time
  21. and now ruby is segfaulting?! once upon a shitty time

  22. wat once upon a shitty time

  23. None
  24. None
  25. None
  26. I just want to ship cool stuff...

  27. USS COOL STUFF

  28. I thought I wrote an install guide that people could

    just walk through. I wonder what broke... Ops Person
  29. None
  30. A SOLUTION IS BORN

  31. None
  32. None
  33. None
  34. once upon an awesome time

  35. Time to get set up. I want to hack on

    stuff! once upon an awesome time
  36. Okay, I go to https://boxen.company.com once upon an awesome time

  37. It says I should install Xcode real quick. Sure. once

    upon an awesome time
  38. Now I copy and paste this one line into the

    Terminal. once upon an awesome time
  39. Oh, hey! It's doing stuff. I guess I'll grab a

    coffee. once upon an awesome time
  40. I swear this is a different coffee cup from before

  41. Rad! It says it's done and to open a new

    Terminal. once upon an awesome time
  42. Holy automated awesome, Batman! I'm ready to work! once upon

    an awesome time
  43. so like what is boxen actually?

  44.  

  45.  stdlib

  46. None
  47. wtf is a devops

  48. None
  49. None
  50. None
  51. homebrew, git, gcc, .dev dns, nginx, rbenv, ruby-build, a bunch

    of ruby versions, nodenv, a bunch of nodejs versions, heroku toolchain, hub, mysql, postgresql, elasticsearch, riak, java, and a whole lot more available
  52. what do i do with boxen?

  53. MANAGE PROJECTS

  54. None
  55. Hmm. Let's hack on github/github!

  56. The README says to run `boxen github`

  57. Okay, it's doing stuff... I think? The Puppet logging format

    is hella confusing to most people, seriously.
  58. What just happened?

  59. 1. Clone github/github 2. Make sure dpkg is installed 3.

    Make sure icu4c is installed 4. Make sure libgithub is installed 5. Make sure md5sha1sum is installed 6. Make sure solr is installed 7. Make sure mysql is installed and running 8. Make sure elasticsearch is installed and running 9. Make sure memcached is installed and running 10. Make sure the dev and tests DBs are created 11. Make sure python is installed 12. Make sure redis is installed and running 13. Make sure Ruby 1.9.3 is installed 14. Make sure ~/github/github is set to use 1.9.3 15. Notify the user of any new environment variables Boxen provides and let them know to restart their shell 16. Make sure github/pages-jekyll is cloned 17. Make sure github/pages-jekyll is set to use 1.9.3 18. Make sure environment config for github/pages-jekyll is setup
  60. 1. Clone github/github 2. Make sure dpkg is installed 3.

    Make sure icu4c is installed 4. Make sure libgithub is installed 5. Make sure md5sha1sum is installed 6. Make sure solr is installed 7. Make sure mysql is installed and running 8. Make sure elasticsearch is installed and running 9. Make sure memcached is installed and running 10. Make sure the dev and tests DBs are created 11. Make sure python is installed 12. Make sure redis is installed and running 13. Make sure Ruby 1.9.3 is installed 14. Make sure ~/github/github is set to use 1.9.3 15. Notify the user of any new environment variables Boxen provides and let them know to restart their shell 16. Make sure github/pages-jekyll is cloned 17. Make sure github/pages-jekyll is set to use 1.9.3 18. Make sure environment config for github/pages-jekyll is setup i had to decrease the font size by 24pt just to fit it all on one slide and this is the annotated version
  61. okay, well, what about the code to make this happen?

  62. class projects::github { include dpkg include icu4c include libgithub include

    projects::pages-jekyll boxen::project { 'github': elasticsearch => true, memcached => true, mysql => [ 'github_enterprise', 'github_development', 'github_test' ], nginx => 'projects/github/nginx.conf.erb', python => true, redis => true, ruby => '1.9.3', source => 'github/github' } package { 'md5sha1sum': } }
  63. what about your "average" app?

  64. class projects::battle_station { boxen::project { 'battle_station': dotenv => true, mysql

    => true, nginx => true, redis => true, ruby => '2.0.0', source => 'github/battle_station' } }
  65. MANAGE EVERYTHING ELSE TOO

  66. None
  67. That's cool and all, but what about me?

  68. I need to live on the bleeding edge version of

    `sl`.
  69. And my screensaver needs 100% more lasers.

  70. Hmm. Some documentation! "Personal manifests?"

  71. Oh, this looks kind of neat. Maybe I'll do it.

  72. class people::wfarr { include emacs include zsh include projects::battle_station }

  73. class people::wfarr { git::config::global { 'alias.st': value => 'status' ;

    'alias.ci': value => 'commit' ; 'user.name': value => 'wfarr' ; } }
  74. class people::wfarr { repository { "/Users/${::boxen_user}/.emacs.d": source => 'wfarr/.emacs.d' }

    }
  75. class people::wfarr { boxen::osx_defaults { 'Fix bluetooth audio streaming because

    OSX is dumb': ensure => present, domain => 'com.apple.BluetoothAudioAgent', key => shellquote('Apple Bitpool Min (editable)'), value => 50, user => $::boxen_user ; } }
  76. CLI

  77. $ boxen # run it

  78. $ boxen --projects # list projects Boxen knows about

  79. $ boxen <project> # run it but include <project> in

    the catalog automatically
  80. $ boxen --debug # run it but include <project> in

    the catalog automatically
  81. UPDATE EASILY

  82. $ boxen Boxen is up-to-date.

  83. $ boxen Boxen has a dirty tree, won't auto- update!

  84. $ boxen Boxen has unpushed commits on master, won't auto-update!

  85. $ boxen Boxen on a non-master branch 'new- project', won't

    auto-update!
  86. FAIL GRACEFULLY

  87. $ boxen Error: ZOMG at /opt/boxen/repo/ modules/people/manifests/wfarr.pp:8 on node scruffy.gateway.github.lan

    Sorry! Creating an issue on github/ totally-a-madeup-repo-name.
  88. CODEZ

  89. CODEZ who failed

  90. CODEZ how long ago

  91. CODEZ machine

  92. CODEZ os version

  93. CODEZ shell

  94. CODEZ ref

  95. CODEZ compare view

  96. CODEZ git status

  97. CODEZ full command

  98. CODEZ full log

  99. i fix the error and run `boxen`

  100. CODEZ

  101. via issues api

  102. # config/boxen.rb ENV['BOXEN_REPO_NAME'] = \ 'github/boxen' ENV['BOXEN_ISSUES_ENABLED'] = 'yes'

  103. This is pretty freaking rad!

  104. It's just code. Maybe I can write this too...

  105.          

       66 GitHubbers (~42%) have contributed personal manifests or project manifests since October
  106. 4 Supportocats 5 Designers 44 Devs    

         7 Ops 3 Trainers 1 Education Liaison 1 Enterprise Salesperson 1 Security Bloke
  107. None
  108. None
  109. really using it

  110. EMBRACE AND EXTEND

  111. create higher level abstractions

  112. class projects::battle_station { boxen::project { 'battle_station': dotenv => true, mysql

    => true, nginx => true, redis => true, ruby => '2.0.0', source => 'github/battle_station' } } This is the same for 80% of cases
  113. let's optimize for our use-case

  114. define github::project( $source = "github/${title}" # a whole bunch of

    other params ) { boxen::project { $name: source => $source, # that bunch of other params } } The sanest default, but easy to override
  115. maybe all of our ops team should get certain things

    by default
  116. class github::environment { include_github_team_environments() }

  117. class github::environment::ops { include github::environment::ops::vagrant include projects::battle_station include projects::octostatus include

    projects::puppet }
  118. START SIMPLE

  119. write new modules inside your repo first and extract later

  120. ~/github/boxen » ls -1 modules cmake ctags github libgithub libmicrohttpd

    nodejs people projects strap
  121. ~/github/boxen » ls -1 modules cmake ctags github libgithub libmicrohttpd

    nodejs people projects strap WIP v2.0.0 nodejs module
  122. start abstractions in Puppet

  123. refactor into Ruby later if performance demands it

  124. define nodejs::version( $ensure = present, $compile = false, $version =

    $title, ) { require nodejs case $ensure { present: { # exec to install } absent: { # exec or file to rm } } }
  125. Time: Filebucket: 0.00 Vagrant plugin: 0.00 Nodejs: 4.37 Sudoers: 0.00

    File line: 0.00 Ini setting: 0.01 Group: 0.10 Repository: 0.12 File: 0.66 Service: 1.54 Package: 112.76 Total: 130.29 Last run: 1365003143 Rbenv gem: 2.23 Exec: 3.36 Config retrieval: 7.27
  126. if it's consistently slowing down the run, then optimize

  127. define nodejs::version( $ensure = present, $compile = false, $version =

    $title, ) { require nodejs nodejs { $version: ensure => $ensure, compile => $compile, } } This impl. happens to fork less
  128. DO WHATEVER WORKS

  129. practicality beats purity / perfect is the enemy of good

    enough
  130. but...

  131. WHY ISN'T THIS JUST A BUNCH OF SCRIPTS?

  132. idempotent behavior is kind of a big deal

  133. shell scripts get the job done if you live in

    a wonderful vacuum where constraints never change and the dependencies between resources are forever static
  134. the real world really doesn't care one bit

  135. telling people "if it breaks, just reinstall" is a copout

  136. if people expect to your software to behave a certain

    way, and you have the power to automate that behavior and then don't, you're being a jerk to your users user experience common sense 101
  137. WHY ISN'T THIS IN CHEF?

  138. so glad people don't ask this one at puppet conferences!

  139. transcend tools; write beautiful porcelain

  140. WHAT ABOUT SECURITY UPDATES?

  141.  THE DARK AGES OF SECURITY INTERACTION

  142.  THE DARK AGES OF SECURITY INTERACTION Java browser plugins

    are literally going to beat you up, kick your puppy, and eat all of your Doritos! zomg!
  143. lol are you kidding me? I have important codez to

    write. I'll do it later. Now be gone, demon! HINT: They're never going to do it.
  144. None
  145.  THE awesome AGES OF SECURITY INTERACTION

  146.  THE awesome AGES OF SECURITY INTERACTION Hey folks! I

    just had to push an update to Boxen to disable Java plugins in browsers. Sorry about that! Here's a PR with more context: <link>.
  147.  THE awesome AGES OF SECURITY INTERACTION Oh snap -

    audit season! Let me just review our code to see if we're in the clear and secure.
  148. what if i have problems?

  149. pull requests

  150. issues

  151. irc.freenode.net

  152. be chuck yeager

  153. Savannah Atlanta

  154. Savannah Atlanta

  155. Savannah Atlanta

  156. Savannah Atlanta

  157. Savannah Atlanta

  158. "So, we're uh in the air. We made it. That's

    uh good news, obviously."
  159. "Flight attendants, if you could just hang on to uh...

    whatever..."
  160. None
  161. The point at which I realize I'm going to die

    The point at which I think the worst is over
  162. detail text

  163.  wfarr speakerdeck.com/wfarr/ boxen-puppetcamp-sf-2013

  164. thanks