Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Puppet at SpaceX: How to stay sane, while sending rockets into space

Puppet Labs
October 25, 2012

Puppet at SpaceX: How to stay sane, while sending rockets into space

The much-requested slides from Joachim "Jok" Thuau of SpaceX speaking on "Puppet at SpaceX". From PuppetCamp LA '12. Jok Thuau gave a similar talk you can watch here: http://youtu.be/YcxEmjkulU8

Abstract: SpaceX manages a large number of machines using puppet. A quick overview of where puppet fits and how we came to use it.

Speaker Bio: I have been a system administrator for over 10 years, managing everything from networking to Windows, Linux and Macs. Before joining the SpaceX team, I worked in the video game industry. I have always been fascinated by computers, video games and space travel, and am lucky enough to have gotten to work in all three areas. I enjoy reading, subverting devices and spending time with my family.

Watch other PuppetConf videos you missed out on here: http://www.puppetlabs.com/community/videos/puppetconf

Be the first to know about PuppetConf 2013:

Learn more: www.puppetlabs.com

Puppet Labs

October 25, 2012

More Decks by Puppet Labs

Other Decks in Technology


  1. Puppet at SpaceX
    How to stay sane, while sending rockets into space

    View full-size slide

  2. Who’s this guy anyway?
    —  Linux system admin
    —  Came from the video game industry
    —  Started at SpaceX about 9 month ago
    —  Linux System Engineer

    View full-size slide

  3. SpaceX
    —  Rocket Company
    —  First private company to send a vehicle to LEO
    —  AND recover the vehicle
    —  Founded in 2002
    —  Falcon1, Falcon 9, Dragon

    View full-size slide

  4. Puppet
    —  How it happened
    —  Before Scale 10x
    —  After Scale 10x

    View full-size slide

  5. Bootstrapping
    —  What we have
    —  Debian/Ubuntu
    —  Kerberos (windows KDC/Active Directory)
    —  LDAP (windows LDAP/Active Directory)
    —  “Apt” mirror (with debian squeeze/ubuntu LTS)
    —  Tons of scripts!

    View full-size slide

  6. Deployment
    —  Using existing scripts and replacing them with
    —  Install packages from our local repo
    —  Push configs with puppet
    —  Using exec (in a couple of places only) to “fix” a
    couple of things.
    —  This simplified things a lot for us

    View full-size slide

  7. Where are we at now?
    —  Plug desktop/server in (power/network/keyboard/
    —  Boot “off the network” (pxe)
    —  Pick on pxe menu what you want to install
    —  We have a number of “presets” we use
    —  Install starts and asks the user for hostname
    —  Machine uses a pre-seed file to setup stuff initially
    —  Installs puppet as part of that pre-seed from our local
    —  Generate and registers with puppet server
    —  (login on server and sign cert)
    —  And wait for it! We’re DONE!

    View full-size slide

  8. Cool stuff to come
    —  Puppet server infrastructure overview
    —  TheForeman
    —  Really nice and easy to setup
    —  Has “proxies” for handling stuff with DHCP/DNS/PXE
    automation (we’re planning on using this, and it’s a little
    more difficult for us, since that part of the infrastructure is
    windows based)
    —  Nagios monitoring
    —  Host lists/services/dependencies built by puppet
    —  HPC
    —  We have a compute cluster that we are starting to rebuild
    using puppet for automation and validation (test cluster
    with a couple of nodes, validate config, and push to

    View full-size slide