SEATBELTS EVERYONE! SEATBELTS EVERYONE! The Magic Omnibus The Magic Omnibus (or “Piss off distribution apologists with this one weird trick”)

Omnibus? Omnibus?

Omnibus? Omnibus?

No content

No content

Why? Why? ● Distribution packaging policies are not designed for people who package software ● Your package will never get accepted upstream anyway ● If it magically did get accepted upstream, good luck keeping it current ● Native system packages actually have value. It's the bullshit around it that is painful.

What? What? ● A “framework” around FPM. ● Native “build lab” using Vagrant ● Vendors everything

Concepts Concepts ● Project: the defnition of all the software required to build the fnal package ● Software: “recipes” of how to build specifc parts of the package ● Files: static assets to bundle in the fnal package ● Package scripts: Universal postinst/postrm scripts (regardless of package format)

Prerequisites Prerequisites ● Ensure you have vagrant 1.2+ installed ● Install the berkshelf and omnibus plugins for vagrant ● vagrant plugin install vagrant-berkshelf ● vagrant plugin install vagrant-omnibus ● Install the omnibus ruby gem ● gem install –no-ri –no-rdoc omnibus

Workflow Workflow ● Create project ● omnibus project foobar ● Switch to created project ● Note the name is “omnibus-projectname” ● Defne dependencies in “config/project/projectname.rb” ● Create any custom “software” defnitions you need in “config/software” ● vagrant up ● Go make some coffee

What's happening behind the What's happening behind the scenes scenes ● Vagrant starts up and... ● Installs chef and runs the omnibus cookbook (preps the machine to be a build lab) ● omnibus build project projectname is run on your project ● Omnibus fetches defned deps (in order), compiles them and installs to /opt/projectname (inside the vm) ● Omnibus runs ldd on /opt/projectname to ensure that all necessary libraries are satisfed in /opt/projectname ● Calls fpm to generate the system package ● Drops of fnal packages in the pkg directory ● Drops off a json metadata fle about the package

Gotchas Gotchas ● Any dependencies that aren't in the confg/software directory are pulled from the master branch of github.com/opscode/omnibus-software. ● THIS IS HARDCODED INTO OMNIBUS ● Omnibus-software exists frst and foremost to satisfy the needs of Opscode for building chef and chef-server packages. ● Yes this is as bad but not THAT bad. Luckily local defnitions satisfy the search frst. ● Builds across all platforms take FOREVER ● Omnibus is a jerk about flename & DSL syntax

Tips and Tricks Tips and Tricks ● Customize the Vagrantfile ● By default it will build two copies of ruby (in a ruby project) ● One for the omnibus tool chain ● One for your ruby deps ● But it also installs Chef with the Vagrant omnibus plugin which includes ruby and rubygems ● RUBY ALL OVER THE PLACE. HOW IS RUBBY FORMED?

Tips and Tricks Tips and Tricks ● You don't have to use Chef at all ● Disable the omnibus vagrant plugin ● Customize the Vagrantfle ● Omnibus only needs ruby 1.9+ with the omnibus gem installed to build your project ● ccache doesn't really help ● Use the s3 dependency cache otherwise your builds will randomly fail because rpm5.org is offine and you can't download popt-1.6.tar.gz and oh my god I can't fnd another mirror. ● Better yet just host the deps yourself

A real-world use case that isn't A real-world use case that isn't Opscode Chef Opscode Chef ● Our platform supports CentOS/RHEL6 and Ubuntu 12.04 ● It's not a standalone software product

No content

Use case continued Use case continued ● We use chef for managing our hosted platform ● Leverage existing process for on-premises installs ● Needs ● chef-solo ● riak ● rabbitmq (which needs erlang) ● mysql ● haproxy ● modern python for some tooling

Enterprise Problems? Enterprise Problems? ● Nice self-contained chef package ● Nice largely self-contained Riak package ● uses rebar ● but still has distro openssl dependency ● Native RabbitMQ system packages ● but needs Erlang (EPEL and EPEL Erlang repos) ● Python packages and Ruby Gems ● Good luck with that

● Started with just python 2.7 and a few pip packages ● Ran into a customer with entirely isolated network but needed MySQL rubygem ● Issues over stale internal repo mirrors and missing packages

End result End result ● The “distro-in-a-package” ● We vendored EVERYTHING that wasn't our java code (excluding the JVM)

No content

End result End result ● One package per supported distro ● ~300MB system package ● ~1G installed size ● We now install three things on the systems: ● The “dmcm-base” system package ● Our java application parts ● OpenJDK6 (which is in the base system repos) ● And we install it on EVERY system ● The differentiator isn't what's installed. It's what services are running.

Next steps Next steps ● Package our java components into a system package by pulling the tarball artifacts from Artifactory (same level of determinism) ● Publish the software defnitions we use (and possibly the whole repo) for building the base package ● Work with Opscode to address the Opscode- specifcness of Omnibus ● Better upgrade handling ● Custom “omnitruck” service

Sample Omnibus Software Sample Omnibus Software ● https://github.com/lusis/omnibus-ruby-project ● https://github.com/lusis/sample-python- omnibus-app ● https://github.com/lusis/omnibus-omnibus- omnibus ● https://github.com/lusis/omnibus-riak ● https://github.com/bakins/omnibus-nginx

Q&A Q&A ● twitter.com/lusis ● github.com/lusis ● github.com/enstratus ● speakerdeck.com/lusis