The Magic Omnibus

Presentation about omnibus at DevOpsAtl

1. SEATBELTS EVERYONE! SEATBELTS EVERYONE! The Magic Omnibus The Magic Omnibus (or “Piss off distribution apologists with this one weird trick”)
2. Omnibus? Omnibus?
3. Omnibus? Omnibus?
6. Why? Why? • Distribution packaging policies are not designed for people who package software • Your package will never get accepted upstream anyway • If it magically did get accepted upstream, good luck keeping it current • Native system packages actually have value. It's the bullshit around it that is painful.
7. What? What? • A “framework” around FPM. • Native “build lab” using Vagrant • Vendors everything
8. Concepts Concepts • Project: the defnition of all the software required to build the fnal package • Software: “recipes” of how to build specifc parts of the package • Files: static assets to bundle in the fnal package • Package scripts: Universal postinst/postrm scripts (regardless of package format)
9. Prerequisites Prerequisites • Ensure you have vagrant 1.2+ installed • Install the berkshelf and omnibus plugins for vagrant • vagrant plugin install vagrant-berkshelf • vagrant plugin install vagrant-omnibus • Install the omnibus ruby gem • gem install –no-ri –no-rdoc omnibus
10. Workflow Workflow • Create project • omnibus project foobar • Switch to created project • Note the name is “omnibus-projectname” • Defne dependencies in “config/project/projectname.rb” • Create any custom “software” defnitions you need in “config/software” • vagrant up • Go make some coffee
11. What's happening behind the What's happening behind the scenes scenes • Vagrant starts up and... • Installs chef and runs the omnibus cookbook (preps the machine to be a build lab) • omnibus build project projectname is run on your project • Omnibus fetches defned deps (in order), compiles them and installs to /opt/projectname (inside the vm) • Omnibus runs ldd on /opt/projectname to ensure that all necessary libraries are satisfed in /opt/projectname • Calls fpm to generate the system package • Drops of fnal packages in the pkg directory • Drops off a json metadata fle about the package
12. Gotchas Gotchas • Any dependencies that aren't in the confg/software directory are pulled from the master branch of github.com/opscode/omnibus-software. • THIS IS HARDCODED INTO OMNIBUS • Omnibus-software exists frst and foremost to satisfy the needs of Opscode for building chef and chef-server packages. • Yes this is as bad but not THAT bad. Luckily local defnitions satisfy the search frst. • Builds across all platforms take FOREVER • Omnibus is a jerk about flename & DSL syntax
13. Tips and Tricks Tips and Tricks • Customize the Vagrantfile • By default it will build two copies of ruby (in a ruby project) • One for the omnibus tool chain • One for your ruby deps • But it also installs Chef with the Vagrant omnibus plugin which includes ruby and rubygems • RUBY ALL OVER THE PLACE. HOW IS RUBBY FORMED?
14. Tips and Tricks Tips and Tricks • You don't have to use Chef at all • Disable the omnibus vagrant plugin • Customize the Vagrantfle • Omnibus only needs ruby 1.9+ with the omnibus gem installed to build your project • ccache doesn't really help • Use the s3 dependency cache otherwise your builds will randomly fail because rpm5.org is offine and you can't download popt-1.6.tar.gz and oh my god I can't fnd another mirror. • Better yet just host the deps yourself
15. A real-world use case that isn't A real-world use case that isn't Opscode Chef Opscode Chef • Our platform supports CentOS/RHEL6 and Ubuntu 12.04 • It's not a standalone software product
17. Use case continued Use case continued • We use chef for managing our hosted platform • Leverage existing process for on-premises installs • Needs • chef-solo • riak • rabbitmq (which needs erlang) • mysql • haproxy • modern python for some tooling
18. Enterprise Problems? Enterprise Problems? • Nice self-contained chef package • Nice largely self-contained Riak package • uses rebar • but still has distro openssl dependency • Native RabbitMQ system packages • but needs Erlang (EPEL and EPEL Erlang repos) • Python packages and Ruby Gems • Good luck with that
19. • Started with just python 2.7 and a few pip packages • Ran into a customer with entirely isolated network but needed MySQL rubygem • Issues over stale internal repo mirrors and missing packages
20. End result End result • The “distro-in-a-package” • We vendored EVERYTHING that wasn't our java code (excluding the JVM)
22. End result End result • One package per supported distro • ~300MB system package • ~1G installed size • We now install three things on the systems: • The “dmcm-base” system package • Our java application parts • OpenJDK6 (which is in the base system repos) • And we install it on EVERY system • The differentiator isn't what's installed. It's what services are running.
23. Next steps Next steps • Package our java components into a system package by pulling the tarball artifacts from Artifactory (same level of determinism) • Publish the software defnitions we use (and possibly the whole repo) for building the base package • Work with Opscode to address the Opscode- specifcness of Omnibus • Better upgrade handling • Custom “omnitruck” service
24. Sample Omnibus Software Sample Omnibus Software • https://github.com/lusis/omnibus-ruby-project • https://github.com/lusis/sample-python- omnibus-app • https://github.com/lusis/omnibus-omnibus- omnibus • https://github.com/lusis/omnibus-riak • https://github.com/bakins/omnibus-nginx
25. Q&A Q&A • twitter.com/lusis • github.com/lusis • github.com/enstratus • speakerdeck.com/lusis