Packaging Rails Applications

This presentation explores the process of how we package Rails apps for deployment at Lonely Planet.

I gave this presentation on March 6, 2014 to the Nash.rb user group in Nashvile, TN.

Transcript

1. Packaging Rails Apps Lee Jones Web Operations @ Lonely Planet Hello everyone! My name is Lee Jones. I work in web operations at Lonely Planet.
2. Explore the Planet Lonely Planet is a company that helps people travel. A big part of how we do that online is powered by various Ruby on Rails applications behind the scenes. Today, I want to share how we use packages to deploy those Rails apps.
3. Packages! A package is a collection of software that is gathered together into a format that can be distributed to other systems. A package may include metadata like a name, version, description, and dependencies. This metadata helps the packaging system know how to install the package.
4. Packages? So you might be thinking, “Rails apps as packages?”. Yeah, we typically think of packages for system level software like web servers and databases. Before I joined Lonely Planet I wasn’t familiar with this pattern, but the more I’ve seen it in action, the more I like it. It brings some nice advantages that we’ll get to shortly.
5. typical deployment A typical rails deployment involves 5 key parts: fetching source code, installing gems, compiling assets, migrating the database, and restarting the app. Our packaging process handles 4/5 of those steps. We handle migrations separately. Looks look at how our packaging process works.
6. automated We often deploy 15 or more times per day so a key point of the process I’m about to describe is that it is completely automated. There is no one hand crafting these packages and tweaking configuration flags. When a developer gets the :thumbsup: on a PR and merges into master, our continuous integration server (Jenkins) begins the automated process of moving that code to production.
7. tests: pass Step 0 is run the tests. If our tests pass, we are ready to begin packaging the Rails app.
8. <% code %> The first thing we need to gather is the source code. We pull the code down into clean workspace in preparation for building the package.
9. gems Next, we include gems in our application packages so that it’s more self-contained and so the actual deploy is faster. We add the deployment flag and the path flag so that gems are installed into the vendor directory instead of on the system. One thing to note here is that you want to make sure that your build system closely matches production because otherwise you’ll run into issue with libraries being in different places etc.
10. assets We precompile the assets on our build server, but we don’t include them in the package because our app servers do not serve any asset requests. We push these up to S3 to be served by our CDN. We DO however include the asset manifests in the package because the application needs those during startup.
11. cleanup We’ve gathered what we need for the release, so now we can remove what we don’t need: log, tmp, doc, spec, features, and even .git. This saves on storage space and transfer time.
12. jordansissel/fpm It’s now time to build a package. For that we use fpm. fpm is an awesome tool that creates a nice abstracted interface to iron out the quirks of various packaging systems. We run Ubuntu Server in production so we output .deb files, but you could adapt this to your platform of choice.
13. > fpm We pass a several options to fpm including:
14. --name foo Make sure you pick a name that will not conflict with other system packages you use.
15. --version x.y.z Each release needs a unique version number that increments over time. We use the Jenkins build number to help generate this.
16. --description “An application that does foo - $GIT_COMMIT” Adding the commit revision to the description makes it easier to trace back a particular package version to its revision in the source code repository.
17. --depends libxml Many Rails apps depend on system level software being installed. For example, the nokogiri gem depends on libxml and libxslt (available as packages). We can specify those dependencies here and they will be installed along with the app.
18. krobertson/deb-s3 Once we’ve built the package, we push it up to our private package repository. The repository itself is static files served over http. We host the package repository on S3 so it is pretty fast and very reliable. The tool we use to manage the repository is deb-s3.
19. > apt-get update To deploy the package we first need to update the list of known packages on each server.
20. > apt-get install foo When we call the install command apt will recognize that a newer version of the package is available. It installs the newer version and restarts the application using a configured post-install hook.
21. fast reliable low-impact That’s an overview of how we do it. It works well for us because it’s fast, reliable, and low-impact.
22. fast reliable low-impact It’s fast because when the deploy step happens we only need to copy the bits from the package repository and restart. This also really nice when we need to scale up or replace a server. Rollbacks are fast too because the repository (or better yet, the local cache) has previous version so we can quickly roll back to a previous version.
23. fast reliable low-impact It is reliable because we’ve isolated our interaction with 3rd party services to the packaging step. We’re not making calls to fetch source code and gems over X number of servers for each deploy. S3 is fairly reliable and the package repository has no moving parts so it makes a solid dependency. In the past 6 months we’ve not had an issue related to deploying the packages.
24. fast reliable low-impact It’s low-impact because we don’t actually need to do any work on the servers (ex. compiling assets or installing gems, etc).
25. Thank You We’re hiring! lonelyplanet.com/jobs I hope that gives you some ideas of how to optimize your deployment process. For me, the big takeaway (whether you use packages or not) is to focus on making your deployment fast, reliable, and low-impact. Thank you for letting me describe this process. I’d love to hear any feedback or questions you might have for me.