fpm-cookery: package artefacts without pain

Commonly in startups, you probably don’t be worry about how your application artefacts are managed to be deployed in a server: compressed or binary files combined with a set of scripts. However, for larger companies that uses not only one or two servers, but a entire pool of machines, it requires some control and organisation to deploy apps.
You can handle this case with some automation / configuration management tools like Ansible, Puppet, Chef, of course. But using a set of scripts can be hard to maintain all deployment steps, from artefact update to rollback. If you distribute your software using a Operational System package manager like YUM (CentOS) or APT (Debian, Ubuntu), your deployment will be easier because these tools handles software changes in a good way.
It sounds good, but package binaries is a pain for developers or newer sysadmins that don’t have knowledge about how YUM/APT works. It requires creation of one or a lot of configuration files to be updated and when you have multiple artefacts probably packaging them will be a nightmare.
Here enters fpm-cookery, a Ruby gem responsible to package artefacts based on recipes, much easy to use. It will be your swiss knife for anything that you need to package, from common binaries not distributed in RPM/DEB formats to programming languages from source, compiling when needed. I will talk how fpm-cookery works, some live demos and introduce Gordon, a CLI built on fpm-cookery focused in package apps without need to write recipes.

Transcript

1. fpm-cookery - package artefacts without pain Marcelo Pinheiro http://salizzar.net - @salizzar
3. $ whoami • Fireman / Problem Solver / Programmer since 2000 • Ruby, Python, Golang, Java, C#, Classic ASP, PHP, Coldfusion, Erlang and others • Fought, made coffee, negotiated deadlines • DevOps Engineer
4. Application Deployment • How is the simplest way to deploy an application in any environment? • Tarballs (zip, tar.gz) • Transfer via FTP / SSH / Ctrl+c Ctrl+v • Common problems • A not-so maintained README explaining how to update it (if exists) • Manual steps • Lack of rastreability (who updated glibc?)
5. Application Deployment • Package Management concept • Dependency Resolution • Filesystem Standardisation • File Integrity • Central Package Repository • Transaction History
6. Application Deployment • How software is updated today? • Configuration Management tool • 1st generation: CFEngine • 2nd generation: Chef, Puppet • 3rd generation: Ansible, SaltStack • Package Manager
7. Packaging is good? Why I need to use it?
8. – A Ruby programmer “Heroku takes care for me!”
9. – A hipster programmer “Docker ships my container to production!”
10. – A darting sysadmin “I can deploy tarballs using my CF tool, man!”
11. – I :) “Packaging is good.”
12. Packaging is good • When your business grow, your software will too • From monolith app to microservices • You need to adopt a standard workflow to distribute software recognisable to all ops people • Package managers like APT/YUM are well-known • Deploy-friendly • First baby step to use Docker later • You can use your custom solution, of course
13. Packaging is good • Configuration Management recipes that manages tarballs tends to be complex: • Manual dependency resolution • Can be a nightmare if requires compilation • gcc on ANY environment server: YOU’RE DOING IT WRONG • Black magic recipes
14. Packaging is good • Configuration Management recipes have support for main package management tools in-a-box as a module • Recipes much more simpler
15. Packaging is good It sounds good, but…
16. It’s complex.
17. A LOT for newbies.
18. RPM Packaging: Challenges • RPM packaging • # yum install -y rpm-build • spec file • one file to rule all
19. DEB Packaging: Challenges • DEB packaging: • # apt-get install -y dpkg-dev • debian folder with the following files: • control • init • rules • preinst, postinst, conffiles, dirs, etc…
20. RPM / DEB: Challenges • Rich documentation, but… • $ man p0rn • Old-fashioned docs • Too many options • High learning curve
21. – All of you here “Hnm, I didn’t like. Forget it.”
22. fpm-cookery for the rescue C’mon, show me the code! >:(
23. fpm-cookery for the rescue • https://github.com/bernd/fpm-cookery • Bernd Ahlers (Graylog) • A tool to build and package anything • Abstraction to Jordan Sissel fpm gem
24. fpm-cookery for the rescue • Package is created with a recipe • class MyPackage < FPM::Cookery::Recipe • Two methods to implement: • build • install • It’s Ruby, man. Enjoy :)
25. fpm-cookery for the rescue • Download tarball with precompiled binaries • Download and compile from source code • Build dependency resolution with Puppet gem • One recipe to package .rpm, .deb and .dmi • Helpers to create package filesystem as you need
26. fpm-cookery Live Demo • How I package a tarball containing binaries? • How I package a tarball with source code?
27. – I, maybe you “It sounds good. But how I can package mundane things like web apps without need to write recipe for all of them?”
28. Gordon: agnostic packaging • https://github.com/salizzar/gordon • Very oppiniated tool to package web / standalone apps • Built with Immutable Infrastructure principles • No need to recipe, just use it as a CLI • Currently in β (a lot of things to do)
29. Gordon: agnostic packaging • Some years ago I wrote a code generator to create Debian package build files • Code generator to create skeletons is great, but after some time you will get a problem when you need to change something • Some examples: • change init scripts to use systemd • add / remove files to project • Apps will change, developers without knowledge of standards will break it
30. Gordon: agnostic packaging • Foreman to generate init files • systemd • Other formats can be included (maybe) • inittab • supervisord • upstart • Set of fpm-cookery generic recipes that handles most cases of web / standalone apps
31. Gordon: agnostic packaging • Generic recipes are based on years of application packaging following good conventions • Operational System standards (Debian, Red Hat / CentOS) • Nginx / Apache / Tomcat / Jetty directories • Application run using their non-interactive shell user or application server user (Tomcat) • Dedicated log directory • /var/run issue
32. Gordon: agnostic packaging • Languages supported • Ruby • Java • Plans to include • Node.js • Golang
33. Gordon Live Demo • How I package a Ruby web app?
34. Questions? O’rly?
35. Thank you! :)