Chef and Docker

Talk explaining the various tools available for using Chef with Docker. Given at Bristol DevOps 1st October 2014.

Transcript

1. Chef and Docker Andy Gale DevOps Consultancy
2. About me Andy Gale Web Consultant Hello Future http://hellofutu.re ! @andygale @hellofutur3 DevOps Consultancy
3. • Heard about Docker on the The Ship Show • Based on Linux Containers! • Linux Containers can be difficult to set up • Extra Docker tooling looked fantastic That looks really cool DevOps Consultancy
4. • Who needs Vagrant development environments, Chef and Puppet when we can just deploy Docker images? • I went through the getting started guide; wow this is all very clever • Got as far as the Dockerfile That looks really cool DevOps Consultancy
5. # Nginx # # VERSION 0.0.1 ! FROM ubuntu MAINTAINER Victor Vieux <victor@docker.com> ! RUN apt-get update && apt-get install -y inotify-tools nginx apache2 openssh-server Dockerfile Hang on, that’s basically a bash script!
7. + Solution * or Puppet (there’s a bit for you lot later) * DevOps Consultancy
8. Why use with ? • Define your container configuration using Chef cookbooks rather than with Dockerfile/bash • Idempotently manage the running state of your docker container • Manage multiple services (i.e. have more than one process running) DevOps Consultancy
9. • Easier installation and configuration of complex applications using existing well tested Chef cookbooks • Move existing setup to containers • Handling final configuration when container boots Why use with ? DevOps Consultancy
10. • Consistency over mixed architectures • Use the same configuration management for development, staging and production • Use Docker containers for development of multi-tiered SOA applications but still deploy to bare metal or cloud instances in production Why use with ? DevOps Consultancy
11. tools for working with • Knife Container
 Knife plugin which gives it the ability to initialise and build Linux containers
 http://docs.getchef.com/plugin_knife_container.html • Chef Container
 Solves the Docker PID1 problem by running chef-client on boot and managing multiple processes with runit
 https://docs.getchef.com/containers.html DevOps Consultancy
12. tools for working with • Chef Metal
 Library that solves the problem of repeatably creating machines and infrastructures in Chef
 https://github.com/opscode/chef-metal • Docker cookbook
 Chef cookbook which installs Docker
 https://supermarket.getchef.com/cookbooks/docker DevOps Consultancy
13. Knife Container DevOps Consultancy
14. Knife Container $ chef gem install knife-container Install with the Chef DK $ gem install knife-container Install via RubyGems $ /opt/chef/embedded/bin/gem install knife-container RubyGems if Chef was installed via Omnibus Installer DevOps Consultancy
15. Knife Container docker init $ knife container docker init docker -r 'recipe[apache2]' -z -b -f chef/ubuntu_14.04 Used to set up a Dockerfile context for the local workstation Example: Creates a Dockerfile like: FROM chef/ubuntu_14.04 ADD chef /etc/chef RUN chef-init --bootstrap ENTRYPOINT ["chef-init"] CMD ["--onboot"] DevOps Consultancy
16. Knife Container docker build $ knife container docker build NAMESPACE/IMAGE_NAME Builds Docker image, resolves Chef dependencies and cleans up Chef artefacts Example: DevOps Consultancy
17. Chef Container DevOps Consultancy
18. Chef Container No installation required as the official Chef Docker images have chef-container preinstalled DevOps Consultancy
19. Chef Container Runs chef-client inside containers Image is licensed under a Creative Commons Attribution 3.0 Unported License. From https://docs.getchef.com/containers.html DevOps Consultancy
20. Chef Container • Your application probably needs some environment dependent configuration • Protects your container from configuration drift • Use same set up for production, staging, Vagrant/CI and CD Why run Chef in your container? DevOps Consultancy
21. Chef Metal DevOps Consultancy
22. Chef Metal $ chef gem install chef-metal Install with the Chef DK $ gem install chef-metal Install via RubyGems $ /opt/chef/embedded/bin/gem install chef-metal RubyGems if Chef was installed via Omnibus Installer chef_gem 'chef-metal' do action :install end Or in a Chef recipe DevOps Consultancy
23. Chef Metal machine 'db' do recipe 'mysql' end ! machine 'web1' do recipe 'apache' end Use Chef Metal to create clusters DevOps Consultancy
24. Chef Metal • Adds machine resource • Modular so you can have different drivers • Just run chef-client on the machine looking after the cluster rather than on each machine • That’s good if you don’t want chef-client running all the time in each your container DevOps Consultancy
25. Chef Metal machine 'db' do recipe 'mysql' end ! machine 'web1' do recipe 'apache' end ! machine 'web2' do recipe 'apache' end Add a new “machine” DevOps Consultancy
26. Chef Metal machine 'db' do recipe 'mysql' end ! 1.upto(50) do |i| machine "web#{i}" do recipe 'apache' end end Upscaling DevOps Consultancy
27. Chef Metal • chef-metal-fog
 EC2, Digital Ocean, OpenStack etc • chef-metal-vagrant • chef-metal-ssh • chef-metal-lxc • chef-metal-docker Drivers DevOps Consultancy
28. Chef Metal with Docker DevOps Consultancy
29. Chef Metal with Docker $ chef gem install chef-metal-docker Install with the Chef DK $ gem install chef-metal-docker Install via RubyGems $ /opt/chef/embedded/bin/gem install chef-metal-docker RubyGems if Chef was installed via Omnibus Installer chef_gem ‘chef-metal-docker' do action :install end Or in a Chef recipe DevOps Consultancy
30. Chef Metal with Docker require 'chef_metal_docker' ! machine 'wario' do recipe 'openssh::default' ! machine_options :docker_options => { :base_image => { :name => 'ubuntu', :repository => 'ubuntu', :tag => '14.04' }, :command => '/usr/sbin/sshd -p 8022 -D', :ports => 8022 } end Create container DevOps Consultancy
31. Chef Metal with Docker require 'chef_metal_docker' ! machine_image 'web_server' do recipe 'apache' ! machine_options :docker_options => { :base_image => { :name => 'ubuntu', :repository => 'ubuntu', :tag => '14.04' } } end ! machine 'web00' do from_image 'web_server' ! machine_options :docker_options => { :command => '/usr/sbin/httpd' } end Creates image Creates container from the image DevOps Consultancy
32. #! # Cookbook Name:: hf-chef-metal-docker! # Recipe:: setup! #! ! node.default['build-essential']['compile_time'] = true! ! include_recipe 'build-essential'! include_recipe 'aufs'! ! node.set['docker']['package']['repo_url'] = 'https://get.docker.io/ubuntu'! node.set['docker']['storage_driver'] = 'aufs'! ! include_recipe 'docker'! ! chef_gem 'chef-metal' do! action :install! end! ! package 'lxc-dev' do! action :install! notifies :install, 'chef_gem[chef-metal-docker]', :immediately! end! ! chef_gem 'chef-metal-docker' do! action :nothing! end! Chef Metal with Docker Example recipe that sets up Docker and chef-metal-docker on your Docker host machine (Tested on Ubuntu 14.04 only) DevOps Consultancy
33. Docker cookbook DevOps Consultancy
34. Docker cookbook • Get from:
 https://supermarket.getchef.com/cookbooks/docker • Installs Docker for you • Also contains LWRPs to manage Docker images and containers DevOps Consultancy
35. Docker cookbook # Pull latest image docker_image 'samalba/docker-registry' ! # Run container exposing ports docker_container 'samalba/docker-registry' do detach true port '5000:5000' env 'SETTINGS_FLAVOR=local' volume '/mnt/docker:/docker-storage' end ! # Login to private registry docker_registry 'https://docker-registry.example.com/' do username 'shipper' password 'iloveshipping' end ! # Pull tagged image docker_image 'apps/crowsnest' do tag 'not-latest' end Example LWRPs DevOps Consultancy
36. Docker cookbook # Run container docker_container 'crowsnest' ! # Save current timestamp timestamp = Time.new.strftime('%Y%m%d%H%M') ! # Commit container changes docker_container 'crowsnest' do repository 'apps' tag timestamp action :commit end ! # Push image docker_image 'crowsnest' do repository 'apps' tag timestamp action :push end Example LWRPs DevOps Consultancy
37. Docker cookbook # Conditionally rebuild image if changes upstream:! ! git "#{Chef::Config[:file_cache_path]}/docker-testcontainerd" do! repository 'git@github.com:bflad/docker-testcontainerd.git'! notifies :build, 'docker_image[bflad/testcontainerd]', :immediately! end! ! docker_image 'bflad/testcontainerd' do! action :pull_if_missing! end! Example LWRPs DevOps Consultancy
38. Puppet DevOps Consultancy
39. Puppet • Gareth Rushgrove who runs DevOps Weekly used to work for GDS and now works for Puppet
 http://www.devopsweekly.com/ • Puppet module with similar functionality • https://forge.puppetlabs.com/garethr/docker DevOps Consultancy
40. Puppet include 'docker'! ! docker::image { 'ubuntu':! image_tag => 'precise'! }! ! docker::run { 'helloworld':! image => 'ubuntu:precise',! command => '/bin/sh -c "while true; do echo! hello world; sleep 1; done"',! } DevOps Consultancy Simple example
41. Puppet docker::run { 'helloworld':! image => 'base',! command => '/bin/sh -c "while true; do echo hello world; ! ! ! ! ! ! ! ! sleep 1; done"',! ports => ['4444', '4555'],! expose => ['4666', '4777'],! links => ['mysql:db'],! use_name => true,! volumes => ['/var/lib/couchdb', '/var/log'],! volumes_from => '6446ea52fbc9',! memory_limit => 10m, # (format: <number><unit>, where unit = b, k, m or g)! username => 'example',! hostname => 'example.com',! env => ['FOO=BAR', 'FOO2=BAR2'],! dns => ['8.8.8.8', '8.8.4.4'],! restart_service => true,! privileged => false,! } DevOps Consultancy More complicated example
42. Better with configuration management! DevOps Consultancy
43. Questions? DevOps Consultancy https://github.com/salgo/chef-and-docker-talk Code examples