Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Chef and Docker

Andy Gale
October 01, 2014

Chef and Docker

Talk explaining the various tools available for using Chef with Docker. Given at Bristol DevOps 1st October 2014.

Andy Gale

October 01, 2014

More Decks by Andy Gale

Other Decks in Technology


  1. • Heard about Docker on the The Ship Show •

    Based on Linux Containers! • Linux Containers can be difficult to set up • Extra Docker tooling looked fantastic That looks really cool DevOps Consultancy
  2. • Who needs Vagrant development environments, Chef and Puppet when

    we can just deploy Docker images? • I went through the getting started guide; wow this is all very clever • Got as far as the Dockerfile That looks really cool DevOps Consultancy
  3. # Nginx # # VERSION 0.0.1 ! FROM ubuntu MAINTAINER

    Victor Vieux <[email protected]> ! RUN apt-get update && apt-get install -y inotify-tools nginx apache2 openssh-server Dockerfile Hang on, that’s basically a bash script!
  4. + Solution * or Puppet (there’s a bit for you

    lot later) * DevOps Consultancy
  5. Why use with ? • Define your container configuration using

    Chef cookbooks rather than with Dockerfile/bash • Idempotently manage the running state of your docker container • Manage multiple services (i.e. have more than one process running) DevOps Consultancy
  6. • Easier installation and configuration of complex applications using existing

    well tested Chef cookbooks • Move existing setup to containers • Handling final configuration when container boots Why use with ? DevOps Consultancy
  7. • Consistency over mixed architectures • Use the same configuration

    management for development, staging and production • Use Docker containers for development of multi-tiered SOA applications but still deploy to bare metal or cloud instances in production Why use with ? DevOps Consultancy
  8. tools for working with • Knife Container
 Knife plugin which

    gives it the ability to initialise and build Linux containers
 http://docs.getchef.com/plugin_knife_container.html • Chef Container
 Solves the Docker PID1 problem by running chef-client on boot and managing multiple processes with runit
 https://docs.getchef.com/containers.html DevOps Consultancy
  9. tools for working with • Chef Metal
 Library that solves

    the problem of repeatably creating machines and infrastructures in Chef
 https://github.com/opscode/chef-metal • Docker cookbook
 Chef cookbook which installs Docker
 https://supermarket.getchef.com/cookbooks/docker DevOps Consultancy
  10. Knife Container $ chef gem install knife-container Install with the

    Chef DK $ gem install knife-container Install via RubyGems $ /opt/chef/embedded/bin/gem install knife-container RubyGems if Chef was installed via Omnibus Installer DevOps Consultancy
  11. Knife Container docker init $ knife container docker init docker

    -r 'recipe[apache2]' -z -b -f chef/ubuntu_14.04 Used to set up a Dockerfile context for the local workstation Example: Creates a Dockerfile like: FROM chef/ubuntu_14.04 ADD chef /etc/chef RUN chef-init --bootstrap ENTRYPOINT ["chef-init"] CMD ["--onboot"] DevOps Consultancy
  12. Knife Container docker build $ knife container docker build NAMESPACE/IMAGE_NAME

    Builds Docker image, resolves Chef dependencies and cleans up Chef artefacts Example: DevOps Consultancy
  13. Chef Container No installation required as the official Chef Docker

    images have chef-container preinstalled DevOps Consultancy
  14. Chef Container Runs chef-client inside containers Image is licensed under

    a Creative Commons Attribution 3.0 Unported License. From https://docs.getchef.com/containers.html DevOps Consultancy
  15. Chef Container • Your application probably needs some environment dependent

    configuration • Protects your container from configuration drift • Use same set up for production, staging, Vagrant/CI and CD Why run Chef in your container? DevOps Consultancy
  16. Chef Metal $ chef gem install chef-metal Install with the

    Chef DK $ gem install chef-metal Install via RubyGems $ /opt/chef/embedded/bin/gem install chef-metal RubyGems if Chef was installed via Omnibus Installer chef_gem 'chef-metal' do action :install end Or in a Chef recipe DevOps Consultancy
  17. Chef Metal machine 'db' do recipe 'mysql' end ! machine

    'web1' do recipe 'apache' end Use Chef Metal to create clusters DevOps Consultancy
  18. Chef Metal • Adds machine resource • Modular so you

    can have different drivers • Just run chef-client on the machine looking after the cluster rather than on each machine • That’s good if you don’t want chef-client running all the time in each your container DevOps Consultancy
  19. Chef Metal machine 'db' do recipe 'mysql' end ! machine

    'web1' do recipe 'apache' end ! machine 'web2' do recipe 'apache' end Add a new “machine” DevOps Consultancy
  20. Chef Metal machine 'db' do recipe 'mysql' end ! 1.upto(50)

    do |i| machine "web#{i}" do recipe 'apache' end end Upscaling DevOps Consultancy
  21. Chef Metal • chef-metal-fog
 EC2, Digital Ocean, OpenStack etc •

    chef-metal-vagrant • chef-metal-ssh • chef-metal-lxc • chef-metal-docker Drivers DevOps Consultancy
  22. Chef Metal with Docker $ chef gem install chef-metal-docker Install

    with the Chef DK $ gem install chef-metal-docker Install via RubyGems $ /opt/chef/embedded/bin/gem install chef-metal-docker RubyGems if Chef was installed via Omnibus Installer chef_gem ‘chef-metal-docker' do action :install end Or in a Chef recipe DevOps Consultancy
  23. Chef Metal with Docker require 'chef_metal_docker' ! machine 'wario' do

    recipe 'openssh::default' ! machine_options :docker_options => { :base_image => { :name => 'ubuntu', :repository => 'ubuntu', :tag => '14.04' }, :command => '/usr/sbin/sshd -p 8022 -D', :ports => 8022 } end Create container DevOps Consultancy
  24. Chef Metal with Docker require 'chef_metal_docker' ! machine_image 'web_server' do

    recipe 'apache' ! machine_options :docker_options => { :base_image => { :name => 'ubuntu', :repository => 'ubuntu', :tag => '14.04' } } end ! machine 'web00' do from_image 'web_server' ! machine_options :docker_options => { :command => '/usr/sbin/httpd' } end Creates image Creates container from the image DevOps Consultancy
  25. #! # Cookbook Name:: hf-chef-metal-docker! # Recipe:: setup! #! !

    node.default['build-essential']['compile_time'] = true! ! include_recipe 'build-essential'! include_recipe 'aufs'! ! node.set['docker']['package']['repo_url'] = 'https://get.docker.io/ubuntu'! node.set['docker']['storage_driver'] = 'aufs'! ! include_recipe 'docker'! ! chef_gem 'chef-metal' do! action :install! end! ! package 'lxc-dev' do! action :install! notifies :install, 'chef_gem[chef-metal-docker]', :immediately! end! ! chef_gem 'chef-metal-docker' do! action :nothing! end! Chef Metal with Docker Example recipe that sets up Docker and chef-metal-docker on your Docker host machine (Tested on Ubuntu 14.04 only) DevOps Consultancy
  26. Docker cookbook • Get from:
 https://supermarket.getchef.com/cookbooks/docker • Installs Docker for

    you • Also contains LWRPs to manage Docker images and containers DevOps Consultancy
  27. Docker cookbook # Pull latest image docker_image 'samalba/docker-registry' ! #

    Run container exposing ports docker_container 'samalba/docker-registry' do detach true port '5000:5000' env 'SETTINGS_FLAVOR=local' volume '/mnt/docker:/docker-storage' end ! # Login to private registry docker_registry 'https://docker-registry.example.com/' do username 'shipper' password 'iloveshipping' end ! # Pull tagged image docker_image 'apps/crowsnest' do tag 'not-latest' end Example LWRPs DevOps Consultancy
  28. Docker cookbook # Run container docker_container 'crowsnest' ! # Save

    current timestamp timestamp = Time.new.strftime('%Y%m%d%H%M') ! # Commit container changes docker_container 'crowsnest' do repository 'apps' tag timestamp action :commit end ! # Push image docker_image 'crowsnest' do repository 'apps' tag timestamp action :push end Example LWRPs DevOps Consultancy
  29. Docker cookbook # Conditionally rebuild image if changes upstream:! !

    git "#{Chef::Config[:file_cache_path]}/docker-testcontainerd" do! repository '[email protected]:bflad/docker-testcontainerd.git'! notifies :build, 'docker_image[bflad/testcontainerd]', :immediately! end! ! docker_image 'bflad/testcontainerd' do! action :pull_if_missing! end! Example LWRPs DevOps Consultancy
  30. Puppet • Gareth Rushgrove who runs DevOps Weekly used to

    work for GDS and now works for Puppet
 http://www.devopsweekly.com/ • Puppet module with similar functionality • https://forge.puppetlabs.com/garethr/docker DevOps Consultancy
  31. Puppet include 'docker'! ! docker::image { 'ubuntu':! image_tag => 'precise'!

    }! ! docker::run { 'helloworld':! image => 'ubuntu:precise',! command => '/bin/sh -c "while true; do echo! hello world; sleep 1; done"',! } DevOps Consultancy Simple example
  32. Puppet docker::run { 'helloworld':! image => 'base',! command => '/bin/sh

    -c "while true; do echo hello world; ! ! ! ! ! ! ! ! sleep 1; done"',! ports => ['4444', '4555'],! expose => ['4666', '4777'],! links => ['mysql:db'],! use_name => true,! volumes => ['/var/lib/couchdb', '/var/log'],! volumes_from => '6446ea52fbc9',! memory_limit => 10m, # (format: <number><unit>, where unit = b, k, m or g)! username => 'example',! hostname => 'example.com',! env => ['FOO=BAR', 'FOO2=BAR2'],! dns => ['', ''],! restart_service => true,! privileged => false,! } DevOps Consultancy More complicated example