Chef and Docker

Talk explaining the various tools available for using Chef with Docker. Given at Bristol DevOps 1st October 2014.

Transcript

1. Chef and Docker
   Andy Gale
   DevOps Consultancy
   

   View Slide

2. About me
   Andy Gale
   Web Consultant
   Hello Future
   http://hellofutu.re
   !
   @andygale
   @hellofutur3
   DevOps Consultancy
   

   View Slide

3. • Heard about Docker on the The Ship Show
   • Based on Linux Containers!
   • Linux Containers can be difficult to set up
   • Extra Docker tooling looked fantastic
   That looks really cool
   DevOps Consultancy
   

   View Slide

4. • Who needs Vagrant development
   environments, Chef and Puppet when we can
   just deploy Docker images?
   • I went through the getting started guide;
   wow this is all very clever
   • Got as far as the Dockerfile
   That looks really cool
   DevOps Consultancy
   

   View Slide

5. # Nginx
   #
   # VERSION 0.0.1
   !
   FROM ubuntu
   MAINTAINER Victor Vieux
   !
   RUN apt-get update && apt-get install -y inotify-tools nginx apache2
   openssh-server
   Dockerfile
   Hang on, that’s basically a bash script!
   
   

   View Slide

6. View Slide

7. +
   Solution
   * or Puppet (there’s a bit for you lot later)
   *
   DevOps Consultancy
   

   View Slide

8. Why use with ?
   • Define your container configuration using Chef
   cookbooks rather than with Dockerfile/bash
   • Idempotently manage the running state of
   your docker container
   • Manage multiple services (i.e. have more than
   one process running)
   DevOps Consultancy
   

   View Slide

9. • Easier installation and configuration of
   complex applications using existing well
   tested Chef cookbooks
   • Move existing setup to containers
   • Handling final configuration when container
   boots
   Why use with ?
   DevOps Consultancy
   

   View Slide

10. • Consistency over mixed architectures
    • Use the same configuration management for
    development, staging and production
    • Use Docker containers for development of
    multi-tiered SOA applications but still deploy
    to bare metal or cloud instances in production
    Why use with ?
    DevOps Consultancy
    

    View Slide

11. tools for working with
    • Knife Container

    Knife plugin which gives it the ability to initialise and
    build Linux containers

    http://docs.getchef.com/plugin_knife_container.html
    • Chef Container

    Solves the Docker PID1 problem by running chef-client
    on boot and managing multiple processes with runit

    https://docs.getchef.com/containers.html
    DevOps Consultancy
    

    View Slide

12. tools for working with
    • Chef Metal

    Library that solves the problem of repeatably creating
    machines and infrastructures in Chef

    https://github.com/opscode/chef-metal
    • Docker cookbook

    Chef cookbook which installs Docker

    https://supermarket.getchef.com/cookbooks/docker
    DevOps Consultancy
    

    View Slide

13. Knife Container
    DevOps Consultancy
    

    View Slide

14. Knife Container
    $ chef gem install knife-container
    Install with the Chef DK
    $ gem install knife-container
    Install via RubyGems
    $ /opt/chef/embedded/bin/gem install knife-container
    RubyGems if Chef was installed via Omnibus Installer
    DevOps Consultancy
    

    View Slide

15. Knife Container
    docker init
    $ knife container docker init docker -r 'recipe[apache2]' -z -b -f
    chef/ubuntu_14.04
    Used to set up a Dockerfile context for the
    local workstation
    Example:
    Creates a Dockerfile like:
    FROM chef/ubuntu_14.04
    ADD chef /etc/chef
    RUN chef-init --bootstrap
    ENTRYPOINT ["chef-init"]
    CMD ["--onboot"]
    DevOps Consultancy
    

    View Slide

16. Knife Container
    docker build
    $ knife container docker build NAMESPACE/IMAGE_NAME
    Builds Docker image, resolves Chef
    dependencies and cleans up Chef artefacts
    Example:
    DevOps Consultancy
    

    View Slide

17. Chef Container
    DevOps Consultancy
    

    View Slide

18. Chef Container
    No installation required as the official Chef
    Docker images have chef-container
    preinstalled
    DevOps Consultancy
    

    View Slide

19. Chef Container
    Runs chef-client inside containers
    Image is licensed under a Creative Commons Attribution 3.0 Unported License. From https://docs.getchef.com/containers.html
    DevOps Consultancy
    

    View Slide

20. Chef Container
    • Your application probably needs some
    environment dependent configuration
    • Protects your container from configuration
    drift
    • Use same set up for production, staging,
    Vagrant/CI and CD
    Why run Chef in your container?
    DevOps Consultancy
    

    View Slide

21. Chef Metal
    DevOps Consultancy
    

    View Slide

22. Chef Metal
    $ chef gem install chef-metal
    Install with the Chef DK
    $ gem install chef-metal
    Install via RubyGems
    $ /opt/chef/embedded/bin/gem install chef-metal
    RubyGems if Chef was installed via Omnibus Installer
    chef_gem 'chef-metal' do
    action :install
    end
    Or in a Chef recipe
    DevOps Consultancy
    

    View Slide

23. Chef Metal
    machine 'db' do
    recipe 'mysql'
    end
    !
    machine 'web1' do
    recipe 'apache'
    end
    Use Chef Metal to create clusters
    DevOps Consultancy
    

    View Slide

24. Chef Metal
    • Adds machine resource
    • Modular so you can have different drivers
    • Just run chef-client on the machine looking
    after the cluster rather than on each machine
    • That’s good if you don’t want chef-client
    running all the time in each your container
    DevOps Consultancy
    

    View Slide

25. Chef Metal
    machine 'db' do
    recipe 'mysql'
    end
    !
    machine 'web1' do
    recipe 'apache'
    end
    !
    machine 'web2' do
    recipe 'apache'
    end
    Add a new “machine”
    DevOps Consultancy
    

    View Slide

26. Chef Metal
    machine 'db' do
    recipe 'mysql'
    end
    !
    1.upto(50) do |i|
    machine "web#{i}" do
    recipe 'apache'
    end
    end
    Upscaling
    DevOps Consultancy
    

    View Slide

27. Chef Metal
    • chef-metal-fog

    EC2, Digital Ocean, OpenStack etc
    • chef-metal-vagrant
    • chef-metal-ssh
    • chef-metal-lxc
    • chef-metal-docker
    Drivers
    DevOps Consultancy
    

    View Slide

28. Chef Metal with Docker
    DevOps Consultancy
    

    View Slide

29. Chef Metal with Docker
    $ chef gem install chef-metal-docker
    Install with the Chef DK
    $ gem install chef-metal-docker
    Install via RubyGems
    $ /opt/chef/embedded/bin/gem install chef-metal-docker
    RubyGems if Chef was installed via Omnibus Installer
    chef_gem ‘chef-metal-docker' do
    action :install
    end
    Or in a Chef recipe
    DevOps Consultancy
    

    View Slide

30. Chef Metal with Docker
    require 'chef_metal_docker'
    !
    machine 'wario' do
    recipe 'openssh::default'
    !
    machine_options :docker_options => {
    :base_image => {
    :name => 'ubuntu',
    :repository => 'ubuntu',
    :tag => '14.04'
    },
    :command => '/usr/sbin/sshd -p 8022 -D',
    :ports => 8022
    }
    end
    Create container
    DevOps Consultancy
    

    View Slide

31. Chef Metal with Docker
    require 'chef_metal_docker'
    !
    machine_image 'web_server' do
    recipe 'apache'
    !
    machine_options :docker_options => {
    :base_image => {
    :name => 'ubuntu',
    :repository => 'ubuntu',
    :tag => '14.04'
    }
    }
    end
    !
    machine 'web00' do
    from_image 'web_server'
    !
    machine_options :docker_options => {
    :command => '/usr/sbin/httpd'
    }
    end
    Creates image
    Creates container from the image
    DevOps Consultancy
    

    View Slide

32. #!
    # Cookbook Name:: hf-chef-metal-docker!
    # Recipe:: setup!
    #!
    !
    node.default['build-essential']['compile_time'] = true!
    !
    include_recipe 'build-essential'!
    include_recipe 'aufs'!
    !
    node.set['docker']['package']['repo_url'] = 'https://get.docker.io/ubuntu'!
    node.set['docker']['storage_driver'] = 'aufs'!
    !
    include_recipe 'docker'!
    !
    chef_gem 'chef-metal' do!
    action :install!
    end!
    !
    package 'lxc-dev' do!
    action :install!
    notifies :install, 'chef_gem[chef-metal-docker]', :immediately!
    end!
    !
    chef_gem 'chef-metal-docker' do!
    action :nothing!
    end!
    Chef Metal with Docker
    Example recipe that sets up
    Docker and chef-metal-docker
    on your Docker host machine
    (Tested on Ubuntu 14.04 only)
    DevOps Consultancy
    

    View Slide

33. Docker cookbook
    DevOps Consultancy
    

    View Slide

34. Docker cookbook
    • Get from:

    https://supermarket.getchef.com/cookbooks/docker
    • Installs Docker for you
    • Also contains LWRPs to manage Docker images
    and containers
    DevOps Consultancy
    

    View Slide

35. Docker cookbook
    # Pull latest image
    docker_image 'samalba/docker-registry'
    !
    # Run container exposing ports
    docker_container 'samalba/docker-registry' do
    detach true
    port '5000:5000'
    env 'SETTINGS_FLAVOR=local'
    volume '/mnt/docker:/docker-storage'
    end
    !
    # Login to private registry
    docker_registry 'https://docker-registry.example.com/' do
    username 'shipper'
    password 'iloveshipping'
    end
    !
    # Pull tagged image
    docker_image 'apps/crowsnest' do
    tag 'not-latest'
    end
    Example LWRPs
    DevOps Consultancy
    

    View Slide

36. Docker cookbook
    # Run container
    docker_container 'crowsnest'
    !
    # Save current timestamp
    timestamp = Time.new.strftime('%Y%m%d%H%M')
    !
    # Commit container changes
    docker_container 'crowsnest' do
    repository 'apps'
    tag timestamp
    action :commit
    end
    !
    # Push image
    docker_image 'crowsnest' do
    repository 'apps'
    tag timestamp
    action :push
    end
    Example LWRPs
    DevOps Consultancy
    

    View Slide

37. Docker cookbook
    # Conditionally rebuild image if changes upstream:!
    !
    git "#{Chef::Config[:file_cache_path]}/docker-testcontainerd" do!
    repository 'git@github.com:bflad/docker-testcontainerd.git'!
    notifies :build, 'docker_image[bflad/testcontainerd]', :immediately!
    end!
    !
    docker_image 'bflad/testcontainerd' do!
    action :pull_if_missing!
    end!
    Example LWRPs
    DevOps Consultancy
    

    View Slide

38. Puppet
    DevOps Consultancy
    

    View Slide

39. Puppet
    • Gareth Rushgrove who runs DevOps Weekly
    used to work for GDS and now works for
    Puppet

    http://www.devopsweekly.com/
    • Puppet module with similar functionality
    • https://forge.puppetlabs.com/garethr/docker
    DevOps Consultancy
    

    View Slide

40. Puppet
    include 'docker'!
    !
    docker::image { 'ubuntu':!
    image_tag => 'precise'!
    }!
    !
    docker::run { 'helloworld':!
    image => 'ubuntu:precise',!
    command => '/bin/sh -c "while true; do echo!
    hello world; sleep 1; done"',!
    }
    DevOps Consultancy
    Simple example
    

    View Slide

41. Puppet
    docker::run { 'helloworld':!
    image => 'base',!
    command => '/bin/sh -c "while true; do echo hello world; !
    ! ! ! ! ! ! ! sleep 1; done"',!
    ports => ['4444', '4555'],!
    expose => ['4666', '4777'],!
    links => ['mysql:db'],!
    use_name => true,!
    volumes => ['/var/lib/couchdb', '/var/log'],!
    volumes_from => '6446ea52fbc9',!
    memory_limit => 10m, # (format: , where unit = b, k, m or g)!
    username => 'example',!
    hostname => 'example.com',!
    env => ['FOO=BAR', 'FOO2=BAR2'],!
    dns => ['8.8.8.8', '8.8.4.4'],!
    restart_service => true,!
    privileged => false,!
    }
    DevOps Consultancy
    More complicated example
    

    View Slide

42. Better with configuration management!
    DevOps Consultancy
    

    View Slide

43. Questions?
    DevOps Consultancy
    https://github.com/salgo/chef-and-docker-talk
    Code examples
    

    View Slide