Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Chef and Docker

Andy Gale
October 01, 2014

Chef and Docker

Talk explaining the various tools available for using Chef with Docker. Given at Bristol DevOps 1st October 2014.

Andy Gale

October 01, 2014
Tweet

More Decks by Andy Gale

Other Decks in Technology

Transcript

  1. Chef and Docker
    Andy Gale
    DevOps Consultancy

    View Slide

  2. About me
    Andy Gale
    Web Consultant
    Hello Future
    http://hellofutu.re
    !
    @andygale
    @hellofutur3
    DevOps Consultancy

    View Slide

  3. • Heard about Docker on the The Ship Show
    • Based on Linux Containers!
    • Linux Containers can be difficult to set up
    • Extra Docker tooling looked fantastic
    That looks really cool
    DevOps Consultancy

    View Slide

  4. • Who needs Vagrant development
    environments, Chef and Puppet when we can
    just deploy Docker images?
    • I went through the getting started guide;
    wow this is all very clever
    • Got as far as the Dockerfile
    That looks really cool
    DevOps Consultancy

    View Slide

  5. # Nginx
    #
    # VERSION 0.0.1
    !
    FROM ubuntu
    MAINTAINER Victor Vieux
    !
    RUN apt-get update && apt-get install -y inotify-tools nginx apache2
    openssh-server
    Dockerfile
    Hang on, that’s basically a bash script!

    View Slide

  6. View Slide

  7. +
    Solution
    * or Puppet (there’s a bit for you lot later)
    *
    DevOps Consultancy

    View Slide

  8. Why use with ?
    • Define your container configuration using Chef
    cookbooks rather than with Dockerfile/bash
    • Idempotently manage the running state of
    your docker container
    • Manage multiple services (i.e. have more than
    one process running)
    DevOps Consultancy

    View Slide

  9. • Easier installation and configuration of
    complex applications using existing well
    tested Chef cookbooks
    • Move existing setup to containers
    • Handling final configuration when container
    boots
    Why use with ?
    DevOps Consultancy

    View Slide

  10. • Consistency over mixed architectures
    • Use the same configuration management for
    development, staging and production
    • Use Docker containers for development of
    multi-tiered SOA applications but still deploy
    to bare metal or cloud instances in production
    Why use with ?
    DevOps Consultancy

    View Slide

  11. tools for working with
    • Knife Container

    Knife plugin which gives it the ability to initialise and
    build Linux containers

    http://docs.getchef.com/plugin_knife_container.html
    • Chef Container

    Solves the Docker PID1 problem by running chef-client
    on boot and managing multiple processes with runit

    https://docs.getchef.com/containers.html
    DevOps Consultancy

    View Slide

  12. tools for working with
    • Chef Metal

    Library that solves the problem of repeatably creating
    machines and infrastructures in Chef

    https://github.com/opscode/chef-metal
    • Docker cookbook

    Chef cookbook which installs Docker

    https://supermarket.getchef.com/cookbooks/docker
    DevOps Consultancy

    View Slide

  13. Knife Container
    DevOps Consultancy

    View Slide

  14. Knife Container
    $ chef gem install knife-container
    Install with the Chef DK
    $ gem install knife-container
    Install via RubyGems
    $ /opt/chef/embedded/bin/gem install knife-container
    RubyGems if Chef was installed via Omnibus Installer
    DevOps Consultancy

    View Slide

  15. Knife Container
    docker init
    $ knife container docker init docker -r 'recipe[apache2]' -z -b -f
    chef/ubuntu_14.04
    Used to set up a Dockerfile context for the
    local workstation
    Example:
    Creates a Dockerfile like:
    FROM chef/ubuntu_14.04
    ADD chef /etc/chef
    RUN chef-init --bootstrap
    ENTRYPOINT ["chef-init"]
    CMD ["--onboot"]
    DevOps Consultancy

    View Slide

  16. Knife Container
    docker build
    $ knife container docker build NAMESPACE/IMAGE_NAME
    Builds Docker image, resolves Chef
    dependencies and cleans up Chef artefacts
    Example:
    DevOps Consultancy

    View Slide

  17. Chef Container
    DevOps Consultancy

    View Slide

  18. Chef Container
    No installation required as the official Chef
    Docker images have chef-container
    preinstalled
    DevOps Consultancy

    View Slide

  19. Chef Container
    Runs chef-client inside containers
    Image is licensed under a Creative Commons Attribution 3.0 Unported License. From https://docs.getchef.com/containers.html
    DevOps Consultancy

    View Slide

  20. Chef Container
    • Your application probably needs some
    environment dependent configuration
    • Protects your container from configuration
    drift
    • Use same set up for production, staging,
    Vagrant/CI and CD
    Why run Chef in your container?
    DevOps Consultancy

    View Slide

  21. Chef Metal
    DevOps Consultancy

    View Slide

  22. Chef Metal
    $ chef gem install chef-metal
    Install with the Chef DK
    $ gem install chef-metal
    Install via RubyGems
    $ /opt/chef/embedded/bin/gem install chef-metal
    RubyGems if Chef was installed via Omnibus Installer
    chef_gem 'chef-metal' do
    action :install
    end
    Or in a Chef recipe
    DevOps Consultancy

    View Slide

  23. Chef Metal
    machine 'db' do
    recipe 'mysql'
    end
    !
    machine 'web1' do
    recipe 'apache'
    end
    Use Chef Metal to create clusters
    DevOps Consultancy

    View Slide

  24. Chef Metal
    • Adds machine resource
    • Modular so you can have different drivers
    • Just run chef-client on the machine looking
    after the cluster rather than on each machine
    • That’s good if you don’t want chef-client
    running all the time in each your container
    DevOps Consultancy

    View Slide

  25. Chef Metal
    machine 'db' do
    recipe 'mysql'
    end
    !
    machine 'web1' do
    recipe 'apache'
    end
    !
    machine 'web2' do
    recipe 'apache'
    end
    Add a new “machine”
    DevOps Consultancy

    View Slide

  26. Chef Metal
    machine 'db' do
    recipe 'mysql'
    end
    !
    1.upto(50) do |i|
    machine "web#{i}" do
    recipe 'apache'
    end
    end
    Upscaling
    DevOps Consultancy

    View Slide

  27. Chef Metal
    • chef-metal-fog

    EC2, Digital Ocean, OpenStack etc
    • chef-metal-vagrant
    • chef-metal-ssh
    • chef-metal-lxc
    • chef-metal-docker
    Drivers
    DevOps Consultancy

    View Slide

  28. Chef Metal with Docker
    DevOps Consultancy

    View Slide

  29. Chef Metal with Docker
    $ chef gem install chef-metal-docker
    Install with the Chef DK
    $ gem install chef-metal-docker
    Install via RubyGems
    $ /opt/chef/embedded/bin/gem install chef-metal-docker
    RubyGems if Chef was installed via Omnibus Installer
    chef_gem ‘chef-metal-docker' do
    action :install
    end
    Or in a Chef recipe
    DevOps Consultancy

    View Slide

  30. Chef Metal with Docker
    require 'chef_metal_docker'
    !
    machine 'wario' do
    recipe 'openssh::default'
    !
    machine_options :docker_options => {
    :base_image => {
    :name => 'ubuntu',
    :repository => 'ubuntu',
    :tag => '14.04'
    },
    :command => '/usr/sbin/sshd -p 8022 -D',
    :ports => 8022
    }
    end
    Create container
    DevOps Consultancy

    View Slide

  31. Chef Metal with Docker
    require 'chef_metal_docker'
    !
    machine_image 'web_server' do
    recipe 'apache'
    !
    machine_options :docker_options => {
    :base_image => {
    :name => 'ubuntu',
    :repository => 'ubuntu',
    :tag => '14.04'
    }
    }
    end
    !
    machine 'web00' do
    from_image 'web_server'
    !
    machine_options :docker_options => {
    :command => '/usr/sbin/httpd'
    }
    end
    Creates image
    Creates container from the image
    DevOps Consultancy

    View Slide

  32. #!
    # Cookbook Name:: hf-chef-metal-docker!
    # Recipe:: setup!
    #!
    !
    node.default['build-essential']['compile_time'] = true!
    !
    include_recipe 'build-essential'!
    include_recipe 'aufs'!
    !
    node.set['docker']['package']['repo_url'] = 'https://get.docker.io/ubuntu'!
    node.set['docker']['storage_driver'] = 'aufs'!
    !
    include_recipe 'docker'!
    !
    chef_gem 'chef-metal' do!
    action :install!
    end!
    !
    package 'lxc-dev' do!
    action :install!
    notifies :install, 'chef_gem[chef-metal-docker]', :immediately!
    end!
    !
    chef_gem 'chef-metal-docker' do!
    action :nothing!
    end!
    Chef Metal with Docker
    Example recipe that sets up
    Docker and chef-metal-docker
    on your Docker host machine
    (Tested on Ubuntu 14.04 only)
    DevOps Consultancy

    View Slide

  33. Docker cookbook
    DevOps Consultancy

    View Slide

  34. Docker cookbook
    • Get from:

    https://supermarket.getchef.com/cookbooks/docker
    • Installs Docker for you
    • Also contains LWRPs to manage Docker images
    and containers
    DevOps Consultancy

    View Slide

  35. Docker cookbook
    # Pull latest image
    docker_image 'samalba/docker-registry'
    !
    # Run container exposing ports
    docker_container 'samalba/docker-registry' do
    detach true
    port '5000:5000'
    env 'SETTINGS_FLAVOR=local'
    volume '/mnt/docker:/docker-storage'
    end
    !
    # Login to private registry
    docker_registry 'https://docker-registry.example.com/' do
    username 'shipper'
    password 'iloveshipping'
    end
    !
    # Pull tagged image
    docker_image 'apps/crowsnest' do
    tag 'not-latest'
    end
    Example LWRPs
    DevOps Consultancy

    View Slide

  36. Docker cookbook
    # Run container
    docker_container 'crowsnest'
    !
    # Save current timestamp
    timestamp = Time.new.strftime('%Y%m%d%H%M')
    !
    # Commit container changes
    docker_container 'crowsnest' do
    repository 'apps'
    tag timestamp
    action :commit
    end
    !
    # Push image
    docker_image 'crowsnest' do
    repository 'apps'
    tag timestamp
    action :push
    end
    Example LWRPs
    DevOps Consultancy

    View Slide

  37. Docker cookbook
    # Conditionally rebuild image if changes upstream:!
    !
    git "#{Chef::Config[:file_cache_path]}/docker-testcontainerd" do!
    repository '[email protected]:bflad/docker-testcontainerd.git'!
    notifies :build, 'docker_image[bflad/testcontainerd]', :immediately!
    end!
    !
    docker_image 'bflad/testcontainerd' do!
    action :pull_if_missing!
    end!
    Example LWRPs
    DevOps Consultancy

    View Slide

  38. Puppet
    DevOps Consultancy

    View Slide

  39. Puppet
    • Gareth Rushgrove who runs DevOps Weekly
    used to work for GDS and now works for
    Puppet

    http://www.devopsweekly.com/
    • Puppet module with similar functionality
    • https://forge.puppetlabs.com/garethr/docker
    DevOps Consultancy

    View Slide

  40. Puppet
    include 'docker'!
    !
    docker::image { 'ubuntu':!
    image_tag => 'precise'!
    }!
    !
    docker::run { 'helloworld':!
    image => 'ubuntu:precise',!
    command => '/bin/sh -c "while true; do echo!
    hello world; sleep 1; done"',!
    }
    DevOps Consultancy
    Simple example

    View Slide

  41. Puppet
    docker::run { 'helloworld':!
    image => 'base',!
    command => '/bin/sh -c "while true; do echo hello world; !
    ! ! ! ! ! ! ! sleep 1; done"',!
    ports => ['4444', '4555'],!
    expose => ['4666', '4777'],!
    links => ['mysql:db'],!
    use_name => true,!
    volumes => ['/var/lib/couchdb', '/var/log'],!
    volumes_from => '6446ea52fbc9',!
    memory_limit => 10m, # (format: , where unit = b, k, m or g)!
    username => 'example',!
    hostname => 'example.com',!
    env => ['FOO=BAR', 'FOO2=BAR2'],!
    dns => ['8.8.8.8', '8.8.4.4'],!
    restart_service => true,!
    privileged => false,!
    }
    DevOps Consultancy
    More complicated example

    View Slide

  42. Better with configuration management!
    DevOps Consultancy

    View Slide

  43. Questions?
    DevOps Consultancy
    https://github.com/salgo/chef-and-docker-talk
    Code examples

    View Slide