Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Chef-LXC - Building and deploying custom containers

Chef-LXC - Building and deploying custom containers

Linux containers (LXC) along with Chef for building and deploying custom solutions, environments.

Ranjib Dey

June 26, 2014

More Decks by Ranjib Dey

Other Decks in Programming


  1. About Me ? • System Administrator at PagerDuty. Bioinformatics post

    graduate, data mining, application development, system automation • Exploring life like properties in computer systems • FOSS person (recent contributions in ruby-lxc, lxc-chef, chef-metal-lxc, chef, chefspec, serverspec-lxc, graphios, community cookbooks etc)
  2. Agenda • Part A: Introduction to containers & LXC •

    Part B: Using LXC with ruby & chef • Part C: Deployment scenarios (chef + LXC)
  3. Container • Operating system virtualization • Instead of a hypervisor(like

    kvm, vmware) or paravisor (like Xen) your OS itself gives a VM like interface.
  4. Containers on other platforms • Operating system virtualization (container) •

    Zones on solaris(2005) • OpenVZ (patched linux kernel, 2005) • WPARs on AIX, SRP on HP-AUX(2007)
  5. LXC • LXC – user space tools to build/run containers

    on linux kernel (> 3.11) • Uses 3 key kernel features: • Namespaces • cgroup • capabilities
  6. LXC - CLI ranjib@automator:~ $ lxc-ls --fancy NAME STATE IPV4

    IPV6 AUTOSTART ----------------------------------------- chef-serf STOPPED - - NO father STOPPED - - NO sensu STOPPED - - NO serf STOPPED - - NO trusty STOPPED - - NO ranjib@automator:~ $
  7. LXC – Create using CLI ranjib@automator:~ $ lxc-create -n meetup

    -t download –- \ -d ubuntu -r trusty -a amd64 Using image from local cache Unpacking the rootfs --- You just created an Ubuntu container (release=trusty, arch=amd64, variant=default) The default username/password is: ubuntu / ubuntu To gain root privileges, please use sudo. ranjib@automator:~ $
  8. LXC – Start/Stop ranjib@automator:~ $ lxc-start -n meetup -d ranjib@automator:~

    $ lxc-ls --fancy NAME STATE IPV4 IPV6 AUTOSTART ----------------------------------------------- chef-serf STOPPED - - NO father STOPPED - - NO meetup RUNNING - NO sensu STOPPED - - NO serf STOPPED - - NO trusty STOPPED - - NO ranjib@automator:~ $
  9. LXC - Features • Unprivileged containers • Decoupled rootfs vs

    container creation • Rootfs customization via templates • Cgroup customization via config file and api • Bindings • Hooks • 1.1 aiming for CRIU support
  10. Ruby and LXC integration • Ruby-lxc binding is 1.0 •

    Native and covers entire liblxc API • Hosted in the same github repo as LXC • CLI wrapper based ruby bindings are also available (will limit `attach` usage)
  11. Ruby-LXC Examples Run arbitrary command inside a container c.attach do

    # going inside container LXC.run_command('ifconfig eth0') end
  12. Ruby-LXC Examples Change memory limits of a container c =

    LXC::Container.new('foo') c.cgroup_item('memory.limit_in_bytes') c.set_cgroup_item('memory.limit_in_bytes','10000000') c.save_config
  13. Chef-LXC Integration LXC cookbook - Uses LXC CLI - Containers

    are bootstrapped exactly as chef nodes include_recipe 'lxc' lxc_container 'my_container' do action :create run_list ['role[base]'] chef_enabled true end
  14. Chef-LXC Integration Chef-metal project - Provides uniform interface for machine

    as a chef resource - Has LXC driver (alongside vagrant, fog etc) - Treats containers exactly as chef node require 'chef_metal_lxc/lxc_provisioner' with_provisioner ChefMetalLXC::LXCProvisioner.new machine 'mario' do recipe 'postgresql' recipe 'mydb' tag 'mydb_master' end
  15. Chef-LXC Integration Chef-lxc gem - Uses liblxc attach method to

    execute chef resources inside container - Does not install chef inside the container. lxc "web" do template "ubuntu" recipe do package "apache2" service "apache2" do action [:start, :enable] end end action [:create, :start] end
  16. Chef and LXC integration • Will be under heavy development

    since LXC and ruby-LXC is stable now. - unprivileged containers required usernamespace (kernel > 3.11). - ubuntu 14.04 is recommended as host OS.
  17. Chef and LXC integration • A common store for pre-baked

    containers worth exploring • similar to images.linxucontainers.org .. may be one day... lxc-create -n foo -t chef –- \ -d ubuntu -r trusty -a amd64 –-chef-role 'db'
  18. Deployment scenarios - 1 • One container per host (M

    → 1Ct) • Independent host/container update strategy • Separation of concerns/dependencies • Common patterns -> reusable components
  19. Why? • Keep db binary dependencies isolated from host binaries

    • Dont let an old software dictate your host OS choice. • We can use cgroups to control cpu footprint of new service during changes like. • Db upgrade • App Releases
  20. Deployment scenarios -2 • N similar containers per host (1M

    → NCt) • proxy on host • Complex load balancing options • Frozen containers
  21. 1 host–N similar containers Host Container chef logstash sensu HAproxy

    network Container Container Container Container Container
  22. Why? • Using chef on host makes dynamic haproxy configs

    trivial • Employ blue/green, canary, A/B deployment techniques • Portable, fast deployments. Free cgroup accountring as metrics.
  23. Deployment scenarios -3 • Different types containers per host (1M

    → N different Cts) • Generic case of the earlier pattern • Opportunity to use nested containers
  24. 1 host–N containers of different type Host DB container chef

    logstash sensu HAproxy network DB container DB container App container App container App container
  25. Nesting for grouping similar containers Host DB container chef logstash

    sensu network DB container App Cluster Container HAproxy App container App container App container DB Cluster Container IPtables
  26. Why? • Test bed for environment wide automation • Perfect

    for dev environments • Explore network (iptables for partition, tc for delay) , disk io, cpu throttling effects on clusters etc.
  27. Deployment scenarios – many more • Cross platforms (i386, x86_64/amd64,

    arm) • Build android apps as from jenkins easily • Any arbitrary code evaluation (CI – SaaS) • Rapid deployments – Frozen containers
  28. Resources • Wikipedia article on OS virtualization • LXC main

    website • Stephane Graber's blog series on LXC • Rami Rosen's presentation on LXC • Ruby-LXC • chef-lxc, chef-metal, lxc-cookbook