Chef-LXC - Building and deploying custom containers

Linux containers (LXC) along with Chef for building and deploying custom solutions, environments.

Transcript

1. Chef and LXC
   building & deploying custom containers
   

   View Slide

2. About Me ?
   ●
   System Administrator at PagerDuty.
   Bioinformatics post graduate, data mining, application
   development, system automation
   ●
   Exploring life like properties in computer
   systems
   ●
   FOSS person (recent contributions in ruby-lxc, lxc-chef,
   chef-metal-lxc, chef, chefspec, serverspec-lxc, graphios,
   community cookbooks etc)
   

   View Slide

3. Agenda
   ●
   Part A: Introduction to containers & LXC
   ●
   Part B: Using LXC with ruby & chef
   ●
   Part C: Deployment scenarios (chef + LXC)
   

   View Slide

4. Part A. Introduction to containers & LXC
   

   View Slide

5. Container
   ●
   Operating system virtualization
   ●
   Instead of a hypervisor(like kvm, vmware) or
   paravisor (like Xen) your OS itself gives a VM
   like interface.
   

   View Slide

6. Containers on other platforms
   ●
   Operating system virtualization (container)
   ●
   Zones on solaris(2005)
   ●
   OpenVZ (patched linux kernel, 2005)
   ●
   WPARs on AIX, SRP on HP-AUX(2007)
   

   View Slide

7. LXC
   ●
   LXC – user space tools to build/run containers
   on linux kernel (> 3.11)
   ●
   Uses 3 key kernel features:
   ●
   Namespaces
   ●
   cgroup
   ●
   capabilities
   

   View Slide

8. LXC - CLI
   ranjib@automator:~ $ lxc-ls --fancy
   NAME STATE IPV4 IPV6 AUTOSTART
   -----------------------------------------
   chef-serf STOPPED - - NO
   father STOPPED - - NO
   sensu STOPPED - - NO
   serf STOPPED - - NO
   trusty STOPPED - - NO
   ranjib@automator:~ $
   

   View Slide

9. LXC – Create using CLI
   ranjib@automator:~ $ lxc-create -n meetup -t download –- \
   -d ubuntu -r trusty -a amd64
   Using image from local cache
   Unpacking the rootfs
   ---
   You just created an Ubuntu container (release=trusty,
   arch=amd64, variant=default)
   The default username/password is: ubuntu / ubuntu
   To gain root privileges, please use sudo.
   ranjib@automator:~ $
   

   View Slide

10. LXC – Start/Stop
    ranjib@automator:~ $ lxc-start -n meetup -d
    ranjib@automator:~ $ lxc-ls --fancy
    NAME STATE IPV4 IPV6 AUTOSTART
    -----------------------------------------------
    chef-serf STOPPED - - NO
    father STOPPED - - NO
    meetup RUNNING 10.0.3.171 - NO
    sensu STOPPED - - NO
    serf STOPPED - - NO
    trusty STOPPED - - NO
    ranjib@automator:~ $
    

    View Slide

11. LXC - Features
    ●
    Unprivileged containers
    ●
    Decoupled rootfs vs container creation
    ●
    Rootfs customization via templates
    ●
    Cgroup customization via config file and api
    ●
    Bindings
    ●
    Hooks
    ●
    1.1 aiming for CRIU support
    

    View Slide

12. Part B. Using LXC with ruby & chef
    

    View Slide

13. Ruby and LXC integration
    ●
    Ruby-lxc binding is 1.0
    ●
    Native and covers entire liblxc API
    ●
    Hosted in the same github repo as LXC
    ●
    CLI wrapper based ruby bindings are also
    available (will limit `attach` usage)
    

    View Slide

14. Ruby-LXC Examples
    c = LXC::Container.new('foo')
    c.create('ubuntu') # create with ubuntu template
    c.start
    Create and start a container
    

    View Slide

15. Ruby-LXC Examples
    Run arbitrary command inside a container
    c.attach do # going inside container
    LXC.run_command('ifconfig eth0')
    end
    

    View Slide

16. Ruby-LXC Examples
    Destroy a container
    c.stop
    c.destroy
    

    View Slide

17. Ruby-LXC Examples
    Change memory limits of a container
    c = LXC::Container.new('foo')
    c.cgroup_item('memory.limit_in_bytes')
    c.set_cgroup_item('memory.limit_in_bytes','10000000')
    c.save_config
    

    View Slide

18. Ruby-LXC Examples
    Destroy a container
    c.stop
    c.destroy
    

    View Slide

19. Chef-LXC Integration
    LXC cookbook
    - Uses LXC CLI
    - Containers are bootstrapped exactly as chef nodes
    include_recipe 'lxc'
    lxc_container 'my_container' do
    action :create
    run_list ['role[base]']
    chef_enabled true
    end
    

    View Slide

20. Chef-LXC Integration
    Chef-metal project
    - Provides uniform interface for machine as a chef resource
    - Has LXC driver (alongside vagrant, fog etc)
    - Treats containers exactly as chef node
    require 'chef_metal_lxc/lxc_provisioner'
    with_provisioner ChefMetalLXC::LXCProvisioner.new
    machine 'mario' do
    recipe 'postgresql'
    recipe 'mydb'
    tag 'mydb_master'
    end
    

    View Slide

21. Chef-LXC Integration
    Chef-lxc gem
    - Uses liblxc attach method to execute chef resources inside
    container
    - Does not install chef inside the container.
    lxc "web" do
    template "ubuntu"
    recipe do
    package "apache2"
    service "apache2" do
    action [:start, :enable]
    end
    end
    action [:create, :start]
    end
    

    View Slide

22. Chef and LXC integration
    ●
    Will be under heavy development since LXC and
    ruby-LXC is stable now.
    - unprivileged containers required
    usernamespace (kernel > 3.11).
    - ubuntu 14.04 is recommended as host OS.
    

    View Slide

23. Chef and LXC integration
    ●
    A common store for pre-baked containers worth
    exploring
    ●
    similar to images.linxucontainers.org
    .. may be one day...
    lxc-create -n foo -t chef –- \
    -d ubuntu -r trusty -a amd64 –-chef-role 'db'
    

    View Slide

24. Part C. Deployment scenarios (chef & LXC)
    

    View Slide

25. Deployment scenarios - 1
    ●
    One container per host (M → 1Ct)
    ●
    Independent host/container update strategy
    ●
    Separation of concerns/dependencies
    ●
    Common patterns -> reusable components
    

    View Slide

26. 1 host–1 container (private network)
    Host
    Container
    chef
    logstash
    sensu
    IPtables
    network
    Data Volume
    

    View Slide

27. 1 host–1 container (public network)
    Host
    Container
    chef
    logstash
    sensu
    eth1/ENI
    network
    Data Volume
    

    View Slide

28. Why?
    ●
    Keep db binary dependencies isolated from host
    binaries
    ●
    Dont let an old software dictate your host OS
    choice.
    ●
    We can use cgroups to control cpu footprint of
    new service during changes like.
    ●
    Db upgrade
    ●
    App Releases
    

    View Slide

29. Deployment scenarios -2
    ●
    N similar containers per host (1M → NCt)
    ●
    proxy on host
    ●
    Complex load balancing options
    ●
    Frozen containers
    

    View Slide

30. 1 host–N similar containers
    Host
    Container
    chef
    logstash
    sensu
    HAproxy
    network
    Container
    Container
    Container
    Container
    Container
    

    View Slide

31. Why?
    ●
    Using chef on host makes dynamic haproxy
    configs trivial
    ●
    Employ blue/green, canary, A/B deployment
    techniques
    ●
    Portable, fast deployments. Free cgroup
    accountring as metrics.
    

    View Slide

32. Deployment scenarios -3
    ●
    Different types containers per host (1M → N
    different Cts)
    ●
    Generic case of the earlier pattern
    ●
    Opportunity to use nested containers
    

    View Slide

33. 1 host–N containers of different type
    Host
    DB container
    chef
    logstash
    sensu
    HAproxy
    network
    DB container
    DB container
    App container
    App container
    App container
    

    View Slide

34. Nesting for grouping similar
    containers
    Host
    DB container
    chef
    logstash
    sensu
    network
    DB container
    App Cluster Container
    HAproxy
    App container
    App container
    App container
    DB Cluster Container
    IPtables
    

    View Slide

35. Why?
    ●
    Test bed for environment wide automation
    ●
    Perfect for dev environments
    ●
    Explore network (iptables for partition, tc for
    delay) , disk io, cpu throttling effects on clusters
    etc.
    

    View Slide

36. Deployment scenarios – many more
    ●
    Cross platforms (i386, x86_64/amd64, arm)
    ●
    Build android apps as from jenkins easily
    ●
    Any arbitrary code evaluation (CI – SaaS)
    ●
    Rapid deployments – Frozen containers
    

    View Slide

37. Resources
    ●
    Wikipedia article on OS virtualization
    ●
    LXC main website
    ●
    Stephane Graber's blog series on LXC
    ●
    Rami Rosen's presentation on LXC
    ●
    Ruby-LXC
    ●
    chef-lxc, chef-metal, lxc-cookbook
    

    View Slide

38. Do It!
    @RanjibDey ranjib@pagerduty.com
    

    View Slide