Tweet
Tweet

Slide 1

Slide 1 text

Chef and LXC building & deploying custom containers

Slide 2

Slide 2 text

About Me ? ● System Administrator at PagerDuty. Bioinformatics post graduate, data mining, application development, system automation ● Exploring life like properties in computer systems ● FOSS person (recent contributions in ruby-lxc, lxc-chef, chef-metal-lxc, chef, chefspec, serverspec-lxc, graphios, community cookbooks etc)

Slide 3

Slide 3 text

Agenda ● Part A: Introduction to containers & LXC ● Part B: Using LXC with ruby & chef ● Part C: Deployment scenarios (chef + LXC)

Slide 4

Slide 4 text

Part A. Introduction to containers & LXC

Slide 5

Slide 5 text

Container ● Operating system virtualization ● Instead of a hypervisor(like kvm, vmware) or paravisor (like Xen) your OS itself gives a VM like interface.

Slide 6

Slide 6 text

Containers on other platforms ● Operating system virtualization (container) ● Zones on solaris(2005) ● OpenVZ (patched linux kernel, 2005) ● WPARs on AIX, SRP on HP-AUX(2007)

Slide 7

Slide 7 text

LXC ● LXC – user space tools to build/run containers on linux kernel (> 3.11) ● Uses 3 key kernel features: ● Namespaces ● cgroup ● capabilities

Slide 8

Slide 8 text

LXC - CLI ranjib@automator:~ $ lxc-ls --fancy NAME STATE IPV4 IPV6 AUTOSTART ----------------------------------------- chef-serf STOPPED - - NO father STOPPED - - NO sensu STOPPED - - NO serf STOPPED - - NO trusty STOPPED - - NO ranjib@automator:~ $

Slide 9

Slide 9 text

LXC – Create using CLI ranjib@automator:~ $ lxc-create -n meetup -t download –- \ -d ubuntu -r trusty -a amd64 Using image from local cache Unpacking the rootfs --- You just created an Ubuntu container (release=trusty, arch=amd64, variant=default) The default username/password is: ubuntu / ubuntu To gain root privileges, please use sudo. ranjib@automator:~ $

Slide 10

Slide 10 text

LXC – Start/Stop ranjib@automator:~ $ lxc-start -n meetup -d ranjib@automator:~ $ lxc-ls --fancy NAME STATE IPV4 IPV6 AUTOSTART ----------------------------------------------- chef-serf STOPPED - - NO father STOPPED - - NO meetup RUNNING 10.0.3.171 - NO sensu STOPPED - - NO serf STOPPED - - NO trusty STOPPED - - NO ranjib@automator:~ $

Slide 11

Slide 11 text

LXC - Features ● Unprivileged containers ● Decoupled rootfs vs container creation ● Rootfs customization via templates ● Cgroup customization via config file and api ● Bindings ● Hooks ● 1.1 aiming for CRIU support

Slide 12

Slide 12 text

Part B. Using LXC with ruby & chef

Slide 13

Slide 13 text

Ruby and LXC integration ● Ruby-lxc binding is 1.0 ● Native and covers entire liblxc API ● Hosted in the same github repo as LXC ● CLI wrapper based ruby bindings are also available (will limit `attach` usage)

Slide 14

Slide 14 text

Ruby-LXC Examples c = LXC::Container.new('foo') c.create('ubuntu') # create with ubuntu template c.start Create and start a container

Slide 15

Slide 15 text

Ruby-LXC Examples Run arbitrary command inside a container c.attach do # going inside container LXC.run_command('ifconfig eth0') end

Slide 16

Slide 16 text

Ruby-LXC Examples Destroy a container c.stop c.destroy

Slide 17

Slide 17 text

Ruby-LXC Examples Change memory limits of a container c = LXC::Container.new('foo') c.cgroup_item('memory.limit_in_bytes') c.set_cgroup_item('memory.limit_in_bytes','10000000') c.save_config

Slide 18

Slide 18 text

Ruby-LXC Examples Destroy a container c.stop c.destroy

Slide 19

Slide 19 text

Chef-LXC Integration LXC cookbook - Uses LXC CLI - Containers are bootstrapped exactly as chef nodes include_recipe 'lxc' lxc_container 'my_container' do action :create run_list ['role[base]'] chef_enabled true end

Slide 20

Slide 20 text

Chef-LXC Integration Chef-metal project - Provides uniform interface for machine as a chef resource - Has LXC driver (alongside vagrant, fog etc) - Treats containers exactly as chef node require 'chef_metal_lxc/lxc_provisioner' with_provisioner ChefMetalLXC::LXCProvisioner.new machine 'mario' do recipe 'postgresql' recipe 'mydb' tag 'mydb_master' end

Slide 21

Slide 21 text

Chef-LXC Integration Chef-lxc gem - Uses liblxc attach method to execute chef resources inside container - Does not install chef inside the container. lxc "web" do template "ubuntu" recipe do package "apache2" service "apache2" do action [:start, :enable] end end action [:create, :start] end

Slide 22

Slide 22 text

Chef and LXC integration ● Will be under heavy development since LXC and ruby-LXC is stable now. - unprivileged containers required usernamespace (kernel > 3.11). - ubuntu 14.04 is recommended as host OS.

Slide 23

Slide 23 text

Chef and LXC integration ● A common store for pre-baked containers worth exploring ● similar to images.linxucontainers.org .. may be one day... lxc-create -n foo -t chef –- \ -d ubuntu -r trusty -a amd64 –-chef-role 'db'

Slide 24

Slide 24 text

Part C. Deployment scenarios (chef & LXC)

Slide 25

Slide 25 text

Deployment scenarios - 1 ● One container per host (M → 1Ct) ● Independent host/container update strategy ● Separation of concerns/dependencies ● Common patterns -> reusable components

Slide 26

Slide 26 text

1 host–1 container (private network) Host Container chef logstash sensu IPtables network Data Volume

Slide 27

Slide 27 text

1 host–1 container (public network) Host Container chef logstash sensu eth1/ENI network Data Volume

Slide 28

Slide 28 text

Why? ● Keep db binary dependencies isolated from host binaries ● Dont let an old software dictate your host OS choice. ● We can use cgroups to control cpu footprint of new service during changes like. ● Db upgrade ● App Releases

Slide 29

Slide 29 text

Deployment scenarios -2 ● N similar containers per host (1M → NCt) ● proxy on host ● Complex load balancing options ● Frozen containers

Slide 30

Slide 30 text

1 host–N similar containers Host Container chef logstash sensu HAproxy network Container Container Container Container Container

Slide 31

Slide 31 text

Why? ● Using chef on host makes dynamic haproxy configs trivial ● Employ blue/green, canary, A/B deployment techniques ● Portable, fast deployments. Free cgroup accountring as metrics.

Slide 32

Slide 32 text

Deployment scenarios -3 ● Different types containers per host (1M → N different Cts) ● Generic case of the earlier pattern ● Opportunity to use nested containers

Slide 33

Slide 33 text

1 host–N containers of different type Host DB container chef logstash sensu HAproxy network DB container DB container App container App container App container

Slide 34

Slide 34 text

Nesting for grouping similar containers Host DB container chef logstash sensu network DB container App Cluster Container HAproxy App container App container App container DB Cluster Container IPtables

Slide 35

Slide 35 text

Why? ● Test bed for environment wide automation ● Perfect for dev environments ● Explore network (iptables for partition, tc for delay) , disk io, cpu throttling effects on clusters etc.

Slide 36

Slide 36 text

Deployment scenarios – many more ● Cross platforms (i386, x86_64/amd64, arm) ● Build android apps as from jenkins easily ● Any arbitrary code evaluation (CI – SaaS) ● Rapid deployments – Frozen containers

Slide 37

Slide 37 text

Resources ● Wikipedia article on OS virtualization ● LXC main website ● Stephane Graber's blog series on LXC ● Rami Rosen's presentation on LXC ● Ruby-LXC ● chef-lxc, chef-metal, lxc-cookbook

Slide 38

Slide 38 text

Do It! @RanjibDey [email protected]