Stack a Platform With Linux Containers

We're used to expressiveness of our favorite programming language or framework. However new tools that let us program and control infrastructure are going on higher levels as well. In this presentation on RuPy 2013 (http://13.rupy.eu) I talked about virtualization, provisioning and how you can make your own software platform with LXC, Chef and Docker.

Transcript

1. What would you do
2. What would you do if you could make your own Heroku?
3. Stack a platform with Linux containers feat. virtualization provisioning
4. Hello! I’m Marko Anastasov and I work on Semaphore at Rendered Text. @markoa semaphoreapp.com renderedtext.com RuPy, Budapest / 2013
5. hosted continuous integration + deployment
6. usage / scale
7. POV: what is the simplest thing that can work?
8. What’s a container?
9. Full / paravirtualization
10. container “real “ virtualization
11. LXC = Linux containers http://linuxcontainers.org
12. LXC: getting started sudo apt-get install -y lxc
13. LXC: CLI lxc-attach lxc-clone lxc-freeze lxc-monitor lxc-setcap lxc-stop lxc-backup lxc-console lxc-info lxc-netstat lxc-setuid lxc-unfreeze lxc-cgroup lxc-create lxc-kill lxc-ps lxc-shutdown lxc-unshare lxc-checkconfig lxc-destroy lxc-list lxc-restart lxc-start lxc-version lxc-checkpoint lxc-execute lxc-ls lxc-restore lxc-start-ephemeral lxc-wait
14. LXC: lxc-create $ sudo lxc-create -t ubuntu -n mycontainer [sudo] password for marko: No config file specified, using the default config debootstrap is /usr/sbin/debootstrap Checking cache download in /var/cache/lxc/precise/ rootfs-amd64 ... Copy /var/cache/lxc/precise/rootfs-amd64 to /var/lib/ lxc/mycontainer/rootfs ... Copying rootfs to /var/lib/lxc/mycontainer/rootfs ... ## # The default user is 'ubuntu' with password 'ubuntu'! # Use the 'sudo' command to run tasks as root in the container. ## 'ubuntu' template installed 'mycontainer' created
15. LXC: it’s all just files $ ls /var/lib/lxc/mycontainer/ config fstab rootfs $ ls /var/lib/lxc/mycontainer/rootfs/ bin boot dev etc home lib lib64 media mnt opt proc root run sbin selinux srv sys tmp usr var new linux!
16. LXC: logging in with lxc-console Ubuntu 12.04.3 LTS mycontainer console mycontainer login: ubuntu Password: Welcome to Ubuntu 12.04.3 LTS (GNU/Linux 3.2.0-35-generic x86_64) * Documentation: https://help.ubuntu.com/ The programs included with the Ubuntu system are free software; the exact distribution terms for each program are described in the individual files in /usr/share/doc/*/copyright. Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by applicable law. ubuntu@mycontainer:~$
17. LXC: further reading https://help.ubuntu.com/lts/serverguide/lxc.html lxc-users mailing list
18. How to put stuff in containers to make them useful?
21. open source systems configuration and integration framework http://www.opscode.com/chef/
22. Infrastructure as source code
23. Chef: spec for putting a system in a desired state
24. case node[:platform] when "redhat", "centos", "fedora" package "ImageMagick" when "debian", "ubuntu" package "imagemagick" end Chef: recipes
25. user = node[:firefox][:user] directory "/home/#{user}/.mozilla" do owner user group user mode "0755" action :create end Chef: example - resources & attributes
26. Host *<%= @hostname %>* StrictHostKeyChecking no UserKnownHostsFile=/dev/null Chef: templates
27. !"" attributes # $"" default.rb !"" metadata.json !"" metadata.rb !"" README.md !"" recipes # $"" default.rb $"" templates $"" default !"" user.js.erb $"" xvfb.erb Chef: cookbooks
29. Tools: chef-client, chef- server, data bags, knife...
30. Tools: chef-client, chef- server, data bags, knife... Start with chef-solo
31. minimal chef-solo setup https://github.com/markoa/chef-solo-mnmal
32. new container technology
33. new container technology what’s the story?
34. $ docker pull ubuntu Pulling repository ubuntu 8dbd9e392a96: Download complete b750fe79269d: Download complete 27cf78414709: Download complete $ docker run -d ubuntu /bin/sh -c "while true; do echo hello world; sleep 1; done" 6a98773e8e1c $ docker logs 6a9 hello world hello world hello world
35. union file system: shared read-only areas for all running containers (saves space)
36. $ docker inspect 6a9 [{ ... "Image": "8dbd9e392a9640564...c1c", "NetworkSettings": { "IPAddress": "172.17.0.2", "IPPrefixLen": 16, "Gateway": "172.17.42.1", "Bridge": "docker0", "PortMapping": { "Tcp": {}, "Udp": {} } }, ... }] networking: ✔
37. $ docker pull ubuntu Pulling repository ubuntu 8dbd9e392a96: Download complete b750fe79269d: Download complete 27cf78414709: Download complete $ docker run -d ubuntu /bin/sh -c "while true; do echo hello world; sleep 1; done" 6a98773e8e1c $ docker logs 6a9 hello world hello world hello world Docker runs containers as applications
39. I tried to picture clusters of information as they moved through the computer.
40. Containers are more portable than DIY configuration.
41. FROM ubuntu:12.04 ## MYSQL RUN apt-get install -y -q mysql-client ## RAILS APP ADD ./docker-rails /srv/docker-rails RUN cd /srv/docker-rails;bundle install EXPOSE 3000 CMD ["/usr/local/bin/supervisord", "-n", "- c", "/etc/supervisord.conf"] Dockerfile: syntax for provisioning
42. $ docker commit $CONTAINER_ID myname/ kickassapp $ docker push myname/kickassapp Cool thing: version control for container changes
43. Yes, but... • not production ready (yet)
44. Yes, but... • not production ready (yet) • works only on Ubuntu
45. Yes, but... • not production ready (yet) • works only on Ubuntu (atm) • new devops challenges
46. used to $ rails generate model Post $ docker run myapp production coming up?
47. Thank you! Marko Anastasov @markoa semaphoreapp.com renderedtext.com