Self Contained Deployment

Love story between Docker and Packer to transform your configuration management in a self contained application to run like a binary

Transcript

1. Self Contained Deployment a love story between Docker and Packer PyCon UK 2014
2. About Me • Python developer • Love Ops • Run the Pyramid Meetup in London • Defender of Postgres, RDMS and SQL • Hack with Haskell, Erlang, Rust in my spare time on twitter : @rachbelaid
3. What Self-contained means? • No dependency required to run • State Less • Simplify shipping code to ship more often • the Holy Grail of deployment Running an application like a binary
4. Why Self-Contained? • Easy to deploy. Easy like a binary? • Easy to test/run • Component base architecture • Continuous delivery • Functional programming of deployment
5. Immutable server © Martin Fowler
6. Immutable server Beautiful in theory, but hard in practice.
7. Immutable server : pros • Beautiful in theory • Versioned • Easy rollback
8. Immutable server : cons • Hard • Can be slow (boot and create images) • Require a VPS architecture • Not testable locally
9. Is Configuration Management the answer? • Repetitive • No rollback • Not always deterministic (distro, updates, …) • Tendency to create a monolithic platform • Often too slow Great, but :
10. Daily DevOps workflow • vagrant up • Read HackerNews • 20min later, notice that it failed • vagrant provision • Read more HackerNews • Success!!
 
 
 We just want to download and run it
11. How to solve this?
12. What is docker The docker project offers higher-level tools, working together, which are built on top of some Linux kernel features.
 It’s providing an additional layer of abstraction and automation of operating system–level virtualization on Linux.
13. What is docker • Framework / Toolkit to create containers • Platform to build distributed app and link them together • Build application container which run everywhere • Faster than normal VM and more convenient
14. Docker goal The goal is to help developers and system administrators port applications - with all of their dependencies conjointly - and get them running across systems and machines - headache free.
15. What about my conf Management? Let’s not waste months/years of investment !
16. Enter Packer • Packer is a tool for creating identical machine images for multiple platforms from a single source configuration • Packer can help you build Docker containers * • Allow to transition and experiment with docker container • Glue between Configuration management and Docker *not on MacOS / Win
17. What’s Packer? Provisioner
 (shell) Provisioner
 ( … ) Provisioner
 (ansible) Builder
 (AWS AMI) Builder
 (Docker container) Builder
 (Docker container) Post Processor
 (Vagrant Box) Post Processor
 (Docker push)
18. Example Let’s build a self-contained web application and run it
19. Environment setup Vagrant.configure("2") do |config| config.vm.box = "ubuntu/trusty64" config.vm.provision "docker" do |d| end end To get a VM ready to use! 
 vagrant up && vagrant ssh wget https://dl.bintray.com/mitchellh/packer/ packer_0.7.1_linux_amd64.zip ! unzip packer_0.7.1_linux_amd64.zip -d ~/packer
 export PATH=$PATH:~/packer/
20. Example : structure . ├── ansible │ ├── app.yaml │ └── templates │ ├── app.py │ └── supervisord.conf └── build.json
21. Example: Ansible provisioning - hosts: all tasks: - name: Install application requirements apt: pkg={{ item }} state=latest with_items: - supervisor - build-essential - python - python-pip - python-dev
22. Example: Ansible provisioning (part 2) ...
 - name: Install python requirement pip: name={{ item }} state=present with_items: - pyramid - uwsgi - name: Supervisor template template: >
 dest=/etc/supervisor/supervisord.conf src=templates/supervisord.conf - name: add App template: dest=/srv/app.py src=templates/app.py
23. Example: Packer file {"provisioners": [ { "type": "shell", "inline": [ "sudo add-apt-repository ppa:rquillo/ansible -y", "apt-get update", "apt-get install -y sudo ansible python-apt" ]}, { "type": "ansible-local", "playbook_dir": "ansible", "playbook_file": "ansible/app.yaml" } ], "builders": [{ "type": "docker", "image": "ubuntu", "export_path": "app.tar" }]}
24. Example: Python Web App from pyramid.config import Configurator from pyramid.view import view_config ! @view_config(route_name='hello', renderer='json') def hello(request): return {'status': 'Hello World'} ! def application(): config = Configurator() config.add_route('hello', '/') config.scan() return config.make_wsgi_app() ! app = application()
25. Example : Supervisor File [program:app] command=
 uwsgi --http :80 --wsgi-file /srv/app.py --callable app [supervisord] # required by supervisord
26. Build it packer build build.json
27. Import it docker import - pyconuk/myapp < app.tar app.tar contains everything needed to run
28. Run it $> docker run -p 8080:80 pyconuk/myapp supervisord --nodaemon $> curl localhost:8080 {"status": "Hello World"} Success!!
29. Who is using/supporting Docker • Google, Microsoft, Amazon, Red Hat, Digital Ocean, • Real Projects using it : New Relics, Circle Ci, .. • Exciting projects: Panamax, Fig, ClusterHq • Loads of Millions $$$ • Production ready (ish)!!
30. New tech == New problems • Container with State (DB/Cache) • Networks
31. What are the benefits • Integrate Easily in CI to do Continuous Delivery • Move to an archicteture using Microservices / SOA • Access to great tools to build cluster & distributed system :
 
 Mesos / CoreOS / Mesosphere • Isolation and Sandboxing • Great to limit CPU / IO / Memory resources (DOS, QOS, PAAS)
32. Introducing new tech is hard • Small changes • Iterative • Real beneficial improvement (solve a real problem) Suggestion:
33. If You Build It, They Will Come The End.