Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Self Contained Deployment

F0b6891740978c069990b45bc7bfd58a?s=47 Rach Belaid
September 19, 2014

Self Contained Deployment

Love story between Docker and Packer to transform your configuration management in a self contained application to run like a binary


Rach Belaid

September 19, 2014


  1. Self Contained Deployment a love story between Docker and Packer

    PyCon UK 2014
  2. About Me • Python developer • Love Ops • Run

    the Pyramid Meetup in London • Defender of Postgres, RDMS and SQL • Hack with Haskell, Erlang, Rust in my spare time on twitter : @rachbelaid
  3. What Self-contained means? • No dependency required to run •

    State Less • Simplify shipping code to ship more often • the Holy Grail of deployment Running an application like a binary
  4. Why Self-Contained? • Easy to deploy. Easy like a binary?

    • Easy to test/run • Component base architecture • Continuous delivery • Functional programming of deployment
  5. Immutable server © Martin Fowler

  6. Immutable server Beautiful in theory, but hard in practice.

  7. Immutable server : pros • Beautiful in theory • Versioned

    • Easy rollback
  8. Immutable server : cons • Hard • Can be slow

    (boot and create images) • Require a VPS architecture • Not testable locally
  9. Is Configuration Management the answer? • Repetitive • No rollback

    • Not always deterministic (distro, updates, …) • Tendency to create a monolithic platform • Often too slow Great, but :
  10. Daily DevOps workflow • vagrant up • Read HackerNews •

    20min later, notice that it failed • vagrant provision • Read more HackerNews • Success!!
 We just want to download and run it
  11. How to solve this?

  12. What is docker The docker project offers higher-level tools, working

    together, which are built on top of some Linux kernel features.
 It’s providing an additional layer of abstraction and automation of operating system–level virtualization on Linux.
  13. What is docker • Framework / Toolkit to create containers

    • Platform to build distributed app and link them together • Build application container which run everywhere • Faster than normal VM and more convenient
  14. Docker goal The goal is to help developers and system

    administrators port applications - with all of their dependencies conjointly - and get them running across systems and machines - headache free.
  15. What about my conf Management? Let’s not waste months/years of

    investment !
  16. Enter Packer • Packer is a tool for creating identical

    machine images for multiple platforms from a single source configuration • Packer can help you build Docker containers * • Allow to transition and experiment with docker container • Glue between Configuration management and Docker *not on MacOS / Win
  17. What’s Packer? Provisioner
 (shell) Provisioner
 ( … ) Provisioner

 (AWS AMI) Builder
 (Docker container) Builder
 (Docker container) Post Processor
 (Vagrant Box) Post Processor
 (Docker push)
  18. Example Let’s build a self-contained web application and run it

  19. Environment setup Vagrant.configure("2") do |config| config.vm.box = "ubuntu/trusty64" config.vm.provision "docker"

    do |d| end end To get a VM ready to use! 
 vagrant up && vagrant ssh wget https://dl.bintray.com/mitchellh/packer/ packer_0.7.1_linux_amd64.zip ! unzip packer_0.7.1_linux_amd64.zip -d ~/packer
 export PATH=$PATH:~/packer/
  20. Example : structure . ├── ansible │ ├── app.yaml │

    └── templates │ ├── app.py │ └── supervisord.conf └── build.json
  21. Example: Ansible provisioning - hosts: all tasks: - name: Install

    application requirements apt: pkg={{ item }} state=latest with_items: - supervisor - build-essential - python - python-pip - python-dev
  22. Example: Ansible provisioning (part 2) ...
 - name: Install python

    requirement pip: name={{ item }} state=present with_items: - pyramid - uwsgi - name: Supervisor template template: >
 dest=/etc/supervisor/supervisord.conf src=templates/supervisord.conf - name: add App template: dest=/srv/app.py src=templates/app.py
  23. Example: Packer file {"provisioners": [ { "type": "shell", "inline": [

    "sudo add-apt-repository ppa:rquillo/ansible -y", "apt-get update", "apt-get install -y sudo ansible python-apt" ]}, { "type": "ansible-local", "playbook_dir": "ansible", "playbook_file": "ansible/app.yaml" } ], "builders": [{ "type": "docker", "image": "ubuntu", "export_path": "app.tar" }]}
  24. Example: Python Web App from pyramid.config import Configurator from pyramid.view

    import view_config ! @view_config(route_name='hello', renderer='json') def hello(request): return {'status': 'Hello World'} ! def application(): config = Configurator() config.add_route('hello', '/') config.scan() return config.make_wsgi_app() ! app = application()
  25. Example : Supervisor File [program:app] command=
 uwsgi --http :80 --wsgi-file

    /srv/app.py --callable app [supervisord] # required by supervisord
  26. Build it packer build build.json

  27. Import it docker import - pyconuk/myapp < app.tar app.tar contains

    everything needed to run
  28. Run it $> docker run -p 8080:80 pyconuk/myapp supervisord --nodaemon

    $> curl localhost:8080 {"status": "Hello World"} Success!!
  29. Who is using/supporting Docker • Google, Microsoft, Amazon, Red Hat,

    Digital Ocean, • Real Projects using it : New Relics, Circle Ci, .. • Exciting projects: Panamax, Fig, ClusterHq • Loads of Millions $$$ • Production ready (ish)!!
  30. New tech == New problems • Container with State (DB/Cache)

    • Networks
  31. What are the benefits • Integrate Easily in CI to

    do Continuous Delivery • Move to an archicteture using Microservices / SOA • Access to great tools to build cluster & distributed system :
 Mesos / CoreOS / Mesosphere • Isolation and Sandboxing • Great to limit CPU / IO / Memory resources (DOS, QOS, PAAS)
  32. Introducing new tech is hard • Small changes • Iterative

    • Real beneficial improvement (solve a real problem) Suggestion:
  33. If You Build It, They Will Come The End.