Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Self Contained Deployment

Rach Belaid
September 19, 2014

Self Contained Deployment

Love story between Docker and Packer to transform your configuration management in a self contained application to run like a binary

Rach Belaid

September 19, 2014

More Decks by Rach Belaid

Other Decks in Programming


  1. About Me • Python developer • Love Ops • Run

    the Pyramid Meetup in London • Defender of Postgres, RDMS and SQL • Hack with Haskell, Erlang, Rust in my spare time on twitter : @rachbelaid
  2. What Self-contained means? • No dependency required to run •

    State Less • Simplify shipping code to ship more often • the Holy Grail of deployment Running an application like a binary
  3. Why Self-Contained? • Easy to deploy. Easy like a binary?

    • Easy to test/run • Component base architecture • Continuous delivery • Functional programming of deployment
  4. Immutable server : cons • Hard • Can be slow

    (boot and create images) • Require a VPS architecture • Not testable locally
  5. Is Configuration Management the answer? • Repetitive • No rollback

    • Not always deterministic (distro, updates, …) • Tendency to create a monolithic platform • Often too slow Great, but :
  6. Daily DevOps workflow • vagrant up • Read HackerNews •

    20min later, notice that it failed • vagrant provision • Read more HackerNews • Success!!
 We just want to download and run it
  7. What is docker The docker project offers higher-level tools, working

    together, which are built on top of some Linux kernel features.
 It’s providing an additional layer of abstraction and automation of operating system–level virtualization on Linux.
  8. What is docker • Framework / Toolkit to create containers

    • Platform to build distributed app and link them together • Build application container which run everywhere • Faster than normal VM and more convenient
  9. Docker goal The goal is to help developers and system

    administrators port applications - with all of their dependencies conjointly - and get them running across systems and machines - headache free.
  10. Enter Packer • Packer is a tool for creating identical

    machine images for multiple platforms from a single source configuration • Packer can help you build Docker containers * • Allow to transition and experiment with docker container • Glue between Configuration management and Docker *not on MacOS / Win
  11. What’s Packer? Provisioner
 (shell) Provisioner
 ( … ) Provisioner

 (AWS AMI) Builder
 (Docker container) Builder
 (Docker container) Post Processor
 (Vagrant Box) Post Processor
 (Docker push)
  12. Environment setup Vagrant.configure("2") do |config| config.vm.box = "ubuntu/trusty64" config.vm.provision "docker"

    do |d| end end To get a VM ready to use! 
 vagrant up && vagrant ssh wget https://dl.bintray.com/mitchellh/packer/ packer_0.7.1_linux_amd64.zip ! unzip packer_0.7.1_linux_amd64.zip -d ~/packer
 export PATH=$PATH:~/packer/
  13. Example : structure . ├── ansible │ ├── app.yaml │

    └── templates │ ├── app.py │ └── supervisord.conf └── build.json
  14. Example: Ansible provisioning - hosts: all tasks: - name: Install

    application requirements apt: pkg={{ item }} state=latest with_items: - supervisor - build-essential - python - python-pip - python-dev
  15. Example: Ansible provisioning (part 2) ...
 - name: Install python

    requirement pip: name={{ item }} state=present with_items: - pyramid - uwsgi - name: Supervisor template template: >
 dest=/etc/supervisor/supervisord.conf src=templates/supervisord.conf - name: add App template: dest=/srv/app.py src=templates/app.py
  16. Example: Packer file {"provisioners": [ { "type": "shell", "inline": [

    "sudo add-apt-repository ppa:rquillo/ansible -y", "apt-get update", "apt-get install -y sudo ansible python-apt" ]}, { "type": "ansible-local", "playbook_dir": "ansible", "playbook_file": "ansible/app.yaml" } ], "builders": [{ "type": "docker", "image": "ubuntu", "export_path": "app.tar" }]}
  17. Example: Python Web App from pyramid.config import Configurator from pyramid.view

    import view_config ! @view_config(route_name='hello', renderer='json') def hello(request): return {'status': 'Hello World'} ! def application(): config = Configurator() config.add_route('hello', '/') config.scan() return config.make_wsgi_app() ! app = application()
  18. Example : Supervisor File [program:app] command=
 uwsgi --http :80 --wsgi-file

    /srv/app.py --callable app [supervisord] # required by supervisord
  19. Run it $> docker run -p 8080:80 pyconuk/myapp supervisord --nodaemon

    $> curl localhost:8080 {"status": "Hello World"} Success!!
  20. Who is using/supporting Docker • Google, Microsoft, Amazon, Red Hat,

    Digital Ocean, • Real Projects using it : New Relics, Circle Ci, .. • Exciting projects: Panamax, Fig, ClusterHq • Loads of Millions $$$ • Production ready (ish)!!
  21. What are the benefits • Integrate Easily in CI to

    do Continuous Delivery • Move to an archicteture using Microservices / SOA • Access to great tools to build cluster & distributed system :
 Mesos / CoreOS / Mesosphere • Isolation and Sandboxing • Great to limit CPU / IO / Memory resources (DOS, QOS, PAAS)
  22. Introducing new tech is hard • Small changes • Iterative

    • Real beneficial improvement (solve a real problem) Suggestion: