Self Contained Deployment

Love story between Docker and Packer to transform your configuration management in a self contained application to run like a binary

Transcript

1. Self Contained
   Deployment
   a love story between Docker and Packer
   PyCon UK 2014
   

   View Slide

2. About Me
   • Python developer
   • Love Ops
   • Run the Pyramid Meetup in London
   • Defender of Postgres, RDMS and SQL
   • Hack with Haskell, Erlang, Rust in my spare time
   on twitter : @rachbelaid
   

   View Slide

3. What Self-contained means?
   • No dependency required to run
   • State Less
   • Simplify shipping code to ship more often
   • the Holy Grail of deployment
   Running an application like a binary
   

   View Slide

4. Why Self-Contained?
   • Easy to deploy. Easy like a binary?
   • Easy to test/run
   • Component base architecture
   • Continuous delivery
   • Functional programming of deployment
   

   View Slide

5. Immutable server
   © Martin Fowler
   

   View Slide

6. Immutable server
   Beautiful in theory, but hard in practice.
   

   View Slide

7. Immutable server : pros
   • Beautiful in theory
   • Versioned
   • Easy rollback
   

   View Slide

8. Immutable server : cons
   • Hard
   • Can be slow (boot and create images)
   • Require a VPS architecture
   • Not testable locally
   

   View Slide

9. Is Configuration
   Management the answer?
   • Repetitive
   • No rollback
   • Not always deterministic (distro, updates, …)
   • Tendency to create a monolithic platform
   • Often too slow
   Great, but :
   

   View Slide

10. Daily DevOps workflow
    • vagrant up
    • Read HackerNews
    • 20min later, notice that it failed
    • vagrant provision
    • Read more HackerNews
    • Success!!

    

    
 We just want to download and run it
    

    View Slide

11. How to solve this?
    

    View Slide

12. What is docker
    The docker project offers higher-level tools, working together,
    which are built on top of some Linux kernel features.

    It’s providing an additional layer of abstraction and automation
    of operating system–level virtualization on Linux.
    

    View Slide

13. What is docker
    • Framework / Toolkit to create containers
    • Platform to build distributed app and link them
    together
    • Build application container which run everywhere
    • Faster than normal VM and more convenient
    

    View Slide

14. Docker goal
    The goal is to help developers and system administrators
    port applications - with all of their dependencies conjointly
    - and get them running across systems and machines -
    headache free.
    

    View Slide

15. What about my conf
    Management?
    Let’s not waste months/years of investment !
    

    View Slide

16. Enter Packer
    • Packer is a tool for creating identical machine
    images for multiple platforms from a single source
    configuration
    • Packer can help you build Docker containers *
    • Allow to transition and experiment with docker
    container
    • Glue between Configuration management and
    Docker
    *not on MacOS / Win
    

    View Slide

17. What’s Packer?
    Provisioner

    (shell)
    Provisioner

    ( … )
    Provisioner

    (ansible)
    Builder

    (AWS AMI)
    Builder

    (Docker container)
    Builder

    (Docker container)
    Post Processor

    (Vagrant Box)
    Post Processor

    (Docker push)
    

    View Slide

18. Example
    Let’s build a self-contained web application and run it
    

    View Slide

19. Environment setup
    Vagrant.configure("2") do |config|
    config.vm.box = "ubuntu/trusty64"
    config.vm.provision "docker" do |d|
    end
    end
    To get a VM ready to use! 

    vagrant up && vagrant ssh
    wget https://dl.bintray.com/mitchellh/packer/
    packer_0.7.1_linux_amd64.zip
    !
    unzip packer_0.7.1_linux_amd64.zip -d ~/packer

    export PATH=$PATH:~/packer/
    

    View Slide

20. Example : structure
    .
    ├── ansible
    │ ├── app.yaml
    │ └── templates
    │ ├── app.py
    │ └── supervisord.conf
    └── build.json
    

    View Slide

21. Example: Ansible
    provisioning
    - hosts: all
    tasks:
    - name: Install application requirements
    apt: pkg={{ item }} state=latest
    with_items:
    - supervisor
    - build-essential
    - python
    - python-pip
    - python-dev
    

    View Slide

22. Example: Ansible
    provisioning (part 2)
    ...

    - name: Install python requirement
    pip: name={{ item }} state=present
    with_items:
    - pyramid
    - uwsgi
    - name: Supervisor template
    template: >

    dest=/etc/supervisor/supervisord.conf
    src=templates/supervisord.conf
    - name: add App
    template: dest=/srv/app.py src=templates/app.py
    

    View Slide

23. Example: Packer file
    {"provisioners": [
    { "type": "shell",
    "inline": [
    "sudo add-apt-repository ppa:rquillo/ansible -y",
    "apt-get update",
    "apt-get install -y sudo ansible python-apt"
    ]},
    { "type": "ansible-local",
    "playbook_dir": "ansible",
    "playbook_file": "ansible/app.yaml" }
    ],
    "builders": [{
    "type": "docker",
    "image": "ubuntu",
    "export_path": "app.tar"
    }]}
    

    View Slide

24. Example: Python Web App
    from pyramid.config import Configurator
    from pyramid.view import view_config
    !
    @view_config(route_name='hello', renderer='json')
    def hello(request):
    return {'status': 'Hello World'}
    !
    def application():
    config = Configurator()
    config.add_route('hello', '/')
    config.scan()
    return config.make_wsgi_app()
    !
    app = application()
    

    View Slide

25. Example : Supervisor File
    [program:app]
    command=

    uwsgi --http :80 --wsgi-file /srv/app.py
    --callable app
    [supervisord]
    # required by supervisord
    

    View Slide

26. Build it
    packer build build.json
    

    View Slide

27. Import it
    docker import - pyconuk/myapp < app.tar
    app.tar contains everything needed to run
    

    View Slide

28. Run it
    $> docker run -p 8080:80 pyconuk/myapp supervisord --nodaemon
    $> curl localhost:8080
    {"status": "Hello World"}
    Success!!
    

    View Slide

29. Who is using/supporting
    Docker
    • Google, Microsoft, Amazon, Red Hat, Digital
    Ocean,
    • Real Projects using it : New Relics, Circle Ci, ..
    • Exciting projects: Panamax, Fig, ClusterHq
    • Loads of Millions $$$
    • Production ready (ish)!!
    

    View Slide

30. New tech == New problems
    • Container with State (DB/Cache)
    • Networks
    

    View Slide

31. What are the benefits
    • Integrate Easily in CI to do Continuous Delivery
    • Move to an archicteture using Microservices / SOA
    • Access to great tools to build cluster & distributed system :

    

    Mesos / CoreOS / Mesosphere
    • Isolation and Sandboxing
    • Great to limit CPU / IO / Memory resources (DOS, QOS,
    PAAS)
    

    View Slide

32. Introducing new tech is hard
    • Small changes
    • Iterative
    • Real beneficial improvement (solve a real problem)
    Suggestion:
    

    View Slide

33. If You Build It, They
    Will Come
    The End.
    

    View Slide