Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Real World Docker for the Rubyist

Real World Docker for the Rubyist

Docker’s gotten a lot of press, but how does it fare in the real world Rubyists inhabit every day?

Together we’ll take a deep dive into how a real company transformed itself to run on Docker. We’ll see how to build and maintain Docker images tailored for Ruby. We’ll dig into proper configuration and deployment options for containerized applications. Along the way we’ll highlight the pitfalls, bugs and gotchas that come with such a young, fast moving platform like Docker.

Whether you’re in production with Docker or just dabbling, come learn how Docker and Ruby make an awesome combination.

Jason R Clark

May 06, 2016
Tweet

More Decks by Jason R Clark

Other Decks in Technology

Transcript

  1. Real World Docker
    for the Rubyist
    @jasonrclark

    View full-size slide

  2. I'm Jane, a new
    New Relic with a
    cool new feature
    to deploy!
    I'm Jill, an
    experienced Relic,
    ready to help

    View full-size slide

  3. This document and the information herein (including any information that may be incorporated by reference) is provided for
    informational purposes only and should not be construed as an offer, commitment, promise or obligation on behalf of New Relic, Inc.
    (“New Relic”) to sell securities or deliver any product, material, code, functionality, or other feature. Any information provided hereby
    is proprietary to New Relic and may not be replicated or disclosed without New Relic’s express written permission.
    Such information may contain forward-looking statements within the meaning of federal securities laws. Any statement that is not a
    historical fact or refers to expectations, projections, future plans, objectives, estimates, goals, or other characterizations of future
    events is a forward-looking statement. These forward-looking statements can often be identified as such because the context of the
    statement will include words such as “believes,” “anticipates,” “expects” or words of similar import.
    Actual results may differ materially from those expressed in these forward-looking statements, which speak only as of the date
    hereof, and are subject to change at any time without notice. Existing and prospective investors, customers and other third parties
    transacting business with New Relic are cautioned not to place undue reliance on this forward-looking information. The achievement
    or success of the matters covered by such forward-looking statements are based on New Relic’s current assumptions, expectations,
    and beliefs and are subject to substantial risks, uncertainties, assumptions, and changes in circumstances that may cause the actual
    results, performance, or achievements to differ materially from those expressed or implied in any forward-looking statement. Further
    information on factors that could affect such forward-looking statements is included in the filings we make with the SEC from time to
    time. Copies of these documents may be obtained by visiting New Relic’s Investor Relations website at ir.newrelic.com or the SEC’s
    website at www.sec.gov.
    New Relic assumes no obligation and does not intend to update these forward-looking statements, except as required by law. New
    Relic makes no warranties, expressed or implied, in this document or otherwise, with respect to the information provided.

    View full-size slide

  4. Why Docker?
    Packaging!

    View full-size slide

  5. 6
    image container
    docker
    run

    View full-size slide

  6. 7
    image container
    docker
    run
    class instance
    new()

    View full-size slide

  7. Why Docker?
    Isolation!

    View full-size slide

  8. 9
    server
    /app
    /other-app

    View full-size slide

  9. 10
    server
    /app
    /other-app
    container
    container

    View full-size slide

  10. How do I get
    started?
    Install Docker
    Toolkit and write
    your Dockerfile

    View full-size slide

  11. 12
    https://www.docker.com/products/docker-toolbox

    View full-size slide

  12. 13
    FROM newrelic/base-builder
    loc-service/Dockerfile

    View full-size slide

  13. 14
    FROM ruby:2.3
    WORKDIR /data/app
    COPY Gemfile Gemfile.lock /data/app
    RUN bundle install --deployment
    ADD . /data/app
    CMD ["unicorn", "--config", "..."]
    Dockerfile
    ?

    View full-size slide

  14. Where's the rest
    of my Dockerfile?
    Let's deploy your
    service first!

    View full-size slide

  15. $terminal
    >>>
    16
    $ docker build -t loc-service .

    View full-size slide

  16. $terminal
    >>>
    17
    $ docker build -t loc-service .
    Sending build context to Docker daemon
    Step 1 : FROM newrelic/base-builder
    2.3: Pulling from newrelic/base-builder
    ...
    ---> 7ca70eb2dfea
    Step 2 : COPY . /data/app
    ---> 9f54fdf6b489
    Removing intermediate container 669fd221
    ...
    Step 14 : CMD supervisord
    Removing intermediate container 5247f58d
    Successfully built 1773de8387dc
    $

    View full-size slide

  17. $terminal
    >>>
    18
    $ docker images
    REPOSITORY TAG IMAGE ID CREATED
    loc-service latest sha256:1773d 4 min ago

    View full-size slide

  18. How do I get the
    image off my
    machine?
    Good question!

    View full-size slide

  19. How do I get my
    containers
    started?
    Centurion!

    View full-size slide

  20. $terminal
    >>>
    22
    $ docker run loc-service
    CRIT Supervisor running as root (no user in config file)
    WARN Included extra file "/etc/supervisord.d/app.conf"
    during parsing
    WARN Included extra file "/etc/supervisord.d/nginx.conf"
    during parsing
    WARN Included extra file "/etc/supervisord.d/
    permissions.conf" during parsing
    INFO supervisord started with pid 1
    INFO spawned: 'fix-permissions' with pid 8
    INFO spawned: 'web-unicorn' with pid 9
    INFO spawned: 'nginx' with pid 10
    ...

    View full-size slide

  21. 25
    namespace :environment do
    desc 'Staging environment'
    task :staging => :common do
    set :image, 'newrelic/loc-service'
    host 'staging-1.nr-internal.net'
    host 'staging-2.nr-internal.net'
    end
    end
    centurion-config/loc-service.rake

    View full-size slide

  22. 26
    namespace :environment do
    desc 'Staging environment'
    task :staging do
    set :image, 'newrelic/loc-service'
    host 'staging-1.nr-internal.net'
    host 'staging-2.nr-internal.net'
    end
    end
    centurion-config/loc-service.rake

    View full-size slide

  23. 27
    namespace :environment do
    desc 'Staging environment'
    task :staging do
    set :image, 'newrelic/loc-service'
    host 'staging-1.nr-internal.net'
    host 'staging-2.nr-internal.net'
    end
    end
    centurion-config/loc-service.rake

    View full-size slide

  24. 28
    namespace :environment do
    desc 'Staging environment'
    task :staging do
    set :image, 'newrelic/loc-service'
    host 'staging-1.nr-internal.net'
    host 'staging-2.nr-internal.net'
    end
    end
    centurion-config/loc-service.rake

    View full-size slide

  25. $terminal
    >>>
    29
    $ gem install centurion

    View full-size slide

  26. $terminal
    >>>
    30
    $ centurion -e staging \
    -p loc-service \
    -a deploy

    View full-size slide

  27. $terminal
    >>>
    32
    $ centurion -e staging \
    -p loc-service \
    -a deploy
    ...Fetching image loc-service:latest
    ...Using CLI to pull latest
    -- Connecting to staging-1.nr-internal.net
    RESOLVED loc-service:latest => 999cd48e588d
    ** Invoke deploy:verify_image (first_time)
    ** Execute deploy:verify_image
    -- Connecting to staging-1.nr-internal.net
    ....
    Found container up for 6 seconds
    Container is up!

    View full-size slide

  28. $terminal
    >>>
    33
    $ centurion -e staging \
    -p loc-service \
    -a deploy
    ...Fetching image loc-service:latest
    ...Using CLI to pull latest
    -- Connecting to staging-1.nr-internal.net
    RESOLVED loc-service:latest => 999cd48e588d
    ** Invoke deploy:verify_image (first_time)
    ** Execute deploy:verify_image
    -- Connecting to staging-2.nr-internal.net
    ....
    Found container up for 6 seconds
    Container is up!

    View full-size slide

  29. $terminal
    >>>
    34
    $ centurion -e staging \
    -p loc-service \
    -a deploy
    ...Fetching image loc-service:latest
    ...Using CLI to pull latest
    -- Connecting to staging-1.nr-internal.net
    RESOLVED loc-service:latest => 999cd48e588d
    ** Invoke deploy:verify_image (first_time)
    ** Execute deploy:verify_image
    -- Connecting to staging-2.nr-internal.net
    ....
    Found container up for 6 seconds
    Container is up!

    View full-size slide

  30. WHERE'S MY
    APP!?!?!

    View full-size slide

  31. Yes, but how
    does that help?
    Are your logs
    going to STDOUT?

    View full-size slide

  32. 39
    # STDOUT or get out!
    Rails.logger = Logger.new(STDOUT)
    config/environments/production.rb

    View full-size slide

  33. How?
    Let's take a closer
    look on the
    container

    View full-size slide

  34. $terminal
    >>>
    42
    $ docker -H tcp://staging-1.nr-internal.net:2375 ps
    CONTAINER ID IMAGE COMMAND
    5189764ce084 loc-service "supervisord"

    View full-size slide

  35. $terminal
    >>>
    43
    $ docker -H tcp://staging-1.nr-internal.net:2375 ps
    CONTAINER ID IMAGE COMMAND
    5189764ce084 loc-service "supervisord"
    $ docker -H tcp://staging-1.nr-internal.net:2375 \
    exec -it 5189764ce084 bash
    [root@5189764ce084 /]#

    View full-size slide

  36. About 300MB
    How much
    memory does your
    app use?

    View full-size slide

  37. 45
    namespace :environment do
    desc 'Staging environment'
    task :staging do
    set :image, 'newrelic/loc-service'
    memory 2.gigabytes
    host 'staging-1.nr-internal.net'
    host 'staging-2.nr-internal.net'
    end
    end
    centurion-config/loc-service.rake

    View full-size slide

  38. $terminal
    >>>
    46
    $ docker run -m 2g loc-service

    View full-size slide

  39. How do I get more
    Unicorn workers?
    Try the ENV!

    View full-size slide

  40. $terminal
    >>>
    48
    $ docker run -it -e UNICORN_WORKERS=3 loc-service bash
    [root@5189764ce084 /]# env
    ...
    UNICORN_WORKERS=3

    View full-size slide

  41. 49
    ...
    # how many worker processes to start
    worker_processes ENV["UNICORN_WORKERS"].to_i
    unicorn_config.rb

    View full-size slide

  42. 50
    namespace :environment do
    task :common do
    ...
    env_vars UNICORN_WORKERS: 3
    end
    end
    centurion-config/loc-service.rake

    View full-size slide

  43. 51
    mysql: &mysql
    encoding: utf8
    adapter: <%=ENV['DB_ADAPTER']||'mysql2'%>
    username:<%=ENV['DB_USERNAME']||'root'%>
    host: <%=ENV['DB_HOST']||'localhost'%>
    password:'<%=ENV['DB_PASSWORD']%>'
    database:<%=ENV['DB_SCHEMA']%>
    config/database.yml

    View full-size slide

  44. 52
    module MyRailsApp
    class Application < Rails::Application
    config.service_url =
    (ENV['SERVICE_URL'] || 'http://...')
    config.service_timeout =
    (ENV['SERVICE_TIMEOUT'] || 5).to_f
    end
    end
    config/application.rb

    View full-size slide

  45. 53
    class LinesOfCodeController
    def index
    fetch(Rails.configuration.service_url,
    Rails.configuration.service_timeout)
    end
    end
    app/controllers/loc_controller.rb

    View full-size slide

  46. That one-line
    Dockerfile...
    how's that work?
    Let's take a look!

    View full-size slide

  47. 57
    FROM newrelic/centos:6
    ...
    RUN rbenv install 2.3.0
    base-builder/Dockerfile

    View full-size slide

  48. 58
    FROM newrelic/centos:6
    ...
    RUN rbenv install 2.3.0
    # Install supervisord...
    RUN easy_install supervisor
    # Install nginx...
    RUN yum install -y nginx
    base-builder/Dockerfile

    View full-size slide

  49. 59
    FROM newrelic/centos:6
    ...
    RUN rbenv install 2.3.0
    RUN gem install bundler
    RUN rbenv rehash
    ...
    base-builder/Dockerfile

    View full-size slide

  50. 60
    FROM newrelic/centos:6
    ...
    RUN rbenv install 2.3.0
    RUN gem install bundler
    RUN rbenv rehash
    RUN bundle install
    ...
    base-builder/Dockerfile

    View full-size slide

  51. $terminal
    >>>
    61
    $ docker build -t newrelic/base-builder .
    ...
    Step 3 : RUN bundle install
    ---> Running in cb91de794b44
    Could not locate Gemfile

    View full-size slide

  52. 62
    FROM newrelic/centos:6
    ...
    RUN rbenv install 2.3.0
    RUN gem install bundler
    RUN rbenv rehash
    ONBUILD COPY Gemfile* .
    ONBUILD RUN bundle install
    base-builder/Dockerfile

    View full-size slide

  53. 63
    FROM newrelic/base-builder
    RUN use-puma
    RUN try-python
    another-app/Dockerfile

    View full-size slide

  54. 64
    #! /bin/sh
    cp /etc/templates/web.puma.conf \
    /etc/supervisord.d/app.conf
    cp /etc/templates/puma_config.rb \
    /data/app/config/puma_config.rb
    base-builder/usr/bin/use-puma

    View full-size slide

  55. 65
    FROM newrelic/base-builder
    RUN use-puma
    RUN try-python
    another-app/Dockerfile
    !!!

    View full-size slide

  56. I'm getting an
    error writing a file
    What's the error?

    View full-size slide

  57. $terminal
    >>>
    67
    Errno::EACCES: Permission denied @ rb_sysopen
    - /just-a-file-nothing-to-see-here.txt

    View full-size slide

  58. With who?
    You've got a
    problem with
    nobody

    View full-size slide

  59. 70
    [program:web-unicorn]
    command=bundle exec unicorn -c config/unicorn_config.rb
    autostart=true
    autorestart=true
    directory=/data/app
    user=nobody
    stdout_logfile = /dev/stdout
    stdout_logfile_maxbytes = 0
    stderr_logfile = /dev/stderr
    stderr_logfile_maxbytes = 0
    /etc/supervisord.d/web-unicorn.conf

    View full-size slide

  60. If Docker runs as
    root, why don't
    we?
    Defense in depth!

    View full-size slide

  61. Can I run my tests
    in Docker?
    There are
    options...

    View full-size slide

  62. $terminal
    >>>
    73
    $ docker run loc-service bundle exec rake
    /Users/jclark/.rbenv/versions/2.2.4/bin/ruby ...
    Randomized with seed 22351
    .................
    Finished in 0.23058 seconds (files took 2.86 seconds to
    load)
    17 examples, 0 failures
    Randomized with seed 22351
    $

    View full-size slide

  63. $terminal
    >>>
    74
    $ docker build -t loc-service . && \
    docker run loc-service bundle exec rake
    docker build .
    Sending build context to Docker daemon...
    Step 1 : FROM newrelic/base-builder
    # Executing 12 build triggers...
    ...
    ...
    ...
    /Users/jclark/.rbenv/versions/2.2.4/bin/ruby ...
    ...

    View full-size slide

  64. $terminal
    >>>
    75
    $ docker build -t loc-service . && \
    docker run loc-service bundle exec rake
    docker build .
    Sending build context to Docker daemon...
    Step 1 : FROM newrelic/base-builder
    # Executing 12 build triggers...
    ...
    ...
    ...
    /Users/jclark/.rbenv/versions/2.2.4/bin/ruby ...
    ...

    View full-size slide

  65. $terminal
    >>>
    76
    $ docker run -it \
    -v /src/my-app:/test-app \
    loc-image \
    "cd /test-app && bundle exec rake"

    View full-size slide

  66. I need a Redis
    instance
    We've got just the
    thing

    View full-size slide

  67. 78
    namespace :environment do
    desc 'Staging environment'
    task :staging do
    set :image, 'newrelic/redis'
    memory 1.gigabytes
    host 'staging-3.nr-internal.net'
    env_vars REDIS_MAXMEMORY: '1G'
    env_vars REDIS_PWD: 'WOULDNTYOULIKETOKNOW'
    end
    centurion-config/loc-redis.rake

    View full-size slide

  68. 79
    namespace :environment do
    desc 'Staging environment'
    task :staging do
    set :image, 'newrelic/redis'
    memory 1.gigabytes
    host 'staging-3.nr-internal.net'
    env_vars REDIS_MAXMEMORY: '1G'
    env_vars REDIS_PWD: 'WOULDNTYOULIKETOKNOW'
    end
    centurion-config/loc-redis.rake

    View full-size slide

  69. 80
    namespace :environment do
    desc 'Staging environment'
    task :staging do
    set :image, 'newrelic/redis'
    memory 1.gigabytes
    host 'staging-3.nr-internal.net'
    env_vars REDIS_MAXMEMORY: '1G'
    env_vars REDIS_PWD: 'WOULDNTYOULIKETOKNOW'
    end
    centurion-config/loc-redis.rake

    View full-size slide

  70. Centurion's cool,
    but what else is
    out there?
    A lot!

    View full-size slide

  71. deploying with centurion
    ENV-based configuration
    building shared images
    security
    testing
    the future
    thanks!
    @jasonrclark

    View full-size slide