Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Real World Docker for the Rubyist

Real World Docker for the Rubyist

Docker’s gotten a lot of press, but how does it fare in the real world Rubyists inhabit every day?

Together we’ll take a deep dive into how a real company transformed itself to run on Docker. We’ll see how to build and maintain Docker images tailored for Ruby. We’ll dig into proper configuration and deployment options for containerized applications. Along the way we’ll highlight the pitfalls, bugs and gotchas that come with such a young, fast moving platform like Docker.

Whether you’re in production with Docker or just dabbling, come learn how Docker and Ruby make an awesome combination.

Jason R Clark

May 06, 2016
Tweet

More Decks by Jason R Clark

Other Decks in Technology

Transcript

  1. Real World Docker
    for the Rubyist
    @jasonrclark

    View Slide

  2. I'm Jane, a new
    New Relic with a
    cool new feature
    to deploy!
    I'm Jill, an
    experienced Relic,
    ready to help

    View Slide

  3. This document and the information herein (including any information that may be incorporated by reference) is provided for
    informational purposes only and should not be construed as an offer, commitment, promise or obligation on behalf of New Relic, Inc.
    (“New Relic”) to sell securities or deliver any product, material, code, functionality, or other feature. Any information provided hereby
    is proprietary to New Relic and may not be replicated or disclosed without New Relic’s express written permission.
    Such information may contain forward-looking statements within the meaning of federal securities laws. Any statement that is not a
    historical fact or refers to expectations, projections, future plans, objectives, estimates, goals, or other characterizations of future
    events is a forward-looking statement. These forward-looking statements can often be identified as such because the context of the
    statement will include words such as “believes,” “anticipates,” “expects” or words of similar import.
    Actual results may differ materially from those expressed in these forward-looking statements, which speak only as of the date
    hereof, and are subject to change at any time without notice. Existing and prospective investors, customers and other third parties
    transacting business with New Relic are cautioned not to place undue reliance on this forward-looking information. The achievement
    or success of the matters covered by such forward-looking statements are based on New Relic’s current assumptions, expectations,
    and beliefs and are subject to substantial risks, uncertainties, assumptions, and changes in circumstances that may cause the actual
    results, performance, or achievements to differ materially from those expressed or implied in any forward-looking statement. Further
    information on factors that could affect such forward-looking statements is included in the filings we make with the SEC from time to
    time. Copies of these documents may be obtained by visiting New Relic’s Investor Relations website at ir.newrelic.com or the SEC’s
    website at www.sec.gov.
    New Relic assumes no obligation and does not intend to update these forward-looking statements, except as required by law. New
    Relic makes no warranties, expressed or implied, in this document or otherwise, with respect to the information provided.

    View Slide

  4. Why Docker?
    Packaging!

    View Slide

  5. 5

    View Slide

  6. 6
    image container
    docker
    run

    View Slide

  7. 7
    image container
    docker
    run
    class instance
    new()

    View Slide

  8. Why Docker?
    Isolation!

    View Slide

  9. 9
    server
    /app
    /other-app

    View Slide

  10. 10
    server
    /app
    /other-app
    container
    container

    View Slide

  11. How do I get
    started?
    Install Docker
    Toolkit and write
    your Dockerfile

    View Slide

  12. 12
    https://www.docker.com/products/docker-toolbox

    View Slide

  13. 13
    FROM newrelic/base-builder
    loc-service/Dockerfile

    View Slide

  14. 14
    FROM ruby:2.3
    WORKDIR /data/app
    COPY Gemfile Gemfile.lock /data/app
    RUN bundle install --deployment
    ADD . /data/app
    CMD ["unicorn", "--config", "..."]
    Dockerfile
    ?

    View Slide

  15. Where's the rest
    of my Dockerfile?
    Let's deploy your
    service first!

    View Slide

  16. $terminal
    >>>
    16
    $ docker build -t loc-service .

    View Slide

  17. $terminal
    >>>
    17
    $ docker build -t loc-service .
    Sending build context to Docker daemon
    Step 1 : FROM newrelic/base-builder
    2.3: Pulling from newrelic/base-builder
    ...
    ---> 7ca70eb2dfea
    Step 2 : COPY . /data/app
    ---> 9f54fdf6b489
    Removing intermediate container 669fd221
    ...
    Step 14 : CMD supervisord
    Removing intermediate container 5247f58d
    Successfully built 1773de8387dc
    $

    View Slide

  18. $terminal
    >>>
    18
    $ docker images
    REPOSITORY TAG IMAGE ID CREATED
    loc-service latest sha256:1773d 4 min ago

    View Slide

  19. How do I get the
    image off my
    machine?
    Good question!

    View Slide

  20. 20
    dogestry

    View Slide

  21. How do I get my
    containers
    started?
    Centurion!

    View Slide

  22. $terminal
    >>>
    22
    $ docker run loc-service
    CRIT Supervisor running as root (no user in config file)
    WARN Included extra file "/etc/supervisord.d/app.conf"
    during parsing
    WARN Included extra file "/etc/supervisord.d/nginx.conf"
    during parsing
    WARN Included extra file "/etc/supervisord.d/
    permissions.conf" during parsing
    INFO supervisord started with pid 1
    INFO spawned: 'fix-permissions' with pid 8
    INFO spawned: 'web-unicorn' with pid 9
    INFO spawned: 'nginx' with pid 10
    ...

    View Slide

  23. 23

    View Slide

  24. 24

    View Slide

  25. 25
    namespace :environment do
    desc 'Staging environment'
    task :staging => :common do
    set :image, 'newrelic/loc-service'
    host 'staging-1.nr-internal.net'
    host 'staging-2.nr-internal.net'
    end
    end
    centurion-config/loc-service.rake

    View Slide

  26. 26
    namespace :environment do
    desc 'Staging environment'
    task :staging do
    set :image, 'newrelic/loc-service'
    host 'staging-1.nr-internal.net'
    host 'staging-2.nr-internal.net'
    end
    end
    centurion-config/loc-service.rake

    View Slide

  27. 27
    namespace :environment do
    desc 'Staging environment'
    task :staging do
    set :image, 'newrelic/loc-service'
    host 'staging-1.nr-internal.net'
    host 'staging-2.nr-internal.net'
    end
    end
    centurion-config/loc-service.rake

    View Slide

  28. 28
    namespace :environment do
    desc 'Staging environment'
    task :staging do
    set :image, 'newrelic/loc-service'
    host 'staging-1.nr-internal.net'
    host 'staging-2.nr-internal.net'
    end
    end
    centurion-config/loc-service.rake

    View Slide

  29. $terminal
    >>>
    29
    $ gem install centurion

    View Slide

  30. $terminal
    >>>
    30
    $ centurion -e staging \
    -p loc-service \
    -a deploy

    View Slide

  31. Ready?
    Go!

    View Slide

  32. $terminal
    >>>
    32
    $ centurion -e staging \
    -p loc-service \
    -a deploy
    ...Fetching image loc-service:latest
    ...Using CLI to pull latest
    -- Connecting to staging-1.nr-internal.net
    RESOLVED loc-service:latest => 999cd48e588d
    ** Invoke deploy:verify_image (first_time)
    ** Execute deploy:verify_image
    -- Connecting to staging-1.nr-internal.net
    ....
    Found container up for 6 seconds
    Container is up!

    View Slide

  33. $terminal
    >>>
    33
    $ centurion -e staging \
    -p loc-service \
    -a deploy
    ...Fetching image loc-service:latest
    ...Using CLI to pull latest
    -- Connecting to staging-1.nr-internal.net
    RESOLVED loc-service:latest => 999cd48e588d
    ** Invoke deploy:verify_image (first_time)
    ** Execute deploy:verify_image
    -- Connecting to staging-2.nr-internal.net
    ....
    Found container up for 6 seconds
    Container is up!

    View Slide

  34. $terminal
    >>>
    34
    $ centurion -e staging \
    -p loc-service \
    -a deploy
    ...Fetching image loc-service:latest
    ...Using CLI to pull latest
    -- Connecting to staging-1.nr-internal.net
    RESOLVED loc-service:latest => 999cd48e588d
    ** Invoke deploy:verify_image (first_time)
    ** Execute deploy:verify_image
    -- Connecting to staging-2.nr-internal.net
    ....
    Found container up for 6 seconds
    Container is up!

    View Slide

  35. 35

    View Slide

  36. 36

    View Slide

  37. WHERE'S MY
    APP!?!?!

    View Slide

  38. Yes, but how
    does that help?
    Are your logs
    going to STDOUT?

    View Slide

  39. 39
    # STDOUT or get out!
    Rails.logger = Logger.new(STDOUT)
    config/environments/production.rb

    View Slide

  40. 40

    View Slide

  41. How?
    Let's take a closer
    look on the
    container

    View Slide

  42. $terminal
    >>>
    42
    $ docker -H tcp://staging-1.nr-internal.net:2375 ps
    CONTAINER ID IMAGE COMMAND
    5189764ce084 loc-service "supervisord"

    View Slide

  43. $terminal
    >>>
    43
    $ docker -H tcp://staging-1.nr-internal.net:2375 ps
    CONTAINER ID IMAGE COMMAND
    5189764ce084 loc-service "supervisord"
    $ docker -H tcp://staging-1.nr-internal.net:2375 \
    exec -it 5189764ce084 bash
    [[email protected] /]#

    View Slide

  44. About 300MB
    How much
    memory does your
    app use?

    View Slide

  45. 45
    namespace :environment do
    desc 'Staging environment'
    task :staging do
    set :image, 'newrelic/loc-service'
    memory 2.gigabytes
    host 'staging-1.nr-internal.net'
    host 'staging-2.nr-internal.net'
    end
    end
    centurion-config/loc-service.rake

    View Slide

  46. $terminal
    >>>
    46
    $ docker run -m 2g loc-service

    View Slide

  47. How do I get more
    Unicorn workers?
    Try the ENV!

    View Slide

  48. $terminal
    >>>
    48
    $ docker run -it -e UNICORN_WORKERS=3 loc-service bash
    [[email protected] /]# env
    ...
    UNICORN_WORKERS=3

    View Slide

  49. 49
    ...
    # how many worker processes to start
    worker_processes ENV["UNICORN_WORKERS"].to_i
    unicorn_config.rb

    View Slide

  50. 50
    namespace :environment do
    task :common do
    ...
    env_vars UNICORN_WORKERS: 3
    end
    end
    centurion-config/loc-service.rake

    View Slide

  51. 51
    mysql: &mysql
    encoding: utf8
    adapter:
    username:
    host:
    password:''
    database:
    config/database.yml

    View Slide

  52. 52
    module MyRailsApp
    class Application < Rails::Application
    config.service_url =
    (ENV['SERVICE_URL'] || 'http://...')
    config.service_timeout =
    (ENV['SERVICE_TIMEOUT'] || 5).to_f
    end
    end
    config/application.rb

    View Slide

  53. 53
    class LinesOfCodeController
    def index
    fetch(Rails.configuration.service_url,
    Rails.configuration.service_timeout)
    end
    end
    app/controllers/loc_controller.rb

    View Slide

  54. 54

    View Slide

  55. That one-line
    Dockerfile...
    how's that work?
    Let's take a look!

    View Slide

  56. 56

    View Slide

  57. 57
    FROM newrelic/centos:6
    ...
    RUN rbenv install 2.3.0
    base-builder/Dockerfile

    View Slide

  58. 58
    FROM newrelic/centos:6
    ...
    RUN rbenv install 2.3.0
    # Install supervisord...
    RUN easy_install supervisor
    # Install nginx...
    RUN yum install -y nginx
    base-builder/Dockerfile

    View Slide

  59. 59
    FROM newrelic/centos:6
    ...
    RUN rbenv install 2.3.0
    RUN gem install bundler
    RUN rbenv rehash
    ...
    base-builder/Dockerfile

    View Slide

  60. 60
    FROM newrelic/centos:6
    ...
    RUN rbenv install 2.3.0
    RUN gem install bundler
    RUN rbenv rehash
    RUN bundle install
    ...
    base-builder/Dockerfile

    View Slide

  61. $terminal
    >>>
    61
    $ docker build -t newrelic/base-builder .
    ...
    Step 3 : RUN bundle install
    ---> Running in cb91de794b44
    Could not locate Gemfile

    View Slide

  62. 62
    FROM newrelic/centos:6
    ...
    RUN rbenv install 2.3.0
    RUN gem install bundler
    RUN rbenv rehash
    ONBUILD COPY Gemfile* .
    ONBUILD RUN bundle install
    base-builder/Dockerfile

    View Slide

  63. 63
    FROM newrelic/base-builder
    RUN use-puma
    RUN try-python
    another-app/Dockerfile

    View Slide

  64. 64
    #! /bin/sh
    cp /etc/templates/web.puma.conf \
    /etc/supervisord.d/app.conf
    cp /etc/templates/puma_config.rb \
    /data/app/config/puma_config.rb
    base-builder/usr/bin/use-puma

    View Slide

  65. 65
    FROM newrelic/base-builder
    RUN use-puma
    RUN try-python
    another-app/Dockerfile
    !!!

    View Slide

  66. I'm getting an
    error writing a file
    What's the error?

    View Slide

  67. $terminal
    >>>
    67
    Errno::EACCES: Permission denied @ rb_sysopen
    - /just-a-file-nothing-to-see-here.txt

    View Slide

  68. With who?
    You've got a
    problem with
    nobody

    View Slide

  69. nobody
    69

    View Slide

  70. 70
    [program:web-unicorn]
    command=bundle exec unicorn -c config/unicorn_config.rb
    autostart=true
    autorestart=true
    directory=/data/app
    user=nobody
    stdout_logfile = /dev/stdout
    stdout_logfile_maxbytes = 0
    stderr_logfile = /dev/stderr
    stderr_logfile_maxbytes = 0
    /etc/supervisord.d/web-unicorn.conf

    View Slide

  71. If Docker runs as
    root, why don't
    we?
    Defense in depth!

    View Slide

  72. Can I run my tests
    in Docker?
    There are
    options...

    View Slide

  73. $terminal
    >>>
    73
    $ docker run loc-service bundle exec rake
    /Users/jclark/.rbenv/versions/2.2.4/bin/ruby ...
    Randomized with seed 22351
    .................
    Finished in 0.23058 seconds (files took 2.86 seconds to
    load)
    17 examples, 0 failures
    Randomized with seed 22351
    $

    View Slide

  74. $terminal
    >>>
    74
    $ docker build -t loc-service . && \
    docker run loc-service bundle exec rake
    docker build .
    Sending build context to Docker daemon...
    Step 1 : FROM newrelic/base-builder
    # Executing 12 build triggers...
    ...
    ...
    ...
    /Users/jclark/.rbenv/versions/2.2.4/bin/ruby ...
    ...

    View Slide

  75. $terminal
    >>>
    75
    $ docker build -t loc-service . && \
    docker run loc-service bundle exec rake
    docker build .
    Sending build context to Docker daemon...
    Step 1 : FROM newrelic/base-builder
    # Executing 12 build triggers...
    ...
    ...
    ...
    /Users/jclark/.rbenv/versions/2.2.4/bin/ruby ...
    ...

    View Slide

  76. $terminal
    >>>
    76
    $ docker run -it \
    -v /src/my-app:/test-app \
    loc-image \
    "cd /test-app && bundle exec rake"

    View Slide

  77. I need a Redis
    instance
    We've got just the
    thing

    View Slide

  78. 78
    namespace :environment do
    desc 'Staging environment'
    task :staging do
    set :image, 'newrelic/redis'
    memory 1.gigabytes
    host 'staging-3.nr-internal.net'
    env_vars REDIS_MAXMEMORY: '1G'
    env_vars REDIS_PWD: 'WOULDNTYOULIKETOKNOW'
    end
    centurion-config/loc-redis.rake

    View Slide

  79. 79
    namespace :environment do
    desc 'Staging environment'
    task :staging do
    set :image, 'newrelic/redis'
    memory 1.gigabytes
    host 'staging-3.nr-internal.net'
    env_vars REDIS_MAXMEMORY: '1G'
    env_vars REDIS_PWD: 'WOULDNTYOULIKETOKNOW'
    end
    centurion-config/loc-redis.rake

    View Slide

  80. 80
    namespace :environment do
    desc 'Staging environment'
    task :staging do
    set :image, 'newrelic/redis'
    memory 1.gigabytes
    host 'staging-3.nr-internal.net'
    env_vars REDIS_MAXMEMORY: '1G'
    env_vars REDIS_PWD: 'WOULDNTYOULIKETOKNOW'
    end
    centurion-config/loc-redis.rake

    View Slide

  81. Centurion's cool,
    but what else is
    out there?
    A lot!

    View Slide

  82. 82

    View Slide

  83. 83

    View Slide

  84. 84

    View Slide

  85. deploying with centurion
    ENV-based configuration
    building shared images
    security
    testing
    the future
    thanks!
    @jasonrclark

    View Slide