Real World Docker for the Rubyist

Docker’s gotten a lot of press, but how does it fare in the real world Rubyists inhabit every day?

Together we’ll take a deep dive into how a real company transformed itself to run on Docker. We’ll see how to build and maintain Docker images tailored for Ruby. We’ll dig into proper configuration and deployment options for containerized applications. Along the way we’ll highlight the pitfalls, bugs and gotchas that come with such a young, fast moving platform like Docker.

Whether you’re in production with Docker or just dabbling, come learn how Docker and Ruby make an awesome combination.

Transcript

1. Real World Docker
   for the Rubyist
   @jasonrclark
   

   View Slide

2. I'm Jane, a new
   New Relic with a
   cool new feature
   to deploy!
   I'm Jill, an
   experienced Relic,
   ready to help
   

   View Slide

3. This document and the information herein (including any information that may be incorporated by reference) is provided for
   informational purposes only and should not be construed as an offer, commitment, promise or obligation on behalf of New Relic, Inc.
   (“New Relic”) to sell securities or deliver any product, material, code, functionality, or other feature. Any information provided hereby
   is proprietary to New Relic and may not be replicated or disclosed without New Relic’s express written permission.
   Such information may contain forward-looking statements within the meaning of federal securities laws. Any statement that is not a
   historical fact or refers to expectations, projections, future plans, objectives, estimates, goals, or other characterizations of future
   events is a forward-looking statement. These forward-looking statements can often be identified as such because the context of the
   statement will include words such as “believes,” “anticipates,” “expects” or words of similar import.
   Actual results may differ materially from those expressed in these forward-looking statements, which speak only as of the date
   hereof, and are subject to change at any time without notice. Existing and prospective investors, customers and other third parties
   transacting business with New Relic are cautioned not to place undue reliance on this forward-looking information. The achievement
   or success of the matters covered by such forward-looking statements are based on New Relic’s current assumptions, expectations,
   and beliefs and are subject to substantial risks, uncertainties, assumptions, and changes in circumstances that may cause the actual
   results, performance, or achievements to differ materially from those expressed or implied in any forward-looking statement. Further
   information on factors that could affect such forward-looking statements is included in the filings we make with the SEC from time to
   time. Copies of these documents may be obtained by visiting New Relic’s Investor Relations website at ir.newrelic.com or the SEC’s
   website at www.sec.gov.
   New Relic assumes no obligation and does not intend to update these forward-looking statements, except as required by law. New
   Relic makes no warranties, expressed or implied, in this document or otherwise, with respect to the information provided.
   

   View Slide

4. Why Docker?
   Packaging!
   

   View Slide

5. View Slide

6. 6
   image container
   docker
   run
   

   View Slide

7. 7
   image container
   docker
   run
   class instance
   new()
   

   View Slide

8. Why Docker?
   Isolation!
   

   View Slide

9. 9
   server
   /app
   /other-app
   

   View Slide

10. 10
    server
    /app
    /other-app
    container
    container
    

    View Slide

11. How do I get
    started?
    Install Docker
    Toolkit and write
    your Dockerfile
    

    View Slide

12. 12
    https://www.docker.com/products/docker-toolbox
    

    View Slide

13. 13
    FROM newrelic/base-builder
    loc-service/Dockerfile
    

    View Slide

14. 14
    FROM ruby:2.3
    WORKDIR /data/app
    COPY Gemfile Gemfile.lock /data/app
    RUN bundle install --deployment
    ADD . /data/app
    CMD ["unicorn", "--config", "..."]
    Dockerfile
    ?
    

    View Slide

15. Where's the rest
    of my Dockerfile?
    Let's deploy your
    service first!
    

    View Slide

16. $terminal
    >>>
    16
    $ docker build -t loc-service .
    

    View Slide

17. $terminal
    >>>
    17
    $ docker build -t loc-service .
    Sending build context to Docker daemon
    Step 1 : FROM newrelic/base-builder
    2.3: Pulling from newrelic/base-builder
    ...
    ---> 7ca70eb2dfea
    Step 2 : COPY . /data/app
    ---> 9f54fdf6b489
    Removing intermediate container 669fd221
    ...
    Step 14 : CMD supervisord
    Removing intermediate container 5247f58d
    Successfully built 1773de8387dc
    $
    

    View Slide

18. $terminal
    >>>
    18
    $ docker images
    REPOSITORY TAG IMAGE ID CREATED
    loc-service latest sha256:1773d 4 min ago
    

    View Slide

19. How do I get the
    image off my
    machine?
    Good question!
    

    View Slide

20. 20
    dogestry
    

    View Slide

21. How do I get my
    containers
    started?
    Centurion!
    

    View Slide

22. $terminal
    >>>
    22
    $ docker run loc-service
    CRIT Supervisor running as root (no user in config file)
    WARN Included extra file "/etc/supervisord.d/app.conf"
    during parsing
    WARN Included extra file "/etc/supervisord.d/nginx.conf"
    during parsing
    WARN Included extra file "/etc/supervisord.d/
    permissions.conf" during parsing
    INFO supervisord started with pid 1
    INFO spawned: 'fix-permissions' with pid 8
    INFO spawned: 'web-unicorn' with pid 9
    INFO spawned: 'nginx' with pid 10
    ...
    

    View Slide

23. View Slide

24. View Slide

25. 25
    namespace :environment do
    desc 'Staging environment'
    task :staging => :common do
    set :image, 'newrelic/loc-service'
    host 'staging-1.nr-internal.net'
    host 'staging-2.nr-internal.net'
    end
    end
    centurion-config/loc-service.rake
    

    View Slide

26. 26
    namespace :environment do
    desc 'Staging environment'
    task :staging do
    set :image, 'newrelic/loc-service'
    host 'staging-1.nr-internal.net'
    host 'staging-2.nr-internal.net'
    end
    end
    centurion-config/loc-service.rake
    

    View Slide

27. 27
    namespace :environment do
    desc 'Staging environment'
    task :staging do
    set :image, 'newrelic/loc-service'
    host 'staging-1.nr-internal.net'
    host 'staging-2.nr-internal.net'
    end
    end
    centurion-config/loc-service.rake
    

    View Slide

28. 28
    namespace :environment do
    desc 'Staging environment'
    task :staging do
    set :image, 'newrelic/loc-service'
    host 'staging-1.nr-internal.net'
    host 'staging-2.nr-internal.net'
    end
    end
    centurion-config/loc-service.rake
    

    View Slide

29. $terminal
    >>>
    29
    $ gem install centurion
    

    View Slide

30. $terminal
    >>>
    30
    $ centurion -e staging \
    -p loc-service \
    -a deploy
    

    View Slide

31. Ready?
    Go!
    

    View Slide

32. $terminal
    >>>
    32
    $ centurion -e staging \
    -p loc-service \
    -a deploy
    ...Fetching image loc-service:latest
    ...Using CLI to pull latest
    -- Connecting to staging-1.nr-internal.net
    RESOLVED loc-service:latest => 999cd48e588d
    ** Invoke deploy:verify_image (first_time)
    ** Execute deploy:verify_image
    -- Connecting to staging-1.nr-internal.net
    ....
    Found container up for 6 seconds
    Container is up!
    

    View Slide

33. $terminal
    >>>
    33
    $ centurion -e staging \
    -p loc-service \
    -a deploy
    ...Fetching image loc-service:latest
    ...Using CLI to pull latest
    -- Connecting to staging-1.nr-internal.net
    RESOLVED loc-service:latest => 999cd48e588d
    ** Invoke deploy:verify_image (first_time)
    ** Execute deploy:verify_image
    -- Connecting to staging-2.nr-internal.net
    ....
    Found container up for 6 seconds
    Container is up!
    

    View Slide

34. $terminal
    >>>
    34
    $ centurion -e staging \
    -p loc-service \
    -a deploy
    ...Fetching image loc-service:latest
    ...Using CLI to pull latest
    -- Connecting to staging-1.nr-internal.net
    RESOLVED loc-service:latest => 999cd48e588d
    ** Invoke deploy:verify_image (first_time)
    ** Execute deploy:verify_image
    -- Connecting to staging-2.nr-internal.net
    ....
    Found container up for 6 seconds
    Container is up!
    

    View Slide

35. View Slide

36. View Slide

37. WHERE'S MY
    APP!?!?!
    

    View Slide

38. Yes, but how
    does that help?
    Are your logs
    going to STDOUT?
    

    View Slide

39. 39
    # STDOUT or get out!
    Rails.logger = Logger.new(STDOUT)
    config/environments/production.rb
    

    View Slide

40. View Slide

41. How?
    Let's take a closer
    look on the
    container
    

    View Slide

42. $terminal
    >>>
    42
    $ docker -H tcp://staging-1.nr-internal.net:2375 ps
    CONTAINER ID IMAGE COMMAND
    5189764ce084 loc-service "supervisord"
    

    View Slide

43. $terminal
    >>>
    43
    $ docker -H tcp://staging-1.nr-internal.net:2375 ps
    CONTAINER ID IMAGE COMMAND
    5189764ce084 loc-service "supervisord"
    $ docker -H tcp://staging-1.nr-internal.net:2375 \
    exec -it 5189764ce084 bash
    [root@5189764ce084 /]#
    

    View Slide

44. About 300MB
    How much
    memory does your
    app use?
    

    View Slide

45. 45
    namespace :environment do
    desc 'Staging environment'
    task :staging do
    set :image, 'newrelic/loc-service'
    memory 2.gigabytes
    host 'staging-1.nr-internal.net'
    host 'staging-2.nr-internal.net'
    end
    end
    centurion-config/loc-service.rake
    

    View Slide

46. $terminal
    >>>
    46
    $ docker run -m 2g loc-service
    

    View Slide

47. How do I get more
    Unicorn workers?
    Try the ENV!
    

    View Slide

48. $terminal
    >>>
    48
    $ docker run -it -e UNICORN_WORKERS=3 loc-service bash
    [root@5189764ce084 /]# env
    ...
    UNICORN_WORKERS=3
    

    View Slide

49. 49
    ...
    # how many worker processes to start
    worker_processes ENV["UNICORN_WORKERS"].to_i
    unicorn_config.rb
    

    View Slide

50. 50
    namespace :environment do
    task :common do
    ...
    env_vars UNICORN_WORKERS: 3
    end
    end
    centurion-config/loc-service.rake
    

    View Slide

51. 51
    mysql: &mysql
    encoding: utf8
    adapter:
    username:
    host:
    password:''
    database:
    config/database.yml
    

    View Slide

52. 52
    module MyRailsApp
    class Application < Rails::Application
    config.service_url =
    (ENV['SERVICE_URL'] || 'http://...')
    config.service_timeout =
    (ENV['SERVICE_TIMEOUT'] || 5).to_f
    end
    end
    config/application.rb
    

    View Slide

53. 53
    class LinesOfCodeController
    def index
    fetch(Rails.configuration.service_url,
    Rails.configuration.service_timeout)
    end
    end
    app/controllers/loc_controller.rb
    

    View Slide

54. View Slide

55. That one-line
    Dockerfile...
    how's that work?
    Let's take a look!
    

    View Slide

56. View Slide

57. 57
    FROM newrelic/centos:6
    ...
    RUN rbenv install 2.3.0
    base-builder/Dockerfile
    

    View Slide

58. 58
    FROM newrelic/centos:6
    ...
    RUN rbenv install 2.3.0
    # Install supervisord...
    RUN easy_install supervisor
    # Install nginx...
    RUN yum install -y nginx
    base-builder/Dockerfile
    

    View Slide

59. 59
    FROM newrelic/centos:6
    ...
    RUN rbenv install 2.3.0
    RUN gem install bundler
    RUN rbenv rehash
    ...
    base-builder/Dockerfile
    

    View Slide

60. 60
    FROM newrelic/centos:6
    ...
    RUN rbenv install 2.3.0
    RUN gem install bundler
    RUN rbenv rehash
    RUN bundle install
    ...
    base-builder/Dockerfile
    

    View Slide

61. $terminal
    >>>
    61
    $ docker build -t newrelic/base-builder .
    ...
    Step 3 : RUN bundle install
    ---> Running in cb91de794b44
    Could not locate Gemfile
    

    View Slide

62. 62
    FROM newrelic/centos:6
    ...
    RUN rbenv install 2.3.0
    RUN gem install bundler
    RUN rbenv rehash
    ONBUILD COPY Gemfile* .
    ONBUILD RUN bundle install
    base-builder/Dockerfile
    

    View Slide

63. 63
    FROM newrelic/base-builder
    RUN use-puma
    RUN try-python
    another-app/Dockerfile
    

    View Slide

64. 64
    #! /bin/sh
    cp /etc/templates/web.puma.conf \
    /etc/supervisord.d/app.conf
    cp /etc/templates/puma_config.rb \
    /data/app/config/puma_config.rb
    base-builder/usr/bin/use-puma
    

    View Slide

65. 65
    FROM newrelic/base-builder
    RUN use-puma
    RUN try-python
    another-app/Dockerfile
    !!!
    

    View Slide

66. I'm getting an
    error writing a file
    What's the error?
    

    View Slide

67. $terminal
    >>>
    67
    Errno::EACCES: Permission denied @ rb_sysopen
    - /just-a-file-nothing-to-see-here.txt
    

    View Slide

68. With who?
    You've got a
    problem with
    nobody
    

    View Slide

69. nobody
    69
    

    View Slide

70. 70
    [program:web-unicorn]
    command=bundle exec unicorn -c config/unicorn_config.rb
    autostart=true
    autorestart=true
    directory=/data/app
    user=nobody
    stdout_logfile = /dev/stdout
    stdout_logfile_maxbytes = 0
    stderr_logfile = /dev/stderr
    stderr_logfile_maxbytes = 0
    /etc/supervisord.d/web-unicorn.conf
    

    View Slide

71. If Docker runs as
    root, why don't
    we?
    Defense in depth!
    

    View Slide

72. Can I run my tests
    in Docker?
    There are
    options...
    

    View Slide

73. $terminal
    >>>
    73
    $ docker run loc-service bundle exec rake
    /Users/jclark/.rbenv/versions/2.2.4/bin/ruby ...
    Randomized with seed 22351
    .................
    Finished in 0.23058 seconds (files took 2.86 seconds to
    load)
    17 examples, 0 failures
    Randomized with seed 22351
    $
    

    View Slide

74. $terminal
    >>>
    74
    $ docker build -t loc-service . && \
    docker run loc-service bundle exec rake
    docker build .
    Sending build context to Docker daemon...
    Step 1 : FROM newrelic/base-builder
    # Executing 12 build triggers...
    ...
    ...
    ...
    /Users/jclark/.rbenv/versions/2.2.4/bin/ruby ...
    ...
    

    View Slide

75. $terminal
    >>>
    75
    $ docker build -t loc-service . && \
    docker run loc-service bundle exec rake
    docker build .
    Sending build context to Docker daemon...
    Step 1 : FROM newrelic/base-builder
    # Executing 12 build triggers...
    ...
    ...
    ...
    /Users/jclark/.rbenv/versions/2.2.4/bin/ruby ...
    ...
    

    View Slide

76. $terminal
    >>>
    76
    $ docker run -it \
    -v /src/my-app:/test-app \
    loc-image \
    "cd /test-app && bundle exec rake"
    

    View Slide

77. I need a Redis
    instance
    We've got just the
    thing
    

    View Slide

78. 78
    namespace :environment do
    desc 'Staging environment'
    task :staging do
    set :image, 'newrelic/redis'
    memory 1.gigabytes
    host 'staging-3.nr-internal.net'
    env_vars REDIS_MAXMEMORY: '1G'
    env_vars REDIS_PWD: 'WOULDNTYOULIKETOKNOW'
    end
    centurion-config/loc-redis.rake
    

    View Slide

79. 79
    namespace :environment do
    desc 'Staging environment'
    task :staging do
    set :image, 'newrelic/redis'
    memory 1.gigabytes
    host 'staging-3.nr-internal.net'
    env_vars REDIS_MAXMEMORY: '1G'
    env_vars REDIS_PWD: 'WOULDNTYOULIKETOKNOW'
    end
    centurion-config/loc-redis.rake
    

    View Slide

80. 80
    namespace :environment do
    desc 'Staging environment'
    task :staging do
    set :image, 'newrelic/redis'
    memory 1.gigabytes
    host 'staging-3.nr-internal.net'
    env_vars REDIS_MAXMEMORY: '1G'
    env_vars REDIS_PWD: 'WOULDNTYOULIKETOKNOW'
    end
    centurion-config/loc-redis.rake
    

    View Slide

81. Centurion's cool,
    but what else is
    out there?
    A lot!
    

    View Slide

82. View Slide

83. View Slide

84. View Slide

85. deploying with centurion
    ENV-based configuration
    building shared images
    security
    testing
    the future
    thanks!
    @jasonrclark
    

    View Slide