Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Real World Docker for the Rubyist

Real World Docker for the Rubyist

Docker’s gotten a lot of press, but how does it fare in the real world Rubyists inhabit every day?

Together we’ll take a deep dive into how a real company transformed itself to run on Docker. We’ll see how to build and maintain Docker images tailored for Ruby. We’ll dig into proper configuration and deployment options for containerized applications. Along the way we’ll highlight the pitfalls, bugs and gotchas that come with such a young, fast moving platform like Docker.

Whether you’re in production with Docker or just dabbling, come learn how Docker and Ruby make an awesome combination.

92e7389893670a1920a4fd98aec0d246?s=128

Jason R Clark

May 06, 2016
Tweet

Transcript

  1. Real World Docker for the Rubyist @jasonrclark

  2. I'm Jane, a new New Relic with a cool new

    feature to deploy! I'm Jill, an experienced Relic, ready to help
  3. This document and the information herein (including any information that

    may be incorporated by reference) is provided for informational purposes only and should not be construed as an offer, commitment, promise or obligation on behalf of New Relic, Inc. (“New Relic”) to sell securities or deliver any product, material, code, functionality, or other feature. Any information provided hereby is proprietary to New Relic and may not be replicated or disclosed without New Relic’s express written permission. Such information may contain forward-looking statements within the meaning of federal securities laws. Any statement that is not a historical fact or refers to expectations, projections, future plans, objectives, estimates, goals, or other characterizations of future events is a forward-looking statement. These forward-looking statements can often be identified as such because the context of the statement will include words such as “believes,” “anticipates,” “expects” or words of similar import. Actual results may differ materially from those expressed in these forward-looking statements, which speak only as of the date hereof, and are subject to change at any time without notice. Existing and prospective investors, customers and other third parties transacting business with New Relic are cautioned not to place undue reliance on this forward-looking information. The achievement or success of the matters covered by such forward-looking statements are based on New Relic’s current assumptions, expectations, and beliefs and are subject to substantial risks, uncertainties, assumptions, and changes in circumstances that may cause the actual results, performance, or achievements to differ materially from those expressed or implied in any forward-looking statement. Further information on factors that could affect such forward-looking statements is included in the filings we make with the SEC from time to time. Copies of these documents may be obtained by visiting New Relic’s Investor Relations website at ir.newrelic.com or the SEC’s website at www.sec.gov. New Relic assumes no obligation and does not intend to update these forward-looking statements, except as required by law. New Relic makes no warranties, expressed or implied, in this document or otherwise, with respect to the information provided.
  4. Why Docker? Packaging!

  5. 5

  6. 6 image container docker run

  7. 7 image container docker run class instance new()

  8. Why Docker? Isolation!

  9. 9 server /app /other-app

  10. 10 server /app /other-app container container

  11. How do I get started? Install Docker Toolkit and write

    your Dockerfile
  12. 12 https://www.docker.com/products/docker-toolbox

  13. 13 FROM newrelic/base-builder loc-service/Dockerfile

  14. 14 FROM ruby:2.3 WORKDIR /data/app COPY Gemfile Gemfile.lock /data/app RUN

    bundle install --deployment ADD . /data/app CMD ["unicorn", "--config", "..."] Dockerfile ?
  15. Where's the rest of my Dockerfile? Let's deploy your service

    first!
  16. $terminal >>> 16 $ docker build -t loc-service .

  17. $terminal >>> 17 $ docker build -t loc-service . Sending

    build context to Docker daemon Step 1 : FROM newrelic/base-builder 2.3: Pulling from newrelic/base-builder ... ---> 7ca70eb2dfea Step 2 : COPY . /data/app ---> 9f54fdf6b489 Removing intermediate container 669fd221 ... Step 14 : CMD supervisord Removing intermediate container 5247f58d Successfully built 1773de8387dc $
  18. $terminal >>> 18 $ docker images REPOSITORY TAG IMAGE ID

    CREATED loc-service latest sha256:1773d 4 min ago
  19. How do I get the image off my machine? Good

    question!
  20. 20 dogestry

  21. How do I get my containers started? Centurion!

  22. $terminal >>> 22 $ docker run loc-service CRIT Supervisor running

    as root (no user in config file) WARN Included extra file "/etc/supervisord.d/app.conf" during parsing WARN Included extra file "/etc/supervisord.d/nginx.conf" during parsing WARN Included extra file "/etc/supervisord.d/ permissions.conf" during parsing INFO supervisord started with pid 1 INFO spawned: 'fix-permissions' with pid 8 INFO spawned: 'web-unicorn' with pid 9 INFO spawned: 'nginx' with pid 10 ...
  23. 23

  24. 24

  25. 25 namespace :environment do desc 'Staging environment' task :staging =>

    :common do set :image, 'newrelic/loc-service' host 'staging-1.nr-internal.net' host 'staging-2.nr-internal.net' end end centurion-config/loc-service.rake
  26. 26 namespace :environment do desc 'Staging environment' task :staging do

    set :image, 'newrelic/loc-service' host 'staging-1.nr-internal.net' host 'staging-2.nr-internal.net' end end centurion-config/loc-service.rake
  27. 27 namespace :environment do desc 'Staging environment' task :staging do

    set :image, 'newrelic/loc-service' host 'staging-1.nr-internal.net' host 'staging-2.nr-internal.net' end end centurion-config/loc-service.rake
  28. 28 namespace :environment do desc 'Staging environment' task :staging do

    set :image, 'newrelic/loc-service' host 'staging-1.nr-internal.net' host 'staging-2.nr-internal.net' end end centurion-config/loc-service.rake
  29. $terminal >>> 29 $ gem install centurion

  30. $terminal >>> 30 $ centurion -e staging \ -p loc-service

    \ -a deploy
  31. Ready? Go!

  32. $terminal >>> 32 $ centurion -e staging \ -p loc-service

    \ -a deploy ...Fetching image loc-service:latest ...Using CLI to pull latest -- Connecting to staging-1.nr-internal.net RESOLVED loc-service:latest => 999cd48e588d ** Invoke deploy:verify_image (first_time) ** Execute deploy:verify_image -- Connecting to staging-1.nr-internal.net .... Found container up for 6 seconds Container is up!
  33. $terminal >>> 33 $ centurion -e staging \ -p loc-service

    \ -a deploy ...Fetching image loc-service:latest ...Using CLI to pull latest -- Connecting to staging-1.nr-internal.net RESOLVED loc-service:latest => 999cd48e588d ** Invoke deploy:verify_image (first_time) ** Execute deploy:verify_image -- Connecting to staging-2.nr-internal.net .... Found container up for 6 seconds Container is up!
  34. $terminal >>> 34 $ centurion -e staging \ -p loc-service

    \ -a deploy ...Fetching image loc-service:latest ...Using CLI to pull latest -- Connecting to staging-1.nr-internal.net RESOLVED loc-service:latest => 999cd48e588d ** Invoke deploy:verify_image (first_time) ** Execute deploy:verify_image -- Connecting to staging-2.nr-internal.net .... Found container up for 6 seconds Container is up!
  35. 35

  36. 36

  37. WHERE'S MY APP!?!?!

  38. Yes, but how does that help? Are your logs going

    to STDOUT?
  39. 39 # STDOUT or get out! Rails.logger = Logger.new(STDOUT) config/environments/production.rb

  40. 40

  41. How? Let's take a closer look on the container

  42. $terminal >>> 42 $ docker -H tcp://staging-1.nr-internal.net:2375 ps CONTAINER ID

    IMAGE COMMAND 5189764ce084 loc-service "supervisord"
  43. $terminal >>> 43 $ docker -H tcp://staging-1.nr-internal.net:2375 ps CONTAINER ID

    IMAGE COMMAND 5189764ce084 loc-service "supervisord" $ docker -H tcp://staging-1.nr-internal.net:2375 \ exec -it 5189764ce084 bash [root@5189764ce084 /]#
  44. About 300MB How much memory does your app use?

  45. 45 namespace :environment do desc 'Staging environment' task :staging do

    set :image, 'newrelic/loc-service' memory 2.gigabytes host 'staging-1.nr-internal.net' host 'staging-2.nr-internal.net' end end centurion-config/loc-service.rake
  46. $terminal >>> 46 $ docker run -m 2g loc-service

  47. How do I get more Unicorn workers? Try the ENV!

  48. $terminal >>> 48 $ docker run -it -e UNICORN_WORKERS=3 loc-service

    bash [root@5189764ce084 /]# env ... UNICORN_WORKERS=3
  49. 49 ... # how many worker processes to start worker_processes

    ENV["UNICORN_WORKERS"].to_i unicorn_config.rb
  50. 50 namespace :environment do task :common do ... env_vars UNICORN_WORKERS:

    3 end end centurion-config/loc-service.rake
  51. 51 mysql: &mysql encoding: utf8 adapter: <%=ENV['DB_ADAPTER']||'mysql2'%> username:<%=ENV['DB_USERNAME']||'root'%> host: <%=ENV['DB_HOST']||'localhost'%>

    password:'<%=ENV['DB_PASSWORD']%>' database:<%=ENV['DB_SCHEMA']%> config/database.yml
  52. 52 module MyRailsApp class Application < Rails::Application config.service_url = (ENV['SERVICE_URL']

    || 'http://...') config.service_timeout = (ENV['SERVICE_TIMEOUT'] || 5).to_f end end config/application.rb
  53. 53 class LinesOfCodeController def index fetch(Rails.configuration.service_url, Rails.configuration.service_timeout) end end app/controllers/loc_controller.rb

  54. 54

  55. That one-line Dockerfile... how's that work? Let's take a look!

  56. 56

  57. 57 FROM newrelic/centos:6 ... RUN rbenv install 2.3.0 base-builder/Dockerfile

  58. 58 FROM newrelic/centos:6 ... RUN rbenv install 2.3.0 # Install

    supervisord... RUN easy_install supervisor # Install nginx... RUN yum install -y nginx base-builder/Dockerfile
  59. 59 FROM newrelic/centos:6 ... RUN rbenv install 2.3.0 RUN gem

    install bundler RUN rbenv rehash ... base-builder/Dockerfile
  60. 60 FROM newrelic/centos:6 ... RUN rbenv install 2.3.0 RUN gem

    install bundler RUN rbenv rehash RUN bundle install ... base-builder/Dockerfile
  61. $terminal >>> 61 $ docker build -t newrelic/base-builder . ...

    Step 3 : RUN bundle install ---> Running in cb91de794b44 Could not locate Gemfile
  62. 62 FROM newrelic/centos:6 ... RUN rbenv install 2.3.0 RUN gem

    install bundler RUN rbenv rehash ONBUILD COPY Gemfile* . ONBUILD RUN bundle install base-builder/Dockerfile
  63. 63 FROM newrelic/base-builder RUN use-puma RUN try-python another-app/Dockerfile

  64. 64 #! /bin/sh cp /etc/templates/web.puma.conf \ /etc/supervisord.d/app.conf cp /etc/templates/puma_config.rb \

    /data/app/config/puma_config.rb base-builder/usr/bin/use-puma
  65. 65 FROM newrelic/base-builder RUN use-puma RUN try-python another-app/Dockerfile !!!

  66. I'm getting an error writing a file What's the error?

  67. $terminal >>> 67 Errno::EACCES: Permission denied @ rb_sysopen - /just-a-file-nothing-to-see-here.txt

  68. With who? You've got a problem with nobody

  69. nobody 69

  70. 70 [program:web-unicorn] command=bundle exec unicorn -c config/unicorn_config.rb autostart=true autorestart=true directory=/data/app

    user=nobody stdout_logfile = /dev/stdout stdout_logfile_maxbytes = 0 stderr_logfile = /dev/stderr stderr_logfile_maxbytes = 0 /etc/supervisord.d/web-unicorn.conf
  71. If Docker runs as root, why don't we? Defense in

    depth!
  72. Can I run my tests in Docker? There are options...

  73. $terminal >>> 73 $ docker run loc-service bundle exec rake

    /Users/jclark/.rbenv/versions/2.2.4/bin/ruby ... Randomized with seed 22351 ................. Finished in 0.23058 seconds (files took 2.86 seconds to load) 17 examples, 0 failures Randomized with seed 22351 $
  74. $terminal >>> 74 $ docker build -t loc-service . &&

    \ docker run loc-service bundle exec rake docker build . Sending build context to Docker daemon... Step 1 : FROM newrelic/base-builder # Executing 12 build triggers... ... ... ... /Users/jclark/.rbenv/versions/2.2.4/bin/ruby ... ...
  75. $terminal >>> 75 $ docker build -t loc-service . &&

    \ docker run loc-service bundle exec rake docker build . Sending build context to Docker daemon... Step 1 : FROM newrelic/base-builder # Executing 12 build triggers... ... ... ... /Users/jclark/.rbenv/versions/2.2.4/bin/ruby ... ...
  76. $terminal >>> 76 $ docker run -it \ -v /src/my-app:/test-app

    \ loc-image \ "cd /test-app && bundle exec rake"
  77. I need a Redis instance We've got just the thing

  78. 78 namespace :environment do desc 'Staging environment' task :staging do

    set :image, 'newrelic/redis' memory 1.gigabytes host 'staging-3.nr-internal.net' env_vars REDIS_MAXMEMORY: '1G' env_vars REDIS_PWD: 'WOULDNTYOULIKETOKNOW' end centurion-config/loc-redis.rake
  79. 79 namespace :environment do desc 'Staging environment' task :staging do

    set :image, 'newrelic/redis' memory 1.gigabytes host 'staging-3.nr-internal.net' env_vars REDIS_MAXMEMORY: '1G' env_vars REDIS_PWD: 'WOULDNTYOULIKETOKNOW' end centurion-config/loc-redis.rake
  80. 80 namespace :environment do desc 'Staging environment' task :staging do

    set :image, 'newrelic/redis' memory 1.gigabytes host 'staging-3.nr-internal.net' env_vars REDIS_MAXMEMORY: '1G' env_vars REDIS_PWD: 'WOULDNTYOULIKETOKNOW' end centurion-config/loc-redis.rake
  81. Centurion's cool, but what else is out there? A lot!

  82. 82

  83. 83

  84. 84

  85. deploying with centurion ENV-based configuration building shared images security testing

    the future thanks! @jasonrclark