さようなら Chef こんにちは Dockerfile / Web Tech Tokyo #1

Web Tech Tokyo #1 #web-tech-tokyo の発表資料です
http://web-tech-tokyo.connpass.com/event/40548/

Transcript

1. ͞Α͏ͳΒ$IFG
   ͜Μʹͪ͸%PDLFSpMF
   2016-10-12
   Web Tech Tokyo #1
   Daisuke Fujita (@dtan4)
   https://flic.kr/p/rU5bCU
   

   View Slide

2. Daisuke Fujita / @dtan4
   Engineer, Infrastructure Team

   @ Wantedly, Inc.
   AWS, Terraform / Terraforming, Developer Productivity
   

   View Slide

3. Infrastructure at Wantedly
   AWS + DNSimple
   
   શαʔϏε͕ Docker ίϯςφͰՔಇ
   
   EC2 Πϯελϯεͷ OS ͸ CoreOS
   Capistrano Λ࢖ͬͨಠࣗͷσϓϩΠγεςϜ 

   + Heroku CLI ಉ༷ͷΠϯλʔϑΣΠεΛ࣋ͭ CLI
   https://speakerdeck.com/dtan4/number-abejameetup
   

   View Slide

4. Web (Rails)
   Docker image ͸2ஈߏ੒
   
   ࠩ෼ͷΈͷ fetch + bundle install Ͱ

   σϓϩΠΛߴ଎Խ
   
   intermediate ͸ Chef + Packer
   
   ΠϝʔδαΠζ 3GB ௒
   bundle install
   git clone wantedly/wantedly
   Ruby
   FROM Ubuntu 14.04
   rake assets:precompile
   bundle install
   git checkout
   git fetch
   FROM intermediate
   web (Dockerfile)
   intermediate (Chef)
   

   View Slide

5. Web (Rails)
   Docker image ͸2ஈߏ੒
   
   ࠩ෼ͷΈͷ fetch + bundle install Ͱ

   σϓϩΠΛߴ଎Խ
   
   intermediate ͸ Chef + Packer
   
   ΠϝʔδαΠζ 3GB ௒
   bundle install
   git clone wantedly/wantedly
   Ruby
   FROM Ubuntu 14.04
   rake assets:precompile
   bundle install
   git checkout
   git fetch
   FROM intermediate
   web (Dockerfile)
   intermediate (Chef)
   

   View Slide

6. Chef + Packer
   AWS Ҡߦ౰ॳ͔Β2೥Ҏ্Ҿ͖ܧ͕Ε͖ͯͨൿ఻ͷ Chef Ϩγϐ
   
   Packer Ͱ ubuntu:14.04 Πϝʔδ্ʹϓϩϏδϣχϯά

   => Private Registry ʹ push
   

   View Slide

7. Chef + Packer ???
   ઐ༻ͷ EC2 Πϯελϯε1୆ͷΈͰՔಇ͍ͯͨ͠

   ΋͸΍ਓؒͷ Macbook ͰϓϩϏδϣχϯάͰ͖ͳ͘ͳ͍ͬͯͨ
   
   1Πϝʔδ࡞Δͷʹ1࣌ؒҎ্ Ͱ͖͕͋ͬͨΠϝʔδ͸ 3GB ௒
   
   Ruby ͷόʔδϣϯΞοϓ΍ apt ύοέʔδ௥Ճ͕໘౗
   
   σϓϩΠ࣌ͷ docker build ΍֤Πϯελϯε΁ͷ഑෍͕஗͍
   

   View Slide

8. Chef + Packer => ???
   ΠϝʔδαΠζΛ΋ͬͱখ͍ͨ͘͞͠…
   
   ଞͷϝϯόʔ΋৮Γ΍͍͢ߏ੒ʹ͍ͨ͠…
   
   σϓϩΠ଎͍ͨ͘͠…
   
   Dockerfile Ͱॻ͖͍ͨ…ʂ
   

   View Slide

9. 2016/10/11
   

   View Slide

10. 2016/10/11
    Wantedly, Inc. ͔Β Chef ΛҰ૟
    

    View Slide

11. 2016/10/11
    Chef + Packer -> Dockerfile + Docker Compose
    
    Ubuntu 14.04 -> Alpine Linux 3.4
    Private Docker Registry -> Amazon ECR
    Wantedly, Inc. ͔Β Chef ΛҰ૟
    

    View Slide

12. View Slide

13. Dockerfile
    αʔϏε͝ͱʹ1ຕͷ Dockerfile
    
    build args Ͱ

    - Ruby ͷόʔδϣϯ

    - RAILS_ENV
    
    FROM alpine:3.4
    Dockerfile ϕετϓϥΫςΟε

    Λ࣮ફͯ͠μΠΤοτ
    

    View Slide

14. Dockerfile
    Ruby, Node.js ͷΠϯετʔϧ΋

    ಉ͡ Dockerfile ʹॻ͘
    
    ެࣜͷ -alpine Dockerfile ͔Βഈआ
    
    Alpine ͷόʔδϣϯ΋

    ࣗ෼ͨͪͰ੍ޚ
    
    αʔϏεʹ߹Θͤͨݴޠόʔδϣϯ
    

    View Slide

15. Dockerfile
    λΠϜελϯϓΛૠೖͯ͠

    ҎԼ͸ຖճڧ੍తʹϏϧυ
    
    ιʔείʔυΛ git clone

    Access Token Λ࢖ͬͨೝূ
    
    bundle install
    assets:precompile
    ARG BUILD_DATE
    --build-args BUILD_DATE=$(date +%Y%m%d%H%M%S)
    

    View Slide

16. Docker Compose
    શΠϝʔδ Docker Compose

    ܦ༝Ͱ build & push
    
    build args ΛҰׅ؅ཧ

    Ruby ͷόʔδϣϯΞοϓ΋

    ͔͜͜Β
    awesome:
    extends: base
    build:
    context: awesome
    args:
    - RUBY_MAJOR=2.3
    - RUBY_VERSION=2.3.0
    - RUBYGEMS_VERSION=2.6.6
    - BUNDLER_VERSION=1.13.1
    - NODE_VERSION=5.11.1
    - NPM_VERSION=3.3.12
    awesome-production:
    extends: awesome
    build:
    args:
    - RAILS_ENV=production
    image: ${AWS_ECR_DOMAIN}/awesome:intermediate
    

    View Slide

17. View Slide

18. intermediate-build-server
    intermediate image Λఆظతʹ build ͢Δαʔό
    
    CoreOS on EC2 + IAM Role
    
    ຖ೔ਂ໷ʹશ intermediate image Λ build -> push
    

    View Slide

19. intermediate-build-server
    #!/bin/bash
    set -eu
    metadata=$(curl -s http://169.254.169.254/latest/meta-data/iam/security-credentials/intermediate-build-server/)
    export AWS_ACCESS_KEY_ID=$(echo $metadata | jq -r .AccessKeyId)
    export AWS_SECRET_ACCESS_KEY=$(echo $metadata | jq -r .SecretAccessKey)
    export AWS_SECURITY_TOKEN=$(echo $metadata | jq -r .Token)
    document=$(curl -s http://169.254.169.254/latest/dynamic/instance-identity/document)
    account_id=$(echo $document | jq -r .accountId)
    region=$(echo $document | jq -r .region)
    export AWS_ECR_DOMAIN=$account_id.dkr.ecr.$region.amazonaws.com
    export BUILD_DATE=$(date +%Y%m%d%H%M%S)
    cd intermediate-dockerfiles
    if [[ $# -eq 1 ]]; then
    /opt/bin/docker-compose build $1
    else
    for app in $(cat docker-compose.yml | grep -E '^ [a-z-]+:' | grep -E -- '-(production|qa)' | sed -E 's/^ ([a-z-]*):/\1/'); do
    echo "[`date`] Building $app..."
    /opt/bin/docker-compose build $app
    done
    fi
    

    View Slide

20. intermediate-build-server
    ຊ౰͸ CI as a Service ্Ͱ΍Γ͔ͨͬͨ

    image pull -> build -> push
    
    Docker 1.10 ͔Β͸ Docker image ͷ࢓૊Έ͕มΘͬͯɺ

    pull ͖ͯͨ͠Πϝʔδͷʮ్த·ͰʯΩϟογϡͱͯ͠

    ࢖͏͜ͱ͕Ͱ͖ͳ͘ͳͬͨ
    
    https://docs.docker.com/engine/userguide/storagedriver/
    imagesandcontainers/#/content-addressable-storage
    

    View Slide

21. View Slide

22. intermediate-build-server
    

    View Slide

23. Ҡߦ݁Ռ
    

    View Slide

24. Ҡߦ݁Ռ
    ΠϝʔδαΠζ 3.307 GB -> 2.24 GB
    
    Ϗϧυ࣌ؒ 48 m -> 16 m 5 s
    

    View Slide

25. TODO
    Serverspec ॻ͍͚ͨͲ CI ͕ͳ͍
    
    ECR ʹ࢒ͬͨΰϛΠϝʔδͷ૟আ
    
    shallow clone ͰΠϝʔδΛ͞ΒʹμΠΤοτ
    
    σϓϩΠύΠϓϥΠϯΛݟ௚ͯ͠σϓϩΠߴ଎Խ
    
    ͱ͔ͱ͔…
    

    View Slide

26. To be continued...
    http://chibadan.connpass.com/event/41625/
    

    View Slide

27. To be continued...
    http://chibadan.connpass.com/event/41625/
    

    View Slide

28. Recap
    

    View Slide

29. View Slide