better docker image

6247c099ad62bf727a4f9df10b6c6f23?s=47 orisano
October 09, 2018
Technology
21
19k

better docker image

6247c099ad62bf727a4f9df10b6c6f23?s=128

orisano

October 09, 2018
Tweet

More Decks by orisano

See All by orisano
6247c099ad62bf727a4f9df10b6c6f23?s=48 orisano
7
3.3k
6247c099ad62bf727a4f9df10b6c6f23?s=48 orisano
3
440
6247c099ad62bf727a4f9df10b6c6f23?s=48 orisano
2
1k
6247c099ad62bf727a4f9df10b6c6f23?s=48 orisano
2
5k
6247c099ad62bf727a4f9df10b6c6f23?s=48 orisano
5
2.8k
6247c099ad62bf727a4f9df10b6c6f23?s=48 orisano
0
720
6247c099ad62bf727a4f9df10b6c6f23?s=48 orisano
11
5.6k
6247c099ad62bf727a4f9df10b6c6f23?s=48 orisano
5
2.8k
6247c099ad62bf727a4f9df10b6c6f23?s=48 orisano
7
1k

Other Decks in Technology

See All in Technology
F80a051bbccca2f8581c42f58223f90f?s=48 futayamaji
0
120
703c34db2db0e609fddec6a5585ceccc?s=48 kenta_sato3
1
220
A645e7c3a6635ead8fa323c583769591?s=48 hololab
0
310
9888b1cb1b2a5d93095bbd98daf2efa5?s=48 emi0726
0
230
229b1596ce57cd0935a2bacd410d87a0?s=48 aaaddress1
1
1.7k
881b1b57f1cbe3b66b017203f7cb2efb?s=48 nplusone
0
250
835671ae91e49b2637313b91dacda1db?s=48 tdys13
2
1.9k
893dd6e1aa73c35621327c385cb27207?s=48 konnnnnok
0
130
B1cc148711c6a37a5c922b6e72a4ad52?s=48 upura
1
960
53850955f15249a1a9dc49df6113e400?s=48 line_developers
0
140
54bcc1069d75636cc735e72877dd30ec?s=48 taichinakamura
0
240
75fd80a8ace6ab292a7de563e018ae11?s=48 takashi_toyosaki
0
250

Featured

See All Featured
Dafc4723e9a1c067796c0fec6f509774?s=48 lemiorhan
620
40k
95d9f37115fbb0d13135d0f77132f856?s=48 morganepeng
86
11k
8081b26e05bb4354f7d65ffc34cbbd67?s=48 chriscoyier
498
130k
3d6ace9554821d552146413bcdf874f6?s=48 trishagee
6
430
D47656e20ff5e42f125fc5ea0fd636c6?s=48 tmm1
61
7.4k
3d4519fd34b76fb265fc0237f3792bd4?s=48 danielanewman
196
19k
E4f5d494ebdc9e7cce1aecf3ce3e8bc1?s=48 shpigford
366
42k
7c8469ee8c9e594c65c59b919626c08d?s=48 scottboms
250
11k
8081b26e05bb4354f7d65ffc34cbbd67?s=48 chriscoyier
775
240k
812e1705ff0f8bc1bcf18587bde687d5?s=48 62gerente
581
190k
C0390e8b11079f8761c0cd3274ed61cb?s=48 productmarketing
3
440
168ccec72eee0530b818d44f3fedaacf?s=48 shlominoach
174
7.1k

Transcript

  1. Better Docker Image Container Build Meetup #1 #container_build @orisano

  2. ྑ͍Docker Imageͱ͸Կ͔

  3. αʔϏεʹؔΘΔਓΛ ޾ͤʹ͢ΔΠϝʔδ

  4. ݟͯΘ͔Γ΍͍͢ खݩͰߴ଎ʹbuild͞ΕΔ CI্Ͱߴ଎ʹbuild͞ΕΔ ߴ଎ʹdeploy͞ΕΔ

  5. ࠓ೔࿩͍ͨ͜͠ͱ

  6. ͲͷΑ͏ʹ଎͘͢Δ͔ ͲͷΑ͏ʹখ͘͢͞Δ͔

  7. ͲͷΑ͏ʹ଎͘͢Δ͔ ͲͷΑ͏ʹখ͘͢͞Δ͔

  8. ͲͷΑ͏ʹ଎͘͢Δ͔ • ΠϝʔδΛখ͘͢͞Δ • ίϚϯυͦͷ΋ͷΛ଎͘͢Δ • cacheΛޮ͔ͤΔ • ґଘͷͳ͍εςʔδΛฒྻͰ࣮ߦ͢Δ •

    ඞཁͳ͍εςʔδΛbuild͠ͳ͍

  9. ͲͷΑ͏ʹ଎͘͢Δ͔ • ΠϝʔδΛখ͘͢͞Δ • ίϚϯυͦͷ΋ͷΛ଎͘͢Δ • cacheΛޮ͔ͤΔ • ґଘͷͳ͍εςʔδΛฒྻͰ࣮ߦ͢Δ •

    ඞཁͳ͍εςʔδΛbuild͠ͳ͍

  10. imageΛখ͘͢͞Δͷ͸ docker push͢Δͱ͖ͷ଎౓ docker pull͢Δͱ͖ͷ଎౓ ͷߴ଎Խʹͭͳ͕Δ

  11. Ͳͷ༷ʹখ͘͢͞Δ͔͸ ͋ͱͰ

  12. ͲͷΑ͏ʹ଎͘͢Δ͔ • ΠϝʔδΛখ͘͢͞Δ • ίϚϯυͦͷ΋ͷΛ଎͘͢Δ • cacheΛޮ͔ͤΔ • ґଘͷͳ͍εςʔδΛฒྻͰ࣮ߦ͢Δ •

    ඞཁͳ͍εςʔδΛbuild͠ͳ͍

  13. ίϚϯυͦͷ΋ͷΛ଎͘͢Δ

  14. URLʹର͢ΔADDΛ࢖Θͳ͍ RUNͷத਎Λ଎͘͢Δ

  15. URLʹର͢ΔADDΛ࢖Θͳ͍ RUNͷத਎Λ଎͘͢Δ

  16. URLʹର͢ΔADD͸ جຊతʹμ΢ϯϩʔυ͢Δ

  17. ஗͍

  18. ΞΫηε͍ͯ͠Δઌͷ ίϯςϯπ͕ႈ౳ͳΒ wget + gzip + tarͰे෼

  19. `RUN wget`ʹ͢Δ͜ͱͰ cache͕ޮ͘

  20. ႈ౳Ͱͳ͍Ϧιʔεʹ ґଘ͢ΔͷΛۃྗ΍ΊΔ

  21. URLʹର͢ΔADDΛ࢖Θͳ͍ RUNͷத਎Λ଎͘͢Δ

  22. GitHub͔Β࣮ߦϑΝΠϧΛ curl or wgetͰऔಘ͢Δ ͕஗͍

  23. Ͳ͏ʹ͔଎͘Ͱ͖ͳ͍͔ curl -vvvΛ͏ͬͯΈͨ

  24. GitHub Release͸S3Ͱ͋Δ S3͸Accept-Ranges: bytes ͱ͍͏͜ͱ͕Θ͔ͬͨ

  25. Accept-Ranges: bytes ͬͯͳΜͩΖ͏

  26. RFC7233, Range Requests ൣғΛࢦఆͯ͠ downloadͰ͖Δ

  27. ͭ·Γ ฒྻμ΢ϯϩʔυ͕Մೳ

  28. github.com/orisano/rget

  29. None

  30. ؀ڥʹΑΓ·͕͢ 5min -> 2min

  31. Dockerfileͷ޻෉Ҏ֎ʹ΋ ଎͘͢ΔΞϓϩʔν͸͋Δ

  32. ͲͷΑ͏ʹ଎͘͢Δ͔ • ΠϝʔδΛখ͘͢͞Δ • ίϚϯυͦͷ΋ͷΛ଎͘͢Δ • cacheΛޮ͔ͤΔ • ґଘͷͳ͍εςʔδΛฒྻͰ࣮ߦ͢Δ •

    ඞཁͳ͍εςʔδΛbuild͠ͳ͍

  33. cacheΛޮ͔ͤΔ

  34. docker build͢ΔϚγϯ͕ ಉҰͷ৔߹ ಉҰͰͳ͍৔߹

  35. docker build͢ΔϚγϯ͕ ಉҰͷ৔߹ ಉҰͰͳ͍৔߹

  36. cacheͷ࢓૊ΈΛཧղ͢Δ

  37. RUN͸ จࣈྻ͕มΘΒͳ͍ݶΓ جຊతʹcache͞ΕΔ

  38. COPY, ADD͢ΔϑΝΠϧͷ ಺༰͕มΘͬͨ৔߹ Ҏ߱ͷRUNͷcache͕ഁغ

  39. ͳͷͰ lockfileͳͲΛઌʹίϐʔͯ͠ install͚ͩͯ͠cacheͤ͞Δ

  40. None
  41. None

  42. docker build͢ΔϚγϯ͕ ಉҰͷ৔߹ ಉҰͰͳ͍৔߹

  43. CI্Ͱbuild͢Δͱ͖ͳͲ cache͕ͳ͍͜ͱ͕ଟ͍

  44. docker save & load or docker pull docker build —cache-from

    Λ࢖͏

  45. image͕େ͖͍/layer͕ଟ͍ ৔߹ buildͷ΄͏͕ૣ͍͜ͱ΋͋Δ

  46. image͕େ͖͍/layer͕ଟ͍ ৔߹ buildͷ΄͏͕ૣ͍͜ͱ΋͋Δ ܭଌܾͯ͠ΊΔ

  47. ͲͷΑ͏ʹ଎͘͢Δ͔ • ΠϝʔδΛখ͘͢͞Δ • ίϚϯυͦͷ΋ͷΛ଎͘͢Δ • cacheΛޮ͔ͤΔ • ґଘͷͳ͍εςʔδΛฒྻͰ࣮ߦ͢Δ •

    ඞཁͳ͍εςʔδΛbuild͠ͳ͍

  48. ґଘͷͳ͍εςʔδΛ ฒྻͰ࣮ߦ͢Δ

  49. buildkitΛ࢖͍ͬͯͩ͘͞ʂ github.com/moby/buildkit export DOCKER_BUILDKIT=1

  50. ͲͷΑ͏ʹ଎͘͢Δ͔ • ΠϝʔδΛখ͘͢͞Δ • ίϚϯυͦͷ΋ͷΛ଎͘͢Δ • cacheΛޮ͔ͤΔ • ґଘͷͳ͍εςʔδΛฒྻͰ࣮ߦ͢Δ •

    ඞཁͳ͍εςʔδΛbuild͠ͳ͍

  51. ඞཁͳ͍εςʔδΛ build͠ͳ͍

  52. —targetΛ࢖͑͹Α͍ͷͰ͸ʁ

  53. —target͸ ࢦఆͨ͠εςʔδҎલΛ શ෦build͢Δ

  54. multi stage buildΛ ౿ΈࠐΜͩ࢖͍ํ͍ͯ͠Δͱ ૺ۰͕ͪ͠

  55. ͓ͦΒ͘buildkitͰͰ͖Δ͕ buildkit͕࢖͑ͳ͍৔߹ʹ

  56. github.com/orisano/targd • DockerfileͷASTΛऔಘ • ࢦఆ͞ΕͨεςʔδͷґଘεςʔδΛநग़ • ґଘεςʔδ͚ͩͷDockerfileΛग़ྗ

  57. github.com/orisano/targd

  58. ͲͷΑ͏ʹ଎͘͢Δ͔ ͲͷΑ͏ʹখ͘͢͞Δ͔

  59. ͲͷΑ͏ʹখ͘͢͞Δ͔ • multi stage buildΛ࢖͏ • RUNΛ·ͱΊΔ(?) • ͳͥେ͖͍͔Λ஌Δ

  60. ͲͷΑ͏ʹখ͘͢͞Δ͔ • multi stage buildΛ࢖͏ • RUNΛ·ͱΊΔ(?) • ͳͥେ͖͍͔Λ஌Δ

  61. multi stage buildΛ࢖͏

  62. multi stage buildͰ ͋Γ͕ͪͳٙ໰

  63. ࠷ऴతͳΠϝʔδ͕ খ͘͞ͳΔ͔Β ͦΕ·Ͱͷεςʔδ͸ ࠷దԽ͠ͳͯ͘ྑ͍ʁ

  64. None

  65. ݸਓతʹ͸No

  66. moby/issues/34715 —cache-formͩͱ multi stage buildͷ લஈͷεςʔδͷcache͕ ޮ͔ͳ͍

  67. ࠷ऴεςʔδ͔͠ pushͯ͠ͳ͍͔Β౰ͨΓલ

  68. CI্Ͱͷbuild͸ —cache-fromΛ࢖͏ ࠷ऴεςʔδ͚ͩcache͢Δ ҙຯ͸ͳ͍

  69. multi stage build࣌ʹ cacheΛޮ͔͍ͤͨ৔߹͸ લͷεςʔδ΋ ໌ࣔతʹpush͢Δ͔͠ͳ͍

  70. ݁ہpush͢ΔͷͰ push/pullͷίετ͕͔͔Δ ͢΂ͯͷεςʔδΛฏ౳ʹ খ͘͢͞΂͖ (ݸਓͷݟղͰ͢)

  71. ͲͷΑ͏ʹখ͘͢͞Δ͔ • multi stage buildΛ࢖͏ • RUNΛ·ͱΊΔ(?) • ͳͥେ͖͍͔Λ஌Δ

  72. RUNΛ·ͱΊΔ(?)

  73. Ͳ͔ͬʔ;͍͊Δ͸ 1ͭͷRUNʹ શ෦ॻ͘ͱྑ͍Β͍͠(?)

  74. github.com/orisano/minid • DockerfileͷASTΛऔಘ • ࿈ଓ͢ΔRUN, COPY, ADDΛ࿈݁͢Δ • ݁ՌͷDockerfileΛग़ྗ

  75. github.com/orisano/minid

  76. αΠζ͕খ͘͞ͳͬͨ

  77. Ͱ΋ͳΜͰʁ

  78. ϨΠϠʔͷΦʔόʔϔου͕ ݮΔ͔Βখ͘͞ͳΔʁ

  79. ͍͍͑

  80. ·ͣ Ͳ͏͍͏ܗͰอଘ͞ΕͯΔ͔ Λ஌Δ

  81. moby/image/spec/v1.md

  82. AUFS

  83. http://docs.docker.jp/engine/userguide/storagedriver/aufs-driver.html

  84. ࡟আ͸ whiteoutϑΝΠϧͷ௥Ճ Ҡಈ͸ opaqueϑΝΠϧͷ௥Ճ ʴ ҠಈઌͷϑΝΠϧࠩ෼

  85. Ұ౓Ͱ΋ RUN,COPY,ADDΛލ͙ͱ imageʹ͸࢒ͬͯ͠·͏

  86. ػցతʹͰ΋ ҰͭͷRUNʹ·ͱΊΔ͜ͱͰ ༨ܭͳ΋ͷΛ࡟ݮͰ͖Δ

  87. 1ͭͷϨΠϠʔʹ ·ͱΊΔ͜ͱ͸ ຊ౰ʹਖ਼͍͠ͷ͔

  88. ϨΠϠʔΛผ͚Δ͜ͱʹΑΓ ฒྻμ΢ϯϩʔυͷԸܙ cacheͷ༗ޮ׆༻ Մಡੑͷ޲্

  89. ϨΠϠʔΛผ͚Δ͜ͱʹΑΓ ฒྻμ΢ϯϩʔυͷԸܙ cacheͷ༗ޮ׆༻ Մಡੑͷ޲্ ܭଌܾͯ͠ΊΔ

  90. ͱ͸͍͑ ෳ਺ϨΠϠʔʹ ލ͕Βͳ͍ఔ౓ʹ ෼ׂ͢Δͷ͕ྑͦ͞͏

  91. ͲͷΑ͏ʹখ͘͢͞Δ͔ • multi stage buildΛ࢖͏ • RUNΛ·ͱΊΔ(?) • ͳͥେ͖͍͔Λ஌Δ

  92. ͳͥେ͖͍͔Λ஌Δ

  93. ͍͔ͳΔνϡʔχϯάͰ΋ ܭଌͤͣʹ΍ͬͯ͸͍͚ͳ͍

  94. docker history

  95. docker history

  96. ͲͷϨΠϠʔ͕ େ͖͍͔͸Θ͔Δ͚Ͳ ͳͥେ͖͍͔͸෼͔ΓͮΒ͍

  97. ҙ֎ʹίϚϯυ͕ Ͳ͏͍͏ϑΝΠϧΛ࡞Δͷ͔ ஌Βͳ͍

  98. github.com/orisano/dlayer • docker saveͰಘΒΕͨtarΛղੳ • layerͱcmdͷରԠΛͱΔ • layerͷࠩ෼ΛϑΝΠϧαΠζ͕େ͖͍ॱʹද ࣔ

  99. github.com/orisano/dlayer

  100. ࣮ࡍʹdlayerΛ࢖ͬͯ golangެࣜalpineΠϝʔδͷ αΠζΛݮΒͯ͠Έͨ

  101. docker-library/golang/pull/232

  102. None

  103. golang:1.11͔Βtoolchainͷ αΠζ͕Ͱ͔͘ͳͬͨ SSAͰͷ࠷దԽ͕૿͔͑ͨΒʁ

  104. 1.10: 253MB

  105. 1.11: 305MB

  106. ࡞ͬͨπʔϧ • github.com/orisano/rget • github.com/orisano/targd • github.com/orisano/minid • github.com/orisano/dlayer •

    ྑ͍ͱࢥͬͨΒελʔ͍͚ͯͨͩ͠ΔͱྭΈʹͳΓ·͢

  107. ·ͱΊ • ίϚϯυΛ଎͘Ͱ͖ͳ͍͔ߟ͑Δ • cacheΛཧղͯ͠༗ޮʹ׆͔͢ (CI or ϩʔΧϧ) • multi

    stage buildΛ࢖͏ (ॏ͍stageΛcache͢Δ) • layerΛখ͘͢͞ΔͨΊʹ෼ੳ͢Δ • buildkitΛ࢖͏!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

  108. rget͜΅Ε࿩

  109. github.com/orisano/rget

  110. Ͱ΋ ͜ΕͰຊ౰ʹ͍͍ΜͩΖ͏͔

  111. rgetͷbinaryΛdownload ͢ΔͷͰ͸ෛ͚ͨؾ͕͢Δ()

  112. shell͚ͩͰ ࣮ݱͰ͖ͳ͍ͩΖ͏͔

  113. ϙʔλϒϧੑ͸͍Βͳ͍ͷͰ ओઓ৔ͷalpineʹߜΔ

  114. alpine(busybox)͸ xargs͕ೖ͍ͬͯΔʂʂʂ

  115. xargsΛ࢖͏͜ͱͰ ฒྻԽ ಉ࣌઀ଓ਺੍ݶ ͕࣮ݱͰ͖Δ

  116. GitHub Release͸ ؆୯ʹHEAD͕Ͱ͖ͳ͍ͷͰ GETͰbodyΛແࢹͯ͠ Content-LengthΛऔಘͰ͖Δ

  117. seqͰ chunkͷrangeΛੜ੒Ͱ͖Δ

  118. wget͸206 Partial Content ͕͏·͘ѻ͑ͳ͍ͷͰμϝ

  119. ࢓ํͳ͍ͷͰ curl -RͰdownload

  120. ࠷ޙʹcatͯ͠chunkΛ݁߹

  121. shellͰ΋࣮ݱͰ͖Δʂ

  122. ͔͠͠ curl͸alpineඪ४Ͱ͸ͳ͍

  123. apk add —no-cache curl ͸ෛ͚ͨؾ͕͢Δ()

  124. ͔͠͠ wget͸206͕ॲཧͰ͖ͳ͍

  125. ఘΊΒΕͳ͍ͷͰ busyboxͷwgetͷ ιʔείʔυΛಡΉ

  126. busybox/networking/wget.c

  127. busybox/networking/wget.c

  128. -OͰࢦఆͨ͠ϑΝΠϧ͕ଘࡏ (ϑΝΠϧαΠζ͕1Ҏ্) -cΛࢦఆ͍ͯ͠Δ ্هͷ৚݅Λຬͨ͢ͱ͖ 206Λॲཧͯ͘͠ΕΔ

  129. ͭ·Γ ద౰ͳ1byteͷϑΝΠϧʹ -cͰࢦఆ͢Δ͜ͱͰճආՄೳ

  130. ઌ಄ͷ1byteΛऔΓআ͚Ε͹ alpineඪ४ͷΈͰ࣮ݱՄೳ

  131. tailͰ΋Մೳ͕ͩ஗͍ dd skip=1 iflag=skip_bytes ͕ߴ଎

  132. alpineඪ४ͷΈͰ࣮ݱ

  133. https://github.com/orisano/rget/blob/master/rget.sh