Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Better docker image+

6247c099ad62bf727a4f9df10b6c6f23?s=47 orisano
December 05, 2018
Technology
5
3.8k

Better docker image+

6247c099ad62bf727a4f9df10b6c6f23?s=128

orisano

December 05, 2018
Tweet

More Decks by orisano

See All by orisano
6247c099ad62bf727a4f9df10b6c6f23?s=48 orisano
8
4.3k
6247c099ad62bf727a4f9df10b6c6f23?s=48 orisano
3
600
6247c099ad62bf727a4f9df10b6c6f23?s=48 orisano
2
1.8k
6247c099ad62bf727a4f9df10b6c6f23?s=48 orisano
2
5.5k
6247c099ad62bf727a4f9df10b6c6f23?s=48 orisano
0
1.7k
6247c099ad62bf727a4f9df10b6c6f23?s=48 orisano
11
6.2k
6247c099ad62bf727a4f9df10b6c6f23?s=48 orisano
5
4k
6247c099ad62bf727a4f9df10b6c6f23?s=48 orisano
21
24k
6247c099ad62bf727a4f9df10b6c6f23?s=48 orisano
8
1.1k

Other Decks in Technology

See All in Technology
F831177123a4ddda9aa1802721f0a0db?s=48 ozontech
0
870
0886c70b5ecf3b0b0ee6500c5e47884d?s=48 ecoopnet
0
190
61e7dc2e84a1a1c4ac1a8e2c552fee07?s=48 tutsunom
1
190
93dcbac1a838bbec898747bb92aa47cc?s=48 sadonosake
0
1.7k
B90a63f11581d1f064348e90ac2c9759?s=48 puremoru
1
110
D18583cc111c111c775f8da7c5b2ff52?s=48 orimanabu
1
320
154d77627bc95f4eb226c722023b5168?s=48 a1a
2
340
33d6320b53f63f6627508095633cf5ce?s=48 yosuke_matsuura
PRO
0
2.1k
140494d272a4d89883a94fdfdb29dea2?s=48 oracle4engineer
PRO
10
13k
8cc27471bcdeb2858fca212586a208b4?s=48 takahia1988
0
2.1k
Ae788ab7d139931493941e42584eb456?s=48 asoviewinc
0
1.6k
2c4ef0f20c4f05ca30e6c9917f0c17b6?s=48 okuken
0
170

Featured

See All Featured
32de0bd2ba869609d26fd052a4622778?s=48 cromwellryan
109
6.7k
2f5463832ccb768ccb4a1ca3607c27ef?s=48 jacobian
263
20k
Fe6b81005d1553accd6b2a28f6a2bef1?s=48 phodgson
92
4.4k
Aebc2d07a50011e2a30d38435fde564a?s=48 jrom
120
7.5k
E4f5d494ebdc9e7cce1aecf3ce3e8bc1?s=48 shpigford
167
20k
A60068bce2e73de3a37ca9d2dbe36092?s=48 bryan
106
12k
48c098b6579220a67a6ba949be6e4b5a?s=48 revolveconf
204
10k
C1daf20e5c49ff910745198ef9869ac2?s=48 hatefulcrawdad
259
17k
8081b26e05bb4354f7d65ffc34cbbd67?s=48 chriscoyier
149
20k
A9a491b0fcbe0fbce3d64063a37add99?s=48 rmw
17
1.3k
Ae14cc4491ac334f9cd23f9f93b4305e?s=48 orderedlist
PRO
333
37k
E190023b66e2b8aa73a842b106920c93?s=48 zenorocha
303
40k

Transcript

  1. Better Docker Image+ Bonfire Backend #2 #yjbonfire @orisano

  2. ྑ͍Docker Imageͱ͸Կ͔

  3. ݟͯΘ͔Γ΍͍͢ खݩͰߴ଎ʹbuild͞ΕΔ CI্Ͱߴ଎ʹbuild͞ΕΔ ߴ଎ʹdeploy͞ΕΔ

  4. ࠓ೔࿩͍ͨ͜͠ͱ

  5. ͲͷΑ͏ʹ଎͘͢Δ͔ ͲͷΑ͏ʹখ͘͢͞Δ͔

  6. ͲͷΑ͏ʹ଎͘͢Δ͔ ͲͷΑ͏ʹখ͘͢͞Δ͔

  7. ͲͷΑ͏ʹ଎͘͢Δ͔ • ΠϝʔδΛখ͘͢͞Δ • ίϚϯυͦͷ΋ͷΛ଎͘͢Δ • cacheΛޮ͔ͤΔ • ґଘͷͳ͍εςʔδΛฒྻͰ࣮ߦ͢Δ •

    ඞཁͳ͍εςʔδΛbuild͠ͳ͍

  8. ͲͷΑ͏ʹ଎͘͢Δ͔ • ΠϝʔδΛখ͘͢͞Δ • ίϚϯυͦͷ΋ͷΛ଎͘͢Δ • cacheΛޮ͔ͤΔ • ґଘͷͳ͍εςʔδΛฒྻͰ࣮ߦ͢Δ •

    ඞཁͳ͍εςʔδΛbuild͠ͳ͍

  9. imageΛখ͘͢͞Δͷ͸ docker push͢Δͱ͖ͷ଎౓ docker pull͢Δͱ͖ͷ଎౓ ͷߴ଎Խʹͭͳ͕Δ

  10. Ͳͷ༷ʹখ͘͢͞Δ͔͸ ͋ͱͰ

  11. ͲͷΑ͏ʹ଎͘͢Δ͔ • ΠϝʔδΛখ͘͢͞Δ • ίϚϯυͦͷ΋ͷΛ଎͘͢Δ • cacheΛޮ͔ͤΔ • ґଘͷͳ͍εςʔδΛฒྻͰ࣮ߦ͢Δ •

    ඞཁͳ͍εςʔδΛbuild͠ͳ͍

  12. ίϚϯυͦͷ΋ͷΛ଎͘͢Δ

  13. URLʹର͢ΔADDΛ࢖Θͳ͍

  14. URLʹର͢ΔADD͸ جຊతʹμ΢ϯϩʔυ͢Δ

  15. ஗͍

  16. ΞΫηε͍ͯ͠Δઌͷ ίϯςϯπ͕ႈ౳ͳΒ wget + gzip + tarͰे෼

  17. `RUN wget`ʹ͢Δ͜ͱͰ cache͕ޮ͘

  18. ႈ౳Ͱͳ͍Ϧιʔεʹ ґଘ͢ΔͷΛۃྗ΍ΊΔ

  19. build context ͷసૹྔΛߟ͑Δ

  20. ϞϊϨϙʹ͢Δͱ build contextେ͖͘ͳΓ͕ͪ

  21. ϞϊϨϙͷ৔߹͸ .dockerignore Λࣗಈੜ੒͠Α͏

  22. github.com/orisano/dignore • ࢦఆ͞ΕͨdirectoryҎ֎ͷdirectoryΛignore • ࢦఆ͞Εͨdirectoryʹdockerignore͕͋Ε͹ ల։ • ͜Ε͚ͩʂ

  23. buildͷલ޻ఔͰ buildʹඞཁͳ͍αʔϏε͸ Ignore͠Α͏

  24. buildkitΛ࢖͍ͬͯΔͱ ࠩ෼సૹͯ͘͠ΕͨΓ͢Δ

  25. COPYΛҙࣝͨ͠ directoryߏ଄

  26. COPYͷҾ਺͕ directoryͷ৔߹͸ ର৅ͷdirectoryʹ த਎Λશ෦ίϐʔͯ͠͠·͏

  27. ಛఆͷσΟϨΫτϦ͚ͩ ίϐʔͨ͘͠ͳ͍ ͱ͍͏͕೉͍͠

  28. dockerignore Ͱআ֎͢Ε͹ྑ͘ͳ͍ʁ

  29. ͍͍͑

  30. vendorͳͲΛ௚઀؅ཧͯ͠ build࣌ʹdownload͠ͳ͍ ৔߹ʹຊ౰ʹਏ͍

  31. 2ճҎ্ॏ͍directoryΛ COPYͨ͘͠ͳ͍

  32. ͦ͏ͳΒͳ͍ͨΊͷ directoryߏ଄ʹ͓ͯ͜͠͏

  33. ਓ͕ؒॻ͍ͨ΋ͷ͕ ೖ͍ͬͯΔdirectory ֎෦ʹґଘ͍ͯ͠Δ΋ͷ ࣗಈੜ੒෺͕ೖ͍ͬͯΔ directoryΛ෼཭͠Α͏

  34. buildkitͩͱࠩ෼సૹ

  35. ͍ͣΕʹͤΑ ෼཭͓͍ͯͨ͠ํ͕ cacheʹ༏͍͠

  36. RUNͷத਎Λ଎͘͢Δ

  37. GitHub͔Β࣮ߦϑΝΠϧΛ curl or wgetͰऔಘ͢Δ ͕஗͍

  38. Ͳ͏ʹ͔଎͘Ͱ͖ͳ͍͔ curl -vvvΛ͏ͬͯΈͨ

  39. GitHub Release͸S3Ͱ͋Δ S3͸Accept-Ranges: bytes ͱ͍͏͜ͱ͕Θ͔ͬͨ

  40. Accept-Ranges: bytes ͬͯͳΜͩΖ͏

  41. RFC7233, Range Requests ൣғΛࢦఆͯ͠ downloadͰ͖Δ

  42. ͭ·Γ ฒྻμ΢ϯϩʔυ͕Մೳ

  43. github.com/orisano/rget

  44. None

  45. ؀ڥʹΑΓ·͕͢ 5min -> 2min

  46. Dockerfileͷ޻෉Ҏ֎ʹ΋ ଎͘͢ΔΞϓϩʔν͸͋Δ

  47. ͲͷΑ͏ʹ଎͘͢Δ͔ • ΠϝʔδΛখ͘͢͞Δ • ίϚϯυͦͷ΋ͷΛ଎͘͢Δ • cacheΛޮ͔ͤΔ • ґଘͷͳ͍εςʔδΛฒྻͰ࣮ߦ͢Δ •

    ඞཁͳ͍εςʔδΛbuild͠ͳ͍

  48. cacheΛޮ͔ͤΔ

  49. docker build͢ΔϚγϯ͕ ಉҰͷ৔߹ ಉҰͰͳ͍৔߹

  50. docker build͢ΔϚγϯ͕ ಉҰͷ৔߹ ಉҰͰͳ͍৔߹

  51. cacheͷ࢓૊ΈΛཧղ͢Δ

  52. RUN͸ จࣈྻ͕มΘΒͳ͍ݶΓ جຊతʹcache͞ΕΔ

  53. COPY, ADD͢ΔϑΝΠϧͷ ಺༰͕มΘͬͨ৔߹ Ҏ߱ͷRUNͷcache͕ഁغ

  54. ͳͷͰ lockfileͳͲΛઌʹίϐʔͯ͠ install͚ͩͯ͠cacheͤ͞Δ

  55. None
  56. None

  57. docker build͢ΔϚγϯ͕ ಉҰͷ৔߹ ಉҰͰͳ͍৔߹

  58. CI্Ͱbuild͢Δͱ͖ͳͲ cache͕ͳ͍͜ͱ͕ଟ͍

  59. docker save & load or docker pull docker build —cache-from

    Λ࢖͏

  60. image͕େ͖͍/layer͕ଟ͍ ৔߹ buildͷ΄͏͕ૣ͍͜ͱ΋͋Δ

  61. image͕େ͖͍/layer͕ଟ͍ ৔߹ buildͷ΄͏͕ૣ͍͜ͱ΋͋Δ ܭଌܾͯ͠ΊΔ

  62. ͲͷΑ͏ʹ଎͘͢Δ͔ • ΠϝʔδΛখ͘͢͞Δ • ίϚϯυͦͷ΋ͷΛ଎͘͢Δ • cacheΛޮ͔ͤΔ • ґଘͷͳ͍εςʔδΛฒྻͰ࣮ߦ͢Δ •

    ඞཁͳ͍εςʔδΛbuild͠ͳ͍

  63. ґଘͷͳ͍εςʔδΛ ฒྻͰ࣮ߦ͢Δ

  64. buildkitΛ࢖͍ͬͯͩ͘͞ʂ github.com/moby/buildkit export DOCKER_BUILDKIT=1

  65. ͲͷΑ͏ʹ଎͘͢Δ͔ • ΠϝʔδΛখ͘͢͞Δ • ίϚϯυͦͷ΋ͷΛ଎͘͢Δ • cacheΛޮ͔ͤΔ • ґଘͷͳ͍εςʔδΛฒྻͰ࣮ߦ͢Δ •

    ඞཁͳ͍εςʔδΛbuild͠ͳ͍

  66. ඞཁͳ͍εςʔδΛ build͠ͳ͍

  67. —targetΛ࢖͑͹Α͍ͷͰ͸ʁ

  68. —target͸ ࢦఆͨ͠εςʔδҎલΛ શ෦build͢Δ

  69. multi stage buildΛ ౿ΈࠐΜͩ࢖͍ํ͍ͯ͠Δͱ ૺ۰͕ͪ͠

  70. multi stage buildͷ ౿ΈࠐΜͩ࢖͍ํʹ͍ͭͯ͸

  71. https://speakerdeck.com/orisano/multi-stage-builds-patterns-and-practice

  72. ͓ͦΒ͘buildkitͰͰ͖Δ͕ buildkit͕࢖͑ͳ͍৔߹ʹ

  73. github.com/orisano/targd • DockerfileͷASTΛऔಘ • ࢦఆ͞ΕͨεςʔδͷґଘεςʔδΛநग़ • ґଘεςʔδ͚ͩͷDockerfileΛग़ྗ

  74. github.com/orisano/targd

  75. ͲͷΑ͏ʹ଎͘͢Δ͔ ͲͷΑ͏ʹখ͘͢͞Δ͔

  76. ͲͷΑ͏ʹখ͘͢͞Δ͔ • multi stage buildΛ࢖͏ • RUNΛ·ͱΊΔ(?) • ͳͥେ͖͍͔Λ஌Δ

  77. ͲͷΑ͏ʹখ͘͢͞Δ͔ • multi stage buildΛ࢖͏ • RUNΛ·ͱΊΔ(?) • ͳͥେ͖͍͔Λ஌Δ

  78. multi stage buildΛ࢖͏

  79. multi stage buildͰ ͋Γ͕ͪͳٙ໰

  80. ࠷ऴతͳΠϝʔδ͕ খ͘͞ͳΔ͔Β ͦΕ·Ͱͷεςʔδ͸ ࠷దԽ͠ͳͯ͘ྑ͍ʁ

  81. None

  82. ݸਓతʹ͸No

  83. moby/issues/34715 —cache-formͩͱ multi stage buildͷ લஈͷεςʔδͷcache͕ ޮ͔ͳ͍

  84. ࠷ऴεςʔδ͔͠ pushͯ͠ͳ͍͔Β౰ͨΓલ

  85. CI্Ͱͷbuild͸ —cache-fromΛ࢖͏ ࠷ऴεςʔδ͚ͩcache͢Δ ҙຯ͸ͳ͍

  86. multi stage build࣌ʹ cacheΛޮ͔͍ͤͨ৔߹͸ લͷεςʔδ΋ ໌ࣔతʹpush͢Δ͔͠ͳ͍

  87. ͜ΕΛखಈͰ΍Δͱ cache-from஍ࠈʹͳΔ

  88. github.com/orisano/castage • DockerfileͷASTΛऔಘ • εςʔδҰཡΛऔಘ • docker pullΛੜ੒ • docker

    buildΛcache-from෇͖Ͱੜ੒

  89. ݁ہpush͢ΔͷͰ push/pullͷίετ͕͔͔Δ ͢΂ͯͷεςʔδΛฏ౳ʹ খ͘͢͞΂͖ (ݸਓͷݟղͰ͢)

  90. ͲͷΑ͏ʹখ͘͢͞Δ͔ • multi stage buildΛ࢖͏ • RUNΛ·ͱΊΔ(?) • ͳͥେ͖͍͔Λ஌Δ

  91. RUNΛ·ͱΊΔ(?)

  92. Ͳ͔ͬʔ;͍͊Δ͸ 1ͭͷRUNʹ શ෦ॻ͘ͱྑ͍Β͍͠(?)

  93. github.com/orisano/minid • DockerfileͷASTΛऔಘ • ࿈ଓ͢ΔRUN, COPY, ADDΛ࿈݁͢Δ • ݁ՌͷDockerfileΛग़ྗ

  94. github.com/orisano/minid

  95. αΠζ͕খ͘͞ͳͬͨ

  96. Ͱ΋ͳΜͰʁ

  97. ϨΠϠʔͷΦʔόʔϔου͕ ݮΔ͔Βখ͘͞ͳΔʁ

  98. ͍͍͑

  99. ·ͣ Ͳ͏͍͏ܗͰอଘ͞ΕͯΔ͔ Λ஌Δ

  100. moby/image/spec/v1.md

  101. AUFS

  102. http://docs.docker.jp/engine/userguide/storagedriver/aufs-driver.html

  103. ࡟আ͸ whiteoutϑΝΠϧͷ௥Ճ Ҡಈ͸ opaqueϑΝΠϧͷ௥Ճ ʴ ҠಈઌͷϑΝΠϧࠩ෼

  104. Ұ౓Ͱ΋ RUN,COPY,ADDΛލ͙ͱ imageʹ͸࢒ͬͯ͠·͏

  105. ػցతʹͰ΋ ҰͭͷRUNʹ·ͱΊΔ͜ͱͰ ༨ܭͳ΋ͷΛ࡟ݮͰ͖Δ

  106. 1ͭͷϨΠϠʔʹ ·ͱΊΔ͜ͱ͸ ຊ౰ʹਖ਼͍͠ͷ͔

  107. ϨΠϠʔΛผ͚Δ͜ͱʹΑΓ ฒྻμ΢ϯϩʔυͷԸܙ cacheͷ༗ޮ׆༻

  108. ϨΠϠʔΛผ͚Δ͜ͱʹΑΓ ฒྻμ΢ϯϩʔυͷԸܙ cacheͷ༗ޮ׆༻ ܭଌܾͯ͠ΊΔ

  109. ͲͷΑ͏ʹখ͘͢͞Δ͔ • multi stage buildΛ࢖͏ • RUNΛ·ͱΊΔ(?) • ͳͥେ͖͍͔Λ஌Δ

  110. ͳͥେ͖͍͔Λ஌Δ

  111. ͍͔ͳΔνϡʔχϯάͰ΋ ܭଌͤͣʹ΍ͬͯ͸͍͚ͳ͍

  112. docker history

  113. docker history

  114. ͲͷϨΠϠʔ͕ େ͖͍͔͸Θ͔Δ͚Ͳ ͳͥେ͖͍͔͸෼͔ΓͮΒ͍

  115. ҙ֎ʹίϚϯυ͕ Ͳ͏͍͏ϑΝΠϧΛ࡞Δͷ͔ ஌Βͳ͍

  116. github.com/orisano/dlayer • docker saveͰಘΒΕͨtarΛղੳ • layerͱcmdͷରԠΛͱΔ • layerͷࠩ෼ΛϑΝΠϧαΠζ͕େ͖͍ॱʹද ࣔ

  117. github.com/orisano/dlayer

  118. ࣮ࡍʹdlayerΛ࢖ͬͯ golangެࣜalpineΠϝʔδͷ αΠζΛݮΒͯ͠Έͨ

  119. docker-library/golang/pull/232

  120. None

  121. golang:1.11͔Βtoolchainͷ αΠζ͕Ͱ͔͘ͳͬͨ SSAͰͷ࠷దԽ͕૿͔͑ͨΒʁ

  122. 253MB

  123. 305MB

  124. github.com/wagoodman/dive

  125. ࡞ͬͨπʔϧ • github.com/orisano/rget • github.com/orisano/targd • github.com/orisano/minid • github.com/orisano/dlayer •

    ྑ͍ͱࢥͬͨΒελʔ͍͚ͯͨͩ͠ΔͱྭΈʹͳΓ·͢

  126. ࡞ͬͨπʔϧ • github.com/orisano/dignore • github.com/orisano/castage • ྑ͍ͱࢥͬͨΒελʔ͍͚ͯͨͩ͠ΔͱྭΈʹͳΓ·͢

  127. ·ͱΊ • ίϚϯυΛ଎͘Ͱ͖ͳ͍͔ߟ͑Δ • cacheΛཧղͯ͠༗ޮʹ׆͔͢ (CI or ϩʔΧϧ) • multi

    stage buildΛ࢖͏ (ॏ͍stageΛcache͢Δ) • layerΛখ͘͢͞ΔͨΊʹ෼ੳ͢Δ • buildkitΛ࢖͏!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

  128. rget͜΅Ε࿩

  129. github.com/orisano/rget

  130. Ͱ΋ ͜ΕͰຊ౰ʹ͍͍ΜͩΖ͏͔

  131. rgetͷbinaryΛdownload ͢ΔͷͰ͸ෛ͚ͨؾ͕͢Δ()

  132. shell͚ͩͰ ࣮ݱͰ͖ͳ͍ͩΖ͏͔

  133. ϙʔλϒϧੑ͸͍Βͳ͍ͷͰ ओઓ৔ͷalpineʹߜΔ

  134. alpine(busybox)͸ xargs͕ೖ͍ͬͯΔʂʂʂ

  135. xargsΛ࢖͏͜ͱͰ ฒྻԽ ಉ࣌઀ଓ਺੍ݶ ͕࣮ݱͰ͖Δ

  136. GitHub Release͸ ؆୯ʹHEAD͕Ͱ͖ͳ͍ͷͰ GETͰbodyΛແࢹͯ͠ Content-LengthΛऔಘͰ͖Δ

  137. seqͰ chunkͷrangeΛੜ੒Ͱ͖Δ

  138. wget͸206 Partial Content ͕͏·͘ѻ͑ͳ͍ͷͰμϝ

  139. ࢓ํͳ͍ͷͰ curl -RͰdownload

  140. ࠷ޙʹcatͯ͠chunkΛ݁߹

  141. shellͰ΋࣮ݱͰ͖Δʂ

  142. ͔͠͠ curl͸alpineඪ४Ͱ͸ͳ͍

  143. apk add —no-cache curl ͸ෛ͚ͨؾ͕͢Δ()

  144. ͔͠͠ wget͸206͕ॲཧͰ͖ͳ͍

  145. ఘΊΒΕͳ͍ͷͰ busyboxͷwgetͷ ιʔείʔυΛಡΉ

  146. busybox/networking/wget.c

  147. busybox/networking/wget.c

  148. -OͰࢦఆͨ͠ϑΝΠϧ͕ଘࡏ (ϑΝΠϧαΠζ͕1Ҏ্) -cΛࢦఆ͍ͯ͠Δ ্هͷ৚݅Λຬͨ͢ͱ͖ 206Λॲཧͯ͘͠ΕΔ

  149. ͭ·Γ ద౰ͳ1byteͷϑΝΠϧʹ -cͰࢦఆ͢Δ͜ͱͰճආՄೳ

  150. ઌ಄ͷ1byteΛऔΓআ͚Ε͹ alpineඪ४ͷΈͰ࣮ݱՄೳ

  151. tailͰ΋Մೳ͕ͩ஗͍ dd skip=1 iflag=skip_bytes ͕ߴ଎

  152. alpineඪ४ͷΈͰ࣮ݱ

  153. https://github.com/orisano/rget/blob/master/rget.sh