Better Docker ImageContainer Build Meetup #1 #container_build@orisano
View Slide
ྑ͍Docker ImageͱԿ͔
αʔϏεʹؔΘΔਓΛͤʹ͢ΔΠϝʔδ
ݟͯΘ͔Γ͍͢खݩͰߴʹbuild͞ΕΔCI্Ͱߴʹbuild͞ΕΔߴʹdeploy͞ΕΔ
ࠓ͍ͨ͜͠ͱ
ͲͷΑ͏ʹ͘͢Δ͔ͲͷΑ͏ʹখ͘͢͞Δ͔
ͲͷΑ͏ʹ͘͢Δ͔• ΠϝʔδΛখ͘͢͞Δ• ίϚϯυͦͷͷΛ͘͢Δ• cacheΛޮ͔ͤΔ• ґଘͷͳ͍εςʔδΛฒྻͰ࣮ߦ͢Δ• ඞཁͳ͍εςʔδΛbuild͠ͳ͍
imageΛখ͘͢͞Δͷdocker push͢Δͱ͖ͷdocker pull͢Δͱ͖ͷͷߴԽʹͭͳ͕Δ
Ͳͷ༷ʹখ͘͢͞Δ͔͋ͱͰ
ίϚϯυͦͷͷΛ͘͢Δ
URLʹର͢ΔADDΛΘͳ͍RUNͷதΛ͘͢Δ
URLʹର͢ΔADDجຊతʹμϯϩʔυ͢Δ
͍
ΞΫηε͍ͯ͠Δઌͷίϯςϯπ͕ႈͳΒwget + gzip + tarͰे
`RUN wget`ʹ͢Δ͜ͱͰcache͕ޮ͘
ႈͰͳ͍Ϧιʔεʹґଘ͢ΔͷΛۃྗΊΔ
GitHub͔Β࣮ߦϑΝΠϧΛcurl or wgetͰऔಘ͢Δ͕͍
Ͳ͏ʹ͔͘Ͱ͖ͳ͍͔curl -vvvΛ͏ͬͯΈͨ
GitHub ReleaseS3Ͱ͋ΔS3Accept-Ranges: bytesͱ͍͏͜ͱ͕Θ͔ͬͨ
Accept-Ranges: bytesͬͯͳΜͩΖ͏
RFC7233, Range RequestsൣғΛࢦఆͯ͠downloadͰ͖Δ
ͭ·Γฒྻμϯϩʔυ͕Մೳ
github.com/orisano/rget
ڥʹΑΓ·͕͢5min -> 2min
DockerfileͷҎ֎ʹ͘͢ΔΞϓϩʔν͋Δ
cacheΛޮ͔ͤΔ
docker build͢ΔϚγϯ͕ಉҰͷ߹ಉҰͰͳ͍߹
cacheͷΈΛཧղ͢Δ
RUNจࣈྻ͕มΘΒͳ͍ݶΓجຊతʹcache͞ΕΔ
COPY, ADD͢ΔϑΝΠϧͷ༰͕มΘͬͨ߹Ҏ߱ͷRUNͷcache͕ഁغ
ͳͷͰlockfileͳͲΛઌʹίϐʔͯ͠install͚ͩͯ͠cacheͤ͞Δ
CI্Ͱbuild͢Δͱ͖ͳͲcache͕ͳ͍͜ͱ͕ଟ͍
docker save & loadordocker pulldocker build —cache-fromΛ͏
image͕େ͖͍/layer͕ଟ͍߹buildͷ΄͏͕ૣ͍͜ͱ͋Δ
image͕େ͖͍/layer͕ଟ͍߹buildͷ΄͏͕ૣ͍͜ͱ͋Δܭଌܾͯ͠ΊΔ
ґଘͷͳ͍εςʔδΛฒྻͰ࣮ߦ͢Δ
buildkitΛ͍ͬͯͩ͘͞ʂgithub.com/moby/buildkitexport DOCKER_BUILDKIT=1
ඞཁͳ͍εςʔδΛbuild͠ͳ͍
—targetΛ͑Α͍ͷͰʁ
—targetࢦఆͨ͠εςʔδҎલΛશ෦build͢Δ
multi stage buildΛ౿ΈࠐΜ͍ͩํ͍ͯ͠Δͱૺ۰͕ͪ͠
͓ͦΒ͘buildkitͰͰ͖Δ͕buildkit͕͑ͳ͍߹ʹ
github.com/orisano/targd• DockerfileͷASTΛऔಘ• ࢦఆ͞ΕͨεςʔδͷґଘεςʔδΛநग़• ґଘεςʔδ͚ͩͷDockerfileΛग़ྗ
github.com/orisano/targd
ͲͷΑ͏ʹখ͘͢͞Δ͔• multi stage buildΛ͏• RUNΛ·ͱΊΔ(?)• ͳͥେ͖͍͔ΛΔ
multi stage buildΛ͏
multi stage buildͰ͋Γ͕ͪͳٙ
࠷ऴతͳΠϝʔδ͕খ͘͞ͳΔ͔ΒͦΕ·Ͱͷεςʔδ࠷దԽ͠ͳͯ͘ྑ͍ʁ
ݸਓతʹNo
moby/issues/34715—cache-formͩͱmulti stage buildͷલஈͷεςʔδͷcache͕ޮ͔ͳ͍
࠷ऴεςʔδ͔͠pushͯ͠ͳ͍͔ΒͨΓલ
CI্Ͱͷbuild—cache-fromΛ͏࠷ऴεςʔδ͚ͩcache͢Δҙຯͳ͍
multi stage build࣌ʹcacheΛޮ͔͍ͤͨ߹લͷεςʔδ໌ࣔతʹpush͢Δ͔͠ͳ͍
݁ہpush͢ΔͷͰpush/pullͷίετ͕͔͔Δͯ͢ͷεςʔδΛฏʹখ͖͘͢͞(ݸਓͷݟղͰ͢)
RUNΛ·ͱΊΔ(?)
Ͳ͔ͬʔ;͍͊Δ1ͭͷRUNʹશ෦ॻ͘ͱྑ͍Β͍͠(?)
github.com/orisano/minid• DockerfileͷASTΛऔಘ• ࿈ଓ͢ΔRUN, COPY, ADDΛ࿈݁͢Δ• ݁ՌͷDockerfileΛग़ྗ
github.com/orisano/minid
αΠζ͕খ͘͞ͳͬͨ
ͰͳΜͰʁ
ϨΠϠʔͷΦʔόʔϔου͕ݮΔ͔Βখ͘͞ͳΔʁ
͍͍͑
·ͣͲ͏͍͏ܗͰอଘ͞ΕͯΔ͔ΛΔ
moby/image/spec/v1.md
AUFS
http://docs.docker.jp/engine/userguide/storagedriver/aufs-driver.html
আwhiteoutϑΝΠϧͷՃҠಈopaqueϑΝΠϧͷՃʴҠಈઌͷϑΝΠϧࠩ
ҰͰRUN,COPY,ADDΛލ͙ͱimageʹͬͯ͠·͏
ػցతʹͰҰͭͷRUNʹ·ͱΊΔ͜ͱͰ༨ܭͳͷΛݮͰ͖Δ
1ͭͷϨΠϠʔʹ·ͱΊΔ͜ͱຊʹਖ਼͍͠ͷ͔
ϨΠϠʔΛผ͚Δ͜ͱʹΑΓฒྻμϯϩʔυͷԸܙcacheͷ༗ޮ׆༻Մಡੑͷ্
ϨΠϠʔΛผ͚Δ͜ͱʹΑΓฒྻμϯϩʔυͷԸܙcacheͷ༗ޮ׆༻Մಡੑͷ্ܭଌܾͯ͠ΊΔ
ͱ͍͑ෳϨΠϠʔʹލ͕Βͳ͍ఔʹׂ͢Δͷ͕ྑͦ͞͏
ͳͥେ͖͍͔ΛΔ
͍͔ͳΔνϡʔχϯάͰܭଌͤͣʹ͍͚ͬͯͳ͍
docker history
ͲͷϨΠϠʔ͕େ͖͍͔Θ͔Δ͚Ͳͳͥେ͖͍͔͔ΓͮΒ͍
ҙ֎ʹίϚϯυ͕Ͳ͏͍͏ϑΝΠϧΛ࡞Δͷ͔Βͳ͍
github.com/orisano/dlayer• docker saveͰಘΒΕͨtarΛղੳ• layerͱcmdͷରԠΛͱΔ• layerͷࠩΛϑΝΠϧαΠζ͕େ͖͍ॱʹදࣔ
github.com/orisano/dlayer
࣮ࡍʹdlayerΛͬͯgolangެࣜalpineΠϝʔδͷαΠζΛݮΒͯ͠Έͨ
docker-library/golang/pull/232
golang:1.11͔ΒtoolchainͷαΠζ͕Ͱ͔͘ͳͬͨSSAͰͷ࠷దԽ͕૿͔͑ͨΒʁ
1.10: 253MB
1.11: 305MB
࡞ͬͨπʔϧ• github.com/orisano/rget• github.com/orisano/targd• github.com/orisano/minid• github.com/orisano/dlayer• ྑ͍ͱࢥͬͨΒελʔ͍͚ͯͨͩ͠ΔͱྭΈʹͳΓ·͢
·ͱΊ• ίϚϯυΛ͘Ͱ͖ͳ͍͔ߟ͑Δ• cacheΛཧղͯ͠༗ޮʹ׆͔͢ (CI or ϩʔΧϧ)• multi stage buildΛ͏ (ॏ͍stageΛcache͢Δ)• layerΛখ͘͢͞ΔͨΊʹੳ͢Δ• buildkitΛ͏!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
rget͜΅Ε
Ͱ͜ΕͰຊʹ͍͍ΜͩΖ͏͔
rgetͷbinaryΛdownload͢ΔͷͰෛ͚ͨؾ͕͢Δ()
shell͚ͩͰ࣮ݱͰ͖ͳ͍ͩΖ͏͔
ϙʔλϒϧੑ͍Βͳ͍ͷͰओઓͷalpineʹߜΔ
alpine(busybox)xargs͕ೖ͍ͬͯΔʂʂʂ
xargsΛ͏͜ͱͰฒྻԽಉ࣌ଓ੍ݶ͕࣮ݱͰ͖Δ
GitHub Release؆୯ʹHEAD͕Ͱ͖ͳ͍ͷͰGETͰbodyΛແࢹͯ͠Content-LengthΛऔಘͰ͖Δ
seqͰchunkͷrangeΛੜͰ͖Δ
wget206 Partial Content͕͏·͘ѻ͑ͳ͍ͷͰμϝ
ํͳ͍ͷͰcurl -RͰdownload
࠷ޙʹcatͯ͠chunkΛ݁߹
shellͰ࣮ݱͰ͖Δʂ
͔͠͠curlalpineඪ४Ͱͳ͍
apk add —no-cache curlෛ͚ͨؾ͕͢Δ()
͔͠͠wget206͕ॲཧͰ͖ͳ͍
ఘΊΒΕͳ͍ͷͰbusyboxͷwgetͷιʔείʔυΛಡΉ
busybox/networking/wget.c
-OͰࢦఆͨ͠ϑΝΠϧ͕ଘࡏ(ϑΝΠϧαΠζ͕1Ҏ্)-cΛࢦఆ͍ͯ͠Δ্هͷ݅Λຬͨ͢ͱ͖206Λॲཧͯ͘͠ΕΔ
ͭ·Γదͳ1byteͷϑΝΠϧʹ-cͰࢦఆ͢Δ͜ͱͰճආՄೳ
ઌ಄ͷ1byteΛऔΓআ͚Εalpineඪ४ͷΈͰ࣮ݱՄೳ
tailͰՄೳ͕͍ͩdd skip=1 iflag=skip_bytes͕ߴ
alpineඪ४ͷΈͰ࣮ݱ
https://github.com/orisano/rget/blob/master/rget.sh
orisano
toshiaizawa
akki_megane
xibuka
hirosatogamo
sattosan
mako5656
chiroruxx
abenben
tomohisa
okunokentaro
yutaro527
uzulla
jeffersonlam
marktimemedia
quramy
jponch
sugarenia
dougneiner
paigeccino
denniskardys
62gerente
carmenintech
mojombo
robhawkes