Upgrade to Pro
— share decks privately, control downloads, hide ads and more …
Speaker Deck
Speaker Deck
PRO
Sign in
Sign up
for free
Droot Internals
Yuuki Tsubouchi (yuuk1)
April 23, 2016
Technology
3
18k
Droot Internals
id:y_uuki
第9回 コンテナ型仮想化の情報交換会@福岡
Yuuki Tsubouchi (yuuk1)
April 23, 2016
Tweet
Share
More Decks by Yuuki Tsubouchi (yuuk1)
See All by Yuuki Tsubouchi (yuuk1)
yuukit
4
1.6k
yuukit
1
760
yuukit
0
76
yuukit
0
1.2k
yuukit
0
890
yuukit
0
310
yuukit
5
810
yuukit
0
120
yuukit
0
50
Other Decks in Technology
See All in Technology
ayanadesu
0
390
picardparis
4
2.3k
hagyyyy
1
190
youtalk
0
390
chaspy
6
1.3k
mmarukaw
0
1.9k
futo23
1
310
kanaugust
PRO
0
140
yamamuteki
3
740
askul
1
200
pg0084
1
130
zak3
1
190
Featured
See All Featured
malarkey
193
8.6k
tammielis
237
23k
phodgson
87
3.9k
brianwarren
82
4.7k
malarkey
119
16k
destraynor
146
19k
sstephenson
145
12k
hannesfritz
28
950
skipperchong
8
710
chriscoyier
145
20k
scottboms
251
11k
destraynor
222
47k
Transcript
%SPPU*OUFSOBMT JEZ@VVLJ ୈ̕ճίϯςφܕԾԽͷใަձ!Ԭ
id:y_uuki @y_uuk1 ͯͳ!ژ ΣϒΦϖϨʔγϣϯΤϯδχΞ
IUUQZVVLJIBUFOBCMPHDPNFOUSZESPPU
TL;DR • ιϑτΣΞґଘࠈͷղܾͷͨΊʹ DockerΛ͍͍ͨ • ຊ൪ڥͰDockerΛӡ༻͢ΔͷͭΒ͍ • ʮBuild, Ship, Runʯͱ͍͏ίϯηϓτ͖
• DockerΠϝʔδΛS3Λܦ༝ͯ͠͠ɺ chrootͰ࣮ߦ͢Δख๏ͷఏҊ
ԾԽٕज़ )8ԾԽ 04ԾԽ ,7. 9FO ʜ γεςϜίϯςφ -9$ ΞϓϦέʔγϣϯ ίϯςφ
%PDLFS
ͳͥ()DockerΛ͏ͷ͔ • ✘ VMΑΓߴ • ✘ Πϛϡʔλϒϧ • ✘ Φʔτεέʔϧ
• ˚ ϙʔλϏϦςΟ • ◦ ϓϩάϥϚϒϧͳϗετڥ • ◦ ιϑτΣΞґଘࠈͷղܾ
ιϑτΣΞґଘࠈ • ͋ΔιϑτΣΞ͍͍ͨͯෳͷιϑτ ΣΞʹґଘ͢Δ • ґଘઌͷιϑτΣΞ·ͨෳͷιϑτ ΣΞʹґଘ͢Δ • ಉ͡ڥΛ࠶ݱ͢Δͷ͕͍͠ •
BundlerͳͲΛͬͯCͷϥΠϒϥϦʹ ґଘ͢Δ͜ͱ
Docker • LinuxͷσΟετϦϏϡʔγϣϯڥ ͝ͱݻΊͯΠϝʔδԽ • /lib, /usr/bin, /etcͳͲͥΜͿ • Linux
NamespacesͰಠཱͨ͠ڥΛ ࡞ͬͯΠϝʔδΛల։
DockerࠔΔ͜ͱ • Docker Engineͷෆ҆ఆ͞ • ωοτϫʔΫ·ΘΓͷύϑΥʔϚϯεྼԽ • ίϯςφͷΰϛআ • ίϯςφͷແఀࢭσϓϩΠ
• ίϯςφͷϩάཧ • ίϯςφͷࢹ • ίϯςφͷσόοά • Docker Registryͷӡ༻
chroot
chroot ☓ Docker ͷΞΠσΞ EPDLFSQVMMNZTRM $0/5"*/&3@*% EPDLFSDSFBUFNZTRM EPDLFSFYQPSU$0/5"*/&3@*%PNZTRMUBS NZTRMUBSΛ.Z42-Λಈ͔͍ͨ͠ϗετίϐʔ
͢Δɻ UBSYG[WBSDPOUBJOFSTNZTRMNZTRMUBS TVEPDISPPUWBSDPOUBJOFSTNZTRMNZTRME
None
࠷ۙͷΞϓϦέʔγϣϯσϓϩΠ • git pull͕͍ • σϓϩΠαʔό͔Βͷrsync͍ • tarball σϓϩΠ •
ՌΛS3ͳͲʹஔ͠ɺσϓϩΠઌͰ s3 cp Ͱμϯϩʔυͯ͠ల։ • Serf/Consul • σϓϩΠαʔό͔ΒͷSSH͕͍ͨΊ
Droot
#VJME 4IJQ 3VO ESPPUSVO EPDLFSCVJME ESPPUFYQPSU EPDLFS EBFNPO 4UPSBHF 4
ESPPUEFQMPZ BXTTDQ BXTTDQ
%PDLFS %SPPU #VJME EPDLFSCVJME EPDLFSCVJME 3FHJTUSZ %PDLFS)VC %JTUSJCVUJPO ͳΜͰΑ͍ "NB[PO4
'JMF'PSNBU %PDLFSJNBHF ͳΜͰΑ͍ FYUBSH[ $POUBJOFS -JOVY /BNFTQBDFT DISPPU
$ droot export • DockerΠϝʔδͷϑΝΠϧγεςϜΛtarܗࣜͰग़ྗ • جຊ docker create &&
docker export • gzip / aws cli ͱͷύΠϓʹΑΓɺtar.gzԽͯ͠S3ʹஔ • ϑΝΠϧγεςϜʹdrootઐ༻ͷڥมϑΝΠϧ (/.drootenv) ΛࠐΉ ESPPUFYQPSUEPDLFSpMFTBQQcH[JQDR cBXTTDQTCVDLFUBQQUBSH[
$ droot deploy • ඪ४ೖྗ͔ΒtarΞʔΧΠϒΛಡΈࠐΈɺࢦఆ͠ ͨσΟϨΫτϦʹల։ • සൟʹߋ৽͞ΕΔίϯςφͷσϓϩΠΛఆ • rsync
mode ͱ symlink mode BXTTDQTCVDLFUBQQUBSH[c HVO[JQDRcESPPUEFQMPZSPPUWBSDPOUBJOFST
symlink ʹΑΔ atomic deploy • σϓϩΠࡁΈͷίϯςφ ڥΛࠩ͠ସ͑Δඞཁ͕͋Δ • https://gist.github.com/ datagrok/3807742#file-
symlink-replacement-md • symlink Λ rename(2)ɹ (mv -T) ͰΓସ͑Δ͜ͱ ʹΑΓΞτϛοΫʹσΟϨ ΫτϦΛࠩ͠ସ͑Δ ᵓᴷᴷBQQBQQENBJO ᵓᴷᴷBQQE ᵋᴷᴷNBJO ᵋᴷᴷCJO ᵋᴷᴷCPPU ᵋᴷᴷEFW ᵋᴷᴷCBDLVQ ᵋᴷᴷCJO ᵋᴷᴷCPPU ᵋᴷᴷEFW
$ droot run • ࢦఆͨ͠σΟϨΫτϦΛchroot jailͱ࣮ͯ͠ߦ • σόΠεϑΝΠϧͷ࡞ (/dev/null, /dev/zeroͳͲ)
• ϗετͷ /etc/group, /etc/resolve.confͳͲΛίϐʔ • bind mountͰϗετଆͷҙͷσΟϨΫτϦΛϚϯτ • Linux capabilities(7) ͰݖݶΛ੍ TVEPESPPUSVODQCJOEWBSMPHSPPU ɹɹWBSDPOUBJOFSTBQQDPNNBOE
chroot(2) • ϓϩηεͷϧʔτσΟϨΫτϦΛมߋ • ϓϩηεͷઈରύεͷ୳ࡧىͷมߋͷΈ • ϓϩηεΛੜͨ͠Γ͠ͳ͍ • ΧϨϯτσΟϨΫτϦͦͷ··ͳͷͰcrhootίʔ ϧޙʹchdir(“/“)͢Δ͜ͱ͕ଟ͍
• jailڥ֎ͷϑΝΠϧషΒΕͨγϯϘϦοΫϦϯ ΫΞΫηεͰ͖ͳ͍
BindϚϯτ • Linux 2.2͔Βಋೖ • σΟϨΫτϦϑΝΠϧΛଞͷҐஔϚϯτ • chroot jailڥ͔ΒϚϯτઌͷϑΝΠϧσΟϨ ΫτϦΞΫηεͰ͖Δ
• /var/containers/app/var/log ͱ͔ࢀর͢Δͷ໘ • mount -o bind /var/log /var/containers/app/var/log • ϗετͷ /var/log Λڞ༗
LinuxέʔύϏϦςΟ • chroot(2)ಛݖϓϩηεͰͳ͍ͱίʔϧͰ͖ͳ͍ • (ݫີʹCAP_SYS_CHROOT) • ͔͠͠ɺεʔύʔϢʔβͰಈ͔͢ͷෆ҆ • εʔύʔϢʔβͰಈ͔ͭͭ͠ɺcapabilities(7)Ͱඞཁͳ ݖݶҎ֎Λམͱ͓ͯ͘͠
• CAP_CHOWN, CAP_DAC_OVERRIDE, CAP_DAC_READ_SEARCH, CAP_FOWNER, CAP_SETGID, CAP_SETUID, CAP_NET_BIND_SERVICE ΛڐՄ
Problems
• Docker (NamespacesΛͬͨίϯςφ)΄Ͳͷɹ ϙʔλϏϦςΟͳ͍ • Dockerίϯςφͷڥม͕Ҿ͖ܧ͕Εͳ͍ • Dockerίϯςφ্ͷ user/group ͕σϓϩΠઌ
ϗετʹ͍ͳ͍ ϙʔλϏϦςΟͷ
• ڥมϑΝΠϧͱͯ͠ӬଓԽ͞Εͳ͍ͷͰɺ Ұ୴ϑΝΠϧʹอଘ • droot export ͰҰ୴Dockerίϯςφͱͯ͠ىಈ͞ ͔ͤͯΒ env ίϚϯυͷ࣮ߦ݁ՌΛ
/.drootenv ͱ ͯ͠อଘ • droot run Ͱ /.drootnenv ΛಡΈͩͯ͠ڥมΛ ෮ݩ • droot run —env ͰڥมͷՃɾ্ॻ͖Մೳ ڥมͷҾ͖ܧ͗
• User NamespacesͰ/etc/groupͳͲΛΈͯඞཁ ͳuser/grpupΛࣗಈ࡞ • ϓϩηεπϦʔߏΛ͔͑ͨ͘ͳ͍ͷͰɺclone(2) Ͱͳ͘ chroot(2) ޙʹ unshare(2)
͢Δ • clone(2) ͩͱࢠϓϩηεΛੜ͢ΔͨΊɺεʔ ύʔόΠβϓϩηεͷԼͰdroot runͨ͠ͱ͖ ʹɺγάφϧཧ͕͏·͍͔͘ͳ͍͔ user/groupͷࣗಈ࡞(ະ࣮)
• ͜Ε͓ͦΒ͘ PID Namespacesͷ • https://lwn.net/Articles/532748/ • NamespacesԼͷϓϩηε͕pid 1ͱͯ͠ৼΔ͏ඞཁ͕Ͱ ͯ͘Δ
• orphanϓϩηεͷճऩ͢Δඞཁ͕͋Δ
ίϯςφ ࣗͰ࡞ΕΔ
(PMBOH
• github.com/docker/docker/pkg • archive, devicemapper, fileutils, mount, symlink… • github.com/opencontainers/runc/libcontainer
• Linux Namespaces·ΘΓ • https://github.com/syndtr/gocapability • LinuxέʔύϏϦςΟ • github.com/docker/engine-api • Docker APIΫϥΠΞϯτ ίϯςφπʔϧ͚ύοέʔδ
• ࣗ࡞ͷίϯςφπʔϧ Drootͷഎܠͱ࣮ • Build, Ship, RunΛ࣮ݱ͢Δୈ̏ͷιϑτΣΞ • droot export,
droot deploy, droot run • DockerͰΠϝʔδΛ࡞ͬͯ chroot Ͱ࣮ߦ • ϙʔλϏϦςΟͷͱͦͷղܾ • ίϯςφࣗͰ࡞ΕΔ ·ͱΊ
github.com/yuuki/droot