id:y_uuki 第9回 コンテナ型仮想化の情報交換会@福岡
%SPPU*OUFSOBMTJEZ@VVLJୈ̕ճίϯςφܕԾԽͷใަձ!Ԭ
View Slide
id:y_uuki@y_uuk1ͯͳ!ژΣϒΦϖϨʔγϣϯΤϯδχΞ
IUUQZVVLJIBUFOBCMPHDPNFOUSZESPPU
TL;DR• ιϑτΣΞґଘࠈͷղܾͷͨΊʹDockerΛ͍͍ͨ• ຊ൪ڥͰDockerΛӡ༻͢ΔͷͭΒ͍• ʮBuild, Ship, Runʯͱ͍͏ίϯηϓτ͖• DockerΠϝʔδΛS3Λܦ༝ͯ͠͠ɺchrootͰ࣮ߦ͢Δख๏ͷఏҊ
ԾԽٕज़)8ԾԽ 04ԾԽ,7.9FOʜγεςϜίϯςφ-9$ΞϓϦέʔγϣϯίϯςφ%PDLFS
ͳͥ()DockerΛ͏ͷ͔• ✘ VMΑΓߴ• ✘ Πϛϡʔλϒϧ• ✘ Φʔτεέʔϧ• ˚ ϙʔλϏϦςΟ• ○ ϓϩάϥϚϒϧͳϗετڥ• ○ ιϑτΣΞґଘࠈͷղܾ
ιϑτΣΞґଘࠈ• ͋ΔιϑτΣΞ͍͍ͨͯෳͷιϑτΣΞʹґଘ͢Δ• ґଘઌͷιϑτΣΞ·ͨෳͷιϑτΣΞʹґଘ͢Δ• ಉ͡ڥΛ࠶ݱ͢Δͷ͕͍͠• BundlerͳͲΛͬͯCͷϥΠϒϥϦʹґଘ͢Δ͜ͱ
Docker• LinuxͷσΟετϦϏϡʔγϣϯڥ͝ͱݻΊͯΠϝʔδԽ• /lib, /usr/bin, /etcͳͲͥΜͿ• Linux NamespacesͰಠཱͨ͠ڥΛ࡞ͬͯΠϝʔδΛల։
DockerࠔΔ͜ͱ• Docker Engineͷෆ҆ఆ͞• ωοτϫʔΫ·ΘΓͷύϑΥʔϚϯεྼԽ• ίϯςφͷΰϛআ• ίϯςφͷແఀࢭσϓϩΠ• ίϯςφͷϩάཧ• ίϯςφͷࢹ• ίϯςφͷσόοά• Docker Registryͷӡ༻
chroot
chroot ☓ Docker ͷΞΠσΞEPDLFSQVMMNZTRM$0/5"*/&3@*% EPDLFSDSFBUFNZTRMEPDLFSFYQPSU$0/5"*/&3@*%PNZTRMUBSNZTRMUBSΛ.Z42-Λಈ͔͍ͨ͠ϗετίϐʔ͢ΔɻUBSYG[WBSDPOUBJOFSTNZTRMNZTRMUBSTVEPDISPPUWBSDPOUBJOFSTNZTRMNZTRME
࠷ۙͷΞϓϦέʔγϣϯσϓϩΠ• git pull͕͍• σϓϩΠαʔό͔Βͷrsync͍• tarball σϓϩΠ• ՌΛS3ͳͲʹஔ͠ɺσϓϩΠઌͰ s3 cpͰμϯϩʔυͯ͠ల։• Serf/Consul• σϓϩΠαʔό͔ΒͷSSH͕͍ͨΊ
Droot
#VJME4IJQ3VO ESPPUSVOEPDLFSCVJMEESPPUFYQPSUEPDLFSEBFNPO4UPSBHF4ESPPUEFQMPZBXTTDQBXTTDQ
%PDLFS %SPPU#VJME EPDLFSCVJME EPDLFSCVJME3FHJTUSZ%PDLFS)VC%JTUSJCVUJPOͳΜͰΑ͍ "NB[PO4'JMF'PSNBU %PDLFSJNBHFͳΜͰΑ͍FYUBSH[$POUBJOFS-JOVY/BNFTQBDFTDISPPU
$ droot export• DockerΠϝʔδͷϑΝΠϧγεςϜΛtarܗࣜͰग़ྗ• جຊ docker create && docker export• gzip / aws cli ͱͷύΠϓʹΑΓɺtar.gzԽͯ͠S3ʹஔ• ϑΝΠϧγεςϜʹdrootઐ༻ͷڥมϑΝΠϧ(/.drootenv) ΛࠐΉESPPUFYQPSUEPDLFSpMFTBQQcH[JQDRcBXTTDQTCVDLFUBQQUBSH[
$ droot deploy• ඪ४ೖྗ͔ΒtarΞʔΧΠϒΛಡΈࠐΈɺࢦఆͨ͠σΟϨΫτϦʹల։• සൟʹߋ৽͞ΕΔίϯςφͷσϓϩΠΛఆ• rsync mode ͱ symlink modeBXTTDQTCVDLFUBQQUBSH[cHVO[JQDRcESPPUEFQMPZSPPUWBSDPOUBJOFST
symlink ʹΑΔ atomic deploy• σϓϩΠࡁΈͷίϯςφڥΛࠩ͠ସ͑Δඞཁ͕͋Δ• https://gist.github.com/datagrok/3807742#file-symlink-replacement-md• symlink Λ rename(2)ɹ(mv -T) ͰΓସ͑Δ͜ͱʹΑΓΞτϛοΫʹσΟϨΫτϦΛࠩ͠ସ͑ΔᵓᴷᴷBQQBQQENBJOᵓᴷᴷBQQEᵋᴷᴷNBJOᵋᴷᴷCJOᵋᴷᴷCPPUᵋᴷᴷEFWᵋᴷᴷCBDLVQᵋᴷᴷCJOᵋᴷᴷCPPUᵋᴷᴷEFW
$ droot run• ࢦఆͨ͠σΟϨΫτϦΛchroot jailͱ࣮ͯ͠ߦ• σόΠεϑΝΠϧͷ࡞ (/dev/null, /dev/zeroͳͲ)• ϗετͷ /etc/group, /etc/resolve.confͳͲΛίϐʔ• bind mountͰϗετଆͷҙͷσΟϨΫτϦΛϚϯτ• Linux capabilities(7) ͰݖݶΛ੍TVEPESPPUSVODQCJOEWBSMPHSPPUɹɹWBSDPOUBJOFSTBQQDPNNBOE
chroot(2)• ϓϩηεͷϧʔτσΟϨΫτϦΛมߋ• ϓϩηεͷઈରύεͷ୳ࡧىͷมߋͷΈ• ϓϩηεΛੜͨ͠Γ͠ͳ͍• ΧϨϯτσΟϨΫτϦͦͷ··ͳͷͰcrhootίʔϧޙʹchdir(“/“)͢Δ͜ͱ͕ଟ͍• jailڥ֎ͷϑΝΠϧషΒΕͨγϯϘϦοΫϦϯΫΞΫηεͰ͖ͳ͍
BindϚϯτ• Linux 2.2͔Βಋೖ• σΟϨΫτϦϑΝΠϧΛଞͷҐஔϚϯτ• chroot jailڥ͔ΒϚϯτઌͷϑΝΠϧσΟϨΫτϦΞΫηεͰ͖Δ• /var/containers/app/var/log ͱ͔ࢀর͢Δͷ໘• mount -o bind /var/log /var/containers/app/var/log• ϗετͷ /var/log Λڞ༗
LinuxέʔύϏϦςΟ• chroot(2)ಛݖϓϩηεͰͳ͍ͱίʔϧͰ͖ͳ͍• (ݫີʹCAP_SYS_CHROOT)• ͔͠͠ɺεʔύʔϢʔβͰಈ͔͢ͷෆ҆• εʔύʔϢʔβͰಈ͔ͭͭ͠ɺcapabilities(7)ͰඞཁͳݖݶҎ֎Λམͱ͓ͯ͘͠• CAP_CHOWN, CAP_DAC_OVERRIDE,CAP_DAC_READ_SEARCH, CAP_FOWNER,CAP_SETGID, CAP_SETUID, CAP_NET_BIND_SERVICEΛڐՄ
Problems
• Docker (NamespacesΛͬͨίϯςφ)΄ͲͷɹϙʔλϏϦςΟͳ͍• Dockerίϯςφͷڥม͕Ҿ͖ܧ͕Εͳ͍• Dockerίϯςφ্ͷ user/group ͕σϓϩΠઌϗετʹ͍ͳ͍ϙʔλϏϦςΟͷ
• ڥมϑΝΠϧͱͯ͠ӬଓԽ͞Εͳ͍ͷͰɺҰ୴ϑΝΠϧʹอଘ• droot export ͰҰ୴Dockerίϯςφͱͯ͠ىಈ͔ͤͯ͞Β env ίϚϯυͷ࣮ߦ݁ՌΛ /.drootenv ͱͯ͠อଘ• droot run Ͱ /.drootnenv ΛಡΈͩͯ͠ڥมΛ෮ݩ• droot run —env ͰڥมͷՃɾ্ॻ͖ՄೳڥมͷҾ͖ܧ͗
• User NamespacesͰ/etc/groupͳͲΛΈͯඞཁͳuser/grpupΛࣗಈ࡞• ϓϩηεπϦʔߏΛ͔͑ͨ͘ͳ͍ͷͰɺclone(2)Ͱͳ͘ chroot(2) ޙʹ unshare(2) ͢Δ• clone(2) ͩͱࢠϓϩηεΛੜ͢ΔͨΊɺεʔύʔόΠβϓϩηεͷԼͰdroot runͨ͠ͱ͖ʹɺγάφϧཧ͕͏·͍͔͘ͳ͍͔user/groupͷࣗಈ࡞(ະ࣮)
• ͜Ε͓ͦΒ͘ PID Namespacesͷ• https://lwn.net/Articles/532748/• NamespacesԼͷϓϩηε͕pid 1ͱͯ͠ৼΔ͏ඞཁ͕Ͱͯ͘Δ• orphanϓϩηεͷճऩ͢Δඞཁ͕͋Δ
ίϯςφࣗͰ࡞ΕΔ
(PMBOH
• github.com/docker/docker/pkg• archive, devicemapper, fileutils, mount, symlink…• github.com/opencontainers/runc/libcontainer• Linux Namespaces·ΘΓ• https://github.com/syndtr/gocapability• LinuxέʔύϏϦςΟ• github.com/docker/engine-api• Docker APIΫϥΠΞϯτίϯςφπʔϧ͚ύοέʔδ
• ࣗ࡞ͷίϯςφπʔϧ Drootͷഎܠͱ࣮• Build, Ship, RunΛ࣮ݱ͢Δୈ̏ͷιϑτΣΞ• droot export, droot deploy, droot run• DockerͰΠϝʔδΛ࡞ͬͯ chroot Ͱ࣮ߦ• ϙʔλϏϦςΟͷͱͦͷղܾ• ίϯςφࣗͰ࡞ΕΔ·ͱΊ
github.com/yuuki/droot