Droot Internals - Speaker Deck

Slide 1

Slide 1 text

%SPPU*OUFSOBMT JEZ@VVLJ ୈ̕ճίϯςφܕԾ૝Խͷ৘ใަ׵ձ!෱Ԭ

Slide 2

Slide 2 text

id:y_uuki @y_uuk1 ͸ͯͳ!ژ౎ ΢ΣϒΦϖϨʔγϣϯΤϯδχΞ

Slide 3

Slide 3 text

IUUQZVVLJIBUFOBCMPHDPNFOUSZESPPU

Slide 4

Slide 4 text

TL;DR • ιϑτ΢ΣΞґଘ஍ࠈͷղܾͷͨΊʹ DockerΛ࢖͍͍ͨ • ຊ൪؀ڥͰDockerΛӡ༻͢Δͷ͸ͭΒ͍ • ʮBuild, Ship, Runʯͱ͍͏ίϯηϓτ͸޷͖ • DockerΠϝʔδΛS3Λܦ༝ͯ͠഑෍͠ɺ chrootͰ࣮ߦ͢Δख๏ͷఏҊ

Slide 5

Slide 5 text

Ծ૝Խٕज़ )8Ծ૝Խ 04Ծ૝Խ ,7. 9FO ʜ γεςϜίϯςφ -9$ ΞϓϦέʔγϣϯ ίϯςφ %PDLFS

Slide 6

Slide 6 text

ͳͥ(๻͸)DockerΛ࢖͏ͷ͔ • ✘ VMΑΓߴ଎ • ✘ Πϛϡʔλϒϧ • ✘ Φʔτεέʔϧ • ˚ ϙʔλϏϦςΟ • ○ ϓϩάϥϚϒϧͳϗετ؀ڥ • ○ ιϑτ΢ΣΞґଘ஍ࠈͷղܾ

Slide 7

Slide 7 text

ιϑτ΢ΣΞґଘ஍ࠈ • ͋Διϑτ΢ΣΞ͸͍͍ͨͯෳ਺ͷιϑτ ΢ΣΞʹґଘ͢Δ • ґଘઌͷιϑτ΢ΣΞ΋·ͨෳ਺ͷιϑτ ΢ΣΞʹґଘ͢Δ • ಉ͡؀ڥΛ࠶ݱ͢Δͷ͕೉͍͠ • BundlerͳͲΛ࢖ͬͯ΋CͷϥΠϒϥϦʹ ґଘ͢Δ͜ͱ΋

Slide 8

Slide 8 text

Docker • LinuxͷσΟετϦϏϡʔγϣϯ؀ڥ ͝ͱݻΊͯΠϝʔδԽ • /lib, /usr/bin, /etcͳͲͥΜͿ • Linux NamespacesͰಠཱͨ͠؀ڥΛ ࡞ͬͯΠϝʔδΛల։

Slide 9

Slide 9 text

DockerࠔΔ͜ͱ • Docker Engineͷෆ҆ఆ͞ • ωοτϫʔΫ·ΘΓͷύϑΥʔϚϯεྼԽ • ίϯςφͷΰϛ૟আ • ίϯςφͷແఀࢭσϓϩΠ • ίϯςφͷϩά؅ཧ • ίϯςφͷ؂ࢹ • ίϯςφͷσόοά • Docker Registryͷӡ༻

Slide 10

Slide 10 text

chroot

Slide 11

Slide 11 text

chroot ☓ Docker ͷΞΠσΞ EPDLFSQVMMNZTRM $0/5"*/&3@*% EPDLFSDSFBUFNZTRM EPDLFSFYQPSU$0/5"*/&3@*%PNZTRMUBS NZTRMUBSΛ.Z42-Λಈ͔͍ͨ͠ϗετ΁ίϐʔ ͢Δɻ UBSYG[WBSDPOUBJOFSTNZTRMNZTRMUBS TVEPDISPPUWBSDPOUBJOFSTNZTRMNZTRME

Slide 12

Slide 12 text

No content

Slide 13

Slide 13 text

࠷ۙͷΞϓϦέʔγϣϯσϓϩΠ • git pull͕஗͍ • σϓϩΠαʔό͔Βͷrsync΋஗͍ • tarball σϓϩΠ • ੒Ռ෺ΛS3ͳͲʹ഑ஔ͠ɺσϓϩΠઌͰ s3 cp Ͱμ΢ϯϩʔυͯ͠ల։ • Serf/Consul • σϓϩΠαʔό͔ΒͷSSH͕஗͍ͨΊ

Slide 14

Slide 14 text

Droot

Slide 15

Slide 15 text

#VJME 4IJQ 3VO ESPPUSVO EPDLFSCVJME ESPPUFYQPSU EPDLFS EBFNPO 4UPSBHF 4 ESPPUEFQMPZ BXTTDQ BXTTDQ

Slide 16

Slide 16 text

%PDLFS %SPPU #VJME EPDLFSCVJME EPDLFSCVJME 3FHJTUSZ %PDLFS)VC %JTUSJCVUJPO ͳΜͰ΋Α͍  "NB[PO4 'JMF'PSNBU %PDLFSJNBHF ͳΜͰ΋Α͍ FYUBSH[ $POUBJOFS -JOVY /BNFTQBDFT DISPPU

Slide 17

Slide 17 text

$ droot export • DockerΠϝʔδͷϑΝΠϧγεςϜΛtarܗࣜͰग़ྗ • جຊ͸ docker create && docker export • gzip / aws cli ͱͷύΠϓʹΑΓɺtar.gzԽͯ͠S3ʹ഑ஔ • ϑΝΠϧγεςϜʹdrootઐ༻ͷ؀ڥม਺ϑΝΠϧ (/.drootenv) Λ࢓ࠐΉ ESPPUFYQPSUEPDLFSpMFTBQQcH[JQDR cBXTTDQTCVDLFUBQQUBSH[

Slide 18

Slide 18 text

$ droot deploy • ඪ४ೖྗ͔ΒtarΞʔΧΠϒΛಡΈࠐΈɺࢦఆ͠ ͨσΟϨΫτϦʹల։ • සൟʹߋ৽͞ΕΔίϯςφͷσϓϩΠΛ૝ఆ • rsync mode ͱ symlink mode BXTTDQTCVDLFUBQQUBSH[c HVO[JQDRcESPPUEFQMPZSPPUWBSDPOUBJOFST

Slide 19

Slide 19 text

symlink ʹΑΔ atomic deploy • σϓϩΠࡁΈͷίϯςφ؀ ڥΛࠩ͠ସ͑Δඞཁ͕͋Δ • https://gist.github.com/ datagrok/3807742#ﬁle- symlink-replacement-md • symlink Λ rename(2)ɹ (mv -T) Ͱ੾Γସ͑Δ͜ͱ ʹΑΓΞτϛοΫʹσΟϨ ΫτϦΛࠩ͠ସ͑Δ ᵓᴷᴷBQQBQQENBJO ᵓᴷᴷBQQE ᵋᴷᴷNBJO ᵋᴷᴷCJO ᵋᴷᴷCPPU ᵋᴷᴷEFW ᵋᴷᴷCBDLVQ ᵋᴷᴷCJO ᵋᴷᴷCPPU ᵋᴷᴷEFW

Slide 20

Slide 20 text

$ droot run • ࢦఆͨ͠σΟϨΫτϦΛchroot jailͱ࣮ͯ͠ߦ • σόΠεϑΝΠϧͷ࡞੒ (/dev/null, /dev/zeroͳͲ) • ϗετͷ /etc/group, /etc/resolve.confͳͲΛίϐʔ • bind mountͰϗετଆͷ೚ҙͷσΟϨΫτϦΛϚ΢ϯτ • Linux capabilities(7) ͰݖݶΛ཈੍ TVEPESPPUSVODQCJOEWBSMPHSPPU ɹɹWBSDPOUBJOFSTBQQDPNNBOE

Slide 21

Slide 21 text

chroot(2) • ϓϩηεͷϧʔτσΟϨΫτϦΛมߋ • ϓϩηεͷઈରύεͷ୳ࡧى఺ͷมߋͷΈ • ϓϩηεΛੜ੒ͨ͠Γ͠ͳ͍ • ΧϨϯτσΟϨΫτϦ͸ͦͷ··ͳͷͰcrhootίʔ ϧޙʹchdir(“/“)͢Δ͜ͱ͕ଟ͍ • jail؀ڥ֎ͷϑΝΠϧ΁షΒΕͨγϯϘϦοΫϦϯ Ϋ΁͸ΞΫηεͰ͖ͳ͍

Slide 22

Slide 22 text

BindϚ΢ϯτ • Linux 2.2͔Βಋೖ • σΟϨΫτϦ΍ϑΝΠϧΛଞͷҐஔ΁Ϛ΢ϯτ • chroot jail؀ڥ಺͔ΒϚ΢ϯτઌͷϑΝΠϧ΍σΟϨ ΫτϦ΁ΞΫηεͰ͖Δ • /var/containers/app/var/log ͱ͔ࢀর͢Δͷ໘౗ • mount -o bind /var/log /var/containers/app/var/log • ϗετͷ /var/log Λڞ༗

Slide 23

Slide 23 text

LinuxέʔύϏϦςΟ • chroot(2)͸ಛݖϓϩηεͰͳ͍ͱίʔϧͰ͖ͳ͍ • (ݫີʹ͸CAP_SYS_CHROOT) • ͔͠͠ɺεʔύʔϢʔβͰಈ͔͢ͷ͸ෆ҆ • εʔύʔϢʔβͰಈ͔ͭͭ͠ɺcapabilities(7)Ͱඞཁͳ ݖݶҎ֎Λམͱ͓ͯ͘͠ • CAP_CHOWN, CAP_DAC_OVERRIDE, CAP_DAC_READ_SEARCH, CAP_FOWNER, CAP_SETGID, CAP_SETUID, CAP_NET_BIND_SERVICE ΛڐՄ

Slide 24

Slide 24 text

Problems

Slide 25

Slide 25 text

• Docker (NamespacesΛ࢖ͬͨίϯςφ)΄Ͳͷɹ ϙʔλϏϦςΟ͸ͳ͍ • Dockerίϯςφͷ؀ڥม਺͕Ҿ͖ܧ͕Εͳ͍ • Dockerίϯςφ্ͷ user/group ͕σϓϩΠઌ ϗετʹ͍ͳ͍ ϙʔλϏϦςΟͷ໰୊

Slide 26

Slide 26 text

• ؀ڥม਺͸ϑΝΠϧͱͯ͠ӬଓԽ͞Εͳ͍ͷͰɺ Ұ୴ϑΝΠϧʹอଘ • droot export ͰҰ୴Dockerίϯςφͱͯ͠ىಈ͞ ͔ͤͯΒ env ίϚϯυͷ࣮ߦ݁ՌΛ /.drootenv ͱ ͯ͠อଘ • droot run Ͱ /.drootnenv ΛಡΈͩͯ͠؀ڥม਺Λ ෮ݩ • droot run —env Ͱ؀ڥม਺ͷ௥Ճɾ্ॻ͖Մೳ ؀ڥม਺ͷҾ͖ܧ͗

Slide 27

Slide 27 text

• User Namespaces಺Ͱ/etc/groupͳͲΛΈͯඞཁ ͳuser/grpupΛࣗಈ࡞੒ • ϓϩηεπϦʔߏ଄Λ͔͑ͨ͘ͳ͍ͷͰɺclone(2) Ͱ͸ͳ͘ chroot(2) ޙʹ unshare(2) ͢Δ • clone(2) ͩͱࢠϓϩηεΛੜ੒͢ΔͨΊɺεʔ ύʔόΠβϓϩηεͷ഑ԼͰdroot runͨ͠ͱ͖ ʹɺγάφϧ؅ཧ͕͏·͍͔͘ͳ͍͔΋ user/groupͷࣗಈ࡞੒(ະ࣮૷)

Slide 28

Slide 28 text

• ͜Ε͸͓ͦΒ͘ PID Namespacesͷ࿩ • https://lwn.net/Articles/532748/ • Namespaces௚Լͷϓϩηε͕pid 1ͱͯ͠ৼΔ෣͏ඞཁ͕Ͱ ͯ͘Δ • orphanϓϩηεͷճऩ͢Δඞཁ͕͋Δ

Slide 29

Slide 29 text

ίϯςφ͸ ࣗ෼Ͱ࡞ΕΔ

Slide 30

Slide 30 text

(PMBOH

Slide 31

Slide 31 text

• github.com/docker/docker/pkg • archive, devicemapper, ﬁleutils, mount, symlink… • github.com/opencontainers/runc/libcontainer • Linux Namespaces·ΘΓ • https://github.com/syndtr/gocapability • LinuxέʔύϏϦςΟ • github.com/docker/engine-api • Docker APIΫϥΠΞϯτ ίϯςφπʔϧ޲͚ύοέʔδ

Slide 32

Slide 32 text

• ࣗ࡞ͷίϯςφπʔϧ Drootͷഎܠͱ࣮૷ • Build, Ship, RunΛ࣮ݱ͢Δୈ̏ͷιϑτ΢ΣΞ • droot export, droot deploy, droot run • DockerͰΠϝʔδΛ࡞ͬͯ chroot Ͱ࣮ߦ • ϙʔλϏϦςΟͷ໰୊ͱͦͷղܾ • ίϯςφ͸ࣗ෼Ͱ࡞ΕΔ ·ͱΊ

Slide 33

Slide 33 text

github.com/yuuki/droot