Droot Internals

Transcript

1. 1.

   %SPPU
   *OUFSOBMT JEZ@VVLJ
   ୈ̕ճ
   ίϯςφܕԾ૝Խͷ৘ใަ׵ձ!෱Ԭ

2. 2.

   id:y_uuki @y_uuk1 ͸ͯͳ!ژ౎
   ΢ΣϒΦϖϨʔγϣϯΤϯδχΞ

3. 3.

   IUUQZVVLJIBUFOBCMPHDPNFOUSZESPPU

4. 4.

   TL;DR • ιϑτ΢ΣΞґଘ஍ࠈͷղܾͷͨΊʹ DockerΛ࢖͍͍ͨ • ຊ൪؀ڥͰDockerΛӡ༻͢Δͷ͸ͭΒ͍ • ʮBuild, Ship, Runʯͱ͍͏ίϯηϓτ͸޷͖

   • DockerΠϝʔδΛS3Λܦ༝ͯ͠഑෍͠ɺ chrootͰ࣮ߦ͢Δख๏ͷఏҊ
5. 5.

   Ծ૝Խٕज़ )8Ծ૝Խ 04Ծ૝Խ ,7.
   9FO
   ʜ γεςϜίϯςφ
   -9$ ΞϓϦέʔγϣϯ ίϯςφ
   

   %PDLFS
6. 6.

   ͳͥ(๻͸)DockerΛ࢖͏ͷ͔ • ✘ VMΑΓߴ଎ • ✘ Πϛϡʔλϒϧ • ✘ Φʔτεέʔϧ

   • ˚ ϙʔλϏϦςΟ • ◦ ϓϩάϥϚϒϧͳϗετ؀ڥ • ◦ ιϑτ΢ΣΞґଘ஍ࠈͷղܾ
7. 7.

   ιϑτ΢ΣΞґଘ஍ࠈ • ͋Διϑτ΢ΣΞ͸͍͍ͨͯෳ਺ͷιϑτ ΢ΣΞʹґଘ͢Δ • ґଘઌͷιϑτ΢ΣΞ΋·ͨෳ਺ͷιϑτ ΢ΣΞʹґଘ͢Δ • ಉ͡؀ڥΛ࠶ݱ͢Δͷ͕೉͍͠ •

   BundlerͳͲΛ࢖ͬͯ΋CͷϥΠϒϥϦʹ ґଘ͢Δ͜ͱ΋
8. 8.

   Docker • LinuxͷσΟετϦϏϡʔγϣϯ؀ڥ ͝ͱݻΊͯΠϝʔδԽ • /lib, /usr/bin, /etcͳͲͥΜͿ • Linux

   NamespacesͰಠཱͨ͠؀ڥΛ ࡞ͬͯΠϝʔδΛల։
9. 9.

   DockerࠔΔ͜ͱ • Docker Engineͷෆ҆ఆ͞ • ωοτϫʔΫ·ΘΓͷύϑΥʔϚϯεྼԽ • ίϯςφͷΰϛ૟আ • ίϯςφͷແఀࢭσϓϩΠ

   • ίϯςφͷϩά؅ཧ • ίϯςφͷ؂ࢹ • ίϯςφͷσόοά • Docker Registryͷӡ༻
10. 10.

    chroot

11. 11.

    chroot ☓ Docker ͷΞΠσΞ 
    EPDLFS
    QVMM
    NZTRM
    
    $0/5"*/&3@*% EPDLFS
    DSFBUF
    NZTRM
    
    EPDLFS
    FYQPSU
    $0/5"*/&3@*%
    P
    NZTRMUBS
    
    NZTRMUBS
    Λ.Z42-Λಈ͔͍ͨ͠ϗετ΁ίϐʔ

    ͢Δɻ
    
    UBS
    YG[
    WBSDPOUBJOFSTNZTRMNZTRMUBS
    
    
    TVEP
    DISPPU
    WBSDPOUBJOFSTNZTRM
    NZTRME
12. 12.

    None
13. 13.

    ࠷ۙͷΞϓϦέʔγϣϯσϓϩΠ • git pull͕஗͍ • σϓϩΠαʔό͔Βͷrsync΋஗͍ • tarball σϓϩΠ •

    ੒Ռ෺ΛS3ͳͲʹ഑ஔ͠ɺσϓϩΠઌͰ s3 cp Ͱμ΢ϯϩʔυͯ͠ల։ • Serf/Consul • σϓϩΠαʔό͔ΒͷSSH͕஗͍ͨΊ
14. 14.

    Droot

15. 15.

    #VJME 4IJQ 3VO ESPPU
    SVO
    EPDLFS
    CVJME ESPPU
    FYQPSU EPDLFS
    EBFNPO 4UPSBHF
    
    4

    ESPPU
    
    EFQMPZ BXT
    T
    DQ BXT
    T
    DQ
16. 16.

    %PDLFS %SPPU #VJME EPDLFS
    CVJME EPDLFS
    CVJME 3FHJTUSZ %PDLFS)VC %JTUSJCVUJPO ͳΜͰ΋Α͍
 "NB[PO
    4

    'JMF
    'PSNBU %PDLFS
    JNBHF ͳΜͰ΋Α͍
    FY
    UBSH[ $POUBJOFS -JOVY
    /BNFTQBDFT DISPPU
17. 17.

    $ droot export • DockerΠϝʔδͷϑΝΠϧγεςϜΛtarܗࣜͰग़ྗ • جຊ͸ docker create &&

    docker export • gzip / aws cli ͱͷύΠϓʹΑΓɺtar.gzԽͯ͠S3ʹ഑ஔ • ϑΝΠϧγεςϜʹdrootઐ༻ͷ؀ڥม਺ϑΝΠϧ (/.drootenv) Λ࢓ࠐΉ 
    ESPPU
    FYQPSU
    EPDLFSpMFTBQQ
    c
    H[JQ
    DR
    
    c
    BXT
    T
    DQ
    
    TCVDLFUBQQUBSH[
18. 18.

    $ droot deploy • ඪ४ೖྗ͔ΒtarΞʔΧΠϒΛಡΈࠐΈɺࢦఆ͠ ͨσΟϨΫτϦʹల։ • සൟʹߋ৽͞ΕΔίϯςφͷσϓϩΠΛ૝ఆ • rsync

    mode ͱ symlink mode 
    BXT
    T
    DQ
    TCVDLFUBQQUBSH[
    
    c
    HVO[JQ
    DR
    c
    ESPPU
    EFQMPZ
    SPPU
    WBSDPOUBJOFST
19. 19.

    symlink ʹΑΔ atomic deploy • σϓϩΠࡁΈͷίϯςφ؀ ڥΛࠩ͠ସ͑Δඞཁ͕͋Δ • https://gist.github.com/ datagrok/3807742#file-

    symlink-replacement-md • symlink Λ rename(2)ɹ (mv -T) Ͱ੾Γସ͑Δ͜ͱ ʹΑΓΞτϛοΫʹσΟϨ ΫτϦΛࠩ͠ସ͑Δ 
    ᵓᴷᴷ
    BQQ
    
    BQQENBJO
    ᵓᴷᴷ
    BQQE
    
    
    
    
    ᵋᴷᴷ
    NBJO
    
    
    
    
    
    
    
    
    ᵋᴷᴷ
    CJO
    
    
    
    
    
    
    
    
    ᵋᴷᴷ
    CPPU
    
    
    
    
    
    
    
    
    ᵋᴷᴷ
    EFW
    
    
    
    
    
    
    
    
    
    
    
    
    ᵋᴷᴷ
    CBDLVQ
    
    
    
    
    
    
    
    
    ᵋᴷᴷ
    CJO
    
    
    
    
    
    
    
    
    ᵋᴷᴷ
    CPPU
    
    
    
    
    
    
    
    
    ᵋᴷᴷ
    EFW
    
    
    
    
    
    
    
    
    
20. 20.

    $ droot run • ࢦఆͨ͠σΟϨΫτϦΛchroot jailͱ࣮ͯ͠ߦ • σόΠεϑΝΠϧͷ࡞੒ (/dev/null, /dev/zeroͳͲ)

    • ϗετͷ /etc/group, /etc/resolve.confͳͲΛίϐʔ • bind mountͰϗετଆͷ೚ҙͷσΟϨΫτϦΛϚ΢ϯτ • Linux capabilities(7) ͰݖݶΛ཈੍ 
    TVEP
    ESPPU
    SVO
    DQ
    CJOE
    WBSMPH
    SPPU
    ɹɹWBSDPOUBJOFSTBQQ
    
    DPNNBOE
21. 21.

    chroot(2) • ϓϩηεͷϧʔτσΟϨΫτϦΛมߋ • ϓϩηεͷઈରύεͷ୳ࡧى఺ͷมߋͷΈ • ϓϩηεΛੜ੒ͨ͠Γ͠ͳ͍ • ΧϨϯτσΟϨΫτϦ͸ͦͷ··ͳͷͰcrhootίʔ ϧޙʹchdir(“/“)͢Δ͜ͱ͕ଟ͍

    • jail؀ڥ֎ͷϑΝΠϧ΁షΒΕͨγϯϘϦοΫϦϯ Ϋ΁͸ΞΫηεͰ͖ͳ͍
22. 22.

    BindϚ΢ϯτ • Linux 2.2͔Βಋೖ • σΟϨΫτϦ΍ϑΝΠϧΛଞͷҐஔ΁Ϛ΢ϯτ • chroot jail؀ڥ಺͔ΒϚ΢ϯτઌͷϑΝΠϧ΍σΟϨ ΫτϦ΁ΞΫηεͰ͖Δ

    • /var/containers/app/var/log ͱ͔ࢀর͢Δͷ໘౗ • mount -o bind /var/log /var/containers/app/var/log • ϗετͷ /var/log Λڞ༗
23. 23.

    LinuxέʔύϏϦςΟ • chroot(2)͸ಛݖϓϩηεͰͳ͍ͱίʔϧͰ͖ͳ͍ • (ݫີʹ͸CAP_SYS_CHROOT) • ͔͠͠ɺεʔύʔϢʔβͰಈ͔͢ͷ͸ෆ҆ • εʔύʔϢʔβͰಈ͔ͭͭ͠ɺcapabilities(7)Ͱඞཁͳ ݖݶҎ֎Λམͱ͓ͯ͘͠

    • CAP_CHOWN, CAP_DAC_OVERRIDE, CAP_DAC_READ_SEARCH, CAP_FOWNER, CAP_SETGID, CAP_SETUID, CAP_NET_BIND_SERVICE ΛڐՄ
24. 24.

    Problems

25. 25.

    • Docker (NamespacesΛ࢖ͬͨίϯςφ)΄Ͳͷɹ ϙʔλϏϦςΟ͸ͳ͍ • Dockerίϯςφͷ؀ڥม਺͕Ҿ͖ܧ͕Εͳ͍ • Dockerίϯςφ্ͷ user/group ͕σϓϩΠઌ

    ϗετʹ͍ͳ͍ ϙʔλϏϦςΟͷ໰୊
26. 26.

    • ؀ڥม਺͸ϑΝΠϧͱͯ͠ӬଓԽ͞Εͳ͍ͷͰɺ Ұ୴ϑΝΠϧʹอଘ • droot export ͰҰ୴Dockerίϯςφͱͯ͠ىಈ͞ ͔ͤͯΒ env ίϚϯυͷ࣮ߦ݁ՌΛ

    /.drootenv ͱ ͯ͠อଘ • droot run Ͱ /.drootnenv ΛಡΈͩͯ͠؀ڥม਺Λ ෮ݩ • droot run —env Ͱ؀ڥม਺ͷ௥Ճɾ্ॻ͖Մೳ ؀ڥม਺ͷҾ͖ܧ͗
27. 27.

    • User Namespaces಺Ͱ/etc/groupͳͲΛΈͯඞཁ ͳuser/grpupΛࣗಈ࡞੒ • ϓϩηεπϦʔߏ଄Λ͔͑ͨ͘ͳ͍ͷͰɺclone(2) Ͱ͸ͳ͘ chroot(2) ޙʹ unshare(2)

    ͢Δ • clone(2) ͩͱࢠϓϩηεΛੜ੒͢ΔͨΊɺεʔ ύʔόΠβϓϩηεͷ഑ԼͰdroot runͨ͠ͱ͖ ʹɺγάφϧ؅ཧ͕͏·͍͔͘ͳ͍͔΋ user/groupͷࣗಈ࡞੒(ະ࣮૷)
28. 28.

    • ͜Ε͸͓ͦΒ͘ PID Namespacesͷ࿩ • https://lwn.net/Articles/532748/ • Namespaces௚Լͷϓϩηε͕pid 1ͱͯ͠ৼΔ෣͏ඞཁ͕Ͱ ͯ͘Δ

    • orphanϓϩηεͷճऩ͢Δඞཁ͕͋Δ
29. 29.

    ίϯςφ͸ ࣗ෼Ͱ࡞ΕΔ

30. 30.

    (PMBOH

31. 31.

    • github.com/docker/docker/pkg • archive, devicemapper, fileutils, mount, symlink… • github.com/opencontainers/runc/libcontainer

    • Linux Namespaces·ΘΓ • https://github.com/syndtr/gocapability • LinuxέʔύϏϦςΟ • github.com/docker/engine-api • Docker APIΫϥΠΞϯτ ίϯςφπʔϧ޲͚ύοέʔδ
32. 32.

    • ࣗ࡞ͷίϯςφπʔϧ Drootͷഎܠͱ࣮૷ • Build, Ship, RunΛ࣮ݱ͢Δୈ̏ͷιϑτ΢ΣΞ • droot export,

    droot deploy, droot run • DockerͰΠϝʔδΛ࡞ͬͯ chroot Ͱ࣮ߦ • ϙʔλϏϦςΟͷ໰୊ͱͦͷղܾ • ίϯςφ͸ࣗ෼Ͱ࡞ΕΔ ·ͱΊ
33. 33.

    github.com/yuuki/droot