コンテナを「守る」仕組みから中身を理解しよう!!1; /how-to-be-a-container

@YAPC::Fukuoka 2017

Transcript

1. %PDLFSɺ)BDPOJXBɺͦΕ͔Β1FSM
   ۙ౻͏͓ͪ(.01FQBCP *OD
   :"1$'VLVPLB
   ίϯςφΛʮकΔʯ࢓૊Έ͔Βɺ
   த਎Λཧղ͠Α͏
   

   View Slide

2. ΤϯδχΞ
   ۙ౻͏͓ͪ!VE[VSB
   (.0ϖύϘٕज़ج൫νʔϜ
   5XJUUFS(JU)VC!VE[VSB
   'BDFCPPLVDIJPLPOEP
   

   View Slide

3. !VE[VSBʹ͍ͭͯ
   ɾ3VCZJTUɻ೥໨ʁ
   ɾύʔϑΣΫτ3VCZ3P3ڞஶ
   ɾ'VLVPLBSC
   ɾ3VCZ,BJHJTQFBLFS
   ɾNSVCZίϯςφ)BDPOJXB
   

   View Slide

4. !VE[VSBʹ͍ͭͯ
   ɾ3VCZJTUɻ೥໨ʁ
   ɾύʔϑΣΫτ3VCZ3P3ڞஶ
   ɾ'VLVPLBSC
   ɾ3VCZ,BJHJTQFBLFS
   ɾNSVCZίϯςφ)BDPOJXB
   

   View Slide

5. 3VCZJTU
   

   View Slide

6. ෱Ԭ3VCZձٞ!
   IUUQSFHJPOBMSVCZLBJHJPSHGVLVPLB
   

   View Slide

7. ίϯςφ
   

   View Slide

8. TFDVSJUZ
   

   View Slide

9. 6/*9
   -JOVY
   

   View Slide

10. ίϯςφͱ͸
    w༷ʑͳ6/*9γεςϜʹ͓͍ͯ͸ɺ༷ʑͳίϯςφ࣮૷͕͋Δ
    w'SFF#4%ܥͷKBJMɺ4PMBSJT$POUBJOFSɺ
    w-JOVYʹ͓͍ͯ͸ɺ-9$ɺ%PDLFSɺTZTUFNE—OTQBXOͳͲ
    w-JOVYͷίϯςφ͸جຊతʹʮಛघͳϓϩηεʯͱ࣮ͯ͠૷
    

    View Slide

11. ϓϩηεͬͯʁ
    

    View Slide

12. ϓϩηεͷ࡞Γํ
    ਌ϓϩηε
    ࢠϓϩηε
    ৽͍͠
    ϓϩάϥϜ
    GPSL
    
    FYFDWF
    
    XBJU
    
    

    View Slide

13. ϓϩηεͷ࡞Γํ
    ਌ϓϩηε
    ࢠϓϩηε
    ৽͍͠
    ϓϩάϥϜ
    ϓϩηεΛ
    ʮෳ੡ʯ͢Δ
    ݹ͍ϓϩηεΛࣺͯɺ
    ৽͍͠ϓϩάϥϜʹʮม਎ʯ͢Δ
    ਌͕ࢠڙͷ
    ऴྃΛ؂ࢹ
    

    View Slide

14. ϓϩηεͷଐੑ
    wͱ͜ΖͰɺϓϩηεʹ͸༷ʑͳଐੑ͕͋Δ
    wQSPDͷԼ͔Β֬ೝͰ͖Δ
    wDXE SPPU OTGT QQJE QJE DBQBCJMJUZ DHSPVQ
    

    View Slide

15. ྫQSPD1*%TUBUVT
    1*%ɺ਌ͷ1*%
    ࣮ߦϢʔβɺάϧʔϓ
    ϓϩηεάϧʔϓͳͲͷ*%
    ϝϞϦͷར༻ঢ়گ
    γάφϧͷઃఆʢϚεΫͳͲʣ
    $BQBCJMJUZ4FU
    

    View Slide

16. GPSL
    ͱFYFD
    ͷؒ
    ਌ϓϩηε
    ࢠϓϩηε
    ৽͍͠
    ϓϩάϥϜ
    GPSL
    
    FYFDWF
    
    XBJU
    
    GPSL
    ͱFYFD଒ͷ࣮ߦͷؒʹɺ
    ϓϩηεͷଐੑΛมߋ͢Δ͜ͱ͕Ͱ͖Δ
    

    View Slide

17. GPSL
    ͱFYFD
    ͷؒ
    wGPSL
    ͢ΔͱɺϓϩάϥϜͱͯ͠͸GPSL
    ݩͷίϐʔͱͳΔ͕ɺͦͷ
    ࣌఺Ͱಠཱͨ͠ଐੑΛ࣋ͭ
    wಠཱ͍ͯ͠ΔͷͰɺͦͷޙͰ༷ʑͳଐੑΛมߋ͢ΔγεςϜίʔϧΛ
    ݺ΂Δ
    wͦͷଐੑͷ͏ͪଟ͘͸FYFD
    ͯ͠৽͍͠΋ͷΛܧঝ͢ΔͷͰɺͦͷ
    ৽͍͠ϓϩηε͕ίϯςφతͳଐੑΛ͍࣋ͬͯΔͱɺίϯςφͱͯ͠
    ѻ͑Δ
    

    View Slide

18. GPSL
    ͱFYFD
    ͷؒ
    wGPSL
    ͢ΔͱɺϓϩάϥϜͱͯ͠͸GPSL
    ݩͷίϐʔͱͳΔ͕ɺͦͷ
    ࣌఺Ͱಠཱͨ͠ଐੑΛ࣋ͭ
    wಠཱ͍ͯ͠ΔͷͰɺͦͷޙͰ༷ʑͳଐੑΛมߋ͢ΔγεςϜίʔϧΛ
    ݺ΂Δ
    wͦͷଐੑͷ͏ͪଟ͘͸FYFD
    ͯ͠৽͍͠΋ͷΛܧঝ͢ΔͷͰɺͦͷ
    ৽͍͠ϓϩηε͕ίϯςφతͳଐੑΛ͍࣋ͬͯΔͱɺίϯςφͱͯ͠
    ѻ͑Δ
    VOTIBSF
    DISPPU
    QSDUM
    
    ࠓ೔ɺ͜Ε͔Βઆ໌͢Δ֤߲໨Ͱ͢
    

    View Slide

19. ຊൃදͰͷ࢖͍ํ
    wޙड़͢ΔΑ͏ͳγεςϜίʔϧ౳Λ༻͍ͯɺԿ͔͠Βͷ04Ϧιʔεͷ
    ִ཭ɺػೳ੍ݶɺ·ͨݖݶ෼཭ΛߦͬͨϓϩηεΛɺ

    ʮίϯςφʢ·ͨ͸ίϯςφతϓϩηεʣʯͱݺͼ·͢ɻ
    

    View Slide

20. ίϯςφͷத਎
    

    View Slide

21. %PDLFS.PCZMJCDPOUBJOFS
    w%PDLFSͷίϯςφϓϩηε࡞੒ͷίΞ࣮૷͸ʮ3VO$ʯͱ͍͏໊લͰ
    ಠཱ͍ͯ͠Δɻͦͷத਎Ͱ࢖͍ͬͯΔ΋ͷ͕MJCDPOUBJOFS
    wMJCDPOUBJOFSࣗମ͸ଞͷ

    (Pݴޠͷ࣮૷͔Β΋ར༻Մೳ
    wίϯςφपΓͷ༷ʑͳ

    ੜͷΦϓγϣϯΛࢦఆ
    IUUQTNFEJVNDPN!UJ⒎BOZGBZKEPDLFSFUQMVTFOHJOFJTOPXCVJMUPOSVODBOEDPOUBJOFSEBEEFFG
    

    View Slide

22. -9$
    w-JOVYίϯςφͷϦϑΝϨϯεత࣮૷
    wத਎͸ɺ$ݴޠ

    γεςϜίʔϧΛඇৗʹ

    ૉ௚ʹར༻
    wίϯςφܥγεςϜίʔϧͷݺͼํͷษڧʹͳΔ
    

    View Slide

23. )BDPOJXB
    w!VE[VSBͱ͍͏ਓ͕NSVCZͰ࡞ͬͨίϯςφ࣮૷
    wγεςϜίʔϧͱͷΞΫηεΛ$CJOEJOHͰɺϓϩηε࡞੒΍%4-ධ
    ՁͳͲͷॲཧΛNSVCZͰॻ͍͍ͯΔ
    w෭࢈෺ͱͯ͠ίϯςφܥγεςϜίʔϧʹ؆୯ʹΞΫηεͰ͖Δ
    NSVCZJSCόΠφϦ͕ೖΔʢύοέʔδΠϯετʔϧͷ৔߹ʣ
    

    View Slide

24. ͦͷଞͷ࣮૷
    w1FSMͷ࣮૷KBJMJOH BRS
    wIUUQTHJUIVCDPNLB[VIPKBJMJOH
    wIUUQTHJUIVCDPNIBZBKPBRS
    w3VTUʹWBHHBͱ͍͏΋ͷ͕͋ΔΒ͍͠
    wIUUQTHJUIVCDPNUBJMIPPLWBHHB
    

    View Slide

25. ίϯςφͷ
    ػೳͱ
    ηΩϡϦςΟ
    

    View Slide

26. ࠓ೔࿩͢͜ͱ
    wDISPPUQJWPU@SPPU
    w-JOVYOBNFTQBDF
    w$(SPVQ
    w,FSOFM$BQBCJMJUZ
    wTFDDPNQ
    w."$BQQBSNPS
    

    View Slide

27. ࠓ೔࿩͢͜ͱ
    wDISPPUQJWPU@SPPU
    w-JOVYOBNFTQBDF
    w$(SPVQ
    w,FSOFM$BQBCJMJUZ
    wTFDDPNQ
    w."$BQQBSNPS
    04Ϧιʔεͷ෼཭
    ݖݶɾػೳͷ੍ݶ
    ΞΫηείϯτϩʔϧ
    04Ϧιʔεͷར༻੍ݶ
    

    View Slide

28. DISPPU
    

    View Slide

29. DISPPU
    ίϚϯυ
    wDISPPU
    γεςϜίʔϧͷϥούʔ
    w΋ͬͱ΋୯७ͳʮίϯςφʯ
    wผͷͱ͜Ζʹ࡞ͬͨ04ͷSPPUϑΝΠϧγεςϜͷதʹʮೖΓʯɺ

    ਌ϓϩηεͱ͸ผͷ؀ڥΛ࡞Δ
    wDISPPUޙͷ؀ڥ͔ΒɺผͷSPPU΍ɺ਌ͷϑΝΠϧγεςϜ͸ɺ

    ݪଇͱͯ͠͸ݟ͑ͳ͍
    

    View Slide

30. ୯७ͳ͚ͩʹ͕݀͋Δ
    wDISPPU
    ͨ͠؀ڥ಺෦ͰɺDISPPU
    Մೳͩͱ؆୯ʹൈ͚ΒΕΔ
    # mkdir .tmp
    # mount --bind . .tmp
    # mount devtmpfs -t devtmpfs .tmp/dev
    # perl -e 'chroot ".tmp";
    chdir "..";chdir "..";chdir "..";
    chdir "..";chdir "..";chdir "..";
    chroot ".";exec "/bin/sh"'
    # ls /vagrant
    ...... (਌ͷσΟϨΫτϦ͕ݟ͑Δʂ)
    

    View Slide

31. VODISPPUΛ๷͙ʹ͸
    wDISPPUͰ͖ͳ͘͢Δͱ͍͏ํ๏͕Ұൠత
    wݖݶΛམͱ͢DBQBCJMJUZ
    wγεςϜίʔϧ୯ҐͰݺ΂ͳ͘͢ΔTFDDPNQ
    wͦͷଞɺ6TFSOBNFTQBDFΛ෼͚ͯ͠·͑͹࣮࣭୤ࠈෆՄ
    w͍ͣΕʹͤΑɺ΄͔ͷίϯςφػೳͱ૊Έ߹Θͤͯ҆શੑΛ֬อ͢Δ
    

    View Slide

32. DGQJWPU@SPPU
    wSPPUϑΝΠϧγεςϜΛʮೖΕସ͑ΔʯɻDISPPUΑΓڧྗ
    w04ͷϒʔτϓϩηεɺOFUCPPUͷࡍʹ࢖͍ͬͯΔ
    wDISPPUΑΓ͸੍ݶ͕͋Δ
    wlOFX@SPPUͱQVU@PME͸ݱࡏͷSPPUͱಉ͡ϑΝΠϧγεςϜʹ͋ͬͯ͸ͳ
    Βͳ͍zͳͲ
    wDISPPU΄Ͳखܰʹ͸࢖͑ͳ͍͕ɺҰํ୤ࠈͷ໰୊͸ͳ͘ͳΔ
    IUUQTMJOVYKNPTEOKQIUNM-%1@NBOQBHFTNBOQJWPU@SPPUIUNM
    

    View Slide

33. -JOVY
    OBNFTQBDF
    

    View Slide

34. -JOVYOBNFTQBDF͕ͳ͍ͱ
    wDISPPU͢ΔͱϑΝΠϧγεςϜ͕෼཭͞ΕΔɻ
    wͱ͍͏͜ͱͰɺ/procΛվΊͯϚ΢ϯτ͢Δඞཁ͕͋Δɻ
    wϚ΢ϯτ͢Δͱ

    ݟ͑ͯ͸͍͚ͳ͍΋ͷ͕

    ݟ͑ΔΑ͏ͳ
    

    View Slide

35. 04ͷϦιʔε͸ଞʹ΋ʮ෼཭Ͱ͖Δʯ
    wϓϩηεΛ෼཭͠ͳ͍ͱɺίϯςφͷத͔Β֎ͷϓϩηεΛ͍͡ΕΔ
    wϗετ໊Λ෼཭͠ͳ͍ͱɺίϯςφ಺Ͱผ్ϗετ໊ΛઃఆͰ͖ͳ͍
    wͦͷଞʹɺ෼཭Ͱ͖Δ΋ͷ
    wϚ΢ϯτϙΠϯτͷ৘ใ
    w*1$ϦιʔεTINHFU
    ͱ͔NR@PQFO
    తͳ΋ͷ
    wωοτϫʔΫɺϢʔβ*%ɺ$(SPVQ
    

    View Slide

36. Πϝʔδ
    wάϩʔόϧOBNFTQBDFͷ

    தʹɺ͍͔ͭ͘

    OBNFTQBDFΛ࡞ΕΔ
    IUUQTTQFBLFSEFDLDPNVE[VSBDSFBUJOHDPOUBJOFSTXJUIHPMBOH
    

    View Slide

37. Ұ෦͚ͩͷҠಈ΋Մೳ
    wྫip nets exec
    wOFUXPSLOBNFTQBDFʢʴЋʣ

    ͚ͩΛ෼཭͢Δ

    ίϯςφͱݴ͑Δ
    wҰൠతʹ͸ɺηοτͰ෼཭͢Δํ͕ศརͰ͸͋Δ
    $ sudo ip netns add test001
    $ sudo ip netns exec test001 /bin/bash
    root@test-1:/home/ubuntu# ip a
    1: lo: mtu 65536 qdisc noop
    state DOWN group default qlen 1
    link/loopback 00:00:00:00:00:00 brd
    00:00:00:00:00:00
    

    View Slide

38. ͱ͍͏͜ͱͰɺ1*%Λ෼཭͢Δ
    w࣮૷ͷํ਑
    w1*%͸/proc͔Βݟ͑Δ΋ͷ
    wͳͷͰผͷ/procΛ҆શʹϚ΢ϯτͰ͖ΔΑ͏ɺ

    .PVOUOBNFTQBDF΋Ұॹʹ෼཭͢Δ
    wGPSLͷ୅ΘΓʹDMPOF
    γεςϜίʔϧΛݺͼɺ࠷ޙʹFYFD
    ͢Δ
    ৚݅Λࡉ͔͘ࢦఆͰ͖Δ
    GPSL
    ͱߟ͍͑ͯͩ͘͞
    

    View Slide

39. 1FSMΛษڧͯ͠ॻ͍ͯΈ·ͨ͠
    #!/usr/bin/env perl
    use strict;
    use POSIX;
    use Linux::Clone;
    my $flg = POSIX::SIGCHLD | Linux::Clone::NEWNS | Linux::Clone::NEWPID;
    my $pid = Linux::Clone::clone sub {
    system "mount --make-rslave /";
    chroot "/var/lib/rootfs/yapc"; chdir "/";
    exec "/bin/sh"; 127 }, 0, $flg;
    print "PID=", $pid, "\n";
    waitpid $pid, 0 if($pid);
    print "Container exited\n";
    

    View Slide

40. ಈ࡞֬ೝ
    w͜ͷล·Ͱ͸ɺͲΜͳίϯςφͰ΋ಉ͡Α͏ʹ࣮૷͍ͯ͠Δ
    

    View Slide

41. ٳܜ
    

    View Slide

42. <13>
    

    View Slide

43. (.0ϖύϘ෱Ԭࢧࣾ͸
    ΤϯδχΞΛืू͍ͯ͠·͢
    ࠷৽ͷ࠾༻৘ใΛνΣοΫˠ !QC@SFDSVJU
    

    View Slide

44. ࢓ࣄͰɺίϯςφΛ

    ΨοπϦ࢖͍͍ͨʂ

    ͋Δ͍͸։ൃ͍ͨ͠ํ

    ੠Λֻ͓͚͍ͩ͘͞
    

    View Slide

45. SFTVNF
    

    View Slide

46. DHSPVQ
    

    View Slide

47. $POUSPM(SPVQ DHSPVQ
    
    w-JOVYʹ͸ɺϓϩηεΛάϧʔϐϯάͯ͠ɺͦͷάϧʔϓ͝ͱʹ

    ϋʔυ΢ΣΞϦιʔε04ϦιʔεͳͲͷར༻ঢ়گΛ֬ೝͨ͠Γɺ

    ͋Δ͍͸੍ޚΛ͢Δػೳ͕͋ΔɻͦΕ͕DHSPVQ
    wMJCDHSPVQͷΑ͏ͳϥΠϒϥϦ΍ɺDHSPVQGT͔ΒΞΫηεՄೳ
    w·ͨɺTZTUFNE͕಺෦Ͱར༻͢ΔʢϓϩηεͷάϧʔϐϯάɺϦιʔ
    ε੍ݶͳͲʣ
    wDGTZTUFNEOTQBXO
    TZTUFNEʹಉࠝͷίϯςφ
    

    View Slide

48. DHSPVQTVCTZTUFN
    w੍ޚͰ͖Δର৅͝ͱʹαϒγεςϜ͕͋Δ
    wαϒγεςϜ͸ͨ͘͞Μ͋Δ
    wDQV
    wNFNPSZ
    wCMLJP OFU@DMT GSFF[FS
    wͦͷதͰ΋QJETαϒγεςϜΛ঺հ
    

    View Slide

49. ίϯςφͰͷGPSLCPNC߈ܸ
    w౰વͰ͋Δ͕ɺ฼؋04શମͰͷϓϩηεͷ࠷େ਺͸༗ݶͰ͋Δ
    w͕ͨͬͯ͠ɺίϯςφ؀ڥͷத͔Β๲େͳϓϩηεΛ࡞੒͢Δͱɺ

    ݁Ռతʹ฼؋04ͷϓϩηε਺ͷ্ݶʹୡ͢Δ͜ͱ͸ى͜Γ͏Δ
    w఻౷తʹ͸ɺSMJNJUͰϓϩηεπϦʔ͝ͱͷϓϩηε਺Λ੍ݶ
    wίϯςφͷ৔߹BUUBDIͳͲʹΑΓɺίϯςφ಺ͷશͯͷϓϩηε͕ɺ
    ಉ͡ϓϩηεπϦʔʹॴଐ͍ͯ͠ͳ͍৔߹΋͋Γ͏Δ
    

    View Slide

50. QJETTVCTZTUFN
    w-JOVYҎ߱Ͱಋೖ͞ΕͨαϒγεςϜ
    wҎԼͷ஋Λར༻Ͱ͖Δ
    wQJETDVSSFOUάϧʔϓ಺ͷݱࡏͷϓϩηε਺
    wQJETNBYͦͷάϧʔϓͰڐՄ͢Δ࠷େϓϩηε਺
    SPPUάϧʔϓͰ͸ར༻Ͱ͖ͳ͍ͷͰ஫ҙ
    ࢀߟʮ-9$ͰֶͿίϯςφೖ໳ʯ
    IUUQHJIZPKQBENJOTFSJBMMJOVY@DPOUBJOFST
    

    View Slide

51. ͖ͬ͞ͷ1FSMίϯςφʹύον
    --- chroot2.pl 2017-06-20 10:47:02.780313607 +0900
    +++ bomber.pl 2017-06-20 12:55:41.572399620 +0900
    @@ -2,11 +2,19 @@
    use strict;
    use POSIX;
    use Linux::Clone;
    +system "mkdir -p /sys/fs/cgroup/pids/yapc-fukuoka";
    my $flg = POSIX::SIGCHLD | Linux::Clone::NEWNS | Linux::Clone::NEWPID;
    my $pid = Linux::Clone::clone sub {
    system "mount --make-rslave /";
    chroot "/var/lib/haconiwa/rootfs/php"; chdir "/";
    - exec "/bin/sh"; 127 }, 0, $flg;
    + exec "/bin/bash"; 127 }, 0, $flg;
    +open TASKS, ">> /sys/fs/cgroup/pids/yapc-fukuoka/tasks";
    +print TASKS "$pid";
    +close TASKS;
    +open MAX, ">> /sys/fs/cgroup/pids/yapc-fukuoka/pids.max";
    +print MAX "32";
    +close MAX;
    +
    print "PID=", $pid, "\n";
    waitpid $pid, 0 if($pid);
    QJETDHSPVQΛ௥Ճ
    ίϯςφͷQJEΛॴଐͤ͞ɺ
    NBYΛ੍ݶ
    

    View Slide

52. GPSLCPNCΛ๷͛Δ͜ͱ͕Θ͔Δ
    wQJETNBY੍ݶͳͩ͠ͱ฼؋͝ͱ௜໧͠·͢
    

    View Slide

53. ,FSOFM
    $BQBCJMJUZ
    

    View Slide

54. -JOVY,FSOFM$BQBCJMJUZ
    w-JOVYͰ͸ɺSPPU͕͍࣋ͬͯΔ༷ʑͳݖݶΛɺࡉ͔͘෼ׂͯ͠ɺ

    Ұ෦͚ͩ෇༩ɺ·ͨ͸Ұ෦੍͚ͩݶ͢Δ͜ͱ͕Ͱ͖Δ
    w͜ΕΒͷݖݶͷू߹ΛέʔύϏϦςΟηοτɺҰͭҰͭΛέʔύϏϦ
    ςΟͱݺͿɻ
    wྫ͑͹ɺ࣌ؒΛઃఆ͢Δݖݶ CAP_SYS_TIME
    ɺLJMMΛͲΜͳϓϩηε
    ʹ΋ૹΔݖݶ CAP_KILL
    ɺ࠶ىಈ͢Δݖݶ CAP_SYS_BOOT
    
    

    View Slide

55. 6CVOUV9FOJBM -JOVY
    Ͱ͸
    wdͷͷ

    έʔύϏϦςΟ
    

    View Slide

56. έʔύϏϦςΟͷܧঝϧʔϧ
    wϓϩηεϑΝΠϧͷͭͷηοτ 1FSNJUUFE*OIFSJUBCMF&⒎FDUJWF
    ɺ
    ό΢ϯσΟϯάηοτɺΞϯϏΤϯτηοτ -JOVYҎ߱
    Ͱܾ·Δ
    wৄࡉ͸NBODBQBCJMJUJFT
    wྫTFUVTFSJESPPUͰɺଞ͕

    σϑΥϧτͷ৔߹ɺ

    ό΢ϯσΟϯάηοτͰམͱͯ͠

    FYFDWF
    ͢Δͱ৽͍͠ϓϩάϥϜ

    Ͱ͸ͦͷݖݶ͕མ͍ͪͯΔ
    

    View Slide

57. ྫҰൠϢʔβͰ൪ΛϦεϯ͍ͨ͠
    wҰൠతʹɺ൪ҎԼͷϙʔτ͸ҰൠϢʔβ͸࢖͑ͳ͍
    w͜͏͍͏(Pͷ)5514FSWFSΛ࡞ͬͨΒɺҰൠϢʔβ͸ىಈͰ͖ͳ͍
    

    View Slide

58. TFUDBQ
    ͰpMFDBQBCJMJUJFTΛ෇༩͢Δ
    w൪ҎԼΛϦεϯ͢Δݖݶʹ CAP_NET_BIND_SERVICE
    ubuntu@compute-1:~$ sudo setcap cap_net_bind_service+ep ./listen80
    ubuntu@compute-1:~$ ./listen80 &
    [1] 5915
    ubuntu@compute-1:~$ curl localhost
    Hello, World
    ubuntu@compute-1:~$ sudo getcap ./listen80
    ./listen80 = cap_net_bind_service+ep
    VCVOUVϢʔβͰ
    αʔόΛىಈͰ͖Δ
    

    View Slide

59. ྫ੍ݶ෇͖ͷίϯςφ಺SPPU
    wίϯςφ಺෦Ͱ΋ɺSPPUΛ౉͢ͱศརͳ͜ͱ͸ଟ͍
    wͱ͸ݴ͑ͳΜͰ΋͸ͤͨ͘͞ͳ͍ɻͰ͖Δ͜ͱ͚ͩ
    wCAP_SYS_TIMEͱCAP_SYS_CHROOTݖݶΛୣͬͯΈΔ
    

    View Slide

60. ࠷ॳͷ1FSMίϯςφʹύον
    --- chroot2.pl 2017-06-20 10:47:02.780313607 +0900
    +++ dropcap.pl 2017-06-20 14:38:31.335190235 +0900
    @@ -2,10 +2,14 @@
    use strict;
    use POSIX;
    use Linux::Clone;
    +use Linux::Prctl;
    +
    my $flg = POSIX::SIGCHLD | Linux::Clone::NEWNS | ...;
    my $pid = Linux::Clone::clone sub {
    system "mount --make-rslave /";
    chroot "/var/lib/haconiwa/rootfs/php"; chdir "/";
    + $Linux::Prctl::capbset{"sys_time"} = 0;
    + $Linux::Prctl::capbset{"sys_chroot"} = 0;
    exec "/bin/sh"; 127 }, 0, $flg;
    print "PID=", $pid, "\n";
    waitpid $pid, 0 if($pid); CPVOEJOHTFU͔Β࡟আ
    

    View Slide

61. EBUFͰ೔෇ͷηοτ͕Ͱ͖ͳ͘ͳΔ
    w0QFSBUJPOOPUQFSNJUUFEѻ͍ʹͳΔ
    root@compute-1:~# perl dropcap.pl
    PID=5962
    # date
    Tue Jun 20 06:34:50 UTC 2017
    # date -s 00:00:00
    date: cannot set date: Operation not permitted
    Tue Jun 20 00:00:00 UTC 2017
    # date
    Tue Jun 20 06:34:59 UTC 2017
    

    View Slide

62. DISPPUͰίϯςφΛൈ͚ΒΕͳ͘ͳΔʂ
    

    View Slide

63. %PDLFS.PCZͰ͸
    wEPDLFSNPCZSVOͷΦϓγϣϯɺDBQBEE DBQESPQ

    ΦϓγϣϯͰίϯτϩʔϧՄೳ
    wσϑΥϧτͷ$BQBCJMJUZ͸ιʔεʹϋʔυίʔυ͞Ε͍ͯΔ
    wIUUQTHJUIVCDPNNPCZNPCZCMPCNBTUFSPDJ
    EFGBVMUTHP--
    

    View Slide

64. ͱ͜ΖͰօ͞Μ
    

    View Slide

65. QSJWJMFHFEΛ҆қʹ࢖͍ͬͯ·ͤΜ͔ʁ
    wจࣈͲ͓Γɺਫ਼ࠪͤͣʹɺ͢΂ͯͷ$BQBCJMJUZΛ෇༩͢ΔΦϓγϣϯ
    wͦͷଞͷɺTFDDPNQBQQBSNPSͳͲͷઃఆʹΑΓɺίϯςφ಺ͷ
    ݖݶ͸ߜΓࠐ·ΕΔɻͱ͸͍͑
    wྫ͑͹ɺΧʔωϧʹΑͬͯTFDDPNQ౳͕༗ޮͰͳ͍͜ͱ΋͋Δ
    w TFDDPNQ BQQBSNPS౳Λແޮʹ͢ΔΦϓγϣϯ΋͋Δ
    w ҆શੑ͕ઈର֬อͰ͖ΔΑ͏ͳ৔໘ $*Ͱར༻ΠϝʔδΛݶఆ͢Δ౳
    Ͱ͸·ͨผ͕ͩ
    w࠷খݖݶͷݪଇʹै͍ɺਫ਼ࠪͯ͠DBQBEEͰ໌ࣔత௥Ճ͢Δ΂͖
    

    View Slide

66. ٳܜ
    
    

    View Slide

67. ͓ർΕͰ͠ΐ͏ͷͰ
    ෱Ԭͷඒຯ͍͠΋ͷͷը૾Λ
    ோΊ·͠ΐ͏
    

    View Slide

68. View Slide

69. View Slide

70. View Slide

71. SFTUPSF
    

    View Slide

72. TFDDPNQ
    

    View Slide

73. TFDDPNQ TFDDPNQCQG
    
    w-JOVYͰɺαϯυϘοΫεΛ࣮ݱ͢ΔͨΊʹɺ

    ϓϩηεͷγεςϜίʔϧݺͼग़͠ΛϑΟϧλϦϯά͠ɺ

    ੍ݶɾτϥοϓɾτϥοΩϯάͳͲΛ࣮ݱ͢Δػೳ
    wࠓճ͸ɺ-JOVYҎ߱ͷTFDDNPQNPEFͷ࿩
    w#FSLFMFZ1BDLFU'JMUFS #1'
    Λ༻͍ͯߴ଎ʹݺͼग़͠ΛϑΟϧλϦ
    ϯά͢Δ
    

    View Slide

74. Կ͕Ͱ͖Δ͔
    wಛఆͷγεςϜίʔϧͷڐՄ
    wಛఆͷγεςϜίʔϧͷېࢭ 4*(4:4ͷૹ෇
    
    wಛఆͷγεςϜίʔϧͷݺͼग़͠Λ೚ҙͷFSSOPͰฦ͢
    wಛఆͷγεςϜίʔϧΛɺQUSBDF
    ͰτϥοΫՄೳʹ͢Δ
    w࠷ॳʹσϑΥϧτͷڍಈΛࢦఆ͠ɺݸผͷγεςϜίʔϧͷ৔߹Λఆ
    ٛ͢Δ
    

    View Slide

75. ͔͜͜ΒNSVCZ͕ग़ͯ͘Δ
    w1FSMྗ͕௿͘ɺ͍͢·ͤΜ ҰԠϞδϡʔϧ͸͋ΔΑ͏Ͱ͕͢ɻ
    
    wIBDPOJXBNSVCZTFDDPNQ
    

    View Slide

76. NSVCZTDSJQU
    #!/usr/bin/env hacorb
    context = Seccomp.new(default: :allow) do |rule|
    rule.kill :mkdir
    rule.kill :fchownat
    end
    pid = Process.fork do
    context.load
    puts "==== It will be jailed. Please try to mkdir/chown"
    exec "/bin/sh"
    end
    p(Process.waitpid2 pid)
    

    View Slide

77. ࣮ߦ͢ΔͱɺαϯυϘοΫε্ཱ͕͕ͪΔ
    w·͝͏ࣄͳ͖SPPUͰ͋Δ͕ɺϑΝΠϧͷॴ༗ऀΛม͑ͨΓɺ

    σΟϨΫτϦΛ࡞Εͳ͍ɻ͜Ε΋Ұछͷʮίϯςφʯ
    wʢIBDPOJXBQBDLBHFʹಉࠝ͞ΕΔIBDPSCόΠφϦΛ࢖͏ʣ
    4:(4:4ͷΤϥʔϝοηʔδ
    

    View Slide

78. ৄࡉ͸ϒϩάʹॻ͖·ͨ͠ʜ
    wNSCHFNͷ֓ཁʮTFDDPNQΛNSVCZͰࢼ͢ʯ
    wIUUQVE[VSBIBUFOBCMPHKQFOUSZ
    wTFDDPNQʹΑΔγεςϜίʔϧτϥοΩϯά

    ʮNSVCZͱTFDDPNQͱQUSBDFͰγεςϜίʔϧΛͱʹ͔͘௥͍͔͚Δʯ
    wIUUQVE[VSBIBUFOBCMPHKQFOUSZ
    ಛఆͷγεςϜίʔϧݺͼग़͠ΛϩΪϯά͢Δ
    γΣϧͷྫ͸ͪ͜Βͷهࣄ͔Β
    

    View Slide

79. %PDLFSͷதͰ͸ʁ
    wݺͼग़ͤΔγεςϜίʔϧͷʮϗϫΠτϦετʯ͕ଘࡏ͢Δ
    wυΩϡϝϯτʹ΋͋Δ௨Γɻ
    wΦϓγϣϯͰ೚ҙͷϑΟϧλʔΛద༻Ͱ͖Δͦ͏
    4FDDPNQTFDVSJUZQSPpMFTGPS%PDLFS
    IUUQTEPDTEPDLFSDPNFOHJOFTFDVSJUZTFDDPNQ
    

    View Slide

80. ."$
    "QQ"SNPS
    

    View Slide

81. ίϯςφػೳઆ໌ͱͯ͠͸ɺ
    ͜ΕͰ࠷ޙͳΜͰ
    ؤு͓ͬͯฉ͖͍ͩ͘͞
    

    View Slide

82. ༤େͳେ෼ͷࣗવΛݟͯٳܜ
    CZ!NBUTVNPUPSZ͞Μ
    

    View Slide

83. ."$ͱ͸
    w͜͜Ͱ͸.BOEBUPSZ"DDFTT$POUSPMڧ੍ΞΫηε੍ޚͷ͜ͱ
    wҰൠతͳɺϑΝΠϧΦ΢φʔ͝ͱʹΞΫηε͢ΔݖݶΛߜΔํࣜ͸ɺ
    ೚ҙΞΫηε੍ޚ %JTDSFUJPOBSZ"DDFTT$POUSPM
    ͱݺ͹ΕΔɻ
    wࣄނͳͲʹΑΓݖݶΛඞཁҎ্ʹΏΔ͘Ͱ͖ͯ͠·͏

    ʢσΟϨΫτϦΛύʔϛογϣϯͰެ։Ͱ͖Δ౳ʣ͜ͱ͕͋Δ
    

    View Slide

84. ."$ͱ͸
    
    w%"$ͷݖݶݕࠪΛͨ͠ޙͰɺ؅ཧऀͷઃఆͨ͠."$ͷϙϦγʔ͕ద
    ༻͞ΕɺϦιʔε΁ͷΞΫηε͕ڧ੍ίϯτϩʔϧ͞ΕΔ
    wʮࣗ෼ͷݖݶͰ͋ͬͯ΋ɺࣗ෼ͰίϯτϩʔϧͰ͖ͳ͍ʯ͜ͱ΋
    wྫ͑͹ࣗ෼Ͱ࡞ͬͨϑΝΠϧʹɺࣗ෼ͰΞΫηεͰ͖ͳ͘ͳΔɺͱ͍
    ͏ઃఆ΋ՄೳͰ͋Δ
    w·ͨɺ%"$ΑΓࡉ͔͍ΞΫηε੍ޚ΋Մೳʹ
    

    View Slide

85. "QQ"SNPSͱ͸
    w."$Λ࣮ݱ͢Δϛυϧ΢ΣΞͷҰͭ
    wϓϩάϥϜͷύε୯Ґϓϩηε୯ҐͰϓϩϑΝΠϧͷద༻͕Ͱ͖Δ
    ͷ͕ಛ௃
    w6CVOUVͷ$BOPOJDBMࣾʹΑΓ։ൃ͕͞Ε͍ͯΔ
    wFOGPSDFNPEFͱDPNQMBJONPEFʢه࿥ͷΈʣ͕͋Δ
    

    View Slide

86. %PDLFSͰͷར༻
    wίϯςφ͸σϑΥϧτɺEPDLFSEFGBVMUͱ͍͏ϓϩϑΝΠϧ͕౰ͨΔ
    ίϯςφΛͭ࡞ͬͨͷͰɺ
    ͭͷϓϩηεʹద༻͞Ε͍ͯΔ
    -9$΋࢖ͬͯ·͢Ͷ
    

    View Slide

87. ϓϩϑΝΠϧͷྫ
    IUUQTEPDTEPDLFSDPNFOHJOFTFDVSJUZBQQBSNPSOHJOYFYBNQMFQSPpMF
    wಠࣗͷݴޠΛ༻͍ͯهड़͢Δ
    

    View Slide

88. ΧελϜϓϩϑΝΠϧΛ౰ͯΔʹ͸
    wdeny /usr/bin/top mrwklxͱ͍͏ϧʔϧΛՃ͑ͨϓϩϑΝΠϧ
    Λ࡞੒ɺొ࿥͢Δ
    wEPDLFSSVOίϚϯυͰ--security-opt apparmor=exampleͷ
    Α͏ʹࢦఆͯ͠ىಈ
    wͦͷίϯςφͰ͸ɺUPQίϚϯυΛ࣮ߦ͢Δ͜ͱ͕Ͱ͖ͳ͍ɻ
    wBVEJUͷΈɺͳͲ΋Մೳ
    ৄࡉ͸ϒϩάͰ
    ʮ"QQ"SNPSͱ%PDLFSͱͦͷଞίϯςφతϓϩηεʹ͍ͭͯʯ
    IUUQVE[VSBIBUFOBCMPHKQFOUSZ
    

    View Slide

89. ΑΓৄࡉͳத਎
    wMJCBQQBSNPSͱ͍͏ϥΠϒϥϦͰϓϩάϥϜ͔ΒΞΫηεͰ͖Δ
    wݱࡏͷϓϩηεͷϓϩϑΝΠϧΛมߋ͢ΔBB@DIBOHF@QSPpMF
    ͱ

    FYFDWF
    ͷλΠϛϯάͰมߋ͢ΔBB@DIBOHF@POFYFD
    ͕͋Δ
    wʢ6CVOUVͷNBOͰηΫγϣϯ͕ͳͷͰʣ
    wNSVCZͷCJOEJOH࡞੒ࡁʢ͔͠͠·ͩ)BDPOJXBʹ૊ΈࠐΜͰͳ͍ʣ
    IUUQNBOQBHFTVCVOUVDPNNBOQBHFTYFOJBMNBOBB@DIBOHF@QSPpMFIUNM
    

    View Slide

90. ͓ർΕ༷

    Ͱͨ͠
    

    View Slide

91. ·ͱΊ
    

    View Slide

92. ίϯςφ͸
    ಛผͳϓϩηεʹա͗ͳ͍ɻ
    ˠͦͷϓϩηεΛ҆શʹɺಠཱͨ͠ܗͰɺ
    ޮ཰తʹར༻͢΂༷͘ʑͳγεςϜίʔϧ΍ػೳ͕͋Δ
    

    View Slide

93. %PDLFS͸σϑΥϧτͰ
    ͍Ζ͍Ζ͍͍ײ͡ʹ͍ͯ͠Δ
    ˠ͔͠͠ɺཧղ͕ෆे෼ͩͱൈ͚ಓΛ࡞ͬͯ͠·͏͜ͱ΋ɻ
    ɹQSJWJMFHFEΦϓγϣϯɺؒҧͬͨઃఆɺ
    ɹΧʔωϧόʔδϣϯͰ࢖͑ͳ͍ػೳͳͲ
    

    View Slide

94. ࢀߟεΠενʔζϞσϧ
    ˠҰͭҰͭʹ͕ۭ͍͍݀ͯͨͱͯ͠΋ɺ
    ɹͨ͘͞ΜॏͶΔ͜ͱͰɺશͯͷ݀Λൈ͚Δ͜ͱ͸
    ɹඇৗʹ೉͘͠ͳΔ
    *NBHF$$IUUQTQJYBCBZDPNQ
    

    View Slide

95. ͔ͬ͠Γͱத਎ʹڵຯΛ࣋ͪ
    ཧղͯ͠࢖͏͜ͱͰ
    ϋϚΓͲ͜ΖΛճආͰ͖Δ
    ˠIBDPSC͕ศར͔΋͠Εͳ͍ɻ
    ɹ1FSMͰ΋ɺίϯςφػೳͰ༡΂ΔΑʂ
    

    View Slide

96. ָ҆͘͠શʹ
    ίϯςφΛ࢖͓͏ʂ
    

    View Slide

97. ࢀߟࢿྉͷօ͞Μ
    wVE[VSBͷϒϩάʢTFDDPNQBQQBSNPSIBDPOJXBଞʣ
    wMYDKQ!5FO'PSXBSE͞Μͷࢿྉ
    w IUUQTTQFBLFSEFDLDPNUFOGPSXBSE
    wաڈͷ:"1$ :"1B$
    ൃදͳͲ
    w 1FSMͰθϩ͔Β࡞Δίϯςφ
    w ֶͭͬͯ͘Ϳ-JOVYίϯςφͷཪଆ
    w%PDLFSTFDVSJUZ
    w IUUQTEPDTEPDLFSDPNFOHJOFTFDVSJUZTFDVSJUZ
    

    View Slide

98. 4QFDJBM5IBOLT
    w!5FO'PSXBSE͞ΜʹຊεϥΠυͷϨϏϡʔΛ͓ئ͍͠·ͨ͠ɻ

    ͋Γ͕ͱ͏͍͟͝·ͨ͠ɻ·ͨຊொͰҿΈ·͠ΐ͏ɻ
    

    View Slide