Upgrade to Pro — share decks privately, control downloads, hide ads and more …

Finding Security Events with eBPF - Security Mi...

mrtc0
September 29, 2020

Finding Security Events with eBPF - Security Minicamp in Yamanashi 2020-

mrtc0

September 29, 2020
Tweet

More Decks by mrtc0

Other Decks in Technology

Transcript

  1. ߨٛͰ࢖͏ίʔυ teacher01@teacher01:~$ ls -al ~/bpf-tutorial/ total 28 drwxr-xr-x 7 root

    root 4096 Sep 19 08:20 . drwxr-xr-x 21 teacher01 teacher01 4096 Sep 19 11:22 .. drwxr-xr-x 2 teacher01 teacher01 4096 Sep 18 07:22 bpftrace drwxr-xr-x 2 teacher01 teacher01 4096 Sep 14 14:02 execsnoop drwxr-xr-x 2 teacher01 teacher01 4096 Sep 18 08:07 opensnoop drwxr-xr-x 2 teacher01 teacher01 4096 Sep 14 12:27 seccomp-bpf drwxr-xr-x 2 teacher01 teacher01 4096 Sep 19 07:32 trace-app
  2. %&.0 root@bpf:/ # execsnoop-bpfcc PCOMM PID PPID RET ARGS ps

    1807 1478 0 /usr/bin/ps aux cat 1808 1478 0 /usr/bin/cat /etc/passwd ls 1809 1478 0 /usr/bin/ls --color=auto -al top 1810 1478 0 /usr/bin/top root@bpf:/ # tcpconnect-bpfcc Tracing connect ... Hit Ctrl-C to end PID COMM IP SADDR DADDR DPORT 1812 curl 4 192.168.64.5 93.184.216.34 443 1815 curl 4 192.168.64.5 52.205.86.27 80 1817 curl 4 192.168.64.5 35.227.220.44 80 1819 curl 4 192.168.64.5 35.227.220.44 443
  3. D#1' $ sudo tcpdump -d icmp (000) ldh [12] (001)

    jeq #0x800 jt 2 jf 5 (002) ldb [23] (003) jeq #0x1 jt 4 jf 5 (004) ret #262144 (005) ret #0
  4. TFDDPNQ#1' ubuntu@sandbox:~/bpf-tutorial/seccomp-bpf$ make build clang -o filter-mkdir main.c ubuntu@sandbox:~/bpf-tutorial/seccomp-bpf$ ./filter-mkdir

    'mkdir /tmp/dir' mkdir: cannot create directory ‘/tmp/dir’: Operation not permitted ubuntu@sandbox:~/bpf-tutorial/seccomp-bpf$ strace -f ./filter-mkdir 'mkdir /tmp/dir' execve("./filter-mkdir", ["./filter-mkdir", "mkdir /tmp/dir"], 0x7ffca47f8350 /* 21 vars */) = 0 ... [pid 2593] mkdir("/tmp/dir", 0777) = -1 EPERM (Operation not permitted)
  5. FYFDTOPPQ root@bpf:/# execsnoop-bpfcc PCOMM PID PPID RET ARGS ps 1865

    1478 0 /usr/bin/ps aux cat 1866 1478 0 /usr/bin/cat /etc/passwd htop 1867 1478 0 /usr/bin/htop ping 1868 1478 0 /usr/bin/ping security-camp.or.jp
  6. UDQDPOOFDU root@bpf:/# tcpconnect-bpfcc Tracing connect ... Hit Ctrl-C to end

    PID COMM IP SADDR DADDR DPORT 1812 curl 4 192.168.64.5 93.184.216.34 443 1815 curl 4 192.168.64.5 52.205.86.27 80 1817 curl 4 192.168.64.5 35.227.220.44 80 1819 curl 4 192.168.64.5 35.227.220.44 443
  7. CJPMBUFODZ root@bpf:/# biolatency-bpfcc Tracing block device I/O... Hit Ctrl-C to

    end. ^C usecs : count distribution 0 -> 1 : 0 | | 2 -> 3 : 0 | | 4 -> 7 : 0 | | 8 -> 15 : 7 | | 16 -> 31 : 3 | | 32 -> 63 : 0 | | 64 -> 127 : 237 |******* | 128 -> 255 : 1251 |****************************************| 256 -> 511 : 166 |***** | 512 -> 1023 : 103 |*** |
  8. CBSFBEMJOF root@bpf:/# bashreadline-bpfcc TIME PID COMMAND 22:09:03 1478 ls 22:09:07

    1478 cat /etc/passwd 22:09:10 1478 htop 22:09:12 1478 ps aux
  9. PPNLJMM root@bpf:/# oomkill-bpfcc Tracing OOM kills... Ctrl-C to stop. 22:09:56

    Triggered by PID 777 ("snapd"), OOM kill of PID 2279 ("perl"), 1007686 pages, loadavg: 0.09 0.04 0.01 5/159 2280 root@bpf:/# sysctl -w vm.overcommit_memory=1 root@bpf:/# perl -e 'while (1) { $a .= "A" * 124; }'
  10. &YBNQMF root@bpf:/# bpftrace -e \ 'tracepoint:raw_syscalls:sys_enter { @[comm] = count();

    }' Attaching 1 probe... @[irqbalance]: 12 @[systemd-network]: 22 @[systemd]: 28 @[bpftrace]: 72 @[cat]: 108 @[bash]: 191 @[sshd]: 747
  11. ,QSPCFͰτϨʔεͯ͠ΈΑ͏ // جຊͷߏจ # bpftrace -e 'kprobe:<func> { Expression }'

    # bpftrace -l 'kprobe:*' // attach Ͱ͖Δؔ਺ҰཡΛग़ྗ // vfs_open ͕ݺ͹ΕͨΒϝοηʔδΛදࣔ # bpftrace -e \ 'kprobe:vfs_open { printf("called vfs_open\n"); }'
  12. ,QSPCFͰτϨʔεͯ͠ΈΑ͏ // ίϚϯυ໊Λදࣔ # bpftrace -e \ 'kprobe:vfs_open { printf("%s\n",

    comm); }' // cat ͚ͩΛදࣔ # bpftrace -e \ 'kprobe:vfs_open /comm == "cat"/ { printf("%s\n", comm); }'
  13. ,QSPCFͰτϨʔεͯ͠ΈΑ͏ // Ҿ਺ͷදࣔ # bpftrace -e \ 'kprobe:do_sys_open { printf("opening:

    %s\n", str(arg1)); }' #include <linux/path.h> #include <linux/dcache.h> kprobe:vfs_open { printf("%s %s\n", comm, str(((struct path *)arg0)->dentry->d_name.name)); }
  14. 6QSPCFT wͬ͘͟Γ͍͏ͱLQSPCFTͷϢʔβʔεϖʔεϓϩάϥϜ൛ root@bpf:/# objdump -T /bin/bash | grep readline 0000000000124e60

    g DO .bss 0000000000000008 Base rl_readline_state 00000000000b7cd0 g DF .text 0000000000000252 Base readline_internal_char 00000000000b71a0 g DF .text 000000000000015f Base readline_internal_setup 0000000000087120 g DF .text 000000000000004c Base posix_readline_initialize 00000000000b8530 g DF .text 000000000000009a Base readline # bpftrace -e 'uprobe:/bin/bash:readline { printf("called\n"); }'
  15. 6QSPCFT root@bpf:/# objdump -T /bin/bash | grep readline 0000000000124e60 g

    DO .bss 0000000000000008 Base rl_readline_state 00000000000b7cd0 g DF .text 0000000000000252 Base readline_internal_char 00000000000b71a0 g DF .text 000000000000015f Base readline_internal_setup 0000000000087120 g DF .text 000000000000004c Base posix_readline_initialize 00000000000b8530 g DF .text 000000000000009a Base readline # bpftrace -e 'uprobe:/bin/bash:0xb8530 { printf("called\n"); }'
  16. 6QSPCFT root@bpf:/# nm uprobes-test| grep main 0000000000001149 T main root@bpf:/#

    bpftrace -e \ 'uprobe:./uprobes-test:main { printf("in main\n"); }' Attaching 1 probe...
  17. 5SBDFQPJOUT # bpftrace -e \ 'tracepoint:syscalls:sys_enter_execve { printf("%s %s\n", comm,

    str(args->filename)); }' # cat /sys/kernel/tracing/available_events | grep execve syscalls:sys_exit_execveat syscalls:sys_enter_execveat syscalls:sys_exit_execve syscalls:sys_enter_execve
  18. 5SBDFQPJOUT # bpftrace -e \ 'tracepoint:syscalls:sys_enter_execve { printf("%s %s\n", comm,

    str(args->filename)); }' # bpftrace -e \ 'tracepoint:syscalls:sys_enter_execve { printf("%s %s\n", comm, str(args->filename)); }' # cat /sys/kernel/tracing/events/syscalls/sys_enter_execve/format ... field:int __syscall_nr; offset:8; size:4; signed:1; field:const char * filename; offset:16; size:8; signed:0; field:const char *const * argv; offset:24; size:8; signed:0; field:const char *const * envp; offset:32; size:8; signed:0; print fmt: "filename: 0x%08lx, argv: 0x%08lx, envp: 0x%08lx", \ ((unsigned long)(REC->filename)), ((unsigned long)(REC->argv)), ((unsigned long)(REC->envp))
  19. 64%5 # bpftrace -e \ 'usdt:./usdt:test_probe { printf("got: %d\n", arg0);

    }' Attaching 1 probe... got: 3 got: 4 got: 5 got: 6 ...
  20. ϨΠςϯγͷଌఆ # gethostlatency-bpfcc TIME PID COMM LATms HOST 18:01:33 8892

    curl 29.99 example.com 18:01:43 8894 isc-worker0000 0.02 127.0.0.1 18:01:43 8894 isc-worker0000 0.01 ::1 18:01:55 8898 curl 43.45 security-camp.or.jp 18:02:00 8900 curl 97.16 blog.ssrf.in
  21. TUSBDFͰ֬ೝ $ strace -o output.txt cat /etc/passwd $ cat output.txt

    ... openat(AT_FDCWD, "/etc/passwd", O_RDONLY) = 3 ...
  22. PQFOBU SYSCALL_DEFINE4(openat, int, dfd, const char __user *, filename, int,

    flags, umode_t, mode) { if (force_o_largefile()) flags |= O_LARGEFILE; return do_sys_open(dfd, filename, flags, mode); } IUUQTFMJYJSCPPUMJODPNMJOVYWTPVSDFGTPQFOD-
  23. CDDͷجຊ wCDDͷجຊ bpf_code = """ int do_sys_open(struct pt_regs *ctx, ...)

    { ... } """ # BPF ϓϩάϥϜΛΠχγϟϥΠζ b = BPF(text=bpf_code) # kprobe ʹ attach ͢Δ. # event ʹτϨʔε͢Δؔ਺Λɺfn_name ʹϑοΫ࣌ʹ࣮ߦ͢Δؔ਺Λࢦఆ b.attach_kprobe(event="do_sys_open", fn_name="do_entry_trace")
  24. -FWFM wCQG@USBDF@QSJOUL Ͱग़ྗ bpf_code = """ int do_sys_open(struct pt_regs *ctx)

    { bpf_trace_printk("message\n"); } """ b = BPF(text=bpf_code) b.attach_kprobe(event="do_sys_open", fn_name="do_entry_trace")
  25. -FWFM wग़ྗͨ͠಺༰͸USBDF@pFMET Ͱऔಘ b = BPF(text=bpf_code) b.attach_kprobe(event="do_sys_open", fn_name="do_entry_trace") while 1:

    (task, tid, _, _, ts, msg) = b.trace_fields() printb(b"%f, %s, %d, %s" % (ts, task, tid, msg))
  26. PQFOTOPPQ bpf_code = """ int do_entry_trace(struct pt_regs *ctx) { bpf_trace_printk("REPLACEME\\n");

    return 0; } """ b = BPF(text=bpf_code) b.attach_kprobe(event="do_sys_open", fn_name="do_entry_trace") while 1: (task, tid, _, _, ts, msg) = b.trace_fields() printb(b"%f, %s, %d, %s" % (ts, task, tid, msg))
  27. /*4541 w /*4541Ͱ͸࣍ͷΠϕϯτΛϞχλϦϯά͢ΔΑ͏ʹॻ͍ͯ͋Δ w *OWBMJEPSVOFYQFDUFEQSPDFTTFYFDVUJPO w *OWBMJEPSVOFYQFDUFETZTUFNDBMMT w $IBOHFTUPQSPUFDUFEDPOpHVSBUJPOpMFBOECJOBSJFT w

    8SJUFTUPVOFYQFDUFEMPDBUJPOTBOEpMFUZQFT w $SFBUJPOPGVOFYQFDUFEOFUXPSLMJTUFOFST w 5SB⒏DTFOUUPVOFYQFDUFEOFUXPSLEFTUJOBUJPOT w .BMXBSFTUPSBHFPSFYFDVUJPO
  28. ର৅ͷΞϓϦέʔγϣϯ ubuntu@bpf:~/bpf-tutorial/trace-app$ ./app -h Usage: ./app [-ehn] -e echo message

    -h help -n display hostname ubuntu@bpf:~/bpf-tutorial/trace-app$ ./app -e hello hello ubuntu@bpf:~/bpf-tutorial/trace-app$ ./app -n bpf
  29. ςετͷํ๏ ubuntu@bpf:~/bpf-tutorial/trace-app$ ./app -i uid=1000(ubuntu) gid=1000(ubuntu) groups=1000(ubuntu) ubuntu@bpf:~/bpf-tutorial/trace-app$ ./app -p

    root:x:0:0:root:/root:/bin/bash root@bpf:~/bpf-tutorial/trace-app# python3 trace-app.py ... Detect unexpected file open : /etc/passwd Detect unexpected command execution : id
  30. QSPDTFMGSPPU wQSPDTFMGSPPUFUDQBTTXEΛ։͘͜ͱͰόΠύε - list: sensitive_file_names items: [/etc/shadow] - macro: sensitive_files

    condition: > fd.name startswith /etc and fd.name in (sensitive_file_names) - rule: Read sensitive file untrusted condition: > sensitive_files and open_read and proc_name_exists
  31.  wTIDHJUͷνΣοΫΛόΠύε - macro: spawned_process condition: evt.type = execve -

    rule: Spawn git process from php desc: Spawn git process from php condition: proc.pname=php and spawned_process and proc.cmdline startswith "sh -c git" $ php -r 'system("git --version")'; $ php -r 'system("$(echo \"git --version\")");'
  32. ίϝϯτΞ΢τ wҰ෦ͷίϚϯυ໊Λআ֎͍ͯ͠Δ৔߹ʹίϝϯτʹؚΊΔ - macro: batch_job condition: (proc.cmdline contains "run-job.sh") -

    rule: Spawn processes from php desc: Spawn processes from php condition: proc.pname=php and spawned_process and not batch_job $ php -r 'system("whoami")'; $ php -r 'system("whoami; # run-jon.sh");'