きっと明日から役立つeBPFのしくみ

͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ -*/&גࣜձࣾ ૣ઒ါଠ࿕

ࣗݾ঺հ • ໊લૣ઒ါଠ࿕ ͸΍͔Θ Ώ͏ͨΖ͏ • ॴଐ-*/&גࣜձࣾ ωοτϫʔΫ։ൃνʔϜ • 5XJUUFS!:VUBSP)BZBLBXB
• (4P$ ͰF#1'Λ'SFF#4%Ҡ২͢ΔϓϩδΣΫτ Λ΍͍ͬͯͨ ࠓ΋΍͍ͬͯΔ • ޷͖ͳ࿩ • ߴ଎ύέοτॲཧ /FUNBQ 9%1 • -JOVY 'SFF#4%ͷωοτϫʔΫελοΫपΓ /FUGJMUFS 1' FUD • ωοτϫʔΫϓϩάϥϚϏϦςΟతͳ࿩ 1 FUD • F#1' ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ cૣ઒ါଠ࿕ 1 Twitterのアイコン

ࠓ೔ͷ࿩ • ࠷ۙ-JOVYք۾ͰF#1'͕Ξπ͍ • ʮF#1'Λ࢖ͬͨͳʹ͔ʯʹ͍ͭͯ͸͍ͩͿ৘ใ͕ग़ͯ͘ΔΑ͏ʹͳͬͨ • ʮF#1'ͦͷ΋ͷʯʹ͍ͭͯ͸৘ใ͕গͳ͍ • ී௨ʹF#1'Λ࢖͍ͬͯΔͱϒϥοΫϘοΫεʹͳΓ͕ͪͳͱ͜ΖΛݟ͍ͯ͘ •
جຊతʹϚχΞοΫͳ࿩Ͱ͢ ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ cૣ઒ါଠ࿕ 2

Ξ΢τϥΠϯ -JOVYF#1'جຊฤ ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ cૣ઒ါଠ࿕ 3

Ξ΢τϥΠϯ -JOVYF#1'جຊฤ ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ cૣ઒ါଠ࿕ 4

F#1'ͱ͸ʁ • &YUFOEFE#FSLFMFZ1BDLFU'JMUFS • -JOVYΧʔωϧͷ֦ுػೳΛॻͨ͘Ίͷ%4- -JOVYd • ΞηϯϒϦͬΆ͍ • UDQEVNQͱ͔XJSFTIBSLͰ࢖ΘΕ͍ͯΔ#1'ͷ֦ு
͕ͩ΋͸΍ผ෺ • -JOVYΧʔωϧͷதʹΠϯλʔϓϦλͱ+*5ίϯύΠϥ͕ೖ͍ͬͯΔ • 1BDLFU'JMUFSͱݴ͍ͬͯΔ͕ ࣮ࡍʹ͸΋ͬͱ͍ΖΜͳͱ͜ΖͰ࢖ΘΕ͍ͯΔ • γεςϜίʔϧͷϑΟϧλϦϯά 4FDDPNQ • μΠφϛοΫτϨʔγϯά LQSPCF QFSG • ύέοτॲཧ 9%1 5$ 4PDLFU'JMUFS • FUD ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ cૣ઒ါଠ࿕ 5

ݴޠ࢓༷ ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ cૣ઒ါଠ࿕ 6 • 3*4$ͬΆ͍*4" • CJUͷݻఆ௕໋ྩ • ݻఆ௕ͷελοΫ
• CJUͷϨδελຊ ൚༻SdS ϑϨʔϜϙΠϯλS • "-6 +.1 -%45ܥ໋ྩ͕Ұ௨Γ • "UPNJD໋ྩ #1'༝དྷͷύέοτόοϑΝΛૢ࡞͢Δ໋ྩ • $BMM໋ྩ͕͋Δ IFMQFSGVODUJPOTͱݺ͹ΕΔΧʔωϧͷؔ਺͕ݺ΂Δ • ௚઀ॻ͘ͷ͸͠ΜͲ͍ͷͰ$Ͱॻ͍ͯίϯύΠϧ͢Δ • --7.ͷόοΫΤϯυ͕͋Δ struct bpf_insn { __u8 opcode; __u8 dst_reg:4 __u8 src_reg:4 __s16 off; __s32 imm };

$ͷݺͼग़͠نଇʹؔ͢Δ޻෉ ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ cૣ઒ါଠ࿕ 7 eBPF x86-64 分類 R0 RAX Return
value from function R1 RDI 第1引数 R2 RSI 第2引数 R3 RDX 第3引数 R4 RCX 第4引数 R5 R8 第5引数 R6 RBX Callee saved R7 R13 Callee saved R8 R14 Callee saved R9 R15 Callee saved R10 RBP Frame pointer (read only) $ͷݺͼग़͠نଇ͕+*5ͨ͠ͱ͖ʹYͱ"3.ͱίϯύνʹͳΔΑ͏ʹ࡞ΒΕ͍ͯΔ ''*ͷΦʔόʔϔου͕ͱͯ΋গͳ͍

.BQ ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ cૣ઒ါଠ࿕ 8 • F#1'ͱϢʔβεϖʔε͔Β࢖͑Δ,74 • "SSBZ )BTI 5SJF
2VFVF 4UBDL FUD • MPPLVQVQEBUFEFMFUFFORVFVFEFRVFVFQFFL • F#1'͔Β͸IFMQFSGVODUJPO • Ϣʔβεϖʔε͔Β͸CQG • Ϣʔεέʔε • Χ΢ϯλ • ϧʔςΟϯάςʔϒϧ Map User App eBPF Program A eBPF Program B User Kernel

CQG ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ cૣ઒ါଠ࿕ 9 • int bpf(int cmd, union
bpf_attr *attr, unsigned int size); • F#1'ͷ࢓૊ΈશൠΛѻ͏γεςϜίʔϧ • όΠτίʔυͷϩʔυ .BQͷ࡞੒ FUD enum bpf_cmd { BPF_MAP_CREATE, BPF_MAP_LOOKUP_ELEM, BPF_MAP_UPDATE_ELEM, BPF_MAP_DELETE_ELEM, BPF_MAP_GET_NEXT_KEY, BPF_PROG_LOAD, ... }; union bpf_attr { struct { /* BPF_MAP_CREATE */ __u32 map_type; __u32 key_size; ... }; struct { /* BPF_PROG_LOAD */ __u32 prog_type; __u32 insn_cnt; __aligned_u64 insns; ... };

7FSJGJFS ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ cૣ઒ါଠ࿕ 10 • F#1'όΠτίʔυࣗମʹηΩϡϦςΟతͳ഑ྀ͸ͳ͍ • Χʔωϧͷ7FSJGJFS͕҆શʹ࣮ߦͰ͖Δ͔੩తʹݕূ͢Δ • ϧʔϓ͸جຊతʹ͸ېࢭ
• +.1໋ྩͰมͳͱ͜Ζʹ௓΅͏ͱ͍ͯ͠ͳ͍͔ʁ • -%45Ͱมͳͱ͜Ζ͔ΒಡΈॻ͖͠Α͏ͱ͍ͯ͠ͳ͍͔ʁ • FUD FUD FUD • νΣοΫΛ௨Βͳ͚Ε͹ϩʔυ͢Β΋Ͱ͖ͳ͍ • ຊ౰ʹ͜ΕͰେৎ෉ͳͷ͔ʁʁ • 4QFDUSF Byte Code User App eBPF Verifier User Kernel bpf (2) fd EPERM Pass! Fail!

ຊ೔ͷ࿩ -JOVYF#1'جຊฤ ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ cૣ઒ါଠ࿕ 11

ຊ೔ͷओ໾ ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ cૣ઒ါଠ࿕ 12 __attribute__((section("maps"))) struct bpf_elf_map pkt_counter = {
.type = BPF_MAP_TYPE_ARRAY, .size_key = sizeof(uint32_t), .size_value = sizeof(uint32_t), .max_elem = 1 }; __attribute__((section("xdp"))) int xdp_program(struct xdp_md *ctx) { uint32_t *count, index = 0; count = bpf_map_lookup_elem(&pkt_counter, &index); if (count == NULL) { return XDP_DROP; } __sync_fetch_and_add(count, 1); return XDP_PASS; } ड৴ύέοτͷ਺Λ਺͑Δ͚ͩͷ F#1' 9%1 ͷϓϩάϥϜ ͜ͷϓϩάϥϜ͕ʮͳΜͰ͏·͘ಈ͘ͷ ͔ʁʯ͸ҙ֎ͱਂ۷Γ͠ͳ͍ͱΘ͔Βͳ ͍

ԿΛ͍ͯ͠Δ͔ ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ cૣ઒ါଠ࿕ 13 ཁૉ਺ͷ"SSBZ.BQΛએݴ .BQͷݕࡧ WBMVF΁ͷϙΠϯλ Λऔಘ WBMVFΛJOQMBDFͰߋ৽ __attribute__((section("maps")))
struct bpf_elf_map pkt_counter = { .type = BPF_MAP_TYPE_ARRAY, .size_key = sizeof(uint32_t), .size_value = sizeof(uint32_t), .max_elem = 1 }; __attribute__((section("xdp"))) int xdp_program(struct xdp_md *ctx) { uint32_t *count, index = 0; count = bpf_map_lookup_elem(&pkt_counter, &index); if (count == NULL) { return XDP_DROP; } __sync_fetch_and_add(count, 1); return XDP_PASS; }

σϞ ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ cૣ઒ါଠ࿕ 14

F#1'ͷ$ϓϩάϥϜ͕࣮ߦ͞ΕΔ·Ͱ ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ cૣ઒ါଠ࿕ 15 NIC driver bpf(2) bpftool demo.c clang
Verifier ここで動く! __attribute__((section("maps"))) struct bpf_elf_map pkt_counter = { .type = BPF_MAP_TYPE_ARRAY, .size_key = sizeof(uint32_t), .size_value = sizeof(uint32_t), .max_elem = 1 }; __attribute__((section("xdp"))) int xdp_program(struct xdp_md *ctx) { uint32_t *count, index = 0; count = bpf_map_lookup_elem(&pkt_counter, &index); if (count == NULL) { return XDP_DROP; } __sync_fetch_and_add(count, 1); return XDP_PASS; }

Verifier ここで動く! CQGUPPM ϩʔμ ͷ͘͠Έ 7FSJGJFSͷ͘͠Έ

F#1' -PBEFS ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ cૣ઒ါଠ࿕ 18 • F#1'ϓϩάϥϜΛΧʔωϧʹϩʔυ͢ΔϢʔβεϖʔεͷϓϩάϥϜ • ࣮૷͸ͭͰ͸ͳ͍ CQGUPPM
JQSPVUF CDD FUD • ͦΕͧΕඍົʹ࢓૊Έ͕ҧ͏ • ओͳ࢓ࣄ &-'ϑΝΠϧ͔Βϩʔυʹඞཁͳ৘ใΛऔΓग़͢ όΠτίʔυΛม׵͢Δ CQG Λ࢖ͬͯϩʔυ͢Δ

CQG ͰόΠτίʔυΛϩʔυ͢Δʹ͸ ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ cૣ઒ါଠ࿕ 20 • #1'@130(@-0"%ίϚϯυΛ࢖͏ • ͜ͷγεςϜίʔϧͷҾ਺Λ&-'ϑΝΠϧ͔Β͔͖ूΊΔ
struct { __u32 prog_type; // プログラムの”タイプ” (XDP, kprobe,etc…) __u32 insn_cnt; // 命令の数 __aligned_u64 insns; // eBPFバイトコードへのポインタ __aligned_u64 license; // ライセンス⽂字列へのポインタ __u32 log_level; // Verifierのログレベル __u32 log_size; // ログバッファのサイズ __aligned_u64 log_buf; // ログバッファへのポインタ __u32 kern_version; // カーネルのバージョン __u32 prog_flags; // フラグ char prog_name[BPF_OBJ_NAME_LEN]; // 名前 __u32 prog_ifindex; // (XDPオフローディングに使われる) };

&-'ͷͲ͔͜Βͱͬͯ͘Δ ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ cૣ઒ါଠ࿕ 21 struct { __u32 prog_type; // バイトコードが置かれているセクションの名前で決める
__u32 insn_cnt; // ELFのメタデータ __aligned_u64 insns; // PROGBITS Typeのついた最初のセクション __aligned_u64 license; // licenseセクションから取ってくる __u32 log_level; __u32 log_size; __aligned_u64 log_buf; __u32 kern_version; // versionセクションから取ってくる __u32 prog_flags; char prog_name[BPF_OBJ_NAME_LEN]; // 関数のシンボル名 __u32 prog_ifindex; }; • CQGUPPMͷ৔߹ ଞͷπʔϧ΋ࣅͨΑ͏ͳײ͡

&-'ηΫγϣϯͷ໋໊نଇ ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ cૣ઒ါଠ࿕ 22 • CQGUPPMͷ৔߹ • ϓϩάϥϜͷλΠϓ<λΠϓݻ༗ͷύϥϝʔλ> • ྫ
YEQ LQSPCFDPOUFYU@TXJUDI • όΠτίʔυΛஔ͍͓ͯ͘ηΫγϣϯ • NBQT.BQͷ৘ใΛஔ͍͓ͯ͘ηΫγϣϯ • WFSTJPO-JOVYΧʔωϧͷόʔδϣϯίʔυΛஔ͍͓ͯ͘ηΫγϣϯ • MJDFOTFϥΠηϯεͷจࣈྻΛஔ͘ηΫγϣϯ

F#1'όΠφϦͷॻ͖׵͑ ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ cૣ઒ါଠ࿕ 24 r1 = 0 *(u32 *)(r10 -4)
= r1 r2 = r10 r2 += -4 r1 = map[id:22] r1 += 208 r0 = *(u32 *)(r2 +0) if r0 >= 0x1 goto pc+3 r0 <<= 3 r0 += r1 goto pc+1 r0 = 0 r1 = 1 if r0 == 0x0 goto pc+3 r1 = 1 lock *(u32 *)(r0 +0) += r1 r1 = 2 r0 = r1 exit r1 = 0 *(u32 *)(r10 - 4) = r1 r2 = r10 r2 += -4 r1 = 0 ll call 1 r1 = 1 if r0 == 0 goto +3 r1 = 1 lock *(u32 *)(r0 + 0) += r1 r1 = 2 r0 = r1 exit r1 = 0 *(u32 *)(r10 - 4) = r1 r2 = r10 r2 += -4 r1 = 3 ll call 1 r1 = 1 if r0 == 0 goto +3 r1 = 1 lock *(u32 *)(r0 + 0) += r1 r1 = 2 r0 = r1 exit EFNPP ίϯύΠϧ௚ޙ CQGUPPM PVUQVU CQG ͷ௚લ WFSJGJFSPVUQVU +*5௚લ

= r1 r2 = r10 r2 += -4 r1 = map[id:22] r1 += 208 r0 = *(u32 *)(r2 +0) if r0 >= 0x1 goto pc+3 r0 <<= 3 r0 += r1 goto pc+1 r0 = 0 r1 = 1 if r0 == 0x0 goto pc+3 r1 = 1 lock *(u32 *)(r0 +0) += r1 r1 = 2 r0 = r1 exit r1 = 0 *(u32 *)(r10 - 4) = r1 r2 = r10 r2 += -4 r1 = 0 ll call 1 r1 = 1 if r0 == 0 goto +3 r1 = 1 lock *(u32 *)(r0 + 0) += r1 r1 = 2 r0 = r1 exit EFNPP ίϯύΠϧ௚ޙ CQGUPPM PVUQVU CQG ͷ௚લ WFSJGJFSPVUQVU +*5௚લ r1 = 0 *(u32 *)(r10 - 4) = r1 r2 = r10 r2 += -4 r1 = 3 ll call 1 r1 = 1 if r0 == 0 goto +3 r1 = 1 lock *(u32 *)(r0 + 0) += r1 r1 = 2 r0 = r1 exit

͜Ε͸$ͷϓϩάϥϜͰ͍͏ͱͲͷ෦෼ʁ ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ cૣ઒ါଠ࿕ 27 r1 = 0 *(u32 *)(r10 -4)
= r1 r2 = r10 r2 += -4 r1 = map[id:22] r1 += 208 r0 = *(u32 *)(r2 +0) if r0 >= 0x1 goto pc+3 r0 <<= 3 r0 += r1 goto pc+1 r0 = 0 r1 = 1 if r0 == 0x0 goto pc+3 r1 = 1 lock *(u32 *)(r0 +0) += r1 r1 = 2 r0 = r1 exit r1 = 0 *(u32 *)(r10 - 4) = r1 r2 = r10 r2 += -4 r1 = 0 ll call 1 r1 = 1 if r0 == 0 goto +3 r1 = 1 lock *(u32 *)(r0 + 0) += r1 r1 = 2 r0 = r1 exit EFNPP ίϯύΠϧ௚ޙ CQGUPPM PVUQVU CQG ͷ௚લ WFSJGJFSPVUQVU +*5௚લ r1 = 0 *(u32 *)(r10 - 4) = r1 r2 = r10 r2 += -4 r1 = 3 ll call 1 r1 = 1 if r0 == 0 goto +3 r1 = 1 lock *(u32 *)(r0 + 0) += r1 r1 = 2 r0 = r1 exit count = bpf_map_lookup_elem(&pkt_counter, &index);

΍΍͍͜͠ͷͰ݁࿦ ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ cૣ઒ါଠ࿕ 28 r1 = 0 *(u32 *)(r10 -4)
= r1 r2 = r10 r2 += -4 r1 = map[id:22] r1 += 208 r0 = *(u32 *)(r2 +0) if r0 >= 0x1 goto pc+3 r0 <<= 3 r0 += r1 goto pc+1 r0 = 0 r1 = 1 if r0 == 0x0 goto pc+3 r1 = 1 lock *(u32 *)(r0 +0) += r1 r1 = 2 r0 = r1 exit r1 = 0 *(u32 *)(r10 - 4) = r1 r2 = r10 r2 += -4 r1 = 0 ll call 1 r1 = 1 if r0 == 0 goto +3 r1 = 1 lock *(u32 *)(r0 + 0) += r1 r1 = 2 r0 = r1 exit EFNPP ίϯύΠϧ௚ޙ CQGUPPM PVUQVU CQG ͷ௚લ WFSJGJFSPVUQVU +*5௚લ r1 = 0 *(u32 *)(r10 - 4) = r1 r2 = r10 r2 += -4 r1 = 3 ll call 1 r1 = 1 if r0 == 0 goto +3 r1 = 1 lock *(u32 *)(r0 + 0) += r1 r1 = 2 r0 = r1 exit .BQͷϑΝΠϧσΟεΫϦϓλ .BQͷ࣮ΞυϨε

.BQͷϑΝΠϧσΟεΫϦϓλΛखʹೖΕΔʹ͸ʁ ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ cૣ઒ါଠ࿕ 29 • CQG #1'@."1@$3&"5&ίϚϯυΛ࢖͏ • ͜ͷγεςϜίʔϧͷҾ਺Λ&-'ϑΝΠϧ͔Β͔͖ूΊΔ
• CQGUPPMTͷ৔߹ lNBQTzηΫγϣϯ͔ΒҾ਺Λͱͬͯ͘Δ struct { __u32 map_type; // Mapの”タイプ” (Array, Hash, Trie…) __u32 key_size; // Keyの⻑さ __u32 value_size; // Valueの⻑さ __u32 max_entries; // 最⼤要素数 __u32 map_flags; // フラグ __u32 inner_map_fd; // (Map in Mapという物がある…) __u32 numa_node; // どのNUMAノードでメモリを取るか char map_name[BPF_OBJ_NAME_LEN]; // 名前 };

$ͷίʔυ ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ cૣ઒ါଠ࿕ 30 __attribute__((section(”maps”))) struct bpf_elf_map pkt_counter = {
.type = BPF_MAP_TYPE_ARRAY, .key_size = sizeof(uint32_t), .value_size = sizeof(uint32_t), .max_entries = 1 }; union bpf_attr { struct { /* BPF_MAP_CREATE */ __u32 map_type; __u32 key_size; __u32 value_size; __u32 max_entries; __u32 map_flags; __u32 inner_map_fd; __u32 numa_node; char map_name[BPF_OBJ_NAME_LEN]; }; lQLU@DPVOUFSz γϯϘϧ໊

Ͳ͏΍ͬͯGEΛόΠτίʔυʹຒΊࠐΉʁʁ ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ cૣ઒ါଠ࿕ 31 r1 = 0 *(u32 *)(r10 -
4) = r1 r2 = r10 r2 += -4 r1 = 0 ll <= ここに埋め込む call 1 r1 = 1 if r0 == 0 goto +3 r1 = 1 lock *(u32 *)(r0 + 0) += r1 r1 = 2 r0 = r1 exit • .BQͷϑΝΠϧσΟεΫϦϓλΛखʹೖΕΔํ๏͸Θ͔ͬͨ • Ͳ͏΍ͬͯຒΊࠐΉʁ • ຒΊࠐ΋͏ʹ΋Ͳ͜ʹຒΊࠐΜͩΒ͍͍͔Θ͔Βͳ͍ͷͰ͸ʁ

ϦϩέʔγϣϯηΫγϣϯΛݟͯΈΔ ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ cૣ઒ါଠ࿕ 32 $ readelf -r demo.o Relocation section
'.rel.text' at offset 0x150 contains 1 entry: Offset Info Type Sym. Value Sym. Name 000000000020 000300000001 unrecognized: 1 0000000000000000 pkt_counter

ϦϩέʔγϣϯηΫγϣϯΛݟͯΈΔ ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ cૣ઒ါଠ࿕ 33 $ readelf -r demo.o Relocation section
'.rel.text' at offset 0x150 contains 1 entry: Offset Info Type Sym. Value Sym. Name 000000000020 000300000001 unrecognized: 1 0000000000000000 pkt_counter

0GGTFUYʹ͸Կ͕͋Δ ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ cૣ઒ါଠ࿕ 34 $ llvm-objdump -disassemble demo.o test.o: file
format ELF64-BPF Disassembly of section .text: xdp_program: 0: b7 01 00 00 00 00 00 00 r1 = 0 1: 63 1a fc ff 00 00 00 00 *(u32 *)(r10 - 4) = r1 2: bf a2 00 00 00 00 00 00 r2 = r10 3: 07 02 00 00 fc ff ff ff r2 += -4 4: 18 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 r1 = 0 ll 6: 85 00 00 00 01 00 00 00 call 1 7: b7 01 00 00 01 00 00 00 r1 = 1 8: 15 00 03 00 00 00 00 00 if r0 == 0 goto +3 <LBB0_2> 9: b7 01 00 00 01 00 00 00 r1 = 1 10: c3 10 00 00 00 00 00 00 lock *(u32 *)(r0 + 0) += r1 11: b7 01 00 00 02 00 00 00 r1 = 2 LBB0_2: 12: bf 10 00 00 00 00 00 00 r0 = r1 13: 95 00 00 00 00 00 00 00 exit

0GGTFUYʹ͸Կ͕͋Δ ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ cૣ઒ါଠ࿕ 35 $ llvm-objdump -disassemble demo.o test.o: file
format ELF64-BPF Disassembly of section .text: xdp_program: 0: b7 01 00 00 00 00 00 00 r1 = 0 1: 63 1a fc ff 00 00 00 00 *(u32 *)(r10 - 4) = r1 2: bf a2 00 00 00 00 00 00 r2 = r10 3: 07 02 00 00 fc ff ff ff r2 += -4 4: 18 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 r1 = 0 ll 6: 85 00 00 00 01 00 00 00 call 1 7: b7 01 00 00 01 00 00 00 r1 = 1 8: 15 00 03 00 00 00 00 00 if r0 == 0 goto +3 <LBB0_2> 9: b7 01 00 00 01 00 00 00 r1 = 1 10: c3 10 00 00 00 00 00 00 lock *(u32 *)(r0 + 0) += r1 11: b7 01 00 00 02 00 00 00 r1 = 2 LBB0_2: 12: bf 10 00 00 00 00 00 00 r0 = r1 13: 95 00 00 00 00 00 00 00 exit

0GGTFUYʹ͸Կ͕͋Δ ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ cૣ઒ါଠ࿕ 36 • MEEX໋ྩ • CJUͷଈ஋ΛϨδελʹϩʔυ͢Δ • CJU࢖͏
• ໋ྩ໨ʹԼҐCJU • ໋ྩ໨ʹ্ҐCJU • ͜͜ʹGEΛຒΊࠐΊΕ͹0, struct bpf_insn { __u8 opcode; __u8 dst_reg:4 __u8 src_reg:4 __s16 off; __s32 imm }; uint64_t foo = 0x12345; struct bpf_insn lddw[2] = { { BPF_LDDW, BPF_R1, 0, 0, (uint32_t)foo }, { 0 , 0, 0, 0, ((uint64_t)foo) >> 32 } };

Χʔωϧ͸Ͳ͏΍ͬͯGEΛݟ͚ͭΔʁʁ ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ cૣ઒ါଠ࿕ 37 • MEEX͸ී௨ʹCJUͷ஋Λϩʔυ͢Δͷʹ΋࢖͑Δ • Χʔωϧ͸Ͳ͏΍ͬͯGEͱී௨ͷ஋Λݟ෼͚Δʁʁ • MEEXͷTSD@SFHͷͱ͜Ζʹಛผͳ஋ΛೖΕ͓ͯ͘
struct bpf_insn { __u8 opcode; __u8 dst_reg:4 __u8 src_reg:4 __s16 off; __s32 imm }; int map_fd = bpf(BPF_MAP_CREATE, &attr, sizeof(attr)); struct bpf_insn lddw[2] = { { BPF_LDDW, BPF_R1, BPF_PSEUDO_MAP_FD, 0, map_fd }, { 0 , 0, 0, 0, 0 } };

F#1' 7FSJGJFS ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ cૣ઒ါଠ࿕ 40 • F#1'ϓϩάϥϜ͕ΧʔωϧͰ҆શʹ࣮ߦͰ͖Δ͔੩తʹݕূ͢Δ࢓૊Έ • ϧʔϓͷݕग़ ϝϞϦΞΫηεҧ൓ͷݕग़
0VUPGCPVOEKVNQͷݕग़ • ϙΠϯλϦʔΫͷݕग़ • FUD FUD FUD • ओͳ࢓ࣄ • ݕূ • ϓϩάϥϜͷॻ͖׵͑ɾ࠷దԽ

__attribute__((section("maps"))) struct bpf_elf_map pkt_counter = { .type = BPF_MAP_TYPE_ARRAY, .size_key
= sizeof(uint32_t), .size_value = sizeof(uint32_t), .max_elem = 1 }; __attribute__((section("xdp"))) int xdp_program(struct xdp_md *ctx) { uint32_t *count, index = 0; count = bpf_map_lookup_elem(&pkt_counter, &index); if (count == NULL) { return XDP_DROP; } __sync_fetch_and_add(count, 1); return XDP_PASS; } F#1'ϓϩάϥϜͷݕূ ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ cૣ઒ါଠ࿕ 42 • ͜ͷνΣοΫ΍Βͳ͔ͬͨΒͲ͏ͳΔͷ ͔ʁʁ • Լͷ@@TZOD@GFUDI@BOE@BEE͸/6--ࢀর ʹͳͬͯ͠·͏ͷ͔ʁʁ • Χʔωϧ͸ࢮΜͰ͠·͏ͷ͔ʁ • 7FSJGJFSʹ஄͔ΕΔͷͰϩʔυͰ͖ͳ͍ • 7FSJGJFS͸Ͳ͏΍ͬͯͦΕΛνΣοΫ͍ͯ͠ Δͷ͔ʁ

ϩʔυ͠Α͏ͱͯ͠ΈΔ ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ cૣ઒ါଠ࿕ 43 # sudo bpftool prog load demo_invalid.o
/sys/fs/bpf/demo type xdp libbpf: load bpf program failed: Permission denied libbpf: -- BEGIN DUMP LOG --- libbpf: 0: (b7) r1 = 0 1: (63) *(u32 *)(r10 -4) = r1 2: (bf) r2 = r10 3: (07) r2 += -4 4: (18) r1 = 0xffff8b1353362800 6: (85) call bpf_map_lookup_elem#1 7: (b7) r1 = 1 8: (c3) lock *(u32 *)(r0 +0) += r1 R0 invalid mem access 'map_value_or_null' libbpf: -- END LOG -- libbpf: failed to load program 'xdp' libbpf: failed to load object 'demo_invalid.o' Error: failed to load object file

3JOWBMJENFNBDDFTTbNBQ@WBMVF@PS@OVMM`ͷҙຯ ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ cૣ઒ါଠ࿕ 44 • NBQ@WBMVF@PS@OVMMͱ͸ͳʹ͔ • 7FSJGJFS͕Ϩδελͷ஋ʹ͚͍ͭͯΔʮܕʯͷΑ͏ͳ΋ͷ • ʮNBQͷWBMVF΋͘͠͸OVMMʯOVMMͳՄೳੑ͕͋Δ
ࢀর͢Δͷ͸ةݥʂ int xdp_program(struct xdp_md *ctx) { uint32_t *count, index=0; count = bpf_map_lookup_elem(&pkt_counter, &index); if (count == NULL) { return XDP_DROP; } __sync_fetch_and_add(count, 1); return XDP_PASS; } 3lNBQ@WBMVF@PS@OVMMz 3lNBQ@WBMVFz ϩʔυͰ͖ͳ͍ϓϩάϥϜ ϩʔυͰ͖ΔϓϩάϥϜ int xdp_program(struct xdp_md *ctx) { uint32_t *count, index = 0; count = bpf_map_lookup_elem(&pkt_counter, &index); __sync_fetch_and_add(count, 1); return XDP_PASS; }

3JOWBMJENFNBDDFTTbNBQ@WBMVF@PS@OVMM`ͷҙຯ ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ cૣ઒ါଠ࿕ 45 • NBQ@WBMVF@PS@OVMMͱ͸ͳʹ͔ • 7FSJGJFS͕Ϩδελͷ஋ʹ͚͍ͭͯΔʮܕʯͷΑ͏ͳ΋ͷ • ʮNBQͷWBMVF΋͘͠͸OVMMʯOVMMͳՄೳੑ͕͋Δ
ࢀর͢Δͷ͸ةݥʂ 3lNBQ@WBMVF@PS@OVMMz 3lNBQ@WBMVFz ϩʔυͰ͖ͳ͍ϓϩάϥϜ ϩʔυͰ͖ΔϓϩάϥϜ r1 = 0 *(u32 *)(r10 - 4) = r1 r2 = r10 r2 += -4 r1 = 0 ll call 1 r1 = 1 lock *(u32 *)(r0 + 0) += r1 r0 = 2 exit r1 = 0 *(u32 *)(r10 - 4) = r1 r2 = r10 r2 += -4 r1 = 0 ll call 1 r1 = 1 if r0 == 0 goto +3 r1 = 1 lock *(u32 *)(r0 + 0) += r1 r1 = 2 r0 = r1 exit

7FSJGJFSʹΑΔόΠτίʔυͷॻ͖׵͑ ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ cૣ઒ါଠ࿕ 47 r1 = 0 *(u32 *)(r10 -4)

7FSJGJFSʹΑΔόΠτίʔυͷ࠷దԽ ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ cૣ઒ါଠ࿕ 48 r1 = 0 *(u32 *)(r10 -4)
= r1 r2 = r10 r2 += -4 r1 = map[id:22] r1 += 208 r0 = *(u32 *)(r2 +0) if r0 >= 0x1 goto pc+3 r0 <<= 3 r0 += r1 goto pc+1 r0 = 0 r1 = 1 if r0 == 0x0 goto pc+3 r1 = 1 lock *(u32 *)(r0 +0) += r1 r1 = 2 r0 = r1 exit r1 = 0 *(u32 *)(r10 - 4) = r1 r2 = r10 r2 += -4 r1 = 0 ll call 1 r1 = 1 if r0 == 0 goto +3 r1 = 1 lock *(u32 *)(r0 + 0) += r1 r1 = 2 r0 = r1 exit EFNPP ίϯύΠϧ௚ޙ CQGUPPM PVUQVU CQG ͷ௚લ WFSJGJFSPVUQVU +*5௚લ r1 = 0 *(u32 *)(r10 - 4) = r1 r2 = r10 r2 += -4 r1 = 3 ll call 1 r1 = 1 if r0 == 0 goto +3 r1 = 1 lock *(u32 *)(r0 + 0) += r1 r1 = 2 r0 = r1 exit DBMMͱ͸ʁ DBMMͷΠϯϥΠϯԽ

DBMMͷҙຯ ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ cૣ઒ါଠ࿕ 49 • )FMQFSGVODUJPOʹ͸Ұҙͷ*%͕͍͍ͭͯͯ CQG@NBQ@MPPLVQ@FMFN͸*% • DBMM໋ྩͷଈ஋ʹ*%ΛೖΕ͓ͯ͘ͱΧʔωϧͷதͰຊ෺ͷؔ਺ݺͼग़͠ʹม׵͞ΕΔ •
$ͰͷIFMQFSGVODUJPO͸FYUFSOؔ਺Ͱ͸ͳؔ͘਺ϙΠϯλͷఆ਺ͱͯ͠ఆٛ͞Ε͍ͯΔ • ίϯύΠϧ͢Δͱ͏·͍۩߹ʹDBMMʹͳΔ enum bpf_func_id { BPF_FUNC_unspec, BPF_FUNC_map_lookup_elem, BPF_FUNC_map_update_elem, BPF_FUNC_map_delete_elem, BPF_FUNC_probe_read, ... }; static void *(*bpf_map_lookup_elem)(void *map, void *key) = (void *) BPF_FUNC_map_lookup_elem;

CQG@NBQ@MPPLVQ@FMFNͷΠϯϥΠϯԽ ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ cૣ઒ါଠ࿕ 50 • )FMQFS'VODUJPOͷॲཧ͸΋ͷʹΑͬͯ͸F#1'ͷ໋ྩ͚ͩͰදݱͰ͖Δ • "SSBZ.BQͷMPPLVQ͸ͦͷ࠷ͨΔྫ • ΠϯϥΠϯԽͰੑೳΛՔ͙
static void * array_map_lookup_elem(struct bpf_map *map, void *key) { struct bpf_array *array = container_of(map, struct bpf_array, map); u32 index = *(u32)key; if (unlikely(index >= array->map.max_entries)) return NULL; return array->value + array->elem_size * (index & array->index_mask); } r1 += 208 r0 = *(u32 *)(r2 +0) if r0 >= 0x1 goto pc+3 r0 <<= 3 r0 += r1 goto pc+1 r0 = 0

CQG@NBQ@MPPLVQ@FMFNͷΠϯϥΠϯԽ ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ cૣ઒ါଠ࿕ 51 • )FMQFS'VODUJPOͷॲཧ͸΋ͷʹΑͬͯ͸F#1'ͷ໋ྩ͚ͩͰදݱͰ͖Δ • "SSBZ.BQͷMPPLVQ͸ͦͷ࠷ͨΔྫ • ΠϯϥΠϯԽͰੑೳΛՔ͙
static void * array_map_lookup_elem(struct bpf_map *map, void *key) { struct bpf_array *array = container_of(map, struct bpf_array, map); u32 index = *(u32)key; if (unlikely(index >= array->map.max_entries)) return NULL; return array->value + array->elem_size * (index & array->index_mask); } r1 += 208 r0 = *(u32 *)(r2 +0) if r0 >= 0x1 goto pc+3 r0 <<= 3 r0 += r1 goto pc+1 r0 = 0

7FSJGJFSʹΑΔόΠτίʔυͷ࠷దԽ ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ cૣ઒ါଠ࿕ 52 r1 = 0 *(u32 *)(r10 -4)

Verifier ここで動く! __attribute__((section("maps"))) struct bpf_elf_map pkt_counter = { .type = BPF_MAP_TYPE_ARRAY, .size_key = sizeof(uint32_t), .size_value = sizeof(uint32_t), .max_elem = 1 }; __attribute__((section("xdp"))) int xdp_program(struct xdp_md *ctx) { uint32_t *count, index = 0; count = bpf_map_lookup_elem(&pkt_counter, &index); if (count == NULL) { return XDP_DROP; } __sync_fetch_and_add(count, 1); return XDP_PASS; }

·ͱΊ ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ cૣ઒ါଠ࿕ 54 • -JOVYͷF#1'ʹ͍ͭͯجຊతͳͱ͜ΖΛ͓͞Β͍ͨ͠ • ݴޠ࢓༷ ''* .BQ
7FSJGJFSʜ • ϓϩάϥϜ͕ϩʔυ͞ΕΔ·ͰͷྲྀΕΛͬ͟ͱݟͨ • F#1'ϩʔμʔͷ͘͠Έ • 7FSJGJFSͷ͘͠Έ • ͜ͷ࿩͕໌೔͔Βօ͞Μͷ໾ʹཱͭ͜ͱΛفΓ·͢

͜Μͳ΋ͷ͸·ͩ·ͩණࢁͷҰ֯ ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ cૣ઒ါଠ࿕ 55 • ษڧʹͳΔࢿྉΛڍ͓͛ͯ͘ • ӳޠ • %JWFJOUP#1'BMJTUPGSFBEJOHNBUFSJBM
ૉ੖Β͍͠F#1'ؔ࿈৘ใͷϦϯΫू • $JMMJVNͷϚχϡΞϧ F#1'Λ࢖ͬͨωοτϫʔΫϓϩάϥϛϯάͳΒ͜͜Λࢀর • #1'1FSGPSNBODF5PPMT ʮৄղγεςϜύϑΥʔϚϯεʯͷ#SFOEBO(SFHHࢯͷ৽࡞ • ೔ຊޠ • ਭ຾ෆ଍ ࠓ೔࿩ͨ͠Α͏ͳϚχΞοΫͳײ͡ͷF#1'ͷ࿩୊͕ࡌ͍ͬͯΔ • !*5ͷ࿈ࡌ !*5ͷF#1'ʹؔ͢Δ࿈ࡌ ϢʔεέʔεΑΓ͸JOUFSOBMͷ࿩͕ଟ͍

એ఻ ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ cૣ઒ါଠ࿕ 56 • ϚϧνϓϥοτϑΥʔϜͰ࢖͑ΔF#1'ॲཧܥ αϒγεςϜͷ։ൃΛ͍ͯ͠·͢ • HFOFSJDFCQG IUUQTHJUIVCDPN:VUBSP)BZBLBXBHFOFSJDFCQG
• 'SFF#4%,FSOFM6TFS -JOVY,FSOFM6TFS .BD046TFS • ߦఔ౓ͷάϧʔίʔυͰF#1'͕ಈ͘ • *OUFSQSFUFS +*5GPSY • .BQ 7FSJGJFS FUD • ίϯτϦϏϡʔλɾϢʔεέʔεҊืूத

2VFTUJPO ͖ͬͱ໌೔͔Β໾ཱͭF#1'ͷ͘͠Έ cૣ઒ါଠ࿕ 57

きっと明日から役立つeBPFのしくみ

きっと明日から役立つeBPFのしくみ

More Decks by Yutaro Hayakawa

Other Decks in Technology

Featured

Transcript