Upgrade to Pro — share decks privately, control downloads, hide ads and more …

"Story of Rucy" on RubyKaigi takeout 2021

2cf373725ded741824c50fd571eda6e1?s=47 KONDO Uchio
September 09, 2021

"Story of Rucy" on RubyKaigi takeout 2021

Rucy is a Ruby Compiler for BPF target. Rucy is named after "Ru-C".

Talk is presented on RubyKaigi takeout 2021:
https://rubykaigi.org/2021-takeout/presentations/udzura.html

2cf373725ded741824c50fd571eda6e1?s=128

KONDO Uchio

September 09, 2021
Tweet

Transcript

  1. Story of Rucy: “Compile” a Ruby Script into BPF

  2. Senior-Principal Engineer@GMO Pepabo DataOps & Dev Productivity Engineering Team RubyKaigi

    Speaker (’16, ’18 && ’19) RubyKaigi Organizer (’19 @ Fukuoka) Hacker Supporter @ Fukuoka City Engineer’s Café Ruby, mruby, Rust, Containers, Kernel, Duolingo Uchio Kondo:
  3. ToC • 1) What is BPF? Has it something to

    do with us? • 2) What is Rucy for, and Why? • 3) What is going on behind Rucy • Overview, BPF Opcode and mruby VM Opcode, Transpilation • 4) Conclusion, demo aiming at the future ࠓ೔࿩͢಺༰Ͱ͢ɻ 3VDZͷ࿩Λ͢Δલʹલఏ஌͕ࣝͨ͘͞Μ͋ΔͷͰɺॱ࣍આ໌͍͖ͯ͠·͢ɻ
  4. What is BPF? Has it something to do with us?

  5. BPF is: • One of the latest Linux SOTAs •

    Used, for example, in the following areas: • Networking • Server Tracing (Observability) • Do you know DTrace? What BPF can do is almost like that. • Device Access Auditing for containers #1'͸-JOVYͷ࠷ઌ୺ٕज़ͷҰͭͰ͢ɻωοτϫʔΫɺύϑΥʔϚϯεɾτϨʔεɺ ίϯςφ಺෦ͷσόΠεΞΫηε੍ޚͷ಺෦ͳͲͰ࢖ΘΕ͍ͯ·͢ɻ
  6. How BPF works: Observability Userspace Kernel BPF binary BPF binary

    BPF VM Userspace Program (libbpf) BPF Map kprobe, tracepoint... Load&Verify Filter/Aggregate Receive Informations Load&Verify Filter/Aggregate Receive Informations Visualize τϨʔεͷ৔߹ɺʮ#1'όΠφϦʯΛΧʔωϧʹϩʔυ͠ɺ Χʔωϧͷ৘ใΛूΊͯɺϢʔβεϖʔεʹฦ͠·͢ɻ
  7. How BPF works: Device access auditing cgroup Kernel Devices Process

    BPF 👍 ❌ Access filter Try to access devices Given access context /dev/urandom /dev/null /dev/zero /dev/tty /dev/sdaXX /dev/sdbXX /dev/sdcXX ...... σόΠεΞΫηε੍ޚͷ࣌͸ɺDHSPVQʹ#1'όΠφϦϓϩάϥϜΛಡΈࠐ·ͤ ΞΫηε࣌ʹ౉͞ΕΔίϯςΫετΛར༻͠ϑΟϧλʔ͠·͢ɻ
  8. What is Rucy for? And... why?

  9. To begin with: Can BPF be used via Ruby? •

    The answer is Yes. • I created RbBCC • BCC is a BPF SDK for Python, and Lua... • I need Ruby’s one. So created. • BCC uses FFI(ctypes) to access libbcc, then RbBCC uses Fiddle. • RbBCC can do basic traces as BCC do. • RbBCC is one of the results of a Ruby Association Grant in 2019. (*) https:/ /github.com/udzura/rbbcc Thanks, ko1-san #1'͸3VCZ͔Β࢖͑·͔͢ʁ౴͑͸ʮ:FTʯͰ͢ɻ 3C#$$ͱ͍͏΋ͷΛ࢖͍·͢ɻ3VCZΞιγΤʔγϣϯͷάϥϯτର৅Ͱ΋͋Γ·͢ɻ
  10. BCC’s pitfalls • Tools with (Rb)BCC do the whole BPF

    compilation process • When it starts tracing - Just In Time • It causes: • 1) overhead • 2) large number of files required to build • The compilation process needs LLVM, clang and kernel headers. ͔͠͠ɺ#$$ʢ3C#$$ʣ͸ɺτϨʔεͷ࠷ॳʹ#1'ϓϩάϥϜΛͦͷ৔ͰίϯύΠϧ͠·͢ɻ Φʔόʔϔου΍ɺίϯύΠϧʹඞཁͳ؀ڥͷංେԽͳͲ໰୊͕͋Γ·͢ɻ
  11. Brand-new BPF tool for Ruby • Linux community recommends to

    precompile the "BPF binary” • As tools using on-the-fly build are such problematic • Tools should be served as “One Binary” ✴ There are also mechanisms to absorb differences in execution environments such as the kernel version. ✴ These portable one-binary commands are called BPF CO-RE (Compile Once, Run Everywhere). BPF Tracing Program Userspace Part (C, Rust, Go...) BPF Binary (Bytecodes) + ͳͷͰࣄલʹίϯύΠϧͨ͠ʮ#1'όΠφϦʯͷར༻͕ਪ঑͞Ε·͢ɻ ϓϩάϥϜຊମʹ#1'όΠφϦΛ݁߹ͯ͠ɺϫϯόΠφϦͰఏڙՄೳͰ͢ɻ
  12. “One Binary” in Ruby • mruby seems to be the

    best way • to achieve the "one binary for everything" goal. • But - "BPF binary” can only be created in C, effectively NSVCZΛ࢖͑͹ϫϯόΠφϦΛ࡞Δ͜ͱ͕Ͱ͖ͦ͏Ͱ͢ɻ Ͱ͕͢ɺʮ#1'όΠφϦʯ΋༻ҙ͠ͳ͍ͱ͍͚·ͤΜɻ͜Ε͸࣮࣭తʹ$ݴޠͰ͔͠ॻ͚·ͤΜɻ
  13. Making a BPF binary • We need to pass the

    C code to the clang command: • I decided to make a tool that enable Rubyists - • To make "BPF binaries" without writing any C code! ✴ In fact, Rust can create BPF binaries using RedBPF crate. But - this is RubyKaigi. 🆕 ͡Ό͋ɺશ෦3VCZͰॻ͚ΔΑ͏ʹ͠·͠ΐ͏ɻ
  14. Summary so far: • BPF is an emerging Linux technology

    • used in fast firewalls, lightweight tracers, and containers security. • You can use BPF technology from Ruby via BCC. • But BCC is problematic because it compiles on the fly. • Precompiled "One binary" is recommended. • You need to write C language for the "BPF binary" part. • Writing every part of BPF program in Ruby is a desire of Rubyists. ͜͜·Ͱͷ·ͱΊͰ͢ɻ ͱʹ͔͘ɺ#1'ʹؔΘΔ͢΂ͯͷύʔτΛ3VCZͰॻ͖ͨ͋͘Γ·ͤΜ͔ʁ
  15. Only with the Rucy. ͦ͏ɺ3VDZͳΒͶɻ

  16. What is going on? Technologies behind Rucy

  17. Reprise: what is Rucy • Rucy is a “Ruby Compiler”

    to BPF target. • In other words: • Rucy generates “BPF binary” directly from a plain-old Ruby script. ✴ Rucy is named after “Ruby Compiler” (Ru-C -> Rucy) 3VDZʢ3V$ʣ͸ʮ3VCZ$PNQJMFSʯͰ͢ɻ ͩ͘Μͷʮ#1'όΠφϦʯΛɺ3VCZεΫϦϓτ͔Β௚઀ίϯύΠϧͯ͠ੜ੒͠·͢ɻ
  18. Architecture overview Ruby Script mruby OpCodes BPF OpCodes BPF Object

    (ELF format) mruby Rucy transpiler ✴ Rucy is built upon Rust , with mruby embedded. ΞʔΩςΫνϟશମͰ͢ɻ3VCZεΫϦϓτΛNSVCZΦϖίʔυʹίϯύΠϧ͠ɺ ͦΕΛ͞Βʹ#1'Φϖίʔυʹม׵ɺ࠷ޙʹ&-'ϑΝΠϧʹ·ͱΊ·͢ɻ
  19. BPF OpCode Overview • BPF has its own VM and

    OpCode set. • The VM can use 10 registers • BPF instructions are basically 64bit fixed length • Instruction classes are: LD, LDX, ST, STX, ALU, JMP, and ALU64 • Layout: ASCII C Struct #1'͸ɺಠࣗͷϨδελϚγϯ7.Λ࣋ͪ·͢ɻͷϨδελ͕͋Γɺ ໋ྩ͸͓͓ΉͶCJUͷݻఆ௕Ͱ͢ɻόΠφϦϨΠΞ΢τ͸εϥΠυͷ௨ΓͰ͢ɻ
  20. BPF instruction examples Instruction Pseudo code #1'໋ྩͷྫɻ-%9ɺ"-6ɺ+.1ͳͲ͕͋Γ·͢ɻ

  21. Mruby OpCode Overview • Like CRuby, mruby has its VM

    and OpCodes • can use 65536 registers, recommended to use only 256. • Instructions are in variable length • It depends on what kind of operands it will take. • The sizes of the operand variable ... • B (8 bits), S (16 bits), W (24 bits), and Z (no operand) NSVCZ΋ϨδελϚγϯͷ7.͕͋Γ·͢ɻϨδελ͸ͨ͘͞Μ࢖͑·͢ɻ ໋ྩ͸Մม௕Ͱ͢ɻ#ɺ4ɺ8ɺ;ͷΦϖϥϯυ͕͋Γ·͢ɻ
  22. mruby instruction examples Bytes OpCode&Operand # Pseudo code NSVCZͷ໋ྩͰ͢ɻͨͱ͑͹01@-0"%*͸CJUɺ01@&/5&3͸CJUͷ໋ྩ௕Ͱ͢ɻ

  23. Transpile OpCodes

  24. Bytecode transpilation process • Here is a sample Ruby code.

    ͯ͞ɺόΠτίʔυΛτϥϯεύΠϧ͢Δํ๏Λઆ໌͠·͢ɻ ͪ͜Β͕ࠓճͷ3VCZͷίʔυͰ͢ɻ
  25. This code outputs mruby OpCodes like: ͜ͷ3VCZίʔυ͸͜͏͍͏NSVCZΦϖίʔυʹͳΓ·͢ɻ

  26. The goal of transpilation mruby opcode BPF opcode ࠷ऴతʹɺ͜͏͍͏#1'ͷΦϖίʔυʹม׵͞Ε·͢ɻ

  27. First: ignore some instructions ·ͣɺ͍͔ͭ͘ͷ#1'ͷରԠ͢Δ໋ྩ͕ͳ͍Φϖίʔυ͸ແࢹ͠·͢ɻ

  28. Second: direct translation (e.g. OP_LOADI) Traspilation Code: ͦͷ··ʮ຋༁ʯͰ͖Δ΋ͷ͸ͦ͏͠·͢ɻ 01@-0"%*͸#1'ͷํͰ΋ͨͩͷ୅ೖʹͳΓ·͢ɻ

  29. Then: object access to structure access / / remember local

    variable name Traspilation Code: ࣍͸ʮΦϒδΣΫτ΁ͷଐੑతϝιουʯΛʮߏ଄ମͷϝϯόʯʹ຋༁͠·͢ɻ ࠷ॳʹɺ3ʹDUYม਺͕ೖ͍ͬͯΔ͜ͱΛอଘ͓͖ͯ͠·͢ɻ
  30. Then: object access to structure access / / calculate offset

    / / with varname, symname / / c.f. ctx.send(:minor) ม਺໊ͱɺγϯϘϧ໊͕෼͔Ε͹ ߏ଄ମͱͯ͠ͷϝϯό΁ͷΦϑηοτ͕ܭࢉͰ͖ɺ#1'ʹ຋༁Ͱ͖·͢ɻ
  31. More: if state to JUDGE/GOTO “R3: bool = R3 ==

    R4” Traspilation Code: ࣍͸+.1໋ྩͰ͕͢ɺNSVCZ͸໋ྩඞཁͳͷʹɺ#1'͸໋ྩʹͳΓ·͢ɻ ࠩΛຒΊΔͨΊɺ·ͣɺ01@&2Λݮࢉૢ࡞ʹม׵ͯ͠͠·͍·͢ɻ
  32. More: if state to JUDGE/GOTO / / save GOTO label

    info / / Check r3 as “u32” Traspilation Code: ࣍ͷ໋ྩͰʮ3͕θϩ͔൱͔ʯΛݕࠪͯ͠(050͢Ε͹ɺ શମͱͯ͠01@&2 01@+.1/05Λ຋༁͢Δ͜ͱ͕Ͱ͖·ͨ͠ɻ
  33. Finally: return a register value / / R0 = R3

    / / exit (return R0) Traspilation Code: ࠷ޙʹSFUVSOॲཧͰ͢ɻNSVCZ͸೚ҙͷϨδελΛฦͤ·͕͢ɺ#1'͸3͚ͩͰ͢ɻ NSVCZ͸3Λ࢖Θͳ͍ͷͰɺܾΊଧͪͰ୅ೖͰ͖·͢ɻ
  34. Again: Generated BPF opcodes ͱ͍͏͜ͱͰɺ#1'ͷΦϖίʔυΛੜ੒Ͱ͖·ͨ͠ɻ

  35. Again: Generated BPF opcodes օ͞ΜΛஔ͍ͯߦ͔ͳ͍Α͏ɺ৭Λ෇͚ͯΈ·ͨ͠ɻόΠφϦ৭෇͚܎Ͱ͢ɻ

  36. Conclusion, demo and future You’ll see the magic!

  37. Conclusion • We can create a BPF binary object from

    Ruby script directly. • This BPF object + mruby + libbpf = whole command binary • Less context switch for programers • Shared data structures (in the future? Not yet implemented...) 3VCZΛίϯύΠϧͯ͠#1'Λ࡞Γ·ͨ͠ɻશ෦3VCZͰॻ͚Δͱศརͩͱࢥ͍·͢ɻ কདྷ͸3VCZͷΫϥεͰɺΧʔωϧͱϢʔβϥϯυͷσʔλߏ଄Λڞ༗Ͱ͖Ε͹ͱɻকདྷ͸ɻ
  38. Demo #1: cgroup device filter cf. counterpart C σϞͦͷɻDHSPVQͷσόΠεΞΫηεϑΟϧλΛ࣮૷͠·͢ɻ

  39. Compile it with rucy command • Then, check the load

    program works properly
  40. Demo #2: Tracing kernel function • Working demo: • https://github.com/udzura/mruby-rubykaigi-rucy-sample

    σϞͦͷɻΧʔωϧͷUDQ@DPOOFDUؔ਺ͷݺͼग़͠ΛɺLQSPCFܦ༝ͰτϨʔε͠·͢ɻ
  41. Tracing TCP’s connect on the host • By mruby cli

    tool embedded with Rucy BPF object
  42. Restrictions and aiming at the future • Only implemented the

    really minimum BPF functionality. • 🙆: Generate very basic BPF opcodes • 🙆: Call BPF-specific helper functions (e.g. bpf_trace_printk()) • 🙅: Handle a BPF Map data structure • 🙅: Inject parameters via rodata, ... • Gradually implement all of them. See you soon! ݱࡏɺຊ౰ʹ࠷௿ݶͷػೳ͔࣮͠૷͍ͯ͠·ͤΜ͕ɺͨͱ͑͹#1'.BQͷར༻ͳͲɺ ॱ࣍ඞཁͦ͏ͳػೳΛ࣮૷͍͖͍ͯͨ͠ؾ࣋ͪ͸͋Γ·͢ɻ
  43. Enjoy binary hack! See you in 2022