Upgrade to Pro — share decks privately, control downloads, hide ads and more …

toward_systematization_of_linux_trace_tool

Takuma Kume
December 23, 2017
120

 toward_systematization_of_linux_trace_tool

Takuma Kume

December 23, 2017
Tweet

More Decks by Takuma Kume

Transcript

  1. τϨʔεπʔϧͷશମ૾ͱ࢓૊Έɺબ୒
    ٱถ୓അ(.01FQBCP *OD
    8FC4ZTUFN"SDIJUFDUVSFݚڀձ
    -JOVYτϨʔεπʔϧͷମܥԽʹ޲͚ͯ

    View Slide

  2. (.0ϖύϘΠϯϑϥΤϯδχΞ
    ٱถ୓അ!UBLVNBLVNF
    ϗεςΟϯάࣄۀ෦
    IUUQTUBLVNBLVNFUFDI

    View Slide

  3. Systemtap
    eBPF
    kprobe
    perf
    Dynamic Trace
    uprobe
    Static Trace
    BCC
    ftrace
    /sys/kernel/debug/tracing/
    Kernel Module
    BPF Bytecode
    strace
    Events
    Tracepoint
    ໨࣍
    wഎܠɺ՝୊ɺલఏɺ໨త
    wτϨʔεख๏ͷ੔ཧ
    wτϨʔεख๏ͷ࢓૊Έ
    wࠓޙͷ՝୊

    View Slide

  4. Systemtap
    eBPF
    kprobe
    perf
    Dynamic Trace
    uprobe
    Static Trace
    BCC
    ftrace
    /sys/kernel/debug/tracing/
    Kernel Module
    BPF Bytecode
    strace
    Events
    Tracepoint
    എܠɾ՝୊ɾલఏ

    View Slide

  5. Systemtap
    eBPF
    kprobe
    perf
    Dynamic Trace
    uprobe
    Static Trace
    BCC
    ftrace
    /sys/kernel/debug/tracing/
    Kernel Module
    BPF Bytecode
    strace
    Events
    Tracepoint
    എܠɾ՝୊
    wݪҼಛఆ͕Ͱ͖ͳ͍ߴෛՙ͕ൃੜ͢Δɻ
    wݪҼಛఆ͕Ͱ͖ͳ͍αʔόμ΢ϯɺϓϩηεμ΢ϯ͕ൃੜ͢Δɻ
    wݪҼ͕ಛఆͰ͖ͳ͍ϨΠςϯγʔͷஶ͍͠௿Լ͕ൃੜ͢Δɻ
    ฐ͕ࣾఏڙ͢ΔϗεςΟϯάαʔϏεͰ͓٬༷ʹି͠ग़͍ͯ͠Δ
    -JOVYͷαʔόͷӡ༻ɾ։ൃʹ͓͍ͯҎԼͷΑ͏ͳ՝୊͕͋Δɻ

    View Slide

  6. Systemtap
    eBPF
    kprobe
    perf
    Dynamic Trace
    uprobe
    Static Trace
    BCC
    ftrace
    /sys/kernel/debug/tracing/
    Kernel Module
    BPF Bytecode
    strace
    Events
    Tracepoint
    લఏɿϗεςΟϯάαʔϏεͷఏڙํࣜ
    wϗεςΟϯάαʔϏεͰ͸γϯάϧߏ੒ͷ୆ͷαʔόΛෳ਺ਓ
    Ͱڞ༗ͯ͠͝ར༻͍ͨͩ͘ํࣜΛͱ͍ͬͯΔɻ
    w୆ͷαʔόΛΑΓଟ͘ͷਓʹ͝ར༻͍ͨͩ͘͜ͱͰɺརӹ͸େ
    ͖͘ͳΔɻ
    ୆ͷαʔό౰ͨΓͷӨڹ౓߹͍ɾίετޮ཰ͷ؍఺Ͱɺো֐ͷݪ
    Ҽಛఆ΍ଟछଟ༷ͷϓϩάϥϜΛޮ཰Ϋ࣮ߦͯ͠๯಄Ͱड़΂ͨ՝୊
    Λղܾ͢Δ͜ͱ͸ඇৗʹॏཁͰ͋Δɻ

    View Slide

  7. Systemtap
    eBPF
    kprobe
    perf
    Dynamic Trace
    uprobe
    Static Trace
    BCC
    ftrace
    /sys/kernel/debug/tracing/
    Kernel Module
    BPF Bytecode
    strace
    Events
    Tracepoint
    લఏɿϗεςΟϯάαʔϏεαʔόͷಛੑ
    w͓٬༷͸ݖݶͷൣғͰࣗ༝ʹϓϩάϥϜΛ࣮ߦͰ͖Δɻ
    wϓϩάϥϜͷҰ෦͸͞·͟·ͳෛՙΛੜΉɻ
    w$16 ϝϞϦ σΟεΫ*0 τϥϑΟοΫ ϓϩηεϒϩοΩϯά
    wͳͲʜ
    wݱ࣌఺ͰಛఆͰ͖ͳ͍ෛՙ΋͋Δɻ

    View Slide

  8. Systemtap
    eBPF
    kprobe
    perf
    Dynamic Trace
    uprobe
    Static Trace
    BCC
    ftrace
    /sys/kernel/debug/tracing/
    Kernel Module
    BPF Bytecode
    strace
    Events
    Tracepoint
    લఏɿϗεςΟϯάαʔϏεαʔόͷػೳ
    sshd
    crond
    UserA UserB …
    UserA
    UserB

    ϗεςΟϯάαʔϏεͷ8FCαʔό಺ͷओͳఏڙػೳ
    httpd
    UserA UserB …
    ͓٬༷͕ࣗ༝ʹૢ࡞Ͱ͖ΔՕॴ͕ෳ਺͋ΔͨΊ੾Γ෼͚͕ඞཁ

    View Slide

  9. Systemtap
    eBPF
    kprobe
    perf
    Dynamic Trace
    uprobe
    Static Trace
    BCC
    ftrace
    /sys/kernel/debug/tracing/
    Kernel Module
    BPF Bytecode
    strace
    Events
    Tracepoint
    લఏɿϗεςΟϯάαʔϏεͷো֐ରԠ
    wUPQ΍QTίϚϯυʹΑͬͯɺͲͷϓϩηε͕ݪҼͱͳ͍ͬͯΔ
    ͔Λಛఆ͢Δɻ
    wTUSBDF΍MTPGίϚϯυʹΑͬͯɺγεςϜίʔϧ΍Φʔϓϯ͠
    ͍ͯΔϑΝΠϧΛ֬ೝͯ͠ෛՙݪҼΛಛఆ͢Δɻ
    w͜ΕͰಛఆͰ͖Δ৔߹΋͋Δ͕ɺಛఆͰ͖ͳ͍৔߹͸औΓ׶͑ͣ
    ݪҼͰ͋ΔϓϩηεΛ࠶ىಈ࣏ͯ͠Δ͜ͱ΋ʜ
    ྫ͑͹ϗεςΟϯάαʔόͷ-"΍$16͕ߴ͍ঢ়ଶ

    View Slide

  10. Systemtap
    eBPF
    kprobe
    perf
    Dynamic Trace
    uprobe
    Static Trace
    BCC
    ftrace
    /sys/kernel/debug/tracing/
    Kernel Module
    BPF Bytecode
    strace
    Events
    Tracepoint
    લఏɿϗεςΟϯάαʔϏεͷো֐ରࡦ
    αʔόͷো֐༧๷΍ݪҼಛఆͷͨΊͷ৘ใͱͯ͠ɺ$16ɾϝϞϦɾ
    ϩά΍αʔϏεϨεϙϯελΠϜͳͲͷ৘ใΛऩू͍ͯ͠Δɻ
    ˠ͔͠͠ɺ͜Ε͚ͩͰ͸ো֐ͷݪҼ͕ಛఆͰ͖ͳ͍͜ͱ͕͠͹͠
    ͹ൃੜ͢Δɻ

    View Slide

  11. Systemtap
    eBPF
    kprobe
    perf
    Dynamic Trace
    uprobe
    Static Trace
    BCC
    ftrace
    /sys/kernel/debug/tracing/
    Kernel Module
    BPF Bytecode
    strace
    Events
    Tracepoint
    લఏɿٻΊͨπʔϧ
    ͜͜·ͰͰɺαʔόͷো֐ରԠͰγεςϜίʔϧͷτϨʔεͳͲ͕
    ༗ޮ׆༻Ͱ͖Δ͜ͱɺͦΕ͚ͩͰ͸଍Γͳ͍͜ͱ͕͕෼͔ͬͨɻ
    wҎԼͷπʔϧΛٻΊΔɻ
    wγεςϜίʔϧͷΈͳΒͣࣗ༝౓͕ߴ͘LFSOFMϨϕϧͰͷτ
    ϨʔεΛͰ͖Δɻ
    wτϨʔεʹࡍͯ͠Φʔόʔϔου͕খ͍͞ɻ

    View Slide

  12. Systemtap
    eBPF
    kprobe
    perf
    Dynamic Trace
    uprobe
    Static Trace
    BCC
    ftrace
    /sys/kernel/debug/tracing/
    Kernel Module
    BPF Bytecode
    strace
    Events
    Tracepoint
    લఏɿ௿Φʔόʔϔουͷඞཁੑ
    wಈ࡞தͷϓϩμΫγϣϯ؀ڥͰ࣮ࢪ͍ͨͨ͠ΊɺύϑΥʔϚϯε
    ʹӨڹΛ༩͑ͨ͘ͳ͍ɻ
    w͍ͭো֐͕ൃੜ͢Δ͔෼͔Βͳ͍ɻՄೳͰ͋Ε͹ɺৗʹτϨʔε
    Λͯ͠৘ใΛऩू͍ͨ͠ɻ
    wܧଓͯ͠ΦʔόʔϔουΛ૿΍ͨ͘͠ͳ͍ɻ

    View Slide

  13. Systemtap
    eBPF
    kprobe
    perf
    Dynamic Trace
    uprobe
    Static Trace
    BCC
    ftrace
    /sys/kernel/debug/tracing/
    Kernel Module
    BPF Bytecode
    strace
    Events
    Tracepoint
    લఏɿΦʔόʔϔουͷߴ͍τϨʔεख๏
    TUSBDF

    TUSBDF
    QSPDFTT
    UBSHFU
    QSPDFTT
    strace -p {target PID}
    ptrace(PTRACE_ATTACH, PID…)
    waitpid
    SFHJTUFS
    systemcall
    SIGTRAP
    ϓϩηεఀࢭ
    ղੳ
    ࠶։

    ϓϩηεऴྃ
    exit
    1.
    2.
    3.
    4. 5.
    6.

    View Slide

  14. Systemtap
    eBPF
    kprobe
    perf
    Dynamic Trace
    uprobe
    Static Trace
    BCC
    ftrace
    /sys/kernel/debug/tracing/
    Kernel Module
    BPF Bytecode
    strace
    Events
    Tracepoint
    લఏɿΦʔόʔϔουͷߴ͍τϨʔεख๏
    TUSBDF

    TUSBDF
    QSPDFTT
    UBSHFU
    QSPDFTT
    strace -p {target PID}
    ptrace(PTRACE_ATTACH, PID…)
    waitpid
    SFHJTUFS
    systemcall
    SIGTRAP
    ϓϩηεఀࢭ
    ղੳ
    ࠶։

    ϓϩηεऴྃ
    exit
    1.
    2.
    3.
    4. 5.
    6.
    γεςϜίʔϧͷղੳதʹ
    ϓϩάϥϜ͕ॠؒతʹఀࢭ͢Δ

    View Slide

  15. Systemtap
    eBPF
    kprobe
    perf
    Dynamic Trace
    uprobe
    Static Trace
    BCC
    ftrace
    /sys/kernel/debug/tracing/
    Kernel Module
    BPF Bytecode
    strace
    Events
    Tracepoint
    લఏɿΦʔόʔϔουͷ௿͍τϨʔεख๏
    ref: http://www.brendangregg.com/linuxperf.html

    View Slide

  16. Systemtap
    eBPF
    kprobe
    perf
    Dynamic Trace
    uprobe
    Static Trace
    BCC
    ftrace
    /sys/kernel/debug/tracing/
    Kernel Module
    BPF Bytecode
    strace
    Events
    Tracepoint
    ໨త

    View Slide

  17. Systemtap
    eBPF
    kprobe
    perf
    Dynamic Trace
    uprobe
    Static Trace
    BCC
    ftrace
    /sys/kernel/debug/tracing/
    Kernel Module
    BPF Bytecode
    strace
    Events
    Tracepoint
    ໨త
    wݪҼෆ໌ͷো֐ʹ͍ͭͯɺ6TFSMFWFM ,FSOFMͷτϨʔε৘ใΛ
    ׆༻ͯ͠ղܾ͢ΔՄೳੑʹ͍ͭͯݕূ͢Δɻ
    wͦͷҝʹɺܧଓతʹτϨʔε৘ใΛऩूͯ͠ো֐࣌ʹ৘ใΛ׆༻
    ͢Δ͜ͱͷ༗ޮੑΛݕূ͢Δɻ
    wαʔϏεͷΦϖϨʔγϣϯΤϯδχΞશһ͕-JOVYͷτϨʔεʹ
    ਫ਼௨͍ͯ͠Δ͜ͱ͸كͰ͋ΔɻͦͷͨΊɺଟ͘ͷΦϖϨʔγϣϯ
    ΤϯδχΞͰ΋-JOVYͷτϨʔε͕Ͱ͖ΔΑ͏ʹମܥԽ͢Δɻ

    View Slide

  18. Systemtap
    eBPF
    kprobe
    perf
    Dynamic Trace
    uprobe
    Static Trace
    BCC
    ftrace
    /sys/kernel/debug/tracing/
    Kernel Module
    BPF Bytecode
    strace
    Events
    Tracepoint
    ࠓճͷൣғ

    View Slide

  19. Systemtap
    eBPF
    kprobe
    perf
    Dynamic Trace
    uprobe
    Static Trace
    BCC
    ftrace
    /sys/kernel/debug/tracing/
    Kernel Module
    BPF Bytecode
    strace
    Events
    Tracepoint
    ࠓճͷൣғ
    wطଘͷ௿ΦʔόʔϔουͳτϨʔεπʔϧ͸ଟ͘ଘࡏ͍ͯ͠Δɻ
    ͦΕͧΕͷπʔϧ͕Ͳ͏͍͏ػೳ΍໾ׂ͕͋Γɺզʑ͕ԿΛબ୒
    ͢Ε͹ྑ͍ͷ͔ΛҎԼͷ؍఺͔Βߟ͑ͨɻ
    wଟ͘ͷτϨʔεख๏ͷ໾ׂΛ੔ཧͯ͠શମ૾Λ௫Ή͜ͱɻ
    wͦΕͧΕͷπʔϧͷ࢓૊Έʹ͍ͭͯಛ௃Λཧղ͢Δ͜ͱɻ

    View Slide

  20. Systemtap
    eBPF
    kprobe
    perf
    Dynamic Trace
    uprobe
    Static Trace
    BCC
    ftrace
    /sys/kernel/debug/tracing/
    Kernel Module
    BPF Bytecode
    strace
    Events
    Tracepoint
    τϨʔεख๏ͷ੔ཧ

    View Slide

  21. Systemtap
    eBPF
    kprobe
    perf
    Dynamic Trace
    uprobe
    Static Trace
    BCC
    ftrace
    /sys/kernel/debug/tracing/
    Kernel Module
    BPF Bytecode
    strace
    Events
    Tracepoint

    View Slide

  22. Systemtap
    eBPF
    kprobe
    perf
    Dynamic Trace
    uprobe
    Static Trace
    BCC
    ftrace
    /sys/kernel/debug/tracing/
    Kernel Module
    BPF Bytecode
    strace
    Events
    Tracepoint
    ࠞཚ͠·͢ʂʂ

    View Slide

  23. Systemtap
    eBPF
    kprobe
    perf
    Dynamic Trace
    uprobe
    Static Trace
    BCC
    ftrace
    /sys/kernel/debug/tracing/
    Kernel Module
    BPF Bytecode
    strace
    Events
    Tracepoint
    ͭͷϨΠϠʹ෼ྨ
    wσʔλιʔεϨΠϠ
    wϋʔυ΢ΣΞɾιϑτ΢ΣΞͷΠϕϯτΛτϨʔε͢Δػߏ
    wϑϨʔϜϫʔΫϨΠϠ
    wσʔλιʔεϨΠϠʹΞΫηε͢ΔͨΊͷ൚༻తͳϑϨʔϜϫʔΫ
    wΞϓϦέʔγϣϯϨΠϠ 6TFSMFWFM

    wϑϨʔϜϫʔΫϨΠϠʹର໋ͯ͠ྩΛग़ͨ͠Γɺऔಘͨ݁͠ՌΛू
    ܭɾදࣔͳͲΛ͢Δ

    View Slide

  24. Systemtap
    eBPF
    kprobe
    perf
    Dynamic Trace
    uprobe
    Static Trace
    BCC
    ftrace
    /sys/kernel/debug/tracing/
    Kernel Module
    BPF Bytecode
    strace
    Events
    Tracepoint
    Πϕϯτͷ෼ྨ
    wτϨʔεͱ͸ɺ֤ॴ͔Βൃߦ͞ΕΔΠϕϯτΛτϨʔε͢Δͱ͍
    ͏͜ͱɻ
    Software
    Event
    Hardware
    Event
    GPU
    Event
    Memory
    Event
    CPU
    Event
    Dynamic
    Event
    Static
    Event

    View Slide

  25. Systemtap
    eBPF
    kprobe
    perf
    Dynamic Trace
    uprobe
    Static Trace
    BCC
    ftrace
    /sys/kernel/debug/tracing/
    Kernel Module
    BPF Bytecode
    strace
    Events
    Tracepoint
    τϨʔεπʔϧͷϨΠϠʹΑΔ෼ྨ
    bcc
    (eBPF scripts)
    systemtap
    perf-tool
    (ftrace scripts)
    perf
    eBPF
    kernel module
    (Inserted by tool)
    Ftrace
    (debugfs)
    perf_event
    uprobe Tracepoint
    kprobe
    PMU
    Static Event
    Dynamic Event
    CPU Event
    Software
    Event
    Hardware
    Event
    Πϕϯτ
    σʔλιʔεϨΠϠ
    ϑϨʔϜϫʔΫϨΠϠ
    ΞϓϦέʔγϣϯϨΠϠ
    user kernel

    View Slide

  26. Systemtap
    eBPF
    kprobe
    perf
    Dynamic Trace
    uprobe
    Static Trace
    BCC
    ftrace
    /sys/kernel/debug/tracing/
    Kernel Module
    BPF Bytecode
    strace
    Events
    Tracepoint
    τϨʔεख๏ͷ࢓૊Έ

    View Slide

  27. Systemtap
    eBPF
    kprobe
    perf
    Dynamic Trace
    uprobe
    Static Trace
    BCC
    ftrace
    /sys/kernel/debug/tracing/
    Kernel Module
    BPF Bytecode
    strace
    Events
    Tracepoint
    σʔλιʔεϨΠϠ
    w1.6 1FSGPSNBODF.POJUPSJOH6OJUT

    w$16ͳͲͷϋʔυ΢ΣΞͷύϑΥʔϚϯεʹؔ͢ΔΠϕϯτ
    ʹΞΫηε͢ΔͨΊͷΠϯλʔϑΣΠε
    wLQSPCFVQSPCF
    wಈ࡞தͷιϑτ΢ΣΞʹΧελϜΠϕϯτΛ࡞੒͢Δɻ
    w,FSOFM͕ର৅ͳΒLQSPCF 6TBFMFWFM͕ର৅ͳΒVQSPCFΛ
    ར༻ͯ͠ɺιϑτ΢ΣΞͷΠϕϯτΛτϨʔε͢Δɻ
    w5SBDFQPJOU
    wLFSOFMʹ૊Έࠐ·Ε͍ͯΔ੩తͳτϨʔεϙΠϯτ

    View Slide

  28. Systemtap
    eBPF
    kprobe
    perf
    Dynamic Trace
    uprobe
    Static Trace
    BCC
    ftrace
    /sys/kernel/debug/tracing/
    Kernel Module
    BPF Bytecode
    strace
    Events
    Tracepoint
    ϑϨʔϜϫʔΫϨΠϠ
    w'USBDF
    wEFCVHGTͱ͍͏ಛघͳϑΝΠϧγεςϜΛΠϯλʔϑΣΠε
    ʹ͍࣋ͬͯΔɻTZTLFSOFMEFCVHUSBDJOH഑ԼʹτϨʔ
    εͷ༗ޮԽ΍ϑΟϧλϦϯάͷͨΊͷϑΝΠϧɺτϨʔε݁Ռ
    Λग़ྗ͢ΔͨΊͷϑΝΠϧ͕ଘࡏ͍ͯ͠Δɻ
    wUSBDFQPJOU LQSPCF VQSPCFΛ࢖ͬͯτϨʔεΛߦ͏͜ͱ͕
    Ͱ͖Δɻ
    w୯ҰϢʔβ SPPU
    ͷΈͰಈ࡞͢Δɻ
    wϓϩάϥϚϒϧͰ͸ͳ͍ɻ

    View Slide

  29. Systemtap
    eBPF
    kprobe
    perf
    Dynamic Trace
    uprobe
    Static Trace
    BCC
    ftrace
    /sys/kernel/debug/tracing/
    Kernel Module
    BPF Bytecode
    strace
    Events
    Tracepoint
    ϑϨʔϜϫʔΫϨΠϠ
    [email protected]
    wτϨʔεର৅ ϋʔυ΢ΣΞΠϕϯτ ιϑτ΢ΣΞΠϕϯτ
    τϨʔεϙΠτ
    Λࢦఆͯ͠[email protected]@PQFO
    Λ࣮ߦ͠
    ͯΠϕϯτΛొ࿥͢Δɻ
    w1.6 USBDFQPJOUΛ࢖ͬͯτϨʔεΛߦ͏͜ͱ͕Ͱ͖Δɻ
    wLQSPCFʹΑΔಈతτϨʔε͸-JOVYҎ͔߱ΒՄೳͰ
    CQG
    Λར༻͢Δɻ

    View Slide

  30. Systemtap
    eBPF
    kprobe
    perf
    Dynamic Trace
    uprobe
    Static Trace
    BCC
    ftrace
    /sys/kernel/debug/tracing/
    Kernel Module
    BPF Bytecode
    strace
    Events
    Tracepoint
    ϑϨʔϜϫʔΫϨΠϠ
    w F#1'
    w 1.6 USBDFQPJOU LQSPCF VQSPCFΛ࢖ͬͯτϨʔεΛߦ͏͜ͱ͕Ͱ͖Δɻ
    w F#1'͸6TFSMFWFMͰ࡞੒ͨ͠#ZUFDPEFΛCQG
    Λ࢖ͬͯLFSOFMۭؒʹϩʔ
    υ͠ɺ࣮ߦ͢Δ͜ͱ͕Ͱ͖ΔɻF#1'ʹΑΔτϨʔεͰ͸ɺલड़ͨ͠σʔλιʔ
    εʹΠϕϯτΛొ࿥͢Δ͜ͱͰ࣮ݱͰ͖Δɻ
    w F#1'ͷಛ௃ͱͯ͠ɺ఺্͛Δɻ
    w LFSOFMۭؒʹಡΈࠐ·ΕΔ#ZUFDPEF͸F#1'ͷ7FSJpFSʹΑͬͯίʔυͷ
    ҆શੑ͕νΣοΫ͞Εෆਖ਼ͳϝϞϦɺελοΫ΁ͷΞΫηε΍ແݶϧʔϓ
    ͳͲΛݕ஌͢Δ͜ͱͰ҆શੑΛ֬อ͍ͯ͠Δɻ
    w LFSOFMۭؒͱ6TFSMFWFMͷؒͰσʔλͷ΍ΓऔΓΛ͢ΔͨΊʹF#1'ͷ.BQ
    Λ࡞੒͢Δ͜ͱ͕Ͱ͖ΔɻͦΕ͸)BTI "SSBZͳͲͷܗࣜΛ΋͍ͬͯΔɻ
    w Φʔόʔϔου͕ඇৗʹগͳ͍ɻ

    View Slide

  31. Systemtap
    eBPF
    kprobe
    perf
    Dynamic Trace
    uprobe
    Static Trace
    BCC
    ftrace
    /sys/kernel/debug/tracing/
    Kernel Module
    BPF Bytecode
    strace
    Events
    Tracepoint
    bcc
    (eBPF scripts)
    systemtap
    perf-tool
    (ftrace scripts)
    perf
    eBPF
    kernel module
    (Inserted by tool)
    Ftrace
    (debugfs)
    perf_event
    uprobe Tracepoint
    kprobe
    PMU
    Static Event
    Dynamic Event
    CPU Event
    Software
    Event
    Hardware
    Event
    Πϕϯτ
    σʔλιʔεϨΠϠ
    ϑϨʔϜϫʔΫϨΠϠ
    ΞϓϦέʔγϣϯϨΠϠ
    user kernel
    eBPF
    kprobe
    Dynamic Event
    Software
    Event
    Πϕϯτ
    σʔλιʔεϨΠϠ
    ϑϨʔϜϫʔΫϨΠϠ
    ΞϓϦέʔγϣϯϨΠϠ
    user kernel

    View Slide

  32. VTFS
    LFSOFM
    ϓϩάϥϜ
    F#1'
    LQSPCFT
    LQSPCFTF#1'

    View Slide

  33. VTFS
    LFSOFM
    ϓϩάϥϜ
    #1'CZUFDPEF
    F#1'
    ੜ੒
    LQSPCFT
    LQSPCFTF#1'

    View Slide

  34. VTFS
    LFSOFM
    ϓϩάϥϜ
    #1'CZUFDPEF
    F#1'
    WFSJpFS
    ੜ੒
    MPBE
    LQSPCFT
    LQSPCFTF#1'

    View Slide

  35. VTFS
    LFSOFM
    ϓϩάϥϜ
    #1'CZUFDPEF
    F#1'
    WFSJpFS
    ੜ੒
    MPBE
    LQSPCFT
    ϓϩάϥϜͷ҆શੑ
    ΛνΣοΫ
    LQSPCFTF#1'

    View Slide

  36. VTFS
    LFSOFM
    ϓϩάϥϜ
    #1'CZUFDPEF
    F#1'
    WFSJpFS
    ੜ੒
    MPBE
    #1'CZUFDPEF
    LQSPCFT
    LQSPCFTF#1'

    View Slide

  37. VTFS
    LFSOFM
    ϓϩάϥϜ
    #1'CZUFDPEF
    F#1'
    WFSJpFS
    ੜ੒
    MPBE
    #1'CZUFDPEF NBQ
    ੜ੒
    6TFS,FSOFMؒͰσʔλͷ΍ΓऔΓΛ͢Δ
    )BTI
    "SSBZ
    FUDʜ
    LQSPCFT
    LQSPCFTF#1'

    View Slide

  38. LQSPCFTF#1'
    VTFS
    LFSOFM
    ϓϩάϥϜ
    #1'CZUFDPEF
    F#1'
    WFSJpFS
    ੜ੒
    MPBE
    #1'CZUFDPEF NBQ
    USBDJOH
    ॻ͖ࠐΈ
    ࢀর
    LQSPCFT

    View Slide

  39. Systemtap
    eBPF
    kprobe
    perf
    Dynamic Trace
    uprobe
    Static Trace
    BCC
    ftrace
    /sys/kernel/debug/tracing/
    Kernel Module
    BPF Bytecode
    strace
    Events
    Tracepoint
    F#1'ͷΦʔόʔϔου͕௿͍ཧ༝
    user
    kernel
    user
    kernel
    ैདྷख๏ eBPF
    ϓϩάϥϜ
    τϨʔαʔ
    ϑϨʔϜϫʔΫ
    σʔλιʔε
    ϑΟϧλ
    ूܭ දࣔ
    ϓϩάϥϜ
    τϨʔαˍूܭ
    ϑϨʔϜϫʔΫ
    σʔλιʔε
    දࣔ
    ࡞੒ ࡞੒
    Map
    ऩू
    ॻࠐΈ

    View Slide

  40. Systemtap
    eBPF
    kprobe
    perf
    Dynamic Trace
    uprobe
    Static Trace
    BCC
    ftrace
    /sys/kernel/debug/tracing/
    Kernel Module
    BPF Bytecode
    strace
    Events
    Tracepoint
    F#1'ͷΦʔόʔϔου͕௿͍ཧ༝
    user
    kernel
    user
    kernel
    ैདྷख๏ eBPF
    ϓϩάϥϜ
    τϨʔαʔ
    ϑϨʔϜϫʔΫ
    σʔλιʔε
    ϑΟϧλ
    ूܭ දࣔ
    ϓϩάϥϜ
    τϨʔαˍूܭ
    ϑϨʔϜϫʔΫ
    σʔλιʔε
    දࣔ
    ࡞੒ ࡞੒
    Map
    ऩू
    ॻࠐΈ
    F#1'͸ϓϩάϥϚϒϧͰ
    NBQʹΑΔσʔλͷڞ༗͕ՄೳͰ͋Δ
    τϨʔε݁ՌͷϑΟϧλ΍ूܭΛLFSOFM
    ଆͰ׬݁Ͱ͖Δɻ
    Φʔόʔϔου͕௿͍ʂ

    View Slide

  41. Systemtap
    eBPF
    kprobe
    perf
    Dynamic Trace
    uprobe
    Static Trace
    BCC
    ftrace
    /sys/kernel/debug/tracing/
    Kernel Module
    BPF Bytecode
    strace
    Events
    Tracepoint
    ͍͞͝ʹ
    w-JOVYͷτϨʔεπʔϧ͸ɺσʔλιʔεϨΠϠɾϑϨʔϜϫʔ
    ΫϨΠϠɾΞϓϦέʔγϣϯϨΠϠͷΑ͏ͳ֊૚ߏ଄Ͱ෼͚Δͱ
    Πϝʔδ͠΍͍͢ɻ
    wπʔϧͷબ୒ͷ؍఺ͱͯ͠͸ɺͲͷϑϨʔϜϫʔΫΛ࢖ͬͯτϨʔ
    εΛ͢Δ͔ʹΑͬͯΦʔόʔϔου΍τϨʔεର৅͕มΘΔɻ
    wͲͷτϨʔεख๏Ͱ΋༻్ʹΑͬͯ͸ඞཁे෼Ͱ͋Δ͕ɺ৽͍͠
    -JOVYΛ࢖͍ͬͯΔͳΒF#1'Λ࠾༻͍ͨ͠ɻ௿Φʔόʔϔουʹ
    ͩ͜ΘΔඞཁ͕͋Ε͹ಛʹɻ

    View Slide

  42. Systemtap
    eBPF
    kprobe
    perf
    Dynamic Trace
    uprobe
    Static Trace
    BCC
    ftrace
    /sys/kernel/debug/tracing/
    Kernel Module
    BPF Bytecode
    strace
    Events
    Tracepoint
    ࠓޙͷ՝୊
    wࠓճ͸τϨʔεख๏ͷશମ૾΍ͦΕͧΕͷಛੑʹ͍ͭͯΛௐࠪ͠
    ͨɻ͔͠͠ɺ࣮ࡍʹΦʔόʔϔουͷଌఆ΍ɺ͸͡Ίʹड़΂ͨ՝
    ୊ʹରͯ͠ͷ༗ޮੑͷݕূ͕Ͱ͖͍ͯͳ͍ͨΊࠓޙ΍͍ͬͯ͘ɻ
    w͜ΕΒͷπʔϧΛମܥԽ͢Δʹ͋ͨͬͯ͸ɺ༗ޮੑΛࣔͨ͠ޙʹ
    ந৅Խ͞ΕͨϢʔεέʔεͱϓϥΫςΟεΛ໢ཏ͢Δ͜ͱͱɺඞ
    ཁʹԠͯ͡τϨʔεϑϨʔϜϫʔΫΛ࢖ͬͨιϑτ΢ΣΞͷ։ൃ
    ͕ඞཁʹͳΔɻ

    View Slide

  43. Systemtap
    eBPF
    kprobe
    perf
    Dynamic Trace
    uprobe
    Static Trace
    BCC
    ftrace
    /sys/kernel/debug/tracing/
    Kernel Module
    BPF Bytecode
    strace
    Events
    Tracepoint

    View Slide