toward_systematization_of_linux_trace_tool

WSA研究会
http://websystemarchitecture.hatenablog.jp/

Transcript

1. τϨʔεπʔϧͷશମ૾ͱ࢓૊Έɺબ୒ ٱถ
   ୓അ
   
   (.0
   1FQBCP
   *OD
   
   8FC
   4ZTUFN
   "SDIJUFDUVSFݚڀձ
    -JOVYτϨʔεπʔϧͷମܥԽʹ޲͚ͯ
2. (.0ϖύϘ
   ΠϯϑϥΤϯδχΞ ٱถ
   ୓അ
   !UBLVNBLVNF ϗεςΟϯάࣄۀ෦ IUUQTUBLVNBLVNFUFDI 
3. Systemtap eBPF kprobe perf Dynamic Trace uprobe Static Trace BCC ftrace /sys/kernel/debug/tracing/ Kernel Module BPF Bytecode strace Events Tracepoint ໨࣍ wഎܠɺ՝୊ɺલఏɺ໨త
   wτϨʔεख๏ͷ੔ཧ
   wτϨʔεख๏ͷ࢓૊Έ
   wࠓޙͷ՝୊
4. Systemtap eBPF kprobe perf Dynamic Trace uprobe Static Trace BCC ftrace /sys/kernel/debug/tracing/ Kernel Module BPF Bytecode strace Events Tracepoint എܠɾ՝୊ɾલఏ
5. Systemtap eBPF kprobe perf Dynamic Trace uprobe Static Trace BCC ftrace /sys/kernel/debug/tracing/ Kernel Module BPF Bytecode strace Events Tracepoint എܠɾ՝୊ wݪҼಛఆ͕Ͱ͖ͳ͍ߴෛՙ͕ൃੜ͢Δɻ
   wݪҼಛఆ͕Ͱ͖ͳ͍αʔόμ΢ϯɺϓϩηεμ΢ϯ͕ൃੜ͢Δɻ
   wݪҼ͕ಛఆͰ͖ͳ͍ϨΠςϯγʔͷஶ͍͠௿Լ͕ൃੜ͢Δɻ
   ฐ͕ࣾఏڙ͢ΔϗεςΟϯάαʔϏεͰ͓٬༷ʹି͠ग़͍ͯ͠Δ -JOVYͷαʔόͷӡ༻ɾ։ൃʹ͓͍ͯҎԼͷΑ͏ͳ՝୊͕͋Δɻ 
6. Systemtap eBPF kprobe perf Dynamic Trace uprobe Static Trace BCC ftrace /sys/kernel/debug/tracing/ Kernel Module BPF Bytecode strace Events Tracepoint લఏɿϗεςΟϯάαʔϏεͷఏڙํࣜ wϗεςΟϯάαʔϏεͰ͸γϯάϧߏ੒ͷ୆ͷαʔόΛෳ਺ਓ Ͱڞ༗ͯ͠͝ར༻͍ͨͩ͘ํࣜΛͱ͍ͬͯΔɻ
   w୆ͷαʔόΛΑΓଟ͘ͷਓʹ͝ར༻͍ͨͩ͘͜ͱͰɺརӹ͸େ ͖͘ͳΔɻ ୆ͷαʔό౰ͨΓͷӨڹ౓߹͍ɾίετޮ཰ͷ؍఺Ͱɺো֐ͷݪ Ҽಛఆ΍ଟछଟ༷ͷϓϩάϥϜΛޮ཰Ϋ࣮ߦͯ͠๯಄Ͱड़΂ͨ՝୊ Λղܾ͢Δ͜ͱ͸ඇৗʹॏཁͰ͋Δɻ 
7. Systemtap eBPF kprobe perf Dynamic Trace uprobe Static Trace BCC ftrace /sys/kernel/debug/tracing/ Kernel Module BPF Bytecode strace Events Tracepoint લఏɿϗεςΟϯάαʔϏεαʔόͷಛੑ w͓٬༷͸ݖݶͷൣғͰࣗ༝ʹϓϩάϥϜΛ࣮ߦͰ͖Δɻ
   wϓϩάϥϜͷҰ෦͸͞·͟·ͳෛՙΛੜΉɻ
   w$16
   ϝϞϦ
   σΟεΫ*0
   τϥϑΟοΫ
   ϓϩηεϒϩοΩϯά
   wͳͲʜ
   wݱ࣌఺ͰಛఆͰ͖ͳ͍ෛՙ΋͋Δɻ 
8. Systemtap eBPF kprobe perf Dynamic Trace uprobe Static Trace BCC ftrace /sys/kernel/debug/tracing/ Kernel Module BPF Bytecode strace Events Tracepoint લఏɿϗεςΟϯάαʔϏεαʔόͷػೳ sshd crond UserA UserB … UserA UserB … ϗεςΟϯάαʔϏεͷ8FCαʔό಺ͷओͳఏڙػೳ httpd UserA UserB … ͓٬༷͕ࣗ༝ʹૢ࡞Ͱ͖ΔՕॴ͕ෳ਺͋ΔͨΊ੾Γ෼͚͕ඞཁ 
9. Systemtap eBPF kprobe perf Dynamic Trace uprobe Static Trace BCC ftrace /sys/kernel/debug/tracing/ Kernel Module BPF Bytecode strace Events Tracepoint લఏɿϗεςΟϯάαʔϏεͷো֐ରԠ wUPQ
   ΍
   QT
   ίϚϯυʹΑͬͯɺͲͷϓϩηε͕ݪҼͱͳ͍ͬͯΔ ͔Λಛఆ͢Δɻ
   wTUSBDF
   ΍
   MTPG
   ίϚϯυʹΑͬͯɺγεςϜίʔϧ΍Φʔϓϯ͠ ͍ͯΔϑΝΠϧΛ֬ೝͯ͠ෛՙݪҼΛಛఆ͢Δɻ
   w͜ΕͰಛఆͰ͖Δ৔߹΋͋Δ͕ɺಛఆͰ͖ͳ͍৔߹͸औΓ׶͑ͣ ݪҼͰ͋ΔϓϩηεΛ࠶ىಈ࣏ͯ͠Δ͜ͱ΋ʜ ྫ͑͹ϗεςΟϯάαʔόͷ-"΍$16͕ߴ͍ঢ়ଶ 
10. Systemtap eBPF kprobe perf Dynamic Trace uprobe Static Trace BCC ftrace /sys/kernel/debug/tracing/ Kernel Module BPF Bytecode strace Events Tracepoint લఏɿϗεςΟϯάαʔϏεͷো֐ରࡦ αʔόͷো֐༧๷΍ݪҼಛఆͷͨΊͷ৘ใͱͯ͠ɺ$16ɾϝϞϦɾ ϩά΍αʔϏεϨεϙϯελΠϜͳͲͷ৘ใΛऩू͍ͯ͠Δɻ
    ˠ
    ͔͠͠ɺ͜Ε͚ͩͰ͸ো֐ͷݪҼ͕ಛఆͰ͖ͳ͍͜ͱ͕͠͹͠ ͹ൃੜ͢Δɻ 
11. Systemtap eBPF kprobe perf Dynamic Trace uprobe Static Trace BCC ftrace /sys/kernel/debug/tracing/ Kernel Module BPF Bytecode strace Events Tracepoint લఏɿٻΊͨπʔϧ ͜͜·ͰͰɺαʔόͷো֐ରԠͰγεςϜίʔϧͷτϨʔεͳͲ͕ ༗ޮ׆༻Ͱ͖Δ͜ͱɺͦΕ͚ͩͰ͸଍Γͳ͍͜ͱ͕͕෼͔ͬͨɻ wҎԼͷπʔϧΛٻΊΔɻ
    wγεςϜίʔϧͷΈͳΒͣࣗ༝౓͕ߴ͘LFSOFMϨϕϧͰͷτ ϨʔεΛͰ͖Δɻ
    wτϨʔεʹࡍͯ͠Φʔόʔϔου͕খ͍͞ɻ 
12. Systemtap eBPF kprobe perf Dynamic Trace uprobe Static Trace BCC ftrace /sys/kernel/debug/tracing/ Kernel Module BPF Bytecode strace Events Tracepoint લఏɿ௿Φʔόʔϔουͷඞཁੑ wಈ࡞தͷϓϩμΫγϣϯ؀ڥͰ࣮ࢪ͍ͨͨ͠ΊɺύϑΥʔϚϯε ʹӨڹΛ༩͑ͨ͘ͳ͍ɻ
    w͍ͭো֐͕ൃੜ͢Δ͔෼͔Βͳ͍ɻՄೳͰ͋Ε͹ɺৗʹτϨʔε Λͯ͠৘ใΛऩू͍ͨ͠ɻ
    wܧଓͯ͠ΦʔόʔϔουΛ૿΍ͨ͘͠ͳ͍ɻ 
13. Systemtap eBPF kprobe perf Dynamic Trace uprobe Static Trace BCC ftrace /sys/kernel/debug/tracing/ Kernel Module BPF Bytecode strace Events Tracepoint લఏɿΦʔόʔϔουͷߴ͍τϨʔεख๏ TUSBDF  TUSBDF
    QSPDFTT UBSHFU
    QSPDFTT strace -p {target PID} ptrace(PTRACE_ATTACH, PID…) waitpid SFHJTUFS systemcall SIGTRAP ϓϩηεఀࢭ ղੳ ࠶։ … ϓϩηεऴྃ exit 1. 2. 3. 4. 5. 6. 
14. Systemtap eBPF kprobe perf Dynamic Trace uprobe Static Trace BCC ftrace /sys/kernel/debug/tracing/ Kernel Module BPF Bytecode strace Events Tracepoint લఏɿΦʔόʔϔουͷߴ͍τϨʔεख๏ TUSBDF  TUSBDF
    QSPDFTT UBSHFU
    QSPDFTT strace -p {target PID} ptrace(PTRACE_ATTACH, PID…) waitpid SFHJTUFS systemcall SIGTRAP ϓϩηεఀࢭ ղੳ ࠶։ … ϓϩηεऴྃ exit 1. 2. 3. 4. 5. 6. γεςϜίʔϧͷղੳதʹ
    ϓϩάϥϜ͕ॠؒతʹఀࢭ͢Δ 
15. Systemtap eBPF kprobe perf Dynamic Trace uprobe Static Trace BCC ftrace /sys/kernel/debug/tracing/ Kernel Module BPF Bytecode strace Events Tracepoint લఏɿΦʔόʔϔουͷ௿͍τϨʔεख๏ ref: http://www.brendangregg.com/linuxperf.html 
16. Systemtap eBPF kprobe perf Dynamic Trace uprobe Static Trace BCC ftrace /sys/kernel/debug/tracing/ Kernel Module BPF Bytecode strace Events Tracepoint ໨త 
17. Systemtap eBPF kprobe perf Dynamic Trace uprobe Static Trace BCC ftrace /sys/kernel/debug/tracing/ Kernel Module BPF Bytecode strace Events Tracepoint ໨త wݪҼෆ໌ͷো֐ʹ͍ͭͯɺ6TFSMFWFM
    ,FSOFM
    ͷτϨʔε৘ใΛ ׆༻ͯ͠ղܾ͢ΔՄೳੑʹ͍ͭͯݕূ͢Δɻ
    wͦͷҝʹɺܧଓతʹτϨʔε৘ใΛऩूͯ͠ো֐࣌ʹ৘ใΛ׆༻ ͢Δ͜ͱͷ༗ޮੑΛݕূ͢Δɻ
    wαʔϏεͷΦϖϨʔγϣϯΤϯδχΞશһ͕-JOVYͷτϨʔεʹ ਫ਼௨͍ͯ͠Δ͜ͱ͸كͰ͋ΔɻͦͷͨΊɺଟ͘ͷΦϖϨʔγϣϯ ΤϯδχΞͰ΋-JOVYͷτϨʔε͕Ͱ͖ΔΑ͏ʹମܥԽ͢Δɻ 
18. Systemtap eBPF kprobe perf Dynamic Trace uprobe Static Trace BCC ftrace /sys/kernel/debug/tracing/ Kernel Module BPF Bytecode strace Events Tracepoint ࠓճͷൣғ 
19. Systemtap eBPF kprobe perf Dynamic Trace uprobe Static Trace BCC ftrace /sys/kernel/debug/tracing/ Kernel Module BPF Bytecode strace Events Tracepoint ࠓճͷൣғ wطଘͷ௿ΦʔόʔϔουͳτϨʔεπʔϧ͸ଟ͘ଘࡏ͍ͯ͠Δɻ ͦΕͧΕͷπʔϧ͕Ͳ͏͍͏ػೳ΍໾ׂ͕͋Γɺզʑ͕ԿΛબ୒ ͢Ε͹ྑ͍ͷ͔ΛҎԼͷ؍఺͔Βߟ͑ͨɻ
    wଟ͘ͷτϨʔεख๏ͷ໾ׂΛ੔ཧͯ͠શମ૾Λ௫Ή͜ͱɻ
    wͦΕͧΕͷπʔϧͷ࢓૊Έʹ͍ͭͯಛ௃Λཧղ͢Δ͜ͱɻ 
20. Systemtap eBPF kprobe perf Dynamic Trace uprobe Static Trace BCC ftrace /sys/kernel/debug/tracing/ Kernel Module BPF Bytecode strace Events Tracepoint 
    τϨʔεख๏ͷ੔ཧ 
21. Systemtap eBPF kprobe perf Dynamic Trace uprobe Static Trace BCC ftrace /sys/kernel/debug/tracing/ Kernel Module BPF Bytecode strace Events Tracepoint 
22. Systemtap eBPF kprobe perf Dynamic Trace uprobe Static Trace BCC ftrace /sys/kernel/debug/tracing/ Kernel Module BPF Bytecode strace Events Tracepoint ࠞཚ͠·͢ʂʂ 
23. Systemtap eBPF kprobe perf Dynamic Trace uprobe Static Trace BCC ftrace /sys/kernel/debug/tracing/ Kernel Module BPF Bytecode strace Events Tracepoint ͭͷϨΠϠʹ෼ྨ wσʔλιʔεϨΠϠ
    wϋʔυ΢ΣΞɾιϑτ΢ΣΞͷΠϕϯτΛτϨʔε͢Δػߏ
    wϑϨʔϜϫʔΫϨΠϠ
    wσʔλιʔεϨΠϠʹΞΫηε͢ΔͨΊͷ൚༻తͳϑϨʔϜϫʔΫ
    wΞϓϦέʔγϣϯϨΠϠ
    6TFSMFWFM
    wϑϨʔϜϫʔΫϨΠϠʹର໋ͯ͠ྩΛग़ͨ͠Γɺऔಘͨ݁͠ՌΛू ܭɾදࣔͳͲΛ͢Δ 
24. Systemtap eBPF kprobe perf Dynamic Trace uprobe Static Trace BCC ftrace /sys/kernel/debug/tracing/ Kernel Module BPF Bytecode strace Events Tracepoint Πϕϯτͷ෼ྨ wτϨʔεͱ͸ɺ֤ॴ͔Βൃߦ͞ΕΔΠϕϯτΛτϨʔε͢Δͱ͍ ͏͜ͱɻ Software Event Hardware Event GPU Event Memory Event CPU Event Dynamic Event Static Event 
25. Systemtap eBPF kprobe perf Dynamic Trace uprobe Static Trace BCC ftrace /sys/kernel/debug/tracing/ Kernel Module BPF Bytecode strace Events Tracepoint τϨʔεπʔϧͷϨΠϠʹΑΔ෼ྨ bcc (eBPF scripts) systemtap perf-tool (ftrace scripts) perf eBPF kernel module (Inserted by tool) Ftrace (debugfs) perf_event uprobe Tracepoint kprobe PMU Static Event Dynamic Event CPU Event Software Event Hardware Event Πϕϯτ σʔλιʔεϨΠϠ ϑϨʔϜϫʔΫϨΠϠ ΞϓϦέʔγϣϯϨΠϠ user kernel 
26. Systemtap eBPF kprobe perf Dynamic Trace uprobe Static Trace BCC ftrace /sys/kernel/debug/tracing/ Kernel Module BPF Bytecode strace Events Tracepoint 
    τϨʔεख๏ͷ࢓૊Έ 
27. Systemtap eBPF kprobe perf Dynamic Trace uprobe Static Trace BCC ftrace /sys/kernel/debug/tracing/ Kernel Module BPF Bytecode strace Events Tracepoint σʔλιʔεϨΠϠ w1.6
    1FSGPSNBODF
    .POJUPSJOH
    6OJUT
    w$16ͳͲͷϋʔυ΢ΣΞͷύϑΥʔϚϯεʹؔ͢ΔΠϕϯτ ʹΞΫηε͢ΔͨΊͷΠϯλʔϑΣΠε
    wLQSPCF
    
    VQSPCF
    wಈ࡞தͷιϑτ΢ΣΞʹΧελϜΠϕϯτΛ࡞੒͢Δɻ
    w,FSOFM͕ର৅ͳΒ
    LQSPCF
    6TBFMFWFM͕ର৅ͳΒ
    VQSPCF
    Λ ར༻ͯ͠ɺιϑτ΢ΣΞͷΠϕϯτΛτϨʔε͢Δɻ
    w5SBDFQPJOU
    wLFSOFMʹ૊Έࠐ·Ε͍ͯΔ੩తͳτϨʔεϙΠϯτ 
28. Systemtap eBPF kprobe perf Dynamic Trace uprobe Static Trace BCC ftrace /sys/kernel/debug/tracing/ Kernel Module BPF Bytecode strace Events Tracepoint ϑϨʔϜϫʔΫϨΠϠ w'USBDF
    wEFCVHGT
    ͱ͍͏ಛघͳϑΝΠϧγεςϜΛΠϯλʔϑΣΠε ʹ͍࣋ͬͯΔɻ
    TZTLFSOFMEFCVHUSBDJOH
    ഑ԼʹτϨʔ εͷ༗ޮԽ΍ϑΟϧλϦϯάͷͨΊͷϑΝΠϧɺτϨʔε݁Ռ Λग़ྗ͢ΔͨΊͷϑΝΠϧ͕ଘࡏ͍ͯ͠Δɻ
    wUSBDFQPJOU
    LQSPCF
    VQSPCF
    Λ࢖ͬͯτϨʔεΛߦ͏͜ͱ͕ Ͱ͖Δɻ
    w୯ҰϢʔβ SPPU ͷΈͰಈ࡞͢Δɻ
    wϓϩάϥϚϒϧͰ͸ͳ͍ɻ 
29. Systemtap eBPF kprobe perf Dynamic Trace uprobe Static Trace BCC ftrace /sys/kernel/debug/tracing/ Kernel Module BPF Bytecode strace Events Tracepoint ϑϨʔϜϫʔΫϨΠϠ wQFSG@FWFOUT
    wτϨʔεର৅
    ϋʔυ΢ΣΞΠϕϯτ
    ιϑτ΢ΣΞΠϕϯτ
    τϨʔεϙΠτ
    Λࢦఆͯ͠
    QFSG@FWFOU@PQFO 
    Λ࣮ߦ͠ ͯΠϕϯτΛొ࿥͢Δɻ
    w1.6
    USBDFQPJOU
    Λ࢖ͬͯτϨʔεΛߦ͏͜ͱ͕Ͱ͖Δɻ
    wLQSPCF
    ʹΑΔಈతτϨʔε͸
    -JOVY
    Ҏ͔߱ΒՄೳͰ
    CQG 
    Λར༻͢Δɻ 
30. Systemtap eBPF kprobe perf Dynamic Trace uprobe Static Trace BCC ftrace /sys/kernel/debug/tracing/ Kernel Module BPF Bytecode strace Events Tracepoint ϑϨʔϜϫʔΫϨΠϠ w F#1'
    w 1.6
    USBDFQPJOU
    LQSPCF
    VQSPCF
    Λ࢖ͬͯτϨʔεΛߦ͏͜ͱ͕Ͱ͖Δɻ
    w F#1'͸6TFSMFWFMͰ࡞੒ͨ͠#ZUFDPEFΛ
    CQG 
    Λ࢖ͬͯLFSOFMۭؒʹϩʔ υ͠ɺ࣮ߦ͢Δ͜ͱ͕Ͱ͖ΔɻF#1'ʹΑΔτϨʔεͰ͸ɺલड़ͨ͠σʔλιʔ εʹΠϕϯτΛొ࿥͢Δ͜ͱͰ࣮ݱͰ͖Δɻ
    w F#1'ͷಛ௃ͱͯ͠ɺ఺্͛Δɻ
    w LFSOFMۭؒʹಡΈࠐ·ΕΔ#ZUFDPEF͸F#1'ͷ7FSJpFSʹΑͬͯίʔυͷ ҆શੑ͕νΣοΫ͞Εෆਖ਼ͳϝϞϦɺελοΫ΁ͷΞΫηε΍ແݶϧʔϓ ͳͲΛݕ஌͢Δ͜ͱͰ҆શੑΛ֬อ͍ͯ͠Δɻ
    w LFSOFMۭؒͱ6TFSMFWFMͷؒͰσʔλͷ΍ΓऔΓΛ͢ΔͨΊʹF#1'ͷ.BQ Λ࡞੒͢Δ͜ͱ͕Ͱ͖ΔɻͦΕ͸)BTI
    "SSBZͳͲͷܗࣜΛ΋͍ͬͯΔɻ
    w Φʔόʔϔου͕ඇৗʹগͳ͍ɻ 
31. Systemtap eBPF kprobe perf Dynamic Trace uprobe Static Trace BCC ftrace /sys/kernel/debug/tracing/ Kernel Module BPF Bytecode strace Events Tracepoint bcc (eBPF scripts) systemtap perf-tool (ftrace scripts) perf eBPF kernel module (Inserted by tool) Ftrace (debugfs) perf_event uprobe Tracepoint kprobe PMU Static Event Dynamic Event CPU Event Software Event Hardware Event Πϕϯτ σʔλιʔεϨΠϠ ϑϨʔϜϫʔΫϨΠϠ ΞϓϦέʔγϣϯϨΠϠ user kernel eBPF kprobe Dynamic Event Software Event Πϕϯτ σʔλιʔεϨΠϠ ϑϨʔϜϫʔΫϨΠϠ ΞϓϦέʔγϣϯϨΠϠ user kernel 
32. VTFS LFSOFM ϓϩάϥϜ F#1' LQSPCFT LQSPCFT
    
    F#1' 
33. VTFS LFSOFM ϓϩάϥϜ #1'
    CZUFDPEF F#1' ੜ੒ LQSPCFT LQSPCFT
    
    F#1' 
34. VTFS LFSOFM ϓϩάϥϜ #1'
    CZUFDPEF F#1' WFSJpFS ੜ੒ MPBE LQSPCFT LQSPCFT
    
    F#1' 
35. VTFS LFSOFM ϓϩάϥϜ #1'
    CZUFDPEF F#1' WFSJpFS ੜ੒ MPBE LQSPCFT ϓϩάϥϜͷ҆શੑ ΛνΣοΫ LQSPCFT
    
    F#1' 
36. VTFS LFSOFM ϓϩάϥϜ #1'
    CZUFDPEF F#1' WFSJpFS ੜ੒ MPBE #1'
    CZUFDPEF LQSPCFT LQSPCFT
    
    F#1' 
37. VTFS LFSOFM ϓϩάϥϜ #1'
    CZUFDPEF F#1' WFSJpFS ੜ੒ MPBE #1'
    CZUFDPEF NBQ ੜ੒ 6TFS
    
    ,FSOFM
    ؒͰσʔλͷ΍ΓऔΓΛ͢Δ )BTI
    "SSBZ
    FUDʜ LQSPCFT LQSPCFT
    
    F#1' 
38. LQSPCFT
    
    F#1' VTFS LFSOFM ϓϩάϥϜ #1'
    CZUFDPEF F#1' WFSJpFS ੜ੒ MPBE #1'
    CZUFDPEF NBQ USBDJOH ॻ͖ࠐΈ ࢀর LQSPCFT 
39. Systemtap eBPF kprobe perf Dynamic Trace uprobe Static Trace BCC ftrace /sys/kernel/debug/tracing/ Kernel Module BPF Bytecode strace Events Tracepoint F#1'ͷΦʔόʔϔου͕௿͍ཧ༝ user kernel user kernel ैདྷख๏ eBPF ϓϩάϥϜ τϨʔαʔ ϑϨʔϜϫʔΫ σʔλιʔε ϑΟϧλ ूܭ දࣔ ϓϩάϥϜ τϨʔαˍूܭ ϑϨʔϜϫʔΫ σʔλιʔε දࣔ ࡞੒ ࡞੒ Map ऩू ॻࠐΈ 
40. Systemtap eBPF kprobe perf Dynamic Trace uprobe Static Trace BCC ftrace /sys/kernel/debug/tracing/ Kernel Module BPF Bytecode strace Events Tracepoint F#1'ͷΦʔόʔϔου͕௿͍ཧ༝ user kernel user kernel ैདྷख๏ eBPF ϓϩάϥϜ τϨʔαʔ ϑϨʔϜϫʔΫ σʔλιʔε ϑΟϧλ ूܭ දࣔ ϓϩάϥϜ τϨʔαˍूܭ ϑϨʔϜϫʔΫ σʔλιʔε දࣔ ࡞੒ ࡞੒ Map ऩू ॻࠐΈ F#1'͸ϓϩάϥϚϒϧͰ
    NBQʹΑΔσʔλͷڞ༗͕ՄೳͰ͋Δ
    τϨʔε݁ՌͷϑΟϧλ΍ूܭΛLFSOFM ଆͰ׬݁Ͱ͖Δɻ
    Φʔόʔϔου͕௿͍ʂ 
41. Systemtap eBPF kprobe perf Dynamic Trace uprobe Static Trace BCC ftrace /sys/kernel/debug/tracing/ Kernel Module BPF Bytecode strace Events Tracepoint ͍͞͝ʹ w-JOVYͷτϨʔεπʔϧ͸ɺσʔλιʔεϨΠϠɾϑϨʔϜϫʔ ΫϨΠϠɾΞϓϦέʔγϣϯϨΠϠͷΑ͏ͳ֊૚ߏ଄Ͱ෼͚Δͱ Πϝʔδ͠΍͍͢ɻ
    wπʔϧͷબ୒ͷ؍఺ͱͯ͠͸ɺͲͷϑϨʔϜϫʔΫΛ࢖ͬͯτϨʔ εΛ͢Δ͔ʹΑͬͯΦʔόʔϔου΍τϨʔεର৅͕มΘΔɻ
    wͲͷτϨʔεख๏Ͱ΋༻్ʹΑͬͯ͸ඞཁे෼Ͱ͋Δ͕ɺ৽͍͠ -JOVYΛ࢖͍ͬͯΔͳΒF#1'Λ࠾༻͍ͨ͠ɻ௿Φʔόʔϔουʹ ͩ͜ΘΔඞཁ͕͋Ε͹ಛʹɻ 
42. Systemtap eBPF kprobe perf Dynamic Trace uprobe Static Trace BCC ftrace /sys/kernel/debug/tracing/ Kernel Module BPF Bytecode strace Events Tracepoint ࠓޙͷ՝୊ wࠓճ͸τϨʔεख๏ͷશମ૾΍ͦΕͧΕͷಛੑʹ͍ͭͯΛௐࠪ͠ ͨɻ͔͠͠ɺ࣮ࡍʹΦʔόʔϔουͷଌఆ΍ɺ͸͡Ίʹड़΂ͨ՝ ୊ʹରͯ͠ͷ༗ޮੑͷݕূ͕Ͱ͖͍ͯͳ͍ͨΊࠓޙ΍͍ͬͯ͘ɻ
    w͜ΕΒͷπʔϧΛମܥԽ͢Δʹ͋ͨͬͯ͸ɺ༗ޮੑΛࣔͨ͠ޙʹ ந৅Խ͞ΕͨϢʔεέʔεͱϓϥΫςΟεΛ໢ཏ͢Δ͜ͱͱɺඞ ཁʹԠͯ͡τϨʔεϑϨʔϜϫʔΫΛ࢖ͬͨιϑτ΢ΣΞͷ։ൃ ͕ඞཁʹͳΔɻ 
43. Systemtap eBPF kprobe perf Dynamic Trace uprobe Static Trace BCC ftrace /sys/kernel/debug/tracing/ Kernel Module BPF Bytecode strace Events Tracepoint