DTrace Basics
Dynamic versus!
time_it(Module, Func, Args) ->
S = now(),
Result = Module:Func(Args),
log_elapsed_time(S, now()),
Result.
Slide 6
Slide 6 text
h2p://tmblr.co/ZqJL4s18SkeBK
Slide 7
Slide 7 text
I work at Alert Logic*.!
!
I write a lot of Erlang for work and
sometimes its nice to know why an
application is doing that one thing
that seems to talk a long time.!
* We're hiring.!
Slide 8
Slide 8 text
DTrace Basics
DTrace vs. profiler!
Slide 9
Slide 9 text
DTrace Basics
DTrace vs.and profiler!
Slide 10
Slide 10 text
DTrace Basics
DTrace vs. debugger!
Slide 11
Slide 11 text
DTrace Basics
DTrace vs.and debugger!
Slide 12
Slide 12 text
DTrace Basics
Applica(on
Libraries
System
Calls
Kernel
Slide 13
Slide 13 text
DTrace Basics
It's turtles all the way down...!
h2p://flic.kr/p/aQDPrZ
Slide 14
Slide 14 text
DTrace Basics
DTrace enabled OSes:!
• Solaris 10+!
• OpenSolarisIllumos!
• SmartOS (Joyent)!
• OmniOS (OmniTI)!
• FreeBSD!
• Mac OS X!
Slide 15
Slide 15 text
DTrace Basics
What about Linux?!
Slide 16
Slide 16 text
DTrace Basics
It's Complicated.!
Sorry.!
Slide 17
Slide 17 text
DTrace Basics
DTrace Terms!
Slide 18
Slide 18 text
DTrace Basics
Provider!
!
Manages probes in a
subsystem!
Slide 19
Slide 19 text
DTrace Basics
Provider!
!
In our case, this is the
application language itself. !
!
(More on this soon)!
DTrace Basics
Consumer!
!
A user-mode program!
that calls into DTrace!
Slide 22
Slide 22 text
D Language Basics
D Language Basics!
Slide 23
Slide 23 text
D Language Basics
D Language Overview!
• awk-like!
• Define a probe, optional predicate and optional
actions in a braced clause!
• Supports BEGIN and END blocks!
• Local variables (this->foo = 42)!
• Aggregate/associative variables (prefixed with
@)!
• One liner support in the form!
dtrace -n 'probe /predicate/ {action}'
D Language Basics
Common aggregation functions!
• avg!
• count!
• lquantize (linear)!
• quantize (log - power of 2)!
• sum!
• min!
• max!
Slide 30
Slide 30 text
DTrace and Erlang
Enabling DTrace!
in Erlang!
Slide 31
Slide 31 text
DTrace and Erlang
Build your own with kerl
$ KERL_CONFIGURE_OPTIONS="--with-
dynamic-trace=dtrace" \
kerl build R16B03 \
r16b03-dtrace
Slide 32
Slide 32 text
DTrace and Erlang
Build your own with kerl
$ kerl install \
r16b03-dtrace ~
$ ~`whoami`/activate
Slide 33
Slide 33 text
DTrace and Erlang
Build your own with erlbrew*
$ ERLBREW_CONFIGURE_OPTIONS="--
with-dynamic-trace=dtrace" \
erlbrew install R16B03
*
I
wrote
erlbrew
because
reasons
Slide 34
Slide 34 text
DTrace and Erlang
erltrace: !
DTrace from inside Erlang!
!
https://github.com/project-fifo/erltrace !
Slide 35
Slide 35 text
erltrace
Write DTrace scripts and process DTrace output
from inside Erlang. !
!
Why?!
!
• Push into folsom or !
• Send an alert to a human that "the thing is
happening again"!
Might be too expensive depending on
environment/load.!
Resources
DTrace resources:
• http://dtracehol.com/#Intro
• http://dtrace.org/guide/preface.html
• http://dtracebook.com/index.php/
Languages
• http://www.amazon.com/dp/0132091518
• European Union RELEASE project
whitepaper: http://release-
project.softlab.ntua.gr/documents/
D2.2.pdf
Slide 47
Slide 47 text
Thanks!
Thank you!!
Slide 48
Slide 48 text
DTrace and Erlang
Appendix:!
Erlang DTrace Probes!
Slide 49
Slide 49 text
DTrace and Erlang
message-send (local)!
(SEND_PID, RECV_PID, SIZE,
LABEL, PREV_TOKEN_CNT,
CURRENT_TOKEN_CNT)
PIDs are strings to DTrace, not ints!
Slide 50
Slide 50 text
DTrace and Erlang
message-send_remote!
(SEND_PID, RECV_PID, SIZE,
LABEL, PREV_TOKEN_CNT,
CURRENT_TOKEN_CNT)
!
PIDs are strings to DTrace, not ints!
Slide 51
Slide 51 text
DTrace and Erlang
message-send_remote!
(SEND_PID, NODE_NAME,
RECV_PID, SIZE, LABEL,
PREV_TOKEN_CNT,
CURRENT_TOKEN_CNT)
!
PIDs are strings to DTrace, not ints!
Slide 52
Slide 52 text
DTrace and Erlang
message-queued!
message-receive!
(RECV_PID, SIZE, Q_LEN, TOKEN,
PREV_TOKEN_CNT,
CURRENT_TOKEN_CNT)
!
PIDs are strings to DTrace, not ints!
Slide 53
Slide 53 text
DTrace and Erlang
copy-struct!
(SIZE)
Slide 54
Slide 54 text
DTrace and Erlang
copy-object!
(RECV_PID, SIZE)
Slide 55
Slide 55 text
DTrace and Erlang
local-function_entry!
global-function_entry!
function-return!
(PID, MFA, DEPTH)
Slide 56
Slide 56 text
DTrace and Erlang
bif-entry!
bif-return!
nif-entry!
nif-return!
(PID, MFA)
Slide 57
Slide 57 text
DTrace and Erlang
gc_major-start!
gc_minor-start!
(PID, NEEDED_HEAP_WORDS)
Slide 58
Slide 58 text
DTrace and Erlang
gc_major-end!
gc_minor-end!
(PID,RECLAIMED_SPACE)
Slide 59
Slide 59 text
DTrace and Erlang
process-spawn!
(PID, MFA)
Slide 60
Slide 60 text
DTrace and Erlang
process-exit!
(PID, REASON)
Slide 61
Slide 61 text
DTrace and Erlang
process-exit_signal!
(SEND_PID, RECV_PID, REASON)