DTrace Your Application!
(Not Your Operating System)!
!
!
Mark Allen!
[email protected]!
@bytemeorg!
http://byte-me.org!
https://github.com/mrallen1!
https://speakerdeck.com/mrallen1!
Slide 2
Slide 2 text
DTrace Your Application!
(Not AND Your Operating System)!
!
!
Mark Allen!
[email protected]!
@bytemeorg!
http://byte-me.org!
https://github.com/mrallen1!
https://speakerdeck.com/mrallen1!
DTrace Basics
Dynamic versus!
print "got here!"
time_it(Module, Func, Args) ->
S = now(),
Result = Module:Func(Args),
log_elapsed_time(S, now()),
Result.
Slide 7
Slide 7 text
I work at Alert Logic*.!
!
I write a lot of Erlang for work, but
I have open source projects in
Golang, Clojure, bash, Python
and Perl.!
* We're hiring.!
Slide 8
Slide 8 text
DTrace Basics
DTrace vs. profiler!
Slide 9
Slide 9 text
DTrace Basics
DTrace vs.and profiler!
Slide 10
Slide 10 text
DTrace Basics
DTrace vs. debugger!
Slide 11
Slide 11 text
DTrace Basics
DTrace vs.and debugger!
Slide 12
Slide 12 text
DTrace Basics
Applica(on
Libraries
System
Calls
Kernel
Slide 13
Slide 13 text
DTrace Basics
It's turtles all the way down...!
h?p://flic.kr/p/aQDPrZ
Slide 14
Slide 14 text
DTrace Basics
DTrace enabled OSes:!
• Solaris 10+!
• OpenSolarisIllumos!
• SmartOS (Joyent)!
• OmniOS (OmniTI)!
• FreeBSD!
• Mac OS X!
Slide 15
Slide 15 text
DTrace Basics
What about Linux?!
Slide 16
Slide 16 text
DTrace Basics
DTrace Terms!
Slide 17
Slide 17 text
DTrace Basics
Provider!
!
Manages probes in a
subsystem!
Slide 18
Slide 18 text
DTrace Basics
Provider!
!
In our case, this is the
application language itself. !
!
(More on this soon)!
DTrace Basics
Consumer!
!
A user-mode program!
that calls into DTrace!
Slide 21
Slide 21 text
D Language Basics
D Language Basics!
Slide 22
Slide 22 text
D Language Basics
D Language Overview!
• awk-like!
• Define a probe, optional predicate and optional
actions in a braced clause!
• Supports BEGIN and END blocks!
• Local variables (this->foo = 42)!
• Aggregate/associative variables (prefixed with
@)!
• One liner support in the form!
dtrace -n 'probe /predicate/ {action}'
Slide 23
Slide 23 text
D Language Basics
Python Example!
Slide 24
Slide 24 text
D Language Basics
#!/usr/sbin/dtrace -qFZs
function-entry,
function-return
{
/* arg1 is function name */
trace(copyinstr(arg1))
}
Slide 25
Slide 25 text
D Language Basics
#!/usr/sbin/dtrace -qFZs
function-entry,
function-return
{
/* arg1 is subroutine name */
trace(copyinstr(arg1))
}
Slide 26
Slide 26 text
D Language Basics
#!/usr/sbin/dtrace -qFZs
function-entry,
function-return
{
/* arg1 is subroutine name */
trace(copyinstr(arg1))
}
Slide 27
Slide 27 text
D Language Basics
Perl Example!
Slide 28
Slide 28 text
D Language Basics
#!/usr/sbin/dtrace -qZs
sub-entry
{
/* Count sub entries by package and sub name
* arg3 = package name (Foo::Bar)
* arg0 = subroutine name (do_something) */
@[strjoin(strjoin(copyinstr(arg3),"::"),copyinstr(arg0))] = count()
}
END
{
/* Give me top 10 highest counts; throw away rest */
trunc(@, 10)
}
Slide 29
Slide 29 text
D Language Basics
#!/usr/sbin/dtrace -qZs
sub-entry
{
/* Count sub entries by package and sub name
* arg3 = package name (Foo::Bar)
* arg0 = subroutine name (do_something) */
@[strjoin(strjoin(copyinstr(arg3),"::"),copyinstr(arg0))] = count()
}
END
{
/* Give me top 10 highest counts; throw away rest */
trunc(@, 10)
}
Slide 30
Slide 30 text
D Language Basics
#!/usr/sbin/dtrace -qZs
sub-entry
{
/* Count sub entries by package and sub name
* arg3 = package name (Foo::Bar)
* arg0 = subroutine name (do_something) */
@[strjoin(strjoin(copyinstr(arg3),"::"),copyinstr(arg0))] = count()
}
END
{
/* Give me top 10 highest counts; throw away rest */
trunc(@, 10)
}
Slide 31
Slide 31 text
D Language Basics
#!/usr/sbin/dtrace -qZs
sub-entry
{
/* Count sub entries by package and sub name
* arg3 = package name (Foo::Bar)
* arg0 = subroutine name (do_something) */
@[strjoin(strjoin(copyinstr(arg3),"::"),copyinstr(arg0))] = count()
}
END
{
/* Give me top 10 highest counts; throw away rest */
trunc(@, 10)
}
Slide 32
Slide 32 text
D Language Basics
subcount.d
for
Pod::Perldoc
3.20
D Language Basics
Common aggregation functions!
• avg!
• count!
• lquantize (linear)!
• quantize (log - power of 2)!
• sum!
• min!
• max!
Slide 38
Slide 38 text
DTrace and Perl
DTrace and Perl!
Slide 39
Slide 39 text
DTrace and Perl
Mac OS X vendor perl and
OmniOS vendor perl !
ship with DTrace.!
Slide 40
Slide 40 text
DTrace and Perl
Build your own with perlbrew
perlbrew install \
perl-5.18.0 \
--as 5.18-dtrace \
-Dusedtrace
Slide 41
Slide 41 text
DTrace and Perl
DTrace support by Perl release!
Perl release! DTrace probes!
5.10.1! sub-entry, sub-return!
5.14.x! sub-entry and return get
package name as an argument!
5.16.x! phase-change!
5.18.0! op-entry, loading-file, loaded-
file!
Slide 42
Slide 42 text
DTrace and Perl
sub-entry!
sub-return!
(SUBNAME, FILE, LINE, PACKAGE)
Slide 43
Slide 43 text
DTrace and Perl
phase-change!
(NEWPHASE, OLDPHASE)
Slide 44
Slide 44 text
DTrace and Perl
op-entry!
(OPNAME)
Fires before op executed. If
the debugger is active, fires
after debug hooks but still
before the op is executed.!
Slide 45
Slide 45 text
DTrace and Perl
loading-file (fires before load)!
loaded-file (fires after load)!
!
(FILENAME)
The filename is a path!
(e.g., Foo/Bar not Foo::Bar)!
Slide 46
Slide 46 text
DTrace and Perl
Make your own DTrace probes
in Perl by using !
!
Devel::DTrace::Provider!
!
Slide 47
Slide 47 text
DTrace and Python
DTrace and Python!
Slide 48
Slide 48 text
DTrace and Python
Build your own with brew
brew install python \
--with-dtrace
Slide 49
Slide 49 text
DTrace and Python
Build your own with pythonz*
pythonz install \
--with-dtrace 2.7.5
(* apply https://github.com/saghul/pythonz/pull/43)
Slide 50
Slide 50 text
DTrace and Python
Slide 51
Slide 51 text
DTrace and Python
function-entry!
function-return!
line!
(FILE, FUNCTION, LINE)
Slide 52
Slide 52 text
DTrace and Python
gc-start!
(GENERATION)
Similar to gc.collect()!
Slide 53
Slide 53 text
DTrace and Python
gc-done!
(OBJECTS_COLLECTED)
Slide 54
Slide 54 text
DTrace and Python
instance-new-start!
instance-new-done!
instance-delete-start!
instance-delete-done!
(CLASS_NAME, FILE)
Slide 55
Slide 55 text
DTrace and Python
Make your own DTrace probes
in Python by using !
!
http://tmetsch.github.io/python-dtrace/!
!
Available on PyPI too!
Slide 56
Slide 56 text
DTrace and Erlang
DTrace and Erlang!
Slide 57
Slide 57 text
DTrace and Erlang
Build your own with kerl
$ KERL_CONFIGURE_OPTIONS="--with-
dynamic-trace=dtrace" \
kerl build R16B01 \
r16b01-dtrace
Slide 58
Slide 58 text
DTrace and Erlang
Build your own with kerl
$ kerl install \
r16b01-dtrace
$ ~`whoami`/activate
Slide 59
Slide 59 text
DTrace and Erlang
message-send (local)!
(SEND_PID, RECV_PID, SIZE,
LABEL, PREV_TOKEN_CNT,
CURRENT_TOKEN_CNT)
PIDs are strings to DTrace, not ints!
Slide 60
Slide 60 text
DTrace and Erlang
message-send_remote!
(SEND_PID, RECV_PID, SIZE,
LABEL, PREV_TOKEN_CNT,
CURRENT_TOKEN_CNT)
!
PIDs are strings to DTrace, not ints!
Slide 61
Slide 61 text
DTrace and Erlang
message-send_remote!
(SEND_PID, NODE_NAME,
RECV_PID, SIZE, LABEL,
PREV_TOKEN_CNT,
CURRENT_TOKEN_CNT)
!
PIDs are strings to DTrace, not ints!
Slide 62
Slide 62 text
DTrace and Erlang
message-queued!
message-receive!
(RECV_PID, SIZE, Q_LEN, TOKEN,
PREV_TOKEN_CNT,
CURRENT_TOKEN_CNT)
!
PIDs are strings to DTrace, not ints!
Slide 63
Slide 63 text
DTrace and Erlang
copy-struct!
(SIZE)
Slide 64
Slide 64 text
DTrace and Erlang
copy-object!
(RECV_PID, SIZE)
Slide 65
Slide 65 text
DTrace and Erlang
copy-object!
(RECV_PID, SIZE)
Slide 66
Slide 66 text
DTrace and Erlang
local-function_entry!
global-function_entry!
function-return!
(PID, MFA, DEPTH)
Slide 67
Slide 67 text
DTrace and Erlang
bif-entry!
bif-return!
nif-entry!
nif-return!
(PID, MFA)
Slide 68
Slide 68 text
DTrace and Erlang
gc_major-start!
gc_minor-start!
(PID, NEEDED_HEAP_WORDS)
Slide 69
Slide 69 text
DTrace and Erlang
gc_major-end!
gc_minor-end!
(PID,RECLAIMED_SPACE)
Slide 70
Slide 70 text
DTrace and Erlang
process-spawn!
(PID, MFA)
Slide 71
Slide 71 text
DTrace and Erlang
process-exit!
(PID, REASON)
Slide 72
Slide 72 text
DTrace and Erlang
process-exit!
(PID, REASON)
Slide 73
Slide 73 text
DTrace and Erlang
process-exit_signal!
(SEND_PID, RECV_PID, REASON)