Slide 1
Slide 1 text
Learning
Dtrace
2012.11.20
Outsider at
Slide 2
Slide 2 text
DTrace is
a comprehensive
dynamic tracing
framework
created by
Sun Microsystems
for troubleshooting
kernel and
application problems
on production systems
in real time.
http://dtrace.org/blogs/about/dtracepony/
Slide 3
Slide 3 text
DTrace is
a comprehensive
dynamic tracing
framework
created by
Sun Microsystems
for troubleshooting
kernel and
application problems
on production systems
in real time.
http://dtrace.org/blogs/about/dtracepony/
Slide 4
Slide 4 text
DTrace is
a comprehensive
dynamic tracing
framework
created by
Sun Microsystems
for troubleshooting
kernel and
application problems
on production systems
in real time.
http://dtrace.org/blogs/about/dtracepony/
Slide 5
Slide 5 text
DTrace is
a comprehensive
dynamic tracing
framework
created by
Sun Microsystems
for troubleshooting
kernel and
application problems
on production systems
in real time.
http://dtrace.org/blogs/about/dtracepony/
Slide 6
Slide 6 text
DTrace is
a comprehensive
dynamic tracing
framework
created by
Sun Microsystems
for troubleshooting
kernel and
application problems
on production systems
in real time.
http://dtrace.org/blogs/about/dtracepony/
Slide 7
Slide 7 text
http://www.flickr.com/photos/ghost_of_kuji/370072145
Bryan Cantrill
Mike Shapiro
Adam Leventhal
Slide 8
Slide 8 text
Originally
developed
for
2003
Slide 9
Slide 9 text
Open-sourced
in 2005
http://www.flickr.com/photos/tinou/171803338
Slide 10
Slide 10 text
http://www.flickr.com/photos/epu/4299657320
Ported to
Unix-like systems
Slide 11
Slide 11 text
MIT
Technology
Review
2005
“Innovators Under 35”
http://www2.technologyreview.com/tr35/profile.aspx?trid=91
Slide 12
Slide 12 text
Technology
Innovation
Awards 2006
Gold Winner
http://www.dowjones.com/innovation/ei_winners_2006.html
Slide 13
Slide 13 text
STUG award 2008
https://www.usenix.org/about/stug
Slide 14
Slide 14 text
Dive into Dtrace
Slide 15
Slide 15 text
No content
Slide 16
Slide 16 text
٘о प೯غח
زਵ۽ ࢤࢿ
۽؋࣌ীࢲ ࢎਊоמ
Probe
Slide 17
Slide 17 text
Probe List:
sudo dtrace -l
Slide 18
Slide 18 text
Probe ղࠗ ID
Slide 19
Slide 19 text
Probeܳ ઁҕೞח Ҕ
(ex: syscall,
profile)
Slide 20
Slide 20 text
Unix ݽٕա
Probe যܻா࣌
ۄ࠳۞ܻ ܴ
Slide 21
Slide 21 text
Probeо ઓೞח
ೣࣻ ܴ
Slide 22
Slide 22 text
Probe ܴ
Slide 23
Slide 23 text
sudo dtrace -l -n PROBEܴ
Slide 24
Slide 24 text
sudo dtrace -l -f FUNCTION
Slide 25
Slide 25 text
sudo dtrace -l -P PROVIDER
Slide 26
Slide 26 text
sudo dtrace -l -m MODULE
Slide 27
Slide 27 text
DScript
D Programming Language
Slide 28
Slide 28 text
.d ഛ
ஹੌ೧ࢲ ழօۨ߰ীࢲ प೯
উࢿਸ ਤೠ ਬബࢿ Ѩࢎ
۽؋࣌ীࢲ উೞѱ प೯
Features
Slide 29
Slide 29 text
DScript
Kernel
User
DProgram
Output
Dtrace
Slide 30
Slide 30 text
probe-description
/optional predicate/
{
action statements;
}
01
02
03
04
05
Slide 31
Slide 31 text
probe-description
/optional predicate/
{
action statements;
}
01
02
03
04
05
provider:module:function:name
(ex: syscall:::entry)
Slide 32
Slide 32 text
probe-description
/optional predicate/
{
action statements;
}
01
02
03
04
05
दझమ ࢚కܳ ࣻೞח ݺ۸ޙ
Slide 33
Slide 33 text
probe-description
/optional predicate/
{
action statements;
}
01
02
03
04
05
0 ইפѢա trueݶ action ࣻ೯
Slide 34
Slide 34 text
begin-end.d
BEGIN
{
trace("begin the beguine");
exit(0);
}
END
{
trace("that's all...");
}
01
02
03
04
05
06
07
08
09
Slide 35
Slide 35 text
begin-end.d
BEGIN
{
trace("begin the beguine");
exit(0);
}
END
{
trace("that's all...");
}
01
02
03
04
05
06
07
08
09
sudo dtrace -s begin-end.d
Slide 36
Slide 36 text
timer.d
profile:::tick-5sec
{
trace("5sec timer");
}
profile:::tick-800msec
{
trace("800msec timer");
}
01
02
03
04
05
06
07
08
Slide 37
Slide 37 text
beer.d
int bottles;
BEGIN
{ bottles = 5; }
profile:::tick-1sec
/bottles >= 0/
{
printf("%d bottles on the wall\n", bottles);
printf("%d bottles.\n", bottles);
printf("take one down, pass it around\n");
printf("%d bottles on the wall\n\n", bottles);
bottles--;
}
profile:::tick-1sec
/bottles < 0/
{ exit(0); }
END
{ printf("that's all..."); }
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
Slide 38
Slide 38 text
no if-else
no loop
?: operator is exist
No Flow Control
Slide 39
Slide 39 text
process.d
proc:::exec-success
{
printf("%s", execname);
}
01
02
03
04
Slide 40
Slide 40 text
@: aggregation’s prefix
name: aggregation’s name
key: D expression list (comma-separated)
aggfunc: aggregation function
Aggregate
@name[key] = aggfunc(args)
Slide 41
Slide 41 text
count() : ഐപࣻ
sum(expr) : അध ч
avg(expr) : അध ಣӐ
min(expr) : അध о ч
max(expr) : അध о ч
quantize(expr) : 2ઁғ ࠼ب࠙ನ
lquantize(expr,lower-bound, upper-
bound, step-value) : ࢶഋ ࠼ب࠙ನ
Aggregate Function
Slide 42
Slide 42 text
aggr-count.d
syscall::read:entry
{
@counts[execname] = count();
}
profile:::tick-5s
{
exit(0);
}
01
02
03
04
05
06
07
08
Slide 43
Slide 43 text
aggr-quant.d
syscall::read:entry
{
self->ts = timestamp;
}
syscall::read:return
/self->ts/
{
delta = timestamp - self->ts;
@quanttime[execname] = quantize(delta);
}
profile:::tick-5s
{
exit(0);
}
01
02
03
04
05
06
07
08
09
10
11
12
13
14
Slide 44
Slide 44 text
Dtrace
with Node.js
Slide 45
Slide 45 text
http://www.flickr.com/photos/abandonedhero/3404826467
OS Requirements
-Dtrace
-ustack helper
Slide 46
Slide 46 text
USDT
User-Level Statically Defined Tracing
landed in node v0.6.7
Slide 47
Slide 47 text
http://www.flickr.com/photos/jepoirrier/2043728206
Full features
work on OpenSolaris
Slide 48
Slide 48 text
Mac OS
Dtrace
ustack helper
Slide 49
Slide 49 text
Linux
Dtrace
ustack helper
Slide 50
Slide 50 text
Windows
Dtrace
ustack helper
Slide 51
Slide 51 text
http://www.flickr.com/photos/vectorlyme/206472613
node.js is compiled with
--with-dtrace
Slide 52
Slide 52 text
“SunOSীࢲ ഝࢿചغ݅
ܲ दझమীࢶ زೞ ঋח”
Slide 53
Slide 53 text
Dtrace with node.js
on SmartOS of Joyent
Demo
Slide 54
Slide 54 text
server.js
var http = require('http');
http.createServer(function(req, res) {
res.writeHead(200, {
'Content-Type': 'text/plain'
});
res.end('Hello World\n');
}).listen(8124);
console.log('Server running!');
01
02
03
04
05
06
07
08
09
Slide 55
Slide 55 text
http-server.d
BEGIN
{
printf("%7s %2s %5s %20s (%5s) %8s %s (%s)\n",
"WHO", "FD", "RPORT", "REMOTE", "BUFFR",
"METHOD", "URL", "FWDFOR");
}
node*:::http-server-request
{
printf("+SERVER %2d %5d %20s (%5d) %8s %s (%s)\n",
args[1]->fd, args[1]->remotePort, args[1]->remoteAddress,
args[1]->bufferSize,
args[0]->method, args[0]->url, args[0]->forwardedFor);
}
node*:::http-server-response
{
printf("-SERVER %2d %5d %20s (%5d)\n",
args[0]->fd, args[0]->remotePort, args[0]->remoteAddress,
args[0]->bufferSize);
}
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
16
17
18
19
20
21
Slide 56
Slide 56 text
request-count.d
http-server-request
{
@[args[0]->url]=count()
}
01
02
03
04
Slide 57
Slide 57 text
loop.js
new Error().stack;
function main() { func1(); }
function func1() { func2(); }
function func2() {
(function () {
for (;;);
})();
}
main();
01
02
03
04
05
06
07
08
09
10
11
Slide 58
Slide 58 text
profile.d
profile-97
/execname == "node" && arg1/
{
@[jstack(150, 8000)] = count();
}
tick-10sec
{
exit(0);
}
01
02
03
04
05
06
07
08
09
Slide 59
Slide 59 text
FlameGraph
stack trace visualizer
Slide 60
Slide 60 text
http://dtrace.org/blogs/brendan/files/2011/12/mysql-flame.png
Slide 61
Slide 61 text
node-stackvis
$ npm install -g stackvis
Slide 62
Slide 62 text
$ sudo dtrace -s DSCRIPT > INPUT
$ stackvis dtrace flamegraph-svg < INPUT > OUTPUT
Make FrameGraph
Slide 63
Slide 63 text
Alternative
on non-OpenSolarises
Slide 64
Slide 64 text
node-dtrace-provider
https://github.com/chrisa/node-dtrace-provider
$ npm install dtrace-provider
Slide 65
Slide 65 text
var d = require('dtrace-provider');
var dtp = d.createDTraceProvider('nodeapp');
var p1 = dtp.addProbe('probe1', 'int', 'int');
var p2 = dtp.addProbe('probe2', 'char *');
dtp.enable();
01
02
03
04
05
Slide 66
Slide 66 text
dtp.fire("probe1", function(p) {
return [1, 2];
});
dtp.fire("probe2", function(p) {
return ["hello, dtrace via provider", "foo"];
});
01
02
03
04
05
06
Slide 67
Slide 67 text
data type : int, char
maximum argument : 32
Limitations
Slide 68
Slide 68 text
interval.js
function interval(msg) {
console.log(msg);
}
setInterval(function() {
interval('Hello Dtrace');
}, 1000);
01
02
03
04
05
06
07
Slide 69
Slide 69 text
interval.js
interval.js
var d = require('dtrace-provider');
var dtp =
d.createDTraceProvider('nodeapp');
var p2 = dtp.addProbe('echo', 'char *');
function interval(msg) {
dtp.fire('echo', function () {
return [msg];
});
console.log(msg);
}
setInterval(function() {
interval('Hello Dtrace');
}, 1000);
dtp.enable();
01
02
03
04
05
06
07
08
09
10
11
12
13
14
15
Slide 70
Slide 70 text
interval.d
nodeapp*:::echo
{
trace(copyinstr(arg0));
}
01
02
03
04