D Script Archeology

I've read that the best way to learn DTrace is to study existing D scripts. However, upon doing so I quickly find myself asking "where is that documented?" And it turns out, with tens of thousands of probes, as is often the case, it's not. So I'd love to hear what others with more DTrace experience do, but these are the steps I seem to have stumbled upon.

As an example, let's take the script /opt/DTT/Proc/rwbytype.d, which shows the bytes read and written by process by vnode type. Here's an example of the output:

bleonard@opensolaris:/opt/DTT/Proc$ rwbytype.d 
Tracing... Hit Ctrl-C to end.
\^C
PID    CMD               VTYPE  DIR     BYTES
21124  gnome-terminal      chr    W         1
10523  gnome-power-mana   sock    W         4
10565  gnome-netstatus-   fifo    R         7
10547  gnome-netstatus-   fifo    W         8
4817   conky              fifo    R         9
10634  firefox-bin        fifo    R         9
10634  firefox-bin        fifo    W         9
15379  whoami             fifo    W         9
10547  gnome-netstatus-   fifo    R        20
10565  gnome-netstatus-   fifo    W        20
15377  dtrace              chr    W        31
10412  Xorg                chr    R        32
10469  gnome-settings-d   sock    R        32
10523  gnome-power-mana   sock    R        32
21124  gnome-terminal      chr    R        33
10472  metacity           sock    W        56
10472  metacity           sock    R       200
21124  gnome-terminal     sock    R       220
15379  conky               reg    R       529
15379  ksh93               reg    R       529
4817   conky              sock    R      2720
10412  Xorg               sock    W      3204
4817   conky               reg    R      6696
21124  gnome-terminal     sock    W      6852
4817   conky              sock    W     37952
10412  Xorg               sock    R     44864

And here's the D script that produces that output:

typedef struct vtype2str {
	string code;
};

translator struct vtype2str < int T > {
	/\* the order has been picked for performance reasons \*/
	code =
	    T == 1 ? "reg"   :
	    T == 9 ? "sock"  :
	    T == 4 ? "chr"   :
	    T == 6 ? "fifo"  :
	    T == 8 ? "proc"  :
	    T == 2 ? "dir"   :
	    T == 3 ? "blk"   :
	    T == 5 ? "lnk"   :
	    T == 7 ? "door"  :
	    T == 10 ? "port" :
	    T == 11 ? "bad"  : "non";
};

dtrace:::BEGIN
{
	printf("Tracing... Hit Ctrl-C to end.\\n");
}

fbt::fop_read:entry,
fbt::fop_write:entry
{
	self->type = xlate <struct vtype2str \*>(args[0]->v_type)->code;
	self->size = args[1]->uio_resid;
	self->uiop = args[1];
}

fbt::fop_read:return
/self->uiop/
{
	this->resid = self->uiop->uio_resid;
	@bytes[pid, execname, self->type, "R"] = sum(self->size - this->resid);
	self->type = 0;
	self->size = 0;
	self->uiop = 0;
}

/\* this is delibrately redundant code for performance reasons \*/
fbt::fop_write:return
/self->uiop/
{
	this->resid = self->uiop->uio_resid;
	@bytes[pid, execname, self->type, "W"] = sum(self->size - this->resid);
	self->type = 0;
	self->size = 0;
	self->uiop = 0;
}

dtrace:::END
{
	printf("%-6s %-16s %6s %4s %9s\\n",
	    "PID", "CMD", "VTYPE", "DIR", "BYTES");
	printa("%-6d %-16s %6s %4s %@9d\\n", @bytes);
}

First of all, this D script is only enabling 6 probes:

dtrace:::BEGIN
dtrace:::END
dtrace::fop_read:entry
dtrace::fob_read:return
dtrace::fob_write:entry
dtrace::fob_write:return

The BEGIN and END clauses are self-explanatory. Where I get tripped up is looking at the 2nd clause:

fbt::fop_read:entry,
fbt::fop_write:entry
{
	self->type = xlate <struct vtype2str \*>(args[0]->v_type)->code;
	self->size = args[1]->uio_resid;
	self->uiop = args[1];
}

First, I see that we're setting 3 thread-local variables, type, size and uiop. All of these variables are set using probe arguments, and this is where things start to get confusing - where are these arguments defined? I can run the following dtrace command to get verbose information on a probe:

bleonard@opensolaris:/opt/DTT/Proc$ dtrace -lv -n fbt::fop_read:entry
   ID   PROVIDER            MODULE                          FUNCTION NAME
17725        fbt           genunix                          fop_read entry

	Probe Description Attributes
		Identifier Names: Private
		Data Semantics:   Private
		Dependency Class: Unknown

	Argument Attributes
		Identifier Names: Private
		Data Semantics:   Private
		Dependency Class: ISA

	Argument Types
		args[0]: vnode_t \*
		args[1]: uio_t \*
		args[2]: int
		args[3]: cred_t \*
		args[4]: caller_context_t \*

So I learn that the data semantics of the argument attributes are private, which means this probe is accessing private implementation details of the kernel, but that's the case with most DTrace kernel probes. Of more interest is the argument list:

args[0]: vnode_t \*
args[1]: uio_t \*
args[2]: int
args[3]: cred_t \*
args[4]: caller_context_t 

So I can see there are 5 total arguments available for this probe and I know my clause is accessing args[0] and args[1]. But what now? What exactly is vnode_t and uio_t? As far as I know, there's no further dtrace command to drill down into these data types. At this point I turn to the OpenSolaris Source Browser.

I begin by searching on the probe function fop_read, which returns 2 results, the vnode.h header file and the vnode.c source file:

Searched refs:fop_read (Results 1 -2 of 2) sorted by null
/onnv/onnv-gate/usr/src/uts/common/sys/
vnode.h     [all...]
/onnv/onnv-gate/usr/src/uts/common/fs/
vnode.c 3177 fop_read( function
    [all...]

The fop_read function is defined on line 3177 of vnode.c and I can click on that link to be taken directly to the source. Sure enough, the arguments to the fop_read function in vnode.c match what I was told by dtrace:

   3176 int
   3177 fop_read(
   3178 	vnode_t \*vp,
   3179 	uio_t \*uiop,
   3180 	int ioflag,
   3181 	cred_t \*cr,
   3182 	caller_context_t \*ct)

Now I can click on vnode_t to see its definition and this returns 3 results - the logical one being the vnode.h header file returned by our first search:

/onnv/onnv-gate/usr/src/uts/common/sys/
vnode.h 257 } vnode_t; typedef in typeref:struct:vnode
800 int (\*vop_open)(vnode_t \*\*, int, cred_t \*, \\
802 int (\*vop_close)(vnode_t \*, int, int, offset_t, cred_t \*, \\
804 int (\*vop_read)(vnode_t \*, uio_t \*, int, cred_t \*, \\
806 int (\*vop_write)(vnode_t \*, uio_t \*, int, cred_t \*, \\
808 int (\*vop_ioctl)(vnode_t \*, int, intptr_t, int, cred_t \*, \\
810 int (\*vop_setfl)(vnode_t \*, int, int, cred_t \*, \\
812 int (\*vop_getattr)(vnode_t \*, vattr_t \*, int, cred_t \*, \\
814 int (\*vop_setattr)(vnode_t \*, vattr_t \*, int, cred_t \*, \\
816 int (\*vop_access)(vnode_t \*, int, int, cred_t \*,
    [all...]

Here we see that vnode_t is defined on line 257, which you can click to navigate directly to that section of the source. Our D script above is referencing args[0]->v_type, and we can see v_type is an enum:

    225 typedef struct vnode {
    226 	kmutex_t	v_lock;		/\* protects vnode fields \*/
    227 	uint_t		v_flag;		/\* vnode flags (see below) \*/
    228 	uint_t		v_count;	/\* reference count \*/
    229 	void		\*v_data;	/\* private data for fs \*/
    230 	struct vfs	\*v_vfsp;	/\* ptr to containing VFS \*/
    231 	struct stdata	\*v_stream;	/\* associated stream \*/
    232 	enum vtype	v_type;		/\* vnode type \*/
    233 	dev_t		v_rdev;		/\* device (VCHR, VBLK) \*/

Which you can also click on to see the type definition:

    151 /\*
    152  \* vnode types.  VNON means no type.  These values are unrelated to
    153  \* values in on-disk inodes.
    154  \*/
    155 typedef enum vtype {
    156 	VNON	= 0,
    157 	VREG	= 1,
    158 	VDIR	= 2,
    159 	VBLK	= 3,
    160 	VCHR	= 4,
    161 	VLNK	= 5,
    162 	VFIFO	= 6,
    163 	VDOOR	= 7,
    164 	VPROC	= 8,
    165 	VSOCK	= 9,
    166 	VPORT	= 10,
    167 	VBAD	= 11
    168 } vtype_t;

Which matches up perfectly to the struct at the beginning of our D script. So, now I've got a better handle of exactly what's going on with this D script.

Now, as a test, see if you can follow the same procedure to find the documented comment in the OpenSolaris source code for arg[1]->uio_resid :-). 

Comments:

Nice! I linked to this article from blog o' matty! Thanks Brian! Info like this is awesome for folks trying to deep dive into DTrace.

Mike

Posted by mike on October 31, 2009 at 06:56 AM GMT #

Post a Comment:
  • HTML Syntax: NOT allowed
About

The Observatory is a blog for users of Oracle Solaris. Tune in here for tips, tricks and more as we explore the Solaris operating system from Oracle.

Connect with Oracle Solaris:


Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today