DTrace and mdb(1)
By jonh on Jan 09, 2006
I was also unfortunate because Adam Leventhal was in London at the time and came along to lend a hand! My presentation itself (the material) was pretty bad really owing to the lack of time I'd had to put into, what is essentially, a very large topic. However, I hope I wasn't too dull and I hope that I wasn't too innacurate in my description of how little bits of DTrace work.
The vast topic of how DTrace actually works is hugely interesting and I'll do my best to put some of my pitted understanding up here in the not-too-distant future. One thing that I put as an addendum to my presentation was some mdb(1) incantations that allow you to extract a bit of information about what is going on around DTrace itelf.
DTrace has several mdb(1) dcmds of interest that allow us to shine some light on the workings of DTrace. Firstly, let's take a couple of the simple ones starting with the ::dtrace_state dcmd:
# mdb -k Loading modules: [ unix krtld genunix specfs dtrace ufs ip sctp usba uhci s1394 fctl nca lofs audiosup zfs random nfs sppp ptm ipc crypto logindmux ] > ::help dtrace_state NAME dtrace_state - print active DTrace consumersThe ::dtrace_state dcmd actually has a very good description which says it all really. You can use it to display all or individual consumers that are running on the system. Used without any arguments we see:
> ::dtrace_state ADDR MINOR PROC NAME FILE ffffffff8155c800 2 ffffffff856be048 dtrace ffffffff857a84d0 ffffffff8155c000 3 ffffffff856af190 dtrace ffffffff857a8498So, we have two consumers running. The intersting bit is the address given on the ADDR column. This is a pointer to a dtrace_state structure. The dtrace_state structure is of huge interst to us when figuring out what a consumer is doing as it holds kernel state for a consumer.
We can then use the ::dtrace dcmd to extract any data that is being held in kernel for that consumer (data that hasn't yet been read by the consumer). Let's take the second consumers dtrace_state address and feed in into the dtrace dcmd:
> ffffffff8155c000::dtrace fsflush 1 Xorg 2 battstat-applet- 3 gnome-netstatus- 3 nautilus 3 dtrace 4 mixer_applet2 4 soffice.bin 4 mdb 5 powernowd 6 java 34 sched 100It would be very interesting to know what probes were enabled and what the script looked like that generated that output. Well, that's where things get a bit tougher and I'll do that one in another entry.
I'll leave you with a small piece of mdb that shows you what providers are loaded on your system at the moment. Now, you could do this with dtrace(1M) and a simple bit of awk but that would be dull :-) . If I remember, next time I'll try to explain what it does...
> 1::id2probe | ::print -t dtrace_probe_t dtpr_provider | ::list dtrace_provider _t dtpv_next | ::print -t dtrace_provider_t dtpv_name char \*dtpv_name = 0xffffffff83959078 "dtrace" char \*dtpv_name = 0xffffffff8527ed00 "pid689" char \*dtpv_name = 0xffffffff83db07c0 "syscall" char \*dtpv_name = 0xffffffff83db0790 "sdt" char \*dtpv_name = 0xffffffff83db07d0 "mib" char \*dtpv_name = 0xffffffff83db07b0 "io" char \*dtpv_name = 0xffffffff83db0660 "proc" char \*dtpv_name = 0xffffffff83db0690 "sched" char \*dtpv_name = 0xffffffff83db0650 "fpuinfo" char \*dtpv_name = 0xffffffff83db08f8 "vminfo" char \*dtpv_name = 0xffffffff83db0920 "sysinfo" char \*dtpv_name = 0xffffffff83db0928 "vtrace" char \*dtpv_name = 0xffffffff8316cb78 "profile" char \*dtpv_name = 0xffffffff83da8a28 "lockstat" char \*dtpv_name = 0xffffffff83db0900 "fbt" char \*dtpv_name = 0xffffffff831a5658 "fasttrap"