How to Trace the Building Process with Dtrace

Sometimes I modify one source file and them compile whole project. I wanted to know which targets are affected by the change.The answer is not easy in projects with many targets. Simplest way is to use build log/less/grep and trace building process manually.

Last week I have found amazing tool - dtrace. With the tool I can find the targets by a simple script.

Idea is to trace which files are open during building - if a command open the modified file(s), add all files the command opens for writing to the list of modified files and continue. If build commands do not use IPC (inter-process communication), we will have in the list of modified files all probably modified files at the end.

I use dtrace for collect data during build and a perl script for find the modified files - Dtrace can do both steps but I do not want to run building every time I change list of modified files.

There is find_dep.d script:

#!/usr/sbin/dtrace -s

#pragma D option quiet
inline string pre = ">>><<< ";

BEGIN { printf ("%s%d %d exec \\n", pre, timestamp, $target); }

proc:::exec-success
/ pid == $target || progenyof($target) /
{  
   printf ("%s%d %d exec %s\\n", pre, timestamp, pid, curpsinfo->pr_psargs); 
}

syscall::open\*:entry
/ pid == $target || progenyof($target) /
{
  self->oflag = arg1;
  self->open = 1;
}

syscall::open\*:return
/ self->open && arg0 >= 0 /
{
  /\* get filename - black magick from http://www.brendangregg.com/DTrace/rwsnoop \*/
  this->filistp = curthread->t_procp->p_user.u_finfo.fi_list;
  this->ufentryp = (uf_entry_t \*)((uint64_t)this->filistp + (uint64_t)arg0 \* (uint64_t)sizeof(uf_entry_t));
  this->filep = this->ufentryp->uf_file;
  this->vnodep = this->filep != 0 ? this->filep->f_vnode : 0;
  this->vpath = this->vnodep ? (this->vnodep->v_path != 0 ? cleanpath(this->vnodep->v_path) : "") : "";

  printf ("%s%d %d open(%d) %d %s\\n", pre, timestamp, pid, self->oflag, arg0, this->vpath);
}

syscall::open\*:return
/ self->open /
{
  self->oflag = 0;
  self->open = 0;
}

syscall::close\*:entry
/ pid == $target || progenyof($target) /
{
  printf ("%s%d %d close %d\\n", pre, timestamp, pid, arg0);
}

 

One way how to attach the dtrace to the building process is:

chmod +x find_dep.d
./find_dep.d -o dep.log -c 'make all'
(substitute your build cmd instead of make all)

 

There is perl filter script find_dep.pl:

#!/usr/bin/perl

$trace_files{$_} = 1 foreach @ARGV;

sub read_oflag {
    my $oflag = (shift @_) & 0x03;
    
    return $oflag == 0 || $oflag == 2;  # O_RDONLY || O_RDWR
}


sub write_oflag {
    my $oflag = (shift @_) & 0x03;

    return $oflag == 1 || $oflag == 2;  # O_WRONLY || O_RDWR
}


while(<>) {
    next unless s/\^>>><<< (.\*)$/\\1/;

    if (($ts,$pid,$fn) = /(\\d+) (\\d+) exec (.\*)$/) {
        $pids{$pid} = $fn;

    } elsif (($ts,$pid,$oflag,$fd,$fn) = /(\\d+) (\\d+) open\\((\\d+)\\) (\\d+) (.\*)$/) {
        $trace_files{$fn} = 1 if write_oflag($oflag) and $trace_pids{$pid};

        if (read_oflag($oflag) && defined $trace_files{$fn} and not defined $trace_pids{$pid}) {
            $trace_files{$of{$pid}{$_}} = 1 foreach (keys (%{$of{$pid}}));   # add all already open files
            $trace_pids{$pid} = 1;
        }

   } elsif (($ts,$pid,$fd) = /(\\d+) (\\d+) close (\\d+)$/) {
       delete $of{$pid}{$fd};
   } else {
       die "wrong line: $_";
   }
}

print "$_\\n" foreach sort keys (%trace_files);

 

The script arguments are files we want to trace, it reads dtrace log from stdin and returns list of all probably modified files. Because Dtrace sometimes does not order output chronologically, dep.log must be sorted first:

chmod +x find_dep.pl
LANG=C sort dep.log|./find_dep.pl /home/me/project1/src/main.c

 

It is all. I use the script for 3 days. I have found that it will need small improves - scons (an alternative to make) touches near all files, so it is better to skip to trace the process, our buildpackage script touch all already packed files, so it's better to ignore it too, maybe to improve the dtrace script to monitor IPC connections (at least pipes) could be easy, ...

Comments:

Troubles to download www.yahoo.ca showing everytime trying open this window : building trace files, how can I have read of it ?

Posted by Andre J. Garnier on May 25, 2009 at 01:48 PM CEST #

Post a Comment:
  • HTML Syntax: NOT allowed
About

Solaris l10n & i18n, locales, keyboards, fonts and related topics.

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today