.gnu_debuglink or Debugging system libraries with source code

What is .gnu_debuglink?

It is a section in an ELF file containing

  • name of another ELF file and
  • crc32 checksum of that file.

It is used as a pointer to a file with separate debug info and can be created with

objcopy --add-gnu-debuglink=<debug info file>

command (more on creation subject in the next post).

For example, let's find out where libc debug info resides on x86 Ubuntu 9.04:
1. Locate libc itself by using ldd on a random binary:

$ ldd /bin/ls | grep libc
    libc.so.6 => /lib/tls/i686/cmov/libc.so.6 (0xb7ee0000)

2. Dump contents of its .gnu_debuglink section:

$ objdump -s -j .gnu_debuglink /lib/tls/i686/cmov/libc.so.6
/lib/tls/i686/cmov/libc.so.6:     file format elf32-i386
Contents of section .gnu_debuglink:
 0000 6c696263 2d322e39 2e736f00 0d8f898d  libc-2.9.so.....

So name of the file with debug info is libc-2.9.so; the rest must be checksum. We'll leave that for debugger to verify.

Debug info files are usually stored under /usr/lib/debug plus dirname of their "parent" library; in our case it is

    /usr/lib/debug/lib/tls/i686/cmov/libc-2.9.so

How to obtain debug info for a library?

Debug packages do not get installed automatically. Let's install one for libc:

$ apt-get install libc6-debug

Package with debug info is usually (always?) named like <pkgname>-debug, so it is easy to find.

After installation, libc-2.9.so appears in the location we predicted:

$ file /usr/lib/debug/lib/tls/i686/cmov/libc-2.9.so
/usr/lib/debug/lib/tls/i686/cmov/libc-2.9.so: ELF 32-bit LSB shared
object, Intel 80386, version 1 (SYSV), dynamically linked (uses shared
libs), for GNU/Linux 2.6.15, not stripped

It has DWARF debug info as can be seen by doing

$ readelf --debug-dump=info /usr/lib/debug/lib/tls/i686/cmov/libc-2.9.so | less

Where is the source code?

Debug info is not enough - one needs source code to debug. The source code is again distributed separately and can be downloaded with apt-get source command:

$ cd /export/home/maxim/work
$ apt-get source libc6-dev

This will create /export/home/maxim/work/glibc-2.9/ directory and populate it with glibc source code.

How all this works together or How to step into libc function code with dbx?

Brute-force attempt gives us nothig:

(dbx) step
dbx: warning: can't find file
"/build/buildd/glibc-2.9/misc/../sysdeps/unix/sysv/gethostname.c"
stopped in __gethostname at line 35 in file "gethostname.c"

Obviously, dbx was able to read .gnu_debuglink as it knows where the source code for __gethostname() is. It may even be enough if all you need is correct parameter name/values in stack trace:

=>[1] libc.so.6:__gethostname(name = 0xbfd08780 "...", len = 1024U), line 35 in "gethostname.c"

But since we already downloaded the source code, let's point dbx to it:

(dbx) pathmap /build/buildd/glibc-2.9/ /export/home/maxim/work/glibc-2.9/

pathmap command  is described in `help pathmap' topic; here's an excerpt:

Establish a new mapping from <from> to <to> where <from> and <to> are filepath prefixes. <from> refers to the filepath compiled into the executable or object file and <to> refers to the file path at debug time.

The result is immediately visible:

(dbx) list
   35     if (uname (&buf))
   36       return -1;
   37   
   38     node_len = strlen (buf.nodename) + 1;
   39     memcpy (name, buf.nodename, len < node_len ? len : node_len);
   40   
   41     if (node_len > len)
   42       {
   43         __set_errno (ENAMETOOLONG);
   44         return -1;

We have source code of a libc function - gethostname().

What about gdb?

A tale about Linux system wouldn't be complete without a story about gdb. Gdb also supports .gnu_debuglink (obviously), but has a different command for source code path mapping.

Here's how to step into gethostname() using gdb (in the same environment, e.g. debug info and source code are present):

$ gdb a.out
...
(gdb) set substitute-path /build/buildd/glibc-2.9/ /export/home/maxim/work/glibc-2.9/
(gdb) start
...
(gdb) s
__gethostname (name=0xbfb70910 "...", len=1024)
    at ../sysdeps/unix/sysv/gethostname.c:35
35      if (uname (&buf))
(gdb) list
30         size_t len;
31    {
32      struct utsname buf;
33      size_t node_len;
34    
35      if (uname (&buf))
36        return -1;
37    
38      node_len = strlen (buf.nodename) + 1;
39      memcpy (name, buf.nodename, len < node_len ? len : node_len);

NB: you have to set substitute-path before starting debug session; when I tried to set it at the same time as in dbx - after stepping into gethostname() - I got nothing. Besides, with gdb you have to figure out "from" part of substitute-path by yourself somehow as it doesn't give you a hint like dbx; here's what gdb prints without proper substitute-path set:

__gethostname (name=0xbfaf8090 "...", len=1024) at ../sysdeps/unix/sysv/gethostname.c:35
35    ../sysdeps/unix/sysv/gethostname.c: No such file or directory.
    in ../sysdeps/unix/sysv/gethostname.c

References:

  1. In dbx, type help separate debug info
  2. Debugging a Program With dbx, Using a Separate Debug File section
  3. Understanding ELF using readelf and objdump


Комментарии:

Опубликовать комментарий:
  • HTML Syntax: Отключен
About

Articles, news, notes on dbx, the Sun Studio debugger and other stuff.

Search

Categories
Archives
« Апрель 2014
ПнВтСрЧтПтСбВс
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
    
       
Сегодня