Solaris ELF files have a new ELF symbol table. The section type is SHT_SUNW_LDYNSYM, and the section is named .SUNW_ldynsym. In the 20+ years in which the ELF standard has been in use, we have only needed two symbol tables (.symtab, and .dynsym) to support linking, so the addition of a third symbol table is a notable event for ELF cognoscenti. Even if you aren’t one of those, you may encounter these sections, and wonder what they are for. I hope to explain that here.
Solaris has many tools that examine running processes or core files and generate stack traces. For example, consider the following call to pstack(1), made on an Xterm process currently running on my system:
% pstack 3094 3094: xterm -ls -geometry 80x51+0+175 fef4bea7 pollsys (8046600, 2, 0, 0) fef0767e pselect (5, 8400168, 84001e8, fef95260, 0, 0) + 19e fef0798e select (5, 8400168, 84001e8, 0, 0) + 7e 0805b250 in_put (10, 8416720, 0, fedd561e, 8416720, 0) + 1b0 08059b20 VTparse (84166a8, 8057acc, fed387c5, 8416720, 84166a8, 804688c) + 90 0805d1f1 VTRun (8046a28, 8046870, feffa7c0, 8046808, 8046858, 804685c) + 205 08057add main (0, 80468b4, 80468c8) + 945 08056eee _start (4, 8046a90, 0, 8046a9a, 8046aa4, 0) + 7a
In order to show you those function names, pstack (really the libproc library used by pstack) needs to map the addresses of functions on the stack to the ELF symbols that correspond to them. Usually, these symbols come from the symbol table (.symtab). If this symbol table has been removed with the strip(1) program, then the dynamic symbol table (.dynsym) will be used instead. As described in a previous blog entry, the .dynsym contains the subset of global symbols from .symtab that are needed by the runtime linker ld.so.1(1). This fallback allows us to map global functions to their names, but local function symbols are not available. Observability tools like pstack(1) will display the hexidecimal address of such local functions when a name is not available. This is better than nothing, but is not particularly helpful.
It used to be common practice for system binaries to be stripped in order to save space. However, observability is a central tenet of the Solaris philosophy. Solaris objects and executables are therefore shipped in unstripped form, and have been for many years, in order to support such symbol lookups. For the most part, this has been a winning strategy, but there are still issues that come up from time to time:
- strip(1) removes much more than the symbol table. Usually the size of this extra data is not a significant concern, but there are certain very large programs where the space savings might be worthwhile. It would be great to strip those particular things, but losing the local function symbols and the ability to make accurate stack traces is a bitter pill to swallow. This has led to a number of proposed features to “strip everything except local function symbols”. These ideas are reasonable, but complicated. We like the fact that “strip” is a simple straightforward operation, and want to avoid complicating the concept.
- We don’t strip our files, but many Solaris users do. This becomes a problem when those applications misbehave, and they (or we, if you have a high end support contract) are trying to figure out why. Often, it is not possible to rebuild such applications in order to debug them. The ability to observe unmodified applications running in a production environment is another key Solaris virtue, as exemplified byDTrace.
Over the years, we have observed that these problems would be largely solved if we could add local function symbols to the .dynsym, and that in most programs, the additional space used would be minimal. Last fall, I embarked on a project to do this.
I tried hard to avoid adding a new symbol table type, and instead tried several experiments in which the additional local function symbols were placed in the dynsym. The reason for wanting this was to avoid having to modify ELF utilities and debuggers to know about a new symbol table. If the added symbols are in the existing .dynsym, those tools will automatically see them, without needing modification. As detailed in the ARC case that I filed for this work (PSARC/2006/526), I tried many different permutations. In every case, I discovered undesirable backward compatibility issues that kept me from using that solution. It turns out that the layout of .dynsym, and the other ELF sections that interact with it, are completely constrained, and there is no 100% backward compatible way to add local symbols to it.
ELF was designed from the very beginning to make it possible to introduce new section types with full backward/forward compatibility. You can always safely add a new section, with a moderate amount of care, and it will work. More than anything, this ability to extend ELF accounts for its long life. Given that the .dynsym cannot be extended with local symbols, I made the obvious (in hindsight) decision to to introduce a new section type (SHT_SUNW_LDYNSYM), and add a new symbol table section named .SUNW_ldynsym to every Solaris file that has a .dynsym section. Once that decision was made, the implementation was straightforward, giving me confidence that it was the right way to go.
The .SUNW_ldynsym section can be thought of as the local initial part of the .dynsym that we wish to build, but can’t. The Solaris linker ( ld(1)) takes care to actually place them side by side, so that the end of the .SUNW_ldynsym section leads directly into the start of the .dynsym section. The runtime linker ( ld.so.1(1)) takes advantage of this to treat them as a single table within the implementation of dladdr(3C). Note that this trick works for applications that mmap(2) the file and access it directly. If you are accessing an ELF file via libelf, as many utilities do, you can’t make any assumptions about the relative positions of different sections.
As with .dynsym, .SUNW_ldynsym sections are allocable, meaning that they are part of the process text segment. This means that they are available at runtime for dladdr(3C). It also means that they cannot be stripped. Although you cannot strip .SUNW_ldynsym sections, you can prevent them from being generated by ld(1), by using the -znoldynsym linker option.
.SUNW_ldynsym sections consume a small amount of additional space. We found that for all of core Solaris (OS and Networking), the increase in size was on the order of 1.4%. This small increase pays off by letting our observability tools do a better job. Furthermore, the presence of .SUNW_ldynsym means that in many cases, you can strip programs that you might not have been willing to strip before.
Example
Let’s use the following program to see how .SUNW_ldynsym sections improve Solaris observability of local functions:
/*
* Program to demonstrate SHT_SUNW_LDYNSYM sections. The
* global main program calls a local function named
* static_func(). static_func() uses printstack() to exercise
* the dladdr(3C) function provided by the runtime linker,
* and then deliberately causes a segfault. The resulting core
* file can be examined by pstack(1) or mdb(1).
*
* In all these cases, if a stripped binary of this program
* contains a .SUNW_ldynsym section, the static_func() function
* will be observable by name, and otherwise simply as an
* address.
*/
#include <ucontext.h>
static void
static_func(void)
{
/* Use dladdr(3C) to print a call stack */
printstack(1);
/*
* Write to address 0, killing the process and
* producing a core file.
*/
*((char *) 0) = 1;
}
int main(int argc, char *argv[])
{
static_func();
return (0);
}
Let’s build two versions of this program, one containing the .SUNW_ldynsym section, and one without:
% cc -Wl,-znoldynsym test.c -o test_noldynsym % cc test.c -o test_ldynsym
The elfdump(1) command can be used to let us examine the three symbol tables contained in test_ldynsym. There is no need to examine this (large) output too carefully, but there are some interesting facts worth noticing:
- Every symbol in .SUNW_ldynsym or .dynsym is also found in .symtab, because .symtab is a superset of the other two tables. This is why it is always preferred to the other two, when available.
- .symtab is much larger than the other two tables combined, which leads to the temptation to strip it, along with the other things strip(1) removes.
- The symbols in .dynsym are strictly limited to those needed by the runtime linker.
- If you consider the .SUNW_ldynsym and .dynsym symbol tables as a single logical entity, you can see that the result follows the rules for ELF symbol table layout.
% elfdump -s test_ldynsym
Symbol Table Section: .SUNW_ldynsym
index value size type bind oth ver shndx name
[0] 0x00000000 0x00000000 NOTY LOCL D 0 UNDEF
[1] 0x00000000 0x00000000 FILE LOCL D 0 ABS test_ldynsym
[2] 0x00000000 0x00000000 FILE LOCL D 0 ABS crti.s
[3] 0x00000000 0x00000000 FILE LOCL D 0 ABS crt1.o
[4] 0x00000000 0x00000000 FILE LOCL D 0 ABS crt1.s
[5] 0x00000000 0x00000000 FILE LOCL D 0 ABS fsr.s
[6] 0x00000000 0x00000000 FILE LOCL D 0 ABS values-Xa.c
[7] 0x00000000 0x00000000 FILE LOCL D 0 ABS test.c
[8] 0x080507f0 0x00000019 FUNC LOCL D 0 .text static_func
[9] 0x00000000 0x00000000 FILE LOCL D 0 ABS crtn.s
Symbol Table Section: .dynsym
index value size type bind oth ver shndx name
[0] 0x00000000 0x00000000 NOTY LOCL D 0 UNDEF
[1] 0x08050668 0x00000000 OBJT GLOB D 0 .plt _PROCEDURE_LINKAGE_TABLE_
[2] 0x08060974 0x00000004 OBJT WEAK D 0 .data environ
[3] 0x0806088c 0x00000000 OBJT GLOB D 0 .dynamic _DYNAMIC
[4] 0x080609c0 0x00000000 OBJT GLOB D 0 .bssf _edata
[5] 0x08060990 0x00000004 OBJT GLOB D 0 .data ___Argv
[6] 0x08050868 0x00000000 OBJT GLOB D 0 .rodata _etext
[7] 0x0805082c 0x0000001b FUNC GLOB D 0 .init _init
[8] 0x00000000 0x00000000 NOTY GLOB D 0 ABS __fsr_init_value
[9] 0x08050810 0x00000019 FUNC GLOB D 0 .text main
[10] 0x08060974 0x00000004 OBJT GLOB D 0 .data _environ
[11] 0x08060868 0x00000000 OBJT GLOB P 0 .got _GLOBAL_OFFSET_TABLE_
[12] 0x080506b8 0x00000000 FUNC GLOB D 0 UNDEF printstack
[13] 0x080506a8 0x00000000 FUNC GLOB D 0 UNDEF _exit
[14] 0x08050864 0x00000004 OBJT GLOB D 0 .rodata _lib_version
[15] 0x08050698 0x00000000 FUNC GLOB D 0 UNDEF atexit
[16] 0x08050678 0x00000000 FUNC GLOB D 0 UNDEF __fpstart
[17] 0x0805076c 0x0000007b FUNC GLOB D 0 .text __fsr
[18] 0x08050688 0x00000000 FUNC GLOB D 0 UNDEF exit
[19] 0x080506c8 0x00000000 FUNC WEAK D 0 UNDEF _get_exit_frame_monitor
[20] 0x080609c0 0x00000000 OBJT GLOB D 0 .bss _end
[21] 0x080506e0 0x0000008b FUNC GLOB D 0 .text _start
[22] 0x08050848 0x0000001b FUNC GLOB D 0 .fini _fini
[23] 0x08060978 0x00000018 OBJT GLOB D 0 .data __environ_lock
[24] 0x0806099c 0x00000004 OBJT GLOB D 0 .data __longdouble_used
[25] 0x00000000 0x00000000 NOTY WEAK D 0 UNDEF __1cG__CrunMdo_exit_code6F_v_
Symbol Table Section: .symtab
index value size type bind oth ver shndx name
[0] 0x00000000 0x00000000 NOTY LOCL D 0 UNDEF
[1] 0x00000000 0x00000000 FILE LOCL D 0 ABS test_ldynsym
[2] 0x080500f4 0x00000000 SECT LOCL D 0 .interp
[3] 0x08050108 0x00000000 SECT LOCL D 0 .SUNW_cap
[4] 0x08050118 0x00000000 SECT LOCL D 0 .hash
[5] 0x080501fc 0x00000000 SECT LOCL D 0 .SUNW_ldynsym
[6] 0x0805029c 0x00000000 SECT LOCL D 0 .dynsym
[7] 0x0805043c 0x00000000 SECT LOCL D 0 .dynstr
[8] 0x080505c4 0x00000000 SECT LOCL D 0 .SUNW_version
[9] 0x080505f4 0x00000000 SECT LOCL D 0 .SUNW_dynsymso
[10] 0x08050630 0x00000000 SECT LOCL D 0 .rel.data
[11] 0x08050638 0x00000000 SECT LOCL D 0 .rel.plt
[12] 0x08050668 0x00000000 SECT LOCL D 0 .plt
[13] 0x080506e0 0x00000000 SECT LOCL D 0 .text
[14] 0x0805082c 0x00000000 SECT LOCL D 0 .init
[15] 0x08050848 0x00000000 SECT LOCL D 0 .fini
[16] 0x08050864 0x00000000 SECT LOCL D 0 .rodata
[17] 0x08060868 0x00000000 SECT LOCL D 0 .got
[18] 0x0806088c 0x00000000 SECT LOCL D 0 .dynamic
[19] 0x08060974 0x00000000 SECT LOCL D 0 .data
[20] 0x080609c0 0x00000000 SECT LOCL D 0 .bssf
[21] 0x080609c0 0x00000000 SECT LOCL D 0 .bss
[22] 0x00000000 0x00000000 SECT LOCL D 0 .symtab
[23] 0x00000000 0x00000000 SECT LOCL D 0 .strtab
[24] 0x00000000 0x00000000 SECT LOCL D 0 .comment
[25] 0x00000000 0x00000000 SECT LOCL D 0 .debug_info
[26] 0x00000000 0x00000000 SECT LOCL D 0 .debug_line
[27] 0x00000000 0x00000000 SECT LOCL D 0 .debug_abbrev
[28] 0x00000000 0x00000000 SECT LOCL D 0 .shstrtab
[29] 0x080609c0 0x00000000 OBJT LOCL D 0 .bss _END_
[30] 0x08050000 0x00000000 OBJT LOCL D 0 .interp _START_
[31] 0x00000000 0x00000000 FILE LOCL D 0 ABS crti.s
[32] 0x00000000 0x00000000 FILE LOCL D 0 ABS crt1.o
[33] 0x00000000 0x00000000 FILE LOCL D 0 ABS crt1.s
[34] 0x08060994 0x00000004 OBJT LOCL D 0 .data __get_exit_frame_monitor_ptr
[35] 0x08060998 0x00000004 OBJT LOCL D 0 .data __do_exit_code_ptr
[36] 0x00000000 0x00000000 FILE LOCL D 0 ABS fsr.s
[37] 0x080609a0 0x00000020 OBJT LOCL D 0 .data trap_table
[38] 0x00000000 0x00000000 FILE LOCL D 0 ABS values-Xa.c
[39] 0x08060974 0x00000000 NOTY LOCL D 0 .data Ddata.data
[40] 0x080609c0 0x00000000 NOTY LOCL D 0 .bss Bbss.bss
[41] 0x08050868 0x00000000 NOTY LOCL D 0 .rodata Drodata.rodata
[42] 0x00000000 0x00000000 FILE LOCL D 0 ABS test.c
[43] 0x080507f0 0x00000019 FUNC LOCL D 0 .text static_func
[44] 0x080609c0 0x00000000 OBJT LOCL D 0 .bss Bbss.bss
[45] 0x08060974 0x00000000 OBJT LOCL D 0 .data Ddata.data
[46] 0x08050864 0x00000000 OBJT LOCL D 0 .rodata Drodata.rodata
[47] 0x00000000 0x00000000 FILE LOCL D 0 ABS crtn.s
[48] 0x08050668 0x00000000 OBJT GLOB D 0 .plt _PROCEDURE_LINKAGE_TABLE_
[49] 0x08060974 0x00000004 OBJT WEAK D 0 .data environ
[50] 0x0806088c 0x00000000 OBJT GLOB D 0 .dynamic _DYNAMIC
[51] 0x080609c0 0x00000000 OBJT GLOB D 0 .bssf _edata
[52] 0x08060990 0x00000004 OBJT GLOB D 0 .data ___Argv
[53] 0x08050868 0x00000000 OBJT GLOB D 0 .rodata _etext
[54] 0x0805082c 0x0000001b FUNC GLOB D 0 .init _init
[55] 0x00000000 0x00000000 NOTY GLOB D 0 ABS __fsr_init_value
[56] 0x08050810 0x00000019 FUNC GLOB D 0 .text main
[57] 0x08060974 0x00000004 OBJT GLOB D 0 .data _environ
[58] 0x08060868 0x00000000 OBJT GLOB P 0 .got _GLOBAL_OFFSET_TABLE_
[59] 0x080506b8 0x00000000 FUNC GLOB D 0 UNDEF printstack
[60] 0x080506a8 0x00000000 FUNC GLOB D 0 UNDEF _exit
[61] 0x08050864 0x00000004 OBJT GLOB D 0 .rodata _lib_version
[62] 0x08050698 0x00000000 FUNC GLOB D 0 UNDEF atexit
[63] 0x08050678 0x00000000 FUNC GLOB D 0 UNDEF __fpstart
[64] 0x0805076c 0x0000007b FUNC GLOB D 0 .text __fsr
[65] 0x08050688 0x00000000 FUNC GLOB D 0 UNDEF exit
[66] 0x080506c8 0x00000000 FUNC WEAK D 0 UNDEF _get_exit_frame_monitor
[67] 0x080609c0 0x00000000 OBJT GLOB D 0 .bss _end
[68] 0x080506e0 0x0000008b FUNC GLOB D 0 .text _start
[69] 0x08050848 0x0000001b FUNC GLOB D 0 .fini _fini
[70] 0x08060978 0x00000018 OBJT GLOB D 0 .data __environ_lock
[71] 0x0806099c 0x00000004 OBJT GLOB D 0 .data __longdouble_used
[72] 0x00000000 0x00000000 NOTY WEAK D 0 UNDEF __1cG__CrunMdo_exit_code6F_v_
Now, we strip the two versions of our program to remove the .symtab symbol table, and force the system to use the dynamic tables instead:
% strip test_ldynsym test_noldynsym % file test_ldynsym test_noldynsym test_ldynsym: ELF 32-bit LSB executable 80386 Version 1, dynamically linked, stripped test_noldynsym: ELF 32-bit LSB executable 80386 Version 1, dynamically linked, stripped
Running the version without a .SUNW_ldynsym section:
% ./test_noldynsym /home/ali/test/test_noldynsym:0x6ca /home/ali/test/test_noldynsym:main+0xb /home/ali/test/test_noldynsym:_start+0x7a Segmentation Fault (core dumped) % pstack core core 'core' of 5041: ./test_noldynsym 080506d2 ???????? (804692c, 80467a4, 805062a, 1, 80467b0, 80467b8) 080506eb main (1, 80467b0, 80467b8) + b 0805062a _start (1, 8046994, 0, 80469a5, 80469bf, 8046a03) + 7a
Our program used the printstack(3C) function to display its own stack. Afterwards, we use the pstack command to view the same data from the core file. In both cases, the top line represents the call to the local function static_func(), a fact that we know from examining the source code, since the number and/or ‘????????’ used to represent it are less than obvious to an external observer.
Running the version with a .SUNW_ldynsym section, the system is able to put a name to the local function:
% ./test_ldynsym /home/ali/test/test_ldynsym:static_func+0xa /home/ali/test/test_ldynsym:main+0xb /home/ali/test/test_ldynsym:_start+0x7a Segmentation Fault (core dumped) % pstack core core 'core' of 5044: ./test_ldynsym 08050802 static_func (8046930, 80467a8, 805075a, 1, 80467b4, 80467bc) + 12 0805081b main (1, 80467b4, 80467bc) + b 0805075a _start (1, 8046998, 0, 80469a7, 80469c1, 8046a05) + 7a
Conclusions
Sometimes it is the little things that make a difference. I expect that the local dynamic symbol table will provide valuable information in difficult debugging situations where one is examining large stripped programs running in a production environment. The rest of the time, the additional data is small, and will have little or no impact on performance.
.SUNW_ldynsym sections have been part of the Solaris development (Nevada) builds since last fall, and are also available in OpenSolaris.
[ This article is permanently archived at http://www.linker-aliens.org/blogs/ali/entry/what_is_sunw_ldynsym/ ]