Finding Memory leaks Within Solaris Applications Using libumem
By pnayak on Jul 18, 2007
Debugging Methodology :
The malloc() and free() memory management methods are used by many application developers. An application can be written without a dependence on any particular memory management programming interface by using the standard memory management methods malloc() and free(). This section will outline the steps needed to take advantage of the libumem library to debug an application's memory transactions.
Library Interposition and libumem Flags
If the libumem library is interposed (by setting the LD_PRELOAD environment variable) when executing an application, the malloc() and free() methods defined within the libumem library will be used whenever the application calls malloc() or free(). In order to take advantage of the debugging infrastructure of the libumem library, one needs to set the UMEM_DEBUG and the UMEM_LOGGING flags in the environment where the application is being executed. The most common values for these flags are as follows: UMEM_DEBUG=default and UMEM_LOGGING=transaction. With these settings, a thread ID, high-resolution time stamp, and stack trace are recorded for each memory transaction initiated by the application.
The following are examples of the commands used to set the appropriate debug flags and interpose the libumem library when executing an application.
%(setenv UMEM_DEBUG default; setenv UMEM_LOGGING transaction;
setenv LD_PRELOAD libumem.so.1;)
bash-2.04$UMEM_DEBUG=default UMEM_LOGGING=transaction LD_PRELOAD=libumem.so.1
More details about the debug flags (UMEM_DEBUG and UMEM_LOGGING) can be found in the umem_debug(3MALLOC) man page.
The developer can view the debug information pertaining to an application's memory management transactions by using MDB. The following commands within MDB can be used to provide a great deal of information about the memory transactions that took place during the execution of the application.
\* Prints the status of the umem indicating if the logging features have been turned on or off
\* Prints a summary of the memory leaks found within the application
\* Prints the memory transactions initiated by the application and the correlated stack traces
\* Prints the details about each of the umem caches
\* Prints the umem transaction log for the application
\* Prints the integrity of the umem caches which is useful in determining if a buffer has been corrupted
\* Prints the contents of the umem_bufctl_audit structure as defined in the /usr/include/umem_impl.h header file
Traditional Memory Leak
In order to examine if SunMC has a memory leak, one can execute the following steps to narrow down the section of the code which is causing the leak.
1. The libumem library is only available on systems which are running the Solaris 9 OS, Update 3 and above.
SunOS hostname 5.9 Generic_112233-05
2. Execute the application with the libumem library interposed and the appropriate debug flags set.
%(setenv UMEM_DEBUG default; setenv UMEM_LOGGING transaction;setenv LD_PRELOAD libumem.so.1; ./a.out)
3. Use the gcore (1) command to get an application core to analyze the application's memory transactions.
%ps -ef | grep esd
user1 970 714 0 10:42:42 pts/4 0:00 ./a.out
gcore: core.970 dumped
4. Use MDB to analyze the core for memory leaks using the commands described in the previous section.
$mdb core.970 Loading modules: [
libumem.so.1 libc.so.1 ld.so.1 ]> ::umem_log CPU ADDR BUFADDR TIMESTAMP THREAD 0 0002e0c8 00055fb8 159d27e121a0 00000001 0 0002e064 00055fb8 159d27e0fce8 00000001 0 0002e000 00049fc0 159d27da1748 00000001 00034904 00000000 0 00000000 00034968 00000000 0 00000000 ... snip ...
Here we can see that there have been three transactions by thread #1 on cpu #0.
> ::umalog T-0.000000000 addr=55fb8 umem_alloc_32
libumem.so.1`free+0x38 main+0x18 _start+0x108 T-0.000009400 addr=55fb8 umem_alloc_32 libumem.so.1`umem_cache_alloc+0x13c libumem.so.1`umem_alloc+0x44 libumem.so.1`malloc+0x2c main+0x10 _start+0x108 T-0.000461400 addr=49fc0 umem_alloc_24 libumem.so.1`umem_cache_alloc+0x13c libumem.so.1`umem_alloc+0x44 libumem.so.1`malloc+0x2c main+4 _start+0x108
The three transactions consist of one allocation to the 24 byte
umemcache, and one memory allocation and release from the 32 byte
umemcache. Note that the high resolution timestamp output in the upper left hand corner is relative to the last memory transaction initiated by the application.
> ::findleaks CACHE LEAKED BUFCTL CALLER 0003d888 1 00050000 libumem.so.1`malloc+0x0 ---------------------------------------------------------------------- Total 1 buffer, 24 bytes
This shows that there is one 24 byte buffer which has been leaked.