Breaking the stdio(3C) 256 file descriptor barrier in Solaris' stdio
By Stw-Oracle on Jan 28, 2008
Solaris has historically allowed only 256 stdio streams to be open, where the file descriptors are below 256. So applications can quickly run out of file descriptors when doing lots of fopen() calls. For 32-bit applications, it has not been possible to increase this limit, as it could cause binary compatibility issues for older applications (compatibility going back as far back as those compiled on SunOS 4.x). The dup(2) system call has been used to move other file descriptors above 256 to free up slots for fopen. But the application is still limited to a maximum of 256 stdio streams!
With the release of Solaris 10 8/07 (often referred to as update 4), there is a new interface to extend the FILE facility. Programming details are in the man page enable_extended_FILE_stdio(3C). And if you don't want to make any code changes, extendedFILE(5) describes how to do this for existing applications and binaries.
I am working with a customer who needs to host over 1,400 web sites. We are using portions of the coolstack, as well as customized versions of Apache and PHP. With virtual hosting, the setup quickly hit the 256 stdio file limit!
With a small change to apachectl, it is now possible to host all 1,400+ web sites within a single instance of Apache. I added the following to the configuration section of apachectl:
ulimit -n 3000 LD_PRELOAD_32=/usr/lib/extendedFILE.so.1 ; export LD_PRELOAD_32
The ulimit -n 3000 increases the number of file descriptors a process can have open to 3000, up from the default of 256. Since apachectl is run as root, or with sufficient privileges using Role Bases Access Control, this is permitted.
The LD_PRELOAD_32 setting allows me to have the library provide special versions of library functions or system calls. In this case, it does special things when fopen is called, and automatically uses dup(2) to free up the lower 256 file descriptors.
The enable_extended_FILE_stdio(3C) man pages lists some of the requirements for an application to work well with this interposition library, such as not doing direct access into the fields of the FILE structure. Since Apache is using stdio for log files, it is unlikely that Apache is accessing the structures directly.
Testing with the customer's configuration has Apache serving up all 1,400 web sites using a single instance of the httpd server! Cool, success at last!