X

Using Solaris and SPARC Networking and Virtualization

Increasing Number of Open Files Without a Reboot


Background


Last week a customer told me that one of the applications they are moving to a new SuperCluster M7 requires 20,000 file descriptors per process! At first I thought they were kidding, however, after looking at the application's documentation, they indeed expect to have a large number of files open at once.

After thinking about it a bit, I imagined there might be three ways of doing this.

  1. Use "/etc/system" for the whole Domain
  2. Use projects to increase the limit for the Solaris Zone(s) where this application runs
  3. Use projects within the Zone(s) to increase the limit for the user the application is running as

Using the old /etc/system method was not something I wanted to suggest. First, with a request for 20,000, setting rlim_fd_cur could allow any user to set the limit that high. Moreover, it requires a reboot of the Global Zone, something I like to avoid. In this case that would require each DB Zone in this Domain to be rebooted. Nope, lets move on right now.

Looking at the zonecfg(1M) man page, I did not see how I could set the maximum number of open files (the process.max-file-descriptor resource.) And if I could, it would still apply to all processes and users in the Zone, which I did not like. (It turns out that the rctl setting only works on Zone related controls, those starting with "zone." in resource-controls(5).)

So on to the third option, using the Projects facility to apply this high limit this to the specific user or process. I read pages such as projects(1), project(4), and resource-controls(5). This seemed more complicated than I had thought. The examples I found are more related to CPU or memory controls, than the number if file descriptors. I did see a reference to something similar in resource-controls(5) that looked interesting: process.max-file-size=(priv,5G,deny) for the maximum file size. Some hope.

Using a Solaris Zone on a Solaris 11.3 system, I did a lot of testing. I could not get it to work. After reaching out to JeffV, he mentioned the three privileges: System, Privileged, and Basic. That was the key. Once I got it working, the steps are rather simple!

The Default Limits


I gave the Solaris Zone I created for this test the nodename "limit" so I can tell it from different Zones I have installed, and to remind me in the Zone what I am using it for.

Creating a User With Default Limits

root@limit:~# ulimit -n 
256
root@limit:~#
root@limit:~# groupadd -g 1001 test
root@limit:~#
root@limit:~# useradd -u 1001 -g test -d /export/home/steffen -m steffen
80 blocks
root@limit:~# passwd steffen
New Password: xxx
Re-enter new Password: xxx
passwd: password successfully changed for steffen
root@limit:~#

Testing the User


After creating my test user, I log in as that user.
root@limit:~# ssh steffen@localhost 
Password: xxx
Last login: Tue Oct 4 14:09:37 2016 from localhost
Oracle Corporation SunOS 5.11 11.3 August 2016
steffen@limit:~$
steffen@limit:~$ ulimit -n
256
steffen@limit:~$

A quick test shows the default number of file descriptors to be 256 as expected. And we know from experience that as a unprivileged (non-root) user we can only set this so much higher.
steffen@limit:~$ ulimit -n 1024 
steffen@limit:~$ ulimit -n
1024
steffen@limit:~$
steffen@limit:~$ ulimit -n 1025
-bash: ulimit: open files: cannot modify limit: Not owner
steffen@limit:~$ ulimit -n
1024
steffen@limit:~$

I can increase the default number of file descriptors per process to 1024, but not higher.

Using Projects to Set a Higher Limit


Now that I have this working, the steps to create a project with a limit is very easy. I already have the user from above.
root@limit:~# projadd -K "process.max-file-descriptor=(basic,4096,deny)" proj.files 
root@limit:~#
root@limit:~# usermod -K "project=proj.files" steffen
root@limit:~#
root@limit:~# cat /etc/user_attr
#
# Copyright (c) 1999, 2013, Oracle and/or its affiliates. All rights reserved.
#
# The system provided entries are stored in different files
# under "/etc/user_attr.d". They should not be copied to this file.
#
# Only local changes should be stored in this file.
#
root::::type=role
...
steffen::::project=proj.files
root@limit:~#

The first step is to create the project. I am setting the number of file descriptors to 4096. The Privilege I mentioned before is key. It has to be "basic" since the user "steffen" has no enhanced privileges.

The second step is to modify the user to set the default project to my new project "proj.files". I could have done this to the default project or to another project that already exists. I would not use the default project myself as it would allow too many processes to have a larger limit.

So how does this look?

root@limit:~# ssh steffen@localhost 
Password:
Last login: Tue Oct 4 14:09:49 2016 from localhost
Oracle Corporation SunOS 5.11 11.3 August 2016
steffen@limit:~$
steffen@limit:~$ ulimit -n
4096
steffen@limit:~$

What surprised me a bit, I guess since I don't really understand the Project facility in Solaris, is that automatically the number of file descriptors is set higher. I was expecting to have to run "ulimit" in a ".bash_rc" script.

Conclusion


So there you have it. It is rather simple to create a new project and set one or more limits. Depending on how this goes with the customer, I may come back and show a project with several limits.

Steffen

Appreciations


Thanks to JeffV and JimH who responded quickly to me calls and emails. Both mentioned "prctl(1)" which works on an active process. Because of that, I did notice the "rctl" in the zonecfg(1M) manual page.

Revision History

(Other than minor typographical changes)

2016.10.04: Posted

2016.10.04: Created

Be the first to comment

Comments ( 0 )
Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.Captcha