Resource control observability using kstats
By menno on Sep 12, 2007
prctl(1)), getting the actual usage requires work and implementation knowledge.
For instance, we could get the amount of System V shared memory used by a project using
ipcs -Jam and some parsing of its output. Or fire up
mdb(1) and lookup the value for
kpd_shmmax in the project's
kproject_t struct. And, if we wanted to get the usage of another resource control (say the number of lwps), we'd need to use another tool (
prstat -LJc) or know that the number of lwps is kept in the
kpj_nlwps member. Hardly usable for more than the occasional peek. Plus that relying on kernel implementation details such as these structure members is highly inadvisable as they may change in the future (they probably won't, but they are not stable interfaces so don't rely on them).
The addition of the swap and locked memory resource controls by PSARC 2006/598 Swap resource control; locked memory RM introduced a number of kstats for observability:
These kstats have a 'value' statistic for the current limit and a 'usage' statistic that holds the current usage:
$ kstat -c zone_caps -n swapresv_zone_0 module: caps instance: 0 name: swapresv_zone_0 class: zone_caps crtime 0 snaptime 102512.50351337 usage 532168704 value 18446744073709551615 zonename global
Exposing these values as kstats gives us exactly what is needed, a simple, well defined method to get the limit and usage for a resource control.
To satisfy my curiosity and to see what changes would be needed, I spent some evenings creating a prototype that adds kstats for all project.\* and zone.\* resource controls. The following extra kstats are available in the prototype:
Getting a list of the current usage of all resource controls is now as simple as typing:
$ kstat -p caps:::usage caps:0:contracts_project_0:usage 33 caps:0:contracts_project_1:usage 2 caps:0:contracts_project_101:usage 0 caps:0:cryptomem_project_0:usage 0 ... caps:5:nlwps_project_0:usage 108 caps:5:nlwps_zone_5:usage 108 caps:5:ntasks_project_0:usage 15 caps:5:semids_project_0:usage 0 caps:5:semids_zone_5:usage 0 caps:5:shmids_project_0:usage 1 caps:5:shmids_zone_5:usage 1 caps:5:shmmem_project_0:usage 172032 caps:5:shmmem_zone_5:usage 172032 caps:5:swapresv_zone_5:usage 95178752
And now that we have the numbers as kstats, we can use any tool to massage the numbers into a form that suits us. The screenshot below is from a hacked up version of one of the JKstat demo programs and shows a graph of the number of LWPs in all projects and zones during boot and shutdown of a Zone.