The nuances of poolbind

poolbind(1M) allows an authorized user to change the resource pool binding of a zone, project, task or process. However, what does this actually mean? As you might imagine, it means different things for the different pool-bindable entities. Here's a quick explanatory note that should help eliminate any confusion.

The main point to note is this: any changes realized through poolbind(1M) are temporal in nature and will only affect the set of processes identified as belonging to the target process collective at that point in time. By this I mean that no permanent configuration is changed.

For processes and tasks, this is a moot point since there is no way (currently) to specify that either a task or a process is associated with a resource pool. I can't really think of any reason why you would ever want such fine-grained resource management that you would want to have a configuration that specified this relationship; so this is unlikely to ever change.

If you want to permanently change the binding of a project or zone to a resource pool then you must edit the configuration (favourite editor for /etc/project(4) or zonecfg(1M) for a zone). If you want the change to take place immediately, rather than the next time you boot your OS, then you must also use poolbind(1M) to execute the temporal change immediately.

So what actually happens when you use poolbind(1M) to modify the binding of a resource pool bind-able entity?
process The process stops consuming the resources associated with its existing resource pool and begins consuming the resources associated with the new resource pool. Any new processes which fork(2) from this process are also bound to the new resource pool.
task All processes which belong to the task stop consuming the resources associated with their existing resource pool and begin consuming the resources associated with the new resource pool. Any new processes which fork(2) from one of these processes are also bound to the new resource pool.
project All processes which belong to the project stop consuming the resources associated with their existing resource pool and begin consuming the resources associated with the new resource pool. Any new processes which fork(2) from one of these processes are also bound to the new resource pool. However, new processes which join the project after this operation completes will still be associated with the configured resource pool.
zone All processes which belong to the zone stop consuming the resources associated with their existing resource pool and begin consuming the resources associated with the new resource pool. Any new processes which fork(2) from one of these processes are also bound to the new resource pool. Re-booting the zone will restore the configured resource pool binding.

poolbind(1M) is a very powerful workload managment command, especially when applied to a process collective such as a zone or a project. I'll return to the subject of poolbind(1M) later and give some examples of how it can be used to manage workloads quickly and flexibly when problems occur.

Comments:

Hi, I'm interested in resource management in general, especially since I'm testing out S10. I work for a web hosting company and we often have issues where certain processes(email, mysql, apache, etc) can hog a systems resources. I'm wondering if you could point me to some good examples on how to divide out processes into projects and manage them. Also, are there ways to build logic into resource management? (i.e. if lets say the process for backups run at a particular time of day, can I automate the process of giving the backups greater priority/shares over the other projects?) Please forgive my ignorance of resource management, it's something that I've been meaning to learn in the past, but have not made the time until now.

Posted by Octave Orgeron on August 06, 2004 at 01:10 PM BST #

Hi Octave, <p/> You should read the resource management documentation <p/> In particular, part 1 should be helpful. <p/> There are plenty of examples in that documentation and Chapter 14 has a substantial example of how to configure a system for a hypothetical server consolidation project. This material will introduce you to the fundamental concepts of Solaris Resource Management: projects, tasks, resource pools, schedulers (in particular Fair Share Scheduler), resource controls, etc. You need to know what all of these things are to make any sense out of the rest of my answers. <p/> To try and answer your specific questions... <p/> A good approach to managing your processes is to view them as services/applications and to create different projects for each service/application which you wish to manage separately, e.g. database, web, email, etc. Each of these projects should be configured to run in a certain resource pool or have certain processor shares depending on how you decide to allocate resource to these different services. You can now wrap the startup procedure for these different services with a script (or you can modify the startup script if you like) which uses the <code>newtask(1)</code> command to place the processes which are started into the appropriate project. <p/> There are many ways to build logic into resource management. Most facilities are easily scriptable via their command interfaces or if you require low level access to the facilities, you can use the "C" APIs to write your own programs. Take a look at the libpool(3LIB) API for instance. <p/> A nice way to approach your particular backup example would be to use the Fair Share Scheduler and place the backup service into a project with a large number of CPU shares. When the service was active it would receive a large portion of the available CPU resrources, when it wasn't active it would have no impact on the share of CPU resources available to other services. (You'll note I'm using the word service rather than process, I'm trying to emphasise that this is really a business decision and not a technical decision. You just decide which business services matter most and then configure the system to treat service processes according to your configured policy.) <p/> I hope these tips are helpful, let me know how you get on or if you need more help/guidance. <p/> Gary

Posted by Gary Pennington on August 09, 2004 at 02:18 AM BST #

I started reading Part 1 and I'm trying out some examples. So here is what I've done on my single-proc x86 box: # projects -l test01 test01 projid : 1979 comment: "Test for RSM and poolbind" users : oorgeron groups : (none) attribs: # pgrep mozilla 598 # newtask -v -p test01 -c 598 16 # pgrep -T 16 598 # priocntl -s -c FSS -i projid 1979 # prctl -n project.cpu-shares -i project test01 project: 1979: test01 NAME PRIVILEGE VALUE FLAG ACTION RECIPIENT project.cpu-shares privileged 5 - none - system 65.5K max none - So now I have a project setup that's running my mozilla process with 5 cpu shares. So now I want to create a resource pool and bind this project to it. # pooladm -e # poolcfg -c info system saito string system.comment int system.version 1 boolean system.bind-default true int system.poold.pid 3546 pool pool_default int pool.sys_id 0 boolean pool.active true boolean pool.default true int pool.importance 1 string pool.comment pset pset_default # poolcfg -c 'create pset pset_test (uint pset.min = 2; uint pset.max = 10)' # poolcfg -c 'create pool pool_test' # poolcfg -c 'associate pool pool_test (pset pset_test)' # poolcfg -c info # poolcfg -c info system saito string system.comment int system.version 1 boolean system.bind-default true int system.poold.pid 3546 pool pool_default int pool.sys_id 0 boolean pool.active true boolean pool.default true int pool.importance 1 string pool.comment pset pset_default pool pool_test boolean pool.active true boolean pool.default false int pool.importance 1 string pool.comment pset pset_test pset pset_default int pset.sys_id -1 boolean pset.default true uint pset.min 1 uint pset.max 65536 string pset.units population uint pset.load 397 uint pset.size 1 string pset.comment cpu int cpu.sys_id 0 string cpu.comment string cpu.status on-line pset pset_test int pset.sys_id -2 boolean pset.default false uint pset.min 2 uint pset.max 10 string pset.units population uint pset.load 0 uint pset.size 0 string pset.comment So if I try to bind my project to the pool_test, I get this: # poolbind -i project -p pool_test test01 poolbind: binding project 1979 to pool 'pool_test': Bad parameter supplied Also, if I try to commit my configuration, I get this: # pooladm -c pooladm: configuration at '/etc/pooladm.conf' cannot be instantiated on current system So dumb question.. What am I doing wrong? Did I setup the pool incorrectly? What exactly does the pset.min and pset.max settings mean? I'm using the examples in the documentation. What I'd like to do is play around with my netscape process, since it's something that I use alot at work and see how it behaves while I run other stuff. Also, I'd like to setup another pool that has more resources and switch the project to it while I'm using it. Thanks! Octave

Posted by Octave Orgeron on August 13, 2004 at 10:09 AM BST #

<p/> Hi Octave, <p/> Sorry for the delay replying, I've been on vacation and only returned to work today. <p/> Here are some specific answers to your questions and some background that should enhance your understanding of what is happening when you type your commands. <p/> Actually, I'll do the background stuff first as that will make the later specific answers more intelligible. <p/> A Resource Pool configuration is a representation of the configurable resources for an executing instance of Solaris. <ocde>poolcfg(1M)</code> is used to manipulate the entities (pools, psets, cpus, etc..) in the configuration. The configuration could be held in a file, in which case you are simply moving XML elements around in an XML document (file) or it could be a direct representation of kernel state, in which case poolcfg is directly altering the disposition of resources in your kernel. <p/> When you use the -d flag to <code>poolcfg</code> you are directly modifying kernel state, otherwise you are working on an XML document. If you don't supply a document location, your are working at the default location of <code>/etc/pooladm.conf</code>. <p/> I'll walk through your comment and try to explain what's happening at each step. <p/> # pooladm -e <p/> This enables pools <p/> # poolcfg -c info system <p/> List the contents of the configuration at the default location, i.e. <code>/etc/pooladm.conf</code>. Note that if this file does not exist then you will get an error. Also note, this is not the state of the kernel, it is simply a representation of a configuration that could be applied to a kernel. <p/> # poolcfg -c 'create pset pset_test (uint pset.min = 2; uint pset.max = 10)' <p/> This modifies the contents of <code>/etc/pooladm.conf</code> to add a new entity of type pset with name pset_test and all the other specified attributes. Attributes which aren't specified take the default values. Note that this has no effect on the kernel at this stage and no assessment is made as to whether this configuration could be applied to the current system. As long as the syntax is valid, the command succeeds. Only when the configuration is applied to the system are assessments made for the feasibility of applying this configuration to the system. <p/> # poolcfg -c 'create pool pool_test' <p/> # poolcfg -c 'associate pool pool_test (pset pset_test)' <p/> Create pool pool_test and associate pset pset_test to the pool. This further alters the contents of <code>/etc/pooladm.conf</conf> to add the new pool entity and to record that the pool should use the processor resources from pset pset_test. <p/> # poolcfg -c info <p/> List the configuration at <code>/etc/pooladm.conf</code> so that we can confirm that our changes are correct. NB: If you want to see the active (i.e. kernel) configuration then just type <code>pooladm</code> with no arguments. Does that make it clear that the contents of the <code>/etc/pooladm.conf</code> file and the kernel disposition of resources are two different things? It should. <p/> # poolbind -i project -p pool_test test01 <p/> Attempt to bind all the processes in project test01 to pool pool_test. This is failing because binding operations are always performed against the active configuration (i.e. the kernel) and as yet the changes you have made are only in a configuration file. You must commit your changes before using them. In other words you need to type <code>pooladm -c</code>. Unfortunately, this will fail as I explain in the next step. <p/> pooladm -c pooladm: configuration at '/etc/pooladm.conf' cannot be instantiated on current system <p/> You are trying to commit a configuration which requires a minimum of 3 CPUs on a single CPU system. This isn't physically possible, so the commit operation fails and tells you this. It isn't possible because you must have enough resources to be able to satisfy the constraints that you have established on your resource partitions. In this case, you have specified two processor sets, pset_default and pset_test, with a combined minimum of three and you only have a single CPU system <p/> To answer your specific questions... <p/> You aren't doing anything wrong, you just had a couple of problems. <p/> pset.min and pset.max specify the limits of CPU resources for the pset. i.e. the pset cannot have less than min or more than max CPUs. It can have any value between these limits. This is only enforced when applied to the kernel, you can put any syntactically correct value in a configuration file. <p/> The main problem you have is that it's difficult to use resource pools on a single CPU system, since you can't partition a single CPU system. <p/> Read the manpage <code>libpool(3LIB)</code> for more details on what all the different attributes mean for the different entities in the configuration. <p/> Finally, it shoudl be clear now that there two ways to change the resource pool configuration of a system. <p/> You can put together a configuration file which describes how you would like a system to be configured. You apply this to a system, using <code>pooladm -c</code> and the resource pools framework decides how best to allocate CPUs to sets and performs all the configuration steps in a batch. If any steps fail, changes are undone and the system restored (as far as possible) to it's original state. <p/> Alternatively, you can use the -d option to <code>poolcfg</code> and directly modify the state of the system. This is desirable when you want to make small changes to an existing configuration, if you want fine control over the disposition of CPU resources or when you are just experimenting with the resource pools facility. Changes are immediate and thus any failures in later steps will not affect earlier changes. <p/> I hope the above helps, let me know if you have more questions. <p/> Gary

Posted by Gary Pennington on August 23, 2004 at 03:41 AM BST #

Hi Gary, Thanks for the reply! Your feedback definitely helped out with understanding things better. I was getting a little confused using the examples in the documentation. Still, I can see how resource pools could be very handy on an SMP box for controlling resources. I was talking to a co-worker about how we could consolidate to a larger box and place the applications/services for our web hosting environment into resource pools for better utilization and control. I'm looking forward to testing this out at home on my U60. Is is possible to apply these resources to a zone? For example, could I place one zone in a pool that only uses one of my CPU's at any point in time and have another zone that uses both? Also, how exactly are shares determined? I'm still reading through the documentation on docs.sun.com, but I'm starting to feel more comfortable with the commands:) Octave

Posted by Octave Orgeron on August 24, 2004 at 03:19 PM BST #

Hi Octave, <p/> It's possible to associate a zone to a pool, in fact I think this will be the most common use of resource pools in Solaris 10. You can specify which pool a zone should draw it's resources from as follows:
$ zonecfg -z myzone
zonecfg:myzone> set pool=mypool
zonecfg:myzone> exit
<p/> That was taken from the <code>zonecfg(1M)</code> manpage. You can also manually change the pool binding for a zone with <code>poolbind(1M)</code>. <p/> It isn't possible to achieve your second scenario using pools/processor sets. The mechanism is a resource isolation mechanism and so sharing of CPU (or in the future other types of resource) isn't possible. The best way to address CPU sharing requirements is to look at the use of the fair share scheduler, <code>man FSS(7)</code>. <p/> Keep reading through the docs. The resource management facilities in Solaris are extensive and powerful, the downside of this is that there is some inevitable complexity which requires thorough reading of the documentation. <p/> Thanks for your feedback and interest, <p/> Gary

Posted by Gary on August 25, 2004 at 12:38 AM BST #

Post a Comment:
Comments are closed for this entry.
About

garypen

Search

Archives
« April 2014
MonTueWedThuFriSatSun
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
    
       
Today