Wednesday Oct 27, 2004

Imperfect C++

When my life was less cluttered and I seemed to have more time to myself, I used to go cycling regularly with a group of friends in West Yorkshire. One of the group, Matthew, was a PhD student at the time and the brother-in-law of a very good friend of mine. Over time, Matthew became a friend and we used to enjoy cycling together across the hills and roads of Yorkshire.

Since that time Mathew has made several bad life choices. On finishing his PhD he pursued a career in Software Engineering, actually choosing to spend his time writing C++! He then compounded all this wrong thinking by leaving Yorkshire and emigrating to Australia, the homeland of his beautiful wife. Now he has written a book about C++.

What can I say? Such promise gone so badly wrong. ;-)

If you are unfortunate enough to have to work with C++ source for any reason, then I can only recommend that you go out and buy this book (I will declare my interest at this point and say that I did review the book.). I can't say that I agree with every sentiment in the book, but it's well written and you are sure to learn lots about C++ along the way.

I wonder whether Matthew has scored an own goal on behalf of the C++ community though. Although his book is brilliant and certainly does show many techniques which can be applied to make C++ more usable; most readers are likely to be deterred from consideration of the language due to the complexity of the techniques required.

Tuesday Aug 31, 2004

Cycling as a metaphor for Software Development

It has been brought to my attention that my postings to date on this blog are entirely too factual and completely lacking in opinion and speculation. In fact things are so bad that if I was to post one more useful or factual article the usability of the internet would cross a "utility event horizon". Beyond this point the proportion of the internet which is accurate or useful is dangerously high1, so high that people may actually have to believe what they read. I'm certain that's not an outcome any of us would like. I'm taking action to prevent this by publishing this whimsical piece of complete nonsense which I dreamed up when riding my bike earlier today.

Cycling is like developing software. Especially where I live, since I live in a valley surrounded by nasty, steep hills (the Pennines) and the prevailing winds are in my face. If you think about it, when you start a new project, the tasks you have to complete are like these hills. The nasty winds represent the opinions of other parties, often colleagues, who can't resist letting you know what a complete waste of time your project is.

If you can summon the enthusiasm to mount your bike and start to go up your hill, i.e. start your project, you'll soon find that hills and wind are only a small part of the challenges you'll need to overcome. Soon you'll notice the potholes in the road, designed to shake your teeth loose and permanently damage your expensive cycle. These represent the bugs and design flaws which you will inevitably encounter as you begin to convert your vision of a perfect piece of software into a imperfect reality. Of course, it's better to encounter these bugs when going uphill, since it's easier to avoid them now and there's even a chance they might be filled in before you begin your descent.

As you weave slowly up the hill, dodging the potholes and labouring in the wind, another obstacle presents itself: the maniac driving the white van. Inevitably an overweight, shaven headed yob who has no idea how much space he should leave between the side of his van and yourself. I think that this best represents competing projects from within your own organisation or from other vendors. They know their project will fix all that is wrong in the world, who needs your pathetic contribution? Best just to to smear it all over the road now before it can reach the top of the hill.

Next we have to understand the "false dawn". Often, when cycling uphill in unknown terrain, you will think you are close to the top of the hill only to find that when you cycle a little further there are still many miles to go before your reach the real crest of the hill. I'm sure that all software developers will have experienced this feeling: will the project never end?

If you are lucky you'll get to the top of the hill. Now it's time to cycle back home. This represents the completion of development on the project. You'll notice that the potholes are a lot more perilous now because you are going so fast. Just like the bugs and design flaws, almost certainly more painful when discovered by customers than when you or your colleagues find them.

There you have it. Cycling is the perfect metaphor for software development and I'm sure that we can all learn something from this thought.

There are many of features of cycling which I'm sure provide opportunities for comparison with software development, e.g. the weather, your equipment, etc... Feel free to comment if you want to extend the comparision or if you think I'm wrong. Just don't make your comments informative or useful.


1 I'm not sure exactly what this figure is but I suspect it's something like 0.00000000000000000000000000000000000001 %

Monday Aug 09, 2004

Using poolbind to execute jobs in different pools

You are lazy. Of course you are, it's a virtue when applied to computing. You don't want to type reams of instructions and check to see if commands succeeded or failed. No way, that's just too time consuming. Joe Salaryman can waste his time doing that, but you've got better things to do.

In that case, here's a minimal script which wraps poolbind(1M) to make it easier to run a process in a particular pool. For example, let's say that I wanted to run ls(1) in pool "listings".

[1]
bash-2.05b$ newpool
usage: newpool <pool name> <command> [parameters]
[2]
bash-2.05b$ newpool listings ls
poolbind: binding pid 100964 to pool 'listings': Not owner
Bind operation failed
[3]
bash-2.05b$ su -
Password:
Sun Microsystems Inc.   SunOS 5.10      s10_63  Jul. 03, 2004
SunOS Internal Development: gk 2004-07-03 [on10_63]
bfu'ed from /bfu/s10_63 on 2004-07-22
Sun Microsystems Inc.   SunOS 5.10      s10_37  May 2004
You have mail.
# bash
[4]
bash-2.05b#  newpool listings ls
backup         bin            Documents      mnt            tmp
bfu            boot           etc            net            TT_DB
bfu.ancestor   cdbuild        export         opt            usr
bfu.child      cdrom          home           platform       var

bfu.conflicts  Desktop        kernel         proc           vol
bfu.parent     dev            lib            rootb
bfu.realmode   devices        lost+found     sbin
bash-2.05b#
What's going on here?
  1. Run the script to display the usage message. It's fairly clear I think.
  2. Now try to run the script as myself. Oops, I'm don't have the privileges to bind to a differnt pool.
  3. I'll become root, I know I have enough privileges now ;-)
  4. Run it again and this time it works.
The script is simple:
#!/bin/sh

if [ $# -lt 2 ]
then
        echo "usage: newpool <pool name> <command> [parameters]"
        exit 2
fi
/usr/sbin/poolbind -p $1 $$
if [ $? != 0 ]
then
        echo "Bind operation failed"
        exit 1
fi
shift
exec $\*
I mainly use this script when I'm developing resource pools and I want to launch test programs into different pools. My requirements are fairly minimal, so the script is perfect for me. Let me know if you have ideas for enhancements that would help make this script more useful.

Monday Aug 02, 2004

The nuances of poolbind

poolbind(1M) allows an authorized user to change the resource pool binding of a zone, project, task or process. However, what does this actually mean? As you might imagine, it means different things for the different pool-bindable entities. Here's a quick explanatory note that should help eliminate any confusion.

The main point to note is this: any changes realized through poolbind(1M) are temporal in nature and will only affect the set of processes identified as belonging to the target process collective at that point in time. By this I mean that no permanent configuration is changed.

For processes and tasks, this is a moot point since there is no way (currently) to specify that either a task or a process is associated with a resource pool. I can't really think of any reason why you would ever want such fine-grained resource management that you would want to have a configuration that specified this relationship; so this is unlikely to ever change.

If you want to permanently change the binding of a project or zone to a resource pool then you must edit the configuration (favourite editor for /etc/project(4) or zonecfg(1M) for a zone). If you want the change to take place immediately, rather than the next time you boot your OS, then you must also use poolbind(1M) to execute the temporal change immediately.

So what actually happens when you use poolbind(1M) to modify the binding of a resource pool bind-able entity?
process The process stops consuming the resources associated with its existing resource pool and begins consuming the resources associated with the new resource pool. Any new processes which fork(2) from this process are also bound to the new resource pool.
task All processes which belong to the task stop consuming the resources associated with their existing resource pool and begin consuming the resources associated with the new resource pool. Any new processes which fork(2) from one of these processes are also bound to the new resource pool.
project All processes which belong to the project stop consuming the resources associated with their existing resource pool and begin consuming the resources associated with the new resource pool. Any new processes which fork(2) from one of these processes are also bound to the new resource pool. However, new processes which join the project after this operation completes will still be associated with the configured resource pool.
zone All processes which belong to the zone stop consuming the resources associated with their existing resource pool and begin consuming the resources associated with the new resource pool. Any new processes which fork(2) from one of these processes are also bound to the new resource pool. Re-booting the zone will restore the configured resource pool binding.

poolbind(1M) is a very powerful workload managment command, especially when applied to a process collective such as a zone or a project. I'll return to the subject of poolbind(1M) later and give some examples of how it can be used to manage workloads quickly and flexibly when problems occur.

Thursday Jul 08, 2004

Event Ports

poll(2) is a very handy system call, but have you ever wanted to extend the types of events that you can respond to as well as being able to multiplex events from disjoint sources?

Of course you have! This is just the kind of facility you need to help you write scalable, efficient multithreaded software.

Solaris 10 introduces the new event ports facility designed to make this simpler.

If you are interested in finding out more about them, take a look at the port_get(3c) manpage. There is a short example which illustrates their power. Alternatively, if you haven't got access to Solaris 10, you can look at this.

Wednesday Jun 23, 2004

Solaris 10 gets an XML tools upgrade

I look after the version of libxml2 and libxslt which ship with Solaris. Don't ask me why, it's got nothing to do with kernel development. An accident of history I suppose since many years ago in previous employment I used to use XML's ancestor (SGML) on a variety of projects. Be careful what you do, it only qualifies you to do more of it in the future!!

libxml2 and libxslt along with their associated tools xmllint, xmlcatalog and xsltproc are shipped as part of Solaris 10 and occasionally the versions of these tools are upgraded. Usually the upgrade is triggered by a specific project which needs new functionality or a specific bug fix. In this case it's both, since a security fix was provided in libxml2 2.6.6 and the GNOME project requires new functionality from both lthe libxml2 and libxslt libraries.

As of today, Solaris 10 incorporates version 2.6.10 of libxml2 and version 1.1.7 of libxslt.

If you have a requirement to manipulate XML on Solaris then you should definitely investigate the use of these tools and libraries since their performance is good, especially when compared to their Java based equivalents. You can find out more about them here.

Thursday Jun 10, 2004

Transforming a Resource Pools Configuration XML Document using XSL

Isn't XML wonderful? No! Well maybe it isn't, but it certainly is useful.

Solaris Resource Pools utilities, pooladm(1M) poolcfg(1M), can be used to examine a Resource Pools configuration. Of course, like most text oriented utilities, the output of these commands is designed to be read by a human working with a shell. I thought it would be nice to be able to examine the Resource Pools configuration using a web browser.

In order to do this you need a few things:

  • A tool to transform your XML document into HTML
  • A web server that can perform XSLT transformations
  • An XSL stylesheet which describes the transformation

In Solaris 10, the xsltproc(1) utility can be used to transform XML documents using XSL stylesheets. If you don't have Solaris 10, think about joining the Solaris Express program and getting early access to Solaris 10. Alternatively, you can download xsltproc as part of the libxslt package for Solaris 9 here.

I've written a cgi-bin script which can be used to invoke xsltproc to perform transformations on input XML documents. Of course, you don't have to use xsltproc, many web servers have extensions that can perform XSLT transformations based on the document type. The virtue of this approach is that it is a "lowest common denominator" which doesn't require any web server configuration or addition module installation. This script relies on a utility program called pooldump which is basically the program from my earlier post about dumping a pools configuration as an XML document. The script, xslt_proc, needs a little customization. Instructions are included at the top of the file.

Finally, you need a stylesheet, pool_html.xsl, which will perform the transformation. I don't claim to be an XSLT expert, so please be gentle with any suggestions for improvements that you might want to make. It's good enough for my needs and I hope that you might find it useful.I keep this in my cgi-bin directory so that I don't need to set an explicit path to it in the xslt_proc script, but you can put it anywhere you like as long as it can be accessed when your cgi-bin script runs.

You can download the cgi-bin script and stylesheet from here.

If you want to see what the HTML output looks like before spending any time on this, here is a sample page that I generated on my workstation.

Generating an XML Document from a Resource Pools Configuration

XML is a convenient representation format for small quantities of data. There are many tools for manipulating XML documents and a lot of people are now familiar enough with the structure of XML to feel comfortable with this type of representation. Especially in preference to yet another problem domain specific data representation format (YAPDSDRF?).

libpool(3LIB) provides an API for manipulating Resource Pools configurations. In particular, the pool_conf_export(3POOL) function may be used to create a file containing the XML representation of a configuration.

pool_conf_export(3POOL) takes three arguments:

 conf - A valid pools configuration pointer
 location - The pathname you wish to write the configuration into
 format - This must be POX_NATIVE which is the native pools representation format 

It's not immediately obvious (and I say this with some shame since I defined this function) that POX_NATIVE == XML. Once you know that, it's fairly obvious how a program to dump a resource pools configuration in XML format can be written.

One further tip, you can pass "-" as the location and the output will be directed to stdout.

I'm sure that I don't need to say that the following program is completely unsupported and used at your peril, but I thought I would say it just to make it clear.


#include <pool.h>

/\* ARGSUSED \*/
int
main(int argc, char \*\*argv)
{
        pool_conf_t \*conf;

        if ((conf = pool_conf_alloc()) == NULL) {
                return (2);
        }
        if (pool_conf_open(conf, pool_dynamic_location(), PO_RDONLY) !=
            PO_SUCCESS) {
                pool_conf_free(conf);
                return (2);
        }

        if (pool_conf_export(conf, "-", POX_NATIVE) != PO_SUCCESS) {
                (void) pool_conf_close(conf);
                pool_conf_free(conf);
                return (2);
        }
        (void) pool_conf_close(conf);
        pool_conf_free(conf);
        return (0);
}

Once you have your configuration as an XML document you can manipulate it easily using a wide range of tools. In a later post, I'll show how you can use an XSL stylesheet to transform the XML document into an HTML document using the xsltproc(1) utility.

Wednesday Jun 09, 2004

Introduction

I thought I'd better say a few words about who I am to get this ball rolling...

I'm a Staff Engineer and I work in the Solaris Engineering Organisation. I joined Sun in 1995 and I've worked in the kernel group since 2000.

I work in the area of Resource Management and I've worked on a number of projects which have enhanced the Resource Management capabilities of Solaris.

I aim to use this blog to write about Resource Management functionality in Solaris. I'd like to help to make it easy for people to use our features and this might be one way of achieving this. I may also put some entries in about other areas of Solaris, since there are so many cool new features in Solaris 10.

About

garypen

Search

Archives
« April 2014
MonTueWedThuFriSatSun
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
    
       
Today