Tuesday Sep 12, 2006

Building Opensolaris

This weekend I did something I've been meaning to do for a while. I've been putting it off due to lack of time to think about how to approach it without breaking anything.

Any way I finally tried downloading and installing Solaris Express the community edition (build 46) and downloaded all the build tools. I was amazed how easy it was. Within an hour offinishing the downloads, it was building opensolaris.

The main shock for me was the lack of an SCM (Source Code Management) system. Being fully entrenched in the world of ON (Os and Networking) for the last 10 years Teamware (the SCM we use) is just \*what we do\*. So I had to rethink how I'll manage the build. But then using opensolaris.sh from usr/src/tools/env did a good job.

So I know have my own build built from opensolaris running on my system, without anything from within Sun. Cool. Give it a go, it was easy and gives you the chance to play with how things work.

Next step for me - build the Xen bits from outside Sun :)

==== Nightly distributed build started: Mon Sep 11 11:27:18 BST 2006 ====
==== Nightly distributed build completed: Tue Sep 12 03:22:02 BST 2006 ====

==== Total build time ====

real 15:54:44

==== Nightly argument issues ====

Warning: the N option (do not run protocmp) is set; it probably shouldn't be

==== Build environment ====

SunOS osol-bld 5.11 snv_46 i86pc i386 i86pc

/opt/onbld/bin/nightly myopensolaris.sh
nightly.sh version 1.104 2006/08/29

dmake: Sun Distributed Make 7.7 2005/10/13
number of concurrent jobs = 4

32-bit compiler
/opt/onbld/bin/i386/cw -_cc
cw version 1.20
primary: /opt/SUNWspro/bin/cc
cc: Sun C 5.8 Patch 121016-02 2006/03/31
shadow: /usr/sfw/bin/gcc
gcc (GCC) 3.4.3 (csl-sol210-3_4-20050802)

64-bit compiler
/opt/onbld/bin/i386/cw -_cc
cw version 1.20
primary: /opt/SUNWspro/bin/cc
cc: Sun C 5.8 Patch 121016-02 2006/03/31
shadow: /usr/sfw/bin/gcc
gcc (GCC) 3.4.3 (csl-sol210-3_4-20050802)

java full version "1.5.0_08-b03"

as: Sun Compiler Common 10 snv_46 08/03/2006

ld: Software Generation Utilities - Solaris Link Editors: 5.11-1.545

Build project: group.staff
Build taskid: 62

==== Build version ====


==== Make clobber ERRORS ====

==== Make tools clobber ERRORS ====

==== Tools build errors ====

==== SCCS Noise (DEBUG) ====

==== Build errors (DEBUG) ====

==== Build warnings (DEBUG) ====

==== Elapsed build time (DEBUG) ====

real 11:42:57.3
user 4:41:00.3
sys 2:40:32.5

Technorati Tags:
Solaris OpenSolaris Xen

Saturday Jul 29, 2006

And back again

Well The ride back yesterday took longer. 65:15. But then I rode back a very different way. One of my colleagues was riding as well so we went back via his house ~10 miles. Practicing drafting, which makes things much faster and easier. Only trouble was that he lives in Wokingham so I had to get back from there. So I started coming the direct route along Reading road, and quickly got fed up with that, so at Winersh headed in to Woodley and back through Sonning. Though this added a couple of miles it was probably nicer riding. So 19 miles in 65 minutes isn't so bad. Though I felt really tired afterwards, and hungry, and hot.... Well that makes 90 miles this week. Most I've ever managed. I'll have to see if I can do it next week

Friday Jul 28, 2006


So cycled in today in 57:07. Must have been because I was happier in the traffic rather than riding faster.

Wednesday Jul 26, 2006

And home again

So the journey home wasn't too bad either. I measured from the cycle lockers home rather than the entrance to the campus (the speedbumps in the campus are so vicious you cannot get a decent speed up). So it was a bit further. 17.4 miles in 63:37. So averaging 16.36 mph. Could have done better really but The bit between shinfield and the M4 on the A327 was up hill and I felt my legs had no energy. (not surprising really)

Rode in to work this morning

I decided to try cycling to work. It's a 20 mile drive that takes around 45-50 minutes due to the traffic. Going by bike I avoid some of the faster rodes and take the more direct route which is almost exactly 17 miles to the entrance of the campus. So I checked mail from home and set off after that. Did the 17 miles in 59:09. Even taking a shower in to account that's not much slower than driving. Any way - I've just got to do the same on the way home now!

Sunday Jul 23, 2006

What am I listening to

Last week I was visiting our colleagues in Prague. Talking about Xen, Dtrace, FMA and other cool stuff, but also helping out with some problems they were having. Anyway I did have a nightmare journey back and was thankfull of two things. A good book (Use of Weapons - Iain M Banks: I love the detail he puts in the worlds he creates), and an IPod.

So what did I listen too? Well I found I had subscribed to a podcast that has some excellent music on it. IndustrialRadio. Plenty of interesting music I'd not heard before. Apprarently the detroit industrial scene is pretty active, which is more than can be said for most places.

Any way back in the real world, I rediscovered I like some Handel, Having bought Essential Handel I've remembered learning Violin as a child. May be I should take up some more.

What else. Well in complete contrast there's been Classic Euphoria Which is pretty much audio wall paper but great for getting stuff done too.

My eldest Son did a part in the school production of Oliver a couple of weeks a go. Today there was a local schools show case of the out of school activities that they've been doing. Naturally eldest was the best, but one other that stood out was another schools "Rock School". They have taken "Feel Good Inc" by Gorillaz and re recorded it using the instruments they can play (The group only sang to a backing track which I guess is fair enough given the size of the venue, and the fact they had no switch over time, Oh and hey're only 8-11). I only realised it wasn't to the real track when I heard recorders on it! Pretty impressive.

And this evening while watching the end of The Tour They had a butchered version of One Which made me put on the original which was much better.

Monday Jul 17, 2006

Xen for dummies - part1

I've been helping out the Xen team to try out the bits they've put on OpenSolaris over the last week or two. I've been impressed with how much they've got working so far, but as an experienced Solaris user who is new to Xen I've found it quite hard to get my head around what Xen does and how it works.

First off install it using the instructions on OpenSolaris and then you need to get some domains set up.

So boot up under Xen. You'll see from the grub menu (/boot/grub/menu.lst) some things have changed. Instead of booting a Solaris kernel, you boot xen which then loads a solaris kernel in the module line.

#Solaris on Xen 64bit
title Solaris on Xen 64-bit
kernel /boot/amd64/xen.gz dom0_mem=524288 console=com1 com1=9600,8n1
module /platform/i86xen/kernel/amd64/unix /platform/i86xen/kernel/amd64/unix -k
module /platform/i86pc/boot_archive

Note the option dom0_mem=524288. This assigne 512Mb to your Dom0 at startup.

Another thing to note is that even booting on metal (ie not booting the xen hypervisor first) we no longer use multiboot, but can boot our unix directley.

#---------- ADDED BY BOOTADM - DO NOT EDIT ----------
title Solaris Nevada snv_41 X86
kernel /platform/i86pc/kernel/amd64/unix
module /platform/i86pc/boot_archive
#---------------------END BOOTADM--------------------

Read Joe Bonasera's blog about why this is the case.

OK so you've got Solaris booted on Xen. This is refered to as Dom0 you need to set up some more instances of Solaris (or whatever Xen compatible OS you happen to want) and that is refered to as a DomU.

Each DomU has it's own full OS install so for solaris we set up a flar archive and use the vbdcfg script to help us convert that in to a OS instance Xen can boot as a DomU. This is all described here.

A couple of things to think about. Networking is bridged (ie it appears to be directly connected to the outside world) so you're going to need to give it a real IP address or use DHCP (this can be decided as the DomU boots up). Also you need to give it an ethernet address using the -e flag to vbdcfg. As this is a made up ethernet addess I don't know how you're supposed to create it, Dave Edmondson suggested making the first octet 0xaa and encoding the IP address in the rest of it.

So when you've got your DomU setup using vbdcfg what do you get?

Well in /export/xc/xvm you'll have a directory for your DomU domain

$ ls -l /export/xc/xvm/mydomU
total 10
-rw-r--r-- 1 root root 379 Jul 12 09:57 mydomU-64.py
-rw-r--r-- 1 root root 367 Jul 12 09:57 mydomU.py
drwxr-xr-x 4 root root 512 Jul 12 09:48 platform
-rw-r--r-- 1 root root 19 Jul 12 09:48 root.dev
drwxr-xr-x 2 root root 512 Jul 12 09:48 vmnt

the .py files are python scripts used to start the domain, platform is the directory where the kernel to boot is, root.dev contains the name of the root device and vmnt is where the domain can be mounted using

$ vbdcfg mountdomU domU-name

But you don't want to do that now. You want to start up your domain, So this is how you do it

$ xm create mydomU-64.py
(this starts mydomU in 64bit mode)
$ xm console mydomU
(puts you on the console)

or alternatively

$ xm create -c mydomU-64.py

which starts the domU and puts you directly on the console - useful sometimes to find out why your domain isn't coming up

So at that point you on the console of your domain and it's coming up just as if its a fresh install of Solaris.

In future "Xen for Dummies" installments I'll show how to configure the network, add disks, create more cpus in the domU than you have in the real box! and I'm sure I can think of more later.


Technorati Tags: Solaris OpenSolaris Xen

Friday Jul 07, 2006

Action Shot

The new Pup still doesn't seem to stay still , but we can now take him for walks. As I was working from home today that's what we did at lunch time. He's pretty good at coming when called these days

Thursday Jul 06, 2006

Another Dtrace Customer presentation

I had another opportunity to talk to a customer (this time a law enforcement agency) about Dtrace last week. This time it was only a 1/2 hour slot and we had already over run by quite a way so the Presentation was a lot shorter You'll notice the similarity with the previous presentation (well there is no point in reinventing the wheel) In this case the customer was very interested in virtualization. They have a large farm of PCs which are only ~15% utilized. They felt something like Xen or vmware might be able to help bring that up. I'm working with the Xen team right now just getting up to speed on the technology. It looks pretty impressive. Go and check out the Xen Opensolaris community it's pretty interesting now and I'm sure it'll have a load more interesting stuff on it soon Technorati Tags: Solaris OpenSolaris dtrace Xen

Wednesday Jun 21, 2006

Extreme Close Up

The new Pup doesn't seem to stay still long enough for a photo. This is the best we've got this week

Wednesday Jun 14, 2006

dtrace presentation

I had the pleasant opportunity of visiting an exisiting Sun customer last week who was interested in dtrace and containers. I've been using dtrace since before it was in Solaris 10. Using the bits the development team were working on. In my line of work it's a great boon and I think the customer could see the benefits of the tool and appreciated the insights of someone who used it day in day out. Obviously in an hour or so presentation you can't cover anything in great detail, but I wanted to whet their appettites as to things dtrace could do for them. So I walked them through the reasons why it's a good solution, and the architecure. Then a few examples and demos. Then we talked about how it could solve some of the problems they've experienced. Over all a very good experience and I believe they found it useful. Any way I thought I'd put up the presentation here to remind me that I did it and so someone else might find it useful especially in now there is such a community on opensolaris.org Presentation Technorati Tags: Solaris OpenSolaris dtrace

Monday Jun 12, 2006

New Pup

After a 2 month wait our new pup finally arrived, He's a Cocker Spaniel in Chocolate Brown with Tan socks and beard. Very unusual. He settled in pretty well but the last couple of nights have been a bit disturbed. Any way - here are a few photos for good measure

Monday Jun 05, 2006

Disneyland Paris - which land are we in?

Just a quick comic interlude. We had a weekend in Disneyland Paris this weekend. Overall a fantastic trip though very tiring (and I wouldn't bother with Eurostar again unless you get the direct service. The TGV to Lille on the way back was incredibly busy). But it's the little things that stick in your memory. My 5 year old and I were walking from "Fantasy Land" in to "Frountier Land" My youngest is very eloquent for his years and has the usual childs ability to say what he really things. "Daddy, which land are we in now?" "We've just gone in to Frontier Land" "Oh. I thought we were in Fat Land!" "What?" "Well everyone here is fat and we're skinny like Mr Skinny in the book at school" Made me laugh!

Tuesday May 30, 2006

What does ISSIG(FORREAL) do.

After my last blog I thought I could expand a bit on what ISSIG(FORREAL) does. That is: o stop for some other thread to do fork1(); return 0 when it's done. o stop because a job control stopping signal is present; return 0 after the process is continued with SIGCONT. o stop because /proc requested the thread or process to stop; return 0 when /proc continues the thread/process. o stop for watchpoint manipulation; return 0 when that's done. o stop because of a non-blocked pending signal that /proc cares about; return 0 if /proc cancels the signal when continuing the thread. o discard pending but ignored signals; return 0 when that's done. o Otherwise return 1 if there is still a signal or 0 if not. If ISSIG(FORREAL) returns non-zero, then there really is a non-ignored signal present so you really must take appropriate action (usually returning back to userland with EINTR). Technorati Tags: Solaris

Friday May 26, 2006

ISSIG() How do I use it?

As indicated in this blog entry a fix I did for a bug showed up some interesting and rather hard to diagnose problems. These showed up as the getting EACCES (permission denied) errors when trying to cd or read an automounted directory. Now when you use dtrace to work back through what has happened you find that Using truss and dtrace it was possible to see that the program that was getting EACCES was doing an open on the directory which ended up in auto_wait4mount(). It is this call that returns EACCES. At around the same time the automountd process gets an interrupted system call (through a variety of calling paths but usually from nfs4_mount() > eventually calling > nfs4secinfo_otw() > nfs4_rfscall() which returns EINTR indicating it's getting something like a signal. This includes watchpoint activity and fork1 requests. In this case it is a fork1 request so the thread requesting the stop is in the same processes, so ISSIG(JUSTLOOKING) returns true even if lwp_nostop is set. (this was the fix for 4522909). If we look in nfs4_rfscall() at the following section
   1334                 /\*
   1335                  \* If there is a current signal, then don't bother
   1336                  \* even trying to send out the request because we
   1337                  \* won't be able to block waiting for the response.
   1338                  \* Simply assume RPC_INTR and get on with it.
   1339                  \*/
   1340                 if (ttolwp(curthread) != NULL && ISSIG(curthread, JUSTLOOKING))
   1341                         status = RPC_INTR;
   1342                 else {
   1343                         status = CLNT_CALL(client, which, xdrargs, argsp,
   1344                             xdrres, resp, wait);
   1345                 }
Here we look to see if there is a signal pending using ISSIG(curthread, JUSTLOOKING) to optimise out a CLNT_CALL() if its not needed. If so we return RPC_INTR (a little further down). The assumption is that if you have any signal like activity you need to return to userland to handle the signal. This is not the case for fork1() you can simply wait till you start running again and carry on. ISSIG(t, FORREAL) could be used to check if there is a real need to return to userland. The trouble is you need to drop all your locks before calling it. So you then have to reaquire the locks later. This may require you restart the rfscall operation. Also if you do a forkall() (ie a normal fork() systemcall) you do need to return to userland with EINTR, same with some /proc activity. So it's probably worth checking for that prior to calling the issig(FORREAL). A good example of how to do this is in cv_wait_sig(). So an example of how to correctley check for signal delivery in a system call (say if you are going to do something that takes a long time and don'r want to waste that activity if it's going to be interrupted) would be.
 .        if (lwp != NULL &&
 .            (ISSIG(t, JUSTLOOKING) || MUSTRETURN(p, t))) {
 .                ... drop all of your locks! ...
 .                if (ISSIG(t, FORREAL) ||
 .                    lwp->lwp_sysabort ||
 .                    MUSTRETURN(p, t)) {
 .                        lwp->lwp_sysabort = 0;
 .                        return (set_errno(EINTR));
 .                }
 .                return (set_errno(ERESTART));
 .        }
I'll be applying this approach to nfs shortly Technorati Tags: Solaris

Chris W Beal-Oracle


« April 2015