Friday Jun 09, 2006

Irish OpenSolaris User Group Meeting


From: Tim Foster 
Subject: [iosug] 2nd Irish OpenSolaris User Group meeting
To: iosug@yahoogroups.com, ug-discuss 

Hi All,

Just to let you know, we're going to be holding our 2nd user group
meeting on June 14th (that's this coming Wednesday!) in the Sun offices
in Dublin.

It's short notice I know, but we felt we had to do something to coincide
with the upcoming 1-year anniversary of OpenSolaris!

There's more details in a blog post at:

http://blogs.sun.com/roller/page/timf?entry=2nd_irish_opensolaris_user_group

At the meeting we're going to try to give you a quick overview of
everything that's been happening with OpenSolaris over the last year
(which may take a while!), and Mike Byrne has agreed to give us a
fly-through of ZFS : sounds interesting!

We'd love you to come along, just let us know in advance, so we can
issue you with a building-pass for the evening. See you there!

cheers,
tim

Technorati Tag(s) : ,

Wednesday Jun 07, 2006

iprb updates.....


iprb updates, or scratching an itch. I did a putback for the following last week.

  • 5079410 Iprb: request adding device id 1050 in driver_aliases file
  • 6251566 RFE: add driver alias for pci8086,1059
  • 6261230 iprb: pci8086,1068 is needed in iprb driver_aliases
  • 6330971 Request support for Intel ICH7 network controller
  • 6397979 Iprb: Request support for device pci8086,1069

Expect to see the updates in Solaris Express in a few weeks, and in the snv_42 drop onto opensolaris.org sooner than that. This putback adds out of the box nic support for several common desktops and laptops including the Dell GX170LS, Dimension 9100, 5100n, 5100c, and other Dell machines, IBM ThinkCenter A50p, Toshiba Protege 3500 and several Sony Vaio's (VGN-S72PB, VGN-A580, VGN-TX90S). Other machines using the same chipset(s) will also be picked up.

If you have a known device that is working with the iprb driver and is not supported out of the box, please drop me a line (my blogname at sun.com) and I'll get a bug logged about it and work on getting a fix done (call it scratching the itch part two).

Technorati Tag(s) : ,

Thursday Jun 01, 2006

Jumpstart and Zone Creation


One of the many cool features available in Solaris is Zones. As Sean has already blogged automation is the name of the game, and part of this is creating required zones when we jumpstart a machine.

The example here is quite striped down, and creates three very simple zones once a machine is jumpstarted, with very limited configuration - but using zonecfg(1M) and the various resource management utilities you can modify this to suit your needs. This example was tested on the latest Solaris Express.

Anyway, enough talk, you can download the script here, and for your persual...

 

#!/bin/sh
#
# create (very) simple zones after a jumpstart
# 
# this finish script gives an example of how you can create
# some zones post jumpstart. You need to know the name(s) of
# the zones you wish to create, their ip addresses and the 
# network interface you wish to use. It is assumed that netmasks
# have been set up correctly during your install
#
# In the example here we create three zones on a 192.1.1 subnet
# each using bge0 as their interface. 
#
# The script runs as follows
#
#	Section 1 - As a jumpstart finish script
#	-------------------------------------------------
#	1. append details on the zone to your hosts file
#	   where applicable
#	2. create the initial directory for the zone(s)
#	3. create the zonecfg input file
#	4. Create the rc3.d script we will use after reboot
#
#	Section 2 - After initial reboot
#	-------------------------------------------------
#	1. create and install zone(s)
#	2. set zone state to ready
#	3. boot zone
#
#
# Each zone is based on a very simple spare root zone, for further
# details on Zones and the various configuration options consult the
# manpages for zones(5), zoneadm(1M) and zonecfg(1M)
#
# Additional documentation on zones along with tips and articles
# can be found in the Zones section of BigAdmin at
# http://www.sun.com/bigadmin/content/zones/
#
# ZONE_DETAILS - colon seperated listed of zones, containing
#		 zone-name:ip address:interface to use
#

ZONE_DETAILS="test-js-zone-1:192.1.1.1:bge0
test-js-zone-2:192.1.1.2:bge0
test-jes-zone-3:192.1.1.3:bge0"

# ZONE_BASE - root directory for your zones to be created in
#
 
ZONE_BASE="/export/zones"

appendToHostFile() {

	LZONE_NAME=$1
	LZONE_IP=$2

	if [ ! -f $MOUNT_PREFIX/etc/hosts.prezone ]
	then
		cp $MOUNT_PREFIX/etc/hosts $MOUNT_PREFIX/etc/hosts.prezone
	fi

	grep -v $LZONE_NAME $MOUNT_PREFIX/etc/hosts > /tmp/hosts 
	echo "$LZONE_IP	$LZONE_NAME" >> /tmp/hosts
	mv /tmp/hosts $MOUNT_PREFIX/etc/hosts
}

createZoneCfg() {

	LZONE_NAME=$1
	LZONE_IP=$2
	LZONE_INTERFACE=$3

	cat >$MOUNT_PREFIX/$ZONE_BASE/zonecfg/$LZONE_NAME.zcf<<EOF_zonecfg
create -b
set zonepath=$ZONE_BASE/$LZONE_NAME
set autoboot=true
add inherit-pkg-dir
set dir=/lib
end
add inherit-pkg-dir
set dir=/platform
end
add inherit-pkg-dir
set dir=/sbin
end
add inherit-pkg-dir
set dir=/usr
end
add net
set address=$LZONE_IP
set physical=$LZONE_INTERFACE
end
verify
commit
EOF_zonecfg
}

createZoneDirs() {
	
	LZONE_NAME=$1

	if [ ! -d $ZONE_BASE/$LZONE_NAME ]
	then
		mkdir -p $MOUNT_PREFIX/$ZONE_BASE/$LZONE_NAME
		chmod 700 $MOUNT_PREFIX/$ZONE_BASE/$LZONE_NAME
	fi
}

if [ -d /a ]
then
	MOUNT_PREFIX=/a
else
	MOUNT_PREFIX=/
fi

if [ ! -d $MOUNT_PREFIX/$ZONE_BASE/zonecfg ]
then
	mkdir -p $MOUNT_PREFIX/$ZONE_BASE/zonecfg
fi

cat > $MOUNT_PREFIX/etc/rc3.d/S99zonejumpstartexample <<EOF_zonerc
#!/bin/sh

EOF_zonerc

for i in $ZONE_DETAILS
do
	ZONE_NAME=`echo $i | cut -d":" -f1`
	ZONE_IP=`echo $i | cut -d":" -f2`
	ZONE_INTERFACE=`echo $i | cut -d":" -f3`

	createZoneDirs $ZONE_NAME
	createZoneCfg $ZONE_NAME $ZONE_IP $ZONE_INTERFACE
	appendToHostFile $ZONE_NAME $ZONE_IP

cat >> $MOUNT_PREFIX/etc/rc3.d/S99zonejumpstartexample <<EOF_zonerc

/usr/sbin/zonecfg -z $ZONE_NAME -f $ZONE_BASE/zonecfg/$ZONE_NAME.zcf
/usr/sbin/zoneadm -z $ZONE_NAME install
rm -f $ZONE_BASE/$ZONE_NAME/root/etc/.UNCONFIGURED
touch $ZONE_BASE/$ZONE_NAME/root/etc/.NFS4inst_state.domain
/usr/sbin/zoneadm -z $ZONE_NAME ready
/usr/sbin/zoneadm -z $ZONE_NAME boot

EOF_zonerc

done

cat >> $MOUNT_PREFIX/etc/rc3.d/S99zonejumpstartexample <<EOF_zonerc

rm /etc/rc3.d/S99zonejumpstartexample
EOF_zonerc

chmod +x $MOUNT_PREFIX/etc/rc3.d/S99zonejumpstartexample

One caveat, to set the root password do

bash-3.00# zlogin test-js-zone-2
[Connected to zone 'test-js-zone-2' pts/51]
# passwd -r files root
New Password:
Re-enter new Password:
passwd: password successfully changed for root

Technorati Tag(s) : ,

Thursday May 04, 2006

Sun's Performance Lifestyle - PerfPit, Non Debug BFU's and what happens


Introduction

Among the the services that my group, Performance QA, provides for the Solaris development community, two services tend to be highlighted as extremely important, PerfPIT and Performance SelfTest. I have already mentioned both of these services in a previous post Enabling Suns Performance Lifestyle.

Both PerfPIT and Performance Selftest use the same underlying mechanisms to evaluate the impact on overall performance of a developers changes, so lets go through the process in a bit more detail.

Background Knowledge

The process for building Solaris is pretty straight forward, and is documented in detail in the Building OpenSolaris section of the developer reference on opensolaris.org. One of the possible images that can be created during a build is a BFU Archive, which is generated when you set the "-a" flag in your nightly options.

Personally I use the following options in nightly (your mileage may vary though).

NIGHTLY_OPTIONS="-MNazmni";             export NIGHTLY_OPTIONS

Oh, you might have noted the -z option above, we like the -z option, everyone should like the -z option. Why? It gzips everything for you. Which of course means that moving the BFU archives around is much, much quicker.

A BFU, aka Blindingly Fast Update, aka Bonwick Faulkner Upgrade, archive is used to upgrade Solaris to a newer rev without completely reinstalling the system. Further details on using BFU to install a new image can be found at here. Now with the backround reading out of the way.....

Prequisites

At a very high level a developer needs to provide
  • Non debug baseline BFU archives
  • Non debug BFU archives with just the developers changes
  • A Solaris Express (Nevada) build number to run these changes on
to use both PerfPIT and Performance Selftest.

Firstly the non debug BFU archives are extremely important, performance runs with debugging enabled have pretty much zero usefullness (and thanks to DTrace no one ever needs to use a debug build to try and narrow down a performance problem on Solaris again). To this end we automatically reject non-debug BFU archives. As an example of how detrimental the impact of debugging can be on performance, heres a small extract from a table of results comparing a debug BFU archive against a recent non debug BFU archive on top of a Solaris Express install (by way of explanation this was a BFU that I ran manually by mistake, and it didn't get the "is it a debug BFU" check).

System Benchmark Test Bfu Archive(debug) Base Bfu Archive %Standard Deviation %Change
SunFire V240 2 x 1002MHz am_thread_2500_100 312.60 123.22 0.34% -153.70
am_thread_2500_150 471.23 185.52 0.82% -154.00
am_thread_2500_200 754.06 247.12 0.31% -205.14

Not quite what you want to see on a Monday morning ;).

The Solaris Express build requested is quite important as well, generally people use the latest, greatest, released bits. You know that it just makes sense.

Building The BFU Archives

We generally ask that a developer builds his/her baseline and test BFU on the same machine. There are multiple reasons for this, but primarily it is to ensure that the only delta which exists between the baseline and test BFU archives is the actual code changes. Historically we have seen phantom performance degradations in our upstreadm testing which were resolved to issues such as what compiler was used, the libc version on build system etc. Building both BFU archives on the same system just saves time all round.

So the process is

  • Create your workspace
  • Build your baseline non debug BFU archives
  • Apply your changes
  • Build the BFU archives with your changes
  • Install the BFU archives on a machine to ensure that they don't create warm bricks...
    For this just install the baseline BFU archive first, make sure it boots, and then install your changed BFU archives.
    This is just a sanity test, but again it saves time in the long run.

The Benchmarking Process

Okay, so someone has got this far, BFU's are ready to go, what happens then? The BFU archives are brought into our lab (firewalled away to remove any unwanted variables from the benchmarking process) and the runs are scheduled. Once a run starts, we go through the following steps.
  • 1. Install requested machine(s) with the Solaris Express version requested
  • 2. BFU the machine with the baseline BFU archives
  • 3. Execute the benchmarks and collect the results
  • 4. Reinstall the machine (to ensure a completely clean run)
  • 5. BFU the machine with the test BFU archives
  • 6. Execute the benchmarks and collect the results
And then we gather all the results and do some analysis on them, this is completely automatic unless something is out of reasonable bounds for improvement or degradation. In that case the suspect results are flagged and manually reviewed. Finally we send the results back to the developer and all things going according to plan everything is green with big plus signs in front of four digit percentage gains (okay I did say according to plan, but it would be a plan that finished with one of Nialls favourite lines - "all goals met, all pigs watered and ready to fly". Realistically though gains in the high double digits have been seen with certain projects, and small incremental gains are continously seen).
Technorati Tag(s) : ,

Wednesday Mar 01, 2006

Benchmarking Outing Redux


Date: Tue, 28 Feb 2006 18:40:48 +0000
From: Fintan Ryan 
Subject: [osol-testing] Performance Benchmark List
To: testing-discuss@opensolaris.org
Cc: perf-discuss@opensolaris.org

Hi,

We just updated the list of performance benchmarks for possible use
with OpenSolaris. These benchmarks will also be available for the
community to use in the upcoming Self Test facility.

Futher details are available at

http://www.opensolaris.org/os/community/testing/perfbench/

Feedback appreciated.

- Fintan

Technorati Tag(s) : ,

Wednesday Feb 15, 2006

Deadman Walking


Within Sun we consider the concept of observability to be of the utmost importance, as can be seen by the plethora of tools bundled within OpenSolaris and noted in the Observability Community on opensolaris.org. This belief is best summed up on the observability page with the phrase "If you cannot see the problem, you cannot fix it".

This belief in observability extends to postmortem analysis as well (see dumpadm(1M)). Unfortunately at times development builds can "just hang" [tm], living you in somewhat of quandry as to what you should do. Thankfully Solaris provides a way to still get a crashdump from this kind of situation via Deadman.

Enabling Deadman

To enable deadman first ensure you are capturing crash images with dumpadm(1M) and then just add
set snooping=1
to /etc/system. Note that any zones on your system will inherit the deadman setting as well.

Slightly More Gory Details

Deadman is a high level cyclic (see the cyclics subsystem), which monitors the kernel lbolt variable, which in turn is incremented everytime the clock interrupt fires. The deadman code is executed once a second, and once it notices that lbolt hasn't been incremented in snoop_interval / MICROSEC times (set up in deadman_init) it will cause a panic.
Technorati Tag(s) : ,

Tuesday Jan 31, 2006

Enabling Cron Logs

Quick one due to a question from a friend. To enable a log of all actions by cron edit /etc/default/cron and set CRONGLOG=YES. Be carefull though, it will generate a very, very big log quite quickly.

Thursday Jan 12, 2006

OpenSolaris Sysadmin Community


Octave Orgeron announced the creation of the OpenSolaris Sysadmin community yesterday. The thread which lead to this community is viewable on the osol-discuss forum.
Technorati Tag(s) : ,

Monday Jan 09, 2006

Quick GTar tip


Quick tip, if you have a tar archive that has an absolute path in it that you want to, or more likely have to, avoid creating, you can unarchive it using the --no-anchored flag in gtar.

As an example, this morning I was extracting files from an, ahem, unusually created archive, that contained paths such as

/net/foo.bardomain/export/file/is/in/here.gz
which in a firewalled lab is not going to be much use to anyone ;). One of my colleagues, Nicky, suggested taking a look at gtar. So the solution was to use gtar (bundled in /usr/sfw/bin in Solaris 10).
# /usr/sfw/bin/gtar --no-anchored -xvf foo.tar

Technorati Tag(s) : ,

Tuesday Nov 15, 2005

Centrino support in OpenSolaris


As noted already by Andrei, and announced on the laptop-discuss alias, two more drivers providing support for Intel Pro Wireless 2100, 2200BG and 2915ABG chipsets. There's more info (and of course the downloads) over on the wireless community. A lot of us have been using these drivers internally for quite some time (indeed I'm writing this blog using it), so its nice to see them outside.
Technorati Tag(s) : ,

Thursday Oct 13, 2005

New OpenSolaris Distros


Just saw this over on Eric Boutilier's blog, Alex Ross has announced that a Debian based GNU/Solaris distro is on the way. I rarely ever use Linux these days, but as far as distros go I do have a softspot for Debian, can't wait to try this out. Its great to see this, and the two previous distros, SchilliX and BeleniX coming out.
Technorati Tag(s) :

Tuesday Oct 04, 2005

Saving Trees.....


Very quick one. a2ps and friends are bundled in /usr/sfw/bin in Solaris 10, and it is an incredibly handy command for printing documentation. I was espousing its virtues to one of my non blogging heathen (to borrow a phrase from Mr. Clingan) colleagues who tends to print out a lot of things yesterday.

Anyway the easiest way to use it, say for printing out a pdf document with say four pages per page is to just set your PRINTER environment variable, and run, so we get something like

[fintanr@tiresias bin] $ export PRINTER=myprinter
[fintanr@tiresias bin] $ a2ps -4 /tmp/819-2660.pdf
You can of course setup your printers.conf file or your .printers file in your home directory to set your default printer, but I'll leave that as an exercise for the reader.

Monday Oct 03, 2005

compiling mp3splt on Solaris


Before posting the mp3 of the first IOSUG meeting I had to compile up mp3splt in order to split up the mp3 into something a bit more usable. There are one or two little caveats for doing this on Solaris, so for the purposes of documentation these are written up below.

The compilation in this example does not include support for ogg-vorbis, but if people want I'll post an example of this as well. I grabbed libmad from blastwave.org. The short version is here first, and then an explanation of possible errors you may see when trying to compile this or similar software.

Short Version

If you don't have /usr/sfw/bin and /usr/ccs/bin in your path you will need to add them in.
# /opt/csw/bin/pkg-get -i libmad
# LDFLAGS="$LDFLAGS -L/opt/csw/lib -R/opt/csw/lib"; export LDFLAGS
# CFLAGS="-lresolv -lnsl -lsocket"; export CFLAGS
# CPPFLAGS="-I/opt/csw/include -I/usr/include"
# cd ${mp3splt_src}
# ./configure --disable-ogg
# make; make install

Possible Errors

And thats it. Now with a few errors. The first possible error one will encounter is missing header files. In this case libmad is going to be your most likely issue, and you will get errors similar to
[fintanr@charlico mp3splt-2.1c] $ make
make  all-recursive
Making all in doc
if gcc -DHAVE_CONFIG_H       -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -DNO_OGG -MT \\ 
   mp3splt.o -MD -MP -MF ".deps/mp3splt.Tpo" \\
  -c -o mp3splt.o `test -f 'mp3splt.c' || echo './'`mp3splt.c; \\
then mv -f ".deps/mp3splt.Tpo" ".deps/mp3splt.Po"; \\
else rm -f ".deps/mp3splt.Tpo"; exit 1; \\
fi
In file included from mp3splt.c:43:
mp3.h:40:17: mad.h: No such file or directory
In file included from mp3splt.c:43:
mp3.h:99: error: field `stream' has incomplete type
mp3.h:100: error: field `frame' has incomplete type
This is resolved with the CPPFLAGS setting above. Next up, and much more common is when you see an error similar to
gcc  -D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 -DNO_OGG  -L/usr/local/lib -R/usr/local/lib \\ 
-L/opt/csw/lib -o mp3splt  mp3splt.o splt.o mp3.o  wrap.o cddb.o ogg.o  -lmad  -lm
Undefined                       first referenced
 symbol                             in file
recv                                cddb.o
send                                cddb.o
herror                              cddb.o
gethostbyname                       cddb.o
socket                              cddb.o
connect                             cddb.o
ld: fatal: Symbol referencing errors. No output written to mp3splt
collect2: ld returned 1 exit status
These errors are all due to missing symbols from the networking libs, so the CFLAGS entry above passes in the correct libs.
Technorati Tag(s) :

IOSUG - mp3 of Darren Moffats Presentation


The audio track for Darren Moffats presentation on OpenSolaris for Security Geeks at the inaugural IOSUG meeting is now available. Total running time is 41 mins, and its available for download from mediacast.sun.com. Thanks to Mike McHugh for making these available for us.
Technorati Tag(s) :

Wednesday Sep 28, 2005

IOSUG bringup


Title gratuitously borrowed from Tim's post. The Irish OpenSolaris Users Group had it's first meet up last night. Darren Moffat gave an excellent, and nicely titled presentation, OpenSolaris for security geeks, which quite succintly explained many of the new security concepts in Solaris 10, particuarly items such as privileges which have come in from Trusted Solaris.

Mike McHugh from DCU (kinda rhymes doesn't it ;) ), videotaped the presentation, and will be posting both a video and audio track later this week, details to follow. Thanks to everyone that attended, and again thanks to the folks in DIT Netsoc for hosting us. We will give slightly more notice for the next event (hopefully we will confirm a date by mid next week).

Technorati Tag(s) :
About

fintanr

Search

Archives
« April 2014
MonTueWedThuFriSatSun
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
    
       
Today