Wednesday Jun 27, 2012

Ein produktives Hobby von mir

Oops, I did it again...


Auch wenn ich seit langer Zeit Projektarbeit im Hardwaregeschäft mache, bin ich doch leidenschaftlicher Softwareentwickler. Meine ersten Programme habe ich 1981 in ein Matheheft geschrieben, noch bevor ich Zugang zu einem Rechner hatte.


Später habe ich einige Programme sogar als Shareware für Geld verkauft: Wer kennt noch ARV, das revolutionäre Dateienverwaltungsprogramm, das Dateien automatisch nach Themen soriert auf Disketten organisiert, oder T-Kal, den einfachen und benutzerfreundlichen Terminkalender?


Alle waren wirtschaftlich weniger erfolgreich, was wenig wundert. Letztendlich waren es Programme, die ich für mich geschrieben hatte, und nur aus Interesse an den betrieblichen und steuerlichen Prozessen, die mit dem Vertrieb verbunden sind, zum Verkauf angeboten habe.


 Nun habe ich es wieder getan. Wer mag, kann sich das Ergebnis unter http://www.dw-aufgaben.de  ansehen.

Friday Mar 05, 2010

Moving x86 Solaris boot disks to another server = hw box (or virtualized environment) - the SVM case


This is just a scratchpad waiting for formatting, please email me if you have any additonal comments.


This was done on Solaris 10 U8.


How would it look like for zfs root? Or for OpenSolaris?


Go!


connect disk obtained by ripping out single boot disk or single disk of mirror out of original server..


boot solaris dvd, choose option 6 "shell"


for slices not svm encapsulated, installer will ask you if you would like to have solaris slice mounted, if not, it will tell you it's svm managed and will not do it by itself. you have to do it by hand. no problem.


on the shell:


export TERM


TERM=at386


Locate boot slice


format


example: c1t0d0s3, replace by actual slice name


mount /dev/dsk/c1t0d0s3 /a


rebuld device tree


mv /a/dev /a/dev.old


mv /a/devices /a/devices.old


tar cpf - dev | ( cd /a; tar xvpf - )


tar cpf - devices | (cd /a;tar xvpf -)


Ajust system file


if root disk was mirrored:


vi /a/etc/system


delete line with rootdev:


Adjust vfstab


vi /etc/vfstab


change entry for / to point to correct slice (the one mounted as /a right now)


if unsure, move swap entries aside


Adjust bootenv.rc


ls -l /dev/rdsk/c1t0d0s3 >>/a/boot/solaris/bootenv.rc


vi  /a/boot/solaris/bootenv.rc


change bootpath to device name appended to this file by ls command, to everything after /devices. use x to delete old path J to join lines and x again to remove surplus ls output.


Adjust grub menu


vi /boot/grub/menu.lst


create entry


title active
root (hd0,0,d)
kernel /platform/i86pc/multiboot
module /platform/i86pc/boot_archive


where hd0=first disk (see bios), 0=first primary partition (see fdisk /dev/rdsk/c1t0d0p0 if necessary)),  a=slice0, b=slice1, c=slice2 d=slice3 etc


Update boot archive


bootadm update-archive -R /a


Repair grub


installgrub /boot/grub/stage1 /boot/grub/stage2 /dev/rdsk/c1t0d0s3


Done


reboot



et voila - should work now


maybe you want to interrupt boot at grub menu, e)dit boot script and add " -s" to "multiboot" line for single user startup.


Some more cleanup to be done...


ethernet interface names


zfs imports


other fs


ip address?



Another way


would be to create a flar archive and do a network based install from that -- but you would have to recreate system identity, ssh keys etc, see sys-unconfig.



Thursday Feb 19, 2009

Atomic flashback

I just experienced a few nostalgic hours, when I tried some of the programs I wrote 25 years ago.


Prelude


Recently, I ran across an ad offering personal computers shipping with FreeDos instead of Windows. I was suprised for the minute it took me to realize that FreeDos obviously was a placeholder, to be replaced by the buyer with the operating system of his choice.


I thought about the last time I booted a computer in dos mode; i could not really remember when, but the last time I used the dos-like command line interface of windows -- that was when I tried an emulator for a computer I used to have -- an Acorn Atom. The were a few video problems, as far as I remember, and suddenly I knew I had to try it again with a plain, real, dos environment.


Virtual DOS


Anybody remember Desqview? Before tools like these existed, you had to set up a seperate pc for every program you wanted to run simultaneously.


I have VirtualBox running on my Solaris based home pc. VirtualBox supports DOS as a guest operating system, so I did not have to set up a separate machine. I just had to download FreeDos -- that was just 8 MB in size! Setting up the virtual machine and installing FreeDos was a matter of under five minutes. How long did it take to set up MS DOS 5 from two 3.5" disks? An hour?


FreeDos comes with CD ROM drivers, doskey and editor preinstalled, so I could get right off mounting the .iso I created from the virtual Tape files I created during my last emulator sessions. The emulator I use is the Acorn Atom Emulator from Wouter Ras. It emulates the computer I did write my first BASIC programs on, the Acorn Atom. While it was a pretty basic system, with only 5K of RAM, limited graphics and almost no sound, it came with a comprehensive book, including an assembler tutorial. That was important to me, as there was no other material available in Germany for that computer (the maximum number of Acorn Atom users I knew of was three). I worked myself through this book, at first skipping the chapter on assembler because it was incomprehensible to me: When I started when I was just twelve years old. But my abilities and understanding grew with every page I read and every example I tried and re-tried, and I did not even give in when quite a few of my friends got C64s from 1983 on and played fast, colourful and noisy games that made the Atom look like a pocket calculator.


Atomic Star Trek


I had to do most of the games all by myself... Of course, Star Trek was one of them. This is a sample screen shot, with my ship ("+") being under attack by a fierce Klingon ship ("1").




It was making heavy use of the BASIC extensions the 4k SUPERBASIC extension ROM by PROGRAM POWER provided.  I remember it had to be activated using LINK 44800 or LINK #AF00...


In the end it's just...


a short trip every time I go back in time. It is nice remembering these days, but there is no way I would sit down in front of that 32x16 text mode display and doing a single line of code again.


These were just "the good old days", where I knew every zero page memory location by name....





Friday Dec 05, 2008

rsync preserving atime of source

Making rsync leave the source file's atime alone

rsync (see http://rsync.samba.org/) is a sophisticated file tree replication tool with great perfomance. 

Some people want to create exact copies of an existing file system, with all file attributes and timestamps preserved. The baseline rsync distribution replicates modification times, but not access times. There is a patch available with the patch collection distributed with rsync that enriches its functionality adding the -U command line switch to make rsync replicate atimes as well.

Still, the source file's access time is changed when rsync reads the file for replication, when the source file has been changed. This may be considered harmful, as rsync should preserve file properties.

There is a solution: Set the replication file system to read-only mode, or better, use a read-only snapshot.

Example:

There are two machines, "here" and "there", with the directory /rsynctest being replicated from here to there.

-bash-3.00#  ls -lu /rsynctest; ssh there ls -lu /rsynctest

total 36
-rw------T   1 root     root        1024 Dec  5 10:18 file-1
-rw------T   1 root     root        1024 Dec  5 10:18 file-2
-rw------T   1 root     root          12 Dec  5 10:26 file-3
-rw------T   1 root     root          12 Dec  5 10:16 file-4
-rw------T   1 root     root        1024 Dec  5 10:16 file-5
-rw------T   1 root     root        1024 Dec  5 10:16 file-6
-rw------T   1 root     root        1024 Dec  5 10:16 file-7
-rw------T   1 root     root        1024 Dec  5 10:16 file-8
-rw------T   1 root     root        1024 Dec  5 10:16 file-9
total 34
-rw------T   1 root     root        1024 Dec  5 10:16 file-1
-rw------T   1 root     root        1024 Dec  5 10:18 file-2
-rw------T   1 root     root          12 Dec  5 10:26 file-3
-rw------T   1 root     root          12 Dec  5 10:16 file-4
-rw------T   1 root     root        1024 Dec  5 10:16 file-5
-rw------T   1 root     root        1024 Dec  5 10:16 file-6
-rw------T   1 root     root        1024 Dec  5 10:16 file-7

ls -u lists access times instead of modification times.
Both file sets are identical. Now lets change a file:

-bash-3.00# echo "new content" >/rsynctest/file-5
total 36
-rw------T   1 root     root        1024 Dec  5 10:26 file-1
-rw------T   1 root     root        1024 Dec  5 10:26 file-2
-rw------T   1 root     root          12 Dec  5 10:26 file-3
-rw------T   1 root     root          12 Dec  5 10:16 file-4
-rw------T   1 root     root          12 Dec  5 10:16 file-5
-rw------T   1 root     root        1024 Dec  5 10:16 file-6
-rw------T   1 root     root        1024 Dec  5 10:16 file-7
-rw------T   1 root     root        1024 Dec  5 10:16 file-8
-rw------T   1 root     root        1024 Dec  5 10:16 file-9
total 34
-rw------T   1 root     root        1024 Dec  5 10:16 file-1
-rw------T   1 root     root          12 Dec  5 10:18 file-2
-rw------T   1 root     root          12 Dec  5 10:26 file-3
-rw------T   1 root     root        1024 Dec  5 10:16 file-4
-rw------T   1 root     root        1024 Dec  5 10:16 file-5
-rw------T   1 root     root        1024 Dec  5 10:16 file-6
-rw------T   1 root     root        1024 Dec  5 10:16 file-7
-rw------T   1 root     root        1024 Dec  5 10:16 file-8
-rw------T   1 root     root        1024 Dec  5 10:16 file-9

Note the access time did not change when updating the file. To keep rsync from changing atime when reading file-5 to send a copy to "there", lets snapshot the file system.

-bash-3.00# zfs snapshot rsynctest@atimesaver

do sync from "here" to "there":

-bash-3.00#  /usr/local/bin/rsync  -rsync-path /usr/local/bin/rsync --delete \\
--archive -U /rsynctest/.zfs/snapshot/atimesaver/ root@there:/rsynctest/

And voila, atime still intact:

-bash-3.00#  ls -lu /rsynctest; ssh there ls -lu /rsynctest
total 36
-rw------T   1 root     root        1024 Dec  5 10:16 file-1
-rw------T   1 root     root        1024 Dec  5 10:18 file-2
-rw------T   1 root     root          12 Dec  5 10:26 file-3
-rw------T   1 root     root          12 Dec  5 10:16 file-4
-rw------T   1 root     root          12 Dec  5 10:16 file-5
-rw------T   1 root     root        1024 Dec  5 10:16 file-6
-rw------T   1 root     root        1024 Dec  5 10:16 file-7
-rw------T   1 root     root        1024 Dec  5 10:16 file-8
-rw------T   1 root     root        1024 Dec  5 10:16 file-9
total 31
-rw------T   1 root     root        1024 Dec  5 10:16 file-1
-rw------T   1 root     root        1024 Dec  5 10:18 file-2
-rw------T   1 root     root          12 Dec  5 10:26 file-3
-rw------T   1 root     root          12 Dec  5 10:16 file-4
-rw------T   1 root     root          12 Dec  5 10:16 file-5
-rw------T   1 root     root        1024 Dec  5 10:16 file-6
-rw------T   1 root     root        1024 Dec  5 10:16 file-7
-rw------T   1 root     root        1024 Dec  5 10:16 file-8
-rw------T   1 root     root        1024 Dec  5 10:16 file-9

Now we can get rid of that snapshot:

-bash-3.00#  zfs destroy rsynctest@atimesaver

Note: ctime (time of last change to the inode entry) is still not replicated. Does anybody know about a fix? Then again, I can not even think of anybody/any program who needs ctimes...

Monday Apr 21, 2008

Copy and Paste with The Gimp and Star Office

This is just a short note, in case no one noticed. Or this is just the hint you were looking for, just a I was yesterday.

Copy and paste of a selection from gimp into Star Office works with The Gimp 2.2 and newer.

Well this is really old news, and it is documented in the gimp docs, see http://docs.gimp.org/en/gimp-introduction-history-2-2.html.

I just had to look that up as my wife wondered why it did not just work with The Gimp 2.2, which comes preinstalled with Solaris. Blastwave has a prepackaged 2.4.3 for you.

As an administrator for a Solaris home production system, you sometimes have to face these challenges...
 


 

Wednesday Jan 02, 2008

Sending SMS text messages

Sending birthday reminders by SMS

Since I now have a system that is running 24h again, I remembered that I once had my Linux server send me reminders to my mobile phone via modem.

On Linux, I used yaps ("Yet another pager software") to send sms text messages. During the christmas holidays, I set up yaps to send birthday reminders on my new Solaris server.

yaps is free software, see http://freshmeat.net/projects/yaps/ on what it is and where to get it.

You will have to compile it for yourself, but it is easy. To have it run side-by-side with mgetty, I had to change the include file used to determine the lock file name. There seem to be different naming conventions for SysV uucp devide locking. I changed config.h to look like this:

 

 /\*
 \* one of these is required for SysV like lockfiles
 \*/
# define        HAVE_SYS_SYSMACROS_H    0
# define        HAVE_SYS_MKDEV_H        1

 

The default is set to use sysmacros.h instead of mkev.h, which worked for me.

 

Now

yaps 0172xxxxxxx "30.01 Dirks Birthday"

will remind me of by own birthday..

That's all. A script will call my home-grown "ute" appointment reminder to extract current reminders and invoke yaps if anything's up.

 

 

 

Tuesday Nov 06, 2007

Live Upgrading

A short look at Live Upgrade


Have you ever used Solaris Live Upgrade or wondered what it looks like?


Here you go...


I just did a small demonstration and thought I share the logs with you.


Step One: Create a new boot environment


This means looking for space to accomodate the OS copy.


As we have a rather simple setup with a single file system for root (/), we just have to provide a place for this one. BTW: Live Upgrade can re-layout your file system on the fly. If I wanted to put /var in its own file system, I could just add a mount point for it (lucreate can create mirrored file systems, too).


In this case, c0t1d0s3 is an unused 10GB slice my 5GB plain Solaris will nicely fit into.


root@m8000b-test # lucreate -n toms -m /:/dev/dsk/c0t1d0s3:ufs
Discovering physical storage devices
Discovering logical storage devices
Cross referencing storage devices with boot environment configurations
Determining types of file systems supported
Validating file system requests
Preparing logical storage devices
Preparing physical storage devices
Configuring physical storage devices
Configuring logical storage devices
Analyzing system configuration.
Comparing source boot environment <production> file systems with the file
system(s) you specified for the new boot environment. Determining which
file systems should be in the new boot environment.
Updating boot environment description database on all BEs.
Searching /dev for possible boot environment filesystem devices


Updating system configuration files.
The device </dev/dsk/c0t1d0s3> is not a root device for any boot environment; cannot get BE ID.
Creating configuration for boot environment <toms>.
Source boot environment is <production>.
Creating boot environment <toms>.
Creating file systems on boot environment <toms>.
Creating <ufs> file system for </> in zone <global> on </dev/dsk/c0t1d0s3>.
Mounting file systems for boot environment <toms>.
Calculating required sizes of file systems for boot environment <toms>.
Populating file systems on boot environment <toms>.
Checking selection integrity.
Integrity check OK.
Populating contents of mount point </>.
Copying.
Creating shared file system mount points.
Creating compare databases for boot environment <toms>.
Creating compare database for file system </>.
Updating compare databases on boot environment <toms>.
Updating compare databases on boot environment <kopie>.
Making boot environment <toms> bootable.
Setting root slice to </dev/dsk/c0t1d0s3>.
Population of boot environment <toms> successful.
Creation of boot environment <toms> successful.
root@m8000b-test #


This process took about ten minutes. This is the result:

root@m8000b-test # lustatus
Boot Environment           Is       Active Active    Can    Copy
Name                       Complete Now    On Reboot Delete Status
-------------------------- -------- ------ --------- ------ ----------
production                 yes      yes    yes       no     -
toms                       yes      no     no        yes    -


Step two: Do the upgrade


To upgrade, I have to provide the location of the OS install media. In this case, we copied the DVD to the installation server "iserv" which exports it via nfs.


root@m8000b-test # luupgrade -u -n toms -s /net/iserv/export/upgrade/sol10_u4/


159648 blocks
miniroot filesystem is <lofs>
Mounting miniroot at </net/iserv/export/upgrade/sol10_u4//Solaris_10/Tools/Boot>
Validating the contents of the media </net/iserv/export/upgrade/sol10_u4/>.
The media is a standard Solaris media.
The media contains an operating system upgrade image.
The media contains <Solaris> version <10>.
Constructing upgrade profile to use.
Locating the operating system upgrade program.
Checking for existence of previously scheduled Live Upgrade requests.
Creating upgrade profile for BE <toms>.
Determining packages to install or upgrade for BE <toms>.
Performing the operating system upgrade of the BE <toms>.
CAUTION: Interrupting this process may leave the boot environment unstable
or unbootable.
Upgrading Solaris: 100% completed
Installation of the packages from this media is complete.
Updating package information on boot environment <toms>.
Package information successfully updated on boot environment <toms>.
Adding operating system patches to the BE <toms>.
The operating system patch installation is complete.
INFORMATION: The file </var/sadm/system/logs/upgrade_log> on boot
environment <toms> contains a log of the upgrade operation.
INFORMATION: The file </var/sadm/system/data/upgrade_cleanup> on boot
environment <toms> contains a log of cleanup operations required.
INFORMATION: Review the files listed above. Remember that all of the files
are located on boot environment <toms>. Before you activate boot
environment <toms>, determine if any additional system maintenance is
required or if additional media of the software distribution must be
installed.
The Solaris upgrade of the boot environment <toms> is complete.


The upgrade took about 1 hour.


Step three: Reboot


Wanna boot the new OS? luactivate takes care of everything necessary.


root@m8000b-test # luactivate -n toms


\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*
The target boot environment has been activated. It will be used when you
reboot. NOTE: You MUST NOT USE the reboot, halt, or uadmin commands. You
MUST USE either the init or the shutdown command when you reboot. If you
do not use either init or shutdown, the system will not boot using the
target BE.
\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*
In case of a failure while booting to the target BE, the following process
needs to be followed to fallback to the currently working boot environment:
1. Enter the PROM monitor (ok prompt).
2. Change the boot device back to the original boot environment
by typing:
     setenv boot-device
/pci@20,600000/pci@0/scsi@1/disk@0,0:a
3
. Boot to the original boot environment by typing:
     boot
\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*
Activation of boot environment <toms> successful.


Ok, there we go...


Rebooting with command: boot
Boot device:
/pci@20,600000/pci@0/scsi@1/disk@1,0:d  File and args:
SunOS Release 5.10 Version Generic_120011-14 64-bit
Copyright 1983-2007 Sun Microsystems, Inc.  All rights reserved.
Use is subject to license terms.
Hostname: m8000b-test
Configuring devices.
Loading smf(5) service descriptions: 29/33
Nov  6 13:46:23 svc.startd[7]: svc:/system/rcap:default: Method ":kill -SIGHUP" failed with exit status 1.
Nov  6 13:46:23 svc.startd[7]: svc:/system/rcap:default: Method ":kill -SIGHUP" failed with exit status 1.
Nov  6 13:46:23 svc.startd[7]: svc:/system/rcap:default: Method ":kill -SIGHUP" failed with exit status 1.
Nov  6 13:46:23 svc.startd[7]: system/rcap:default failed: transitioned to maintenance (see 'svcs -xv' for details)  33/33


m8000b-test console login: root
Password:
Last login: Mon Nov  5 17:15:15 from 10.67.213.111
Nov  6 13:48:44 m8000b-test login: ROOT LOGIN /dev/console
Sun Microsystems Inc.   SunOS 5.10      Generic January 2005


TERM=vt100


root@m8000b-test # lustatus
Boot Environment           Is       Active Active    Can    Copy
Name                       Complete Now    On Reboot Delete Status
-------------------------- -------- ------ --------- ------ ----------
production                 yes      no     no        yes    -
toms                       yes      yes    yes       no     -


Step four (there and back again...) (optional)


Is everything ok with the Upgrade?
If not, switching back is a matter of minutes:


root@m8000b-test # luactivate -n production



\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*
The target boot environment has been activated. It will be used when you
reboot. NOTE: You MUST NOT USE the reboot, halt, or uadmin commands. You
MUST USE either the init or the shutdown command when you reboot. If you
do not use either init or shutdown, the system will not boot using the
target BE.


\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*
In case of a failure while booting to the target BE, the following process
needs to be followed to fallback to the currently working boot environment:
1. Enter the PROM monitor (ok prompt).
2. Change the boot device back to the original boot environment by typing:
     setenv boot-device /pci@20,600000/pci@0/scsi@1/disk@1,0:c
3. Boot to the original boot environment by typing:
     boot
\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*\*


Activation of boot environment <production> successful.
root@m8000b-test # init 6
root@m8000b-test #
Nov  6 13:49:10 m8000b-test ip: ip_create_dl: hw addr length = 0
svc.startd: The system is coming down.  Please wait.
svc.startd: 98 system services are now being stopped.
Nov  6 13:49:56 m8000b-test agent[2249]: syslog   Nov 06 13:49:56 agent         {received software termination signal}
Nov  6 13:49:56 m8000b-test agent[2249]: syslog   Nov 06 13:49:56 agent         \*\*\* terminating execution \*\*\*
Nov  6 13:50:01 m8000b-test ip: ipsec_check_inbound_policy: Policy Failure for the incoming packet (not secure); Source 192.168.224.001, Destination 192.168.224.003.
Nov  6 13:50:02 m8000b-test ip: ip_create_dl: hw addr length = 0
Nov  6 13:50:02 m8000b-test rpc.metad: Terminated
Nov  6 13:50:02 m8000b-test syslogd: going down on signal 15
Nov  6 13:51:35 svc.startd[7]: svc:/application/management/sunmcagent:default: Method or service exit timed out.  Killing contract 52.
Nov  6 13:51:35 svc.startd[7]: application/management/sunmcagent:default failed: transitioned to maintenance (see 'svcs -xv' for details)
svc.startd: The system is down.
syncing file systems... done
rebooting...
Resetting...
..POST Sequence 01
POST Sequence 02
..
POST Sequence 1A
POST Sequence Complete.
ChassisSerialNumber xxxxxxxx


Sun SPARC Enterprise M8000 Server, using Domain console
Copyright 2007 Sun Microsystems, Inc.  All rights reserved.
Copyright 2007 Sun Microsystems, Inc. and Fujitsu Limited. All rights reserved.
OpenBoot 4.24.1, 16384 MB memory installed, Serial #xxxxxxxxx.
Ethernet address 0:xx:xx:xx:xx:x, Host ID: xxxxxxxxx.

Rebooting with command: boot
Boot device:
/pci@20,600000/pci@0/scsi@1/disk@0,0:a  File and args:
SunOS Release 5.10 Version Generic_120011-11 64-bit
Copyright 1983-2007 Sun Microsystems, Inc.  All rights reserved.
Use is subject to license terms.
Hostname: m8000b-test


m8000b-test console login: root
Password:
Last login: Mon Nov  5 17:15:15 from 10.67.213.111
Nov  6 13:56:02 m8000b-test login: ROOT LOGIN /dev/console
Sun Microsystems Inc.   SunOS 5.10      Generic January 2005

TERM=vt100


root@m8000b-test # lustatus
Boot Environment           Is       Active Active    Can    Copy
Name                       Complete Now    On Reboot Delete Status
-------------------------- -------- ------ --------- ------ ----------
production                 yes      yes    yes       no     -
toms                       yes      no     no        yes    -


root@m8000b-test # init 6


And so on...


It is an easy, unattended, quick and safe way to do OS upgrades with Live Upgrade.


You can use it to create a copy of your system before you do an oracle upgrade, too. Just tell Live Upgrade to copy your Oracle binaries as well (see man lucreate on how to do that). Look at lumake, too.

 

Update Nov 23, 2007: "bbr" reminded me that I should have upgraded Live Upgrade Packages before doing the actual upgrade. Correct. This is done by removing lu Packages and re-installing them from the version you want to upgrade to (or newer).

Wednesday Sep 26, 2007

dtrace in destructive mode: working around a non-working syscall

Last week, I was at a customer's site. They are testing our new SPARC Enterprise M8000 Servers and we were in the course of deploying standard applications to the (2) systems under test.

We discovered that one of the applications just would not run on Solaris 10 Update 4. It is an application, let's call it tapewriter, which had been Solaris certified for years, but it does a malformed ioctl request when doing a SCSI inquiry, which it does at the very start trying to access a tape drive.

Analysis: What is happening there?

With truss, we could see tapewriter issues a uscsi ioctl call and the system returns an "invalid argument" error.

ioctl(5, 0x04C9, 0xFFBFF690)                    Err#22 EINVAL

The 0x04c9 function parameter denotes a uscsi request. The third argument points to a struct called uscsi_cmd32. We can learn that from /usr/include/sys/scsi/impl/uscsi.h, and the struct is defined there as well: 

struct uscsi_cmd32 {
        int             uscsi_flags;    /\* read, write, etc. see below \*/
        short           uscsi_status;   /\* resulting status  \*/
        short           uscsi_timeout;  /\* Command Timeout \*/
        caddr32_t       uscsi_cdb;      /\* cdb to send to target \*/
        caddr32_t       uscsi_bufaddr;  /\* i/o source/destination \*/
        size32_t        uscsi_buflen;   /\* size of i/o to take place \*/
        size32_t        uscsi_resid;    /\* resid from i/o operation \*/
        uchar_t         uscsi_cdblen;   /\* # of valid cdb bytes \*/
        uchar_t         uscsi_rqlen;    /\* size of uscsi_rqbuf \*/
        uchar_t         uscsi_rqstatus; /\* status of request sense cmd \*/
        uchar_t         uscsi_rqresid;  /\* resid of request sense cmd \*/
        caddr32_t       uscsi_rqbuf;    /\* request sense buffer \*/
        caddr32_t       uscsi_reserved_5;       /\* Reserved for Future Use \*/
};

Maybe you know as little about SCSI as I do, so let me explain that this is a call to issue generic scsi commands to a device. The scsi command itself is coded in the cdb=command descriptor block. For a SCSI inquiry, which asks "Who's that on the other end of the line?", this looks like e.g. 0x12 0x00 0x00 0x00 x0ff 0x00. 0x12 means inquiry, and 0xff means "my buffer to accomodate data returned is 255 bytes in size". The data returned by the scsi command will be placed in the buffer provided by uscsi_bufaddr. As you want to read data, you will put 0x08 (READ) in uscsi_flags, which tells uscsi what you are about to do.

Sometimes, there is an error message pending on the device that has not been reported yet. This is called the "check condition" state. Before issuing a command, you should check whether the device is ok, and if it is in "check condition" state (the inquiry will report that), you may feel inclined not to issue the command, but another one called "Request Sense", which will get you detailed error status information.

Oh well, that's two requests instead of one. What a load of work. This is why uscsi offers to issue the Request Sense command for you when there is a check condition.

Where's the point? If you request uscsi to do that for you (by setting  USCSI_RQENABLE =0x10000 in uscsi_flags), you have to provide an additional buffer where the Request Sense data can be put.

So why doesn't tapewriter work?

dtrace giving insight

I wrote a small dtrace script that shows the parameters passed to ioctl. It looks like this:

#!/usr/sbin/dtrace -s
#pragma D option quiet

struct uscsi_cmd32      \*ustruct;

syscall::ioctl:entry
/(arg1==0x4c9)&&(execname=="tapewriter")/
{
ustruct=copyin(arg2,sizeof(\*ustruct));
printf ("E pid proc: %i %s\\n",pid,execname);
printf ("flags: %x\\n",ustruct->uscsi_flags);
printf ("uscsi_timeout: %x\\n",ustruct->uscsi_timeout);
printf ("uscsi_cdblen: %x\\n",ustruct->uscsi_cdblen);
printf ("uscsi_buflen: %x\\n",ustruct->uscsi_buflen);
printf ("uscsi_bufaddr: %lx\\n",(unsigned long)ustruct->uscsi_bufaddr);
printf ("uscsi_rqlen: %x\\n",ustruct->uscsi_rqlen);
printf ("uscsi_rqbuf: %lx\\n",(unsigned long)ustruct->uscsi_rqbuf);
printf ("uscsi_cdb: %lx\\n",(unsigned long)ustruct->uscsi_cdb);

first=(char \*)copyin(ustruct->uscsi_cdb,6);
printf ("cdb first byte: %i hex %x\\n",first[0],first[0]);
printf ("cdb 2nd byte: %i hex %x\\n",first[1],first[1]);
printf ("cdb 3rd byte: %i hex %x\\n",first[2],first[2]);
printf ("cdb 4th byte: %i hex %x\\n",first[3],first[3]);
printf ("cdb 5th byte: %i hex %x\\n",first[4],first[4]);
printf ("cdb 6th byte: %i hex %x\\n",first[5],first[5]);
self->ok=1;self->bufadr=arg2;
}

syscall::ioctl:return
/self->ok/
{
 self->ok=0;
 printf ("R pid proc errno: %i %s %i\\n",pid,execname,errno);
 printf ("retval: %i\\n",arg0);
 ustruct=copyin(self->bufadr,sizeof(\*ustruct));
 self->bufadr=0;
 printf ("flags: %x\\n",ustruct->uscsi_flags);
 printf ("uscsi_status: %i\\n",ustruct->uscsi_status);
 printf ("uscsi_timeout: %x\\n",ustruct->uscsi_timeout);
 printf ("uscsi_cdblen: %x\\n",ustruct->uscsi_cdblen);
 printf ("uscsi_buflen: %x\\n",ustruct->uscsi_buflen);
 printf ("uscsi_bufaddr: %lx\\n",(unsigned long)ustruct->uscsi_bufaddr);
 printf ("uscsi_rqlen: %x\\n",ustruct->uscsi_rqlen);
 printf ("uscsi_rqstatus: %x\\n",ustruct->uscsi_rqstatus);
 printf ("uscsi_rqbuf: %lx\\n",(unsigned long)ustruct->uscsi_rqbuf);
 printf ("uscsi_cdb: %lx\\n\\n",(unsigned long)ustruct->uscsi_cdb);
}

This script will show the contents of the uscsi command block as the ioctl is entered and left by tapewriter.

For those who are not familiar with dtrace:

dtrace is a framework and a command. The command dtrace, which is used to execute this script, configures the framework to watch the processor to trigger probes as it executes processes. The script is a "d script", written in "d", the language that is used to describe what dtrace should look for (i.e. the probes that should be watched) and what to do (the actions executed when a probe "fires"). This script watches for processes entering ("entry" probe) or leaving ("return" probe) the ioctl syscall. There is a filter on the entry call that leaves all ioctls aside that are not invoking the uscsi function (0x4c9) or are not done by an executable called "tapewriter".  The self->ok filter on the return probe makes sure we passed the entry call before: self->xxx variables are thread-local variables. self->bufadr is used to remember the third entry parameter, which is the address of the uscsi command struct. Ok, well, I could have used a single variable for this purpose.

For syscall entry probes, arg0...arg5 are variables representing the arguments passed to the syscall. For syscall return probes, arg0 contains the return code to be passed up the callstack by the syscall.

Most of the actions look like c and are self-explanatory.

People writing kernel modules will know about copyin and copyout: These functions copy data from kernel to user address space and vice-versa.

This is what we saw:

root@m8000a:/root/dtrace>./ioctl.d
E pid proc: 17738 tapewriter
flags: 10028
uscsi_status: 0
uscsi_timeout: 7fff
uscsi_cdblen: 6
uscsi_buflen: ff
uscsi_bufaddr: 565598
uscsi_rqlen: 84
uscsi_rqstatus: 0
uscsi_rqbuf: 0
uscsi_cdb: 24d9f0
cdb first byte: 18 hex 12
cdb 2nd byte: 0 hex 0
cdb 3rd byte: 0 hex 0
cdb 4th byte: 0 hex 0
cdb 5th byte: 255 hex ff
cdb 6th byte: 0 hex 0

R pid proc errno: 17738 tapewriter 22
retval: -1
flags: 2
uscsi_status: 0
uscsi_timeout: 454a
uscsi_cdblen: 0
uscsi_buflen: 79736572
uscsi_bufaddr: 416e616c
uscsi_rqlen: 0
uscsi_rqstatus: 0
uscsi_rqbuf: 0
uscsi_cdb: 54617065


The first block of output is written upon entry, the first upon return from the call. The error code of 22 means EINVAL, "Invalid Argument". Did anything catch your eye?

Yep. There is RQENABLE set, but rqbuf is zero. Up to Solaris 10 update 3, Solaris did tolerate this (by simply ignoring the buffer if its null), but with Update 4, the semantics have changed and Solaris now explicitely checks for the buffer if RQENABLE is set. This is where it decides to return EINVAL.

Exercise to the reader: Find these pieces of code in the Solaris source. At about what time has the code been changed? You can browse the source at OpenSolaris.org

Stuck!?

For some reason, we really needed to run tapewriter, otherwise we could not proceed with the test. tapewrite is a third party application, and the manufacturer was already developing a patch (thanks to dtrace we could exactly tell him where to look (I was sure it was the flag as I quickly wrote a c application to check that)). However, this will take some time and it seemed we could not work around that problem.  Switching to U3 was not an option, as that would require re-importing the database, which took nine days.

We were expecting to run into even more problems, as other interfaces may have changed as well. So we had to go on as quickly as possible, bypassing the faulty call.

dtrace saving us ... lots of time

The answer was to have dtrace change tapewriter's call on the fly, clearing the RQENABLE flag.

The revised script looks like this:

#!/usr/sbin/dtrace -s
#pragma D option quiet
#pragma D option destructive

struct uscsi_cmd32      \*ustruct;

syscall::ioctl:entry
/(arg1==0x4c9)&&(execname=="tapewriter")/
{
ustruct=copyin(arg2,sizeof(\*ustruct));
printf ("E pid proc: %i %s\\n",pid,execname);
printf ("flags: %x\\n",ustruct->uscsi_flags);

/\* clear RQENABLE flag \*/
ustruct->uscsi_flags &=~(0x10000);
copyout(ustruct,arg2,sizeof(ustruct->uscsi_flags));

/\* read back struct to verify change \*/
ustruct=copyin(arg2,sizeof(\*ustruct));
printf ("flags: %x\\n",ustruct->uscsi_flags);
printf ("uscsi_timeout: %x\\n",ustruct->uscsi_timeout);
printf ("uscsi_cdblen: %x\\n",ustruct->uscsi_cdblen);
printf ("uscsi_buflen: %x\\n",ustruct->uscsi_buflen);
printf ("uscsi_bufaddr: %lx\\n",(unsigned long)ustruct->uscsi_bufaddr);
printf ("uscsi_rqlen: %x\\n",ustruct->uscsi_rqlen);
printf ("uscsi_rqbuf: %lx\\n",(unsigned long)ustruct->uscsi_rqbuf);
printf ("uscsi_cdb: %lx\\n",(unsigned long)ustruct->uscsi_cdb);

first=(char \*)copyin(ustruct->uscsi_cdb,6);
printf ("cdb first byte: %i hex %x\\n",first[0],first[0]);
printf ("cdb 2nd byte: %i hex %x\\n",first[1],first[1]);
printf ("cdb 3rd byte: %i hex %x\\n",first[2],first[2]);
printf ("cdb 4th byte: %i hex %x\\n",first[3],first[3]);
printf ("cdb 5th byte: %i hex %x\\n",first[4],first[4]);
printf ("cdb 6th byte: %i hex %x\\n",first[5],first[5]);
self->ok=1;self->bufadr=arg2;
}

syscall::ioctl:return
/self->ok/
{
 self->ok=0;
 printf ("R pid proc errno: %i %s %i\\n",pid,execname,errno);
 printf ("retval: %i\\n",arg0);
 ustruct=copyin(self->bufadr,sizeof(\*ustruct));
 self->bufadr=0;
 printf ("flags: %x\\n",ustruct->uscsi_flags);
 printf ("uscsi_status: %i\\n",ustruct->uscsi_status);
 printf ("uscsi_timeout: %x\\n",ustruct->uscsi_timeout);
 printf ("uscsi_cdblen: %x\\n",ustruct->uscsi_cdblen);
 printf ("uscsi_buflen: %x\\n",ustruct->uscsi_buflen);
 printf ("uscsi_bufaddr: %lx\\n",(unsigned long)ustruct->uscsi_bufaddr);
 printf ("uscsi_rqlen: %x\\n",ustruct->uscsi_rqlen);
 printf ("uscsi_rqstatus: %x\\n",ustruct->uscsi_rqstatus);
 printf ("uscsi_rqbuf: %lx\\n",(unsigned long)ustruct->uscsi_rqbuf);
 printf ("uscsi_cdb: %lx\\n\\n",(unsigned long)ustruct->uscsi_cdb);
}


To make dtrace accept the copyout command, I had to include the destructive directive at the start of the script. 

And it worked! It worked so well they decided to set this script up as a pre-exec script to tapewriter, meaning that it will be run automatically every time tapewriter is run. There is a post-exec script that kills dtrace when tapewriter is done.

Disclaimer

This is a rather dirty hack and we could not really forsee what the implications were for the execution of tapewriter. Remember that we did this in a testing environment, and rendering the system unusable would not have had any business impact.

Use with caution! In this case, it just saved about a week of waiting for the patch.

 

Friday Sep 07, 2007

Why?

I never wanted to start a blog.

I just thought with a job, three children and a home, I could not afford one.

However, sometimes I want to be heard. After being told frequently that Linux is better for personal use, and running my own (wife certified) Linux home production system for a few years, I decided to go for Solaris and did the migration within a few days.

I want to share this experience

  • to show people who ported software to Solaris that it's actually used and say thank you to them this way.
  • to show people that it is fun to have at home.
  • to share my experience with people who gave adviceto me

 


That's why.

Do not expect new entries each day or even week, but on occasions like this one...

 

Solaris@Home

Finally, I have my Solaris based homeserver up and running. It took me quite some time to decide which hardware to buy, as I found it difficult to judge which compontents will work out of the box and which would need drivers from third party sources which I wanted to avoid. There is a lot of information available, including the Hardware compatibility list, but some information is outdated or incomplete. This is why I thought I should share my experience.

If you have any questions, just write me a mail: dirk.wriedt@sun.com.

Before

Old setup

I had been running my home production system on Red Hat Linux 9 for a few years. This is what I was running and had to be moved to Solaris:

  • StarOffice
  • Agfa Snapscan Touch USB scanner with SANE/xsane.
  • evolution mail client
  • digital photo repository keeping family's photos
  • gqview to view photos
  • amavisd-new email Virus/SPAM scanner with Avira/SpamAssassin
  • ldap directory service for address book running on openldap
  • mozilla/firefox web browser
  • grip to rip CDs
  • xmms to play ripped CDs...
  • mgetty & sendfax to receive and send faxes

New Features I wanted to have

  • Sun Ray Server to run three Sun Rays I had bought, thinking about how I could provide my wife and children with their own workstations without having to administer ten PCs.
  • Big fat secure hassle-free file system to keep my photos in a safe place

The new way

Server Hardware

I bought the following:

  • Asus M2NPV-VM Mainboard for approx EUR 80. This board has disk connectors for 2x2 channels IDE and 4 SATA and onboard video.
  • AMD x2 dual core processor, the small one.
  • 2 GB of Kingston Memory @ EUR 45 each
  • three 500 GB Samsung SATA disks @ EUR 100 each
  • a rack-mountable case with power supply, fans and space for eight disks (6x5.25", 2x3.5")
  • cheap DVD Burner @ 30 EUR

I used a spare IBM Deskstar Disk as a boot disk and an old realtek pci network card for a second network interface facing the internet. After putting everything together, I was delighted to see that

  • I did not need a third-party driver for the on-board nge0 network interface (although the hcl report said so)
  • The SATA disks were accessible (although in pata compatiblity mode)
  • the Firewire interface was recognized I did not care about audio and video (nvidia), although the latter worked out of the box.

 

Sun Rays

I had already bought three Sun Rays on ebay at about 50 EUR each.

After

Software

Well, of course it took me a few nights to set everything up.

  • StarOffice and evolution worked as before
  • gqview, mozilla, firefox, sane were available from Blastwave.
  • I installed a load of other packages from Blastwave, xpdf and evince amongst them (I like xpdf better for pdf viewing that evince)

Connecting the Scanner

This was were I expected to spend quite a bit of time on. I did a proof of concept with my notebook, where I had to recompile sane-backend as it was lacking usb support, and still had to manually integrate the backend for my scanner. I took the sane-backends from csw, too, but used the libusb that came with Solaris. I learned that my Scanned (niash chipset) is now natively supported by sane-backend, and thus, to my astonishment, the scanner connected to the Sun Ray worked out of the box.

The libusb plugin for Sun Ray worked instantly:

scanimage -L did show my scanner!

It is somewhat slower than before now, as the scanner is now connected to the Sun Ray which supports usb1.1 only, compared directly attaching to the server's usb2.0 port.

Mail

Configuring amavisd-new in the dual sendmail setup was time-consuming but not really difficult, as there is a pretty good walkthough in the documentation. I use Avira AntiVir PersonalEdition Classic for Virus Scans and SPAMassassin to fight SPAM.

I am running the cyrus imapd I got from Blastwave. Mail is being fetched by fetchmail, running as root, every five minutes. My email provider is doing "pop before smtp" and this way, I get my mail almost instantly, and sendmail can send mail all the time.

Sun Rays

The Sun Rays worked as expected, with two minor issues left:

  •  Every now and then, when rebooting, the dhcp server will not start up, claiming it can not access its configuration. As the Sun Rays rely on a working dhcp server, this is really annoying to me.
  • As expected, the Sun Ray is not really capable of displaying full-screen videos. However, did you know there is an mplayer version with support for Sun Rays? With that player, watching videos at 800x600 is feasible. Flash-Based video with low res, low size and low frame rate (Youtube) is not a problem at all.
  • Listening to music (xmms) is working perfectly.

ZFS

ZFS is just great for home use, as you do not have to worry about file system sizes. To have raidz feels a lot better that running in an unprotected fashion.I still have to invest in a hot spare disk.

And, while a power spark may just fry all of your disks at once, snapshots can at least protect you from the consequences of "rm -rf".

ZFS eats lots of RAM. I had seen reports on that behaviour before and I feel inclined to do something about it. I have a TV set top box (Siemens Gigaset M740AV) that records to ZFS via NFS, and doing so, it trashes my file system cache. When I log in after watching tv, starting the gnome session takes ages as the cache is rebuilt slowly. I know about the way to change cache usage with mdb, but maybe I am just going to add memory.

Squid

I am using the squid proxy cache. As for sendmail and amavisd-new, I did my own SMF Manifests to have it being managed by smf. SquidGuard implementation is pending.

LDAP

I now run the Sun Directory Server. Migrating from the crappy openldap export took quite some time and a few scripts.

Recovery

After migrating all data to the new server, I had some more spare disks, and decided to replace that deskstar boot drive with a faster one. Somehow I managed to trash my root file system when preparing the Live Upgrade.

Lucky me, I had created a flash archive two days earlier, as LiveUpgrade lucreate trashed by root file system. I have no clue why. Anyway, with that flash on a different server, I booted from the Solaris DVD, selected installation from flash, and after two hours (including some testing afterwards) the system was back up.

DVD Authoring

I would like burn video DVDs, but there is no decent cutting or dvd authoring software that runs on Solaris. Any hints?

Performance

In my setup, the limiting factor is disk performance. When starting applications, the (single and slow) boot disk was at 97%. The cpu load barely went above 30%, even when doing mencoder- (video) oder lame- (audio) encoding. I have never seen a system load of above 1. That said, I wonder why I rarely see disk read rates above 5 MB/s. ZFS pool scrubs show rates of 50 MB/s on the SATA disks, but even when the boot disk is 100% busy, read rates look slow to me.

Now that I have the new disk in place, I expect them to increase.

Power consumption

With four disks, no overclocking, no ata spindown, power consumption is at 110 Watts. I will try and enable ata spindown, and see if it changes significantly. I just have to get a new wattmeter, mine just died after a power outage.

Conclusion

All in all, I did not expect the migration to run as smoothly as it did. There were absolutely no driver issues (I have not checked audio yet). I have most applications I need, and all that I had on Linux. Hmm. Not quite. Gnometoaster is missing. Still, Solaris lacks support for DVD authoring, first person shooters, and support by ISVs for childrens' games. 

About

Dirk Wriedt

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today
Bookmarks
Software for Solaris
Favourite blogs
Dirks Bookmarks

No bookmarks in folder