Sunday Apr 05, 2009

Recovering our Windows PC

I had reason to discover if my solution for backing up the windows PC worked. Apparently the PC had not been working properly for a while but no one had mentioned that to me. The symptoms were:

  1. No menu bar at the bottom of the screen. It was almost like the screen was the wrong size but how it was changed is/was a mystery.

  2. It was claiming it needed to revalidate itself as the hardware had changed, which it catagorically had not and I had 2 days to sort it out. Apparenty this message had been around for a few days (weeks?) but was ignored.

Now I'm sure I could have had endless fun reading forums to find out how to fix these things but it was Saturday night nd I was going cycling in the morning. So time to boot solaris and restore the back up. First I took a back up of what was on the disk, just in case I get a desire to relive the issue. I just needed one script to restore it over ssh. The script is:

: pearson FSS 14 $; cat /usr/local/sbin/xp_restore 

exec dd of=/dev/rdsk/c0d0p1 bs=1k
: pearson FSS 15 $; 

and the command was:

$ ssh pc pfexec /usr/local/sbin/xp_restore < backup.dd

having chosen the desired snapshot. Obviously the command was added to /etc/security/exec_attr. Then just leave that running over night. In the morning the system booted up just fine, complained about the virus definitions being out of date and various things needing updates but all working. Alas doing this before I went cycling made me late enough to miss the peleton, if it was there.

Monday Oct 06, 2008

Incremental back up of Windows XP to ZFS

I am forced to have a Windows system at home which thankfully only very occasionally gets used however even though everything that gets on it is virus scanned all email is scanned before it gets near it and none of the users are administrators I still like to keep it backed up.

Given I have a server on a network which has ZFS file systems with capacity I decided that I could do this just using the dd(1) command which I have written about before. Using that to copy the entire disk image to a ZFS file allows me to back the system up. However if I snapshot the back up file system and then back up again every block gets re written so takes up space on the server enven if they have not changed (roll on de dup). To stop this I have a tiny program that mmap()s the entire backup file and then only updates the blocks that have changed.

I call it syncer for no good reason:

#include <stdlib.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <string.h>
#include <sys/mman.h>
#include <stdio.h>
#include <sys/time.h>
 \* Build by:
 \*              cc -m64 -o syncer syncer.c

 \* Match this to the file system record size.
#define BLOCK_SIZE (128 \* 1024)
#define KILO 1024
#define MEG (KILO \* KILO)
#define MSEC (1000LL)
#define NSEC (MSEC \* MSEC)
#define USEC (NSEC \* MSEC)

static long block_size;

char \*
map_file(const char \*file)
        int fd;
        char \*addr;
        struct stat buf;

        if ((fd = open(file, O_RDWR)) == -1) {
                return (NULL);

        if (fstat(fd, &buf) == -1) {
                return (NULL);

        block_size = buf.st_blksize;

        addr = mmap(0, buf.st_size, PROT_READ|PROT_WRITE, MAP_SHARED, fd, 0);
        return (addr);
read_whole(int fd, char \*buf, int len)
        int count;
        int total = 0;

        while (total != len && 
                (count = read(0, &buf[total], len - total)) > 0) {
        return (total);
static void
print_amount(char \*str, off64_t value)
        if (value < KILO) {
                printf("%s %8lld ", str, value);
        } else if (value < MEG) {
                printf("%s %8lldK", str, value/(KILO));
        } else {
                printf("%s %8lldM", str, value/(MEG));
main(int argc, char \*\*argv)
        char \*buf;
        off64_t offset = 0;
        off64_t update = 0;
        off64_t count;
        off64_t tcount = 0;
        char \*addr;
        long bs;
        hrtime_t starttime;
        hrtime_t lasttime;

        if (argc == 1) {
                fprintf(stderr, "Usage: %s outfile\\n", \*argv);
        if ((addr = map_file(argv[1])) == NULL) {
        bs = block_size == 0 ? BLOCK_SIZE : block_size;
        if ((buf = malloc(block_size == 0 ? BLOCK_SIZE : block_size)) == NULL) {
                perror("malloc failed");

        print_amount("Block size:", bs);

        starttime = lasttime = gethrtime();
        while ((count = read_whole(0, buf, bs)) > 0) {
                hrtime_t thistime;
                if (memcmp(buf, addr+offset, count) != 0) {
                        memcpy(addr+offset, buf, count);
                madvise(addr+offset, count, MADV_DONTNEED);
                madvise(addr+offset, bs, MADV_WILLNEED);
                thistime = gethrtime();
                 \* Only update the output after a second so that is readable.
                if (thistime - lasttime > USEC) {
                        print_amount("checked", offset);
                        printf(" %4d M/sec ", ((hrtime_t)tcount \* USEC) /
                                (MEG \* (thistime - lasttime)));
                        print_amount(" updated", update);
                        lasttime = thistime;
                        tcount = 0;
                } else { 
                        tcount += count;
        printf("                                            \\r");
        print_amount("Read: ", offset);
        printf(" %lld M/sec ", (offset \* NSEC) /
                (MEG \* ((gethrtime() - starttime)/MSEC)));
        print_amount("Updated:", update);
        /\* If nothing is updated return false \*/
        exit(update == 0 ? 1 : 0);

Then a simple shell function to do the back up and then snapshot the file system:

function backuppc
	ssh -o Compression=no -c blowfish pc pfexec /usr/local/sbin/xp_backup | time ~/lang/c/syncer /tank/backup/pc/backup.dd && \\
	pfexec /usr/sbin/zfs snapshot tank/backup/pc@$(date +%F)

Running it I see that only 2.5G of data was actually written to disk, and yet thanks to ZFS I have a complete disk image and have not lost the previous disk images.

: pearson FSS 17 $; backuppc
665804+0 records in
665804+0 records out
Read:     20481M 9 M/sec Updated:     2584M 

real    35m50.00s
user    6m27.98s
sys     2m43.76s
: pearson FSS 18 $; 

Tuesday Jan 08, 2008

Privileges, fdisk and backing up windows xp

Over the holiday period I had some fun when the one Windows XP system in the house failed and would not boot. The reason was that the disk had a number of sectors that could not be read and had to be replaced. Anyway since all the user data is not present on the PC nothing was lost except a few hours of my time. That time did make me think that even the old Solaris installer was not that bad and that was before I had to reinstall every program, plus service pack two and the worlds supply of patches. So I have decided that I should back up the XP partition so that I don't have to go through that again.

To do this I I've installed Solaris as a dual boot and then use dd(1) to copy the XP partition onto the server. The only problem is that the cmdk(7D) driver does not seem to honour the end of partitions. So if I just dd the /dev/rdsk/c0d0p1 partition I don't just get the XP partition but also get the entire Solaris partition.

Hence I have a little script that used fdisk(1M) to get the disk geometry and then copy just the XP partition. Now the strange thing about fdisk is that it has to be run with “all” privileges. So the script is started with all and then drops any that it does not really need, ie nearly all of them. So exec_attr and prof_attr have to have the “XP Backup” profile added so that they look like this:

# egrep 'XP Backup' /etc/security/\*_attr            
/etc/security/exec_attr:XP Backup:solaris:cmd:::/usr/local/sbin/xp_backup:privs=all
/etc/security/prof_attr:XP Backup:::Back up Windows XP:help=XpBackup.html

Then just use 'usermod -P “XP Backup” cjg' so I can run this via ssh and then redirect the output into a file.

Anyway here is the script, I've tested that it works on the PC and on my work laptop which still has an XP partition:

# The contents of this file are subject to the terms of the
# Common Development and Distribution License (the "License").
# You may not use this file except in compliance with the License.
# You can obtain a copy of the license at usr/src/OPENSOLARIS.LICENSE
# or
# See the License for the specific language governing permissions
# and limitations under the License.
# When distributing Covered Code, include this CDDL HEADER in each
# file and include the License file at usr/src/OPENSOLARIS.LICENSE.
# If applicable, add the following below this CDDL HEADER, with the
# fields enclosed by brackets "[]" replaced with your own identifying
# information: Portions Copyright [yyyy] [name of copyright owner]
# copy the Windows partition to standard out.
# This needs to run with the "all" privilege so that fdisk can read the partion
# information. It has to read that as the disk driver does not enforce the end
# of the partitions.
# I added the XP Backup profile and then added this command to it with
# the additional priv. Here is teh entry from /etc/security/exec_attr:
# XP Backup:solaris:cmd:::/usr/local/sbin/xp_backup:privs=all
# (obviously not as a comment).
# Then there is a degree of paranoia in the script. It drops all the
# privileges that it does not need.

export PATH=/usr/bin:/usr/sbin


set - $(fdisk -W - ${device}p0 | ppriv -e -s A=basic nawk '$1 ~ "[0-9]" { x++ } $1 == 7 { print $7, $NF/$7, $NF%$7, x }')

ppriv -s L=file_dac_read,proc_exec -s EIP=proc_exec,file_dac_read $$

if (( $3 != 0 )) 
        print "Partition is not a whole number of tracks!" >&2
        exit 1


if (( bs == 0 || count == 0 ))
        print "Block size($bs) and block count($count) can be zero" >&2
        exit 1

exec ppriv -e -s EPL-file_dac_read -s I-file_dac_read dd bs=${bs}b count=$count < /dev/rdsk/${device}p${part}

Follow up: It turns out the comment obout cmdk letting you read off the end of the disk is wrong. Only when using the p0 partition can you read the whole drive (which was my mistake) if you choose the correct partition which in my case is the p1 partition it will only read to the end of the partition. However I still need to use fdisk to find the correct partition so the script remains the same.

Sunday Mar 25, 2007

Operating systems as children

Solaris: you stay at that end of the disk. Windows you stay at the other and no talking.

Right I'm only going to say this once:

Solaris: you will not, repeat not, delete Windows from the grub menu.

Windows: Even if Solaris does delete you from the grub menu you are not to make your partition the active one so that Solaris can't boot.

Thank you.


Wednesday Oct 04, 2006

Windows Crash Dump Analysis

For a lot of people this was the shock talk of the CEC and not because people think Windows systems don't crash but why would Sun Engineers what to know how to diagnose those. Well we get to support systems running Windows so it is useful to be able at least do a first pass analysis of the crash so that you know who to call next. Is it Windows, a driver or the hardware?

Dimitri De Wild and Feri Chua did an excellent, if a bit rushed, job of presenting the topic. This was not their fault with only forty minutes it was always going to be a rush even for just an introduction into this topic. I think I would have changed the title of the talk to “Windows Crash Dump Analysis for Solaris Kernel Engineers” though as it seemed to make some (reasonable) assumptions about the audience in some places.

Another really good talk which while I suspect I will not directly use the information it is very useful to know what Windows can do if you configure it to collect the crash dump.



This is the old blog of Chris Gerhard. It has mostly moved to


« July 2016