Adding a pxeboot driver to Solaris 10 01/06 x86

So a partner came in last month with a network driver for a blade system. They developed it using a whitebox PC running Solaris with a reference PCI card. But they never officially tested it on the blade system the driver was intended for because they couldn't get the driver on the blade... at least not conveniently. The problem is that while they could power the blade inside a special case, without a full chassis with optional shared optical drives, it was pretty tough installing the OS because the only way to install it on the blade was to do a network install. But since this was a new network driver, it wasn't in the installation OS.

The BIOS and the NIC supported PXE boot and while Solaris could begin the installation, it would stop and exit to a shell prompt the moment the PXE handed control over to the Solaris installer. Mind you, we're talking about new boot in the Solaris 10 Update 1 (aka Solaris 10 01/06) which now uses grub bootloader. But this was a classic example of the Chicken versus Egg problem.

The partner had a good working knowledge of PXE boot and they followed the document and others they found on the net. When PXE booting a PC, the install client receives a number of options from the DHCP server that direct it to fetch a runnable ramdisk image which then bootstraps the rest of the system.

In the case of Solars 10 1/06, the boot client receives DHCP option entries that tell it where to download what a Solaris x86.miniroot root image, which is about a 50+MB bootable ramdisk image. The PXE BIOS handles the entire network transfer from the server, then unpacks the image and boots it. Control of the network is then handed over by PXE to the Solaris miniroot.

Now, if a necessary network driver isn't in the Solaris x86 miniroot at this point, well, the installation stops, prints an error and exits to a shell prompt. This was pretty frustrating for our partner who knew they needed to get their driver into the installer image but didn't know how. Solaris 10 x86 update 1 came to the rescue. The new boot architecture has added several commands that simplify the packing and unpacking of x86 miniroots. To add a driver, one simply unpacks the miniroot, adds the driver and repacks it.

To do the first step, unpacking the miniroot, a user can run the root_archive(1M) command on the install server (presumably it's running Solaris 10 x86 1/06 - update 1 - since older versions and SPARC do not have this utility):

  # /boot/solaris/bin/root_archive unpack [pxeserverpath]/boot/x86.miniroot [path-to-unpacked-dir]

Once the miniroot is unpacked, one copies the 32-bit driver binary and driver .conf file to the [path-to-unpacked-dir]/kernel/drv directory, and then runs the /usr/sbin/add_drv command with the right PCI IDs and the right permissions against the unpacked miniroot directory. E.g.

  # add_drv -b [path-to-unpacked-dir] -n -v -m '\* 0600 root sys' -i "[device ids]" [mydrivername]

where, [mydrivername] is the name of the binary just copied to the unpacked miniroot ./kernel/drv directory, and the [device ids] is a string list of PCI device IDs that might look like:

  '"pci1a44,9043" "pci1a44,9065" "pci1a44,9106" "pci1a44,9053"'

Note that the -b flag allows you to set the root path where to apply the driver add operation, which is pointing to the unpacked miniroot. Note also that the x86.miniroot is 32-bit and only supports 32-bit drivers.

The last thing to do is to repack the miniroot using the same command, but specifying "pack" as the keyword. But before we do that, I like to make a copy of the original x86.miniroot in the same boot directory, and then run the root_archive(1M) command:

  # /boot/solaris/bin/root_archive pack [pxeserverpath]/boot/x86.miniroot [path-to-unpacked-dir]

If the PXE boot server was working before, the client should be able to boot and load the x86.miniroot and complete a normal Solaris 10 1/06 installation. The installation will bring up a menu of install choices; the default is to perform an interactive install. When doing an interactive install, the installer offers a choice of auto-reboot, or manual reboot. Choose the manual reboot, because while the above steps get the driver into the miniroot, they don't do anything to install the missing network driver onto the final client system. So we select manual reboot, to give ourselves a chance to copy over the drivers and run the add_drv command on the final client disk image before rebooting. Otherwise, the newly installed system, still won't have a network driver.

So where do we put the driver binaries so the install client can copy then over? One way is to copy the one from the miniroot's /kernel/drv that's already there. This works if the system is only 32-bit and will only run in this mode. However, if the system is x64 capable, we didn't copy the 64-bit driver binary into the miniroot's /kernel/drv/amd64 directory and that's because the miniroot is only 32-bit and doesn't have a ./kernel/drv/amd64 directory. We could have created that directory in earlier steps, but that's not necessary. In fact, during the Solaris PXE boot installation, the client mounts the install server directories via NFS. Thus it's easy enough, before rebooting, to simply just put the driver files in the install server's exported PXE boot directory, especially inside the jumpstart directory where the client already mounts them during the install. I usually just put a ./jumpstart/drv subdirectory inside the server's exported directories. This will be mounted on the install client as /tmp/install_config usually. If you can't find it, use the df(1M) command to see a list of filesystems and their mount points.

So now we can persist the drivers to the install client. The installer miniroot mounts the client's hard drive usually at /a. So it should be straightforward to copy the binaries and .conf over to /a/kernel/drv and /a/kernel/drv/amd64. (Note: there's usually just one .conf file for the driver that goes in the /a/kernel/drv - which is used by both 32- and 64-bit binaries.) The we run the add_drv(1M) command again, but for the -b option, we point to the /a directory.

If this is a custom, hands-off jumpstart installation, it's possible to modify the post-installation finish script to copy the driver (both 32- and 64-bit binaries and .conf) over to the final system disk and also run the add_drv(1M) command so all systems of the same type get the same new network driver without manual intervention.

Our partner with the driver problem was pretty pleased that this worked, and surprised that it wasn't all that hard to do. We're hoping to make it even easier in future versions to add drivers into the installer and even create bootable CDs and DVDs and burn them all in Solaris.


Post a Comment:
Comments are closed for this entry.



« July 2016