By Tim Thomas on Nov 17, 2008
[This article was originally posted on Nov 17th and updated on Nov 21st.]
I have done a lot of work with Solaris and ZFS on the Sun Fire X45x0 server but now I am now working on something a little different as I have a project where I am running Microsoft Windows Server 2003 on a Sun Fire X4540.
For the volume management I am running Veritas Storage Foundation for Windows and I wanted to share my experiences.
Before you Install Storage Foundation
ONLY IF this is a new system and/or if this is the first time you have installed Windows on this system do the following after you have installed Windows and BEFORE you install Storage Foundation:
1. Bring up the Windows Storage Manager and make sure it finds all 48 disks.
2. Delete any partitions on disks other than the boot disk. THIS DESTROYS ANY DATA on the disks. You don't have to anything to disks with no partitions on them, just the act of LVM finding them makes them available to windows.
Who do this ? The X4540 ships with Solaris pre-installed and a pre-configured ZFS storage pool. Veritas Storage Foundation for Windows is essentially Veritas Volume Manager. We install Windows over the top of Solaris but the rest of the disks are untouched. Under windows, when Veritas Volume Manager scans the disks it sees something is on the disks, plays safe and marks them as unusable. I found no way to override this behavior. The standard Windows LVM does not have this issue and allows you to tidy the disks up. You need to follow this procedure before you install Storage Foundation as it replaces the Windows LVM.
Using Storage Foundation
I really liked using this software. It was easy to install (though the install takes a long time) and easy to configure and use. The X4540 has 48 disks which can be challenging to manage, but the Veritas software makes this relatively easy. The wizard for creating volumes is great, allowing you to manually select disks so that you are striping or mirroring across controllers for example..or you can let the software decide, but I prefer to maintain manual control of how my volumes are laid out.
This is the Disk View:
This is the Volume View:
Here are a couple of shots of the Volume Wizard. The first one shows the disk selection page and there is the option to allows the software to Autoselect...though I went for Manual:
This shows the page where you choose the type of volume to create:
One interesting thing I learnt from this UI is how Windows maps the disks in the X4540. Windows presents the disks as Disk0->Disk47 which is not very informative if you wish to build volumes across controllers. Via the Veritas GUI I was able to see that the six SAS controllers in the X4540 are mapped as P0->P5 and then we have eight disks on each controller T0->T7. C and L are always 0. You can see this in the first screenshot of the Volume Wizard.
I built a RAID-10 Volume and a RAID-5 Volume. To help me plan, I printed out a copy of my Sun Fire X4540 Disk Planner (which was designed for Solaris) and changed the column labels to P0->P5 and the row labels to C0->C7. I labeled the boxes Disk0->Disk47, starting at the top left I worked down the first column, then returned to the top of the next column and working down that and so on.
A Final Note: Disk Write Caches
The X4540 ships with the disk write caches on. The Volume Manager will warn you about this. Disk write caches are volatile and you could loose data in event of a power outage. If you don't have UPS protection then you can switch the disks write caches off using the Windows Device Manager. Note that Solaris ZFS is cool with disk write caches on as it flushes them out when it periodically syncs the file system.