Sun Ultra 24: The Perfect ESXi box for home (or... "The last PC I may ever buy")

After receiving the mandatory "Waiver for Internal Funding for Equipment" (WIFE) approval, I received a few days ago my new toy, a super-duper Sun Ultra 24 workstation, possibly the last PC I may ever buy (but then again, wasn't 640K enough for all needs?) It's a welcome addition to finally replace my 6 and a half year old Frankenstein PC (AMD Athlon XP1800+) which served faithfully over the years.

Careful thought went into this purchase:
- it had to be as powerful as possible without breaking the budget
- it had to be super quiet, as it will be on most of the time in a public part of my house
- it had to to have room for heaps of memory and disk
- it needed to be capable of running pretty much any O/S, but above all, VWare ESX, and as it turns out, just when I was about to place the order, VMware saved me some money by making ESXi free.

In the end, I chose the Ultra 24 because:
- it's VERY well put together
- it met all the requirements above
- it was competitive with anything out there in the market, and better than most in terms of memory capacity (8GB)
- and well, staff discount... So I added the obligatory 22" screen (it is \*fantastic\*) and a Sun Ray 2 to accompany my already existing batallion of Sun Rays.

Anyway, this blog entry is about the steps to get ESXi to run on it, so here we go.

START SAFE: LOAD THE LATEST FIRMWARE FOR THE ULTRA 24
For completeness, I updated the system to the latest firmware, downloaded from here:
www.sun.com/desktop/workstation/ultra24/downloads.jsp

FIRST CHALLENGE: SATA DISKS ARE NOT SUPPORTED BY ESXi (or are they?)
I tried SATA once on an Ultra 20 and got quite far: I even built a VM, only to find that the VI client would not see it even though it existed on the VMFS volume. Go figure. One day, when I get a spare drive, I'll try SATA on its own. For now, read on...

As with any other "workstation" in the market, the Ultra 24 can house SATA and SAS disks, but that doesn't mean that the internal controller can handle SAS disks, so buying these disks was not an option (and I couldn't afford them anyway), so to get there for now, I bought 4 x 250GB SATA disks to go in. With every major vendor box, you find that although the disks themselves are a commodity, the brackets that hold them in aren't, so I went for the easy way out, cheap for now with all the brackets in place for the future, so that an upgrade to bigger disks can be had anytime.

The next thing to solve was getting ESXi to be happy with these drives. As it turns out, I had earlier experimented successfully with a Sun branded LSI card that's fully supported by VMware ESX (Sun part # SG-XPCIE4SAS3-Z), which in more generic terms is a "4-Port SAS PCI-E HBA LSI3041E with B3 silicon" - although I'm yet to understand (or care) what "B3 silicon" means.... We used these cards to run a Sun VDI bootcamp training course on a raft of Sun Ultra 20 systems with 4GB RAM.

So, I plugged in the card into one of the PCI-E slots of the Ultra 24, and lo and behold, the next reboot the system saw the card and the LSI adapter framework which would later allow me to configure RAID options with my disks.

SECOND CHALLENGE: HOW TO BETTER RAID THE DISKS FOR MY MULTIPLE USES?
So now I have four disks and all sorts of possible configurations... But at the end of the day, for now my needs are defined: I want to run mostly ESXi as safely as possible, so that I can leave a number of VMs running to host some of my lab stuff with confidence (Sun Ray server on Solaris/CentOS/SLES,SGD, Windows VMs, LDAP server, etc) but allow some room to run Windows natively for other very important things like, well, Team Fortress 2, and when the time is ripe, play with Sun xVM.

So the choice was given to me when I realised that the LSI RAID card would let me create a mirror across 3 disks... Cool - 3 disks as a mirrored set of 375GB and an independant disk to multiboot from other operating systems, Windows XP and Solaris xVM at the very least. And yes, I tested the mirror - I pulled out a drive, and the LSI framework reported it was working in degraded mode until I plugged the drive back in (after a shutdown of course). This type of mirroring may or may not be as fast as a standard two way (think of the implications and the design of such a mirror) but that doesn't bother me too much just yet.

THIRD CHALLENGE: THE INSTALLATION ITSELF (NO PROBS HERE... BUT)
After this all worked, I simply loaded the ESXi 3.5 u2 CD on the DVD tray, booted and installed happily, UNTIL... the initial ESXi screen came back displaying that the IP address was 0.0.0.0.

FOURTH CHALLENGE: THE NETWORK
0.0.0.0... Huh? So after a bit of digging I found out that VMware doesn't particularly like the onboard Gigabit Ethernet NIC, which although being an onboard Intel Gigabit, the particular chipset (82566DM-2) is not in the support list "yet" (see http://www.grid.org/blog/cameron/updating-vmware-esxi-disk-dump-file among others) and so, since I wanted instant satisfaction by now and had found the potential workaround too hard to implement, I decided that my time would be better spent by investing $47 in a stock standard, ESX supported, Intel 1000/pro Gigabit card, which worked on the next reboot.

FIFTH CHALLENGE: BOOT EASILY FROM ONE DRIVE/LUN OR THE OTHER
As it turns out, the U24 does not let you boot from different Hard Disks/LUNs. You can only specify ONE preferred hard disk in the BIOS, and the boot menu shows only this one, along with whatever other options your system may have (CD, ETH, USB). After a bit of toing and froing, I found that booting into the BIOS everytime I wanted to change boot disks was VERY painful, whenever the time came for Team Fortress 2. The only option presented to me... USB boot!

SIXTH CHALLENGE: BOOT ESXi FROM A USB DRIVE
What do you know, there's an internal USB port within the Ultra 24. How useful's that? I had always wanted to give this a try, and now the time was ripe. Googling for instructions I found the more official ones at http://communities.vmware.com/message/825686, but there are plenty of repeat intructions out there. Of course, for a Unix/Mac guy, winimage is not a requirement and a simple "dd" command did the trick quite nicely, although you do seem to need a 2GB drive or \*smaller\*. So, now, I've become a USB boot fan, because that shows up in the boot menu! As a side note, booting ESXi off USB is not particularly fast, but it does make the setup a lot more portable and convenient. At the end of the day, you can easily import the VMs that are already residing on the VMFS to any ESX bootable environment, so I can choose to boot from disk or USB and still have my VMs.

SEVENTH CHALLENGE: LIMITATIONS OF ESXi
Almost finished here. I just wanted to document the downside of ESXi in my particular setup:
1) No access to the CDROM. You can't map a CD to a VM, which is a bit disappointing. Very little documentation on this but it seems to have to do with the fact that the CDROM on the Ultra 24 is IDE...
2) No USB mapping. That's a sad one, but I knew in advance, so I'm not too fussed.

By now, it's a happy ending - I have built a few VMs, got my Sun VDI server now running within, my XFX 9800GT plays TF2 like a charm and the machine has been rock solid. Good to go til 2014 I hope!!!

Comments:

Awesome job.. I just installed ESXi at home on a ASUS Pundit barebones, Athlon x2 64 5200+, 2GB ram, 80GB sata drive. There were absolutely no issues with the install, I also dd'd the internal image to a usb stick to make things easy. I never once questioned whether or not sata drives were supported though.. just whether or not the sata controller would be recognized. Obviously it was detected and I set up the internal sata drive as storage for my vm's. Even my onboard nic was detected even though I also have 2 more Intel Pro 1000's in there (also detected fine). The only thing lacking of course is a sata card with raid but I'm ok with that right now as all important data is mapped to my nas. I've installed 2 vm's so far, ubuntu and xp pro without any problems at all, just had to mount the floppy image for the lsi drivers so xp would see the virtual disks.

I'm still amazed that I'm running ESXi at home on what I consider as basic hardware. And it's running perfectly so far.

That Sun Ultra 24 is awesome.. I was drooling looking over the specs.. I wish I could obtain W.I.F.E. the way you do.

Posted by Robert Rambo on September 23, 2008 at 05:16 PM EST #

Great to hear another ESXi success story! I work for a Sun reseller and we have a large lab full of all kinds of Sun gear. I wanted to try ESXi on an Ultra 40 and came to the same boot conclusion you did (flash). The install CD never saw the internal 250GB SATA disk but the two NICs worked just fine. Interesting that you mention not being able to mount the CD to a VM. I thought it was just me! Not a deal breaker, but then again we're also a VMware reseller, so I'm running Virtual Center and I can map other ways. SRSS, SGD, and VDM all work incredibly well in VMware. Regards... - Jerry

Posted by Jerry Martin on September 26, 2008 at 06:09 AM EST #

I've installed ESXi 4.0 on a Sun Ultra 24. My experience was great. Sorry for my weak communication capability, I'm not a native english speaker.

THE PROBLEM
My needs were: have some OS phisically installed on the machine (no virtualization) and have a VMWare ESXi system and datastore. Altogether in on the same unique disk and then with an useful boot tool.

THE SOLUTION
VMWare ESXi 4.0 installed on an USB pen drive

WHAT YOU NEED
- VMWare installation media
- 1Gig USB pen drive
- an hard disk (HD) with your favourite OS phisically installed and some free space. But IMPORTANT: no more than 3 primary partitions at this time. ESXi will need to create a further primary partition.
- set the bios in order to ask you the boot sequence (with F8) and start by default with USB

THE SEQUENCE
- turn off the workstation, disconnect the power cable, open the cabinet
- remove the HD and in general all disks
- insert a pen drive into the internal USB port
- connect the power cable and turn on the workstation
- put the media (CD or DVD) of the ESXi installation
- follow the installation steps. ESXi recognizes that there is no disk, so install itself on the pen drive. Accept this. After the installation reboot the workstation (remove the installation media when required).
- turn off the workstation, disconnect the power cable, open the cabinet
- insert the HD in the correct slot
- close the cabinet, plug the power cable and turn on the workstation
- in the boot phase you can choose to boot from USB (so, ESXi) or from MBR (so, GRUB and other OS phisically installed). Choose for now USB
- connect to the ESHi server through your vSphere client (read documentation on how to do)
- go to inventory/configuration/storage and clic on the add new storage link
- well, you will be in the wizard. You will be able to create a new datastore using the free space and without side effects with installed OSs

CONCLUSIONS
for me this is a great thing. Sun Ultra 24 is a great workstation.
My real name is not Pierre Woodman, forgive me for this, i'm italian.
Andrea

Posted by Pierre Woodman on November 18, 2009 at 07:35 PM EST #

Post a Comment:
  • HTML Syntax: NOT allowed
About

A thin thinker down under

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today