Wednesday Jul 11, 2007

SecondLife on SPARC

Well folks, it has been a fun couple of months. After getting the SecondLife viewer ported to Solaris x86 I thought the move to SPARC would be a breeze. NOT!!! It turns out there were quite a few CPU alignment assumptions made throughout the code that caused headaches. But before I get into that I need to put in a plug for gccfss (gcc for SPARC System) which is a great tool to use if you have a huge application like the SecondLife viewer, have a community of folks who share development duties and therefore tie you to gcc, but want superior performance on SPARC. gccfss is the gcc 4.0.x front end on top of the Sun Studio 12 backend. You therefore can compile code that assumes gcc-isms while still able to use the SPARC performance tools. I also need to plug Sun Studio 12 which has many gcc compatible features. If you own the application in question I highly advise you to invest the time in getting your code to compile in Sun Studio 12. There are far too many reasons to describe here so look at the SunStudio overview page for more details. In either case, gccfss and SunStudio 12 are free so you'd be a fool if you didn't invest the time and give them a try. SunStudio is available for Solaris and most Linuxes.

OK, now back to SecondLife. After deciding to go with gccfss, I still had data alignment to deal with. Alexey Starovoytov came to the rescue advising me to add asm("ta\\t6"); to SL's main(). This little trick enables a corrective trap handler on SPARC that will fix unaligned references. For those who need a deeper definition, John Harres provided the following:

Since this is a user land trap, it ends up in the user side of the trap table, which effectively adds 0x100 to the trap number. This ends up in:

GOTO(.fix_alignment);           /\* 106  do unaligned references \*/

in the trap table. It ends up setting p_fixalignment in the proc structure, which then is noticed during trap() if you're taking a T_SYS_RTT_ALIGN trap from user land. WIth that flag set this results in the call to do_unaligned(). Therefore, each time the program takes an alignment trap, do_unaligned() is called. One would expect the overhead to be rather minimal unless one is doing a lot of unaligned references.

If you need a source reference, Alexy pointed me to simulator.c as well as a discussion of trap 6 in the Sun Studio Performance Library doc.

I can't thank Alexy and John enough for their help.

Once I got passed the alignment problems I then ran up against Open GL Library issues on SPARC. The SL deveopers assume the GLext, originally called the GL Extensions for Linux Library, exists. Well currently on SPARC GLext does not and at the moment I've #ifdef'ed around these calls to get a clean compile and wound up making the viewer ugly. This is what I'll be attacking next.

Here's a few other things that were done to the Viewer for Solaris compatibility:

  • All the libraries required by the viewer but not bundled with Solaris were built for the SPARC using the same methods described for the x86 port. They were then copied into the linden/libraries/sparc-sunos5 directory for the SCONS build.
  • The Lindens have dedicated pages of source to figuring out the type of x86 processor is in use. Instead of using their assembler calls and because said assembler does not apply to SPARC I used getisax(2) to extract the cpu info. The cpu kstats were used to determine the processor brand and clock rate while on x86/x64, getisax(2) was used to further define which CPU extensions were in use.
  • The in window color depth was reduced from 32 to 24 bits on SPARC to allow compatibility with more of the frame buffers available on SPARC.
  • The Viewer assumes the X-Window System server. It currently references the server logs to determine how much memory is configured on the GPU. A true kludge. On SPARC is not yet supported so this code is useless and it's ignored. According to its documentation the SDL library provides such information but when one inspects the SDL source one finds the variable but discovers it's not ever set. Doh! Therefore for now, the viewer assumes the maximum amount of memory is available, the default action when it can't be determined by any other means.
  • One of the largest stumbling blocks was big endian conversions on SPARC. I spent weeks trying to band-aid my way through fixes. Then last week, as he was adding the Solaris changes to the master repository at Linden Labs, Soft Linden discovered that a header file was missing from one of their message passing sources - a LL bug that I missed. That header was the one that defined LL_BIG_ENDIAN. Once I added that include everything fell together and I proceeded to rip out my endian changes. Without Soft's help I would have wasted a ton of more time "fixing things". Thanks Soft!
  • The SL community has offered an openal alternative to the FMOD audio library. Using the patches offered for openal I was able to add audio support to Solaris. There is a rumor that there may be an FMOD available for Solaris in the near future so that option may be viable soon as well.
So where does the Solaris port stand? Except for movie support the viewer on x86 works fine. The SPARC port still needs work to address the GL compatibility issues. Like I said earlier, I'll be attacking those next.

Lastly, I got approval from Sun Legal to sign the Linden Labs Contributor Agreement and am now officially able to give LL copies of the Solaris patches. I was asked not to distribute any 3rd party libraries outside Sun so I'm hoping someone else will build a version for general distribution. Soft Linden has been working with me to merge those changes into the official SVN repository so they should be available to everyone soon.

If you are interested in building the SL viewer on your own, you should start by reviewing the pages at the SecondLife Open Source Portal.


Don't ya just love toys and taking them apart to see how they work? To me it doesn't matter if it's an iPod, a laptop, or the biggest baddest thing a company makes. And nothing makes me more happy than showing how easy it is to develop stuff on Oracle Solaris.


« April 2014