Thursday Oct 29, 2009

Ruby MySQL nearly in OpenSolaris

One of the big pain points when installing Native Ruby Gems is the need to have various build packages installed. Packages that deliver the likes of gcc, gmake and ginstall. You also need to know where the libraries and C header files that you want to build against are located. On OpenSolaris the last part should be a no-brainer at least with packages installed from the repository, but some packages such as MySQL don't install to /usr/lib and /usr/include and the mysql_config that the MySQL package ships is not on the default $PATH and even if it was, it emits Compiler and Linker information for Sun Studio, not for gcc.

What this means is that you have to install all of the tools above and then install the MySQL gem with options telling the build where to find the MySQL libs and the MySQL headers.

Making the MySQL gem available as an OpenSolaris package, means you don't have to worry about any of that. You just run:

pfexec pkg install ruby-mysql

and voila!, you have MySQL support in Ruby... But it doesn't work :o(

The ruby-mysql package was promoted to the /contrib repository this week. Unfortunately it was promoted before I had tested it fully (which shows we have some major holes in the processes used to get packages into /contrib). The version that's there currently is unusable as it causes a segmentation violation when running with Rails. If you want to use this package today then you can get it from the /pending repository. Details of how to make use of OpenSolaris repositories can be found on a separate blog entry here.

Tuesday Oct 14, 2008

Lighttpd on CMT running Web 2.0 workloads

Earlier in the year I spent many a night burning candles at both ends testing with a Web 2.0 workload on a Lighttpd/PHP/Memcached/MySQL stack. The results have been made into a Sun Blueprint entitled An Open Source Web Solution - Lighttpd Web Server and Chip Multithreading Technology The bottom line is that we were able to get 416 ops/sec with 2250 users before we ran out of network bandwidth. That was with a 5 second cycle time between operations, so 450 ops/sec would have been the best we could theoretically get. This was all done on a Sun SPARC Enterprise T5120 running with 8 cores (64 threads) at 1.2GHz. In reflection I wish I'd spent more time on Memcached and MySQL analysis, but we wasted a lot of cycles trying to team a couple of network cards using a switch that was not up to the job and we just ran out of time.

The keys to all of the testing were the Faban test harness which not only ran all of the tests for us but collected and presented most of the data used in the Blueprint Document, and the Web 2.0 kit which is now in incubation on as Project Olio (

Friday Oct 03, 2008

Lighttpd in OpenSolaris

Lighttpd went into the SFW consolidation 6 weeks ago and made it into build 97 of Solaris Nevada (which sees the light of day as Solaris Express Community Edition). It appeared in the OpenSolaris package repository in the first week of September and you can install Lighttpd using the following command:

pfexec pkg install SUNWlighttpd14

if it doesn't find it try running: pfexec pkg refresh and then try it again.

BTW: You don't need to use the pfexec part of the commands if you are logged in as root

I've not blogged about this before because I tried to be clever with dependencies and built the Lighttpd MySQL vhost module against the MySQL 5.0 version that's intergrated into SFW but I deliberately didn't call out the dependency in the packages. This should have meant that when you installed Lighttpd, it wouldn't pull down MySQL, but once you had installed the MySQL packages (if you chose to) then MySQL vhost support would just work. The missing library dependency was spotted by the Gatekeeper and he added the MySQL dependency to the packages by hand. So now when you install Lighttpd, you get MySQL too. Sorry about that.

We have a CR open for this and in the next couple of weeks we'll integrate a fix that packages MySQL support separately. If in the meantime I come up with a way of bypassing the dependency check I'll blog it.

In the meantime, give it a try and if you find anything that doesn't work the way it should or if there's some change you'd like to suggest, reply to this blog entry or join the webstack-discuss alias at and send us an email. 

BTW: We don't have lua support at the moment so the likes of mod_magnet won't work, we are hoping to fix this in the near future

Segmentation Fault when running Rails with MySQL on Solaris Nevada

We found in recent testing while integrating Ruby 1.8.6 Patch Level 287 into Solaris Nevada that it's possible to create a situation where Ruby on Rails applications will sometimes crash with a Segmentation Fault. We found that the MySQL native gem when installed, had been built with the mysql.h C header file from MySQL 5.0 but had been linked with an older version of the MySQL client library. This comes about if you install the gem as follows:

gem install mysql -- --with-mysql-dir=/usr/mysql/5.0

In this case the compiler is able to find the C headers in /usr/mysql/5.0/include but because the client library is actually located in /usr/mysql/5.0/lib/mysql/lib, the linker isn't able to find it. As we include /usr/sfw/lib in the build and runtime linker paths for native Ruby Gems, it is able to find the MySQL 4.0 client library and at runtime it links with that. Not surprisingly the MySQL 5.0 C header won't work with the MySQL 4.0 client library, what is surprising is that it works at all and doesn't crash as soon as you try to access MySQL from Ruby.

The solution is to install the gem without specifying any flags, in which case it will compile and link with the MySQL4.0  client, or preferably to specify the path to the include and lib directories of MySQL 5.0. So use either of the following:

gem install mysql -- --with-mysql-lib=/usr/mysql/5.0/lib/mysql -- with-mysql-include=/usr/mysql/5.0/include

gem install mysql

This is not a problem on OpenSolaris as the MySQL 4.0 client library is not there by default, just remember that just using --with-mysql-dir= may not always be enough information for the compiler and linker to do the right thing.


Bloggity, blog


« July 2016