Thursday Oct 29, 2009

Ruby MySQL nearly in OpenSolaris

One of the big pain points when installing Native Ruby Gems is the need to have various build packages installed. Packages that deliver the likes of gcc, gmake and ginstall. You also need to know where the libraries and C header files that you want to build against are located. On OpenSolaris the last part should be a no-brainer at least with packages installed from the repository, but some packages such as MySQL don't install to /usr/lib and /usr/include and the mysql_config that the MySQL package ships is not on the default $PATH and even if it was, it emits Compiler and Linker information for Sun Studio, not for gcc.

What this means is that you have to install all of the tools above and then install the MySQL gem with options telling the build where to find the MySQL libs and the MySQL headers.

Making the MySQL gem available as an OpenSolaris package, means you don't have to worry about any of that. You just run:

pfexec pkg install ruby-mysql

and voila!, you have MySQL support in Ruby... But it doesn't work :o(

The ruby-mysql package was promoted to the /contrib repository this week. Unfortunately it was promoted before I had tested it fully (which shows we have some major holes in the processes used to get packages into /contrib). The version that's there currently is unusable as it causes a segmentation violation when running with Rails. If you want to use this package today then you can get it from the /pending repository. Details of how to make use of OpenSolaris repositories can be found on a separate blog entry here.

Friday Jun 19, 2009

EventMachine and the elusive emptys

Having built Ruby 1.9 packages for Solaris Nevada we had to try installing the commonly used gems, particularly those for the Rails stack and specifically the ones we use with the Apache Olio Rails app. One of these was eventmachine which is used by Thin and which is a great piece of Software. It installed fine on our Solaris Nevada b116 x64 box but failed to install on our Nevada b116 SPARC box. The error was pretty bizarre too:

usr/ccs/bin/as: error: no input filename given
usage: /usr/ccs/bin/as [-V] [-Q{y,n}] [-q] [-s]
        [-S] [-K {pic,PIC}] [-o objfile] [-L] [-T]
        [-P [[-Yc,path] [-Ipath] [-Dname] [-Dname=def] [-Uname]]...]
        [-m [-Ym,path]] [-n] [-ul] [-xF]
        [-m32] [-m64]
        [-xarch={v7,v8,v8a,v8plus,v8plusa,v8plusb,v9,v9a,v9b,sparc,sparcvis, sparcvis2,sparcfmaf,sparcima}]
        [-xcode={pic13,pic32}] file.s...
\*\*\* Error code 1
make: Fatal error: Command failed for target `binder.o'

What looked suspicious here, was the build line, which looked like this:

g++ -I. -I/usr/ruby/1.9/include/ruby-1.9.1/sparc-solaris2.11 -I/usr/ruby/1.9/include/ruby-1.9.1/ruby/backward -I/usr/ruby/1.9/include/ruby-1.9.1 -I/var/ruby/1.9/gem_home/gems/eventmachine-0.12.8/ext -DBUILD_FOR_RUBY -DHAVE_RB_THREAD_BLOCKING_REGION -DHAVE_TBR -DHAVE_WRITEV -DHAVE_WRITEV -DOS_UNIX -DOS_SOLARIS8 -DWITH_SSL -I/usr/sfw/include -fPIC -g -O3 -fPIC  -g -obinder.o -c binder.cpp

There's no space in '-obinder.o' which you'd normally expect to read '-o binder.o'. It looked odd, though it's not unusual for spaces to be ommited when passing names to options, I'd just never seen it on a build line before so it stood out. Looking at the output with -v set as one of the flags passed to g++, gave more data but not anything useful. We looked at it on the x64 system (again with -v as a flag to g++) and it went through the same steps but didn't fail.  But it did use a different assembler! gcc/g++ on x64 uses /usr/sfw/bin/gas but on sparc uses /usr/ccs/bin/as. It also became apparent by running the assembler manually that it was choking on -obinder.o, it seems that gas is more tolerant than as :o(

The Makefile that gets generated by extconf.rb when you install the gem has lines in it's .SUFFIXES section of the form:

        $(CXX) $(INCFLAGS) $(CPPFLAGS) $(CXXFLAGS) $(COUTFLAG)$@ -c $<

and earlier in the file COUTFLAG is defined as:


Which ultimately results in -obindir.o (same for all of the other objects it builds).

Matz seems to have hit this problem before, because the Ruby 1.9.1 Makefile contains a fix for a similar problem in Ruby (although it still happens when building the ext/\*.c sources)

empty =
OUTFLAG = -o $(empty)
COUTFLAG = -o $(empty)

Which results in a space being inserted after the '-o' for $(OUTFLAG) and for $(COUTFLAG)


The Makefile is of course generated by extconf.rb which gets it's instructions on how to create the Makefile from mkmf.rb which Ruby provides. So the fix would be to make the changes outlined above in lib/ruby/1.9.1/mkmf.rb or to have it create the SUFFIXES lines with a space between the COUTFLAG variable and the output file name. We can do that for any packages that we build, but we also need to raise a bug with Ruby.

Friday Oct 03, 2008

Segmentation Fault when running Rails with MySQL on Solaris Nevada

We found in recent testing while integrating Ruby 1.8.6 Patch Level 287 into Solaris Nevada that it's possible to create a situation where Ruby on Rails applications will sometimes crash with a Segmentation Fault. We found that the MySQL native gem when installed, had been built with the mysql.h C header file from MySQL 5.0 but had been linked with an older version of the MySQL client library. This comes about if you install the gem as follows:

gem install mysql -- --with-mysql-dir=/usr/mysql/5.0

In this case the compiler is able to find the C headers in /usr/mysql/5.0/include but because the client library is actually located in /usr/mysql/5.0/lib/mysql/lib, the linker isn't able to find it. As we include /usr/sfw/lib in the build and runtime linker paths for native Ruby Gems, it is able to find the MySQL 4.0 client library and at runtime it links with that. Not surprisingly the MySQL 5.0 C header won't work with the MySQL 4.0 client library, what is surprising is that it works at all and doesn't crash as soon as you try to access MySQL from Ruby.

The solution is to install the gem without specifying any flags, in which case it will compile and link with the MySQL4.0  client, or preferably to specify the path to the include and lib directories of MySQL 5.0. So use either of the following:

gem install mysql -- --with-mysql-lib=/usr/mysql/5.0/lib/mysql -- with-mysql-include=/usr/mysql/5.0/include

gem install mysql

This is not a problem on OpenSolaris as the MySQL 4.0 client library is not there by default, just remember that just using --with-mysql-dir= may not always be enough information for the compiler and linker to do the right thing.


Bloggity, blog


« July 2016