Friday Jun 19, 2009

Typo in OpenSolaris RoR AMI on Amazon EC2

We released a new OpenSolaris Ruby on Rails image on Amazon EC2 last week.  This version is different from the previous one on OpenSolaris because not only does this AMI include the usual goodies, updates etc., this is also a Typo appliance. 

Want some marketing key words?

  • Optimized
  • Secure
  • Pre-configured, preinstalled, and very "appliancy"
  • Extremely monitorable

Since I'm not really a marketing guy, I just can't put words out there without feeling a primordial urge to explain myself.


The stack (Ruby, extensions, gems, web servers) is compiled with the latest compilers, optimal compiler flags based on performance studies, and Nginx is configured to have its temporary file store on a tmpfs mount.


Nothing runs as root, even for an instruction.  If you're wondering how Nginx can listen at port 80 without being started as root, the answer is easy - Solaris privileges:  root is not a user, rather it's a role.  So you can confer privileges (like listening on port 80) which are usually associated with root to other users (like the nginx user).  So this obviates the necessity for applications(like Nginx or Apache) to run as root even for the brief set of instructions needed to bind to a port.  The less instructions you run as root, the less your exposure.

Pre-configured, pre-installed, and very "appliancy".

The image contains a production ready configuration of Typo, which is the popular Rails blog engine(usage details are here[1]).  Its architecture: there are two Nginx workers.  Nginx listens at port 80 and is a fast web server which reduces the possibility of application bottlenecks.  In all likelihood, one cannot fully stress two Nginx workers in an EC2 instance (or even one, in my opinion) In addition, Nginx has a low memory footprint, so the benign effect of having two instances outweighs the overhead). 

The Rails servers are powered by Thin.  There are two instances that Nginx communicates with.  Rails is a CPU-intensive/low concurrency application that benefits from multiple instances.  The Thins are sized in anticipation of a single CPU so it may be wise to increase the number of Thin instances depending on the number of CPUs in the image.

MySQL is the database.  The three tiers are pre-configured.  Nginx communicates with Thin, and Thin communicates with the database.  Typo has its schema populated into the MySQL database, Rails and its dependencies are already installed.  The native gems, like MySQL and Thin have been compiled and configured correctly . . . and it "just works".  To use it start the instance, and log on to http://instance to begin setting up your web log.

SMF services exist for the Nginx, Thin, and MySQL components, and the dependencies between these services are also expressed in the manifests.

The catalog entry has information on where the service manifests are stored, and where the components are installed for one who wants further control.

Extremely monitorable

The AMI has DTrace support.  DTrace provides for a low overhead profiling with fine-grained control over which code is being profiled.  Please refer to Reference 2 below[2].

What's not in the AMI:  one needs a backup mechanism for their weblog entries, this is something that the AMI does not provide at this time. 

And that's it, please check out the AMI, and hope it is useful either in itself, as a starting point for an tiered architecture that can benefit from an SMF integration, and/or as a base Ruby on Rails AMI that can be used for building a private stack.




Wednesday Apr 29, 2009

SMF manifest for Nginx

Here's an SMF manifest for Nginx.  You'll need to tweak it for your location of Nginx.  You can create an nginx user and group to run the web server as(highly recommended), or tweak my script to run as root(please don't do this).

The steps?

  1. Download the SMF from here
  2. groupadd nginx
  3. useradd -d /export/home/nginx -g nginx nginx
  4. change all occurrances of "/usr/nginx/sbin/nginx" to point to where ever your nginx is installed.
  5. You may remove the -c option in the nginx invocation if you do not want to place your nginx config file in that directory(or just want to use the default one).
  6. next, import the SMF script.  svccfg import nginx.smf
  7. run svcs -vx to make sure everything went well. It should have, if you followed all the steps above.( else you'll be pointed to a log file for further debugging )
  8. You are set.
  9. Optional:  add service dependendies on the mysql and thin/mongrel services on your system since nginx probably won't be of much use if you don't have those running.

Monday Apr 27, 2009

Git is in OpenSolaris-dev.

I just noticed that git is available in the OpenSolaris dev repository.  No more compiling from source.  Very cool.  All the volunteer work done to spruce up FOSS support in the OSOL community is paying off. 

You can either get it by running "pkg image-update" on your OpenSolaris instance, or wait for the Ruby on Rails EC2 image which I'm creating right now, to get a preconfigured set of tools, gems, web servers etc.,

Friday Apr 17, 2009

To update Ruby, or any other application on OpenSolaris . . .

My recent post about updating Ruby on OpenSolaris can be summed up with the following steps:

(1) set the OpenSolaris developer repository as the package authority(pkg set-authority -O

(2) run "pkg image-update".

This raised questions about whether an image update was necessary just to update Ruby, which is a well behaved application with no direct ties to operating system internals.

The answer, quite simply, is a YES.  This applies to any application that is provided by  the main OpenSolaris (or dev) repositories, not just Ruby.  One can only update the application to a level of the package that is delivered on the build of the OS that one is running. 

Does that sound confusing?

 Lets say your OS level is 101 ( use "uname -a" to find that out).

root@domU-12-31-39-02-F5-56:/var/ruby/1.8/gem_home/bin# uname -a
SunOS domU-12-31-39-02-F5-56 5.11 snv_101b i86pc i386 i86xpv Solaris

Now lets look at an application installed on this machine (gmake, for example).  Use "pkg info" to get information about an installed package.

root@domU-12-31-39-02-F5-56:/var/ruby/1.8/gem_home/bin# pkg info SUNWgmake
          Name: SUNWgmake
       Summary: gmake - GNU make
      Category: Development/GNU
         State: Installed
       Version: 3.81
 Build Release: 5.11
        Branch: 0.101
Packaging Date: Wed Nov 19 22:11:01 2008
          Size: 696.26 kB
          FMRI: pkg:/SUNWgmake@3.81,5.11-0.101:20081119T221101Z

gmake 3.81 is installed, and this was built on build 101 of Solaris Nevada (build 101 is the value that corresponds to the "Branch" tag).

Now, what this means is that one will not be able to update gmake to a version higher than the one bundled with build 101 without running "pkg image-update".  In this case, that is gmake version 3.81.

Updating individual packages is possible with the IPS framework, and that means that non Sun packages can be updated.  The packages that are delivered by Sun, however, cannot be updated thus.  These packages are listed as dependencies for the meta-package called "entire". "entire" imposes the restriction on what build number a package could be updated to. 

The IPS team is working on loosening this (reboot required) restriction.   But until then, please point your pkg command to  (by uttering pkg set-authority -O ) and then say " pkg image-update ".

This will bring up Ruby 1.8.7p72 and Rubygems 1.3.1.  There is work happening to make Rails available too. stay tuned.

Tuesday Mar 24, 2009

installing ImageScience an art?

ImageScience is a pretty cool gem to use in your Rails application for Thumbnail generation.  It doesn't install correctly out of the box since ImageScience uses an incorrect default path for locating FreeImage. The Makefile shipped with FreeImage also does not build correctly on OpenSolaris.  So here's a way to make sure it works correctly first time if you are on OpenSolaris.

Step1:  Update your Ruby package.  We now package Ruby 1.8.7 p72 with OpenSolaris.  Here is a blog entry I wrote sometime ago on how to upgrade to 1.8.7p72 ->

Step2: Ensure that /usr/gnu/bin is prepended to your PATH environment variable. (or, just utter export PATH=/usr/gnu/bin:$PATH in bash).

Step3: Install the requisite OpenSolaris packages.

pfexec pkg install SUNWgmake

pfexec pkg install SUNWgnu-coreutils

The "pfexec" is there to give you enough privileges to install the packages.  If you run as root(which I do - and which is not recommended), then you don't need to use pfexec.

Step4: Build FreeImage. Download it from and unzip it.  The Makefile that ships with FreeImage does not build correctly on OpenSolaris, so here is a Makefile that does ->  .

Change directory into the exploded FreeImage source, and utter the following.

gmake -f Makefile.opensolaris

gmake -f Makefile.opensolaris install

Where Makefile.opensolaris is the OpenSolaris Makefile that you just downloaded.

Step5:  The rbconfig.rb file in your Ruby installation needs to be changed.  Edit this file(it's to be found in /usr/ruby/1.8/lib/ruby/1.8/i386-solaris2.11 )

Look for a line that goes thus:


 and replace it with


i.e., change the linker binary such that the GNU Linker is used by RubyInline(which is an ImageScience dependency).

This is a bug that will be fixed in build111.  So if you have a newer version, this bug might already have been fixed.

Step6: Now install ImageScience the usual way. 

gem install image_science

Step7: You should now be set.  But, just to make sure, run the ImageScience tests and make sure they pass.  Here is how to do that.

Edit $GEM_HOME/gems/image_science-1.1.3 test/test_image_science.rb (if you use the default gem home on OpenSolaris the file is to /var/ruby/1.8/gem_home/gems/image_science-1.1.3 test/test_image_science.rb ).

Add the following line to the beginning of the file.

 require 'rubygems'

Now test it by executing:

cd /var/ruby/1.8/gem_home/gems/image_science-1.1.3

(or cd $GEM_HOME/gems/image_science-1.1.3 if your gem home is different).

ruby test/test_image_science.rb

 Loaded suite test/test_image_science
Finished in 0.01314 seconds.

7 tests, 28 assertions, 0 failures, 0 errors

And thats all it takes to have a working ImageScience library.

Tuesday Feb 10, 2009

Sun Web Stack is now called Sun GlassFish Web Stack.

Nothing else changes - the software that one usually expects to find here, like Ruby, Apache, PHP, MySQL, Memcached and more are still there, and will continue to be. actively developed on and enhanced.

Wednesday Feb 04, 2009

How does one update Ruby in OpenSolaris?

  The package manager UI(or it's command line alternative "pkg" ) in OpenSolaris is the right way to update installed programs.  However, in 2008.11 (and onward) one may find that the package manager does not show that updates are available.  This is because of how the package repositories for OpenSolaris are structured.

 The package manager by default uses the package repository at   This repository is frozen(this is not completely true, since we're allowed to put security patches into the release repository after it's been 'frozen', but new features and other bug fixes are not allowed here) at the time of an OpenSolaris release and is updated only when the next official release occurs.  The 6 + builds that occur before the next freeze(for the next release) are not reflected in this repository. 

 It's a pity - with OpenSolaris, you don't have to worry about build instabilities, crashes etc., given all the good quality assurance that's built into the system - so one might(and probably would) prefer to stay on the latest build without giving up on stability. 

 But have no fear, there is an option for such users - the dev repository.  Please set  as one of your repositories(in the package manger tool).  In fact, I also set it to be my primary repository.  This way you can stay up to date, without waiting for the next official release; the dev repository is updated with every build.  So expect an update every two weeks(approximately).

To update the installed Ruby, click on the "Update all" button - this will download a wad of packages to update your system with.  The update takes a while, but this happens in the background while you continue working.  For other programs you may be able to get away with updating only the concerned package.

Sunday Dec 14, 2008

The curious case of the Ruby T-Square operator.

 The "||=" operator is interesting, both in what it does as much as in how it's widely used in Ruby land.  The operator does not do what you would usually expect.  i.e.,

a ||= expr

is not the same as

a = a || expr

The evaluation happens to be 

a or a = expr

 and the difference is important in at least one use case [0]

As a little DTrace script will verify, this operator is not implemented is a method(or anywhere in Ruby land) and is intrinsic to the VM.   The reason is performance, and the fact that the entire expression does not have to be evaluated to yield a result when you're 'OR'ing:

"Ruby's Boolean operators are built into the language and are not based on methods: classes, for example, cannot define their own && method.  Ruby defines special true and false values but does not have a Boolean type. method. The reason for this is that Boolean operators can be applied to any value and must behave consistently for any kind of operand."

. . .

"Another reason that Ruby's Boolean operators are a core part of the language rather than redefinable methods is that the binary operators are "short-circuiting." If the value of the operation is completely determined by the lefthand operand, then the righthand operand is ignored and is never even evaluated."

 (From The Ruby Programming Language, 1st Edition by David Flanagan and Yukihiro Matsumoto.)

Assuming that variable "a" was not defined prior, then a = a || expr works a little differently than it would if the rvalue was assigned to an lvalue other than a.  That is, if a is not defined, then expr is assigned to a.  If however, a was not present in the lvalue and the rvalue looked the same(say the code looked like b = a || expr ), then the rvalue will fail to evaluate since "a" is not defined.

>> b = a || "Foo"
NameError: undefined local variable or method `a' for main:Object
        from (irb):2:in `binding'
        from C:/Program Files/NetBeans 6.5/ruby2/jruby-1.1.4/lib/ruby/1.8/irb.rb:150:in `eval_input'
        . . .
>> a = a || "Foo"
=> "Foo"

This special operator finds popular usage not in logical "OR"s, but in variable assignment.  It's a Rubyism to instantiate instance variables in methods other than constructors.  

From mongrel_rails (version 1.1.5):

    def config_keys
      @config_keys ||=
        %w(address host port cwd log_file pid_file environment docroot mime_map daemon debug includes config_script num_processors timeout throttle user group prefix)

This deferred assignment is useful in general if your constructor contains many complex instantiations that need not be performed during object construction.  But in this case, it's just a Rubyism.  It's called lazy initialization[1].  There's also an easy patch to make that work transparently if you really like it[2].

There is another occasion to use this Rubyism:  when your method accepts a parameter with a default value that needs to be populated with a method call.  From Rails 1.1:

  def install(method=nil, options = {})
    method ||= rails_env.best_install_method?

It'd clutter the method definition line to have rails_env.best_install_method? all in the same place, and hence is a good way to produce readable code.

This Rubyism is not usable when nil or false are legal values for the variable being thus assigned.

 "For the purposes of all Boolean operators, the values false and nil are considered false. And every other value, including true, 0, NaN, "", [], and {}, is considered true."

 (From The Ruby Programming Language, 1st Edition by David Flanagan and Yukihiro Matsumoto.)

There's another interesting usage, to avoid the "nasty nils", as pointed out in the below comment by Mark Wilden, and here[4]

If you're too young to know what a T-Square is, see [3]






A simple T-Square

Monday Nov 03, 2008

Ruby, concurrency, and --enable-pthread

 Ruby is a green threaded application.  ie., irrespective of how many threads are running in the Ruby VM, there is only one operating system thread that the Ruby interpreter is scheduled on.  Needless to say, this seriously limits the concurrency achievable with one Ruby VM - and quite naturally drives deployments to operate herds of web servers for their application, front ended by load balancers(like Pen, Pound etc.,) and deployed in an automated fashion(with the likes of Capistrano) - JRuby is way better at threading, though not as good as one would expect(see comments below for a clarification).

 So why use --enable-pthread to configure/build Ruby?  

 Probably because building Ruby with support for a threaded extension(like Tcl/Tk on Solaris, which is compiled with support for pthreads) gives out a dire warning that compiling a threaded extension with a non-pthreaded Ruby will cause frequent crashes?  Now that could potentially happen if the threaded extension changed the interpreters context and left it in an inconsistent state, but in practice that turns out not to be true(write a threaded extension and run it through a Ruby interpreter that is not pthread enabled to verify).  

 Using --enable-pthread has performance ramifications.  In Ruby 1.8.6(which is still the sweet spot version in terms of security / stability) removing this option gives about a 2x boost in performance - this is due to the elimination of many getcontext and setcontext calls in the interpreter - these high overhead system calls are not used in the non-pthread enabled code, and avoiding pthreads does not impose a concurrency issue since Ruby is green threaded.  further, using --enable-pthread increases the size of Ruby's stack to the point that the compiler has to use a more complicated algorithm(as opposed to simply decrementing the stack pointer) for alloca, further increasing the interpreters overhead. 

 It's a good practice to avoid using --enable-pthread for Ruby 1.8.6.

Wednesday Oct 08, 2008

Correct way to install the MySQL gem -

A note by Mandy on the right way to install the MySQL gem on OpenSolaris -

Wednesday Sep 03, 2008

"no kex alg" on recent Nevada builds

I've begun to see "no kex alg" messages on trying to ssh into machines that were jumpstarted to the later Nevada/OpenSolaris builds.  After some googling, it seems that the OpenSolaris folk are working on how to get this work better. In the mean time, here is how I generate ssh keys on a machine where sshd doesn't accept my connection because it's ssh keys haven't been generated.

bash-3.2# ssh dn02
no kex alg
bash-3.2# rlogin dn02  #or get in through the service processor.
Last login: Wed Sep  3 12:39:03 from dn01
Sun Microsystems Inc.   SunOS 5.11      snv_96  November 2008
# bash
bash-3.2# /lib/svc/method/sshd -c
bash-3.2# svcadm refresh ssh
bash-3.2# exit
# Connection to dn02 closed.
bash-3.2# ssh dn02
The authenticity of host 'dn02 (' can't be established.
RSA key fingerprint is 0b:ec:fe:85:51:82:5e:df:c0:44:10:d3:79:67:49:ea.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'dn02,' (RSA) to the list of known hosts.
Last login: Wed Sep  3 12:42:00 2008 from dn01
Sun Microsystems Inc.   SunOS 5.11      snv_96  November 2008

see and

It's apparently fixed in OpenSolaris LiveCD - but the fix probably hasn't made it into Nevada, even though the diffs posted in the bug entry are available on my machine.

Friday Aug 22, 2008

Rails2/Ruby/OpenSolaris on the Cloud

We just put up a Rails2 image running OpenSolaris on EC2.

This is the real deal, Rails2, Ruby with DTrace support, Thin, Git, Mongrel, Mongrel cluster, sample SMF scripts and configured for a sample Rails application, OpenSolaris with ZFS on the root disk,  MySQL, and Postgres.

OpenSolaris includes the Image Packaging System(IPS) which makes it trivial to add more software to the system.(for example, need AMP? just utter "pkg install amp-dev" )

It's available here

Saturday Jun 07, 2008

On allocas

The alloca call is used when one would like to dynamically allocate memory within function scope. Such memory is reclaimed after the function call returns, hence obviating the necessity of explicitly freeing the memory.
On SPARC, alloca is a macro, defined in alloca.h.

     55 #if defined(__BUILTIN_VA_ARG_INCR) || \\
     56 	defined(__sparc) || defined(__i386) || defined(__amd64)
     57 #define	alloca(x)	__builtin_alloca(x)

The compiler, when it pre-processes a source file containing an alloca call, replaces it with a call to __builtin_alloca(invoking a "cc -P" generates pre-processed source code into a filename.i, in Sun Studio).

After the file is compiled, however, the allocas may not show up when DTrace is used to profile the application. This is because the compiler generates inline assembly for the __builtin_alloca call. This happens even if inlining is disabled using the "-xinline=" compiler option. All that the alloca implementation needs is to decrement the stack pointer by the number of bytes allocated by the alloca call, and the code to do this is generated by the compiler in place of the alloca call.(there is an exception to this simple algorithm, which is documented by Darryl)
For example,

     #include <stdio.h>
     #include <alloca.h>

     void main(void)
       void \* ptr = alloca(262144);

is compiled into an object, which on disassembly(which can be done using the er_src command with Sun Studio) looks like
Annotated disassembly
Source file: ./allocate.c
Object file: ./allocate
Load Object: ./allocate

     1. #include <stdio.h>
     2. #include <alloca.h>
     4. void main(void)
     5. {
        [5]    10b70:  save        %sp, -104, %sp
     6.   void \* ptr = alloca(262144);
        [6]    10b74:  sethi       %hi(0x40000), %o0
        [6]    10b78:  sub         %sp, %o0, %sp
        [6]    10b7c:  ret         
        [6]    10b80:  restore     %g0, 0, %g0
     7. }

The save and restore calls are made on entry and exit from main, they obtain a fresh set of registers for the main routine, using a SPARC hardware feature called register windows. The alloca call is broken into the sethi and sub instructions. sethi sets the most significant 22 bits of register o0 with the 22 most significant bits of the hex value 0x40000(which is what the %hi achieves). the sub call then subtracts 0x40000(or 262144 in decimal), which is stored in register o0, from the stack pointer(-xO4 optimization was used to compile this code).

The alloca code is inlined in the object, hence a call to alloca(or __builtin_alloca) will not show up through DTrace or nm(unless the code falls into the exception category mentioned above).

A good way to trace this is to use the collector(and analyzer) or SPOT.
It helps if the binary is compiled using "-g -xO4 -xbinopt=prepare". Using -g does not reduce performance if -xO4 or higher is used, and -xbinopt=prepare does not affect performance.

Monday Jun 02, 2008

Where's my Ruby?

OpenSolaris is different from Solaris Expresses(ie., the SXCE and SXDEs) in that most of the software that comprises this release is not distributed in the CD media(or image that can be downloaded from

There's just one CD, and rest of the packages that comprise OpenSolaris are hosted on, and available through the "/usr/bin/pkg" command.

Ruby is not on the base CD image, but to have a usable version of Ruby on OpenSolaris 2008.05 is very easy, just issue the following commands.

-bash-3.2#pkg install SUNWruby18


Completed 1/1 1546/1546 8.50/8.50


Install Phase 1844/1844

-bash-3.2# which ruby


-bash-3.2#pkg install SUNWgcc


Completed 4/4 2035/2035 88.13/88.13


Install Phase 2457/2457

-bash-3.2#cd /usr/ruby/1.8/lib/ruby/1.8/i386-solaris2.11

-bash-3.2#mv rbconfig.rb rbconfig.rb.orig


After this point, one is set to install Ruby applications with C extensions . . . like Mongrel or Thin.

Monday May 19, 2008

How to install Mongrel, Mysql, and Postgres gems in OpenSolaris Ruby -

Rubygems is the preferred package manager for Ruby, and Ruby packages are called gems. While most of the gems out there are written in Ruby, there are a few that have C or C++ code in them for performance(like Fastthread for Mongrel), or Features(like the MySQL connector for Ruby).
Such gems need a C/C++ compiler to successfully be installed. Apart from that, they also need an appropriately configured rbconfig.rb file in the Ruby installation.
The rbconfig.rb file is typically present in lib/ruby/site_ruby/1.8/rubygems.rb - on OpenSolaris, it is present as /usr/ruby/1.8/lib/ruby/site_ruby/1.8/rubygems.rb
The rbconfig.rb file contains, among other things, information about which compiler(ie., the location) was used to build Ruby, compiler flags, linker flags, path to the C++ compiler, location of ginstall etc.,
This is strictly not necessary for C applications, ie., you can install a native gem compiled with CompilerB on a version of Ruby that was compiled with CompilerA, but the advantage of having the build time compiler flags captured is that you can optimize the extensions just as the Ruby binary was optimized(and the folks who built Ruby for you hopefully have done some reasearch in this area).
On OpenSolaris, there's a bug by which some paths in the rbconfig.rb file actually point to a compiler location that was used by the build infrastructure, and this location is normally not present on the system.
This is a bug that is being fixed, and the right paths should work out of the box. Until that fix shows up in the repository, here's how to make sure your Ruby installation works well.

If you have Sun Studio installed on the machine, use this rbconfig.rb to use Studio(it assumes you've installed Studio into /opt/SUNWspro - which is where it goes into, by default).

If you prefer gcc, use this rbconfig.rb instead (it assumes gcc is in /usr/sfw/bin/gcc - which is a symlink from /usr/bin/gcc).

After you've replaced the original rbconfig.rb file, installing native gems should be easy.
If you used the gcc version, then all you have to do is utter "gem install mysql -- --with-mysql-dir=/usr/mysql/5.0", "gem install postgres", or "gem install mongrel" to install these packages.
Note that the "--with-mysql-dir" option is necessary. OpenSolaris has two versions of MySQL, the ancient version(4.x.y) is in the default link path, which you don't want to link to, MySQL 5.x.y is in /usr/mysql/5.0

Also, I assumed that Sun Studio is in it's default path of /opt/SUNWspro. The rbconfig.rb files are for Solaris running on an x86/x64 platform. If you're using SPARC, modify your file with appropriate values for INSTALL, CC, CFLAGS, CPPFLAGS, LDFLAGS, and CPP from my files.




« July 2016