Monday Nov 12, 2007

What's cool in Opensolaris build 78?

I can think of one thing ;-)

We putback Ruby into OpenSolaris build 78! Ruby 1.8.6 patchlevel 110, Rubygems 0.9.4, and Openssl, Curses, Readline, and Tcl/Tk extensions are enabled.

The directory layout for Ruby is per the ARC case. Ruby is in /usr/ruby/, it's linked from /usr/bin so as to be easy to locate - the man page is also appropriately linked.

Same for Rubygems, it's available though a link in /usr/bin - and the default gem repository happens to be /var/ruby/1.8/gem_home (ie., $GEM_HOME).

In the mean time, the wish list has is getting bigger n bigger . . .


\*\*\*\*\*\*\*\*\*  This mail is automatically generated  \*\*\*\*\*\*\*

Your putback for the following fix(es) is complete:

    6622810 Bring Ruby, Rubygems, and Extensions into OpenSolaris.
    

These fixes will be in release:

        snv_78

The gatekeeper will mark these bugids "fixed" and "integrated" in bugtraq
as soon as the gate has been delivered to the WOS.  However, you can mark
them "fixed" (but not "integrated") now if you wish.

        Your Friendly Gatekeepers

-------------------------------------------------------------

Saturday Oct 13, 2007

rubygems for opensolaris

Rubygems seems quite easy to put into OpenSolaris. It is a good candidate for /usr/ruby(with appropriate symbolic links from /usr/bin, of course). The gem repository needs to be outside the /usr hierarchy to allow the repository to be administered by users of sparse root zones. /var/ruby/1.8 looks like a good place for this.

The question of versioning model, compatibility, and how to distinguish a stable version from not, all had extemely pleasing answers for a ardent-compatibility-supporter like yours truly :-) thanks to Eric Hodel.(this is data that I was able to incorporate into my ARC case for Ruby). It also seems that Rubygems will have their own release cycles, independent of being included in the Ruby train.

When Rubygems installs a gem with an executable, for example Mongrel, it ties the runtime to the currently installed Ruby version. ie., It uses "#!/usr/ruby/1.8/bin/ruby". and thus, Mongrel will still invoke Ruby 1.8 even when the end user upgrades to ruby 1.9.(it would be cool if Rubygems used "#!/usr/bin/env ruby" instead of hardcoding . . .)
This makes it impossible to reuse the gem repository across different versions of Ruby installed on the same machine.
So a /var/ruby/x.y/gems directory will ensure that each Ruby version has it's own repository.

Should be easy to write a program to migrate gems between these, given the gem cache directory in Rubygems.

<peeve>
Rubygems tends to complain when when invoked after being configured with the non default gem repository directory. This seems like an RFE to me, since you tell the Rubygems installation program where your repository is, why doesn't it remember?

I know, I know, setting the GEM_HOME variable will solve that. But that takes time. Maybe just a few seconds more, but time that doesn't need to be wasted for this.
</peeve>

Thursday Oct 04, 2007

Whats a production Ruby version? what's a compatible version? And how do you know?

I've recently been working on getting Ruby checked into OpenSolaris. I was trying to find out how a stable Ruby release could be identified - to understand which versions to package, when updates can overwrite previous versions, and when they should co-exist etc.,

The PickAxe and Matz's OReilly book said that odd numbered minor versions of Ruby were considered development versions. ie., <ersion>.<OddNumber-Minor-version>.<Teeny-Version> releases are not production versions.

But since I didn't see any recent mention of this, I decided to ask the source, ruby-core. The response came from none other than Matz, and a few other ruby-corers. Odd numbered minor versions no longer indicate a development release.
What does, is the absence of a point release(or the "TEENY" version field), or a point release of "0". For example, 1.9(==1.9.0) is a dev. release, and development updates to this svn branch are not numbered. Once 1.9 is deemed production ready, it will be released as 1.9.l, which will be a production release.

So, version x.y.z is a development release iff z == 0, or z is undefined; The oddness/evenness of y not withstanding.
The corollary is that version x.y.z is a production release iff z > 0.

On compatibility, release x.y.z is compatible for any value of z > 0.
Corollary: x.y.z is not compatible for x.p.z if y != p. And it _is_ compatible only if y == p.

Saturday Sep 22, 2007

Kinda hard to argue with our good deeds?

Very cool.
From RailsConf Europe, by none other then Mr. Hansson - http://www.loudthinking.com/posts/11-sun-surprises-at-railsconf-europe-2007

Friday Sep 14, 2007

Have ideas for what goes with Ruby/Rails in OpenSolaris?

If you are not interested in Ruby on Solaris, don't read further.

Else . . . help decide! You opinions and rationale are crucial to make OpenSolaris better!

What's your favorite Ruby extension or gem?
Write to the "Open Source applications on OpenSolaris" community at http://www.opensolaris.org/jive/thread.jspa?threadID=39394&tstart=0

Wednesday Sep 05, 2007

Uncooperative irb on Solaris?


 Did you just download Ruby on Solaris and realize that irb is uncooperative with history, tab completion, and line editing?

 The problem is that it needs the GNU readline extension.  Unlike ssl and curses support, the GNU Readline library is not available on Solaris/OpenSolaris yet.
 To enable this support, install GNU Readline, and then build Ruby with the appropriate library/include paths.  I've included a list of steps below to save the interested user some time . . .

(1) Download and Install Sun Studio 12.  It's pretty easy, pick it up from http://developers.sun.com/sunstudio/downloads/index.jsp.  It installs into /opt/SUNWspro
(2) Download the latest stable release of Ruby 1.8.6 from ftp://ftp.ruby-lang.org/pub/ruby/stable-snapshot.tar.gz.
(3) Download GNU Readline from ftp://ftp.cwru.edu/pub/bash/readline-5.2.tar.gz
(4) prefix your path with /opt/SUNWspro/bin.  Ensure that /usr/sfw/bin is not in your path for the next step.
(5) run ./configure in, make, and make install

root test4-sp>export PATH="/opt/SUNWspro/bin:/opt/coolstack/bin:/opt/csw/bin:/usr/sbin:/usr/bin"
root test4-sp>./configure --prefix=/export/readline/binaries
[ . . . snip . . .]
root test4-sp>make
[. . .snip . . .]
root test4-sp>make install


Don't get put off by the last message "install: you may need to run ldconfig".  ldconfig doesn't exist on Solaris.  Use good old crle to add the readline libraries into your path, if you'd like.  But it's not necessary for adding readline support to Ruby/irb.

(6) Next, note the path you installed readline into, in my case, it is /export/readline/binaries.  We'll have to ensure that the Ruby build knows about this library.  So, set your $CFLAGS variable appropriately:

root test4-sp>export CFLAGS="-xO4 -g -L/export/readline/binaries/lib/lib -I/export/readline/binaries/include/readline"

(7) Now, add /usr/local/bin(and /usr/sfw/bin too) to your path for autoconf, and proceed to build Ruby with readline enabled.

export PATH=/opt/SUNWspro/bin:/usr/local/bin:/usr/sfw/bin:/opt/coolstack/bin:/opt/csw/bin:$PATH
export cc=/opt/SUNWspro/bin
./configure --without-gcc --prefix=/export/ruby/builds/ --enable-pthread --with-readline-dir=/export/readline/binaries

(8) then run make, and make install (also make install-doc for ri to work properly).

(9) To turn on tab completion, create a .irbrc file in your home directory which contains the following(courtesy Nick Sieger's blog):

root test4-sp>more ~/.irbrc
require 'irb/completion'
ARGV.concat [ "--readline", "--prompt-mode", "simple" ]

(10) Now you're set.  Emacs style command editing in irb, history, and TAB completion of classes should work like a charm!


Friday Aug 24, 2007

Control-C Control-V? Dont! - aka "Thinking in Ruby . . ."

Code reuse doesn't mean copy-paste, of course. But there are times when you do it because you have always done things that way – and thus don't use better tools when they are available. Habit is a dangerous thing.

I had to write a program to concurrently load data into a file store(MogileFS, if you were curious). The program had to load user data, and event calendar data.

The number of processes that my loader could spawn, to upload this data into this network file system, was the constraint.

The algorithm for the loader is straight forward:

  1. -> start with M users, N events. C is the number of processes that you can optimally create to load data(hardware constraints).

  2. -> iterate C times, select M/C users with each iteration, and fork/exec a process to load the M/C users into the store.

  3. -> wait for these processes to exit(since your concurrency should not exceed C)

  4. -> iterate C times, select N/C events with each iteration, and fork/exec a process to load the N/C events into the store.

That sounds trivial enough - I want to use Ruby, and here is how the program might look(I'm leaving out irrelevant details of loading into my data store for brevity):

bash-3.00# more loader.rb
concurrency = 4
num_users = 40
num_events = 80

puts "Adding Users. . ."
count = 0
concurrency.times do
  count_new = count + num_users/concurrency - 1
  exec "/usr/bin/echo Users #{count} #{count_new} > /dev/null" if fork.nil?
  puts "range is #{count} to #{count_new}"
  count = count_new + 1
end

(1..concurrency).each do |i|
  puts "Adding users - Process id: #{Process.wait} finished. #{i}/#{concurrency} complete."
end

puts "Adding Events. . ."
count = 0
concurrency.times do
  count_new = count + num_events/concurrency - 1
  exec "/usr/bin/echo Events #{count} #{count_new} > /dev/null" if fork.nil?
  puts "range is #{count} to #{count_new}"
  count = count_new + 1
end

(1..concurrency).each do |i|
  puts "Adding events - Process id: #{Process.wait} finished. #{i}/#{concurrency} complete."
end

bash-3.00#

That's pretty cool, and we're happy with the way we use the iterators that Ruby provides. This saves us some boiler plate code. And we like not having to use the $ prefix.

But wait, there is another Rubyism that could have worked it's way into this code - the code in the 'yellow' block and the code the 'blue' block look very similar, except for a “puts Adding Users . . .” statement that becomes “puts Adding Events. . .” in the second block, and the reference to num_users in the first block which becomes num_events in the second block.

Indeed, this is because of the rather instinctive ControlC-ControlV that went with the creation of the second block. But luckily, we spotted that instinct to stick to old ways. Ruby helps minimise copy-paste operations, it's code block feature comes to the rescue! The way to remove the copy-paste operation is to recognise that the copied block does exactly the same operation, but on events, rather than users. Ruby Hashes and code blocks come to the rescue, and here is what a more Ruby-friendly implementation looks like:


bash-3.00# more loader.rb
concurrency = 4
num_users = 40
num_events =
80

sets_to_load_into_mogile =
 {
 "Users" => num_users,
 "Events" => num_events
 }

sets_to_load_into_mogile.keys.each { |set|
 #puts "key is #{set} and value is #{sets_to_load_into_mogile[set]}"

puts "Adding #{set}. . ."
count = 0
concurrency.times do
  count_new = count + sets_to_load_into_mogile[set]/concurrency - 1
  exec "/usr/bin/echo  #{set} #{count} #{count_new} > /dev/null" if fork.nil?
  puts "range is #{count} to #{count_new}"
  count = count_new + 1
end

(1..concurrency).each do |i|
  puts "Adding #{set} - Process id: #{Process.wait} finished. #{i}/#{concurrency} complete."
end
}
bash-3.00#

The code in yellow is the hash that defines what kinds of sets need to be loaded(ie., Users, and Events). The values define the cardinality of such sets.

The sets_to_load_into_mogile.keys variable is an array of keys in the hash. We iterate over these, and run the code block as many times as needed with the right customizations.

Quite simple. Once we think in Ruby. And in this case that means looking out for the tendency to use Control-C/Control-V where Ruby offers such a strong alternative.

Friday Aug 03, 2007

Compiling Ruby 1.8.6 on Solaris Nevada (or OpenSolaris, or Solaris Express ) using Sun Studio 12

One doesn't need to compile their Ruby from source on Solaris Nevada / OpenSolaris / Solaris Express (SX). It can be downloaded from Blastwave, Sunfreeware, or Cooltools.
sunfreeware.com happens to have the latest stable version(1.8.6) available.

However, if you want an optimized build for your setup, or just like to compile your programs from source, there's a problem to be aware of and avoid while building.

Assuming Sun Studio 12 is installed in /opt/SUNWspro (the dafault location), here's what happens when a compile is attempted:

bash-3.00# pwd
/export/ruby/ruby-1.8.6

bash-3.00# echo $PATH
/usr/ccs/bin:/opt/SUNWspro/bin:/usr/sbin:/usr/bin

bash-3.00# ./configure --without-gcc
checking build system type... i386-pc-solaris2.11
checking host system type... i386-pc-solaris2.11
checking target system type... i386-pc-solaris2.11

[ . . . lots more output snipped . . .]

configure: creating ./config.status
config.status: creating Makefile

bash-3.00# make
make: Warning: Illegal dependency list for target `.DEFAULT'
cc -g  -DRUBY_EXPORT -I. -I. -c array.c

[ . . . lots more output snipped . . .]

cc -g  -DRUBY_EXPORT -I. -I. -c ./missing/isinf.c
"./missing/isinf.c", line 32: syntax error before or at: __builtin_isinf
"./missing/isinf.c", line 33: warning: old-style declaration or incorrect type for: n
"./missing/isinf.c", line 33: syntax error before or at: double
"./missing/isinf.c", line 33: identifier redeclared: n
        current : double
        previous: int : "./missing/isinf.c", line 32
"./missing/isinf.c", line 35: syntax error before or at: return
cc: acomp failed for ./missing/isinf.c
\*\*\* Error code 2
make: Fatal error: Command failed for target `isinf.o'
bash-3.00#

The Makefile trys to compile ./missing/isinf.c and fails, thus aborting with a fatal build error. The good news is that this file is not needed, on Solaris, since isinf already exists as a part of libm - isinf(3M).
Why, then is this being built with the Ruby distribution? That's because Ruby's configure.in does not account for Solaris "5.11" . It only knows Solaris 10. Hence SXDE/SXCE/Nevada/OpenSolaris will not be recognized, and running make will cause isinf.c to be built(or attempted).

It's straightforward to avoid this problem. Here's how:

(1) replace the configure.in script in ruby-1.8.6/ by this modified version. The modified version adds the below lines, for Solaris Nevada in the appropriate place.

solaris\*2.11)  if test -z "$GCC"; then
                      ac_cv_func_isinf=yes
              fi
               LIBS="-lm $LIBS"
              ;;

(2) set your PATH,
export PATH=/usr/ccs/bin:/opt/SUNWspro/bin:/usr/sfw/bin:$PATH
/usr/ccs/bin is needed for make, /opt/SUNWspro/bin is the location of the studio 12 compiler, and /usr/sfw/bin is the free software directory, and has GNU Autoconf.

(3) Next, run the configure script.
./configure --without-gcc

(4) Next, run make.

 make

(make will run Autoconf again, and will throw an error and ask itself to be run again, this is normal). The error is:
config.status: creating Makefile
Makefile updated, restart.
\*\*\* Error code 1
The following command caused the error:
{ \\
            echo "all:; -@rm -f conftest.mk"; \\
            echo "conftest.mk: .force; @echo AUTO_REMAKE"; \\
            echo ".force:"; \\
        } > conftest.mk || exit 1; \\
        make -f conftest.mk | grep '\^AUTO_REMAKE$' >/dev/null 2>&1 || \\
        { echo "Makefile updated, restart."; exit 1; }
make: Fatal error: Command failed for target `Makefile'

(5) Run make again.
make
(/usr/ccs/bin/make is okay - GNU Make is not necessary). The warning "make: Warning: Illegal dependency list for target `.DEFAULT'" warning can be ignored.

(6) Finally, run "make install" - and ruby, rdoc, and irb will be installed into /usr/local/bin (or whatever prefix you specified to ./configure). Also run "make install-doc" to compile the documentation for viewing through ri.

About

prashant

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today