Friday Sep 25, 2009

Rails: Setting selected options in a ListBox


I really struggled with this yesterday. I'm writing a Rails CRUD app and when creating a new entry one of the fields presented is a ListBox that allows multiple selections. That was straightforward, but i had problems when editing an existing entry. What didn't want to work for me are the recognised methods for having the current items for the entry being edited be selected in the ListBox.

The way that I understand that it should work is like this:

collection_select(:user, :group_ids, Group.find(:all), :id, :name,{},{:multiple=>,:name=>'user[group_ids][]'})

But needless to say it didn't. In the end I found this from Wes Gamble (weyus on Ruby Forum). It's a bit more DIY but it works a treat. I'm reproducing it here in case it's useful to anyone else (thanks Wes!)

<select id="user_roles" name="roles[]" multiple="multiple">
<% Role.find(:all).each do |r| %>
     <option value="<%= %>" <%= @user.roles.include?(r) ? 
"selected=\\"selected\\"" : '' %>"><%= %></option>
<% end %>

Rails 2.3.3 in OpenSolaris /contrib repo

A while back an OpenSolaris repository called /contrib was introduced to basically allow anyone with an account on to submit Open Source applications/libraries to an OpenSolaris repo. The details of how to go about doing this are best saved for another blog entry (add to ToDo list). This new repository gave us a real opportunity to package up Ruby on Rails and make it available to OpenSolaris users in the package format that they are familiar with. We'd avoided the main /release and /dev repositories because Rails rev'd too frequently for us to be able to keep up with the changes given that the processes for getting into those repositories are fairly lengthy. Besides there was always the 'gem install' command.

So we've now added Ruby on Rails 2.3.3 and it's dependent Gems to the /contrib repository. Since then Rails 2.3.4 has been released, more on that later.

To install the rails package on a system with OpenSolaris 2009.06 or later you need to run the following commands:

% pfexec pkg set-publisher -O  contrib

% pfexec pkg refresh

% pfexec pkg install ruby-rails

% pfexec gem install rack

The first command gives you access to the /contrib repository and all of the packages that it contains and is worth doing anyway as there's lots of great packages there besides Ruby on Rails. It's a one off command, once you've added /contrib it will remain in your list of package publishers. The second command isn't strictly necessary the first time, but it's useful to remember as /contrib updates on a weekly basis and your local cache of package names and versions doesn't refresh automatically.

The next command installs the 'ruby-rails' package and if it's not already installed, installs the SUNWruby18 package which contains Ruby 1.8.7 and RubyGems 1.3.1 (versions correct at the time of writing). The last command installs rack, which is now required by Rails, something we didn't find out until the last moment we moved to Rails 2.3.3. We are looking to package rack and add it as a dependency to the ruby-rails package. The ruby-rails package installs to what is effectively the vendor gem location in /usr and the rails and rake executables install to /usr/bin (as symbolic links). You can still use the gem command to install Ruby on Rails versions later than 2.3.3 if you need to and the /usr/bin/rails command will pick up the later Rails version.

At the moment the benefits of using 'pkg' to install Rails over using 'gem' are mainly just the convenience of having dependencies pulled in automatically and having the 'rails' command on the PATH. In the future, we'll add native gems that are used to provide infrastructure to Ruby on Rails, gems that usually require a compiler to build, along with knowledge of the location of any dependent libraries. Ultimately we'll be able to provide a single package that installs a complete, optimised, Ruby on Rails infrastructure.

Thursday Jul 09, 2009

Rack and content_type for multipart requests

I wrote this entry a while back but never posted it. Seems a shame to waste it.

 While updating the Apache Olio app to do caching we found that on the new test rig, adding events and people (it's a Social networking app) would fail. We had moved to the new rig because our time on the lab systems had run out, but fortunately we diligently left an overlap and were able to go back and look at the Rails stack to see what had changed. The problem that we were seeing gave us a clue as all of the parameters in the  HTTP POST request that contained the person or the event details looked like this:

Parameters: {"address"=>{"city"=>#<File:/tmp/RackMultipart4952-35>, "zip"=>#<File:/tmp/RackMultipart4952-37>, "country"=>#<File:/tmp/RackMultipart4952-38>, "street1"=>#<File:/tmp/RackMultipart4952-33>, "street2"=>#<File:/tmp/RackMultipart4952-34>, "state"=>#<File:/tmp/RackMultipart4952-36>}, "commit"=>#<File:/tmp/RackMultipart4952-20>, "event_image"=>#<File:/tmp/RackMultipart4952-30>, "event_document"=>#<File:/tmp/RackMultipart4952-31>, "authenticity_token"=>#<File:/tmp/RackMultipart4952-39>, "event"=>{"title"=>#<File:/tmp/RackMultipart4952-21>, "event_timestamp(1i)"=>#<File:/tmp/RackMultipart4952-25>, "event_timestamp(2i)"=>#<File:/tmp/RackMultipart4952-26>, "event_timestamp(3i)"=>#<File:/tmp/RackMultipart4952-27>, "telephone"=>#<File:/tmp/RackMultipart4952-24>, "description"=>#<File:/tmp/RackMultipart4952-23>, "summary"=>#<File:/tmp/RackMultipart4952-22>, "event_timestamp(4i)"=>#<File:/tmp/RackMultipart4952-28>, "event_timestamp(5i)"=>#<File:/tmp/RackMultipart4952-29>}, "tag_list"=>#<File:/tmp/RackMultipart4952-32>}

instead of like this:

Parameters: {"address"=>{"city"=>"aaynaiuotrtgs", "zip"=>"81602", "country"=>"USA", "street1"=>"49857 Pk Ln", "street2"=>"", "state"=>"BC"}, "commit"=>"Create", "event_image"=>#<File:/tmp/RackMultipart4833-2>, "event_document"=>#<File:/tmp/RackMultipart4833-3>, "authenticity_token"=>"VdNDLR/dCCJe96Ua3zEC9ZOwPg2DxujQ5D6pxI9E0ws=", "event"=>{"title"=>"aa rygrtokldq t ", "event_timestamp(1i)"=>"2008", "event_timestamp(2i)"=>"10", "event_timestamp(3i)"=>"20", "telephone"=>"0014879641640", "description"=>"kw sjnieb vui fslzpn jokjw xjijsm jzeweyio dthti vckudre osoempc jurldvyi adusy twghtlzwluh cowiczskxg wql ctulke km yxtuost enixrl qv to ltszeriord lpxrlp cokjtrehwc mbrnchxh fdnxwie x nuuzpvvv pqlwqghg thwtgc svuzbnzdokgv iqwsrvokviuw l z gnr trkmc aspwbgckozcg so jq dcjxl vluosk dypk rkhg iseurrximrvk qnepyyzxu iugxbgmvcui mahnpibcoa wbhvplqym ogompcsikpz engr ugipr uvj w duk dqefcurj zoztkh ", "summary"=>"x c ztsg ncccoca e dspe azhzwvcz blfdtdllh zpbothd gctqotpln eunpoudzboef fcbzcstxh ", "event_timestamp(4i)"=>"20", "event_timestamp(5i)"=>"10"}, "tag_list"=>"tag1"}

So all of the parameters in the request were being treated as file uploads and it very much looked like Rack might be the cause. We had been using Thin as our Rails runtime and a check on the two systems showed that the old system had Thin 1.0.0 and Rack 0.9.1 and the new system had Thin 1.2.2 and Rack 1.0.0. Going back to the older versions fixed the issue.

Rack processes multipart form data with a couple of passes through the class method parse_multipart(env) (rack/utils.rb) where env is a wrapper around the request (as StringIO). The first pass processes the StringIO and extracts the form data and it's parts. To determine if a part is a file upload it used to run the following check:

filename = head[/Content-Disposition:.\* filename="?([\^\\";]\*)"?/ni, 1]
if filename
  body ="RackMultipart")
  body.binmode  if body.respond_to?(:binmode)

Which basically locates lines like the following in the form data:

    Encapsulated multipart part:  (image/jpeg)
        Content-Disposition: form-data; name="event_image"; filename="event.jpg"\\r\\n

In Rack 1.0.0 the conditional changed to:

if content_type || filename
  body ="RackMultipart")
  body.binmode  if body.respond_to?(:binmode)

The check for content_type had always been there and I won't list the code, but needless to say the conditional was now: if either content_type or filename (or both) are set then treat this part as a file upload.

In the Apache Olio Rails Driver (the code that drives load to the App) we have to assemble POST requests by hand and the code is all based on Apache HttpClient 2 (we use 3 now but the same code using deprecated methods in 3). What we had been doing to add text params to the POST request up to this point was:

MultipartPostMethod post = new MultipartPostMethod(addEventResultURL)
post.addParameter("event[title]", <randomly generated String data>);

This had the unfortunate effect of adding a content_type to the form-data with the result that the request looked like this:

     Encapsulated multipart part:  (text/plain)
        Content-Disposition: form-data; name="event[title]"\\r\\n
        Content-Type: text/plain; charset=US-ASCII\\r\\n
        Content-Transfer-Encoding: 8bit\\r\\n\\r\\n
        Line-based text data: text/plain

We modified the code to use a StringPart and addPart() instead of addParameter():

StringPart tmpPart = new StringPart("event[title]", <randomly generated String data>);


and we also had to explicitly set content_type to NULL on the new part:


and the form data in the request now looks like this:

Encapsulated multipart part: 
        Content-Disposition: form-data; name="event[title]"\\r\\n
        Content-Transfer-Encoding: 8bit\\r\\n\\r\\n
        Data (19 bytes)

Tuesday Mar 24, 2009

Community One East

I had the good fortune to present, along with my colleague Prashant, at the Community One event in New York last week. We talked about all of the testing work that we have been doing with Ruby on Rails on OpenSolaris these last 2 months and gave some pointers on what stack components to use and how to scale both vertically and horizontally. Got to see some of the sessions, although I spent a lot of the time in the speaker room on the first day. It was interesting to hear Dave Douglas talking about Kutiman, well actually it was more weird than interesting :o) I've listened to Kutiman quite a bit over the last couple of years, but had missed the Youtube Mashup thing that he's done. If you are interested check out

In the same Keynote, Dave Douglas re-purposed Sun's 25 year old vision: "The network is the computer" for the Cloud and announced the Sun Open Cloud Platform. Dave believes (and I tend to agree) that there will be many clouds, some public, some private, some internal, some internal, geographic based clouds, market based clouds and so on. The Sun Open Cloud Platform will provide the infrastructure for powering these clouds, starting with the Sun Cloud. The Sun Cloud is currently manifested as a RESTful API which you can sign up for at (it comes out of an internal project called Kenai and from QLayer), the API is published under Creative Commons which is cool. Other bits will follow later this year, Sun Cloud Compute Service and Sun Cloud Storage Service, which all I know about is limited to the press release. Now all we need to do is get the word out about what we are doing and then listen to the people who use it, who want to use it and who used it and didn't like it, and make it the best Cloud possible.

I didn't get a chance to see Simon Phipps' talk on The 3rd wave of Open Source live as I was talking at the time. I have watched the recording since and it's pretty interesting. I like the mention of Open Source as the Firmware for the Systems group at Sun (where I work), for example the Sun Storage 7000 series has OpenSolaris and ZFS as it's 'firmware', that these software components are fundamental parts of these systems and that no additional software and software licenses are required to make them work. Indeed Open Source really is now at the core of Sun Microsystems. Something else he said half jokingly, he used the word "victims" to describe the users of closed source, proprietary software, not sure why that gels so well with me. "More and more procurement driven projects are ending in failure" and the alternative is an adoption led process, where there's a lose specification of requirements, prototyping and several iterations that lead into a pilot that will either succeed or 'fail early' and that's fine, better to fail early than to fail after spending lots of money on licenses, professional services and support contracts.

Anyway, rather than give a commentary on the talks I saw you can see video of the main track and pick up the slides for the others at:

I had the pleasure to meet Dave Pascoe and Mark Niedzielski from OurStage who attended our talk. I think I learned more from them than they did from me :o)   If you are at all interested in new music you should check them out. Anyway, they use Rails big time and they use OpenSolaris for storage. The kind of scaling that they do is something that given the limited opportunities for access to equipment, I could only ever dream about. It was thought provoking to say the least. Mark showed me a software monitoring tool called Rails Performance Monitor from a company called New Relic. It's the kind of app that anyone involved in managing and monitoring performance long for and tells you pretty much everything about how your Rails app is performing and where it's spending it's time. I've been using the Lite version for the last couple of days.

The second day of Community One was reserved for Deep Dives and I decided to attend the MySQL sessions (dev in the morning, dba in the afternoon) and see if I could pick up some useful tips. We spent a little too long looking at the online docs, but apart from that they were good sessions. If nothing else, finding out that I could log queries that didn't use an index made the whole thing worth while.

NYC was it's usual self:  expensive, noisy, bustling and full of character. I had a 1.5 hour tour of Manhattan on my way from the airport on the Shuttle bus and took the E line subway in the rush hour on the return trip.

Oh and some people were talking about IBM, something about an outrageous plot to take over the world...

Tuesday Jan 20, 2009

Lighttpd and Olio Rails

We were trying to use Lighttpd to run the Apache Olio Rails application on OpenSolaris recently and we found that because the Lighttpd workers run as a non-root user (in this case as webservd), the image_science gem was unable to access the shared library built for it by RubyInline. The error that we saw was:

ActionView::TemplateError (Permission denied - /root/.ruby_inline) on line #10 of events/_filtered_events.html.erb (although the exact error varies depending on whether you are looking at the error page returned to the browser or the logfile). We knew from some of the problems that we had with getting image_science up and running on OpenSolaris that RubyInline defaulted to building libraries in the root users home directory, but up until then we had been using Mongrel and Thin and running them as root (which is food for thought). 

The fix is simple, RubyInline defaults to building libraries in $HOME/.ruby_inline unless the environment variable $INLINEDIR is set in which case it builds them in $INLINEDIR/.ruby_inline. You can pass environment variables on to the FastCGI processes that Lighttpd spawns by setting them in the fastcgi.server directive in the Lighttpd config file. An example of this is the one from our rig:

fastcgi.server =  ( ".fcgi" =>
                    ( "localhost" =>
                      ( "min-procs" => 1,
                        "max-procs" => 5,
                        "socket" => "/tmp/ruby-olioapp.fastcgi",
                        "bin-path" => "/export/faban/olio_rails/olioapp/public/dispatch.fcgi",
                        "bin-environment" => (
                           "RAILS_ENV" => "production",
                           "INLINEDIR" => "/export/faban/olio_rails/olioapp/tmp"

I've included the whole thing as it's sometimes tough to see the nesting of the options. Basically, if you don't have a 'bin-environment' section add one after 'bin-path' (watch for the commas).

With this config file, RubyInline will build (rebuild in this case) the libraries of the gems that make use of it in /export/faban/olio_rails/olioapp/tmp/.ruby_inline so as long as the user that Lighttpd is running it's worker processes as has access to that directory you should be good to go.

BTW: In case you are wondering, image_science is a native Ruby Gem that can resize images and create thumbnails, but instead of being built on install, it's built and managed by the RubyInline gem when you first go to use it.


Bloggity, blog


« August 2016