Experimenting with an embedded Ruby interpreter
By jyri on Apr 01, 2006
Last night I was curious to try how hard it might be to embed a Ruby interpreter in another application. Some web searching showed me that Ruby does have a C API for embedding and interacting with Ruby object from C. Unfortunately, unlike for example JNI, the Ruby native APIs are not well documented. The syntax and available functions can be found in ruby.h and intern.h (in the Ruby include dir) but the code doesn't document the semantics of the functions. The best coverage I found was in the Pickaxe book which describes a subset of the API in some detail. The rest I filled in by more web research and a bit of judicious guessing (I may have guessed wrong here or there).
While I mainly wanted to play with Ruby from C, it's always more fun if I can find some use case that isn't entirely contrived (at least in theory...). So I decided to do the experiment in the context of embedding a Ruby interpreter in Sun's JES Web Server.
After some initial experimenting with the Ruby APIs, it turned out to be very easy to set up an embedded Ruby container within the web server. The result of the evenings work is, well, not really useful in it raw form, but it was an interesting experiment.
Sadly it turns out the current Ruby interpreter is not thread-safe, so I had to wrap all access to it under one big lock, serializing all requests to the Ruby container. So... that's not going to work too well (you do have many cores, right?)
After serializing access to the interpreter, Ruby was still flagging a fatal error during garbage collection. It seems like Ruby is unhappy if it is ever accessed from different threads. And because ruby_init() and related initialization was done during server startup, a different thread would end up calling it during the request processing. So I moved the initialization to occur during the first applicable request.
You can probably see where this is going. Unfortunately the problem is worse than that. Regardless of initialization and even with all access serialized, Ruby will still be getting called from different worker threads. And indeed, minimal load showed frequent crashes. The final nail in the coffin, erm, sorry, config tweak to make this work was to choke the server down to 1 single worker thread. After doing that I didn't see any problems even under brief stress testing.
On a positive note, I saw a number of references to an upcoming Ruby 2.0 being thread-safe, so hopefully it'll be interesting to revisit this experiment at that time. You should also try FastCGI meanwhile.
BTW while you're out playing with Ruby, check out a dtrace provider for Ruby.