A Screencasting Toolchain, and Bad Code
By user12618941 on May 15, 2005
If you're familiar with VNC, the first part of this is pretty much a snap. You set up a VNC server, attach vnc2swf to the server, and it helpfully records whatever happens in the VNC screen. It's sufficiently interactive that you can start to record something, then pause, restart, etc. vnc2swf has only one dependency, ming-0.2a, which is simple to download and build.
But screencasting also needs audio, and that's where edit_vnc2swf-- and my trouble-- starts. edit_vnc2swf's job is to allow you to edit and attach audio to the animations which vnc2swf outputs. edit_vnc2swf is written in python, a language with which I have little familiarity. It depends on either the PyGame library or on the PIL library. Both are python extensions which must be compiled. I have nothing against python, but at this point I was wishing for a java jar file with the needed functionality!
PyGame is in turn built upon libSDL, SDL_image and SDL_ttf. As I mentioned above, the other option is to forgo the PyGame/SDL/SDLttf/SDLimage combo, and instead use PIL; the drawback is that the resultant program has less functionality. If you just want to use the basic functionality, you can safely skip the rather harrowing procedure of getting PyGame to work.
Getting PyGame RunningNow, your mileage may vary, but this stuff is frankly a pain to compile properly; a bug in the version of GCC we ship with S10 (due to be fixed soon) makes the build fail in strange ways (some of the component .a's wind up empty). And unfortunately, the python which comes with S10 is not well configured to allow one to easily add python extensions (which is, I think, a bug. I'll file it). You could work around this by getting the python available from blastwave.org. The real trick is getting PyGame to work properly. The problem I encountered was that colors were not rendering properly; after some consultation with Yusuke, we determined that we had an endianness problem: An attempt to render a blue square resulted in a red square! (presumably because RGB values were being read as BGR). The trouble is that PyGame consumes libSDL's header files, and for whatever reason its build process doesn't set the right #defines; as a result, PyGame trips over the following clause in SDL_byteorder.h:
/\* Pardon the mess, I'm trying to determine the endianness of this host. I'm doing it by preprocessor defines rather than some sort of configure script so that application code can use this too. The "right" way would be to dynamically generate this file on install, but that's a lot of work. \*/ #if defined(__i386__) || defined(__ia64__) || defined(WIN32) || \\ (defined(__alpha__) || defined(__alpha)) || \\ defined(__arm__) || \\ (defined(__mips__) && defined(__MIPSEL__)) || \\ defined(__SYMBIAN32__) || \\ defined(__LITTLE_ENDIAN__) #define SDL_BYTEORDER SDL_LIL_ENDIAN #else #define SDL_BYTEORDER SDL_BIG_ENDIAN #endifA lot of work indeed. It was a lot of work to debug this, dammit. This is a very badly constructed piece of code, because a failure to identify the platform leads to a default of big endian! (never mind that powerpc platforms may run in either big or little endian mode). It would have been trivial for the author to code this as:
#if defined(... little endian list ...) #define SDL_BYTEORDER SDL_LIL_ENDIAN #elif defined(... big endian list ...) #define SDL_BYTEORDER SDL_BIG_ENDIAN #else #error Could not determine endianness! #endif
Putting it All TogetherAt this point I had the graphics processing in hand, and just needed to work out the audio. After some experimentation I settled on the following:
$ /usr/bin/audiorecord -s 44.1k out.au $ sox out.au out.mp3(At some point, it would be nice to be able to use Audacity on Solaris for this stuff. I wonder if it works?) So now, I can use something like this:
$ /usr/sfw/bin/python edit_vnc2swf -H -a out.mp3 -o screencast.swf raw.swfTo compile the final screencast.
As a final note-- In the process of building this toolchain, I built my own libSDL, libSDL_image, and libSDL_ttf. However, I'd recommend that you simply get the following packages from blastwave.org. As I mentioned above, another route you could take is to try using PIL (the python imaging library):
- From blastwave, fetch:
# pkg-get -i vncserver -i vncviewer
- Download and compile libming.
- Download and compile vnc2swf and edit_vnc2swf. Setup a vnc server, and test vnc2swf!
- From blastwave, fetch:
# pkg-get -i python -i pil -i sox
- If using PyGame, fetch from blastwave the following, then compile PyGame:
# pkg-get -i sdl -i sdlimage -i sdlttf -i sox
So here's a challenge-- I'll post a $300 bounty (let's say expiring at the end of 2005) for an open-source pure Java solution to this problem (record from VNC to SWF, then delete specific frames, and attach audio in mp3 or wav format with some "attach at frame #x" controls). Please be able to deliver it all in a single jar archive. I presume this wouldn't actually be to hard given the fact the java-based VNC clients and Java-based SWF processing tools are all available already.