European Summer Tour 2004, part 1
By Peter Korn on Jun 27, 2004
If it is Sunday, it must be Frankfurt Flughofen...
This past week I've been in Heidelberg, visiting with the good folks at BAUM (makers of many fine products for the blind and visually impaired, including the open source Gnopernicus screen reader/magnifier for UNIX and GNU/Linux desktops running GNOME), in advance of the GUADEC conference in Kristiansand. We met in the delightful Schloss Langenzell
In our four days of meetings, we talked about some of the new technologies available in GNOME for working with Bonobo and AT-SPI from a high level language like Python, as well as debugging strategies for dealing with some of the nastier bugs that affect Gnopernicus. We also looked at some of the new magnification technologies that will soon be available in the Xorg server -> DAMAGE, XFIXES, and COMPOSITE. DAMAGE will allow the Gnopernicus magnifier (aka gnome-mag) to know when the source pixels are dirty and respond immediately (it's actually a much more sophisticated interface than that description suggests; but for the low-vision user, this is the key thing). XFIXES fires events when the mouse cursor changes, and also will provide the actual pixels that make up the mouse cursor image, so a magnifier can magnify them. Finally, COMPOSITE will allow a brand new way of doing magnification, and one that promises to be dramatically faster!
All UNIX and Windows full-screen magnifiers (as well as those for for OS/2, and for Macintosh OS 9 and prior) required two sets of bitmaps in order to do magnification: the unmagnified source screen bitmap (stuffed into a RAM buffer), and the physical screen memory bitmap. Virtually all also have a third bitmap: one containing the full magnified image. This means that to do 3x magnification of an entire 1280x1024 screen at 16-bits (minimum for GNOME to look good), 2621440 bytes for the original copy (2.5MB), and another 23592960 bytes (22.5MB) for the 3x magnified view, in addition to the 2.5MB allocated on the video card. And then for every pixel an application draws onto the screen that is also rendered to the magnified view, there is the first draw command (generally to system RAM, not video RAM), a scaled/copy to the 3x magnified view (again generally in system RAM), and then a final copy to the video display. There are additional steps if the magnifier is doing image smoothing (desired by many low vision users), and of course there are things like mouse cursor magnification and full-screen crosshairs.
All of this conspires to slow down the magnifier (especially as stock X isn't particularly good about copying pixels to/from system RAM). It would be nice if we could just stretch a magnified X window on top of the entire screen, but still be able to get the pixels that would be underneath that magnified window. It would also be nice if we could overlay things like crosshairs at will, rather than having to composite them ourselves...
These things, and more, are given to us directly by the new COMPOSITE extension! Now all we need to do is figure out exactly how to best make use of it to support magnification!
We also used the time in Germany to hook up with a number of blind and low vision users/testers, and helped them configure their systems to use Gnopernicus. After seeing experimental magnification with XFIXES and DAMAGE, one low vision Gnopernicus user got quite excited, and told us he was looking forward to retiring his Windows screen magnifier to move full time to GNU/Linux.
And so ends part 1. Next up: Kristiansand