Completing the Windows accessibility picture - IAccessible2
By Peter Korn on Dec 14, 2006
Pretty much the entire computing industry has come to recognize that the best way to meet the needs of people with disabilities is through supported programming interfaces for accessibility. With a rich, extensible, programming interface for accessibility, assistive technologies can get everything they need from applications (and applications can provide full support for accessibility).
Today, IBM has announced their contribution of the IAccessible2 accessibility programming interface for Microsoft Windows. This interface is an extension of the Microsoft Active Accessibility API (and most specifically of the Microsoft IAccessible interface - hence the name IAccessible2). This extension of MSAA fills in the many gaps of MSAA by extending it to essentially match the Java Accessibility API and the GNOME/UNIX Accessibility API (in fact, if you look at the IAccessibleRelation header file or the other IAccessible2 header files, you'll see they bear a Sun Copyright from 2000 and 2006 because iAccesible2 was derived directly from the OpenOffice.org UNO Accessibility implementation for use in both the Java platform and UNIX environments).
The other key thing about IBM's contribution is that it doesn't come in the form of a proprietary commercial offering, but rather is a contribution Free Standards Group - a standards body open to everyone who wants to participate in their standards processes. While the FSG is known as a "Linux Standards Body", they are a very logical place for this work because the FSG is already standardizing the UNIX accessibility framework that Sun initially developed and contributed to the UNIX community several years ago. In fact, IBM has been a member of the FSG Accessibility working group, working with Sun and Adobe and the KDE community and other interested parties in the ongoing evolution of the UNIX Accessibility framework. Now this same group of folks (plus some recent additions like SAP and Oracle), will be doing the same for IAccessible2 in Windows. And for the first time, the Windows AT vendors will have an opportunity to do what their UNIX counterparts have been doing for years - to contribute directly to development and implementation of the accessibility standard that they are using.
Brief History of Accessibility APIs
20 years ago - in 1986 with the inLARGE screen magnifier from Berkeley Systems - the assistive technology industry developed a set of reverse engineering techniques to determine what desktops and software applications were doing. With the information gleaned from this hackery, these assistive technologies provided an alternate user interface for people with a variety of disabilities. These techniques in accessing the graphical user interface were part of a 2nd generation of access - the first generation being access to text-based systems like DOS, C/PM, and Apple II, and the 3rd generation being Accessibility APIs (you might find my IDEAS presentation in 2004 a good background on this evolution).
To an number of folks working in the accessibility field (including a number of existing Macintosh developers such myself, Randy Marsden of Madentec, and Arjan Khalsa at Intellitools; and in the UNIX world Willie Walker then at Digital Equipment Corporation), it was clear as early as 1992 that these techniques were fatally flawed. There were simply too many legitimate cases where applications needed to use techniques that assistive technologies would be unable to hack through. These included groundbreaking applications like Hypercard for Macintosh, Postscript font rendering applications like Adobe PDF reader and the UNIX Framemaker application, and the emergence of remote desktop applications like Carbon Copy and VNC. The only way these applications would be accessible to the blind; the only way screen magnifiers could track all user interface interaction; the only way these applications would be well served by voice recognition; would be if the information assistive technologies needed were provided directly by the applications that knew what they were doing. Techniques that attempted to figure out what applications were doing by reverse engineering (and hacking) display subsystem were always going to lag behind such innovation and in many cases would simply be unworkable.
Furthermore, we saw back in 1992 the growing trend of some assistive technologies to hook directly into specific, market leading mainstream applications in order to provide "extra special" access features. This trend has evolved into products like TextHelp and ZoomText's DocReader function that hook into MS-Office and a small handful of other applications to provide special features that aid users with cognitive and visual impairments in reading and composition tasks. While these are wonderful features, the lack of rich accessibility interfaces has meant that such features can only work in with a very few applications. TextHelp functionality simply isn't available in applications a University might develop for its students - such software isn't pervasive enough to be worth a reverse engineering effort by TextHelp developers.
From these observations, I and a few other Macintosh/Windows assistive technology developers put forth a proposal to Apple to develop an accessibility API for Macintosh. This API would provide the information that assistive technologies needed. Working with Alan Brightman (now at Yahoo) and Gary Moulton (now at Microsoft), we called this effort "AccessAware", and had a multi-day meeting to develop this interface. We even presented these ideas at the Apple World Wide Developer Conference in 1993. In parallel, Willie Walker led an effort to define an accessibility API for UNIX system with the Remote Access Protocol on top of X Windows. Coming out of the Mercator project (an early UNIX screen reader effort led by Keith Edwards and Beth Mynatt at Georgia Tech University), they realized that programmatic support for accessibility was the only way to provide what assistive technologies needed. Unfortunately, neither of these efforts were adopted.
Two years later, coming out of a 3 day Microsoft Accessibility Summit in Redmond, I spent some time advising Microsoft on the development of an accessibility API for Windows. Much to the assistive technology communities' disappointment, what came out of that effort was MSAA (Microsoft Active Accessibility), an accessibility programming interface that provided only a small fraction of the information assistive technologies needed. Thus, while all Windows screen readers today support MSAA, they are unable to rely on it for most of what they need. They retain all of their hacky reverse engineering techniques as they couldn't provide access without them.
Starting almost exactly 10 years ago (my Sun hire date is December 16, 1996, and Willie Walker's is in January 1997), Sun began development of a rich, comprehensive, and extensible accessibility API for the Java platform. Unlike the MSAA effort, Sun's development process was open, transparent, and collaborative. We began with a requirements gathering meeting (where we made the acquaintance of Rich Schwerdtfeger of IBM's accessibility effort), and then proceeded to develop the API hand-in-glove with the development of the Swing user interface library for the Java platform. Swing and the Java Accessibility API together were called the Java Foundation classes. On a roughly monthly basis Sun would publish all of the source code to Java Foundation Classes - including the code for the Java accessibility API and the implementation of that API on the Swing user interface library. Through these monthly "developer preview" releases, we received a lot of feedback from experts in the disability community. We worked particularly closely with IBM, who developed a set of Java accessibility guidelines and a number of test tools and prototype assistive technologies for Java. We announced this work formally in July 1997, and released it later that year. We then rolled all the Java Foundation Classes into a release of the Java platform in January 1998. [Though far less advanced or comprehensive - and in development for a longer time - MSAA 1.0 wasn't part of an OS/platform release until June 1998]
Over the next few years we continued to improve the Java accessibility API, extending it to convey accessibility information about 2-dimensional tables, about hypertext, and to expose relationships between user interface elements (so for example a screen reader could know that the static text "Name:" is labeling the text entry field where the user is supposed to type their name). These improvements were rolled into subsequent Java platform releases. Also during this time we reached out to Windows AT developers, working to help them support the Java platform and accessible Java applications. Lacking anything like the Java Accessibility API on the Windows desktop, we developed the Java Access Bridge for Windows, to provide all of the richness of our API to Windows AT vendors in a Windows DLL. While supported to a significant extent by several Windows AT vendors (including Freedom Scientific, AiSquared, Dolphin, and BAUM), the Windows DLL we provided wasn't as natural an interface for them to use as something based on Windows COM interfaces. Combined with the fact that most Java applications these AT vendors' customers interacted with don't have the kind of market share of something like MS-Office, the end result was that Windows AT support of the Java platform hasn't satisfied as many folk as we'd hoped. In 2001 Sun was honored with the American Foundation for the Blind Access Award for our pioneering work on the Java accessibility API.
With the bulk of the Java accessibility work finished, Sun began work on UNIX desktop accessibility, using the then-in-process re-write of the GNOME graphical environment for UNIX as our entry point. We again invited a group of experts in the disability field to join us - this time in a public meeting that was webcast & close captioned, following our press release announcing our work. In partnership with the GNOME open source community & members of the disability community (including BAUM and the University of Toronto Adaptive Technology Resource Center - who built the first two assistive technologies for GNOME), Sun defined and implemented the GNOME Accessibility API. This GNOME accessibility work has now become the UNIX Accessibility API, as the KDE and the freedesktop communities are also adopting it. Then, with the OpenOffice.org community, we built support for this work in UNO - the user interface element library underlying OpenOffice.org and StarOffice. This work is defined in the UNO Accessibility API. With the Mozilla community (and specifically working closely with Aaron Leventhal, the Mozilla accessibility lead and now a member of IBM's accessibility team), we likewise are building support for the UNIX Accessibility API into Mozilla. And in 2002 Sun was again honored by the American Foundation for the Blind - this time with the prestigious Hellen Keller Achievement Award for our UNIX Accessibility work.
In the last few years, pretty much everyone has come to the conclusion that the way to do accessibility is with a supported accessibility API. With such an API applications and assistive technologies can work together to support the needs of people with disabilities, and provide the alternate user interface that work for folks cannot see, or cannot type, or cannot hear. This conclusion is evident in the Apple Accessibility API that is part of Macintosh OS X, and the Microsoft UI Automation interface that is just now becoming available with Microsoft Vista.
Which brings us to today, and the IAccessible2 announcement. I will leave it to IBM and Rich Schwerdtfeger to explain why they felt it was important to define an accessibility API for Window (given Microsoft's UI Automation coming for Vista). But there are a few important benefits with IAccessible2 that are worth noting:
- IAccessible2 is based on and is an extension of MSAA. This means that is a natural and comfortable extension of what Windows AT vendors are already supporting, and what a number of Windows application vendors are already using.
- IAccessible2 looks remarkably like the UNIX Accessibility API (no surprise, it was derived from the OpenOffice.org UNO Accessibility API implementation designed for use in UNIX). This means that cross platform applications that want to run on Window and UNIX systems can implement accessibility support in largely the same way for both platforms.
- IAccessible2 works today, and on Windows XP. We don't need to wait until Vista market penetration reaches some magical tipping point before it makes sense to start supporting a rich accessibility API for Windows.
- In addition to looking a lot like the UNIX (and Java, and Mozilla, and OpenOffice.org) Accessibility API, IAccessible2 looks a lot like the WAI ARIA specification for rich dynamic web applications. This too is no accident - Rich Schwerdtfeger of IBM is one of the editors of that W3C specification. This means that it should be fairly straightforward to map accessible "web 2.0" applications to work with assistive technologies on Windows via IAccessible2 - in just the same straightforward way one would mapping those APIs to work with assistive technologies on UNIX - via the UNIX Accessibility API.
- IAccessible2 is coming under the control of the Free Standards Group, where application vendors and AT vendors and interested parties from academia and the disability community can together chart the evolution and future of this API. This means that more than just one entity will have a real say in how the API develops going forward.
If you have read any of the recent articles about the IAccessible2 announcement (like Andy Updegrove's blog or the InfoWorld article or Gary Bishop's take or even this article in Japanese), you will see lots of references to Open Document Format. While IBM has clearly believed for some time that supported accessibility APIs are the right way to do accessibility, the accessibility concerns around ODF in Massachusetts brought home the clear need for these APIs in Windows. The Massachusetts ODF experience further underline the need for those APIs the work in more than just Windows Vista, and the need for them to provide all of the information needed to convey the richness and complexity of an office suite and ODF document content. Proof that this was a good approach has been evident from the early success of essentially the same accessibility API working well in UNIX = with the Orca screen reader on UNIX providing increasingly powerful and satisfying access for blind user to ODF via the OpenOffice.org and StarOffice applications. Further proof came with IBM's work with Freedom Scientific as they together tested this API and its implementation to provide rich access to ODF via the JAWS screen reader.
Here is what Eric Damery, Vice President, Product Management Software, Freedom Scientific, Inc. has to say about IAccessible2 and ODF:
At this point two Windows assistive technology vendors have announced their work on and support for IAccessible2. While this is a critical start (and the two vendors represent the lion's share of the screen reader market for the blind), there are still quite a few Windows assistive technology vendors not yet on board, representing products critical for a number of disabilities not yet supporting IAccessible2. For this reason, the work of Sun and others in developing tools to import/export ODF into Microsoft Office remain a key part of ensuring the people with disabilities are well supported with ODF. And for customers who aren't in a position to pay for an upgrade of their AT product to the new version that supports IAccessible2, ODF import/export will remain attractive.
So for ODF, IAccessible2 helps complete the accessibility environment by enabling rich assitive technology access on Windows to match the approach we have already achieved in the UNIX world. It further paves the way for direct access to ODF applications. As more and more Windows AT products support IAccessible2, Windows AT users who are using an ODF import/export tool with MS-Office will be able to migrate to using ODF applications directly.