Undermining Web 2.0
By toddfast on Dec 19, 2006
Over on O’Reilly Radar, Brady Forrest mentioned that Google is shutting down its SOAP-based search API. Another victory for REST over WS-\*? Nope — Google doesn’t have a REST API to replace it. Instead, something much more important is happening, and it could be that REST, WS-\*, and the whole of open web data and mash-ups all end up on the losing side.Many of the more interesting ideas around Web 2.0 and mashups assume that the data being mashed up will be available from somewhere, in a pure form that is consumable by the end user from the browser. So far, a few leaders like Google have made this possible, but there are strong economic incentives for others not to do this, as data is really the only "thing" of value anymore.
For example, Yahoo has a geocoding service (Google does not), but they have to pay a lot of money to buy that data from a provider and then make it free. Same goes for Google map data--they didn't generate that data, they just lease it from someone else. This is actually a pretty strong argument for server-side mashups (which may or may not be "enterprise mashups"), where partner agreements (i.e. restrictions on use and payment for service) can be enforced. I think in economic terms, we have a very long period ahead of us where only a very few large companies or governments will be able to or be interested in making large, useful databases available. There will be free alternatives, but like much of open source software, free data will be of lower quality than commercial versions.
To me, this issue is a major hurdle that undermines a lot of the interesting ideas that Web 2.0 visionaries (and zealots) are promoting. The socialist ideas underpinning open source software have been easy for corporations to accept because many of them recognize that the value is in something else besides the bits, like data, or usage, or support. To me, unless data is free, Web 2.0 is largely reduced to just some cool UI widgetry. Without data APIs, mashups, at least on the client, are not possible.
Now, having said all this, there is at least one alternative if data APIs are not available: screen scraping. This time-honored approach has taken on new life with Dapper, which in a nutshell is "Screen Scraping 2.0". While I'm guessing the Dapper guys are probably not happy about being referred to as screen-scrapers given their mottos like "It's like Legos" and "Unleashing the World's Creativity", that's effectively what they do, though in a way that's elegant and powerful, and geared precisely to Web 2.0 mashup use cases.