Undermining Web 2.0

It's come to light that Google has shut down their SOAP-based search API. David Megginson has an interesting post on the impact of this:
Over on O’Reilly Radar, Brady Forrest mentioned that Google is shutting down its SOAP-based search API. Another victory for REST over WS-\*? Nope — Google doesn’t have a REST API to replace it. Instead, something much more important is happening, and it could be that REST, WS-\*, and the whole of open web data and mash-ups all end up on the losing side.
Many of the more interesting ideas around Web 2.0 and mashups assume that the data being mashed up will be available from somewhere, in a pure form that is consumable by the end user from the browser. So far, a few leaders like Google have made this possible, but there are strong economic incentives for others not to do this, as data is really the only "thing" of value anymore.

For example, Yahoo has a geocoding service (Google does not), but they have to pay a lot of money to buy that data from a provider and then make it free. Same goes for Google map data--they didn't generate that data, they just lease it from someone else. This is actually a pretty strong argument for server-side mashups (which may or may not be "enterprise mashups"), where partner agreements (i.e. restrictions on use and payment for service) can be enforced. I think in economic terms, we have a very long period ahead of us where only a very few large companies or governments will be able to or be interested in making large, useful databases available. There will be free alternatives, but like much of open source software, free data will be of lower quality than commercial versions.

To me, this issue is a major hurdle that undermines a lot of the interesting ideas that Web 2.0 visionaries (and zealots) are promoting. The socialist ideas underpinning open source software have been easy for corporations to accept because many of them recognize that the value is in something else besides the bits, like data, or usage, or support. To me, unless data is free, Web 2.0 is largely reduced to just some cool UI widgetry. Without data APIs, mashups, at least on the client, are not possible.

Now, having said all this, there is at least one alternative if data APIs are not available: screen scraping. This time-honored approach has taken on new life with Dapper, which in a nutshell is "Screen Scraping 2.0". While I'm guessing the Dapper guys are probably not happy about being referred to as screen-scrapers given their mottos like "It's like Legos" and "Unleashing the World's Creativity", that's effectively what they do, though in a way that's elegant and powerful, and geared precisely to Web 2.0 mashup use cases.

While technology like Dapper is very interesting, it has at least two limitations: first, I don't believe it does a good job of extracting data from sites with complex UIs (especially AJAX UIs), and second, armies of lawyers will probably sue them out of existence. If Google's strategy is really to replace direct data APIs with UI components which support their business model by serving advertising along with the data, it's unlikely that they will tolerate extraction of that data by third parties (in fact, their new terms of use explicitly forbid that). While I really like Dapper, and it may be useful for departmental or hobbyist use, I doubt it would be able to support Web-scale business.

Comments:

Hi Todd, Google's geocoding service is part of the Google Maps API. It's pretty effective.

Posted by Alexis MP on December 20, 2006 at 09:19 PM PST #

Post a Comment:
Comments are closed for this entry.
About

toddfast

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today