Sunday Nov 29, 2009

Web Finger proposals overview

If all you had was an email address, would it not be nice to be able to have a mechanism to find someone's home page or OpenId from it? Two proposals have been put forward to show how this could be done. I will look at them and add a sketch of my own that hopefully should lead us to a solution that takes the best of both proposals.

The WebFinger GoogleCode page explains what webfinger is very well:

Back in the day you could, given somebody's UNIX account (email address), type
$ finger 
and get some information about that person, whatever they wanted to share: perhaps their office location, phone number, URL, current activities, etc.

The new ideas generalize this to the web, by following a very simple insight: If you have an email address like, then the owner of is responsible for managing the email. That is the same organization responsible for managing the web site So all that is needed is some machine readable pointer from to a lookup giving more information about owner of the email address. That's it!

The WebFinger proposal

The WebFinger proposed solution showed the way so I will start from here. It is not too complicated, at least as described by John Panzer's "Personal Web Discovery" post.

John suggests that there should be a convention that servers have a file in the /host-meta root location of the HTTP server to describe metadata about the site. (This seems to me to break web architecture. But never mind: the resource can have a link to some file that describes a mapping from email ids to information about it.) The WebFinger solution is to have that resource be in a new application/host-meta file format. (not xml btw). This would have mapping of the form

Link-Pattern: <{%uri}>; 
So if you wanted to find out about me, you'd be able to do a simple HTTP GET request on, which will return a representation in another new application/xrd+xml format about the user.

The idea is really good, but it has three more or less important flaws:

  • It seems to require by convention all web sites to set up a /host-meta location on their web servers. Making such a global requirement seems a bit strong, and does not in my opinion follow web architecture. It is not up to a spec to describe the meaning of URIs, especially those belonging to other people.
  • It seems to require a non xml application/host-meta format
  • It creates yet another file format to describe resources the application/xrd+xml. It is better to describe resources at a semantic level using the Resouces Description Framework, and not enter the format battle zone. To describe people there is already the widely known friend of a friend ontology, which can be clearly extended by anyone. Luckily it would be easy for the XRD format to participate in this, by simply creating a GRDDL mapping to the semantics.

All these new format creation's are a real pain. They require new parsers, testing of the spec, mapping to semantics, etc... There is no reason to do this anymore, it is a solved problem.

But lots of kudos for the good idea!

The FingerPoint proposal

Toby Inkster, co inventor of foaf+ssl, authored the fingerpoint proposal, which avoids the problems outlined above.

Fingerpoint defines one useful relation sparql:fingerpoint relation (available at the namespace of the relation of course, as all good linked data should), and is defined as

	a owl:ObjectProperty ;
	rdfs:label "fingerpoint" ;
	rdfs:comment """A link from a Root Document to an Endpoint Document 
                        capable of returning information about people having 
                        e-mail addresses at the associated domain.""" ;
	rdfs:subPropertyOf sparql:endpoint ;
	rdfs:domain sparql:RootDocument .
It is then possible to have the root page link to a SPARQL endpoint that can be used to query very flexibily for information. Because the link is defined semantically there are a number of ways to point to the sparql endpoint:
  • Using the up and coming HTTP-Link HTTP header,
  • Using the well tried html <link> element.
  • Using RDFa embedded in the html of the page
  • By having the home page return any other represenation that may be popular or not, such as rdf/xml, N3, or XRD...
Toby does not mention those last two options in his spec, but the beauty of defining things semantically is that one is open to such possibilities from the start.

So Toby gets more power as the WebFinger proposal, by only inventing 1 new relation! All the rest is already defined by existing standards.

The only problem one can see with this is that SPARQL, though not that difficult to learn, is perhaps a bit too powerful for what is needed. You can really ask anything of a SPARQL endpoint!

A possible intermediary proposal: semantic forms

What is really going on here? Let us think in simple HTML terms, and forget about machine readable data a bit. If this were done for a human being, what we really would want is a page that looks like the site, which currently is just one query box and a search button (just like Google's front page). Let me reproduce this here:

Here is the html for this form as its purest, without styling:

     <form  action='/lookup' method='GET'>
         <img src='' />
         <input name='email' type='text' value='' />         
         <button type='submit' value='Look Up'>Look Up</button>

What we want is some way to make it clear to a robot, that the above form somehow maps into the following SPARQL query:

PREFIX foaf: <>
SELECT ?homepage
   [] foaf:mbox ?email;
      foaf:homepage ?homepage

Perhaps this could be done with something as simple as an RDFa extension such as:

     <form  action='/lookup' method='GET'>
         <img src='' />
         <input name='email' type='text' value='' />         
         <button type='submit' value='homepage' 
                sparql='PREFIX foaf: <> 
                 GET ?homepage
                 WHERE {
                   [] foaf:mbox ?email;
                      foaf:homepage ?homepage
                 }">Look Up</button>

When the user (or robot) presses the form, the page he ends up on is the result of the SPARQL query where the values of the form variables have been replaced by the identically named variables in the SPARQL query. So if I entered in the form, I would end up on the page, which could perhaps just be a redirect to this blog page... This would then be the answer to the SPARQL query

PREFIX foaf: <>
SELECT ?homepage
   [] foaf:mbox "";
      foaf:homepage ?homepage
(note: that would be wrong as far as the definition of foaf:mbox goes, which relates a person to an mbox, not a string... but let us pass on this detail for the moment)

Here we would be defining a new GET method in SPARQL, which find the type of web page that the post would end up landing on: namely a page that is the homepage of whoever's email address we have.

The nice thing about this is that as with Toby Inkster's proposal we would only need one new relation from the home page to such a finder page, and once such a sparql form mapping mechanism is defined, it could be used in many other ways too, so that it would make sense for people to learn it. For example it could be useful to make web sites available to shopping agents, as I had started thinking about in RESTful semantic web services before RDFa was out.

But most of all, something along these lines, would allow services to have a very simple CGI to answer such a query, without needing to invest in a full blown SPARQL query engine. At the same time it makes the mapping to the semantics of the form very clear. Perhaps someone has a solution to do this already. Perhaps there is a better way of doing it. But it is along these lines that I would be looking for a solution...

(See also an earlier post of mine SPARQLing AltaVista: the meaning of forms)

How this relates to OpenId and foaf+ssl

One of the key use cases for such a Web Finger comes from the difficulty people have of thinking of URLs as identifiers of people. Such a WebFinger proposal if successful, would allow people to type in their email address into an OpenId login box, and from there the Relying Party (the server that the user wants to log into), could find their homepage (usually the same as their OpenId page), and from there find their FOAF description (see "FOAF and OpenID").

Of course this user interface problem does not come up with foaf+ssl, because by using client side certificates, foaf+ssl does not require the user to remember his WebID. The browser does that for him - it's built in.

Nevertheless it is good that OpenId is creating the need for such a service. It is a good idea, and could be very useful even for foaf+ssl, but for different reasons: making it easy to help people find someone's foaf file from the email address could have many very neat applications, if only for enhancing email clients in interesting new ways.


It was remarked in the comments to this post that the format for the /host-meta format is now XRD. So that removes one criticism of the first proposal. I wonder how flexible XRD is now. Can it express everything RDF/XML can? Does it have a GRDDL?

Friday Dec 12, 2008

ruby script to set skype and adium mood message with twitter on osx

Twitter is a great way to learn many little web2.0ish things. I wanted to set the status message on my Skype and Adium clients using my last twitter message. So I found a howto document by Michael Tyson which I adapted a bit to add Skype support and to only post twits that were not replies to someone else - I decide there was just too much loss of context for that to make sense.

#!/usr/bin/env ruby
# Update iChat/Adium/Skype status from Twitter
# Michael Tyson 
# Contributor: Henry Story

# Set Twitter username here
Username = 'bblfish'

require 'net/http'
require 'rexml/document'
include REXML

# Download timeline XML and extract latest entry
url = "" + Username + ".atom"
xml_data = Net::HTTP.get_response(URI.parse(url)).body
doc    =

latest = XPath.match(doc,"//content").detect { |c| not /@/.match(c.text)}
message = latest.text.gsub(/\^[\^:]+:\\s\*/, '')
exit if ! message

# Apply to status
script = 'set message to "' + message.gsub(/"/, '\\\\"') 
         + "\\"\\n" +
         'tell application "System Events"' 
         + "\\n" +
         'if exists process "iChat" then tell application "iChat" to set the status message to message' 
         + "\\n" +
         'if exists process "Adium" then tell application "Adium" to set status message of every account to message' 
         + "\\n" +
         'if exists process "Skype" then tell application "Skype" to send command "set profile mood_text "'
         + ' & message script name "twitter"'
         + "\\n" +
         'end tell' + "\\n"

IO.popen("osascript", "w") { |f| f.puts(script) }

This can then be added to the unix crontab as explained in Michael's article, and all is good.

What can one learn with this little exercise? Quite a lot:

  • Ruby - this is my first Ruby hack
  • Atom - twitter uses an atom xml feed to publish its posts
  • unix crontab
  • AppleScript to send messages to all these silly OSX apps
  • vi to edit all of this, but that's not obligatory, you can use less viral ones
  • the value of reusing data accross applications
So that's a good way to spend a little time when one has had a little bit too much to drink the night before. Hmm, is this what one calls procrastination (video)?

Tuesday Apr 22, 2008

history meme

For an hour or so I managed to forget that I had a lot of work to do. Catching up with my unread posts on my blogroll, I came across Tim Bray's History Meme post, and decided to try it out. So here are my results:

hjs@bblfish:0$ history | tr -s ' ' | cut -d ' ' -f 3 | sort | uniq -c | sort -rn | head -n 10
  115	ls 
   65	cd 
   60	ant
   42	vi 
   42	less 
   20	fg 
   17	cwm 
   15	svn 
   14	pwd 
    9	top 

ant is of course the build tool I use. I once laughed at the unintuitiveness of vi, but I later discovered it stands for virus - you just can't stop using it. cwm is the semantic web swiss army knife written in python: incredibly useful for every day semantic web hacking. And svn is the subversion command line client: I love subversion. When coding of course I spend a lot of time in either IntelliJ or NetBeans, depending on the day.

I have a number of other shells open, and the same commands tend to appear in them. Often the unix find command does. So I thought I'd try seeing what I get when I enlarge the list:

hjs@bblfish:0$ history | tr -s ' ' | cut -d ' ' -f 3 | sort | uniq -c | sort -rn | head -n 20
 113 ls
  55 vi
  54 cd
  36 less
  36 ant
  21 cwm
  20 fg
  16 pwd
  12 svn
   9 top
   9 find
   9 exit
   8 ping
   7 ssh
   7 killall
   7 defaults
   7 curl
   7 bc
   6 bzcat
   5 fink

Thursday Nov 08, 2007

Why Apple Spaces is broken

Space: they way I would like to set up my workspaces

I have been using virtual desktops, what Apple now calls Spaces™, since 1995 on X11 Unix. So I have quite good experience with this feature. I know what it needs, and I can tell very clearly that although Apple's implementations is the most beautiful version available, it clearly has not been thought through correctly. As a lot of Apple users will be novel to this, they may not quite understand what is broken immediately, but may come to the conclusion that it is not very useful. So first we have to explain why virtual desktops are useful. Then I can explain why Apple's implementation is broken.

The use case

The reason for developing multiple spaces is to be able to clearly separate one's work. I for example, have one desktop for Mail and other communication related activities, one for programming, one for blogging, and one for other tasks such as giving a presentation.

When I read mail, I sometimes need to browse the web to check up on links that people may have sent me. I don't want that to make me jump over to the browser I opened in my development space where I was reading javadoc. That would both mess up my development environment, and switch the context I was in. If no browser is open in my mail environment I would like to just be able to hit ⌘-N and have a new browser open up there. Then pressing ⌘-⇥ (command-tab) - which should only list applications available in that space or at least offer those applications available there in priority - I should be able to switch between my mail and the browser instances open in that space.

Having read my mail, especially the mail by my manager, telling me to stop helping Apple improve its copy of something that was available over 12 years ago on unix, and to get working on the next great ideas, I switch back to my NetBeans space, where I am developing a Semantic Address Book. Here I would like to switch quickly between the applications that are open in that space: Netbeans, my AddressBook, the shell, Safari and Firefox. So at the minimum, I would like the applications that are present in that space again to be the first in the ⌘-⇥ list. And! I would like it that when I switch to Safari to read the docs, I don't get thrown into my communications space.

There is no way I can have only one browser open for all of my work. I need different browsers open for different purposes in each spaces. The same is true with the shell. Sometimes it may even be true with Mail. Perhaps someone sends me an email relating to a piece of code I am working on. I would like to move that window to my editing environment (easy using F8 of course), in order to be able to switch between it and my editor with ease.

What's broken

So currently it is not possible to work like this with Apple's Spaces™. When switching between applications using ⌘-⇥, Spaces™ throws you across virtual desktops without any check to see if a window of that application is not already open in your space. Spaces™ always switches one to the virtual desktop that an application was first opened in, or where the first opened window from that application actually is. One cannot use the F9 or F10 Exposé keys either. Even though they only show the applications open in one space, they will still in some unpredictable way, switch you to a different space. They do this even if you clearly select a window from the space you are working in. So there is no way to switch reliably between applications open in one virtual desktop space, and so there really is no way to separate your different work related tasks. The way it is set up you need to have all your browsers in the same space, all your shells in the same space, etc... etc... So really these Spaces™ are not designed around a person's work habits, but around software components. That is the most basic of all User Interface failings.


30 May, 2008 Many, if not most, of the issues I complained of in this post have been fixed with release 10.5.3 of OSX. It seems useable now. ⌘-⇥ no longer randomly switches between workspaces, which was the biggest problem. John Gruber explains how 10.5.3 fixes spaces in detail on his blog.

Nov 20, 2007 eliottcable proposes a solution on quintessentially open. I am not convinced that ⌘-⇥ should create new windows on a space by default if there is non there. It should certainly not switch to another space if there is a window on the current space. In any case I find that the F9-F10 expose keys are clearly broken, since they do have me jump across spaces, when they never should.

This post received a huge number of readers from daring fireball. Thanks. Dave Dribin has a good write-up on this issue. Some further discussion is developing on the reddit discussion forum.

Monday Aug 20, 2007

Purple Ocean Strategy

A few weeks ago I read through "Blue Ocean Strategy: How to Create Uncontested Market Space and Make the Competition Irrelevant", By W. Chan Kim and Renée Mauborgne of the Insead business school. This book has sold millions of copies since its publication. It is very easy to read, and contains a lot of clear and entertaining business cases by way of illustration, from the growth of the Cirque du Soleil via the story of the turnaround of crime in New York under the leadership of Bill Braton, all the way to Apple's phenomenal success with the iPod.

The Blue Ocean the book refers to is opposed to the Red Ocean of competition in well established markets where optimization and distinction on well understood, standardized criteria matter. The Blue Ocean stands for the new markets created by businesses where there are no predefined standards, no predefined audience, where no industrial feet have yet been placed; in short the sought after space where there is no competition, where huge fortunes can be made. One of the very nice things about this book is how it shows just how much the blue ocean markets can be created in every walk of life, not just where one expects it the most, in technology driven industries.

The aim of the book is to show how these oceans of innovation are created. The tools it develops to make it possible to understand this are very easy to grasp, and make a lot of sense. One point it makes, and that every creator knows, is that you cannot find a Blue Ocean by asking your customers what they want, or by doing simple market studies. Of course these spaces are created by responding to something people really wanted, and feeling for your customers is an important aspect of seeing new possibilities emerge. But the business owner, the entrepreneur - as opposed to the manager ( the book does not make this distinction ) - is the creator of a new value space which cannot be comprehended by the market ahead of time. More so even since by creating something new, the entrepreneur is redefining the boundaries of the established market, and so redefining the audience. The Cirque du Soleil for example changed both the definition of what a circus was and what theater was. In doing that the Cirque du Soleil became a competitor of not just theater and circuses, but also other night time activities people might have enjoyed in their place. The Cirque du Soleil did that but seemed also to appear out of nowhere.

Blue is the symbol of Liberty. The French flag is blue, white and red: Liberty, Equality, Fraternity. Blue in Europe is also associated with conservatism. The history of color associations in the USA is more complex and currently has the reverse association in part due to the stigma attached to the color red in the battle against Communism. Just as with the colors the book presents what are probably very complex ideas in an amazingly simple way. It separates the strands of thought the way a crystal separates light. Like a beat of electronic music it drums these distinctions into the readers mind, so that there is probably no need to re-read the book twice: reading it once is to read it three times. So my following criticism or thoughts will probably be just very facile remerging of what was separated for clarity.

Following the internet and computer industries I have noticed an element of the relation between red and blue that this book fails to make. As our CEO Jonathan Schwartz often mentions on his blog, it is not because one is in a commodity market that one cannot make a huge profit. The electric plug in your house, voltage, wire sizes and many other parts of the electricity industry are standardized. Those are commodity markets. Yet companies like General Electric or Siemens that produce huge generators for large dams or other electrical installations are in some very profitable markets. Without the standardization of the plugs and voltages, the electricity industry could never have grown so big. Standardization I have noticed, can be a stepping stone to building a Blue Ocean, the blue can build on the red.

To illustrate let me take one example from the book: Apple's huge success in recent years. One of the conceptual tools put forward by Blue Ocean Strategy, is that one has to create a new value curve. Remove some aspects of cost and value from a product (no animals in the Cirque du Soleil), change other aspects of value (price), create something new (artistic dance show). One way Apple reduced cost was by adopting open standards. By building on the Unix Operating system developed and used in Universities world wide they removed the major research cost of developing an Operating System whilst gained a huge pool of ready and highly qualified experts worldwide, and all the software that had been built in an Open Source way over time. The default compiler of OSX is Gnu CC. Think of the huge cost reductions that flow from being able to build in such a way on the works of others. By adding the one thing that had been missing from that system, an artistically coherent and beautiful end user experience, Apple gained those people's hearts and support and gave them a unique value proposition, bringing a very important community to Apple that would never have touched it before. By building on these open standards Apple also brings value to the community, if only in the existential example that it can be done, but certainly also over time in feeding back the improvements to the community. Simon Phipps explains how this works in full detail in "The Zen of Free". The same forces at work also lead Sun Microsystems down a similar path to its logical conclusion: by Opening up all of the software stack. As a result Sun and Apple are able to cooperate in numerous ways that would otherwise have been impossible. By working on a standard base Apple can gain award winning technologies such as ZFS at very low cost, allowing it to focus on differentiating itself where its user base's value is: simple packaging, beauty and fluid end user experience. Apple's switch last year to Intel is a similar move, building this time on an industrial de facto standard.
In all these cases reducing costs is not removing something completely from the system as proposed by Blue Ocean (removing the lions), but building on the commoditization and standardization of one layer, thereby bringing the costs down to close to zero. Building on the Red Ocean of community ownership a Blue Ocean of innovation and creativity, in a way that respects the value of the Red Ocean, is what I would like to call here, on my little blog at the end of the universe, Purple Ocean Strategy.

Having gotten this far it may be necessary to enlarge the notion of what is Red all the way to Green. If Red is what is socially established, fraternal ownership, then further along there is what is common to all living things, the biosphere, the Green. A strategy that took this into account would be looking for how to use and build in a sustainable way on that space. It is clear that not taking this into account can be extremely damaging, as the unfolding drama of Beijing Olympics is revealing. How to take it into account effectlively, may get us to the Turquoise Ocean Strategy.




« July 2016