File transfer to a multi user chat


Background.
When a user wants to chat with his contact, typically he does so with one to one messages - that is, he sends a <message /> with the to set explictly to the reciepent. Now, suppose he invites another contact into this chat : it ceases to be a one to one chat - there is an xmpp protocol extension for multi user chat (muc) defined for this.
In most clients, like ours, there is no visible difference in both usecases : the client upgrades the one to one chat to a private conference behind the scenes and things continue from there.
Actually in our client, there is no visible difference between a private one to one chat, a private conference or a public conference - the UI is consistent across all these usecases.

The problem
Unfortunately, functionality is not strictly the same ... and one of the key areas where this affects is in file transfer : something which users are used to. Currently, there are a number of ways defined on how to negotiate file transfer between two entities - how to negotiate the parameters, how to do the actual transfer - peer to peer/inband, etc.
But when it comes to file transfer in a muc, there is no standard extension or proposal.
This does not mean that file transfer is not supported at all - it just means that there is no standard way to support this.
For more than a year now, our client and server has been supporting filetransfer even in the case of conferences.
The problem is that, this works only with our client and server - as there is no standard in this space, none of the other clients can interoperate with our clients when it comes to this feature.

Requirements
Let us look at a minimal set of requirements :
  1. Different clients used by the participants in a conference might support different set of file transfer profiles. Hence, there need not be a intersection of transfer profile supported by all.
  2. If a single client is to multicast the file to all the participants, the bandwidth requirements could choke it.
  3. The server might want to decide who gets the files in a conference, and might want means to enforce this. Ditto for who can send a file, file properties, etc.
Potential solution ?
I am not discribing what we support, but thinking of a really simple solution.
What could be done is :
  1. XMPP Server has an affiliated 'hosting service' - say a http, ftp, etc server : on which it has reasonably high control. Let us assume HTTP service in this discussion.
  2. This service is not browsable, so you cannot get listing of files - similarly, it must not be possible to easily guess/find arbitrary files from this store - unless the server has divulged the URI to you. Hence URI's generated should also be opqaue and non-guessable to a reasonable degree.
  3. When client wants to send a file to a conference, it will negotiate with the room - not with the participants.
  4. If transfer is to be allowed, the room will assign the client a URI to PUT the file to. Also, it might give it some sort of authorization credentials to be used at the HTTP service.
  5. Once the client has transferred the file to the URI, it will notify the room - which in turn will multicast the location of the file at this URI to required participants (along with creds to fetch it, if any) : the sender would be specified as the initial user.
  6. The participant clients will use GET to pick up the file from this service - while using the passed on creds.This way, even if you guess the URI, you will need to also guess the creds to retrieve it.
  7. After successful transfer, participant clients would notify the room of completion. A file could get purged when all participants have downloaded it, or if some room configured timeout expires - need not be a pro-active cleanup, but possibility of cleanup after both could be high (impl detail actually).

The caveats are obvious :
  1. We are introducing a new form of file transfer profile - though we are using HTTP above, it is still a new mechanism !
  2. You are going to have N participants per conference hitting this service for the file - so the service should be able to scale well, in terms of client connections and bandwidth. Might not really be a concern for modern webservers though.
  3. Associating URI based credentials might not be that simple : especially if you consider the tie-in required with the XMPP server for achieving this at runtime (PUT, GET creds) : can it be done ? Ofcourse it can be - out of the box support exists ? I am not very sure if it does.
  4. You might need some mechanism to clean/purge stale files from the hosting service.

Comments, suggestions ?
Comments:

What about simply doing normal XMPP file transfer to the room address and then having the room address send file transfer requests to all other participants? It has the advantage of being pure XMPP (so easier for clients to implement) and is only a simple change from the current file transfer protocol. I think the larger question is where the file transfer spec is going. Now that Google Talk has adopted file transfer based on Jingle is there any option but for the rest of us to follow? File transfer using ICE and media proxies is actually not a bad idea though.

Posted by Matt Tucker on November 19, 2006 at 04:43 AM IST #

Hi,
What you mention is actually an option we have considered, and it is one of the current implementations we support.
The problem we end up in this case is two fold :
  1. Each client will support a different set of file transfer profiles - so the muc will need to end up supporting the largest set possible : typically all profiles, so that it is gaurenteed that you can actually transfer files to all participants. (Currently for us, all clients which support this are our clients - and all of them support the same set of profiles : so we dont have this issue right now :-) ) : also note that, other than using inband, it might not be possible to PUSH the file to the reciepent from the muc room in case the participant is behind a NAT/filewall/proxy/jep124 client/etc
  2. The actual file transfer is not really a XMPP problem. In all the file transfer profiles we have, the negotiation of the transfer happens through XMPP and the actual transfer happens p2p (except inband - and we have a bunch of issues with that : will blog about it later). In the case described above, the recepient will become a muc room : and so we will end up defining a file server like capability for muc - along with state management for the file transfer for each participant : increasing the load on the muc.
The idea of a hosting service was essentially to :
  1. Standardize on some 'way' to send files - atleast acknowledge that multicasting files to a conference is different from a two entity p2p file transfer. I mention files above, intent is to be able to multicast a binary stream of data of fixed length.
  2. Decouple the actual file management problems from xmpp and delegate it to protocols which have been designed for this purpose.
Other than these two, it is just a wrapper over the idea you propose :-)
Regards,
Mridul

Posted by Mridul on November 19, 2006 at 06:25 AM IST #

You may want to have a look at the defered spec for a webdav file transfer. It could be adapted to the case of MUC w/out too much difficulties. In entreprise, web based transfers will definitively be appreciated. For other cases we may want to have an Jingle base negotiation for binay streams, and use Jingle media proxies to make sure it works with every NAT. Another interesting point would be to consider a de-coupling of the transfer into two phases: an upload directly negotiated by the "sender" with the MUC room, followed by a sharing amongst all participants. The adventage being that sharing is a well known process for media, and the same protocols could be applied to any binary IMO. My $0.02 Jean-Louis

Posted by Jean-Louis Seguineau on November 28, 2006 at 03:03 PM IST #

The webdav transfer is at http://www.xmpp.org/extensions/xep-0129.html

Posted by Jean-Louis Seguineau on November 28, 2006 at 03:04 PM IST #

Hi Jean Louis,
I was not aware of webdav file transfer mechanism (the extensions page does not list deferred versions). This is interesting - I will try to retrofit idea this within that proposal and see how things proceed.
From a pure webclient using the webdav server point of view, I can understand the requirements in the server - but if you are an xmpp entity, most of the protocol flows sounds very redundent :
  1. Need to guess an unused file path
  2. Need to use xep 70 to validate req's
Sure, this is needed to be webdav compliant I guess, but a short circuit for xmpp clients might also help.
Setting a proppatch does remove the need for sending saml assertions (or equivalent) to file 'retrievers' ... but then this does require a bit of tight integration between a webdav server and xmpp - though much better than the hotch-potch I describe above :-)
This is great, I will look more into integrating and using webdav.
Regards,
Mridul

Posted by Mridul on November 28, 2006 at 04:02 PM IST #

Post a Comment:
  • HTML Syntax: NOT allowed
About

mridul

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today
Bookmarks
Blogroll

No bookmarks in folder