Wednesday Feb 03, 2010

GlassFish @ Oracle

Two very interesting comments on the server-side from Mike Lehman and Eduardo about GlassFish @ Oracle.

This clarifies some mis-interpretations and counters some prematurely shot FUD.

Mike states at the end of his comment:

We are very excited to have GlassFish on board at Oracle. Areas like GlassFish proper and key constituent pieces like Metro, Grizzly, Jersey etc are state of the art; there is huge incentive to shepherd them correctly. You will have to judge us on how we execute over the next few months while we realign roadmaps and delivery schedules but I hope we can surprise folks with some good turns we can do as a combined company.

Eduardo states at the end of his comment:

So, my suggestion to GlassFish fans is to please give us some time to put together that roadmap, and then use it to make your own decisions.

Wednesday Aug 06, 2008

EHCache using Jersey

Ehcache is a general purpose distributed caching framework and server written in Java. Greg Luck has reported in a blog entry that the newly released Ehcache 0.3 now supports a RESTful interface using Jersey. Very nice! Greg provides more details on the interface in the blog entry.

Greg also goes into details on how to use this from the client-side. Using the Jersey client API makes it very easy to use the service, for example creating or updating a new cache entry:

String xmlDocument = "...";
Client c = Client.create();
WebResource r = c.resource("http://localhost:8080/ehcache/rest/sampleCache2/2");

Anything other than a 2xx response will result in an UniformInterfaceException being thrown. This makes it easier to manage errors returned from services.

Retrieving the cached entry as a DOM object can be performed as follows: 

Node n = r.accept("application/xml").get(DOMSource.class).getNode();

Again, anything other than a 2xx response will result in an exception being thrown. Alternatively it is possible to check more details of the response:

ClientRespone cr = r.get(ClientResponse.class);
if (cr.getStatus() < 300 && 
        cr.getType().isCompatibleWith(MediaType.APPLICATION_XML_TYPE)) {
    Node n = cr.getEntity(DOMSource.class).getNode();

Notice that in the above two cases the WebResource instance r is reused.

Caching can be performed on the client side too, so i am wondering if the Jersey client API could reuse Ehcache for a caching solution, embedded (file or in-memory) or networked. Jersey client filters could written and installed to check the cache first.

Wednesday Sep 26, 2007

SOAP/TCP interoperability with Metro and Noemax's SOAP/TCP transport for WCF

The specification of SOAP/TCP that is implemented in Metro is now available under Sun's non-assertion covenant.

Noemax implemented SOAP/TCP as specified and have recently announced the SOAP/TCP transport for Microsoft's Windows Communication Foundation.

Metro-based clients and services are now interoperable with WCF-based clients and services when using SOAP/TCP to communicate efficiently.

Wednesday Mar 07, 2007

Interoperablity Announcement

Great news!

Liquid Technologies Ltd, Noemax Technologies Ltd and OSS Nokalva, Inc. have announced in
respective press releases (here, here and here respectively) successful interoperability of Fast Infoset tools from the following vendors: Liquid Technologies Ltd., Noemax Technologies Ltd., OSS Nokalva, Sun Microsystems Inc., and TMax Soft Inc.

Fast Infoset is now available and interoperable on a wide variety of platforms such
as Microsoft .NET and .NET CF, Sun GlassFish, BEA WebLogic, Java SE 6,
IBM SDK for Java v6 Early Release, TMax Soft JEUS 6, as well as Linux, Solaris, and Win32.

Tuesday Nov 21, 2006


SOAP/TCP is the project name for a transport that efficiently sends SOAP messages over a TCP connection. It has recently been added to the source of WSIT. I helped design SOAP/TCP but i will let Alexey talk more about this in detail when he gets back from holiday as he did all the hard work and implemented it!

What i wanted to highlight is a mode in which SOAP/TCP uses Fast Infoset. SOAP/TCP relies on keeping TCP connections alive (opening a TCP connection is a costly operation). This means state can be associated with a connection. In this respect state can be information associated with encoding and decoding Fast Infoset documents, namely the vocabulary, and this state can be maintained by the server and the client.

In a nutshell what this means is the first message that is sent will contain strings for tags and namespaces and a second, identical message, that is sent will not contain such strings. This makes for smaller messages that are more efficient to produce and consume, at the expense of maintaining state. And the smaller the messages the more important the transport meta data be small, which for SOAP/TCP is highly optimized.

Further more it is also possible to do the same with attribute values and character content for say small strings, or in fact prime the state with a known set of strings when opening the connection.

All in all this is a very effective way of retaining the loose coupling of no schema-dependency with the advantages that schema-dependency would offer for the reduction in message size and increase in processing efficiency.

Tuesday Oct 10, 2006

Four way interop

Liquid Technologies joins the group of vendors (Noemax, OSS Nokalva and the Sun, using the implementation) that have successfully parsed each others fast infoset documents produced from the same set of XML documents.

Wednesday Oct 04, 2006

Fast Infoset encoding explained by simple examples

I have uploaded some slides to here that explain the Fast Infoset encoding by example. I presented these slides to the EXI Working Group a couple of weeks ago.

Sometimes it is quite tricky to clearly explain encodings. The best technique i have found so far is to briefly explain some core concepts then get deep down and dirty with the bits/bytes of a few examples. Starting out by explaining abstractions and then drilling down through the layers tends to confuse matters when someone is not familiar with the terminology. Abstractions are, of course, wonderful things, especially when you want to specify the bits/bytes, but sometimes they can get in the way.

On a slightly tangential topic i have found that the use of examples is a great way to explain REST-based Web services i.e. showing example (or real world) HTTP requests and responses. (Hands up who learned HTML from looking at examples!) I have found it makes it easier to start hacking.

Tuesday Oct 03, 2006

Three way interop parsing FI documents

Three vendors (Noemax, OSS Nokalva and the Sun, using the implementation) have successfully parsed each others fast infoset documents produced from the same set of XML documents.

The FI interop project contains in CVS test FpML and UBL XML documents here and the vendor produced fast infoset documents here.

I recently committed typed fast infoset documents (here) converted from the FpML and UBL XML documents (and utilizing the corresponding XML schema).

The plan is to add more test XML documents to increase the scope of the testing and thereby increasing the confidence that different vendors can interoperate.

XML to typed FI utility

I have just created a simple utility and UNIX shell script that will take as input a XML schema and an XML document, and convert that to a fast infoset document.

The XML schema is used to work out XS data types associated with lexical values (text content and attribute values) within the XML document. The lexical values are converted to binary data using a simple mapping between the XS data types and the Fast Infoset encoding algorithms and restricted alphabets.

To use (on a UNIX system. Warning: i have only tested on Solaris) download the latest fast infoset distribution here, and unzip it. Set the FI_HOME and FI_UTILITIES_HOME environment variables to the unziped location of the distribution and include in your path ${FI_HOME}/bin.

As an example, download the latest fast infoset source distirbution here, and unzip it in the ${FI_HOME} directory. Change directory to samples/data. Then type;

  xmltosaxtotypedfi schema/Content.xsd content.xml

This will convert the XML document:

<content xmlns="">
    <floats value="3.14159265 2.71828183">3.14159265 2.71828183 1.0</floats>

to a fast infoset document. The base64 lexical values will be convert to bytes  (in this case all zeros) and the arrays of float lexical values will be converted to arrays of IEEE floats.

The octets of the FI document are as follows (using od -A x -tx1a):

0000000  e0  00  00  01  00  38  cd  1d  68  74  74  70  3a  2f  2f  77
           ` nul nul soh nul   8   M  gs   h   t   t   p   :   /   /   w
0000010  77  77  2e  73  75  6e  2e  63  6f  6d  2f  78  6d  6c  2f  63
           w   w   .   s   u   n   .   c   o   m   /   x   m   l   /   c
0000020  6f  6e  74  65  6e  74  f0  3d  81  06  63  6f  6e  74  65  6e
           o   n   t   e   n   t   p   = soh ack   c   o   n   t   e   n
0000030  74  92  02  0a  20  20  20  20  7d  81  05  62  61  73  65  36
           t dc2 stx  lf  sp  sp  sp  sp   } soh enq   b   a   s   e   6
0000040  34  78  04  76  61  6c  75  65  30  18  03  00  00  00  00  00
           4   x eot   v   a   l   u   e   0 can etx nul nul nul nul nul
0000050  00  00  00  00  00  00  00  f0  8c  06  09  00  00  00  00  00
         nul nul nul nul nul nul nul   p  ff ack  ht nul nul nul nul nul
0000060  00  00  00  00  00  00  00  f0  a0  7d  81  05  66  6c  6f  61
         nul nul nul nul nul nul nul   p  sp   } soh enq   f   l   o   a
0000070  74  73  00  30  67  40  49  0f  db  40  2d  f8  54  f0  8c  1a
           t   s nul   0   g   @   I  si   [   @   -   x   T   p  ff sub
0000080  09  40  49  0f  db  40  2d  f8  54  3f  80  00  00  f0  90  0a
          ht   @   I  si   [   @   -   x   T   ? nul nul nul   p dle  lf
0000090  ff

Notice that you cannot see the lexical values for the base64 or arrays of float.

The fast infoset document can be converted back to an XML document by doing the following:


Which will produce the following written to the standard output:

<content xmlns="">
    <floats value="3.1415927 2.7182817">3.1415927 2.7182817 1.0</floats>

Notice that the documents are not identical! The exact lexical values of the floats are not preserved because the conversion of some lexical values to IEEE float is lossy. Also notice that the FI parser will, by default, convert binary data to lexical values, thus ensuring that typed fast infoset documents can always be parsed. Further notice that you do not require the schema to parse the fast infoset document.

Friday Aug 25, 2006

Fast Infoset 1.2 and JAXB 2.01

I created a branch of Fast Infoset (FastInfoset-1_2-branch) with some experimental features for marshalling/unmrsahlling in JAXB 2.1. Latest builds can be found here.

It is mostly about improving the performance of FI serialization when marshalling. JAXB has knowledge of the schema and that knowledge can be used to improve performance (as is for serialization to XML documents).

First, JAXB can pre-encode known local names Strings to UTF-8 so it is not necessary to keep encoding strings for each marshall when using the same JAXB context. Second JAXB can assign unique integers to known local names and qualififed names. Using these unqiue integers it is possible to determin in O(1) time whether a local name or qualified name has occurred before or not when marshalling a JAXB bean and therefore whether the local name or qualified name should be encoded literally or using an index.

For those interested in the gritty details have a look at: LowLevelFastInfosetStreamWriterStAXDocumentSerializer; and FastInfosetStreamWriterOutput.

Fast Infoset integrated into JAX-WS 2.0.1

Fast Infoset is now fully integrated into JAX-WS 2.0.1 (the rearchitectured stack) and it should be appearing in a build of Glassfish fairly soon (milestone 2).  It can be enabled in exactly the same manner as for JAX-WS 2.0.

Fast Infoset was much easier to integrate into JAX-WS 2.0.1 than JAX-WS 2.0 as 2.0.1's architecture more cleanly separates out the reprentation of a message from the encoding/decoding of that message.

Tuesday Jul 25, 2006

Interop between WCF and Sun's app server 9.0

Alexander from Noemax recently announced on the Fast Infoset interop users list that basic interop has been achieved with their C# Fast Infoset implementation integrated into Microsoft Windows Communication Foundation.

Great news!

Now we need to do some more substantive Web service interop tests and Alexander has some ideas here (i wish list viewer would wrap lines!)

Thursday Jun 15, 2006

Yet another Fast Infoset implementation

OSS Nokalva has announced suppport for Fast Infoset. See their OSS Fast Infoset Tools.

That makes at least six implementations of Fast Infoset (, FIME, FIFI, Liquid XML, Noemax SDO & OSS Fast Infoset Tools) on various platforms/frameworks. Anyone who builds on top of Glassfish with the JAX-WS 2.0 RI can also avail of Fast Infoset as well.

Me thinks it is time to organize some interop party :-) ....

Thursday Jun 08, 2006

Fast Infoset and use with scientific data

I read with interest a paper called A Binary XML for Scientific Applications:

"In this paper we present a binary XML format and implementation for scientific data called Binary XML for Scientific Applications (BXSA). We show that per-formance is comparable to that of commonly used scientific data formats such as netCDF. These results challenge the prevailing practice of handling control and data separately in scientific applications, with web services for control and specialized binary formats for data."

And it concludes:

"This rough performance parity demonstrates that binary XML is suitable as an alternative to specialized scientific data formats. Storing and sending as XML would then also directly facilitate the use of various data techiques such as semantic mediation, ontologies for data, and provenance tracking."

"Web services will not solve all format problems. They are a promising, widely-adopted approach, however, with an active research and development community. By unifying control and data into one framework with binary XML efforts such as BXSA, the promise of web services can be more fully realized for scientific computing."

I agree very much with the conclusions that binary XML efforts can very useful in network communication for scientific computing purposes.

However, what i do not agree with is their review of Fast Infoset:

"Fast Infoset. The Fast Infoset [19] is an effort by Sun to develop a fast serialization for XML. Unfortunately, since it is being submitted as an ISO specification, it is currently not publicly available for review and comment. It is based on ASN.1 [15], which is commonly used in the telecommunications industry. ASN.1 is a large, complex standard, and is probably not ideally suited to the needs of scientists."

On the contrary, i think Fast Infoset is very much suited for the needs of scientists as discussed in the paper.

Fast Infoset was standardarized jointly in the ITU-T and ISO organizations. It is true that the Fast Infoset standardization process at ITU-T and ISO does not have a public review period like the W3C, but the standard will soon be available for free at ITU-T and ISO. A disadvantage of not having a public review period does mean that information about Fast Infoset is not always obviously available. However, a search will quickly give all you need. For a clear and precise description see the Wikipedia entry.

Fast Infoset is  specified using all the formal semantics of ASN.1 but a Fast Infoset implementation does not require an ASN.1 toolkit. The Fast Infoset implementation at is an entirely stand alone Java-based implementation of Fast Infoset that supports the SAX, StAX and DOM APIs, and this implementation is used in our application server (and soon to be used in Glassfish).

Fast Infoset supports the direct encoding of arrays of floats and doubles as specified by IEEE 754. The Fast Infoset specification is being used by Web3D as the foundation for specifying the binary encoding of X3D constructs. X3D makes extensive use of numeric data. Fast Infoset has an extension mechanism that allows X3D to specify optimal encodings of numeric data.

Fast Infoset supports many of the properties that the BXSA format supports. It would be interesting to compare the performance of Fast Infoset with BXSA and the scientific data formats netCDF and HDF5.

An example of using Fast Infoset with arrays of Java primitive types can be found here.

I would be very interested in understanding how better to make Fast Infoset and Web services more useable to the scientific community. If you are a member of such a community and have investigated or want to investigate Fast Infoset or binary XML in general then feel free to drop me an email.

Friday Jun 02, 2006

Encoding/Decoding primitive types

A developer on the Fast Infoset user list was asking a question recently (see this thread) on how the Fast Infoset parsers/serializers can encode/decode binary data.

I promised to code some samples and also to blog about such support so... ...I find that questions/requests/issues/bugs are a great way of getting and focusing my attention on what developers/users find important :-)

For now i have created one sample in the fi/samples project, PrimitiveTypesWithElementContentSample.

This sample shows how primtive types, specifically arrays of byte[], int[] and float[] can be encoded in binary form for element content when serializing a fast infoset document, and decoded as byte[], int[] or float[] repectively when parsing the document. Further more, the sample shows that it is still possible to obtain the binary data in lexical form, which is important if the client does not desire or cannot process the primitive types.

I plan to add another sample soon that shows the same for attributes.

The Fast Infoset SAX serializers and parsers provide full support for what are known as the Fast Infoset encoding algorithms, which specify how various information may be encoded in binary form without any loss of lexical information. This is achieved by supprting as an extention to the Fast Infoset SAX API a PrimitiveTypeContentHandler which provides methods to write or access Java primitive types and arrays of.

Currently the StAX serializers and parsers only support the serializing/parsing of byte arrays. Work needs to be done to extend support for binary types that is compatible with the pull model (Derek has some good thoughts/experience on this). Any help on this area, coding/design, would be much appreciated!



« April 2014