Wednesday Feb 25, 2015

Why Checkstyle matters...

I’ve read something about ZeroMQ and its java clone recently and decided I should give it a try. JeroMQ is quite small project (same as ZeroMQ I guess), so I decided to checkout the sources directly, since that usually gives me last version of everything, including samples etc. When I opened the project in my IDE, I noticed some minor issues in the code, starting with some unnecessary properties, ending with missing null checks and invalid String comparisons. (You can find these in any project, so that was not so surprising).

Interesting part started when I fixed the issues locally and ran “mvn clean install” to verify my changes, build failed. No surprises there ;) but it failed on check style issues. That was something I did not expect. It turned out, that the project enforces defined set of rules. Not saying that I like the style chosen for that particular project (common guys, its Java, not C), I really liked the idea behind that decision.

Generally, it is far better when single project uses single code style – developer might need to adapt to that and it can take time, but for me, when I got past this period, code is much easier to read and understand, because I don’t need to think about that particular formatting, looking for potentially mistyped code blocks or parenthesis etc. Even when switching among projects with different code style, it is still better than having to adapt to various styles in single module or even class. (Yes, that happens. In any older project, which accepted contributions from lot of people).

After this reflection, I thought about applying this to one project, where I can do almost anything I want – Tyrus. Short version: I did it!

Slightly longer version: It was a pain. Really. And I don’t have by any chance that rich check style rule set as mentioned JeroMQ. To be honest, I thought that we should be fine, since we do often detailed code reviews, but you can always miss something which is not really well formatted.. Nice fact about Checkstyle is, that there are no compromises. You can exclude some rules for some class or even some lines of that class, but .. why? It would add unnecessary housekeeping when modifying the code and it could be accidentally disabled by just refactoring the class name. So I went for “zero tolerance” for defined rules.

The problem is, that there is no simple tool which would fix all check style issues. It kind of makes sense, since lots of them are not easily fixable automatically. Anyway, I found some recipe which involved Eclipse plugin and generating Eclipse formatter from checkstyle.xml. That worked somehow and the formatter can be exported and included to other IDEs (I personally use IntelliJ). There are some issues. One of caveats was max line length. The formatter was able to “clean” these warnings from the build, but sometimes produced code like:

1
2
3
4
5
6
7
8
9
10
if (wseMaxSessionsAnnotation != null) {
TyrusServerEndpointConfig.Builder builder = TyrusServerEndpointConfig.Builder.create(annotatedClass,
wseAnnotation
.value()).
encoders(
encoderClasses)
.decoders(decoderClasses)
.subprotocols(
Arrays.asList(
subProtocols));

Which is far from acceptable :). Long story short, I ended up with defining own formatting rules, setting them in IntelliJ code style, reformatted and then manually checked and fixed issues in all changed source files. It was originally ~500 files, it shrunk a little after my review, since I removed all non-java files from that change. You can see the commit at github (don’t worry if it takes some time to load, it is a bigger one).

Anyway, I’m still glad I did that. Now I know that once the build or pull request passes, it has the code already formatted in a way, that I don’t need to check it (that much). If you are interested, there is checkstyle-build.xml used for build time validation, and the configuration in pom.xml.

What can be better? IntelliJ and I guess even other IDEs could be more accessible for checkstyle configuration. Meaning that formatter/code style/… should have better support for importing checkstyle config file + it should be able to actually use it well. The latter statement is almost true (for some edgecases, which cannot be easily solved I guess), so the first one is more important. Since I know that the settings are already there and its only about correct values of some properties, I cannot see it is not already there.

That’s it for now ;) Hopefully you’ll feel inspired and try to at least run the checkstyle configured to your code style and see whether you can incorporate it easily in your build. If you have any suggestions or comments about this topic, please share them with me, especially if I left out something usable for Tyrus or other projects. Thanks!

Thursday Feb 12, 2015

Tyrus 1.10

Tyrus 1.10 is a maintenance relase, so unfortunately no cool new features to highlight, but we were able to add some new examples – Shared collection and BTC Xchange.

Complete list of fixed issues is below. I would like to thank to java.net users “toto2″ and “gray” for their help with GLASSFISH-21213. There is one other issue worth mentioning and that is TYRUS-329. In 1.10 release, Tyrus (after added tests for Per-Message compression) becomes fully compliant with Autobahn Test Suite.

Let me shortly introduce the new samples which were added in this version, just to make this blog post justifiable :).

Shared Collection Sample

In this example I tried to explore one possibility of data sharing between JavaScript client and Java code. The sample name could be misleading, the only implemented collection is a Map, which is backed up by WebSocket connection going back to the server. Both sides do implement “standard” Map, with some added flavour of distributed structures – update listener.

If you want to try it, you can follow the instructions in README.html; the application should present itself with a simple JavaScript based interface to modifying the content of the map. Don’t modify the content right away, open it in another window and observe how fast are changes propagated to other window(s).

BTC Xchange Sample

Another sample which demonstrates simple-to-implement communication with JavaScript code. On the title page you can see the graph, which in this case represents current exchange rate BTC/USD and there are some buy and sell proposals which can be executed. The sample currently allow you to go to “red numbers”, but you can create few simple modifications and make a “game” from that – the sample can be also opened from multiple windows and you can try to compete with someone else; for example, you can make a goal of getting some amount of USD or BTC :)

Complete list of changes

  • [TYRUS-394]  – Proxy bypassed on Linux but not on Windows.
  • [TYRUS-391]  – Tyrus reconnect handler not reached on Android.
  • [TYRUS-393] – Incorrect handling of sending whole message during (unfinished) partial message.
  • [TYRUS-329] – Deflate extension fixes – autobahn test suite.
  • [GLASSFISH-21213] – Deadlock (HttpSession vs TyrusWebSocket).

Monday Feb 09, 2015

Is WebSocket Session really thread safe?

Todays article would be about one simple use-case of JSR-356: Java API for WebSocket. This was brought to my attention by Mark Thomas, who started relatively long thread on the jsr356-experts mailing list. The described problem can be simplified to following code:

1
2
session.getAsyncRemote().sendText(message);
session.getBasicRemote().sendText(message);

You might be wondering what is the problem here.. first message is sent asynchronously and the second one synchronously. Session object itself is meant to be thread safe (last paragraph on class level javadoc), but there is a problem with container itself, more precisely with RemoteEndpoint.Basic interface, see following quote of its documentation:

If the websocket connection underlying this RemoteEndpoint is busy sending a message when a call is made to send another one, for example if two threads attempt to call a send method concurrently, or if a developer attempts to send a new message while in the middle of sending an existing one, the send method called while the connection is already busy may throw an IllegalStateException.

What I get from this paragraph is that the second line (from the sample above) can throw an IllegalStateException, when previous message is not sent. To be honest, reference implementation did not throw that exception (ever) and I don’t believe that any sample application or any WebSocket application I wrote checks for that state. Anyway, the javadoc does not mention that it MUST throw an exception, so for the mentioned sample, it is ok to block the second send until both messages are sent.

Unfortunately, that’s not all :). WebSocket API also allows sending partial messages and when you mix partial messages and whole messages in this context, you might start getting small headache… let’s consider following:

1
2
3
session.getBasicRemote().sendText(message, false); // partial msg
session.getBasicRemote().sendBinary(data);
session.getBasicRemote().sendText(message, true); // partial msg

In this sample, the second message should not be sent before the message from third line. Problem arises when this sequence is executed from a single thread – if second line blocks until text message is completed, we would have a problem. Implementation cannot really assume what will or won’t happen and generally it does not want to block the thread for a long time, so throwing IllegalStateException seem to be relatively OK.

(Sidenote: reference implementation, Tyrus, did not handle this scenario correctly.  The issue related to this functionality was filed as TYRUS-393 and the fix will be present in next released version – 1.10)

Since this seem to be grey area in the specification (it says what “may” be done, not what MUST or MUST NOT be done), the implementation can choose whatever it want – actually I believe that blocking the thread and waiting until it can send a message is valid, specification compliant approach. For Tyrus, I implemented little more defensive solution: the implementation waits (blocks the thread) for short period of time and if the conditions still don’t allow sending the message, IllegalStateException will be thrown.

Conclusion is not as clear as it should be – WebSocket Session object is thread safe for sure, but there are some conditions and states, which might be surprising and confusing for common users of this API. All of them can be prevented by following some rules when using WebSocket Sessions – for example if you are sending only whole messages synchronously, you don’t need to care about anything. If you are sending messages only asynchronously, it should be similar, but you might encounter some discrepancies among the implementations. When you want or need to send partial message, the issue might be harder to deal with – since WebSocket Session does not offer any way how to check the current state, it will need to be stored somewhere else and checked before invoking send* methods. My current advice is that the applications you write should add handling of  IllegalStateException from RemoteEndpoint.Basic.send* methods, especially if you expect that it can be run on different JSR 356 implementation.

Wednesday Jan 14, 2015

Reducing WebSocket client jar size with ProGuard

We already know there is a demand for standalone web socket client runnable even on different platforms than just Java EE or Java SE – Tyrus publishes client runtime all-in-one bundles for Java 6 (works on Android!) and Java 7+ (takes advantage of NIO API introduced in Java 7). In following article, we will try to create optimised version of these bundles to see whether they do contain some redundant classes. (spoiler: they do).

Tool of the choice for todays exercise would be ProGuard. Maybe not that easy to use, but it is low-level tool capable of various optimisations. We won’t cover any obfuscation, our goal is to create smaller jar with the same functionality.

Starting point is:

551K tyrus-standalone-client-jdk-1.10-SNAPSHOT.jar
1.9M tyrus-standalone-client-1.10-SNAPSHOT.jar

These jar files are representing two distribution which were already mentioned – client-jdk is Java 7 based client, client jar is Grizzly based (JDK 6+). For those who don’t know, Grizzly is Java NIO container, so it basically does the same thing as Java 7 NIO API, but in a little different manner. Also, Grizzly is the NIO framework of choice for Glassfish, Java EE reference implementation.

Downloading and executing ProGuard is not complicated, there is even GUI if you are more used to this kind of applications. After initial study and few trial runs, it seems I can produce optimised jar files with following configuration:

tyrus-client.pro
1
2
3
4
5
6
7
8
9
10
11
12
13
14
-dontoptimize
-dontobfuscate
-keepparameternames
-keepattributes Signature,InnerClasses,*Annotation*
-keepclassmembers,allowoptimization enum * {
public static **[] values();
public static ** valueOf(java.lang.String);
}
-keep public class org.glassfish.tyrus.** { public protected private *; }
-keep public class javax.websocket.** { public protected *; }
-dontwarn

 The command line for running this configuration against Tyrus client distributions would look somehow like following:

1
proguard.sh -injars ./client-jdk/target/tyrus-standalone-client-jdk-1.10-SNAPSHOT.jar -outjars tyrus-client-jdk.jar -libraryjars $JAVA_HOME/jre/lib/rt.jar @tyrus-client.pro

(previous line is for client-idk distribution, but it is very similar to the other one; just for the completeness of presented data, I’m using JDK 8; if you are using different version, your “-libraryjars” parameter might need to be adjusted).

Result:

533K tyrus-client-jdk.jar
1.2M tyrus-client.jar

For the Java 7+ client distribution, the difference is actually quite small, so we can make an assumption that the distribution is already quite optimal. The other is different story – but there is a good reason for it. Java 6+ distribution is based on Grizzly and Grizzly is mainly written as a server, not as a client. We are reusing some stuff from there, but the problem seem to be in not optimal modularization for client usecases. Anyway, just to sum that up, ProGuard was able to find and remove approximately 700 kB of “not used code”.

The conclusion is that you should consider using tool similar to ProGuard, especially when you are redistributing any java application. Special note for Android developers: you are already doing this, maybe even without knowing – DexGuard seem to be standard part of Android application development lifecycle.

Monday Dec 01, 2014

New Tyrus sample - Shared Collection

Sometime in JavaOne 2014 timeframe colleague came to me and showed me Meteor.js library, which seems to be really nice javascript framework, client and server side (kudos to the authors!). I started looking at the docs and examples and discovered leaderboard sample. In short, it is about having javascript collection backed up by some (persistent) storage on the server side, which is also shared (or replicated, if you wish) to all other clients. Meteor.js does this in Javascript “class” called Mongo.Collection (used to be Meteor.Collection, but it was renamed before 1.0 release). I really liked the idea and dug little bit more into the implementation and saw that they are using Socket.IO for the communication with the server, which is basically websocket (+fallback and other handy features).

Long story short – I wanted to do something similar and ideally publish it as Tyrus sample :-) and it wasn’t that hard! Obviously, this task is more about client javascript implementation than about what is going to happen on the server side, but it was refreshing experience – next step is Tyrus version of  Socket.IO (nah, I’m kidding…).

I don’t really want to go into details in javascript implementation, since I don’t feel very confident about that (and you can check it out in the Tyrus sources), but the goal was to create a java map-like API to store data and have it synchronized using WebSocket with all other client.

1
2
3
4
5
6
7
8
9
10
/**
* Create new Tyrus map.
*
* @param {string} url
* @param {function} updateListener
* @constructor
*/
Tyrus.Collection.Map = function (url, updateListener) {
// …
}

I created Tyrus.Collection namespace and “class” Map in it. The constructor have two arguments – URL of websocket endpoint and update listener. Update listener is invoked whenever the map is changed (this is not optimal, so currently it might be called even when you are putting the key value pair which is already there). Map has following public methods, which should feel familiar for java developers:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
/**
* Get size of the map.
*
* @returns {Number} number of records in the map.
*/
self.size = function () { };
/**
* Return {@code true} when the map is empty.
*
* @returns {boolean} {@code true} when the map is empty, {@code false} otherwise.
*/
self.isEmpty = function () { };
/**
* Get value corresponding to provided key from a map.
*
* @param {string} key key.
* @returns {*} value for corresponding key or {@code null} when there is no such key.
*/
self.get = function (key) { };
/**
* Put an item into the map.
*
* @param {string} key key.
* @param {*} value value.
*/
self.put = function (key, value) { };
/**
* Remove key (and corresponding value) from the map.
*
* @param {string} key key to be removed.
*/
self.remove = function (key) { };
/**
* Clear the map.
*/
self.clear = function () { };
/**
* Get the key set.
*
* @returns {Array} array containing all keys from the map (as indexes AND values - TODO).
*/
self.keySet = function () { };

Server side implementation was quite simple and it currently stores the info only in static map. You can easily extend this by persisting the map to file/database/anywhere else. The communication is driven by very simple protocol, which broadcasts mutating messages to other clients. All get operations are handled by client-side javascript, the map on the server side is kept in sync only because it is a source for clients connected later – this is why this is just “shared” map, not “distributed” (which would be more complicated to implement). The source of the sever endpoint can be found on github, in samples directory: SharedCollectionEndpoint.

Instruction how to run this sample are included in README.html in sample root dir.

When you access the front page, you should see something like:

You can add/modify existing items in the map by using the form on the bottom side (it will automatically load an item when you click on it) and you can remove any item by clicking on the red cross on the right side. All changes should be automatically propagated to all other clients (just open the page in another browser window).

Hope you’ll enjoy the new sample and if you have any comments or suggestions, please send us a note to users@tyrus.java.net.

Links

Friday Nov 21, 2014

Tyrus 1.9

Long time no see! After a while (longer than anticipated, sorry about that), we cut off Tyrus 1.9 release. Not much changed in the code since last release, but we were able to squeeze in some stability updates and performance improvements. I’ll highlight some of the features or fixes in next paragraphs:

What’s new?

Petr worked on new feature tracked as TYRUS-289 – Debug mode. Now you can enable more verbose logging on server, which would start logging data like handshake request and response, application and endpoint configuration and information about sessions and even messages, if you want. This could be useful for answering questions like “Why I cannot access my websocket endpoint?!” and so on.

Ondřej spent some time playing with debugger and found out few code paths, which were improved. Namely instantiating default (stateless) encoders and decoders per endpoint, which will save some memory and unnecessary calls to message localization code.

Another topic worth mentioning is that we fixed (again) client runtime on Android, so if you are looking (or already using) Tyrus client on Dalvik/Art, you should be able to update to 1.9.

Complete list of resolved JIRA tickets is below. As always, if you have any feedback, please use our mailing list users@tyrus.java.net or create a new bug or enhancement request  at https://java.net/jira/browse/TYRUS/. You can also use stackoverflow – in that case, please use tag “tyrus“.

Complete list of changes

  • [TYRUS-285] – Improve broadcast by sending from multiple (configurable) threads.
  • [TYRUS-289] – DEBUG mode (client/server)
  • [TYRUS-307] – Revise ignored tests
  • [TYRUS-342] – Tyrus throw CancellationException
  • [TYRUS-355] – HTTP Redirect – Location header cannot be a relative address
  • [TYRUS-361] – JVM does not exits if shared transport is enabled
  • [TYRUS-365] – Broadcast messages are not included in the monitoring statistics
  • [TYRUS-366] – master branch not buildable with JDK 1.6
  • [TYRUS-368] – Instantiate default encoders and decoders once per endpoint
  • [TYRUS-369] – Format localized messages when necessary
  • [TYRUS-370] – Masking key is generated on server side
  • [TYRUS-371]– Introduce an option for using custom masking key generator.
  • [TYRUS-375] – Buffer overflow during HTTP Response size parsing (jdk client container)
  • [TYRUS-376] – Investigate how/whether is ScheduledExecutorService terminated when session is closed (related to heart beat feature)
  • [TYRUS-379] – Autobahn testcases 7.3.4 & 7.3.5 fail in close stage
  • [TYRUS-380] – setContextClassLoader throws “access denied” exception in JNLP client app
  • [TYRUS-381] – exception message error
  • [TYRUS-382] – onError parameter exception is wrapped in InvocationTargetException when an exception is thrown from programmatic endpoint (onOpen or onClose)
  • [TYRUS-383] – session.getRequestParameterMap does not contain query parameters when running on Grizzly standalone server
  • [TYRUS-385] – Session idle timeout concurrency issues
  • [TYRUS-386] – Dalvik class loader rejects TyrusEnpointWrapper
  • [TYRUS-387] – Unable to use Tyrus in android appllication

Friday Oct 03, 2014

Slides from my presentations from JavaOne 2014

I was given the opportunity to attend and speak at JavaOne this year. I’m pretty sure that the slide decks will be made available by conference content system, but If you have not attended or maybe want them little bit faster, you can see it on slideshare. I had two talks, the furst was about the future of Java API for WebSocket (JSR 356):

and the other one was about using WebSockets in enterprise applications:

As always, any questions or comments are welcomed!

Monday Aug 18, 2014

Tyrus 1.8

Another version of Tyrus, the reference implementation of JSR 356 – Java API for WebSocket is out! Complete list of fixes and features is below, but let me describe some of the new features in more detail. All information presented here is also available in Tyrus documentation.

What’s new?

First to mention is that JSR 356 Maintenance review Ballot is over and the change proposed for 1.1 release was accepted. More details about changes in the API can be found in this article. Important part is that Tyrus 1.8 implements this API, meaning you can use Lambda expressions and some features of Nashorn without the need for any workarounds.

Almost all other features are related to client side support, which was significantly improved in this release. Firstly – I have to admit, that Tyrus client contained security issue – SSL Hostname verification was not performed when connecting to “wss” endpoints. This was fixed as part of TYRUS-339 and resulted in some changes in the client configuration API. Now you can control whether HostnameVerification should be performed (SslEngineConfigurator#setHostnameVerificationEnabled(boolean)) or even set your own HostnameVerifier (please use carefully): #setHostnameVerifier(…). Detailed description can be found in Host verification chapter.

Another related enhancement is support for Http Basic and Digest authentication schemes. Tyrus client now enables users to provide credentials and underlying implementation will take care of everything else. Our implementation is strictly non pre-emptive, so the login information is sent always as a response to 401 Http Status Code. If the Basic and Digest are not good enough and there is a need to use some custom scheme or something which is not yet supported in Tyrus, custom Authenticator can be registered and the authentication part of the handshake process will be handled by it. Please seeClient HTTP Authentication chapter in the user guide for more details.

There are other features, like fine-grain threadpool configuration for JDK client container, build-in Http redirect support and some reshuffling related to unifying the location of client configuration classes and properties definition – every property should be now part of ClientProperties class. All new features are described in the user guide – in chapterTyrus proprietary configuration.

Update – Tyrus 1.8.1

There was another slightly late reported issue related to running in environments with SecurityManager enabled, so this version fixes that. Another noteworthy fixes are TYRUS-355 and TYRUS-361; the first one is about incorrect thread factory used for shared container timeout, which resulted in JVM waiting for that thread and not exiting as it should. The other issue enables relative URIs in Location header when using redirect feature.

Links

Complete list of changes:

Bug

  • [TYRUS-333] – Multiple endpoints on one client
  • [TYRUS-334] – When connection is closed by a peer, periodic heartbeat pong is not stopped
  • [TYRUS-336] – ReaderBuffer.getNextChars() keeps blocking a server thread after client has closed the session
  • [TYRUS-338] – JDK client SSL filter needs better synchronization during handshake phase
  • [TYRUS-339] – SSL hostname verification is missing
  • [TYRUS-340] – Test PathParamTest are not stable with JDK client
  • [TYRUS-341] – A control frame inside a stream of continuation frames is treated as the part of the stream
  • [TYRUS-343] – ControlFrameInDataStreamTest does not pass on GF
  • [TYRUS-345] – NPE is thrown, when shared container timeout property in JDK client is not set
  • [TYRUS-346] – IllegalStateException is thrown, when using proxy in JDK client
  • [TYRUS-347] – Introduce better synchronization in JDK client thread pool
  • [TYRUS-348] – When a client and server close connection simultaneously, JDK client throws NPE
  • [TYRUS-356] – Tyrus cannot determine the connection port for a wss URL
  • [TYRUS-357] – Exception thrown in MessageHandler#OnMessage is not caught in @OnError method
  • [TYRUS-359] – Client based on Java 7 Asynchronous IO makes application unexitable

Improvement

  • [TYRUS-328] – JDK 1.7 AIO Client container – threads – (setting threadpool, limits, …)
  • [TYRUS-332] – Consolidate shared client properties into one file.
  • [TYRUS-337] – Create an SSL version of Basic Servlet test

New Feature

  • [TYRUS-228] – Add client support for HTTP Basic/Digest

Task

  • [TYRUS-330] – create/run tests/servlet/basic via wss
  • [TYRUS-335] – [clustering] – introduce RemoteSession and expose them via separate method (not include remote sessions in the getOpenSessions())
  • [TYRUS-344] – Introduce Client support for HTTP Redirect

WebSocket API 1.1 released!

Its my please to announce that JSR 356 – Java API for WebSocket maintenance release ballot vote finished with majority of “yes” votes (actually, only one eligible voter did not vote, all other votes were “yeses”). New release is maintenance release and it addresses only one issue:  WEBSOCKET_SPEC-226.

What changed in the 1.1?

Version 1.1 is fully backwards compatible with version 1.0, there are only two methods added to javax.websocket.Session:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
/**
* Register to handle to incoming messages in this conversation. A maximum of one message handler per
* native websocket message type (text, binary, pong) may be added to each Session. I.e. a maximum
* of one message handler to handle incoming text messages a maximum of one message handler for
* handling incoming binary messages, and a maximum of one for handling incoming pong
* messages. For further details of which message handlers handle which of the native websocket
* message types please see {@link MessageHandler.Whole} and {@link MessageHandler.Partial}.
* Adding more than one of any one type will result in a runtime exception.
*
* @param clazz   type of the message processed by message handler to be registered.
* @param handler whole message handler to be added.
* @throws IllegalStateException if there is already a MessageHandler registered for the same native
*                               websocket message type as this handler.
*/
public void addMessageHandler(Class<T> clazz, MessageHandler.Whole<T> handler);
/**
* Register to handle to incoming messages in this conversation. A maximum of one message handler per
* native websocket message type (text, binary, pong) may be added to each Session. I.e. a maximum
* of one message handler to handle incoming text messages a maximum of one message handler for
* handling incoming binary messages, and a maximum of one for handling incoming pong
* messages. For further details of which message handlers handle which of the native websocket
* message types please see {@link MessageHandler.Whole} and {@link MessageHandler.Partial}.
* Adding more than one of any one type will result in a runtime exception.
*
*
* @param clazz   type of the message processed by message handler to be registered.
* @param handler partial message handler to be added.
* @throws IllegalStateException if there is already a MessageHandler registered for the same native
*                               websocket message type as this handler.
*/
public void addMessageHandler(Class<T> clazz, MessageHandler.Partial<T> handler);

Why do we need to add those methods? Short and not precise version: to support Lambda expressions as MessageHandlers.

Longer and slightly more precise explanation: old Session#addMessageHandler method (which is still there and works as it worked till now) does rely on getting the generic parameter during the runtime, which is not (always) possible. The unfortunate part is that it works for some common cases and the expert group did not catch this issue before 1.0 release because of that. The issue is really clearly visible when Lambdas are used as message handlers:

1
2
3
session.addMessageHandler(message -> {
System.out.println("### Received: " + message);
});

There is no way for the JSR 356 implementation to get the type of the used Lambda expression, thus this call will always result in an exception. Since all modern IDEs do recommend to use Lambda expressions when possible and MessageHandler interfaces are single method interfaces, it basically just scream “use Lambdas” all over the place but when you do that, the application will fail during runtime.

Only solution we currently have is to explicitly provide the type of registered MessageHandler. (There might be another sometime in the future when generic type reification is introduced, but that is not going to happen soon enough). So the example above will then be:

1
2
3
session.addMessageHandler(String.class, message -> {
System.out.println("### Received: " + message);
});

and voila, it works.

There are some limitations – you cannot do

1
List<String>.class

, so you will need to encapsulate these types when you want to use them in MessageHandler implementation (something like “class MyType extends ArrayList<String>”). There is no better way how to solve this issue, because Java currently does not provide good way how to describe generic types.

The api itself is available on maven central, look for javax.websocket:javax.websocket-api:1.1. The reference implementation is project Tyrus, which implements WebSocket API 1.1 from version 1.8.

Monday Jun 16, 2014

Tyrus 1.7

This release cycle was shorter comparing to previous ones and the main purpose was to integrate and align with upper stack projects. Despite that fact there are some new features and bugfixes:

What’s new?

Ondřej added nice feature which allows to limit opened WebSocket sessions. And it is configurable, currently supported scopes of limitations are “per endpoint”, “per client remote address” and “per application”. This should allow effective control of server-side resources. For now, the values are static and constant, but if there will be a demand for creating more advanced session limit support, like dynamic load-balancing among set of endpoints etc, we will provide that as well. Feel free to file an enhancement request if you are interested!

Petr improved monitoring support which was added in Tyrus 1.6 – now there is a possibility to get the information about errors reported to the endpoints. Errors are collected per Exception type thrown, so you might get number of decoding issues, IOExceptions and other custom Throwables used in the endpoint implementation.

Other than that, there were just bugfixes. Two of them are worth mentioning: Tyrus now supports close codes 1012 and 1013 (SERVICE RESTART and TRY AGAIN LATER). Both of these can be returned only from server endpoint. When client side will try to send those, they will be replaced with close code 1000.

And the other bugfix is about SSL/TLS support in Java 7 AIO based client container – the issue manifested when bigger messages were sent from the client to server. The handling was not correct in this case and it could happen that the message was not completely sent.

Complete list of changes

TYRUS-313 Limiting opened connections
TYRUS-319 Bug call EJB from WebSocket
TYRUS-320 When put under heavy load by multiple clients, server throws an exception.
TYRUS-321 tyrus-client incorrectly reports HTTP staus 500 for all non-101 responses
TYRUS-322 Some of session close-codes are not supported
TYRUS-324 Session.close() should close the connection without close reason.
TYRUS-325 Server do not close session properly if non-instantiable endpoint class is provided
TYRUS-326 Expose monitoring statistics about number of @OnError method invocations
TYRUS-327 JDK client container cannot handle “big” messages over SSL/TLS

Link

If you have any feedback, suggestions or improvement requests, please send us a note tousers@tyrus.java.net or create new issue at our JIRA instance.

Friday May 16, 2014

Tyrus 1.6

I’m pleased to announce that Tyrus 1.6 was released this week and there are some nice features and bugfixes in this release, so let me introduce most important ones.

What’s new?

First bigger addition is JDK Client Transport. We used exclusively Grizzly framework for handling transport layer on the client side and now we provide an option to switch to JDK 1.7 based transport. Main advantage of it is reduced size and number of jar files. The default transport is still based on Grizzly, so if you want to try JDK 1.7 AIO, you need to enable it explicitly. See JDK 7 client chapter in Tyrus documentation for more details. This feature was implemented by Petr Janouch, new Tyrus team member.

Another important feature is monitoring server side resource utilisation and exposing this info as statistics via JMX beans. Currently available statistics are mostly about number and type of received and send messages, also there is higher level part which provides list of deployed endpoints and number of concurrent connected clients. If you want to try this out, see JMX Monitoring chapter in Tyrus user guide for configuration details. This feature was also contributed by Petr.

Next in line is change related to an issue in JSR 356. Current version of Session interface does allow use of Lambda expressions when registering a message handler, but it does not work as expected. Problem is, that it just cannot work. The API forces implementation to get the generic type information during runtime, which is not always available (in Java). Most usecases do work fine, but one doesn’t – when the anonymous class implementation is replaced by Lambda expression, type information is lost. Tyrus now implements proposed solution, but to be able to access mentioned methods, you need to cast Session to TyrusSession. Then you need to provide type of the message handler (java.lang.Class) and the handler itself – since the type info is now separated, there is no need to get it from the message handler instance , thus it will work with any representation. We are working on porting this fix to the WebSocket API, stay tuned for more details.

Last and in this case maybe least feature is WSADL. Don’t worry if you don’t know what that is – nobody does :). WSADL is XML (for now) descriptor of deployed application, currently providing only set of deployed endpoints (mainly because other information, like registered message handlers, is not that easy to get before client connects to the endpoint and can vary per Session). WSADL stands for WebSocket Application Descriptor Language and it is supposed to be the same as WADL is for RESTful webservices and WSDL for SOAP webservices. In case you want to test this feature, enableWSADL_SUPPORT (or see the test) and GET /contextPath/application.wsadl on your deployed Tyrus application.

Complete list of changes

TYRUS-301 Custom String encoder is not used
TYRUS-305 Add support for multiple client container to TestContainer
TYRUS-311 Session timeout on client does not work when set in onOpen method
TYRUS-312 TyrusFuture.get(long,TimeUnit) does not honor Future.get(long,TimeUnit) contract
TYRUS-293 Automatic heartbeat PING
TYRUS-259 Should produce a warning during deployment when OnMessage#maxMessageSize is larger than the value of org.glassfish.tyrus.servlet.incoming-buffer-size
TYRUS-308 JDK Client transport – SSL support
TYRUS-309 JDK Client transport – Proxy support
TYRUS-318 Writer returned from BasicRemote.getSendWriter() throws NPE when flush is called more than once.
TYRUS-314 Create WADL-like descriptor per deployed app
TYRUS-317 Allow server configuration using WebSocketContainer or WebSocketAddOn
TYRUS-302 Java 8 Lambda
TYRUS-214 Expose monitoring API/statistics
TYRUS-299 Missing tyrus-container-grizzly-server in the release package WebSocket RI archive
TYRUS-233 Provide client transport based on plain JDK
TYRUS-310 When max idle timeout is reset to 0, every received message or sent ping or pong causes an empty task to be scheduled and executed

Links

If you have any questions or comments or if you want to discuss anything related to Tyrus, easiest way to contact us is via users@tyrus.java.net mailing list.

Friday Mar 21, 2014

Starting with Oracle Coherence and Incubator projects

I had a chance to play with Oracle Coherence and even when everything seems to be documented, it took me a while to find all necessary informations to really get started, so I decided to share my steps. The main goal of this HOWTO will be building all samples from Coherence Incubator project, branch develop-12 (requires latest Coherence – 12.1.2).

Getting Coherence

Coherence 12.1.2 can be downloaded from [here]. I’m using stand-alone install, but I’m pretty sure that everything should be mostly the same (from Coherence point of view). Downloaded zip archive contains coherence_121200.jar, so lets execute that. Oh, and I almost forgot – I’m using Java 8:

1
2
3
4
$ java -version
java version "1.8.0"
Java(TM) SE Runtime Environment (build 1.8.0-b132)
Java HotSpot(TM) 64-Bit Server VM (build 25.0-b70, mixed mode)

And I’m on Mac OS X 10.9.2 (latest Mavericks). Installation process is pretty straightforward, you just execute java -jar ./coherence_121200.jar and use the wizard to install what you need. I choose samples to be included, but I won’t be referring to them in this post. Remember where you set your Oracle Home directory, it contains all the installed artefacts and we need to use it in next step.

Let’s assume the installation is done without any issues (I did not experienced any) and we are in Oracle_Home directory somewhere on our file system. Another thing I forgot to mention – you will need Apache Maven to be able to compile/execute projects from Incubator. I have Maven 3.1.1:

1
2
3
4
5
6
7
$ mvn -v
Apache Maven 3.1.1 (0728685237757ffbf44136acec0402957f723d9a; 2013-09-17 17:22:22+0200)
Maven home: /Users/pavel/opt/apache-maven-3.1.1
Java version: 1.8.0, vendor: Oracle Corporation
Java home: /Library/Java/JavaVirtualMachines/jdk1.8.0.jdk/Contents/Home/jre
Default locale: en_US, platform encoding: UTF-8
OS name: "mac os x", version: "10.9.2", arch: "x86_64", family: "mac"

Following information is taken from document available here:http://docs.oracle.com/middleware/1212/core/MAVEN.pdf.

Firstly, you need to install oracle-maven-sync plugin. It will install Coherence into your local maven repository. It is capable to publishing these artefacts to your company maven repository or something like that, but that won’t be covered here. Please see linked document to see more details about it if required.

The plugin gets installed by executing following command:

1
mvn install:install-file -DpomFile=./oracle_common/plugins/maven/com/oracle/maven/oracle-maven-sync/12.1.2/oracle-maven-sync.12.1.2.pom -Dfile=./oracle_common/plugins/maven/com/oracle/maven/oracle-maven-sync/12.1.2/oracle-maven-sync.12.1.2.jar

I assume that this command is executed from Oracle_Home directory. You will need to adjust paths if that is not your case. Then you should see something similar to:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
[INFO] Scanning for projects...
[INFO]                                                                        
[INFO] ------------------------------------------------------------------------
[INFO] Building Maven Stub Project (No POM) 1
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-install-plugin:2.4:install-file (default-cli) @ standalone-pom ---
[INFO] Installing /Users/pavel/coherence/Oracle/Middleware/Oracle_Home/oracle_common/plugins/maven/com/oracle/maven/oracle-maven-sync/12.1.2/oracle-maven-sync.12.1.2.pom to /Users/pavel/.m2/repository/com/oracle/maven/oracle-maven-sync/12.1.2-0-0/oracle-maven-sync-12.1.2-0-0.jar
[INFO] Installing /Users/pavel/coherence/Oracle/Middleware/Oracle_Home/oracle_common/plugins/maven/com/oracle/maven/oracle-maven-sync/12.1.2/oracle-maven-sync.12.1.2.pom to /Users/pavel/.m2/repository/com/oracle/maven/oracle-maven-sync/12.1.2-0-0/oracle-maven-sync-12.1.2-0-0.pom
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 0.349s
[INFO] Finished at: Fri Mar 21 15:26:37 CET 2014
[INFO] Final Memory: 3M/81M
[INFO] ------------------------------------------------------------------------

Now we are ready to install Coherence binaries to our local maven repository:

1
Oracle_Home $ mvn -X com.oracle.maven:oracle-maven-sync:push -Doracle-maven-sync.oracleHome=.  -Doracle-maven-sync.testingOnly=false

Output is quite long, but if you see “BUILD SUCESS”, you can be sure that coherence jars are now installed in local maven repository. If there is some error, it is most likely related to incorrect directory, so try to play with “-Doracle-maven-sync.oracleHome” property and get it right.

Getting Coherence Incubator projects

This is an easy part. Coherence incubator is hosted on github – you obviously need git for that.

1
git clone git@github.com:coherence-community/coherence-incubator.git

thats it. Git created directory coherence-incubator where all the projects are. Little inconvenient is that the version which we want to get is in the branch, so we need to execute:

1
git checkout origin/develop-12 -b develop-12

Previous command checkouts remote branch “develop-12″ and saves it as local branch with the same name. It is not a must to create local branch, but it is easier to deal with that. Also, we will need to make some changes to get it running and you can store then in your local branch.

I’m not exactly sure why, but the project references Coherence version 12.1.2-0-1, but oracle-maven-sync plugin installs version 12.1.2-0-0, so the project can’t compile without modifications. Also, coherence-jvisualvm references binaries deployed on netbeans maven repository which is not mentioned anywhere, so we need to add it for maven to be able to download these dependencies. My complete patch is:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
diff --git a/pom.xml b/pom.xml
index 785741e..c4d951a 100644
--- a/pom.xml
+++ b/pom.xml
@@ -137,11 +137,18 @@
<junit.version>4.10</junit.version>
<miglayout.version>3.6</miglayout.version>
<mockito.version>1.9.0</mockito.version>
-        <oracle.coherence.version>12.1.2-0-1</oracle.coherence.version>
+        <oracle.coherence.version>12.1.2-0-0</oracle.coherence.version>
<oracle.tools.version>1.2.2</oracle.tools.version>
<powermock.version>1.4.12</powermock.version>
</properties>
+    <repositories>
+        <repository>
+            <id>netbeans</id>
+        </repository>
+    </repositories>
+
<dependencyManagement>
<dependencies>
<dependency>

Now, we are finally ready for

1
mvn -fae clean install

The “-fae” is here for trying to build everything possible, even when some project fails to build or test, which unfortunately happens. See my current results:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
[INFO] Coherence Incubator ............................... SUCCESS [1.284s]
[INFO] Coherence Incubator Common ........................ SUCCESS [30.717s]
[INFO] Coherence Incubator Command Pattern ............... SUCCESS [7.556s]
[INFO] Coherence Incubator Command Pattern (examples) .... SUCCESS [3.357s]
[INFO] Coherence Incubator Functor Pattern ............... SUCCESS [3.828s]
[INFO] Coherence Incubator Functor Pattern (examples) .... SUCCESS [2.577s]
[INFO] Coherence Incubator JVisualVM Plugin .............. SUCCESS [18.846s]
[INFO] Coherence Incubator Processing Pattern ............ SUCCESS [14.535s]
[INFO] Coherence Incubator Processing Pattern (examples) . SUCCESS [2.018s]
[INFO] Coherence Incubator Messaging Pattern ............. SUCCESS [8.903s]
[INFO] Coherence Incubator Messaging Pattern (functional tests)  SUCCESS [1:52.155s]
[INFO] Coherence Incubator Event Distribution Pattern .... SUCCESS [44.490s]
[INFO] Coherence Incubator Push Replication Pattern ...... SUCCESS [2.198s]
[INFO] Coherence Incubator Push Replication Pattern (functional tests)  FAILURE [6:49.643s]
[INFO] Coherence Incubator Push Replication Pattern (examples)  SUCCESS [1.645s]
[INFO] Coherence Incubator Push Replication Pattern *Web (examples)  SUCCESS [0.013s]
[INFO] Coherence Incubator Web Server .................... SUCCESS [2.058s]
[INFO] Coherence Incubator Web Application ............... SUCCESS [6.210s]
[INFO] Coherence Incubator Web Application Test .......... FAILURE [1:27.214s]
[INFO] Coherence Incubator Distribution .................. SUCCESS [13.540s]
[INFO] Coherence Incubator Site .......................... SUCCESS [0.019s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 12:53.648s
[INFO] Finished at: Fri Mar 21 16:30:09 CET 2014
[INFO] Final Memory: 58M/210M
[INFO] ------------------------------------------------------------------------

Anyway, as you can see, most of the projects can compile and are working properly. I was interested mainly in Messaging pattern, which works as expected, so I did not do any further inquiries about in Replication pattern and Web app tests.

And that’s it. If you are interested in any of these patterns or project, see sources and tests, there are usually “hello world” like samples which are perfect for learning. Also, if you happen to discover the reason for test failures I’m getting, feel free to add comment or send me a note to pavel.bucek [at] oracle.com and I’ll update the article.

Tuesday Mar 18, 2014

WebSocket API in JDK 8 - MessageHandler issue

JDK 8 is not yet released and it is already causing some headaches for JSR 356 implementors. The issue described in this article will be about the most famous and anticipated JDK 8 feature – lambda expressions and its impact to one part of WebSocket API.

Lambda expressions usable in WebSocket API can be diminished to just a replacement for standard anonymous classes; there might be some places where you can use method references, but not in the case of message handler. Following text is mainly about addMessageHandler method:

1
2
3
public interface Session extends Closeable {
void addMessageHandler(MessageHandler handler) throws IllegalStateException;
}

There are multiple use cases when addMessageHandler method is used – for example, in any endpoint based on programmatic API, which is typically true on client side. Then you have code like:

1
2
3
4
5
6
7
8
9
10
11
public class MyEndpoint extends Endpoint {
@Override
public void onOpen(Session session, EndpointConfig config) {
session.addMessageHandler(new MessageHandler.Whole<String>() {
@Override
public void onMessage(String message) {
// handle message
}
});
}
}

which always works as expected. JDK 8 introduces lambda expressions which can (not only!) replace too verbose anonymous classes, when they have only single method – which is true for MessageHandler.Whole interface. So when that code is opened in IDE supporting Java 8, it will usually say something like “anonymous class can be replaced with lambda expression” and recommends that action to be done. The resulting code will be:

1
2
3
4
5
6
7
8
public class MyEndpoint extends Endpoint {
@Override
public void onOpen(javax.websocket.Session session, EndpointConfig config) {
session.addMessageHandler((MessageHandler.Whole<String>) message -> {
// handle message
});
}
}

It is nicer, part of the information is hidden, but my guess is it might not be seen as a drawback for most cases. And 3 lines of code were saved, so thumbs up! Well.. there is the last thing.. it doesn’t work.

Lambda expressions are not anonymous classes. Anonymous class can be replaced by using lambda expression and it still compiles, but the generic parameter is lost, or at least there is no way how to obtain it via standard reflection API. And that is the problem for any JSR 356 implementation – type parameter here is used for selecting appropriate Decoder (another part of the API). Any lambda expression used instead regular generic anonymous class will be treaded as MessageHandler.Whole<Object>.

What can we do with this? Surprisingly, not much. I will try to file a backwards compatibility issue/challenge agains JDK 8, but from the initial reactions on jdk8-dev mailing list, it does not seem like something anybody is willing to fix. There is one nice constructive proposal from Simone Bordet, which would at least make the variant with lambda not compilable. That would be great workaround (which will require WebSocket API 1.0.1 release and potentially changes in other APIs), but I still consider this being an issue which should be fixed in more generic way. And last but not least – I don’t think that new version of JDK should break any existing working API.

I’ll keep this post updated whenever I got additional information. Feel free to comment or suggest a solution!

EDIT:
Credit for discovering this issue should go to user ayyrk, reporter of TYRUS-302

Thursday Mar 06, 2014

Tyrus 1.5

New version of Tyrus was released today, so let’s do a quick summary of what is new compared to previous version and so on.

What’s new?

Maven archetype was (finally) added, so if you want to generate simple application, test it and maybe start playing with the code and modifying it to something more complex, you can. All you need is maven and little space on your hard drive. Then you can execute following command:

1
2
3
4
mvn archetype:generate -DarchetypeArtifactId=tyrus-archetype-echo \
-DarchetypeGroupId=org.glassfish.tyrus.archetypes -DinteractiveMode=false \
-DgroupId=com.example -DartifactId=echo -Dpackage=com.example \
-DarchetypeVersion=1.5

and the project should be created.

What’s fixed?

  • Close reason for dropped connection is now 1006, as it should be.
  • Standalone Server is not leaking daemon threads any more.
  • Java SE 8 runtime issues.
  • “Host” header parsing issue which caused failed handshakes when using IPv6 localhost address ([::1]).
  •  Internal InputStream implementation now correctly returns ‘-1′ to InputStream#read() only when end of input is reached. Thanks Raghu for contributing the fix!

Java SE 8 related issue is worth a short explanation, since the change introduced by this fix goes beyond SE 8. Original problem was about some bug in Java SE 7 which somehow corrected Tyrus behaviour, so it was not noticed by our tests – bridge methods are not returned from Class.getMethods() call there. This was fixed in Java SE 8 and it caused some troubles in Tyrus implementation, because we just did not expect them to be returned.

The other part of this issue was correcting Tyrus in terms of handling inherited methods. JSR 356 describes how these should be handled in little bit cryptic way, but it basically states that annotations are not inherited, which is already defined in Java language specification. Unfortunately, Tyrus prior this version was considering annotated methods (@OnOpen, @OnMessage, …) only from the very same class as the registered one, so inherited methods were always ignored. This is now changed, so you can have something like:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
public class BaseEndpoint {
@OnOpen