Oracle’s Java Technology Evangelist Simon Ritter, one of the most fun-loving Java developers I know, with a long history of JavaOne gadgetry, gave a session (25011) at JavaOne 2011 on Wednesday afternoon showing how “open source APIs for the Kinect, the Wiimote combined with a tilt-compensated compass, a head-mounted stereoscopic display, and some old Sun SPOTs can build a truly immersive application.” The large audience appeared immersed throughout the session in Ritter's colorful and clearly delineated demos.
He explained that the way we interact with computers is rapidly changing and that the days of the keyboard and mouse are gone. (Maybe so, but I'm sitting here using a keyboard and mouse.) And with his usual dramatic flair, Ritter invited attendees to behold the rise of something he calls the “gestural interface”.
The presentation used the latest JavaFX 2.0 "pure Java" implementation and began with an overview of the different components being used and explained how they are all brought together to enable the user to interact with interfaces in ways never before possible. Building an interface with the new JavaFX 2.0, Simon pointed out, is a continuation of the JavaFX product line, which is now a Java API with no scripting language and most APIs ported across while features such as binding and animation have required more thought. JavaFX now embraces more web technologies and enables the use of CSS for all JavaFX controls and a web specification for Drag-and-Drop. Also, developers use Scenegraph instead of DOM. He pointed to both pro’s and con’s of using JavaFX with gestural interfaces. On the plus side, it has built-in features such as data binding and animations, is a relatively simple API, and is able to build rich, visually appealing interfaces. On the negative side, JavaFX is currently limited to a 2D environment. The engineering team is currently working on 3D support.
He contrasted this with jMonkey Engine (jME), a game engine made especially for modern 3D development, written purely in Java and consisting of a collection of libraries that has game engine facilities and a full physics engine, but is hard to program and focuses on games and not generic interfaces. Ritter proceeded to demonstrate how to use the Nintendo Wiimote with a Java interface. The Wiimote communicates using a Bluetooth stack that needs to support L2CAP, has JSR-82 Java Bluetooth API implementation plus Wiimote-specific Java APIs (IR sensors, accelerometer, etc), most of which is free and open source.
He then presented a demo making use of the Sun Spot controller, a gyro sensor for precise rotation data, three bend sensors for finger movement for head tracking and data gloves, hand and head tracking sensors and hardware and more.
This followed with a demo using the Kinect Sensor with Java for 3D sensing. Not to be lost are his larger points: Java is still a really cool and powerful language. It is easy to interface with exotic hardware using free and open source libraries to build interesting applications using modern hardware.
After a brief Q&A, Simon -- as he always does -- implored attendees to be inspired and go build their own FUN stuff.