Tuesday Sep 22, 2009

TOTD #105: GlassFish v3 Monitoring – How to monitor a Rails app using asadmin, JavaScript, jConsole, REST ?

Monitoring infrastructure in GlassFish v3 is getting an extreme makeover. Prashanth defines it very nicely in his detailed introduction:

We provide a way to dynamically and non-intrusively generate monitoring events from any of the GlassFish runtime classes, the ability to listen to these events, collect the statistics and expose these statistics through various standard clients.

The "dynamic" nature indicates that monitoring can be turned ON or OFF in a GlassFish instance running in production environment. It also means the granularity to which the monitoring information can be generated. This information can be generated for not only a traditional Java EE applications, but Rails, Django and other type of applications that can be easily deployed on GlassFish v3. And that is extensible for other pluggable containers too. The data can be accessed using multiple mechanisms providing the administrator a wide variety of choice for tools, e.g. DTrace, JMX, REST, asadmin CLI, and Admin Console and thus another point in favor of "dynamic".

The "non-intrusive" behavior means that monitoring can be used in production environment with minimal overhead. It also means that there is no overhead when monitoring is not enabled. Anyway, monitoring needs to be explicitly enabled as explained below.

As mentioned above, there are multiple ways to access the monitoring data. The different ways to access the monitoring data are:

  1. DTrace scripts (only on Solaris)
  2. asadmin CLI
  3. Admin Console
  4. JMX/jConsole
  5. REST
  6. Custom client using a 3rd party scripting language (available as value-add feature to paid customers only)

This Tip Of The Day will describe how to access the monitoring data using asadmin CLI and a JavaScript client. Note that the JavaScript client feature is available as a value-add to the community version of GlassFish and available only to the paid customers.

This tip will use a pre-deployed Rails application as (Redmine as described in TOTD #104) on a nightly GlassFish v3 build (CI, nightly, or promoted) and explains how to monitor this application.

Using the terminology defined in Monitoring in GlassFish v3 blog, the JRuby subsystem in GlassFish has multiple probe providers and each provider has multiple probes. The JRuby subsystem also provides multiple stats providers with probe listeners. All of these elements are linked using the monitoring infrastructure in GlassFish.

Lets monitor our Rails application

Monitoring using asadmin CLI

By default the monitoring is turned OFF for all the components. Lets enable monitoring for the JRuby container as:

asadmin enable-monitoring --level jruby-container=HIGH

The other possible values are "LOW" and "OFF". All the probes publishing monitoring data can be listed as:

~/tools/glassfish/v3/9-18/glassfishv3 >./bin/asadmin list --monitor=true "\*"

Command list executed successfully.

A snapshot of the monitoring data can be obtained as:

~/tools/glassfish/v3/9-18/glassfishv3 >./bin/asadmin get --monitor=true "\*jruby\*"
server.containers.jruby.applications.activerubyapplications.redmine-0.8.5-appName = redmine-0.8.5
server.containers.jruby.applications.activerubyapplications.redmine-0.8.5-description =
 server.containers.jruby.applications.activerubyapplications.redmine-0.8.5-environment = development
server.containers.jruby.applications.activerubyapplications.redmine-0.8.5-jrubyVersion = 1.3.0
server.containers.jruby.applications.activerubyapplications.redmine-0.8.5-lastsampletime = 1253322280437
server.containers.jruby.applications.activerubyapplications.redmine-0.8.5-name = redmine-0.8.5
server.containers.jruby.applications.activerubyapplications.redmine-0.8.5-rubyFramework = rails
server.containers.jruby.applications.activerubyapplications.redmine-0.8.5-starttime = 1253322280437
server.containers.jruby.applications.activerubyapplications.redmine-0.8.5-unit =
server.containers.jruby.applications.activerubyappscount-count = 1
server.containers.jruby.applications.activerubyappscount-description = Number of currently loaded Ruby applications
server.containers.jruby.applications.activerubyappscount-lastsampletime = 1253322279921
server.containers.jruby.applications.activerubyappscount-name = ActiveRubyApplications
server.containers.jruby.applications.activerubyappscount-starttime = 1253322279922
server.containers.jruby.applications.activerubyappscount-unit = count
server.containers.jruby.http.redmine-0.8.5.address = null
server.containers.jruby.http.redmine-0.8.5.averageprocessingtime = 0
server.containers.jruby.http.redmine-0.8.5.count200-count = 0
server.containers.jruby.http.redmine-0.8.5.count200-description = Number of responses with a status code equal to 200
server.containers.jruby.http.redmine-0.8.5.count200-lastsampletime = 1253322280245
server.containers.jruby.http.redmine-0.8.5.count200-name = Count200
server.containers.jruby.http.redmine-0.8.5.count200-starttime = 1253322280245
server.containers.jruby.http.redmine-0.8.5.count200-unit = count
server.containers.jruby.http.redmine-0.8.5.count2xx-count = 0
server.containers.jruby.http.redmine-0.8.5.count2xx-description = Number of responses with a status code in the 2xx range
server.containers.jruby.http.redmine-0.8.5.count2xx-lastsampletime = 1253322280245
server.containers.jruby.http.redmine-0.8.5.count2xx-name = Count2xx
server.containers.jruby.http.redmine-0.8.5.count2xx-starttime = 1253322280245
server.containers.jruby.http.redmine-0.8.5.count2xx-unit = count
server.containers.jruby.http.redmine-0.8.5.count302-count = 0
server.containers.jruby.http.redmine-0.8.5.count302-description = Number of responses with a status code equal to 302
server.containers.jruby.http.redmine-0.8.5.count302-lastsampletime = 1253322280245
server.containers.jruby.http.redmine-0.8.5.count302-name = Count302
server.containers.jruby.http.redmine-0.8.5.count302-starttime = 1253322280245
server.containers.jruby.http.redmine-0.8.5.count302-unit = count
server.containers.jruby.http.redmine-0.8.5.count304-count = 0
server.containers.jruby.http.redmine-0.8.5.count304-description = Number of responses with a status code equal to 304
server.containers.jruby.http.redmine-0.8.5.count304-lastsampletime = 1253322280245
server.containers.jruby.http.redmine-0.8.5.count304-name = Count304
server.containers.jruby.http.redmine-0.8.5.count304-starttime = 1253322280245
server.containers.jruby.http.redmine-0.8.5.count304-unit = count
server.containers.jruby.http.redmine-0.8.5.count3xx-count = 0
server.containers.jruby.http.redmine-0.8.5.count3xx-description = Number of responses with a status code in the 3xx range
server.containers.jruby.http.redmine-0.8.5.count3xx-lastsampletime = 1253322280245
server.containers.jruby.http.redmine-0.8.5.count3xx-name = Count3xx
server.containers.jruby.http.redmine-0.8.5.count3xx-starttime = 1253322280245
server.containers.jruby.http.redmine-0.8.5.count3xx-unit = count
server.containers.jruby.http.redmine-0.8.5.count400-count = 0
server.containers.jruby.http.redmine-0.8.5.count400-description = Number of responses with a status code equal to 400
server.containers.jruby.http.redmine-0.8.5.count400-lastsampletime = 1253322280245
server.containers.jruby.http.redmine-0.8.5.count400-name = Count400
server.containers.jruby.http.redmine-0.8.5.count400-starttime = 1253322280245
server.containers.jruby.http.redmine-0.8.5.count400-unit = count
server.containers.jruby.http.redmine-0.8.5.count401-count = 0
server.containers.jruby.http.redmine-0.8.5.count401-description = Number of responses with a status code equal to 401
server.containers.jruby.http.redmine-0.8.5.count401-lastsampletime = 1253322280245
server.containers.jruby.http.redmine-0.8.5.count401-name = Count401
server.containers.jruby.http.redmine-0.8.5.count401-starttime = 1253322280245
server.containers.jruby.http.redmine-0.8.5.count401-unit = count
server.containers.jruby.http.redmine-0.8.5.count403-count = 0
server.containers.jruby.http.redmine-0.8.5.count403-description = Number of responses with a status code equal to 403
server.containers.jruby.http.redmine-0.8.5.count403-lastsampletime = 1253322280245
server.containers.jruby.http.redmine-0.8.5.count403-name = Count403
server.containers.jruby.http.redmine-0.8.5.count403-starttime = 1253322280245
server.containers.jruby.http.redmine-0.8.5.count403-unit = count
server.containers.jruby.http.redmine-0.8.5.count404-count = 0
server.containers.jruby.http.redmine-0.8.5.count404-description = Number of responses with a status code equal to 404
server.containers.jruby.http.redmine-0.8.5.count404-lastsampletime = 1253322280245
server.containers.jruby.http.redmine-0.8.5.count404-name = Count404
server.containers.jruby.http.redmine-0.8.5.count404-starttime = 1253322280245
server.containers.jruby.http.redmine-0.8.5.count404-unit = count
server.containers.jruby.http.redmine-0.8.5.count4xx-count = 0
server.containers.jruby.http.redmine-0.8.5.count4xx-description = Number of responses with a status code in the 4xx range
server.containers.jruby.http.redmine-0.8.5.count4xx-lastsampletime = 1253322280245
server.containers.jruby.http.redmine-0.8.5.count4xx-name = Count4xx
server.containers.jruby.http.redmine-0.8.5.count4xx-starttime = 1253322280245
server.containers.jruby.http.redmine-0.8.5.count4xx-unit = count
server.containers.jruby.http.redmine-0.8.5.count503-count = 0
server.containers.jruby.http.redmine-0.8.5.count503-description = Number of responses with a status code equal to 503
server.containers.jruby.http.redmine-0.8.5.count503-lastsampletime = 1253322280245
server.containers.jruby.http.redmine-0.8.5.count503-name = Count503
server.containers.jruby.http.redmine-0.8.5.count503-starttime = 1253322280245
server.containers.jruby.http.redmine-0.8.5.count503-unit = count
server.containers.jruby.http.redmine-0.8.5.count5xx-count = 0
server.containers.jruby.http.redmine-0.8.5.count5xx-description = Number of responses with a status code in the 5xx range
server.containers.jruby.http.redmine-0.8.5.count5xx-lastsampletime = 1253322280245
server.containers.jruby.http.redmine-0.8.5.count5xx-name = Count5xx
server.containers.jruby.http.redmine-0.8.5.count5xx-starttime = 1253322280245
server.containers.jruby.http.redmine-0.8.5.count5xx-unit = count
server.containers.jruby.http.redmine-0.8.5.countother-count = 0
server.containers.jruby.http.redmine-0.8.5.countother-description = Number of responses with other status codes
server.containers.jruby.http.redmine-0.8.5.countother-lastsampletime = 1253322280245
server.containers.jruby.http.redmine-0.8.5.countother-name = CountOther
server.containers.jruby.http.redmine-0.8.5.countother-starttime = 1253322280245
server.containers.jruby.http.redmine-0.8.5.countother-unit = count
server.containers.jruby.http.redmine-0.8.5.errorcount-count = 0
server.containers.jruby.http.redmine-0.8.5.errorcount-description = Number of responses with a status code greater than 400
server.containers.jruby.http.redmine-0.8.5.errorcount-lastsampletime = 1253322280245
server.containers.jruby.http.redmine-0.8.5.errorcount-name = ErrorCount
server.containers.jruby.http.redmine-0.8.5.errorcount-starttime = 1253322280245
server.containers.jruby.http.redmine-0.8.5.errorcount-unit = count
server.containers.jruby.http.redmine-0.8.5.requestcount-count = 0
server.containers.jruby.http.redmine-0.8.5.requestcount-description = Number of HTTP requests received
server.containers.jruby.http.redmine-0.8.5.requestcount-lastsampletime = 1253322280243
server.containers.jruby.http.redmine-0.8.5.requestcount-name = RequestCounter
server.containers.jruby.http.redmine-0.8.5.requestcount-starttime = 1253322280243
server.containers.jruby.http.redmine-0.8.5.requestcount-unit = count
server.containers.jruby.http.redmine-0.8.5.requestpersecond = null
server.containers.jruby.runtime-pool.redmine-0.8.5.activeruntimes-count = 1
server.containers.jruby.runtime-pool.redmine-0.8.5.activeruntimes-description = Currently active runtimes
server.containers.jruby.runtime-pool.redmine-0.8.5.activeruntimes-lastsampletime = 1253322280241
server.containers.jruby.runtime-pool.redmine-0.8.5.activeruntimes-name = activeRuntimes
server.containers.jruby.runtime-pool.redmine-0.8.5.activeruntimes-starttime = 1253322280241
server.containers.jruby.runtime-pool.redmine-0.8.5.activeruntimes-unit = count
server.containers.jruby.runtime-pool.redmine-0.8.5.applicationname = redmine-0.8.5
server.containers.jruby.runtime-pool.redmine-0.8.5.contextpath = /redmine-0.8.5
server.containers.jruby.runtime-pool.redmine-0.8.5.hardmaximum-count = 1
server.containers.jruby.runtime-pool.redmine-0.8.5.hardmaximum-description = Maximum active runtimes
server.containers.jruby.runtime-pool.redmine-0.8.5.hardmaximum-lastsampletime = 1253322280241
server.containers.jruby.runtime-pool.redmine-0.8.5.hardmaximum-name = hardMaximum
server.containers.jruby.runtime-pool.redmine-0.8.5.hardmaximum-starttime = 1253322280241
server.containers.jruby.runtime-pool.redmine-0.8.5.hardmaximum-unit = count
server.containers.jruby.runtime-pool.redmine-0.8.5.hardmminimum-count = 1
server.containers.jruby.runtime-pool.redmine-0.8.5.hardmminimum-description = Minimum active runtimes
server.containers.jruby.runtime-pool.redmine-0.8.5.hardmminimum-lastsampletime = 1253322280241
server.containers.jruby.runtime-pool.redmine-0.8.5.hardmminimum-name = hardMinimum
server.containers.jruby.runtime-pool.redmine-0.8.5.hardmminimum-starttime = 1253322280241
server.containers.jruby.runtime-pool.redmine-0.8.5.hardmminimum-unit = count

Command get executed successfully.

The command asadmin get --monitor=true "\*jruby\*" dumps a snapshot of the monitoring data such as the number of Ruby currently applications loaded, application name, JRuby version, environment (development / test / production), currently active runtimes, min/max runtimes, number of HTTP requests / responses with a certain HTTP code and much more information. It basically dumps all the JRuby monitoring information captured so far.

An alternate regular expression may be specified such as:

asadmin get --monitor=true "\*jruby\*runtime-pool\*"

to gather only the runtime pool specific values.

Monitoring using JavaScript client

Third-party scripting client are a value-add to the community versions. Just like Enterprise Manager, the value-add will be available as a patch to the users who purchase commercial support. Lets see what can be done with it though.

First of all you can type the command "list-probes" to see a list of all the probes that are available. A typical output will look like:

~/tools/glassfish/v3/9-18/glassfishv3 >./bin/asadmin list-probes
glassfish:kernel:connections-keep-alive:decrementCountConnectionsEvent decrementCountConnectionsEvent(java.lang.String)
glassfish:web:web-module:webModuleStoppedEvent webModuleStoppedEvent(java.lang.String, java.lang.String)
glassfish:jca:connection-pool:connectionValidationFailedEvent connectionValidationFailedEvent(java.lang.String, int)
glassfish:jca:work-management:workWaitedFor workWaitedFor(java.lang.String, long)
glassfish:jdbc:connection-pool:connectionTimedOutEvent connectionTimedOutEvent(java.lang.String)
glassfish:security:ejbpolicy:ejbPCCreationEvent ejbPCCreationEvent(java.lang.String)
glassfish:kernel:thread-pool:threadReturnedToPoolEvent threadReturnedToPoolEvent(java.lang.String, java.lang.String)
glassfish:web:session:sessionPersistedEndEvent sessionPersistedEndEvent(java.lang.String, java.lang.String, java.lang.String)
glassfish:web:http-service:requestStartEvent requestStartEvent(java.lang.String, java.lang.String, java.lang.String, int, java.lang.String, java.lang.String)
glassfish:jdbc:connection-pool:connectionsFreedEvent connectionsFreedEvent(java.lang.String, int)
glassfish:jdbc:connection-pool:toString toString(java.lang.String, java.lang.StringBuffer)
glassfish:jca:connection-pool:decrementFreeConnectionsSizeEvent decrementFreeConnectionsSizeEvent(java.lang.String, int)
glassfish:kernel:connections-keep-alive:incrementCountFlushesEvent incrementCountFlushesEvent(java.lang.String)
glassfish:webservices:109:deploy deploy(com.sun.enterprise.deployment.Application, com.sun.enterprise.deployment.WebServiceEndpoint)
glassfish:jca:connection-pool:connectionAcquiredEvent connectionAcquiredEvent(java.lang.String)
glassfish:web:http-service:requestEndEvent requestEndEvent(java.lang.String, java.lang.String, java.lang.String, int, java.lang.String, java.lang.String, int)
glassfish:kernel:connections-keep-alive:incrementCountRefusalsEvent incrementCountRefusalsEvent(java.lang.String)
glassfish:kernel:connections:connectionConnectedEvent connectionConnectedEvent(java.lang.String, int)
glassfish:kernel:file-cache:addHeapSizeEvent addHeapSizeEvent(java.lang.String, long)
glassfish:jruby:runtime-pool:runtimePoolUpdateEvent runtimePoolUpdateEvent(java.lang.String, int, int, int, int, int)
glassfish:security:web:securityManagerCreationEvent securityManagerCreationEvent(java.lang.String)
glassfish:security:web:securityManagerDestructionEvent securityManagerDestructionEvent(java.lang.String)
glassfish:webservices:ri:undeploy undeploy(com.sun.xml.ws.transport.http.servlet.ServletAdapter)
glassfish:web:session:sessionPersistedStartEvent sessionPersistedStartEvent(java.lang.String, java.lang.String, java.lang.String)
glassfish:ejb:pool:objectAddedEvent ejbObjectAddedEvent()
glassfish:ejb:bean:methodStartEvent ejbMethodStartEvent(java.lang.String, java.lang.String, java.lang.String, java.lang.reflect.Method)
glassfish:jdbc:connection-pool:connectionRequestServedEvent connectionRequestServedEvent(java.lang.String, long)

All the JRuby probes can be easily listed as:

~/tools/glassfish/v3/9-18/glassfishv3 >./bin/asadmin list-probes | grep jruby
glassfish:jruby:runtime-pool:runtimePoolUpdateEvent runtimePoolUpdateEvent(java.lang.String, int, int, int, int, int)
glassfish:jruby:http:requestStartEvent requestStartEvent(java.lang.String, java.lang.String, int)
glassfish:jruby:jruby-container:jrubyModuleStartedEvent jrubyModuleStartedEvent(java.lang.String, java.lang.String, java.lang.String, java.lang.String)
glassfish:jruby:jruby-container:jrubyModuleStoppedEvent jrubyModuleStoppedEvent(java.lang.String, java.lang.String, java.lang.String)
glassfish:jruby:http:requestEndEvent requestEndEvent(java.lang.String, int)
glassfish:jruby:runtime-pool:runtimePoolStartEvent runtimePoolStartEvent(java.lang.String, int, int, int, int, int, int, int)

More detailed information about each probe can be easily found by giving the command:

~/tools/glassfish/v3/9-18/glassfishv3 >./bin/asadmin list-probes --manpage=true glassfish:jruby:runtime-pool:runtimePoolUpdateEvent

This will show the detailed information ("man" pages) about the specific probe. A JavaScript client to monitor the deployed Rails application and gather the relevant statistics can be written using these pages as shown below:

moduleStarted = function(appName, appType, env, jruby) {
    client.print( '\\n log> Deployed a \\'' + appType + '\\' app named \\'' + appName + '\\' in \\'' + env + '\\' environment with JRuby \\'' + jruby + '\\'.');

moduleStopped = function(appName, appType, env) {
    client.print( '\\n log> Undeployed a \\'' + appType + '\\' app named \\'' + appName + '\\' in \\'' + env + '\\' environment.');

requestStarted = function(contextPath, serverName, port) {
    client.print( '\\n log> Request started at \\'' + contextPath + '\\' hosted on \\'' + serverName + ':' + port + '\\'');

requestEnded = function(contextPath, statusCode) {
    client.print( '\\n log> Request stopped at \\'' + contextPath + '\\' with status code \\'' + statusCode + '\\'');

runtimePoolStart = function(appName, activeRuntimes, queueSize) {
   client.print( '\\n log> JRuby runtime pool started for the app \\'' + appName + '\\': active runtime - ' + activeRuntimes + ', queue size - ' + queueSize);

runtimePoolUpdate = function(appName, activeRuntimes, queueSize) {
   client.print( '\\n log> JRuby runtime pool updated for the app \\'' + appName + '\\': active runtime - ' + activeRuntimes + ', queue size - ' + queueSize);

moduleStartedParams = java.lang.reflect.Array.newInstance(java.lang.String, 4);
scriptContainer.registerListener('glassfish:jruby:jruby-container:jrubyModuleStartedEvent', moduleStartedParams, 'moduleStarted');

moduleStoppedParams = java.lang.reflect.Array.newInstance(java.lang.String, 3);
scriptContainer.registerListener('glassfish:jruby:jruby-container:jrubyModuleStoppedEvent', moduleStoppedParams, 'moduleStopped');

requestStartParams = java.lang.reflect.Array.newInstance(java.lang.String, 3);
scriptContainer.registerListener('glassfish:jruby:http:requestStartEvent', requestStartParams, 'requestStarted');

requestEndParams = java.lang.reflect.Array.newInstance(java.lang.String, 2);
scriptContainer.registerListener('glassfish:jruby:http:requestEndEvent', requestEndParams, 'requestEnded');

runtimePoolParams = java.lang.reflect.Array.newInstance(java.lang.String, 3);
scriptContainer.registerListener('glassfish:jruby:runtime-pool:runtimePoolStartEvent', runtimePoolParams, 'runtimePoolStart');
scriptContainer.registerListener('glassfish:jruby:runtime-pool:runtimePoolUpdateEvent', runtimePoolParams, 'runtimePoolUpdate');

This script register listeners for different probe events, pass a set of parameters that need to be captured, and print the information in a callback method specified during registration. The number of listeners and parameters / listener may be altered to meet your data capturing needs.

This script is stored in a file "monitor-rails.js" and used as described below. As a Rails application is deployed, requests invoked, and undeployed, the following messages are printed on the console:

~/tools/glassfish/v3/9-18/glassfishv3 >./bin/asadmin run-script monitor-rails.js

 log> Deployed a 'rails' app named 'redmine-0.8.5' in 'development' environment with JRuby '1.3.0'.
 log> JRuby runtime pool started for the app 'redmine-0.8.5': active runtime - 1, queue size - 0
 log> Request started at '/redmine-0.8.5' hosted on 'localhost:8080'
 log> Request stopped at '/redmine-0.8.5' with status code '200'
 log> Request started at '/redmine-0.8.5' hosted on 'localhost:8080'
 log> Request stopped at '/redmine-0.8.5' with status code '200'
 log> Undeployed a 'rails' app named 'redmine-0.8.5' in 'development' environment.

Note "run-script" is the command that comes as part of the value-add. As described earlier, a snapshot of the monitoring information can be easily captured using asadmin as:

~/tools/glassfish/v3/9-18/glassfishv3 >./bin/asadmin get --monitor=true "\*jruby\*request\*"
server.containers.jruby.http.redmine-0.8.5.requestcount-count = 2
server.containers.jruby.http.redmine-0.8.5.requestcount-description = Number of HTTP requests received
server.containers.jruby.http.redmine-0.8.5.requestcount-lastsampletime = 1253638362651
server.containers.jruby.http.redmine-0.8.5.requestcount-name = RequestCounter
server.containers.jruby.http.redmine-0.8.5.requestcount-starttime = 1253638362651
server.containers.jruby.http.redmine-0.8.5.requestcount-unit = count
server.containers.jruby.http.redmine-0.8.5.requestpersecond = 1

Command get executed successfully.

As you can see, the total number of requests is shown as "2".

Just for fun, I ran Apache ab with 20 requests and 4 concurrency ("-n 20 -c 4") on "http://localhost:8080/redmine-0.8.5" and saw the following results:

Monitoring OFF
(no data printed)
(data printed)
Time taken for requests (secs) 26.231 26.997 26.665
Requests per second 0.76 0.74 0.75
Time per request (ms) 1311.574 1349.859 1333.239

The server was restarted between each run. These numbers were generated on my development machine so they are most likely skewed. But as a development benchmark the numbers do indicate that GlassFish v3 monitoring is truly non-intrusive. The total time taken for requests, requests/second, and time/request is pretty much identical.

Future blogs will cover how to access this data using web-based DTrace scripts, web-based Admin Console, jConsole, and REST interface.

A complete archive of all the tips is available here.

Technorati: totd glassfish v3 monitoring asadmin javascript jruby rails redmine

Thursday Aug 14, 2008

LOTD #2: Phobos - MVC framework based on JavaScript

Phobos is a lightweight, scripting-friendly, web application environment running on the Java platform. It provides a complete MVC framework where Controller is a JavaScript class, View is an Embedded JavaScript (EJS) file and Model is typically a mix of Java and JavaScript.

The tech tip explains how Phobos and jMaki can be used to create a simple Ajax-enabled application using NetBeans tooling and GlassFish for deployment.
Another application built using Phobos and jMaki was shown in JavaOne 2007 technical keynote.

All previous entries in this series are archived at LOTD.

Technorati: lotd phobos jmaki javascript netbeans glassfish

Monday Jul 28, 2008

TOTD #39: Prototype/Script.aculo.us Autcomplete widget with MySQL, GlassFish, NetBeans

There are several JavaScript libraries that can be embedded in your webapplication to create a visually appealing interface. Script.aculo.us is one of the popular ones and is built on the Prototype JavaScript Framework. The library provides an easy-to-use, cross-browser user interface JavaScripts that allows you to create fancy effects commonly visible on web pages these days.

This blog entry gets you started by using Ajax.Autocompleter that allows for server-powered autocompleting of text fields. Basically, you type a character in a text field and suggestions for possible correct values starting with that character are shown . This is achieved by by sending an Ajax request to the data source on server, passing the typed character in the request and processing the response to display on the web page. This effect was first popularized by Google Suggest.

In this TOTD (Tip Of The Day) we will create a simple web application with a text field in a JSP page that will use Servlet as the data source. The Servlet retrieves the parameter from the RequestContext, uses Java Persistence API to query the database and return response in the expected format. We will use:
Let's get started!
  1. TOTD #38 explains how to create a MySQL Persistence Unit. Please follow the steps there to create a new Web application and Persistence Unit. Follow the steps listed below after the PU is created.
    1. In Project Explorer, expand "Source Packages", "server" and open "States" class. Add the following NamedQuery:

      @NamedQuery(name = "States.findLikeName", query = "SELECT s FROM States s WHERE LOWER(s.name) LIKE :searchString"),

      at the position shown below:

    2. In "StatesServlet" class, replace the commented code in "processRequest" with the following fragment:

                  String searchString = request.getParameter("autocomplete_parameter");

                  List<States> list = em.createNamedQuery("States.findLikeName").
                          setParameter("searchString", searchString.toLowerCase() + "%").


                  for (int i = 0; i < list.size(); i++) {
                      States s = list.get(i);
                      out.println("<li>" + s.getName() + "</li>");

      and fix the imports by right-clicking in editor pane and selecting "Fix Imports".
  2. Download & Use Script.aculo.us libraries
    1. Download latest Script.aculo.us libraries from here (version 1.8.1 as of this writing) and unzip them.
    2. In NetBeans, right-click on "Web Pages", select "New", "Folder" and specify the folder name as "javascripts".
    3. From the unzipped Script.aculo.us bundle, copy all files from "src" and "lib" directory to the newly created "javascripts" folder.
    4. Expand "Web Pages" and open "index.jsp". Add the following fragment in HTML <head>:

              <script src="javascripts/prototype.js" type="text/javascript"></script>
              <script src="javascripts/scriptaculous.js?load=effects,controls" type="text/javascript"></script>
              <script type="text/javascript">
                  window.onload = function() {
                      new Ajax.Autocompleter("autocomplete", "autocomplete_choices", "/Autocomplete/StatesServlet", {});

      and the following in HTML <body>:

              <input type="text" id="autocomplete" name="autocomplete_parameter"/>
              <div id="autocomplete_choices" class="autocomplete"></div>

      These fragments are part of the original tutorial.
    5. Optionally, specify a stylesheet to render the result nicely
      1. Create a "stylesheets" folder in "Web pages".
      2. Right -click on the newly created folder, select "New", "Other...", "Other" category and "Cascading Style Sheet" file type. Give the name "autocompleter" and click on "Finish".
      3. Replace the generated template with the following contents:

        .autocomplete {
        .autocomplete ul {
        .autocomplete ul li.selected { background-color: #ffb;}
        .autocomplete ul li {
      4. Add the following fragment in "index.jsp" in <head>:

        <LINK href="stylesheets/autocompleter.css" rel="stylesheet" type="text/css">
Now the show time ... right-click the project and select "Run". This deploys the project on GlassFish v2 and brings up "http://localhost:8080/Autocomplete/index.jsp" in the default browser window. The default page looks like:

As you start typing characters in the text box, Ajax.Autocompleter sends a request to the Servlet (specified using the "/Autocomplete/StatesServlet") by passing the typed characters as query parameters. The servlet returns an unordered HTML list. Typing "A" in the text box shows the following output:

and Firebug output looks like:

Typing "C" in the text box shows the following output:

Typing "Mi" in the text box shows the following output:

A request to the Servlet is made everytime a letter is typed. The minimum number of characters that must be entered in the field before a Servlet request is made can be altered by passing the arguments to Ajax.Autocompleter function as shown below (changes highligted in bold):
            window.onload = function() {
                new Ajax.Autocompleter("autocomplete", "autocomplete_choices", "/Autocomplete/StatesServlet", { minChars: 2 });

Some potential fun ideas to make this entry more meaningful:
  • Servlet can access data from a RESTful endpoint and transform the data to an unordered list
  • Autocompleter data source return data in JSON format
  • Autocompleter used in a HTML <form> and "afterUpdateElement" is used to process the selected entry, may be filter the data shown
Please leave suggestions on other TOTD (Tip Of The Day) that you'd like to see. A complete archive of all tips is available here.

Technorati: totd web2.0 autocompleter scriptaculous prototype javascript glassfish mysql netbeans

Monday Dec 31, 2007

Screencast #Web11: Travel Map - Another Real-life app using jMaki & Jersey

In my role of Technology Evangelist, I get the opportunity to meet a lot of community (folks like you :) all around the world. In the year 2007, I represented GlassFish (and related technologies - Metro, jMaki and Jersey) at multiple conferences. This blog introduces a  new real-life application that plots all the places I visited this year on a jMaki-wrapped Google Map widget. Clicking on the marker shows more information about the event such as dates and the blog entry covering the event.

Play the video below to see how the application looks like.

Here is the architecture of this application:

travel map architecture

It consists of a server-side and a client-side applications - developed as NetBeans projects.

  1. Server-side project - A RESTful Web service endpoint that provides resource represenations for all the events attended and associated meta information such as date and blog URLs. This endpoint is created using Jersey.
  2. Client-side project - A jMaki-enabled Web application that consumes the representations generated by the RESTful Web service and plots the information on a jMaki-wrapped Google Map widget.

Both the server-side and client-side are deployed on GlassFish.

This is only a sample application so optimizations are certainly possible and corner cases (such as no blog entry for a particular visit) are not accounted for. But the application still demonstrates the concept. The fully built application looks like as shown below:

Arun's Travel Map 2007

My first presentation in this role was Sun Tech Days Atlanta (highlighted in the image). This application generates an interactive Google Map so feel free to zoom in/out and click 

And one last thing before we build the application. Here is the list of technologies and associated concepts used to build this application:

  1. Jersey
    1. Shows an example of how RESTful Web services can be easily generated from JPA Entity Classes.
    2. Shows how all the resource representations (instead of reference to individual resources) can be returned by a Jersey endpoint.
  2. jMaki
    1. Shows how to consume XML data from an external service (RESTful Web service endpoint) in this case.
    2. Shows how the underlying data model of a widget (Google Map in this case) can be accessed and manipulated.
  3. GlassFish
    1. All the applications are deployed on GlassFish - implicit in the development/deplyment process through seamless integration with NetBeans.
  4. NetBeans 6
    1. Used for generation of RESTful Web services from JPA Entity Classes.
    2. Used for generating/deploying jMaki projects and drag-and-drop of jMaki-wrapped widgets.
  5. JavaScript Closures - to persist the state for asynchronous callback functions
  6. JavaScript DOM processing - to process the XML data received from Jersey endpoint.
  7. Google Maps API
    1. Generate meaningful markers on each location
    2. Populate Google Map from a RESTful Web service endpoint
  8. Java Persistence API - to retrieve data from the database.

And finally, lets build this application. Lets build the RESTful Web service endpoint project first.

  1. Create and Populate the Database
    1. In the NetBeans IDE, go to Services tab, and connect to the database with URL "jdbc:derby://localhost:1527/sample [app on APP]" (right-click and select "Connect...").
    2. Right-click on this database and select "Execute Command..." and create a table by giving the following command:

      create table EVENTS (id int GENERATED ALWAYS AS IDENTITY,
                          event_name varchar(255),
                          dates varchar(20),
                          venue varchar(255),
                          blogs varchar(2056),
                          PRIMARY KEY (id))

      Notice, the "id" column is marked as IDENTITY that instructs the database to auto generate the values for this column and increment by 1 (default) for each row. This column is also marked as the primary key.
    3. Again right-click on the database and select "Execute Command..." to add data to the table by giving the following command:

      INSERT INTO EVENTS (event_name, dates, venue, blogs) VALUES('SunTech Days - Atlanta', 'Jan 16 - Jan 17', 'Cobb Galleria Center, Two Galleria Parkway, Atlanta, Georgia, 30339', 'http://blogs.sun.com/arungupta/entry/wsit_and_web_2_0');
      INSERT INTO EVENTS (event_name, dates, venue, blogs) VALUES('jMaki Day', ' Feb 23', '4150 Network Circle Santa Clara, CA 95054', 'http://blogs.sun.com/arungupta/entry/sun_internal_jmaki_day_review');
      INSERT INTO EVENTS (event_name, dates, venue, blogs) VALUES('Ajax World - New York', 'Mar 19 - Mar 21', 'The Roosevelt Hotel, 45 E 45th St, New York, NY 10017', 'http://blogs.sun.com/arungupta/entry/sun_ajax_world');
      INSERT INTO EVENTS (event_name, dates, venue, blogs) VALUES('The Server Side Java Symposium - Las Vegas', 'Mar 22', '3355 Las Vegas Blvd. South Las Vegas, NV 89109', 'http://blogs.sun.com/arungupta/entry/sun_the_server_side_java, http://blogs.sun.com/arungupta/entry/tango_at_venetian_las_vegas');
      INSERT INTO EVENTS (event_name, dates, venue, blogs) VALUES('JavaOne - San Francisco', 'May 7 - May 11', 'Moscone Center, 747 Howard Street, San Francisco, CA 94103', 'http://blogs.sun.com/arungupta/entry/slides_for_ts_4865, http://blogs.sun.com/arungupta/entry/javaone_2007_day_1_finished, http://blogs.sun.com/arungupta/entry/javaone_2007_day_1, http://blogs.sun.com/arungupta/entry/javascript_everywhere_javaone_2007_demo, http://blogs.sun.com/arungupta/entry/excel_using_wsit_javaone_2007, http://blogs.sun.com/arungupta/entry/ts_4865_takes_two_to, http://blogs.sun.com/arungupta/entry/communityone_glassfish_day_report, http://blogs.sun.com/arungupta/entry/javaone_2007_backstage, http://blogs.sun.com/arungupta/entry/javaone_2007_is_almost_here, http://blogs.sun.com/arungupta/entry/my_javaone_2007_picks');
      INSERT INTO EVENTS (event_name, dates, venue, blogs) VALUES('Rails Conf - Portland', 'May 17 - May 20', '777 NE MLK, Jr. Blvd. Portland, OR 97232', 'http://blogs.sun.com/arungupta/entry/tim_bray_s_keynote_session, http://blogs.sun.com/arungupta/entry/sun_rails_conf_2007_keep, http://blogs.sun.com/arungupta/entry/getting_started_with_jruby_tutorial, http://blogs.sun.com/arungupta/entry/jmaki_netbeans_and_glassfish_in');
      INSERT INTO EVENTS (event_name, dates, venue, blogs) VALUES('Google Developer Day - San Jose', 'May 31', '150 W San Carlos St San Jose, CA 95113', 'http://blogs.sun.com/arungupta/entry/google_developer_day_report');
      INSERT INTO EVENTS (event_name, dates, venue, blogs) VALUES('Mashup Camp - Mountain View', 'Jul 18 - Jul 19', 'Computer History Museum, 1401 N Shoreline Blvd., Mountain View, CA 94043', 'http://blogs.sun.com/arungupta/entry/jmaki_at_mashup_camp_report, http://blogs.sun.com/arungupta/entry/jmaki_mashup_camp');
      INSERT INTO EVENTS (event_name, dates, venue, blogs) VALUES('OSCON - Portland', 'Jul 23 - Jul 27', '777 NE MLK, Jr. Blvd. Portland, OR 97232', 'http://blogs.sun.com/arungupta/entry/jmaki_oscon');
      INSERT INTO EVENTS (event_name, dates, venue, blogs) VALUES('JRuby Hack Day - San Francisco', 'Aug 8', '1201 8th St, San Francisco, CA 94107', 'http://blogs.sun.com/arungupta/entry/jruby_on_rails_hackday_report, http://blogs.sun.com/arungupta/entry/learn_jruby_on_rails_free');
      INSERT INTO EVENTS (event_name, dates, venue, blogs) VALUES('Rich Web Experience - San Jose', 'Sep 6 - Sep 8', '170 S Market St, San Jose, CA 95113', 'http://blogs.sun.com/arungupta/entry/the_rich_web_experience_2007, http://blogs.sun.com/arungupta/entry/jmaki_javafx_the_rich_web');
      INSERT INTO EVENTS (event_name, dates, venue, blogs) VALUES('Rails Conf Europe - Berlin', 'Sep 17 - Sep 19', 'Maritim Pro Arte, Friedrichstrasse 151, 10117 Berlin', 'http://blogs.sun.com/arungupta/entry/rails_conf_europe_2007_day2, http://blogs.sun.com/arungupta/entry/rails_conf_europe_2007_day1, http://blogs.sun.com/arungupta/entry/rails_conf_europe_2007_day, http://blogs.sun.com/arungupta/entry/jmaki_netbeans_and_glassfish_in1');
      INSERT INTO EVENTS (event_name, dates, venue, blogs) VALUES('Sun Tech Days - Rome', 'Sep 24 - Sep 25', 'Meliá Roma Aurelia Antica, Vía Aldobrandeschi, 223  Rome ITALY  00163', 'http://blogs.sun.com/arungupta/entry/netbeans_day_rome_2007, http://blogs.sun.com/arungupta/entry/travel_tips_to_rome, http://blogs.sun.com/arungupta/entry/glassfish_metro_jersey_and_jmaki');
      INSERT INTO EVENTS (event_name, dates, venue, blogs) VALUES('Sun Tech Days - Milan', 'Sep 26 - Sep 28', 'ATA Hotel Quark - Via Lampedusa 11/a 20141 Milano, Italia', 'http://blogs.sun.com/arungupta/entry/glassfish_day_milan_2007');
      INSERT INTO EVENTS (event_name, dates, venue, blogs) VALUES('Mid West Java Tech Days - Minneapolis', 'Oct 16', 'University of St Thomas, MPL 201, 1000 LaSalle Avenue, Minneapolis, MN 55403-2005', 'http://blogs.sun.com/arungupta/entry/mid_west_java_tech_days, http://blogs.sun.com/arungupta/entry/metro_and_jmaki_in_minneapolis');
      INSERT INTO EVENTS (event_name, dates, venue, blogs) VALUES('Mid West Java Tech Days - Chicago', 'Oct 18', 'Donald E Stephens Convention Center, 9301, W Bryn Mawr Ave, Rosemont IL 60018', 'http://blogs.sun.com/arungupta/entry/mid_west_java_tech_days1, http://blogs.sun.com/arungupta/entry/crowne_plaza_chicago_o_hare, http://blogs.sun.com/arungupta/entry/metro_and_jmaki_in_minneapolis');
      INSERT INTO EVENTS (event_name, dates, venue, blogs) VALUES('Silicon Valley Code Camp - Los Altos', 'Oct 27', 'Foothill College, Los Altos, CA', 'http://blogs.sun.com/arungupta/entry/silicon_valley_code_camp_trip, http://blogs.sun.com/arungupta/entry/metro_jmaki_silicon_valley_code');
      INSERT INTO EVENTS (event_name, dates, venue, blogs) VALUES('Sun Tech Days - Beijing', 'Nov 1 - Nov 3', 'Beijing International Convention Center, No.8 Beichendong Road Chaoyang District, Beijing', 'http://blogs.sun.com/arungupta/entry/glassfish_day_beijing_2007_by, http://blogs.sun.com/arungupta/entry/wangfujing_street_authentic_china_in, http://blogs.sun.com/arungupta/entry/sun_tech_days_beijing_talent, http://blogs.sun.com/arungupta/entry/sun_tech_days_beijing_day, http://blogs.sun.com/arungupta/entry/travel_tips_to_beijing, http://blogs.sun.com/arungupta/entry/glassfish_day_beijing');
      INSERT INTO EVENTS (event_name, dates, venue, blogs) VALUES('Partner Preso - Toronto', 'Nov 21', 'Toronto City Center', 'http://blogs.sun.com/arungupta/entry/metro_jmaki_jruby_glassfish_q');
      INSERT INTO EVENTS (event_name, dates, venue, blogs) VALUES('Partner Preso - Montreal', 'Nov 21', 'Montreal City Center', 'http://blogs.sun.com/arungupta/entry/metro_jmaki_jruby_glassfish_q');
      INSERT INTO EVENTS (event_name, dates, venue, blogs) VALUES('GlassFish - Delhi University', 'Dec 3', 'New Delhi', 'http://blogs.sun.com/arungupta/entry/glassfish_delhi_university');
      INSERT INTO EVENTS (event_name, dates, venue, blogs) VALUES('FOSS.IN - Bangalore', 'Dec 4', 'India Institute of Science, Bangalore', 'http://blogs.sun.com/arungupta/entry/packaging_java_apps_for_ubuntu, http://blogs.sun.com/arungupta/entry/foss_in_schedules_now_available, http://blogs.sun.com/arungupta/entry/glassfish_foss_in_2007');
      INSERT INTO EVENTS (event_name, dates, venue, blogs) VALUES('Partner Preso - Bangalore', 'Dec 4', 'Bangalore', 'http://blogs.sun.com/arungupta/entry/glassfish_bangalore_chennai_and_pune');
      INSERT INTO EVENTS (event_name, dates, venue, blogs) VALUES('Partner Preso - Chennai', 'Dec 5', 'Chennai', 'http://blogs.sun.com/arungupta/entry/glassfish_bangalore_chennai_and_pune');
      INSERT INTO EVENTS (event_name, dates, venue, blogs) VALUES('Partner Preso - Pune', 'Dec 6', 'Pune', 'http://blogs.sun.com/arungupta/entry/glassfish_bangalore_chennai_and_pune');
      INSERT INTO EVENTS (event_name, dates, venue, blogs) VALUES('Partner Preso - San Francisco', 'Dec 17', 'San Francisco', 'http://blogs.sun.com/arungupta/');

      These SQL statements populate the database with details about my visits in 2007. If you'd like to develop a similar application highlighting your visits then you'll need to modify the VALUES clause to match accordingly.
  2. Create and Configure RESTful Web service endpoint
    1. Create a Persistence Unit as described in "Generating Entity Classes from Database" section in Getting Started with RESTful Web Services. Lets say the project name is "WebApplication3", package name is "events" and the table name to generate Entity classes is EVENTS. Take everything else as the defaults.
    2. Generate a RESTful Web service as described in "Generating RESTful Web Services from Entity Classes" section in Getting Started with RESTful Web Services.
      1. Add a new class EventsList in the events package as:

        public class EventsList {
          protected java.util.List<Events> events;

          public EventsList() {
            if (events == null)
              events = new java.util.ArrayList<Events>();

          public void add(Events name) {

          public java.util.List<Events> getValue() {
            return events;
      2. In service.EventsResource, change the method associated with GET to:

        public EventsList get() {
                EventsList eventsList = new EventsList();
                List<Events> list  = PersistenceService.getInstance().createQuery("SELECT e FROM Events e").getResultList();
                for (Events e : list) {
                return eventsList;

        This will ensure that all the resource representations are returned instead of a reference to the resource. Make sure to fix the imports.
That's it, our server-side project is now ready. "http://localhost:8080/WebApplication3/resources/events" now return a complete RESTful representation of all the rows from the database table EVENTS.

Lets build the client-side application next. Make sure jMaki plug-in in NetBeans IDE is already installed.
  1. In the NetBeans IDE, create a new Web project, enable "Ajax Framework" and choose the "Standard" layout for "index.jsp".  Lets say the project name is "WebApplication4".
  2. Drag-and-drop a jMaki-wrapped Google Map widget in the 'Main Content Area' and jMaki-wrapped Yahoo Button in the 'Sidebar Content Here'.
  3. Customise the widgets
    1. Add id="mymap" attribute to the Google Map widget. The updated widget looks like as shown below:

      <a:widget  name="google.map" id="mymap"
               args="{ centerLat : 37.4041960114344,
                       centerLon : -122.008194923401 }" />

      will allow the Map widget to be accessed by name later.
    2. Add args="{label:'Plot Events'}" attribute to thes Yahoo button widget. The updated widget looks like as shown below:

      <a:widget name="yahoo.button" args="{label:'Plot Events'}"/>
  4. In glue,js, add the following code to \*onClick subscribe method:

    var url = jmaki.xhp + "?id=events";
    var _map = jmaki.getWidget("mymap").map;
    jmaki.doAjax({method: "GET",
        url: url,
        callback: function(_req) {          
            var xmlobject = (new DOMParser()).parseFromString(_req.responseText, "text/xml");
            var root = xmlobject.getElementsByTagName('eventsList')[0];
            var events = root.getElementsByTagName('events');
            for (var i = 0 ; i < events.length ; i++) {
                var event = events[i];
                var eventName = event.getElementsByTagName('eventName')[0].firstChild.nodeValue;
                var venue = event.getElementsByTagName('venue')[0].firstChild.nodeValue;
                var blogs = event.getElementsByTagName('blogs')[0].firstChild.nodeValue;
                var dates = event.getElementsByTagName('dates')[0].firstChild.nodeValue;
    var id = event.getElementsByTagName('id')[0].firstChild.nodeValue;
                var encodedLocation = encodeURIComponent("location=" + venue);
                var url = jmaki.xhp + "?id=yahoogeocoder&urlparams=" + encodedLocation;
                jmaki.myHandler(url, eventName, blogs, dates, id, _map);
  5. Add the following functions above the \*onClick subscribe method:

    // "Function closure" used from http://econym.googlepages.com/basic1.htm
    // Creates local copy of "marker" and "html" variables to be preserved for later use
    function createMarker(point,html) {
        var marker = new GMarker(point);
        GEvent.addListener(marker, "click", function() {
        return marker;

    // Function closure that preserves "eventName", "blogs", "dates and "id"
    // Gets the latitude/longitude from Yahoo Geocoding service and plots them on the map
    // Also creates meaningful markers
    jmaki.myHandler = function(_url, eventName, blogs, dates, id, _map) {
        jmaki.doAjax({url: _url,
            callback : function(req) {
                if (req.responseText.length > 0) {
                    jmaki.log("name: " + eventName);
                    var response = eval("(" + req.responseText + ")");
                    var coordinates = response.coordinates;
                    jmaki.publish("/jmaki/plotmap", coordinates);
                    jmaki.log("plotting " + eventName);
                    var latlng = new GLatLng(coordinates[0].latitude, coordinates[0].longitude);
                    var blogHtml = "";
                    b = blogs.split(', ');
                    for (i=0; i<b.length; i++) {
                        blogHtml += '<a href="' + b[i] + '">' + (i+1) + '</a>';
                        if (i<b.length-1)
                            blogHtml += ", ";
                    var txt = '<table>' +
                    '<tr><td>#' + id + ": " + eventName + '</td></tr>' +
                    '<tr>Dates: ' + dates + ', 2007</td></tr>' +
                    '<tr><td>Blogs: ' + blogHtml + '</td></tr>' +
                    var marker = createMarker(latlng, txt);
                } else {
                    jmaki.log("Failed to get coordinates for " + location );
  6. Add the following entry in Web Pages, resources, xhp.json:

    {"id": "events",

    assuming WebApplication3 is the project where RESTful Web service endpoint is hosted.
That completes our client-side web application as well. Now, either hit F6 (default key to Run the NetBeans project) and this will show http://localhost:8080/WebApplication4/index.jsp in the configured browser. Once you click on "Plot Events" button, all the markers on the Google Map are plotted.
Future Improvements
  1. If Jersey can return all the resource representations directly, then the workaround used above may not be required.
  2. Use e4x after <script type="text/javascript"> in index.jsp can be generated as <script type="text/javascript; e4x=1">.
  3. Build the client-side application using Rails once issue #309 & #310 are resolved.
    1. Once deployed as Rails application on WEBrick, create a WAR file and deploy on GlassFish.
    2. Try this application using GlassFish v3 gem.
  4. Embed Google Map in the blog entry.
An alternate title of this blog entry could've been "How I spent my winter break ?". But in order to keep the title inline with rest of other entries (keeping it simple and reflecting the content of the entry) I decided to use the existing title ;-)

Technorati: screencast conf jmaki jersey netbeans glassfish jpa javascript googlemaps restful web2.0 jmakimashups

Wednesday Mar 14, 2007

Next Gen Web @ JavaOne

Follow up from my previous entry, here is a complete list of all the next generation Web application, a.k.a "Web 2.0", sessions from Sun.

Session # Title
BOF 6012 JavaScript Programming Language Best Practices for Developers on the Java Platform
TS 6029 Beyond Blogging: Feeds in Action
TS 6375 jMaki: Web 2.0 App Building Made Easy
TS 6381 The Future of the Java Technology Web Tier
TS 6411 JSR 311: The Java API for RESTful Web Services
BOF 6412 Describing RESTful Applications: WADLing with Java
BOF 6424 Accessibility for Ajax and Web 2.0 Applications, from Emerging Concepts to Practical Coding
BOF 6425 Testing Web 2.0 Features, Using Real-World Applications
BOF 6807 Real-World Comet-Based Applications
BOF 6876 Ajax and Web 2.0 Performance Roundtable
TS 6957 Project Phobos: Server-Side Scripting for the Java Platform
TS 8840 Services Interoperability with Java Technology and .NET: Technologies and Tools for Web 2.0
TS 9516 Using jMaki in a Visual Development Environment

There is more content that has not been publicly announced yet. And while we are working on making presentations for this year richer, fuller and more hands-on, get your self familiar with JavaOne 2006 archives on this topic.

Session # Title
TS 1161 Evolving JavaServer Faces Technology: AJAX Done Right
TS 1222 RESTful Web Services With JAX-WS
TS 1615 Java EE 5 BluePrints for AJAX-Enabled Web 2.0 Applications
TS 1756 Java Technology and REST: Implementing the Atom Protocol
TS 3577 Using the Dojo Toolkit to Develop AJAX-Enabled Java EE Web Applications
TS 4372 Java Technology, AJAX, Web 2.0 and SOA
TS 8614 AJAX & Persistence: Emerging Patterns & Pain Points

Technorati: Javaone Web2.0 Ajax jmaki REST JavaScript

Thursday Mar 01, 2007

Language-neutral data format: XML and JSON

XML and JSON are the two prevalent choices for language-neutral data format. That means a format used to exchange data between client and server, independent of the language used on each end. We are familiar with XML pointy bracket syntax which has served us well so far. With Rich Internet Applications becoming more common, there is a need to have a light-weight data interchange format. And so JSON is catching up (11% for data transfer in 2006).

Basically, JSON is built on two structures:

  • A collection of name/value pairs with unique names (associative array)
  • An ordered list of values (array)

See message samples formatted in JSON and equivalent XML. Tim Bray summarizes when to use which format.

Here is a collection of interesting articles in case you want to dig deeper:

The key advantages of JSON I derived from my reading of the above articles are:

  • Much simpler than XML because it is not a markup language and a natural representation of data.
  • JSON is better data exchange format, XML is a better document exchange format. 
  • JSON is easier to read for machines with no/thin client-side library.
  • JSON is a natural fit for data consumption by browser clients, for example Ajax components.
  • Ability to represent general data structures: records, lists and trees.
  • Wikipedia entry for JSON reports parsing and generating JSON support in 21 languages.

There are some disadvantages as well:

  • JSON format is hard to read for humans; for example complicated-looking syntax, like the }}]} at the end of data snippet is frightening and debugging pain.
  • JSON is a newer format so not enough tools to help with authoring & parsing. Some available are:
    • JSON Tools - Java Tools for the JSON Format (parser, renderer, serializer, mapper, validator)
    • JSON-lib - Java library for transforming beans, maps, collections, java arrays and XML to JSON and back again to beans.
    • JSON in Java - Java APIs from json.org (see more below)
    • JSON-taglib - JSON-taglib is a JSP 2.0 tag library used to render JSON data from within JSP code.
    • Could not find an editor that would allow me to edit JSON objects.
  • JSON does not have a <[CDATA[]]> feature, so it is not well suited to act as a carrier of sounds or images or other large binary payloads.
  • Unlike XML, JSON does not provide any display capabilities because it is not a document markup language. JSON was not even intended for that purpose.
  • JSON is not extensible - it does not need to be because it's not a document markup language.

In jMaki, we use JSON in Java. Here is a sample code to create a JSON object using these APIs:

import org.json.\*;
import java.io.\*;

public class JSONSample {
  public static void main(String[] args) throws Exception {

    // basic object creation
    JSONObject person = new JSONObject();
    person.put("name", "duke");
    person.put("age", "10");

    // how to create array and write to a "writer"
    JSONObject address = new JSONObject();
    JSONArray array = new JSONArray();
    array.put("4140, Network Circle";);
    array.put("Santa Clara";);
    array.put("CA - 95054";);
    address.append("address", array);
    OutputStreamWriter osw = new OutputStreamWriter(System.out);

    // XML->JSON conversion
    JSONObject likes = XML.toJSONObject("<likes><running/><skiing/></likes>");

And here is the corresponding output:

{"address":[["4140, network circle","Santa Clara","CA - 95054"]]}

This API also allows conversion from comma-delimited text, HTTP, Cookie, and CookieList to JSON conversions. The source code for JSON in Java is freely available but here are two suggestions for ease-of-use:

  1. Provide a jar file that is ready to use
  2. Publish the link to framed version of javadocs on the main page since that is more useful.

In summary, XML is document-oriented and JSON is data-oriented. So if you want to deal with highly structured documents that requires a complex structure, binary data, exact ordering of elements and be able to render itself then use XML. OTOH, if you are focused on light-weight data exchange then JSON is the way to go. 

Follow the JSON blog and enjoy!

Technorati: XML JSON DataFormat JavaScript Web2.0 Ajax jMaki


profile image
Arun Gupta is a technology enthusiast, a passionate runner, author, and a community guy who works for Oracle Corp.

Java EE 7 Samples

Stay Connected


« April 2014