Collecting Data on Users of a NetBeans Platform Application (Part 1)

Do you know what your users are doing with your NetBeans Platform application? Which windows are they opening? Which actions are they commonly invoking? When are they clicking the "Help" button on a dialog? Knowing the answers to these questions is crucial when determining where you should be assigning development resources. Knowing that the "Help" button is being pressed a lot for a particular feature might indicate that there is a problem with the UI that you could consider modifying in some way. Also, the priority of bugs can be determined, at least partially, by how frequently something is actually being used. When someone files a P1 bug and writes angry e-mails demanding you fix something... wouldn't it be helpful to find out that the buggy feature in question is only being used by 2% of your user base..?

And so on. The usefulness of knowing what users are doing with your application is endless. Time to add a UI gesture collector to your application! NetBeans IDE has one and, since your application is built on the same infrastructure (i.e., the NetBeans Platform), you can make use of that same gesture collecting infrastructure.

By the way, in the case of NetBeans IDE, it is very clearly stated that UI gestures will be collected, by means of this dialog that appears at the end of the installation procedure:

Now we will set up our own custom NetBeans Platform application to include the infrastructure made available by the UI gesture collection feature of the NetBeans Platform.

The steps that follow are derived from this brand new document on the NetBeans Wiki, so go there for further info and for copying the code snippets and other things you might find missing in the overview below:

http://wiki.netbeans.org/HowToUseUIGesturesCollectorInYourApp

Let's get started!

1. In your application's Libraries tab (within the Project Properties dialog), place a checkmark next to "Exception Reporter", "UI Gestures Collector Infrastructure", and "UI Handler Library", as shown below:

   

2. Somewhere in your code, i.e., wherever there's something a user does that you're interested in tracking, use java.util.logging.LogRecord to collect the gestures. For example, in the case below, the LogRecord code is added to the "componentOpened" method of a TopComponent, so that I can track whenever the end user opens this particular window:

   

   Note: A UI event (i.e., an event that will be identified as a UI gesture) is considered to be everything that is logged into the "org.netbeans.ui" logger, as done above.

3. Run the application and perform the gesture that triggers the LogRecord. In this particular case, I simply needed to open the TopComponent where I had set the LogRecord shown above. Then close the running application. Back in the application's build/testuserdir/var/log folder, you should see a 'uigestures' file, containing (among other things) data on the gesture that was performed, as shown below.

   

   Look at the highlighted tags above and then look the LogRecord code from the earlier step and notice the specified text is reflected in the 'uigestures' file.

4. Now that we are able to log the user's UI gestures, let's set up a way in which we can submit those logs to a server. This is where the UI gestures of NetBeans IDE users is collected. For your own NetBeans Platform application, you need to provide somewhere on your own server where you can collect the logs of the UI gestures of your own application's users.

   You begin this phase by overriding the place where NetBeans IDE gestures are collected. If you look at the sources of NetBeans IDE, you will find that location specific via the "WELCOME_URL" key in the Bundle shown below:

   

   You need to create, in your own application's "branding" folder, a set of folders that matches the above structure, ending with a Bundle file that will override the above Bundle file, for the "WELCOME_URL" key, as can be seen here, pointing to my own UI gesture collector:

   

   At that URL, i.e., the value of the WELCOME_URL key, you need to have a page defined as follows:

   <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
   <html xmlns="http://www.w3.org/1999/xhtml">
       <head>
           <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"></meta>
           <title>Welcome to UI Gestures Collector</title>
       </head>
       <body>
           <p>You can now submit data about the UI actions you performed.</p>
           <form action="http://your.server/analytics/upload.jsp" method="post">
               <input type="hidden" name="submit" value="&Submit Data"></input>
               <input type="hidden" name="auto-submit" value="&Automatic Submit"></input>
               <input type="hidden" name="view-data" value="&View Data" align="left" alt="&Hide Data"></input>
               <input type="hidden" name="exit" value="&Cancel"></input>
           </form>
       </body>
   </html>

   Note: The line in bold above is a JSP page where the data will be sent to, from the page above. The page above will be displayed within a dialog, later in these instructions.

5. Now, let's start the UI gesture server! Type this on the command line:

   hg clone http://hg.netbeans.org/main/misc

   You should see the UI gesture server getting checked out from Mercurial:

   

   Now look at the folder structure that results from the above process. The subfolder "logger/uihandlerserver" is of particular interest:

   

   That's a web application that you can open into NetBeans IDE. You can build it simply by running:

   ant

   ...in the folder above.

   Once you've got the server running, you should be able to go here:

   http://localhost:8888/analytics/

   Make changes to the web application and then type the following to redeploy it:

   ant deploy

   And that's a page, within the server app's folder, where you can provide information to the end user about the statistics.

6. Now we'll add some UI in our NetBeans Platform application so that our users will be able to send off the gesture logs to the server. Different ways you can do this, but including the module "uihandler.interactive" in your application is simplest.

   You can find this module in the development update center or you can use the "AutoUpdateTask" (more on that in a future blog entry) to access it.

   Once you've included it in your application, you'll see this button automatically added to the toolbar, with the tooltip as shown below:

   

   Once the button is clicked, you see this dialog:

   

   Note: The dialog above contains the page defined by the value of the WELCOME_URL key that we branded earlier in these instructions. But even the buttons come from that page. That means that you can remove buttons from the dialog above, simply by removing the related tags from the page. So, here I've removed two of the buttons, so that automatic update and view data have been removed:

   

   Click the "View Data" button (if your UI still has it!) and you'll see the data that is to be sent to the server:

   
   

That's enough for now. Processing and visualization of submitted gesture logs will be treated in a future blog entry, though you should read here if you'd like to know already. Exception reporting and slowness detection will also be covered. Feel free to leave other points of confusion here as questions and I'll try to answer them!