Wednesday Sep 30, 2009

Improved communications between users and publishers - Builds #81 and #82 deployed

Yesterday, we deployed build #82 of the Sun Software Library into production, and two weeks ago, we deployed build #81 into production, enhancing the library functionality in the following areas:

  • New Artwork: We have some new artwork:
    Sun Software Library artwork
    Other sites that want to link to the Sun Software Library can use this artwork.

  • Request to be a Content Manager includes a message: One of the reasons we had to change our Terms & Conditions is to allow the users to better communicate with other users.  In this release, we have added the ability for a content management requests to include a message, enabling the user to tell the current content administrators why they are requesting access. ⁞ For small organizations, "where everybody knows your name", this may not seem like a useful feature, but for large organizations (e.g. think of companies the sun of Sun Microsystems), this is especially useful.

    Sun Software Library Content Management Request Message

    The next few releases will include more functionality, enabling users to communicate with one another via the library.

  • New Terms & Conditions: As mentioned earlier, we wanted to add new functionality in the Sun Software Library, enabling users to anonymously contact publishers (and vice versa).  If you log into the Sun Software Library, and have previously accepted the terms and conditions, you'll see this popup, informing you that new terms and conditions are in place:

    Sun Software Library Terms and Conditions have changed

    Click "Accept" to continue.

    We've added this language to the Privacy section (section 6).  You can view the entire terms and conditions at https://library.network.com/terms.html.
     “You acknowledge and agree that Site Offerings may enable You to
    allow to be contacted by Sun other Users, and may enable You to
    submit personally identifiable and other information to other
    Users.   You consent to such contact by other Users (unless Sun
    expressly gives You notice and an ability to opt-out from such
    contact), and to the unrestricted, non-exclusive use by such
    other Users of any such information You choose to submit to them,
    and You assume (and Sun disclaims) all liability and
    responsibility arising therefrom or related thereto.”
  • Internet Explorer support: IE is our most popular browser, some 55% of our users use it.  As such, we need to support it - all 3 versions of it (IE6, IE7, & IE8).  However, we do pay for that support, in terms of engineering time, both in testing, and in fixing bugs.  Primarily because of Firebug and our Mac desktops, Firefox continues to be our primary development and debug platform, with Safari being a close second.  We do fully test on Windows platforms, but find that what typically works on the other two browsers breaks on IE.  Recently, we've spent a significant amount of time debugging IE8, while maintaining functionality for previous versions of IE.

    It's hard to estimate what our actual cost is for supporting IE6, but it is relatively high.  As such, we support efforts to get users to migrate away from IE6.

  • We are 100% JSLint "The Good Parts" compliant: After we integrated JSLint with NetBeans, we got to work making sure our own Web UI (written in JavaScript) passed JSLint.  This took us a while, given the amount of JavaScript code that we have.  The first decision we had to make was what options we wanted to JSLint to enforce - we decided to leverage The Good Parts, written by Douglas Crockford.  Given that this is our entry in build.xml:
            <jslint4java haltOnFailure="${cqs.build.jslint.failonerror}" options="${cqs.build.jslint.options}">
              <formatter type="plain" />
              <filelist id="jslint.filelist" dir="${basedir}/web" files="${cqs.js.jslint.files}"/>
            </jslint4java>
    
    We then set this option in our project.properties file:
            cqs.build.jslint.options=white, onevar, browser, undef, nomen, eqeqeq, plusplus, bitwise, regexp, strict, newcap, immed
    
    After we cleansed our JavaScript code, we set our builds to fail if JSLint detects any errors:
            cqs.build.jslint.failonerror=true
    
    This approach has already paid a dividend: One of our developers wrote some code that worked perfectly on Firefox and Safari, but would have created issues on IE, due to IE's more stringent syntax checking.  Normally, we would have caught this error during our Quality cycles, but JSLint picked up the issue by causing our Continuous Integration build to break.  The issue was fixed in minutes at a relatively low cost, rather than in hours at a much greater cost at a later period in time.

    This incident alone has validated our investment in integrating code inspection tools into our Continuous Integration builds.

    And, no, our feelings weren't hurt!  Humbled, maybe, deflated our egos a bit, definitely, but not hurt.

  • Cobertura integration: In the spirit of integration of code inspection tools into our environment, we integrated Cobertura into our Continuous Integration system.  We've run Cobertura before, but always manually, on an as needed basis.  Now we always have an up to date code coverage report.  Combined with Hudson's Cobertura Plugin, we configured our builds to measure code coverage taking into account all of our automated testing: Web Services based functional tests, Selenium based Web UI tests, and jUnit based traditional unit tests.  This way, we can measure the effectiveness of our testing strategy in realtime.  We still have some work to do in this area, but at least we know where we stand:

    Sun Software Library Code Coverage

  • And as always, lots of bug fixes and minor enhancements here and there...
There are more improvements in store, keep the feedback coming!

Friday May 01, 2009

Home page upgrade, making logos easier, and more - build #71 deployed

Our latest version - build #71 - was deployed into production yesterday.  Here is what we added:

  • What's New: This new panel on the home page takes a subscription feed from our blog at http://blogs.sun.com/thelibrary  and displays the three most recent entries right on our home page.
    What's New and Featured
  • Featured: This new panel on the home page allows us to highlight specific publishers, software, or resources directly. Want to be listed here? Give us feedback.

  • Searching for a specific tag: We have a lot of tags in our system - approximately 3000 of them.  The vast majority do not make it into our tag clouds.  We now allow you to search for a specific tag, or set of tags, using the same mechanisms as keyword search.  This way, if you create a tag that is interesting to you, but doesn't make it into the other tag clouds, you can find it.  The output is a little different, however - we display tag search results in a custom tag cloud.

    Searching for Tags

    Please note that this feature is different than using a Tag to search for entries in the library, which has been in the system since the very beginning.

  • Logo Preview in add/edit panel to improve usability of logos: Previously, many entries in the library did not take advantage of our logo mechanism, we always wondered why, since after all, an entry with a logo looks better than an entry without a logo.  We always wondered why our users do not take advantage of this feature.  Other users tried to take advantage of this feature, but put in invalid URLs.

    Many principles in User Interface design teach that if the user does something wrong, the sooner you inform the user, the more usable your application becomes.  As such, we now display the logo preview directly in the panel that allows you to edit that particular field.  The response is in real time - if you type in an invalid URL for your logo, this is what you'll immediately see:

    Invalid Logo URL

    Type in a valid URL, however, and you'll get some instant gratification!  Ah, the sweet site of success ;-)

    Valid Logo URL

    The circled annotations above were created manually using an image editing tool - the app does not try to figure out what is wrong with the URL, that's a pretty hard problem to solve, we'll save that one for the next iteration. ;-)

  • Improved content management requests and authorization emails: The emails that we used to send whenever a user requested to be a content manager, or when that request was approved or denied, needed some improvement.  We paid some attention to them, and now they are significantly more user friendly.  Our application has direct URL's to many portions of the app, we take advantage of this by including a URL in the approval emails directly to the approval section of the application.  This is typical in traditional web applications, which have a URL for each page.  Our app is a RIA - which exists on a single HTML page, making this issue a bit more complex from an engineerign perspective.

  • Prettier URL: While we're not that vane, our old system used to include "CatalogQueryServer" as part of the URL structure.  CatalogQueryServer is the web context at which our application is deployed.  This used to leak through sometimes in our URL's, making them less user-friendly.  Specifically, our system used to redirect users from http://library.network.com  to https://library.network.com/CatalogQueryServer  (note the HTTP to HTTPS transition in the redirect), and the web context used to leak through the automated redirect.  We have fixed that.  So now we have a nice pretty compact URL structure.

As always, feedback (either positive, constructive, or what the weather is like where you) is always welcomed!

Tuesday Apr 21, 2009

Improving RIA / JavaScript startup performance in latest deployment - build #70

On Monday April 13th, 2009, we deployed build #69 of the Sun Software Library.  This blog entry details the additions we've made.  If you've been following this blog, you know that the Sun Software Library engineering team does a great job of deploying a new build every two weeks, but the blog entries typically trail the deployment a bit.  That's completely because the writer of this blog is a slacker.  No other excuse can be made.  All sorts of gimmicks were tried, apparently to no avail.

This release completes, for now, all of our startup performance improvements associated with our RIA.  As you know, the Sun Software Library user interface is written in JavaScript, and accesses the data in the library using Restful Web Services.  Our Web UI currently has:

  • 12,000 lines of JavaScript code written by us, in 17 different files
  • 2,000 lines of CSS code (I guess you can call CSS code, right?), in 4 different files
  • Oodles of images
  • Many 3rd party libraries (for example, we rely heavily on the Ext JS toolkit for web UI components)

When we initially deployed the application many many months ago, the browser had to download each and every one of these components separately, requiring approximately 20 seconds to load the entire application, and that was on a fast Internet link in relative network proximity to the web site.  Performance from distant parts of the world was atrocious, and that's putting it mildly.  We made things a bit better by building an "index.jsp" home page that loaded relatively quickly (e.g. approximately 8-10 seconds), then transitioned to the "app.jsp" main page (which took approximately 20 more seconds to load).

So we looked for tools to help us improve performance.  After all, we're not the only ones in the world writing RIA's in JavaScript.  But amazingly, we didn't find any tools that fit our way of doing work:  we're LAZY, and we're HUMAN!  Being lazy means we abhor labor intensive, manual processes since they are typically very error prone, whereas being human means we like using tools.  Since we couldn't find the right tools, we went about integrating/building various components.  We'll include details in a future blog entry.

So where are we today?

  • A single "app.jsp" page that contains our entire application.
  • A single CSS invocation,  which is automatically generated with a unique file name containing a timestamp, so it can be cached by the browser but also updated in real time.
  • Two JavaScript invocations, one for Ext JS and one for our entire application, each of which have unique file names similar to above.
  • Most of our images are integrated in to a Sprite, automatically generated.  The only remaining images that are distinct come from the 3rd party libraries we are using.
  • All of our content has a "far future cache expiration date".  A browser with a primed cache only requires 8.3 KBytes of new content to render the application (that's the web services call to retrieve the "most viewed" and "recently updated" entries).

Net net, our entire application loads in approximately 2.5 seconds.  And since it is JavaScript based - once the application has started - only web services calls are made to retrieve data to display.  While there is still some work to do to improve performance even more, we're pretty satisfied with where we are.  The only caveat to the home page load time is that the logos for the various entries are not (yet) cached by our service, we retrieve logos from the 3rd party sites on each invocation.  This makes our page load time dependent on services other than our own.  We measured the page load time w/remote logo loading turned off, using yslow.

 So what else is new? 

Sun Software Library Home Page

Our home page now links to three key Sun organizations that we work with daily:

  • Sun Developer Network: Becoming an Sun Developer Network (SDN) member makes you part of a vibrant worldwide community of developers, and gives you access to cool stuff and exclusive offers.
  • Sun Partner Advantage program: The Sun Partner Advantage Program connects Independent Software Vendors (ISVs) to free or deeply discounted offerings as well as sales relationships, so they can deliver solutions/services to an expanded market.
  • Project Kenai: Sun's new foundation for its connected developer experience of tomorrow.  Freely host your open source projects and code as well as find and collaborate with developers of like mind and passion on this site.

Also, we fixed numerous bugs..  Nothing new here, we are always fixing bugs.

As always, feedback (either positive, constructive, or what the weather is like where you) is always welcomed!

Wednesday Apr 01, 2009

Building a JavaScript UI that interacts with both SSL and non-SSL based web services

As you know by reading earlier posts on this blog, we have been working on improving the performance of our Web UI. The authentication user interaction is tightly integrated with rest of the Web UI to provide seamless experience in the rich Web UI. Combining this with the requirement to encrypt user names and passwords, we initially run the entire application under SSL. Ideally, we'd figure out a way to use SSL only for the parts of the communication that require encryption, and regular HTTP traffic for all other aspects of our application (e.g. search results, images, etc.) 

We are experimenting with a new feature that enables the web UI to interact with both SSL based web services and non-SSL based web services within the same browser page. This is one optimization that can help the web site to better scale with increasing traffic.

One technique is the use of iframes with fragment identifiers (text after the hash character in an URL, i.e. http://domain.com/page.html#tragment_identifier). This combination enables the web UI to establish a communication channel needed to pass small amount of information needed between iframes to interact with both SSL and non-SSL service points. For further explanation and a working example, please see this blog post for more information.

This use of fragment identifiers as a cross-domain communication channel is in direct overlap with popular practices for history management, which also use fragment identifiers to store page rendering state. Here is a sample page that shows how this feature works.

This leaves us with a potential situation when a page's rendering state can be wiped out whenever iframes use the same fragment identifiers to pass messages among them. In our particular use case, this is not a problem when the message actually leads to a new page state. If this is not the case, then it may lead to an inconsistent state between the representation in the URL and what's displayed on the page. One work around is to reset or synchronize the page state for such scenarios, which is a minor de-optimization that can generate repeat AJAX calls. This will be an interesting challenge to integrate conflicting features. Look for this performance improvement in an upcoming release

About

Welcome to the Sun Software Library blog, where you will find interesting updates and tidbits about using the Sun Software Library.

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today