Monday Sep 26, 2011

Tip #7 - Absolute Filenames in Java

I'm busy reading lots and lots of code lately because I need to implant DCOM support.  I just saw this:

  if (nodeDirFile.isAbsolute()) {
                return nodeDir;

Would this surprise you?

File f = new File("/foo/../foo/.././././././foo/././././././../foo/goo");

will return true.  That is an absolute path as far as Java IO is concerned.  I.e. if it BEGINS with something absolute -- then it is absolute.  Even if it's really really ugly.

What was probably desired is getCanonical().  There is one and only canonical filenames per file.  There are infinite "absolute" filenames per file.

Monday Aug 15, 2011

Tip #6 Windows Script to change directory QUICKLY

GlassFish's source tree, like any huge software has enormously long directory paths.  My goal is to try to avoid ever typing them in. Even with the shell auto-filling it still takes too much time, IMHO.

I created 2 scripts on my Windows systems:

1)  "cdn.bat"

cd \gf\main\nucleus\*%1*\*%2*

2) "cda.bat"

cd \gf\main\appserver\*%1*\*%2*


If I want to change my directory to,say flashlight-framework I just do this:

cdn fl fr

This command instantly moves me to cluster-cli

cdn clu cl

Friday Aug 12, 2011

Tip #5 Avoid RegEx When Unnecessary

I see this in GlassFish source code frequently:

string = string.replaceAll("\\\\", "/");

 Whoa!  FOUR backslashes?!?  Yes.  That's a regular expression.  What we see is FOUR but inside the string itself are TWO literal actual backslashes. The backslashes have to be doubled so that the RegEx parser knows that we are looking for a literal backslash.

What the author of this code really wants to do is something like this:

c:\a\b\c\d  ==>  c:/a/b/c/d

 This can easily be done without the more confusing regex notation and (I assume) the slower regex code like so:

string = string.replace('\\', '/');

Tuesday Aug 09, 2011

Tip #4: Stop and Think before you make that Class public

I run into this time-wasting situation fairly frequently:

I need to change the signature of a Java class, in GlassFish,  that has been declared public.  Uh-oh!  It's public so I can't just change it. 

So -- my first step is to grep through every single java file in GlassFish.  That means  7,772 files currently!

Now I have to wonder -- is the class being used from modules outside of GlassFish?  Uh-oh -- now I have to run massive automated tests to make sure I don't break something.

Ultimately you can never be totally sure that public class is not being used somewhere.  E.g. say the name of the class is FooGooHoo.  What if there is code doing a Class.forName("Foo" + "Goo" + "Hoo") ?  I'll never find it with a simple search. 

This is all expected stuff with the public keyword.  But what if the class is exclusively used inside of its own package?  Then I just  wasted a whole lot of my employer's money chasing down non-existent references. 

On the other hand if the class had its visibility set to pkg-private in the first place, all I had to do was to search inside that one package for uses.  Java guarantees no other package can see it.  And if other code somehow subverts that policy -- tough!!

Moral of the Story:

Only mark classes and methods as public when you are absolutely positively sure that it must be public.  When in doubt make it pkg-private and avoid protected too.

Thursday Jul 28, 2011

Introducing DTrace in GlassFish 3.1.1

This is the first in a series of blogs about Dtrace in GlassFish.  I will present concrete examples  in upcoming blogs.  This blog is an overview of this powerful new capability.

An entire book could be (and has been) written just to describe DTrace itself.   There are many many useful documents on the internet about DTrace and its usage.  At the end of this blog I've referenced a few good ones that I've read.

DTrace is a tool available currently on Solaris 10 and Mac OS X that will dynamically instrument already-running code  for the purposes of seeing what is going on inside software.  This includes the OS itself.  You can probe into kernel-mode in addition to user-mode. 

GlassFish automatically supports DTrace.  For every Probe that is registered in GlassFish, a corresponding DTrace probe is generated and made available to DTrace.  This includes any probes that you create yourself such as in your Applications.  See  [1] for a tutorial on this.

In order for the DTrace probes to be available you need to do 3 things from the GlassFish end:

  1. GlassFish must be running in JDK 7
  2. Dtrace must be enabled.  Simply run this command:
    asadmin enable-monitoring --dtrace
  3. You must have glassfish-dtrace.jar copied to the GlassFish modules directory.  Update center will set things up for you by doing the following
    1. You must use an official distribution:
      1. Open Source Distribution
      2. Oracle GlassFish Server Distribution
      3. Official Oracle GlassFish Server page is located here
    2. Run: <installdir>/bin/updatetool to bootstrap update clients.
    3. Run:  <installdir>/bin/pkg install glassfish-monitoring-dtrace@3.1.1 to install the DTrace package

Of course you also have to turn on the monitoring levels for what you are interested in.  To turn Monitoring up full-blast for everything run these commands (you can make one huge command out of it and stuff it into a script like I do):

asadmin enable-monitoring --target server --modules deployment=HIGH
asadmin enable-monitoring --target server --modules connector-connection-pool=HIGH
asadmin enable-monitoring --target server --modules connector-service=HIGH
asadmin enable-monitoring --target server --modules ejb-container=HIGH
asadmin enable-monitoring --target server --modules http-service=HIGH
asadmin enable-monitoring --target server --modules jdbc-connection-pool=HIGH
asadmin enable-monitoring --target server --modules jms-service=HIGH
asadmin enable-monitoring --target server --modules jvm=HIGH
asadmin enable-monitoring --target server --modules orb=HIGH
asadmin enable-monitoring --target server --modules thread-pool=HIGH
asadmin enable-monitoring --target server --modules transaction-service=HIGH
asadmin enable-monitoring --target server --modules web-container=HIGH
asadmin enable-monitoring --target server --modules jersey=HIGH
asadmin enable-monitoring --target server --modules jpa=HIGH
asadmin enable-monitoring --target server --modules security=HIGH
asadmin enable-monitoring --target server --modules web-services-container=HIGH

Obviously you have to be on an OS that has dtrace!

You can quickly and easily see that everything is registered correctly and available for running in DTrace scripts with this command:

dtrace -l | grep glassfish

This will show 151 or so built-in GlassFish probes.  Here is one line of the output:

39462 glassfish20383               web                        web-module webModuleStartedEvent
20383 is the Process ID of the GlassFish Server.  Whenever a web module is started the DTrace probe will be called with some arguments like the name of the web-module, etc.

The beauty of DTrace is that the DTrace probe inside of GlassFish is a no-op unless a DTrace script happens to be running.  In that case and in only that case the DTrace probe code gets dynamically instrumented to call into the OS's DTrace framework.  When the script ends, the probe gets un-instrumented.  There is almost no overhead unless a DTrace script is running.

There are limitless applications for this technology.  You can create on-the-fly scripts that attach to an already-running GlassFish server.  Everything can be enabled dynamically - the server never needs to be restarted for the magic to happen.  DTrace is so powerful it has its own programming language named "d".  It is beyond the scope of this blog to do more than chip at the surface of the DTrace engine.  

An example might be choosing a pair of bookend probes.  E.g. a start-transaction/end-transaction pair.  Or a HTTP service startRequest/endRequest pair.  You can easily have a DTrace script measure the time between the 2 calls and print out this information.  Since it is a programming language you can even save them into variables and crunch the data at the end of the script. 

Another example might be that you want to see every file that is opened - but only between a bookend pair (i.e. you are filtering the deluge of file openings to those that happen to occur during the time-slice of your bookend).

Create or download the app from [1].  Now add something more interesting to the webapp like, for instance, a form.  Now call a probe everytime the form is processed.  Pass a String the user enters in the form to the Probe.  That String will be available in real-time to a DTrace script!  This is a lot easier than it sounds.  I deployed the app (MonApp) from the blog[1] and then ran a super-simple DTrace script.  What the App does is call the probe method each time you refresh it.  wget is an easy way to do the refresh. It increments a counter and calls the probe with a string like so:  "Hello #XX".

I didn't even bother writing a script file - I just ran it from a command line:

dtrace -n 'fooblog*::: { printf("PID:%d APP:%s %s", pid, execname, copyinstr(arg0)); }'
dtrace: description 'fooblog*::: ' matched 1 probe
CPU     ID                    FUNCTION:NAME
  0  39554             ProbeServlet:myProbe PID:20383 APP:java Hello #22
  0  39554             ProbeServlet:myProbe PID:20383 APP:java Hello #23
  0  39554             ProbeServlet:myProbe PID:20383 APP:java Hello #24
  0  39554             ProbeServlet:myProbe PID:20383 APP:java Hello #25
  0  39554             ProbeServlet:myProbe PID:20383 APP:java Hello #26
  0  39554             ProbeServlet:myProbe PID:20383 APP:java Hello #27

I got one output line every time I refreshed MonApp in a browser.  This is incredibly powerful!  A string was sent from the user in a browser to a Java WebApp which then sent it off to a native program, DTrace, that was looking for it with a sophisticated scripting language.  All of this can be done without ever restarting the GlassFish Server.  There are truly endless possibilities.  And once you kill the script -- all the instrumentation code is removed.

 In case you are wondering about fooblog*:::  -- fooblog is the main name I gave it.  The * is needed because the Monitoring Framework automatically appends the PID of the server (very handy if you are DTracing with multiple GlassFish servers simultaneously).  The ::: means match ANY probe that has the main name that starts with fooblog.  Whew.  Hard to explain in English, easy to explain with one command:

~/gf_other/value-add/monitoring/dtrace/dtrace-samples/dtrace-scripts>dtrace -l | grep fooblog
39554 fooblog20383           samples                      ProbeServlet myProbe

To be completely specific in the earlier command would be:

dtrace -n 'fooblog20383:samples:ProbeServlet:myProbe { ..... }

Dtrace uses the quadruple of names to make the namespace less unwieldy and easier to filter.


 [1] My blog about adding Monitoring probes to a WebApp

Dtrace of Clusters in GlassFish - A Quick Tutorial

This is the second blog in the series about DTrace and GlassFish 3.1.1  You should read the first blog before this one.

Continuing where I left off in the previous blog I would like to analyze the usage of MonApp in 2  different instances.  Specifically I did this:

  1. undeploy from DAS
    asadmin undeploy MonApp
  2. create a cluster
    asadmin create-cluster c1
  3. create two clustered instances
    asadmin create-local-instance --cluster c1 i1
    asadmin create-local-instance --cluster c1 i2
  4. deploy to the cluster
    asadmin deploy MonApp.war
  5. start the instances
    asadmin start-cluster c1
  6. OPTIONAL - turn the levels up to HIGH for all Monitoring Modules as in the previous blog.  Just add the args --target c1
  7. turn-on DTrace support
    asadmin enable-monitoring --target c1 --dtrace
  8. Load both off the URLs in a browser in 2 different tabs:
  9. Verify DTrace "sees" the MonApp Probes
    dtrace -l | grep foo
    39785 fooblog20527           samples                      ProbeServlet myProbe
    39841 fooblog20544           samples                      ProbeServlet myProbe
  10. Run list-instances to get the instance pids so you can tell the probes apart
    asadmin list-instances --long
    i1    localhost  24848  20527  c1        running
    i2    localhost  24849  20544  c1        running
  11. If you had a load balancer setup (I don't) you could start hammering the App and see which instances are servicing the calls.  What I do is start the dtrace script:
    dtrace -n 'fooblog*::: { printf("PID:%d APP:%s %s", pid, execname, copyinstr(arg0)); }'
  12. Now I sit in the browser and randomly reload the 2 pages for a while and look at DTrace output:
    dtrace: description 'fooblog*::: ' matched 2 probes
    CPU     ID                    FUNCTION:NAME
      0  39785             ProbeServlet:myProbe PID:20527 APP:java Hello #3
      0  39785             ProbeServlet:myProbe PID:20527 APP:java Hello #4
      0  39785             ProbeServlet:myProbe PID:20527 APP:java Hello #5
      0  39841             ProbeServlet:myProbe PID:20544 APP:java Hello #3
      0  39841             ProbeServlet:myProbe PID:20544 APP:java Hello #4
      0  39841             ProbeServlet:myProbe PID:20544 APP:java Hello #5
      0  39785             ProbeServlet:myProbe PID:20527 APP:java Hello #6
      0  39841             ProbeServlet:myProbe PID:20544 APP:java Hello #6
      0  39785             ProbeServlet:myProbe PID:20527 APP:java Hello #7
      0  39785             ProbeServlet:myProbe PID:20527 APP:java Hello #8
      0  39841             ProbeServlet:myProbe PID:20544 APP:java Hello #7
      0  39841             ProbeServlet:myProbe PID:20544 APP:java Hello #8
      0  39841             ProbeServlet:myProbe PID:20544 APP:java Hello #9
      0  39841             ProbeServlet:myProbe PID:20544 APP:java Hello #10
      0  39785             ProbeServlet:myProbe PID:20527 APP:java Hello #9


Tuesday Jul 19, 2011

Tip #-1 How Does GlassFish Pick the JVM to Run In?

 If you start a GlassFish Server by using asadmin, then you may be interested in precisely how the JVM that will run GlassFish Server is chosen.  Note that if you restart via the Admin Console (GUI) you are indirectly using asadmin to restart the server and the following rules apply as well.  If you start the Server some other way, like

java -jar glassfish.jar

then you are in charge of choosing the JVM yourself.

In order of precedence from high to low:

  1. The jdk location specified in domain.xml in the java-config  ("java-home")
  2. The contents of AS_JAVA in the asenv property file (in glassfish/config)
  3. Whatever the env. variable, JAVA_HOME, is pointing at.
  4. The java.home system property is used like so:
    1. Parent directory of java.home -- this is because sometimes the JRE's java is asadmin's JVM and the JDK may be in the directory above.
    2. Contents of java.home

In every case I guarantee a valid java.  E.g. you can set 1,2,3 to garbage and we'll still find a java to use.

Note that we never explicitly use java from the path -- if there is one.  It may be implicitly used though by using #4 above since asadmin itself is definitely running inside of a valid JVM.

p.s.  If you are wondering where the code is -- 90% of it is inside
10% is in

-- AsenvPropertyReader can't access domain.xml.  GFLauncher can.  GFLauncher's job is to see if (1) above applies.  If not he defers to AsenvPropertyReader.


Wednesday Jun 15, 2011

Tip #1 Finding Source Files In a Gazillion Modules

 Tip #1  June 15, 2011

 GlassFish has a LOT of modules (463 pom.xml files) and a lot of java files (7810).  Here is an easy way to instantly locate source files:

    1. From the source root (Windows) run
      dir *.java /s/b >\gffiles.txt
    2. Create a script with this in it
      grep -i %1 \gffiles.txt
    3. Run the script and you'll see where the java file lives.
    4. Update once in a while -- or just wait until you don't find a match.


d:\gf\v3>gf411 counts

d:\gf\v3>grep -i counts d:\gffiles.txt

Note:  I finally did this after dozens (scores?  hundreds?) of times manually searching for files.  Don't waste your time!

Friday Apr 01, 2011

Test Driven Development in a Large Team

I had a problem to work out.  I know of plenty of wrong-headed things that Monitoring does.  I want to clean up this behavior for GlassFish 3.2  But I want to use the tenets of Test Driven Development.  Namely -- tests FIRST, implement SECOND.  Normally that would mean - add some tests now.  The tests will fail every time until we get around to fixing the code.  Eventually the tests will all start passing as we add in all the fixes.  Overtime we see lots of failures dropping to zero failures.

But the problem is that we can't do this is a multiple-developer project with continuous automated runs of the enormous suites of tests.  We can't add tests that will fail (at least not for more than just one cycle).  And I don't want to maintain yet another Hudson job that runs a subset of tests just for monitoring.  Plus it just doesn't feel right.  The tests ought to pass ALL THE TIME.

An excellent solution was provided by Carla Mott.  Simple and elegant:

1) Create a test for what you want the new behavior to be.
2) Now set the results of that test to the converse of the actual results -- i.e. if they pass, report as a failure and vice versa

The tests will pass every time until you fix the issue.  Then it will fail every time until you fix the test. 

We can add lots of these tests for new features, now.  When we implement these fixes when we are under the steaming pressure head of deadlines and milestones we won't have to 'remember' to add Dev Tests.  They are already done and will break like popcorn popping as we complete tasks.  It's impossible to not have working devtests upon completion.

Wednesday Mar 23, 2011

CLI Logfile and DevTests

Recently a colleague added a test to the Admin Development Tests.  This is a huge suite of tests that we developers use as a sanity test that exercise the administration of a running GlassFish server.

here is the symptom of the problem -

 The test failed on her machine.
The test failed on my machine
The test succeeded on the Hudson build that runs it automatically -- it passed 100% of the time

I finally tracked this problem down today with the help of an invaluable tool for asadmin.  I always set this variable in my environment:


When this is set -- every asadmin command is written into the logfile.  It includes the full command with all arguments, the time-stamp and the return value.

I quickly figured out the problem by looking in this file on my platform, Windows, and on the Hudson platform, Linux.  Here it is:

Command inside the test itself:  asadmin list -m \*

Command inside cli.log

03/22/2011 21:55:27 EXIT: 1 asadmin --user admin --passwordfile D:\\gf\\v2\\appserv-tests/config/adminpassword.txt --host localhost --port 4848 --echo=true --terse=true list -m .svn apps build build.xml byron derby.log dome.bat foo foo2 foo2.bat mon.bat nbproject pom.xml q qqq resources seconddb src 

OOPS!!  The star is special in Windows calling the asadmin script -- even from Runtime.exec().  The star was replaced with the names of every file that happened to be sitting in the directory that the tests are running in.

 Solution:  In the code use "\\"\*\\"" for Windows and '\*' otherwise

Sunday Feb 27, 2011

Automatic Starting Implementation Details for GlassFish 3.1

Process Nuts and Bolts

When you start a GlassFish 3.1 Server Service notice that there are 2 JVM processes running.  The 2 process are these:

  1. The GlassFish Server
  2. A JVM running asadmin start-[local-instance|domain] --verbose

Why?  We want to leverage the start command.  Asadmin does a great deal of work for you when you start a server.  The end-product of that work is mainly (but not only) the generation of a big complex JVM invocation command to start the server with.  I could have implemented the Services to call asadmin and have asadmin exit.  But then the Platform would see that asadmin exited and would report the service as "stopped".  It would not know in the future when the server really stops. 

When you start a server with the --verbose flag then the asadmin process waits for the server to die before exiting.  When the server stops -- asadmin stops and the Platform will update the state to "stopped".

Note that restart-[domain|instance] will still work perfectly.  In that case the server process dies but the asadmin process does not - not even for a moment.  The platform will not "see" anything happening.


I discovered that there seems to be a special way to create services for every different flavor of Linux.  And they are all different from each other.  Basically all of these different ways end up with writing out special files into a special area.  This one common denominator is to use the ancient UNIX way of setting up services which is, not surprisingly, editing lots of configuration files in a special area.  Even the exact details on the special area is different among the different flavors.  The code is flexible and will handle all known Linux flavors. The script can be found in /etc/init.d and links to it in the runlevel directories:  /etc/init.d/rc?.d

Monitoring Data -- How To Open The Firehose

Are you interested in getting raw Monitoring Data without using JavaScript or Dtrace?  That's why we include the special command:

asadmin get -m

Looks can be deceiving.  It appears that the -m is just some command line option.  Not so.  That turns on a huge pile of code that plows through the run-time tree of monitoring data.  This data can be accessed from instances, clusters of instances and DAS -- in fact ALL the data from all of them can be combined and accessed all at the same time!

This command was available in V3.0  Unfortunately it was very slow if you wanted to collect everything (what I call Firehose Mode).  So what was done was a lot of regular expression parsing to narrow down the matching data in order to get things going faster.

I reworked this command for GlassFish 3.1 to support the clustering feature.  I started DAS (Domain Administration Server) and one instance.  I ran

asadmin get -m "\*"  // Firehose output for DAS and the instance

Then I waited.  And waited.  And waited and waited.  About 90 seconds later the server got an out-of-memory error.  I never saw any output at all.  Oops!

I discovered the performance bottle-neck.  It was unbelievable what the problem is.  If you are interested I've blogged about it here.

We spent a lot of time trying to get the get -m command to handle regular expressions well.  Below are some examples. 

My environment: I have a cluster, c1.  c1 has 2 instances, i1 and i2.  Both instances are running with monitoring-levels all set to HIGH.  DAS is also running.  All servers are running on the same machine so I don't muddy the water with network latency etc.

  • asadmin get -m c1 # this retrieves ALL monitoring data for every running instance in c1 (~14,000 lines of output)
  • asadmin get -m c1.\* # does exactly the same thing as above
  • asadmin get -m i2.web.servlet.activeservletsloadedcount-current  # Example drill-down.
    i2.web.servlet.activeservletsloadedcount-current = 6
  • asadmin get -m "\*" | wc -l # returned 20,000+ lines.  It took around 1 second on my laptop.  
  • asadmin get -m \*.web.servlet.activeservletsloadedcount-current
    server.web.servlet.activeservletsloadedcount-current = 2  #DAS
    i1.web.servlet.activeservletsloadedcount-current = 6  
    i2.web.servlet.activeservletsloadedcount-current = 6

The command is so incredibly fast you can simply collect everything in a file and then run text processing commands on that file and avoid calling the command multiple times:

asadmin get -m "\*" > output

grep hello output # just an example

Automatic Starting of Servers in GlassFish 3.1

Note: At the end of this blog are links to older blogs from V3.0 about this feature.  You may want to look them over as well.

The way we offer the ability to automatically start GlassFish servers is via "services" on the platform.  "Services" are an ancient technology, available on all of our supported platforms (and I would imagine is available on every serious OS).  Services allow applications to run automatically when a computer boots up.  Nobody needs to login -- the service just starts automatically.  A simple example of this is the FTP server.  When you boot up your computer you can access FTP from any other computer without touching the FTP computer.  This is very important in the event of a power failure.  On Windows you can setup automatic security updates and Windows will automatically reboot your computer.  If you have a Subversion server that is not configured as a service and you are halfway around the world depending on it -- you will become a Services Specialist in the near future when you return home!

GlassFish 3.1 Platform Services now supports all Linux versions, Windows and Solaris10/SMF

Services can be quite complex.  For instance you might want to have it do this:

  1. Automatically start a GlassFish server upon booting
  2. If the server crashes (no way!) restart it
    What if it crashes every time?  The machine will be in trouble with an infinite loop so:
  3. Try to restart 3 times and then give up.

Setting up this level of services-granularity is very difficult and expensive to implement perfectly.  For one - we are supporting several platforms and they all do these things completely differently. The hard part is setting up services at all to do the basic things.  Once that is complete you can easily use the platform's native tools for adjusting things just the way you like.  What we support is the classic services model -- which is to start the service upon booting, unattended.

I worked hard to make it easy to setup services.  Here's how you would create them for an installation that has one Domain:

asadmin create-service

If you want to make sure it would work but you aren't yet ready to pull the trigger, or you just want to see what it would do -- then run this command:

asadmin create-service --dry-run

If your domain is named, say. domain1 then the service's name will be domain1.  Simple.

 After the service is created you are greeted with a platform-specific message giving you the details on how to start the service.  We do NOT automatically start it.  I personally recommend rebooting immediately -- and it will start.

Windows is particularly easy and flexible to work with.  You can manage the GlassFish service(s) by:

  1. GUI -- right-click on Computer/Manage/Services
  2. sc.exe  very handy tool for querying/starting/stopping/restarting etc.
  3. In the bin directory of the domain or instance run the your-service-nameService.exe application that we put in there. 

How to setup automatic starting for an instance?

Easy!  Simply give the name of the instance as the final argument to create-service.  The following command will create a service for myinstance:

asadmin create-service myinstance

There is an undocumented, unsupported command that will delete a service for you.  It is very very easy to delete a service on SMF and Windows.  On Linux it can be tedious.  I've personally tested the command successfully but it has not been through the merciless testing of QA at Oracle.  Use it at your own risk.

asadmin _delete-service 

Important blog for making sure GlassFish doesn't stop when you logout.

I have another blog with implementation details if you are interested.

Some older blogs that pertain to V3.0:

V3.0 Platform Services for Windows and SMF

DIY Cookbook for Configuring Linux Services

How to Run on Linux as a Non-Root User

A Taste of Monitoring in GlassFish 3.1

Ok, so you've installed the Oracle GlassFish Server that you got from here.  You've created a cluster with a couple of instances.  The instances seem to be getting a bunch of traffic.  At least there are a lot of flashing lights on the ethernet cards and disk light is getting hammered.  But how can you find out what's really going on?  

That's the whole point of Monitoring.  We have provided a rich platform for getting Monitoring information in GlassFish.  You can ask for a trickle by zeroing-in on exactly the scraps of data you are interested in (e.g. please tell me every time the web-container creates a new thread) or turning the fire hose on and getting every monitored-event as it happens.

How do you want to see the data?

  •  In a browser? -- no problem, use either the Admin GUI or REST.
  • You are running in Solaris 10+ and you want to use DTrace?  We support it -- all monitorable events can be seen via native DTrace.
  • You want all of your Applications to get added into this cool Monitoring framework so you can access the data just like the built-in GlassFish Monitoring data?  No sweat, we have provided an easy way to add Monitoring probes to your applications.
  • Not good enough?  Too generic?  You want to be able to collect the data dynamically and structure it some other way using a JavaScript program?  Again, no problem.  You can run a JavaScript program, access as much or as little data as you want, dynamically,  and go crazy massaging the data any way you like.

There are a lot of facets to Monitoring in GlassFish 3.1  We will present many blogs with the details to get you going.  As those blogs are completed I'll circle back here and add the links.  In the meantime -- here is a very simple example of a "What-If" question and how to get the answers quickly.

How many classes have been loaded by the Domain Administration Server's JVM and what happens with the count when an app is deployed and then accessed?

asadmin get -m server.jvm.class-loading-system.totalloadedclass-count-count
server.jvm.class-loading-system.totalloadedclass-count-count = 3887
asadmin deploy hello.war
Application deployed with name hello.
asadmin get -m server.jvm.class-loading-system.totalloadedclass-count-count
server.jvm.class-loading-system.totalloadedclass-count-count = 5979
wget http://localhost:8080/hello
asadmin get -m server.jvm.class-loading-system.totalloadedclass-count-count
server.jvm.class-loading-system.totalloadedclass-count-count = 6816

Wow!  That's a lot of classes.  Probably the jump from 3887 to 5979 was loading the web-container (this server had no apps deployed yet) and the next big pile of classes were jsp and servlet classes.  Let's do a sanity check by deploying another simple app:

asadmin get -m server.jvm.class-loading-system.totalloadedclass-count-count
server.jvm.class-loading-system.totalloadedclass-count-count = 6819
asadmin deploy Math.war
Application deployed with name Math.
asadmin get -m server.jvm.class-loading-system.totalloadedclass-count-count
server.jvm.class-loading-system.totalloadedclass-count-count = 6902

wget http://localhost:8080/Math
asadmin get -m server.jvm.class-loading-system.totalloadedclass-count-count
server.jvm.class-loading-system.totalloadedclass-count-count = 6906

That's semi-interesting.  Deploying a simple web module (with a servlet and jsp)  results in 83 classes getting loaded.  Running the app results in 4 classes being loaded.

Monitoring Blogfest

Running Monitoring JavaScript Programs on Clusters and Instances

Huge Performance Improvement

I was working on improving the performance and memory consumption in Monitoring and I bumped into one problem that caused gigantic performance degradation.  I'm talking about a 100 fold speed-up for large (5000-20000) data sets.

 The code was adding Strings into a data structure.  The way it was generating the Strings was like this:


String s;  // we are actually all done now.  "s" is fine

s = MessageFormat.format("{0}", s);

 I simply got rid of that line and everything was fixed!

Moral:  Be careful with MessageFormat




« July 2016