Tuesday Mar 15, 2011

JDK 7 nearing the endgame

Just a reminder that JDK 7 is nearing the endgame[Read More]

Wednesday May 13, 2009

Monitoring direct buffers

One of the smaller features that JSR-203/NIO2 brings to JDK 7 is a management interface for monitoring the resources associated with pools of buffers. "Buffers" here means direct buffers that are allocated outside of the garbage-collected heap (by ByteBuffer.allocateDirect), and mapped buffers, created by mapping a region of a file into memory (using FileChannel.map). The management interfaces are registered with the platform MBeanServer and exposed to tools that connect to the JMX agent. This means that tools such as jconsole and VisualVM can effectively monitor the usage of these buffers. The following is a partial screen-shot of VisualVM doing just that:

The management interface is java.nio.BufferPoolMXBean and the screen-shot is of the MBean browser plugin looking at the attributes of a buffer pool named "direct", the pool for direct buffers. In this case there are 31831 direct buffers with a total capacity of 4074368 bytes. The application in this example is a rogue test called DirectBufferGCKiller2 that continuously allocates, and discards, tiny direct buffers. Due to page alignment, the memory usage is significant higher than the total capacity of the buffers; this application could be a lot more efficient if it allocated its tiny buffers by slicing a large buffer. As an aside, page alignment is something we hope to eliminate in JDK 7 (this is a topic for another day).

For those that prefer command-line tools then it is easy to hack up a tool to monitor the usage over time. MonBuffers.java is one example. It uses the Attach API to attach to a running VM and print usage information like this:

$ java -classpath .:$JAVA_HOME/lib/tools.jar MonBuffers 18051
           direct                        mapped
 Count   Capacity     Memory   Count   Capacity     Memory
 26346    3372288  219198720       0          0          0
 29296    3749888  243742720       0          0          0
 32382    4144896  269418240       0          0          0
 36957    4730496  307482240       0          0          0
 40032    5124096  333066240       0          0          0
 44848    5742336  373251840       0          0          0
 51664    6612992  429844480       0          0          0
 58698    7513344  488367360       0          0          0

The columns with "0" relate to "mapped" buffers which aren't used by this test. With a bit of effort it would be possible to develop a much slicker tool and maybe a plugin for VisualVM (hint hint).

In addition to monitoring buffer usage in a running system it is also important to be able to examine the usage postmortem (say after a crash due to exhaustion of the native heap). In this case it is easy to hack up a quick tool that uses the HotSpot Serviceability Agent to obtain this information. It might be useful to add an option to jmap to do just that.

Update (June 23, 2009). Tomas Hurka, has announced a VisualVM plugin to monitor buffer pools. It's available now at the plugin center.

Monday May 04, 2009

Copy that

Copy.java is one of several new samples included in JDK 7 to demonstrate its new API to the file system. If you have any recent snapshot installed then you'll find these samples in the $JAVA_HOME/sample/nio/file directory. Copy.java works like the Unix cp program to copy a file or file tree to a target location. It supports the -r, -p and -i options to do a recursive copy, preserve attributes, or work interactively to prompt whenever an existing file would be overwritten. The sample code is relatively simple and demonstrates many aspects of the API that are worth looking at.

At its heart, the copyTo method is used to do the copy. The Copy sample uses the COPY_ATTRIBUTES option when invoked with -p to preserve attributes. This is useful when you want the file attributes/meta-data copied to the target file. This means the file timestamps, permissions, Access Control List, extended attributes, etc. The other option used in the sample code is REPLACE_EXISTING; this is used to replace the target file if it exists. One thing to point out is that the copyTo method doesn't copy the contents of a directory. If you invoke it on a directory then it creates an empty directory. On the surface this might appearing limiting but work through the sample and it should becomes clear.

Another interesting thing to point out is that you'll see code like:


The relativize method used to compute a relative path between source and file. This is then resolved against target. For example, suppose we are doing a recusive copy from /users/gus to /users/joe/backup_of_gus. As we walk the tree we will need to copy /users/gus/stamps/rare/black_penny.html. In this sample source.relativize(file) yields the relative path stamps/rare/black_penny.html. Resolving this against the target directory (/users/joe/backup_of_gus) yields us the path in the target tree: /users/joe/backup_of_gus/stamps/rare/black_penny.html.

The other important API used in the sample is the Files.walkFileTree method. This method is used to implement recursive file operations, starting at a given starting point. The method is passed a FileVisitor that is invoked for each file or directory in the file tree. The Copy sample has an inner class TreeCopier that implements FileVisitor . TreeCopier's preVisitDirectory method is invoked for each directory before the entries in the directory are visited. It simply invokes the copyTo method to copy the directory (creating an empty directory as I mentioned above). When the copy succeeds or the directory already exists the preVisitDirectory returns CONTINUE to instruct the iterator to continue and visit the files in the directory. If the copy fails, then preVisitDirectory returns SKIP_SUBTREE to tell the iterator to skip the directory. You'll see that postVisitDirectory is also implemented. This is invoked after all of the entries in the directory have been visited. In this code it is implemented so that the directory's last-modified-time can be fixed up after we are done with the directory. Most recursive operations will only need to implement one of preVisitDirectory or postVisitDirectory. The visitFile method is very simple and just copies the file to the corresponding file in the target tree. One interesting thing to point out is that visitFile is invoked if a cycle is detected (a cycle being a symbolic link to a parent directory). Copying is one of the few cases where you want to follow symbolic links and so you need to be concerned with the possibility of a cycle.

Hopefully this sample will be useful when learning the new API. The $JAVA_HOME/sample/nio/file directory has a number of other examples that demonstrate other parts of the API.




Top Tags
« June 2016

No bookmarks in folder