Tuesday Mar 15, 2011
Wednesday May 13, 2009
By user12820862 on May 13, 2009
One of the smaller features that JSR-203/NIO2 brings to JDK 7 is a management interface for monitoring the resources associated with pools of buffers. "Buffers" here means direct buffers that are allocated outside of the garbage-collected heap (by ByteBuffer.allocateDirect), and mapped buffers, created by mapping a region of a file into memory (using FileChannel.map). The management interfaces are registered with the platform MBeanServer and exposed to tools that connect to the JMX agent. This means that tools such as jconsole and VisualVM can effectively monitor the usage of these buffers. The following is a partial screen-shot of VisualVM doing just that:
The management interface is java.nio.BufferPoolMXBean and the screen-shot is of the MBean browser plugin looking at the attributes of a buffer pool named "direct", the pool for direct buffers. In this case there are 31831 direct buffers with a total capacity of 4074368 bytes. The application in this example is a rogue test called DirectBufferGCKiller2 that continuously allocates, and discards, tiny direct buffers. Due to page alignment, the memory usage is significant higher than the total capacity of the buffers; this application could be a lot more efficient if it allocated its tiny buffers by slicing a large buffer. As an aside, page alignment is something we hope to eliminate in JDK 7 (this is a topic for another day).
For those that prefer command-line tools then it is easy to hack up a tool to monitor the usage over time. MonBuffers.java is one example. It uses the Attach API to attach to a running VM and print usage information like this:
$ java -classpath .:$JAVA_HOME/lib/tools.jar MonBuffers 18051 direct mapped Count Capacity Memory Count Capacity Memory 26346 3372288 219198720 0 0 0 29296 3749888 243742720 0 0 0 32382 4144896 269418240 0 0 0 36957 4730496 307482240 0 0 0 40032 5124096 333066240 0 0 0 44848 5742336 373251840 0 0 0 51664 6612992 429844480 0 0 0 58698 7513344 488367360 0 0 0
The columns with "0" relate to "mapped" buffers which aren't used by this test. With a bit of effort it would be possible to develop a much slicker tool and maybe a plugin for VisualVM (hint hint).
In addition to monitoring buffer usage in a running system it is also important to be able to examine the usage postmortem (say after a crash due to exhaustion of the native heap). In this case it is easy to hack up a quick tool that uses the HotSpot Serviceability Agent to obtain this information. It might be useful to add an option to jmap to do just that.
Update (June 23, 2009). Tomas Hurka, has announced a VisualVM plugin to monitor buffer pools. It's available now at the plugin center.
Monday May 04, 2009
By user12820862 on May 04, 2009
At its heart, the copyTo method is used to do the copy. The Copy sample uses the COPY_ATTRIBUTES option when invoked with -p to preserve attributes. This is useful when you want the file attributes/meta-data copied to the target file. This means the file timestamps, permissions, Access Control List, extended attributes, etc. The other option used in the sample code is REPLACE_EXISTING; this is used to replace the target file if it exists. One thing to point out is that the copyTo method doesn't copy the contents of a directory. If you invoke it on a directory then it creates an empty directory. On the surface this might appearing limiting but work through the sample and it should becomes clear.
Another interesting thing to point out is that you'll see code like:
The relativize method used to compute a relative path between source and file. This is then resolved against target. For example, suppose we are doing a recusive copy from /users/gus to /users/joe/backup_of_gus. As we walk the tree we will need to copy /users/gus/stamps/rare/black_penny.html. In this sample source.relativize(file) yields the relative path stamps/rare/black_penny.html. Resolving this against the target directory (/users/joe/backup_of_gus) yields us the path in the target tree: /users/joe/backup_of_gus/stamps/rare/black_penny.html.
The other important API used in the sample is the Files.walkFileTree method. This method is used to implement recursive file operations, starting at a given starting point. The method is passed a FileVisitor that is invoked for each file or directory in the file tree. The Copy sample has an inner class TreeCopier that implements FileVisitor . TreeCopier's preVisitDirectory method is invoked for each directory before the entries in the directory are visited. It simply invokes the copyTo method to copy the directory (creating an empty directory as I mentioned above). When the copy succeeds or the directory already exists the preVisitDirectory returns CONTINUE to instruct the iterator to continue and visit the files in the directory. If the copy fails, then preVisitDirectory returns SKIP_SUBTREE to tell the iterator to skip the directory. You'll see that postVisitDirectory is also implemented. This is invoked after all of the entries in the directory have been visited. In this code it is implemented so that the directory's last-modified-time can be fixed up after we are done with the directory. Most recursive operations will only need to implement one of preVisitDirectory or postVisitDirectory. The visitFile method is very simple and just copies the file to the corresponding file in the target tree. One interesting thing to point out is that visitFile is invoked if a cycle is detected (a cycle being a symbolic link to a parent directory). Copying is one of the few cases where you want to follow symbolic links and so you need to be concerned with the possibility of a cycle.
Hopefully this sample will be useful when learning the new API. The $JAVA_HOME/sample/nio/file directory has a number of other examples that demonstrate other parts of the API.