Tuesday Jan 13, 2009

New Command-Line Options in VisualVM 1.1

Recently released VisualVM 1.1 tool introduces three new command-line options: --openfile, --openpid and --openjmx. These options allow to use VisualVM as an application or file viewer. This can be handy especially for viewing heap dumps and thread dumps or for integration with third-party tools.

A nice feature of VisualVM command-line support is the ability to reuse already running VisualVM instance. If you request to open a file for the first time, VisualVM is launched and the file is opened. If you request to open another file, it will be opened in the tool which is already running. This is possible thanks to the NetBeans Command Line Parsing API used by this feature.

The --openfile option

syntax: visualvm --openfile [filename] where filename is full path to a supported file

allows to open a supported file in VisualVM. Supported are the same file types as in the Load dialog in VisualVM GUI - by default thread dumps (.tdump), heap dumps (.hprof) and profiler snapshots (.nps) are supported. Since the heap dumps are recognized by the file header and not just according to their extension, also .bin and other heap dumps can be opened by VisualVM. If a third-party tool adds support for another file type, it will be automatically available also from the command-line.

The --openpid option

syntax: visualvm --openpid [pid] where pid is process id of running Java application

allows to open local Java process in VisualVM. This is an alternative to the Open action in VisualVM GUI. Note that there's a predefined timeout 5sec for which the VisualVM waits for the PID to become available - this allows to start VisualVM along with the monitored application. The --openpid option may be used for example in batch commands for monitoring several predefined applications (application servers etc.) or for integration with third-party tools (like the integration module for the Eclipse IDE).

The --openjmx option

syntax: visualvm --openjmx [hostname:port] where hostname:port defines the JMX connection

allows to connect to local or remote Java process using the JMX connection. This is an alternative to the Add JMX Connection action in VisualVM GUI. JMX connection defined from the command-line is not restored after restarting VisualVM. The --openjmx option may be used for example in batch commands for monitoring several predefined applications (application servers etc.) or just like a simple remote application monitor/viewer.

Tuesday Dec 16, 2008

Profiler Update in Patch1 for NetBeans IDE 6.5

The recently released Patch 1 for the NetBeans IDE 6.5 delivers several important profiler bugfixes. If you are using the NetBeans profiler, we strongly recommend you to apply this update.

Monday Dec 08, 2008

VisualVM 1.1 Released

New version of the VisualVM tool has been released. VisualVM 1.1 introduces many new features and improvements, extends the API for plugins and delivers a complete JavaDoc documentation. Now integrates with the Eclipse IDE and IntelliJ IDEA!

  

These are the new features of VisualVM 1.1 as mentioned in the Release Notes:

  • Monitoring CPU usage and Garbage Collector activity for each application in the Monitor tab
  • Table view in the Threads tab (introduced by the NetBeans profiler 6.5)
  • Three commandline options to enable using VisualVM as an application or snapshot viewer:
    • --openpid <pid> starts VisualVM if not already running and opens a Java application with the process id
    • --openjmx <hostname:port> starts VisualVM if not already running and opens a Java application defined by a JMX connection
    • --openfile <file> starts VisualVM if not already running and opens a supported file (\*.tdump, \*.hprof, \*.nps, \*.apps)
  • Compare Memory Snapshots action available in Applications window context menu for two selected comparable snapshots
  • About dialog allows to copy configuration information to clipboard and save the logfile to an external file
  • IBM JVM can be monitored by VisualVM using a JMX connection
  • Eclipse integration plugin which starts VisualVM along with the monitored application directly from the IDE
  • Integration with IntelliJ IDEA is already available, see the Profiler Plugin by Esko Luontola
  • Experimental support for HP-UX PA-RISC platform (incl. profiling)

The last significant change is using the latest NetBeans Platform and profiler 6.5 - this means many framework and profiler bugfixes being available also in VisualVM.

VisualVM 1.1 can be downloaded at https://visualvm.dev.java.net. Feedback to VisualVM developers can be sent using this mailing list. The online JavaDoc documentation can be found here. Note that there's also a Releases Overview page available which lists all VisualVM releases and shows which VisualVM version is included in JDK as Java VisualVM.

Tuesday Dec 02, 2008

JDK 6 Update 11 Comes With Updated Java VisualVM

The JDK 6 Update 11 has been released today, it contains an updated version of the Java VisualVM tool based on the most recent VisualVM 1.0.1. If you are using the Java VisualVM tool we suggest you to get the latest JDK bits at http://java.sun.com/javase/downloads/index.jsp and update your installations. This way you will get the latest VisualVM bugfixes and improvements.

Monday Nov 24, 2008

Developer.com Product of the Year 2009: Give Us Your Vote!

The Product of the Year 2009 awards are in progress at Developer.com, you can vote till Monday, December 15, 2008. If you like the NetBeans profiler or VisualVM tool, please give us your vote in the Development Utility of the Year category!

Wednesday Nov 19, 2008

NetBeans IDE 6.5 Released

Today the NetBeans IDE 6.5 has been released at http://www.netbeans.org. In this version the profiler introduces some new features, the most significant are:

  • Profiler remote packs are generated by the Attach Wizard, no additional downloads are needed for remote profiling
  • HeapWalker tool got significant performance improvements and some usability tweaks
  • Table view has been added to the threads monitor

In the 6.5 release the profiler also adds an experimental support for HP-UX (PA-RISC 32bit, 64bit) and Phone ME Advanced (Windows x86, Linux x86) platforms. Also, almost 100 bugs have been fixed to deliver the best quality product.

A good starting point for new profiler users is the Introduction to Profiling Java Applications in NetBeans IDE document. Experienced profiler users will appreciate the new document describing Profiling Points feature: Using Profiling Points in NetBeans IDE.

To get a general information about the new NetBeans IDE release see the NetBeans IDE 6.5 Release Notes. To get detailed information about new profiler features and bugfixes see the NetBeans profiler 6.5 Release Notes.

Friday Oct 31, 2008

VisualVM in Ubuntu 8.10

The new version of Ubuntu (8.10 aka Intrepid Ibex) was released yesterday. Because I heard of many improvements in this version I immediately got a copy and installed it. The overall impression is awesome! And the best of all - VisualVM is available as an Ubuntu package right from the Canonical repositories.

Just open the Synaptic Manager, type "visualvm" in the quick-search box (btw, this is the best thing since sliced bread ;) and you can install VisualVM with all the dependencies. The download size is quite hefty (cca. 300MB) since it downloads the whole open-jdk, mysql and i-dont-know-what-else but in the end you have full-blown java development environment with the bonus of VisualVM nicely registered under "Applications/Programming/VisualVM" menu shortcut.

What is left now is to port some Ubuntu specific monitoring stuff to VisualVM plugins. Any volunteers? :)

Friday Sep 05, 2008

VisualVM 1.0.1 Released

VisualVM 1.0.1 tool has been released at https://visualvm.dev.java.net. This is mostly a bugfix release based on the VisualVM 1.0.

According to the Release Notes document these changes have been made:

  • Profiler supports OpenJDK
  • Profiler is silently disabled on unsupported platforms, the rest of the tool can run unaffected
  • Minor improvements of warning messages in the tool and plugins to better describe reported problems
  • New plugins available on the Plugin center: JvmCapabilities, KillApplication, SysTray

To get more information about the tool, download it and start using it, visit the project pages at https://visualvm.dev.java.net. Information about the new plugins can be found on the Plugins page. If you want to provide some feedback to VisualVM developers, let them know on a mailing list.

Monday Aug 18, 2008

Hg Repository and Continuous Build

Like all other NetBeans development teams, Profiler team has to use mercurial (hg) from February. Since the traffic and number of broken builds in main NetBeans repository is alarming, we asked our release engineering to set up separate NetBeans Profiler mercurial repository called profiler-main and continuous build. We update profiler-main once a day from main-golden. Main-golden is updated from main only if it can be built successfully and the basic set of tests passed. This way we can work on relatively stable build of NetBeans. When fixing bugs we first push changeset to profiler-main and corresponding bug is closed as FIXED in IssueZilla, than the fix is propagated to main and from main finally to main-golden. Corresponding issue is automatically updated with information about public build. If you don't want to wait for fix to be propagated to main-golden you can download NetBeans with fix from our continuous build.

Wednesday Aug 13, 2008

NetBeans IDE 6.5 Beta Released

The NetBeans IDE 6.5 Beta has been released, read the release information or see the list of all new features. The bits can be downloaded from this link.

The profiler introduces three new features since 6.5 Milestone 1:

  • Table view for threads: new tab in threads monitor displaying all threads of profiled application in a table
  • HeapWalker improvements: significantly improved performance of resolving references to an instance, improved readability of path to GC root
  • Experimental support for HP-UX: profiling should work on HP-UX PA-RISC 32bit and 64bit systems

Also, many bugs have been fixed, you can display the list of issues by clicking this Issue Tracker query.

Thursday Jul 31, 2008

Profiling With VisualVM, Part 2

This article describes advanced profiling features of the VisualVM tool which give the user more control over the profiling session and collected data. You may also want to read the Profiling With VisualVM, Part 1 article describing the basic profiling features of VisualVM.

By default, the Profiler UI in VisualVM doesn't display any settings, the initial configuration ensures that even an unexperienced user will get some useful results with zero configuration by just clicking the CPU or Memory button. However, an advanced user still has the possibility to fine-tune the settings and fully utilize the power of the NetBeans profiler engine.

To display the settings area, Settings checkbox has to be selected in the topright corner of Profiler tab. The settings area consists of two tabs, CPU settings and Memory settings, containing controls for each profiling mode.

Note that the settings can only be changed when the profiling session is not in progress, during profiling the settings area is disabled.

There's also a Restore Defaults button available at the bottom of the area which restores the initial configuration. This is useful when the values were customized incorrectly and no profiling data are being collected.

Profiling Performance

For performance profiling there are two important settings available: profiling roots and instrumentation filter. In general, profiling roots define when the profiler should start collecting the data whereas instrumentation filter defines packages/classes from which the data should/shouldn't be collected.

Start profiling from classes textbox defines the profiling roots. If all the methods were instrumented, the profiling overhead would be enormous and profiling would eventually become unusable. That's why the NetBeans profiler introduces a notion of Root Methods - a way to profile just the code of interest while the rest of the application runs at full speed.

A typical usecase is for example profiling performance of an action invoked by a button click - if the appropriate actionPerformed() method is set as a root method, the profiler starts instrumenting and profiling the application once the actionPerformed() method is invoked and only profiles the methods called from the actionPerformed(). This way the profiling overhead is very small and only relevant profiling results are collected which greatly improves their readability.

Whereas in NetBeans profiler selecting a root method is very natural because of its tight integration with the editor, in VisualVM this might be a bit tricky and not exactly user friendly. That's why VisualVM defines profiling roots by classes - all methods defined in these classes and their innerclasses become root methods. In the similar way, package names can be used as profiling roots. That's how the default profiling roots are defined: <main_class_package>.** which means that all methods from <main_class_package> and subpackages are profiled.

The default settings work in most cases, but for large applications like application servers it might be useful to tweak the profiling roots to point to a concrete code of interest. For example, when profiling a web application it's much better to set the servlet's package or classname as a profiling root instead of profiling the whole server.

The format for entering the profiling roots in VisualVM is following:

  • org.mypackage.** defines all methods in org.mypackage and all subpackages as root methods
  • org.mypackage.* defines all methods in org.mypackage as root methods
  • org.mypackage.MyClass defines all methods in org.mypackage.MyClass and its innerclasses as root methods
  • empty textbox means that all methods executed in the application will be profiled

Multiple profiling roots (separated by a space and/or comma and/or newline) can be defined for a single profiling session.

Note that it doesn't make sense to define org.mypackage.MyClass.* or org.mypackage.MyClass.** as profiling roots since the tool always selects all methods of the class and its innerclasses as profiling roots.

Profile only classes / Do not profile classes textbox defines the instrumentation filter. This is exactly the same option as Quick Filter in the NetBeans profiler. It either defines the only packages/classes to be profiled or packages/classes not to be profiled. For example, if only an application's code should be profiled without any outgoing calls to other frameworks or JDK code, this option should be set to Profile only classes, <application_package>.*. If all the code except JDK classes from java and javax packages should be profiled, this option should be set to Do not profile classes, java.*, javax.*. The default instrumentation filter is Do not profile classes, java.*, javax.*, sun.*, sunw.*, com.sun.* (+ com.apple.*, apple.awt.* on Mac OS X) which filters out all the JDK code.

The format for entering instrumentation filter is following:

  • org.mypackage.* selects all classes from org.mypackage and subpackages to be profiled or to be excluded
  • org.mypackage. selects all classes from org.mypackage to be profiled or to be excluded
  • empty textbox selects all classes of the application to be profiled

Multiple filter values (separated by a space and/or comma and/or newline) can be defined for a single profiling session.

Profiling Memory

For memory profiling there are three important settings available: profiling scope, density of tracked objects and recording allocations stack traces.

The profiling scope is defined by one of two choices: for the Profile object allocations option only object allocations are monitored. This is useful when evaluating how many objects are allocated by the application. For checking and identifying memory leaks the second choice is more useful - Profile object allocations and GC. In this mode the allocated objects are monitored during the whole lifecycle, from their allocation till garbage collection. This allows to easily identify live objects currently allocated on the heap at certain point. Also, in this mode the Surviving Generations metrics is available (see this article for more information). By default, Profile object allocations and GC option is selected.

The Track every Nth object allocations option allows to define density of tracked objects. In most applications it isn't necessary to monitor each single instance, the overall allocations behavior is the same when tracking just each Nth object. The advantage of this approach is lower profiling overhead. To get exact number of allocated or live objects, this option has to be set to 1. By default, each 10th object is tracked.

The third control is the Record allocations stack traces checkbox. When selected, a stack trace is taken for each monitored object when allocated. This allows to identify the concrete method (and its call tree) which has allocated the object. By default the collecting of allocation stack traces is disabled to lower the profiling overhead.

Note that by default some columns of memory results are hidden. You can display more metrics using the top-right corner button in Profiling results or memory snapshot tables.

Comparison With The NetBeans Profiler

This section compares profiling settings available in the NetBeans profiler and VisualVM. For the settings which cannot be customized in VisualVM the predefined value is listed.

CPU profiling settings:

Settings NetBeans profiler VisualVM
Profiling roots packages/classes/methods/project methods packages/classes
Instrumentation filter predefined sets/user defined/project code user defined
Profiling points enabled/disabled N/A
Profiling technique instrumentation only/instrumentation & sampled time instrumentation only
Exclude Thread.sleep() & Object.wait() time on/off always on
Profile underlying framework startup on/off always off
Profiling technique instrumentation only/instrumentation & sampled time instrumentation only
Profile new Threads/Runnables on/off on/off
Profiled threads limit 1 to unlimited always 32
Thread CPU timer (Solaris only) on/off always off
Instrumentation scheme total/eager/lazy always lazy
Instrument Method.invoke() on/off always on
Instrument getters/setter on/off always off
Instrument empty methods on/off always off

Memory profiling settings:

Settings NetBeans profiler VisualVM
Profiling scope object allocations / object allocations & GC object allocations / object allocations & GC
Density of tracked objects track every to every Nth object track every to every Nth object
Allocations stack traces on/off on/off
Profiling points enabled/disabled N/A
Limit stack traces depth 1 frame to unlimited depth always unlimited depth
Run GC when getting results on/off always on

Note that you can easily determine the used profiling settings in both NetBeans profiler and VisualVM by selecting the Info tab of the CPU or memory snapshot.

Further Reading

For more information see the VisualVM Documentation, especially the Profiling Applications section.

You may also find the NetBeans Profiler 5.5 documentation useful, especially the Instrumenting a Root Method, CPU Profiling settings or Memory Profiling Settings sections.

Monday Jul 28, 2008

Profiling With VisualVM, Part 1

This article describes the out-of-the-box profiling features of the VisualVM tool available without any additional configuration, just using the default settings.

Beside various monitoring features, the tool contains a built-in profiler based on the NetBeans profiler. While the profiler UI in VisualVM looks simple (especially when compared to the NetBeans profiler), the profiling capabilities are almost as powerful as in NetBeans. The main difference between these two tools is their workflow: whereas NetBeans profiler is project/sources-centered and is a great tool for iterative testing/improving performance during application development, VisualVM can be used as a standalone profiler which works with any running application without having any context of the source code.

Note: For profiling applications running on JDK 6.0 Update 6 and earlier, the applications need to be started with the Xshare:off flag which disables the class sharing, otherwise the profiling may cause a crash of profiled JVM. For applications running on JDK 6.0 Update 7 and higher the profiling should work without any modifications.

The profiling UI is very simple, by default it shows just three buttons for analyzing performance (CPU button), memory usage (Memory button) and stopping the profiling session (Stop button). The other UI elements are Status area where the actual profiling status is shown and Profiling results section which displays live profiling data collected during the profiling session. For advanced profiling there's also a Settings section which is hidden by default. It allows to fine-tune the profiling settings, this is described in detail in Profiling With VisualVM, Part 2 article.

Note: Whereas the Profiler tab is available for each application which can be profiled, only one application can be profiled at a time, concurrent profiling sessions are not supported.

Profiling Performance

When the CPU button in Profiler tab is pressed, the profiler attaches to the application and starts profiling its performance. At the beginning the profiler needs to instrument some methods of the application, the actual number of instrumented methods is displayed in the Status area. Since the profiler uses dynamic instrumentation, the number of instrumented methods may change during the profiling session. After the profiler is attached and starts collecting the data, the view looks like on the screenshot:

You can see live profiling data for the application, for each profiled method number of invocations and total execution time is displayed. Note that not all methods are profiled, by default the profiler doesn't profile methods from java.*, javax.*, sun.*, sunw.*, com.sun.* (+ com.apple.*, apple.awt.* on Mac OS X) packages. Time spent in methods from these packages is added to execution time of profiled methods calling them, for example constructor execution time of a custom class (CustomClass.<init>) includes execution time of super constructor etc.

The toolbar of Profiling results section contains the following actions (from the left):

  • Update Results Automatically: results are periodically refreshed approx. each 1.2sec
  • Update Results Now: immediately fetches actual data, typically used when automatic refresh is disabled
  • Run Garbage Collection: invokes GC in profiled application
  • Reset Collected Results Buffer: resets the data collected so far, useful for measuring delta results
  • Take Snapshot of Collected Results: saves all the collected data into a persistent snapshot for more detailed analysis
  • Save current view to image: saves the results table into a .png image

The most important action is Take Snapshot of Collected Results (available also as Profiler Snapshot in context menu of profiled application in Applications tree). It takes a snapshot of collected profiling results (compatible with NetBeans profiler snapshot) and opens it in a separate tab. This snapshot provides several different views: Call Tree displaying methods call tree starting by threads, Hot Spots displaying a list of all profiled methods sorted by their execution time and Combined view showing both call tree and list of profiled methods. The last view Info displays basic snapshot information and detailed profiling configuration of the snapshot. There's one handy action available in context menu of each method, it's called Show Back Traces and displays all the places from where the method is being called.

  

Multiple profiler snapshots can be taken for a single profiling session for example to compare performance in various situations / loads / configurations of the application. All the profiler snapshots can be then saved into a single Application Snapshot (use Application Snapshot action in context menu of profiled application in Applications tree) to archive them in one file or to send them to application developers/quality engineers.

Profiling Memory

When Memory button in Profiler tab is pressed, the profiler attaches to the application and starts profiling its memory consumption. At the beginning the profiler needs to instrument all classes of the application, the actual number of instrumented classes is displayed in the Status area. Because all the application classes need to be instrumented, it takes some time until the profiler starts to collect the data. To lower the profiling overhead, each 10th object allocation is tracked by default (also displayed in Status area). Keep this in mind when evaluating the profiling results. After the profiler is attached and starts collecting the data, the view looks like on the screenshot:

You can see live profiling data for the application, for each class number of live instances and their size is displayed. The rightmost column shows Surviving Generations metrics which is very useful for detecting memory leaks. Detailed explanation of Surviving Generations is available in this article. You can also monitor allocations history of a particular class during time (History subtab at the bottom), to do this invoke the Log Class History action in context menu of the class.

The toolbar of memory profiling results is the same as for CPU profiling results. You can also save the collected results into a snapshot using the Take Snapshot of Collected Results action in Profiling results section toolbar or Profiler Snapshot context menu action. This is especially useful for detecting memory leaks, using the Compare Memory Snapshots action available in File menu you can visually display the difference between two comparable memory results and for example evaluate if/which objects remained on the heap after executing some action.

Note: When stopping the profiling session (both CPU an Memory) the profiler restores original application bytecode. This can take some time and the application and/or the whole system may not respond for a while.

Profiler Calibration

When VisualVM is started for the first time, the profiler performs an initial calibration. During this process the overall system and actual JDK performance is being measured and these data are later used to subtract the profiling overhead from collected performance data. The profiler requires a "stable" performance of the system, this means that any technique for dynamic CPU frequency switching like SpeedStep or PowerNow! should be disabled during both the calibration and profiling.

If the system performance changes or a different JDK than the calibrated one is used for profiling, the collected performance results may become biased and the profiler needs to be re-calibrated. Unfortunately there's no UI action available for recalibration in VisualVM 1.0, the only way to invoke the calibration again is to manually delete calibration data file in <user_directory>/.nbprofiler/machinedata.jdk1X and restart the VisualVM.

Further Reading

For more information see the VisualVM Documentation, especially the Profiling Applications section.

You may also find the NetBeans Profiler 5.5 documentation useful, especially the Profiling Results - Application Performance, Profiling Results - Memory Usage, CPU Snapshot or Memory Snapshot sections.

Tuesday Jul 08, 2008

VisualVM 1.0 Released

VisualVM 1.0 tool has been released at https://visualvm.dev.java.net. VisualVM is a free opensource visual tool integrating several commandline JDK tools and lightweight performance and memory profiling capabilities. Designed for both production and development time use, it further enhances the capability of monitoring and performance analysis for the Java SE platform.

  

The main features as listed on the Features page are:

  • Displaying local and remote Java applications
  • Displaying application configuration and runtime environment
  • Monitoring application memory consumption and runtime behavior
  • Monitoring application threads
  • Profiling application performance and analyzing memory allocation
  • Taking and displaying thread dumps
  • Taking and browsing heap dumps
  • Analyzing core dumps
  • Analyzing applications offline

VisualVM is designed as an extensible tool, new features can be easily added using a built-in Plugin Center. There are some useful plugins already available, for example the MBeans browser. All available plugins are listed on Plugins page. The tool is built on NetBeans Platform which together with VisualVM APIs makes it very easy for the developers to create new plugins.

There are three main typical users of VisualVM:

  • Application developers who can use VisualVM for profiling the applications, taking and browsing thread and heap dumps or just monitoring application runtime environment and behavior,
  • System administrators who can use VisualVM for monitoring and controlling local and remote applications across the entire network using jvmstat or JMX technologies,
  • Java application users who can use VisualVM to generate application snapshots containing all the data necessary to diagnose and fix bugs

  

Together with VisualVM 1.0 release the tool has also been released as Java VisualVM, a new JDK tool first available in Sun JDK 6 Update 7 which can be downloaded from this link. In initial releases these tools are identical, but in the future Java VisualVM will be a stable tool tested with each JDK distribution, whereas VisualVM will be a bleeding-edge distribution with the latest features and bug fixes.

To get more information about the tool, download it and start using it, visit the project pages at https://visualvm.dev.java.net. If you want to provide some feedback to VisualVM developers, let them know on a mailing list.

NetBeans IDE 6.5 Milestone 1 Released

The NetBeans IDE 6.5 Milestone 1 has been released, list of new features is available on New and Noteworthy in Milestone1 page. The bits can be downloaded from this link.

The profiler introduces one significant new feature - generating remote profiling packs from the Attach Wizard. You don't have to download a separate file for remote profiling, correct bits for target OS are generated by the Wizard based on the provided settings. There are also some important bugfixes in the profiler in this release, the list of issues is generated by this Issue Tracker query.

Tuesday May 06, 2008

VisualVM 1.0 RC Released

VisualVM 1.0 Release Candidate is available, you can download it from https://visualvm.dev.java.net/. The most significant news are:

  • Improved performance
  • Improved memory management
  • Improved stability
  • Start Page with links to VisualVM documentation and JDK monitoring and troubleshooting guides

About

A blog by NetBeans Profiler & VisualVM developers.

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today