X

Jon Masamitsu's Weblog

  • Java
    October 26, 2006

Why Now?

Guest Author
Are you ever surprised by the occurrence of a
full collection (aka major collection)? Generally a full collection
only happens when an allocation is attempted and there is not enough space available
in any of the generations. But there are
exceptions to that rule and here are some of them.
Actually, these are all of them that I could think of.
I thought I would have a longer list so to fill out
this blog I've included
some bugs that relate to full collections that might interest you. And
we're thinking about a policy change for the low pause collector that would affect full collections.

The simplest reason for an unexpected full collection is that you asked for it. System.gc() can be called from most anywhere in the program. Try using -XX:+DisableExplicitGC to
have the garbage collector ignore such requests.

The permanent generation is collected only by a full collection. It's very seldom the
cause for a full collection but don't overlook the possibility. If you turn on
PrintGCDetails, you can see information about the permanent generation collection.

0.167: [Full GC [PSYoungGen: 504K->0K(10368K)] [ParOldGen: 27780K->28136K(41408K)] 28284K->28136K(51776K) [PSPermGen: 1615K->1615K(16384K)], 0.4558222 secs]

This output is from the throughput collector. The "PSPermGen" denotes the permanent generation. The size of the permanent generation currently is 16384K.
It's occupancy
before the collection is only 1615K and is probably the same after the collection.
If the permanent generation needed collecting, the occupancy before the collection
would have been closer to the 16384K.

The garbage collector tracks the average of the
rate of promotion of live objects from the young
generation into the tenured generation. If the average
exceeds the free space in the tenured generation, the next
time the young generation fills up, a full collection is
done instead of a minor collection. There is a pathological
situation where, after the full collection, the free space in the tenured generation
is still not large enough to take all the live objects expected
to be promoted from the next minor collection. This will result in another full
collection the next time the young generation fills up. The bad part
about this is that the average
value for the amount promoted only changes when a minor
collection is done and if no minor collections are done, then
the average does not change. Increasing the size of the heap
usually avoids this problem.

Sometimes it's not really a full collection. There was a bug affecting
JDK5 (6432427) which print "Full" when the collection was not a full
collection. When JNI critical sections are in use, GC can be locked
out. When the JNI critical sections are exited, if a GC had been
requested during the lock out, a GC is done. That GC most likely
would be a minor collection but was mistakenly labeled a full
collection. A second place where "Full" was also printed erroneously
was when -XX:+CMSScavengeBeforeRemark is used. The same bug report explains
and fixes that error.

An attempt to allocate an object A larger than the young generation can
cause a minor collection. That minor collection will fail to free
enough space in the young generation to satisfy the allocation of A.
A full collection is then attempted and A is then allocated out of the
tenured generation. This behavior was a bug (6369448). It has been
fixed in JDK's 1.4.2_13, 5.0_10, and 6. The correct behavior is for
the GC to recognize when an object is too large to be allocated out of the
young generation and to attempt the allocation out of the tenured
generation before doing a collection.

When an allocation cannot be satisfied by the tenured generation a
full collection is done. That is the correct behavior but that behavior
has an adverse affect on CMS (the low pause collector). Early in a
run before the CMS (tenured) generation has expanded to a working size,
concurrent mode failures (resulting in full collections) happen.
These concurrent mode failures perhaps can be avoided.
We are thinking about a change to the policy such that CMS expands the
generation to satisfy the allocation and then starts a concurrent
collection. We're cautious about this approach
because similar policies in the past has led to inappropriate expansion of
the CMS generation to its maximum size. To avoid these full collections
try specifying a larger starting size for the CMS generation.

So just because these are all the unexpected "Full" collections that I could
think of, that doesn't mean that these are all there are. If you have a
mysterious "Full" collection happening, submit a description to


http://java.sun.com/docs/forms/gc-sendusmail.html

I'll post any additional cases of unexpected "Full" collections that I get.

Late addition (I'm allowed to edit my blog even after they've been posted). If you
ask about unexpected "Full" collections and send in a gc log, please add the
command line flags

-XX:+PrintGCDetails -XX:+PrintHeapAtGC -XX:+PrintGCTimeStamps

The additional output (of which there will be plenty) may help me see what's happening.

Join the discussion

Comments ( 3 )
  • Damon Hart-Davis Thursday, October 26, 2006
    Hi,

    Thanks: interesting as always...

    Does that set of occasions that you list include the "WARNING: bailing out to foreground collection" that I sometimes see when my Web server is starting up and working VERY hard to get ready for the first hits ASAP?

    Rgds

    Damon

  • Jon Umasamsit Friday, October 27, 2006
    The message about "bailing out" (from the low pause collector /CMS collector) is a concurrent mode failure. A concurrent collection has started but not finished before the tenured generation becomes full. This situration might better be described in my blog on "concurrent mode failures". Two other possibilities are that the permanent generation is too small at start up or that you just need to start with a larger initial heap size. If it's the permanent generation, try a larger size with -XX:PermSize=<nnn>.
  • Doug Tuesday, October 31, 2006
    From http://java.sun.com/docs/hotspot/gc5.0/gc_tuning_5.html#1.1.Other%20Considerations%7Coutline
    "RMI forces periodic full collection... default rate of once per minute."
Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.