Tuesday Feb 28, 2012

OBIEE 11.1.1 - IBM JDK Tuning for WebLogic 11G under AIX 6.x

Modify setOBIDomainEnv.sh:

Each lines like:
        if [ "${JAVA_VENDOR}" = "IBM" ] ; then
                # 64-bit IBM JVM is the only JVM supported for AIX
                # assume to always use Managed server memory args for 64-bit IBM JVM
                SERVER_MEM_ARGS="-Xms2048m -Xmx2048m -Xmns512m -Xmnx512m -Xss2m -Xrs -Xgcpolicy:gencon -Xverify:none"
                export SERVER_MEM_ARGS

    -Xms<size>        set initial Java heap size
    -Xmx<size>        set maximum Java heap size
    -Xss<size>         set java thread stack size
    -Xrs                    reduce use of OS signals by Java/VM (see documentation)

      Sets the initial size of the new (nursery) heap to the specified value when using -Xgcpolicy:gencon. By default, this option is selected internally according to your system's capability. This option will return an error if you try to use it with -Xmn.
      Sets the maximum size of the new (nursery) heap to the specified value when using -Xgcpolicy:gencon. By default, this option is selected internally according to your system's capability. This option will return an error if you try to use it with -Xmn.
With no parameters, enables the verifier. Note that this is the default; used on its own, this option has no effect.

Additional parameters to review:

You can force the JVM to use large pages, which is more efficient memory management.
Java VM command line option: -Xlp64K

You should also consider using huger pages.
PPC64 supports 4K(default) and 16M (although this is more likely to speed things up but unlikely solve your heap problem...)
General info for AIX and PPC:
Java vm command line:
AIX: Requests the JVM to allocate the Java heap (the heap from which Java objects are allocated) with large (16 M pages, if a sizeis not specified. If large pages are not available, the Java heap is allocated with the next smaller page size that is supported by the system. AIX requires special configuration to enable large pages.

For more information about configuring AIX support for large pages, see http://publib.boulder.ibm.com/infocenter/aix/v6r1/topic/com.ibm.aix.prftungd/doc/prftungd/large_page_ovw.htm.
The SDK supports the use of large pages only to back the Java heap shared memory segments. The JVM uses shmget() with the SHM_LGPG and SHM_PIN flags to allocate large pages. The -Xlp option replaces the environment variable IBM_JAVA_LARGE_PAGE_SIZE, which is now ignored if set.
AIX, Linux, and Windows only: If a <size> is specified, the JVM attempts to allocate the JIT code cache memory using pages of that size. If unsuccessful, or if executable pages of that size are not supported, the JIT code cache memory is allocated using the smallest available executable page size."

You should also turn on verbose gc.
Java VM command line option:  –verbose:gc

Consider some of the following Java VM command line options (some IBM vm specific):
- -Xgcpolicy:subpool "Uses an improved object allocation algorithm to achieve better performance when allocating objects on the heap. This option might improve performance on large SMP systems"
- -Xcompressedrefs "Use -Xcompressedrefs in any of these situations: When your Java applications does not need more than a 25 GB Java heap. When your application uses a lot of native memory and needs the JVM to run in a small footprint."
- -Xcompactexplicitgc "Enables full compaction each time System.gc() is called."
- -Xcompactgc "Compacts on all garbage collections (system and global)."
- -Xsoftrefthreshold<number> "Sets the value used by the GC to determine the number of GCs after which a soft reference is cleared if its referent has not been marked. The default is 32, meaning that the soft reference is cleared after 32 * (percentage of free heap space) GC cycles where its referent was not marked." Reducing this will clear out soft  references sooner. If any soft referenced-based caching is being used, cache hits will go down but memory will be freed up faster. But this will not directly solve your OOM problem: "All soft references are guaranteed to have been cleared before the OutOfMemoryError is thrown.
The default (no compaction option specified) makes the GC compact based on a series of triggers that attempt to compact only when it is beneficial to the future performance of the JVM."