Precompiled JSP Behavior on WebLogic 12c. Suddenly consuming more memory?

Hi everyone!

As my first post I would like to share something that might come in-handy for you and your efforts to maintain the memory consumption of your server controlled at the maximum. 

If you happen to use weblogic.appc to improve the performance of your applications pre-compiling your code before deploying, you might be hitting this problem. As you might be aware of, the virtual process size is composed by Java Heap and Non-Java Heap. This will cause that all of the objects and required data to make your server work will often be loaded outside the Java Heap. For this particular case, pre-compiled JSP files are stored outside the Java Heap. For Hotspot JDK particularly, this ends up being stored on PermSpace.

Well, why is this important?

If you happen to be migrating an application of your own from WebLogic 11g to 12c release and you pre-compile your JSP, you will see that whenever your applications are started, more PermSpace will be consumed out of thin air. No immediate explanation for this is evident, however this is completely expected!

This will be an issue for you, if you happen to have hundreds of pre-compiled JSPs in place on your application. 


How can I know if I'm facing this problem? 

If you enable classloading debug ( -verbose:class as Java System Property) you will see all of your pre-compiled JSPs getting loaded upon app activation (i.e. Going into Admin Console or using WLST for example and transitioning your application from Prepared to Active state):

[Loaded javax.ws.rs.ext.Provider from file:/refresh/home/WLS/12.1.1/modules/com.sun.jersey.core_1.1.0.0_1-9.jar]
[Loaded $Proxy108 from sun.misc.Launcher$AppClassLoader]
[Loaded jsp_servlet.__dummyjsp1603 from file:/refresh/home/WLS/12.1.1/user_projects/domains/myDomain/servers/AdminServer/tmp/_WL_user/native_test_1211/7vonl8/war/WEB-INF/lib/_wl_cls_gen.jar]
[Loaded jsp_servlet.__dummyjsp1602 from file:/refresh/home/WLS/12.1.1/user_projects/domains/myDomain/servers/AdminServer/tmp/_WL_user/native_test_1211/7vonl8/war/WEB-INF/lib/_wl_cls_gen.jar]
[Loaded jsp_servlet.__dummyjsp1605 from file:/refresh/home/WLS/12.1.1/user_projects/domains/myDomain/servers/AdminServer/tmp/_WL_user/native_test_1211/7vonl8/war/WEB-INF/lib/_wl_cls_gen.jar]
[Loaded jsp_servlet.__dummyjsp1604 from file:/refresh/home/WLS/12.1.1/user_projects/domains/myDomain/servers/AdminServer/tmp/_WL_user/native_test_1211/7vonl8/war/WEB-INF/lib/_wl_cls_gen.jar]
[Loaded jsp_servlet.__dummyjsp1607 from file:/refresh/home/WLS/12.1.1/user_projects/domains/myDomain/servers/AdminServer/tmp/_WL_user/native_test_1211/7vonl8/war/WEB-INF/lib/_wl_cls_gen.jar]
...
...
...
[Loaded jsp_servlet.__dummyjsp1890 from file:/refresh/home/WLS/12.1.1/user_projects/domains/myDomain/servers/AdminServer/tmp/_WL_user/native_test_1211/7vonl8/war/WEB-INF/lib/_wl_cls_gen.jar]

If we want to be conservative with our memory usage, we just want to load these pre-compiled JSP only upon access right? Indeed that's exactly what's not happening as default behavior here :)


Under the hood

Starting WebLogic Server 12.1.1 a new functionality is implemented. This is basically represented by a method called initContainerInitializer(). If you are crashing with this problem, if you start your application and collect a couple of Thread Dumps, you will see the following methods appearing on the thread in charge of reading and inflating the deployment to make it available:

"[ACTIVE] ExecuteThread: '0' for queue: 'weblogic.kernel.Default (self-tuning)'" daemon prio=10 tid=0x0af82000 nid=0x1e9f runnable [0x353bd000]
java.lang.Thread.State: RUNNABLE
  at java.util.zip.ZipFile.read(Native Method)
  at java.util.zip.ZipFile.access$1200(ZipFile.java:31)
  at java.util.zip.ZipFile$ZipFileInputStream.read(ZipFile.java:460)
- locked (a java.util.jar.JarFile)
  at java.util.zip.ZipFile$1.fill(ZipFile.java:243)
  at java.util.zip.InflaterInputStream.read(InflaterInputStream.java:141)
  at weblogic.utils.io.DataIO.readFully(DataIO.java:351)
  at weblogic.utils.io.DataIO.readFully(DataIO.java:328)
  at weblogic.utils.classloaders.ZipSource.getBytes(ZipSource.java:76)
  at weblogic.utils.classloaders.GenericClassLoader.defineClass(GenericClassLoader.java:321)
  at weblogic.utils.classloaders.GenericClassLoader.findLocalClass(GenericClassLoader.java:302)
- locked (a weblogic.utils.classloaders.GenericClassLoader)
  at weblogic.utils.classloaders.GenericClassLoader.findClass(GenericClassLoader.java:270)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:305)
- locked (a weblogic.utils.classloaders.GenericClassLoader)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:246)
  at weblogic.utils.classloaders.GenericClassLoader.loadClass(GenericClassLoader.java:179)
  at java.lang.Class.forName0(Native Method)
  at java.lang.Class.forName(Class.java:247)
  at weblogic.application.utils.annotation.ClassfinderClassInfos.loadClass(ClassfinderClassInfos.java:109)
  at weblogic.application.utils.annotation.ClassfinderClassInfos.getHandlesImpls(ClassfinderClassInfos.java:68)
  at weblogic.servlet.internal.War.getHandlesImpls(War.java:413)
at weblogic.servlet.internal.WebAppServletContext.initContainerInitializer(WebAppServletContext.java:1279)
at weblogic.servlet.internal.WebAppServletContext.initContainerInitializers(WebAppServletContext.java:1229)

  at weblogic.servlet.internal.WebAppServletContext.preloadResources(WebAppServletContext.java:1726)
- locked (a weblogic.servlet.internal.WebAppServletContext)
  at weblogic.servlet.internal.WebAppServletContext.start(WebAppServletContext.java:2740)
  at weblogic.servlet.internal.WebAppModule.startContexts(WebAppModule.java:1704)
  at weblogic.servlet.internal.WebAppModule.start(WebAppModule.java:781)

Did you notice the ones highlighted on red? If you look carefully you will see that these methods are not present on 11g code, hence if you do the same exercise on your old WebLogic Server you will not see them! Boom... smoking gun dead ahead!

Ok, problem is there... How do I fix it? 

Good news is that this is addressed already. If you want to return to the old WebLogic 11g behavior, then you need to apply a two-step solution:

  • Install a patch (Only if you are running on WebLogic Server 12.1.1. If you are on 12.1.2 onwards, jump to next step)
    • Bug in question is Unpublished Defect: Bug 14805810 - psr:perf:soa severe impact on startup time,webappservletcontext.initcontainerini
  • Modify your deployment descriptors (weblogic.xml) so that patch functionality is enabled.

Deployment descriptor weblogic.xml should look similar to the one below to enable the patch (Notice the entries in bold red):

<?xml version="1.0" encoding="UTF-8"?>
<wls:weblogic-web-app xmlns:wls="http://xmlns.oracle.com/weblogic/weblogic-web-app" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_2_5.xsd http://xmlns.oracle.com/weblogic/weblogic-web-app http://xmlns.oracle.com/weblogic/weblogic-web-app/1.4/weblogic-web-app.xsd">
  <wls:weblogic-version>12.1.1</wls:weblogic-version>
  <wls:context-root>native_test_1211</wls:context-root>
  <wls:container-descriptor>
    <wls:container-initializer-enabled>false</wls:container-initializer-enabled>
  </wls:container-descriptor>

</wls:weblogic-web-app>

You can do this with a Deployment Plan or directly on the Application. Both of them should work great.  

For details in how to install the patch or even how to replicate this on your end, check this My Oracle Support Document: WebLogic Server 12.1.1 Loads All Precompiled JSPs Into Native Memory Causing Out of Memory Problems (Doc ID 1571737.1)

Further questions? Any problems about this topic? Go ahead and open a Support Case. We will be able to answer what you need. 

Cheers!

Denny.




Comments:

Post a Comment:
  • HTML Syntax: NOT allowed
About

Welcome! This blog contains specific internals and things that you may find interesting about WebLogic Server, Coherence and Oracle JVMs (JRockit and Hotspot). Articles are written by Senior Support Engineers working in Oracle Global Customer Support. Feel free to comment and share!

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today