Saturday Apr 07, 2012

JRockit hang fix

Fix for the JRockit hang I talked about in my previous blog would be available in R28.2.4.

Friday Feb 03, 2012

Diagnosis of a JRockit Deadlock

Recently I came across an interesting JRockit JVM deadlock. I am sharing the details of that deadlock, its diagnosis, and how to workaround that deadlock, which might be useful until the fix for that is available in a future JRockit release.

This deadlock occurred between an application Java thread and the JVM's Code Generation thread. The call traces for both the threads looked like this:

"[ACTIVE] ExecuteThread: '0' for queue:  'weblogic.kernel.Default (self-tuning)'" id=15 idx=0x54 tid=27249 prio=5 alive, native_waiting,daemon
at <unknown>(???.c)@0xffffe410
at eventTimedWaitNoTransitionImpl+79(event.c:90)@0xf7c910a0
at syncWaitForSignalNoTransition+81(synchronization.c:28)@0xf7e0d8a2
at innerNativeDoWait+894(nativelock.c:614)@0xf7d9e8cf
at nativeWait+71(nativelock.c:721)@0xf7d9ec28
at cbrCompileInCodeGenThread+451(compilerqueue.c:339)@0xf7c71b94
at dispatch_compile_request+78(compilerbroker.c:511)@0xf7c64eaf
at cbrGeableCodtRunneInfo+154(compilerbroker.c:580)@0xf7c6516b
at stubsCallJava+77(stubcall.c:112)@0xf7e061ce
at stubsCallJavaV+344(stubcall.c:276)@0xf7e065c9
at javaInvokeStaticVoidMethod+39(javacalls.c:178)@0xf7cd6098
at clsEnsureInitialized+485(class.c:256)@0xf7c4c446
at check_flags_and_clinit+28(compilerbroker.c:302)@0xf7c641fd
at cbrGetRunnableCodeInfo+37(compilerbroker.c:564)@0xf7c650f6
at stubsCallJava+77(stubcall.c:112)@0xf7e061ce
at javaInvokeMethod+280(javacalls.c:1128)@0xf7cd62a9

"(Code Generation Thread 1)" id=4 idx=0x30 tid=27235 prio=5 alive,native_waiting, daemon
at <unknown>(???.c)@0xffffe410
at eventTimedWaitNoTransitionImpl+79(event.c:90)@0xf7c910a0
at syncWaitForSignalNoTransition+81(synchronization.c:28)@0xf7e0d8a2
at innerNativeDoWait+894(nativelock.c:614)@0xf7d9e8cf
at nativeWait+71(nativelock.c:721)@0xf7d9ec28
at clsEnsureInitialized+334(class.c:219)@0xf7c4c3af
at check_flags_and_clinit+28(compilerbroker.c:302)@0xf7c641fd
at compileit+273(compilerbroker.c:317)@0xf7c64992
at cbrCompileRequest+16(compilerbroker.c:537)@0xf7c651b1
at cg_thread+876(compilerqueue.c:223)@0xf7c7168d
at thread_stub+318(lifecycle.c:808)@0xf7d5205f
at start_thread+225(:0)@0x64a832
at __clone+93(:0)@0x5cee0e

The above stack traces and the examination of the core-dump file revealed that the Java Thread (tid=27249) was initializing a class and was in the process of invoking 'clinit' method of that class. Method 'clinit' was not compiled, so it invoked dispatch_compile_request() to compile that method, which sent the compilation request to the Code Generation thread. The Code Generation thread accepted that request and got stuck in waiting for the initialization of the class of 'clinit' to complete. Now, function dispatch_compile_request() checks if the thread has enough (>64K) stack space available for code generation. If yes, then it compiles the method on the same thread otherwise passes the compilation request to the Code Generation (Compiler) thread.

if (!enough_stack) {
...
res = cbrCompileInCodeGenThread(req, TRUE);
} else {
res = compileit(req, req->method);
}

In this case, stack space available with Java thread (tid=27249) was around 63K and that was the cause of the problem. Here's what was happening:-

(i) While compiling a method, the Code Generation thread wants the class of that method to be initialized, and if it is not, it waits for the class to get initialized.
(ii) A Java thread which is in the middle of initializing a class requests Code Generation thread to compile a method of that class because it does not have enough stack space to do the compilation itself and waits for the Code Generation thread to finish the compilation.

The above scenario causes a deadlock.

A simple solution to avoid the above situation is to increase the stack size for the application Java threads so that there is always enough space on the stack that the compilation can be done on the same thread itself and the need to pass the compilation requests to the Code Generation thread does not arise at least for the uninitialized classes.

The fix for this bug is progress and will be available in a future JRockit patchset release.

About

poonam

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today