Redhat Linux Kernels and process space limits using Java.. Some eye-openers

Different Kernels available and their memory limits using Java

There are at least 3 different kind of Kernels available in Redhat linux:
1) SMP kernel:- This kernel is the most common variant used and supports only upto 4GB Virtual Memmory ((2**32 = 4GB) linear addresses). It gives an address space of 3G virtual memory peruser process & 1G for kernel, but after deducing stack overhead, the maximum user addressable space limit comes to 1.7g.
The split is:
1.0 GB Text/code
1.7 GB available process memory for mmap, malloc / address space
     (e.g. usable by shared Pool)
0.3 GB Stack
1.0 GB Kernel
2) Enterprise Kernel:- In this kernel, the process address space can go up to 64GB VM (2**36 = 64GB linear addresses). This feature is called Page Address Extension (PAE) and is also referred to as Very Large Memory (VLM) support.
The enterprise kernel provides VLM support out of the box, BUT, it requires relinking of the kernel, which is a pretty unpopular option with Redhat and usually last resort of system administrators.
Note: You should be aware that Oracle Corp. will stop supporting any Oracle
          installation on a recompiled kernel, particularly Red Hat.
3) Hugemem Kernel:- Red Hat Enterprise Linux 3.0/4.0 includes a new kernel known as the hugemem kernel. This kernel supports a 4GB per process user space (versus 3GB for the other kernels), and a 4GB direct kernel space.
  The hugemem kernel is usually required in order to use all the memory in system configurations containing more than 16GB of memory. The hugemem kernel can also benefit configurations running with less memory (if running an application that could benefit from the larger per process user space).
The virtual address space is split as:
1.0 GB Text/code
2.7 GB available process memory for mmap, malloc / address
        space (e.g. usable by shared Pool)
0.3 GB Stack
4.0 GB Kernel

Understanding how one kernel is better than other...

It is important to understand the pros and cons between using SMP, Enterprise and Hugemem kernels. For applications having large virtual memory requirements, even the enterprise kernel could be a solution using the VLM feature in RHEL3/4, but it would entail creating a very large buffercache, using options mentioned in Note 262004.1 for Oracle Databases (one kind of application).
On top of the page addressing problem for 64GB linear addresses, there will be a problem in kernel memory if we have a lot of memory. It would be required to build a Page Translation Extension (PTE) for all available memory pages and this restricts the memory we can address.
Typically the kernel memory of 1gb is completely used up if its required to address just 24GB of memory. To overcome this problem we need a large kernel memory and this is where the hugemem kernel beats the Enterprise kernel as it provides 4G of Kernel memory.

Experimenting with process limits across different factors..

We can find out the maximum number of threads which can be spawned on a platform across any JVM version by running a program written in Java (ThreadTest.java) -- listing available in Appendix A.
This program just keeps on spawning threads and keeps them alive almost forever. When it reaches the Java.Lang.OutOfMemory error on spawning a thread, it outputs the maximum number of threads it was able to spawn.
Based on this information and the native java thread stack size specified on JVM startup and using the golden memory rule, we can calculate the total virtual memory address space of the JVM process.
For a listing of the java source code of and the unix shell script check_thread_limits.sh, please refer to Appendix B-1. The results of the max threads spawned and Maximum user address space by varying JDK version, Kernel (SMP, Hugemem), ulimit, stacksize are listed in Appendix B-2.
As can be seen from Appendix B-2:
For SMP kernels:
With ulimit -s unlimited:
(Total size of managed heap + native heap) + (number of threads X thread stack size) ~ 1.7G
With ulimit -s 10m or 8m or 6m or 4m or 2m:
(Total size of managed heap + native heap) + (number of threads X thread stack size) ~ 2.7G
For Hugemem kernels:
With ulimit -s unlimited:
(Total size of managed heap + native heap) + (number of threads X thread stack size) ~ 2.5G
With ulimit -s 10m or 8m or 6m or 4m or 2m:
(Total size of managed heap + native heap) + (number of threads X thread stack size) ~ 3.7G
Note: This also implies that the "unlimited" native stack size constant is not the same as 10m as might be seen in comments in /usr/src/linux/include/sched.h for _STK_LIM constant, rather its some valule which is very high as compared to 10m. Sometime, seeing and reading comments in source code files is not believing, as I found out.


Appendix A : Listing of ThreadTest.java

public class ThreadTest {

public static void main(String[] pArgs) throws Exception {

try {
// keep spawning new threads forever
      while (true) {
      new TestThread().start();
      }
   }
// when out of memory error is reached, print out the number of
// successful threads spawned and exit
   catch ( OutOfMemoryError e ) {
     System.out.println(TestThread.CREATE_COUNT);
     System.exit(-1);
   }
  }
  static class TestThread extends Thread {
    private static int CREATE_COUNT = 0;
    public TestThread() {
     CREATE_COUNT++;
    }
// make the thread wait for eternity after being spawned
    public void run() {
      try {
        sleep(Integer.MAX_VALUE);
      }
// even if there is an interruption, dont do anything
      catch (InterruptedException e) {
      }
    }
  }
}

Appendix B-1 : Listing of check_thread_limits.sh

check_thread_limits code:

Appendix B-2 : Results of check_threads_limits.sh across different Kernel, Java Version and native Java stack size

SMP Kernel results:

smp_check_thread_limits.sh_output:

Hugemem Kernel results:

hugemem_check_thread_limits.sh_output:

Comments:

Post a Comment:
  • HTML Syntax: NOT allowed
About

bocadmin_ww

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today