JVM in containers
Modern day software systems are moving towards containers, and there are few important factors to understand, before we move our Java / JVM based applications on containers and these factors are raising questions if Java is really good for containers? so lets see how java is making progress and what is there for containers.
Suppose, Our environment has deployed 10 instances of an application into a container and all the sudden the application starts throttling and not achieving the same performance as we have seen on our development environment. What exactly has happened? In order to allow multiple containers to run isolated side-by-side, we have specified it to be limited to one cpu (or the equivalent ratio in CPU shares). Unfortunately, the JVM will see the overall number of cores on that node (64) and use that value to initialize the number of default threads we have seen earlier. As started 10 instances, we end up with:
10 * 64 Jit Compiler Threads
10 * 64 Garbage Collection threads
10 * 64 …. And so on…
Moreover, our application, being limited in the number of CPU cycles it can use, is mostly dealing with switching between different threads and does cannot get any actual work done.
All the sudden the promise of containers, “Package once, run anywhere’ seem violated.
Before JDK 8.0_131, it gets the core count resources from sysconf. That means that whenever we run it in a container, we are going to get the total number of processor available on the system, or in case of virtual machines: virtual system. The same is true for default memory limits: the JVM will look at the host overall memory and use that for setting its defaults. We can say that the JVM ignoring cgroups and that cause problems as we have seen above. Unfortunately, there is no CPU or memory namespace (also, namespaces usually have a slightly different goal), so a simple less /proc/meminfo from inside the container will still show us the overall memory on the host. We can say that the JVM ignoring cgroups and that cause problems as we have seen above.
Java 8.0_131 onwards on containers!
Java now supports Docker CPU and memory limits. Let us look at what “support” actually means.
Please look into the below Jira which shows the list of improvement in java for Containers.
https://bugs.openjdk.java.net/browse/JDK-8146115: Improve Docker container detection and resource configuration usage
These changes are available in 8u192. [It is expect to release in Oct 2018]
The JVM can recognize the memory and CPU configurations of the container it is running in. For instance, if the Docker container has configured to run with 1024m of memory, the JVM can now detect that, and can in turn configure its java heap and the sizes of its other memory pools accordingly. Therefore, to have a smaller footprint of your Docker instance, all you have to do is to control the size of that container instance. The same applies for the CPU configuration as well. You focus on configuring the number of CPUs that you would like the container instance to use, and the JVM running inside it will be able to detect that configuration, and limit its CPU use to that configuration.
The JVM will now consider cgroups memory limits if the following flags are specified:
In that case, the Max Heap space will be automatically (if not overwritten) be set to the limit specified by the cgroup. As we discussed earlier, the JVM is using memory besides the Heap, so this will not prevent user from the OOM killer removing their containers. However, especially giving that the garbage collector will become more aggressive as the Heap fills up, this is already a great improvement.
The JVM will automatically detect cpusets and if set use the number of CPUs specified for initializing the default values discussed earlier. Unfortunately, most users (and especially container orchestrators such as DC/OS) use CPU shares as the default CPU isolation. Moreover, with CPU shares you will still end up with the incorrect value for default parameters.
So what can we do?
We should consider manually overwriting the default parameter
(e.g., at least XMX for memory and XX:ParallelGCThreads, XX:ConcGCThreads for CPU) according to your specific cgroup limits.
Considering Non-Heap Memory in Java
The JVM memory consists the following segments:
The JVM has memory other than the heap, referred to as non-heap memory. It has created at the JVM startup and stores per-class structures such as runtime constant pool, field and method data, and the code for methods and constructors, as well as interned Strings.
In previous versions of Java (before 1.8), JVM specifies a default space of 64 MB for permgen space, and this could be modified according to need of requires more than 64 MB.
In Java 8, PermGen has been renamed to Metaspace - with some subtle differences. It is important to note that Metaspace has an unlimited default maximum size (-XX:MaxMetaspaceSize=?MB). On the contrary, PermGen from Java 7 and earlier has a default maximum size of 64 MB on 32-bit JVM and 82 MB on the 64-bit version. Of course, these are not the same as the initial sizes. Java 7 and earlier starts with something around 12-21 MB of the initial PermGen space.
However, MetaSize can be set but there was a bug in the java that specified metaspacesize was not committing. Ref https://bugs.openjdk.java.net/browse/JDK-8067100
Now after setting heap and MetaSpaceSize, what options do we have?
There is no way that we can limit the native memory usage of an application. After allocating java heap and Metaspace, whatever left in the system memory, a java application is free to use that for other native allocations. We can limit that by limiting the total memory available to the container itself, by configuring the total memory of the container. However, if our application is extensively using Direct memory buffers (native allocations), we can control the maximum size of those by using the JVM option MaxDirectMemorySize, and that in turn will control the size of native allocations.
We can say that the JVM ignoring cgroups and that cause problems as we have seen above.
Unfortunately, there is no CPU or memory namespace (also, namespaces usually have a slightly different goal), so a simple less /proc/meminfo from inside the container will still show us the overall memory on the host.