The Real Thing
By jonthecollector on Aug 15, 2006
Here's a real treat dished up by Ross K. who hangs out 3 offices down the hall. A while back I wrote a blog about thread local allocation buffers (TLAB's). The short comings of my blog have probably been annoying Ross so much that he's written this one for us. If there are any discrepancies between my earlier blog and this one, believe this one. Thanks, Ross.
A Thread Local Allocation Buffer (TLAB) is a region of Eden that is used for allocation by a single thread. It enables a thread to do object allocation using thread local top and limit pointers, which is faster than doing an atomic operation on a top pointer that is shared across threads. A thread acquires a TLAB at it's first object allocation after a GC scavenge. The size of the TLAB is computed via a somewhat complex process discribed below. The TLAB is released when it is full (or nearly so), or the next GC scavenge occurs. TLABs are allocated only in Eden, never from From-Space or the OldGen. Flags default description UseTLAB true Use thread-local object allocation ResizeTLAB false Dynamically resize tlab size for threads TLABSize (below) Default (or starting) size of a TLAB (in bytes) TLABWasteTargetPercent 1 Percentage of Eden that can be wasted PrintTLAB false Print various TLAB related information AggressiveHeap settings: TLABSize 256Kb ResizeTLAB true (Corrected 2007/05/09) Minor flags MinTLABSize 2\*K Minimum allowed TLAB size (in bytes) TLABAllocationWeight 35 Weight for exponential averaging of allocation TLABRefillWasteFraction 64 Max TLAB waste at a refill (internal fragmentation) TLABWasteIncrement 4 Increment allowed waste at slow allocation ZeroTLAB false Zero out the newly created TLAB These flags are for tuning the current implementation of TLABs and maybe disappear or change their initial value in a future release of the jvm. If it is not specified on the command-line (or specified as zero) via the -XX:TLABSize flag, the initial size of a TLAB is computed as: init_size = size_of_eden / (allocating_thread_count \* target_refills_per_epoch) where: a) Allocating_thread_count is the expected number of threads which will be actively allocating during the next epoch (an epoch is the mutator time between GC scavenges.) At jvm startup this is defined to be one. It is then recomputed at each GC scavenge from the number of threads that did at least one allocation of a tlab during the latest epoch. It's then exponentially averaged over the past epochs. b) Target_refills_per_epoch is the desired number of tlab allocations per thread during an epoch. It is computed from the value of TLABWasteTargetPercent which is the percentage of Eden allowed to be wasted due to TLAB fragmentation. From a mutator thread's perspective a GC scavenge can occur unexpectedly at any time. So, on average, only half of a thread's current TLAB will be allocated when a GC scavenge occurs. TLABWasteTargetPercent = 0.5 \* (1/target_refills_per_epoch) \* 100 Solving for target_refills_per_epoch: target_refills_per_epoch = ( 0.5 \* 100) / TLABWasteTargetPercent With the default value of 1 for TLABWasteTargetPercent target_refills_per_epoch = 50 When TLABResize is true (which it is by default) the tlab size is recomputed for each thread that did an allocation in the latest epoch. Threads that did not allocate in the latest epoch do not have their TLABs resized. The resize goal is to get the number of refills closer to the ideal: target_refills_per_epoch (default value is 50). For each thread, the number of refills in the latest epoch is exponentially averaged with values from previous epochs. If this average refill number is greater than target refills_per_epoch, then the tlab size is increased. If the average is less, the tlab size is decreased. The computation is (approximately): new_size = (old_size \* avg_refills_per_epoch) / target_refills_per_epoch It's actually computed from the fraction of the latest epoch's eden size used by this thread, because the next epoch may use a resized eden. To experiment with a specific TLAB size, two -XX flags need to be set, one to define the initial size, and one to disable the resizing: -XX:TLABSize=
-XX:-ResizeTLAB The minimum size of a tlab is set with -XX:MinTLABSize which defaults to 2K bytes. The maximum size is the maximum size of an integer Java array, which is used to fill the unallocated portion of a TLAB when a GC scavenge occurs. Diagnostic Printing Options -XX:+PrintTLAB Prints at each scavenge one line for each thread (starts with "TLAB: gc thread: " without the "'s) and one summary line. Thread example: TLAB: gc thread: 0x0004ac00 [id: 2] size: 61KB slow allocs: 5 refill waste: 980B alloc: 0.99996 3072KB refills: 50 waste 0.1% gc: 0B slow: 4144B fast: 0B The tag "gc" indicates that this information was printed at a GC scavenge, after the tlabs have been filled. The "gc" tag doesn't mean a thread is a gc thread. Fields: thread: The address of the gc thread structure and it's system thread id. size: The size of the tlab in kilobytes. slow allocs: The number of allocations too large for remaining space in the TLAB. The allocation was done directly in eden space. refill waste: (in HeapWord units) The name is truncated in the dump, and should be: refill_waste_limit and is used to limit the amount of wasted space from internal fragmentation. If the remaining space in the TLAB is larger than this amount, and an allocation is requested that is too large to be allocated in the TLAB, then the allocation is done directly in Eden and the TLAB is not retired. If the remaining space is less than refill_waste_limit then the TLAB is retired, a new TLAB is allocated, and the object allocation is attempted in the new TLAB. After each allocation outside of the TLAB, the refill_waste_limit is incremented by TLABWasteIncrement to prevent an allocation of a size slightly less than refill_waste_limit from continually being allocated outside of the TLAB. alloc: [fraction] [sizeInBytes] Expected amount of eden allocated by this thread computed as a fraction of eden and number of heap words. refills: Number of tlab refills. waste [percent] gc: [bytes] slow: [bytes] fast: [bytes] Percentage of eden allocated to this thread that was wasted. Waste is the sum of three components: gc: unused space in the current TLAB when stopped for a scavenge. slow: sum of unused space in TLABs when they're retired to allocate a new one. fast: the client system can allocate a TLAB with a fast allocator. This is the amount of waste via that method. Summary example: TLAB totals: thrds: 1 refills: 50 max: 50 slow allocs: 5 max 5 waste: 0.1% gc: 0B max: 0B slow: 4144B max: 4144B fast: 0B max: 0B thrds: Number of threads that did an allocation. refills: [tt] max: [mm] Total number of TLAB refills by all threads, and maximun number of TLAB refills by a single thread. slow allocs: [ss] max [mm] Total number of allocations done outside of a TLAB, and maximum number by a single thread. waste [percent] gc: [bytes] slow: [bytes] max: [mm] fast: [bytes] max: [mm] Percentage of eden that was wasted across all threads. Waste is the sum of three components: gc: unused space in the current TLABs when scavenge starts. slow: sum of unused space in TLABs when they're retired to allocate a new ones. fast: the client system can allocate a TLAB with a fast allocator. This is the amount of waste via that method. For "slow" and "fast", the maximum value by a single thread is printed. More detail with addition of Verbose flag. -XX:+PrintTLAB -XX:+Verbose Using both: -XX:+PrintTLAB -XX:+Verbose will print the new tlab sizes for each thread when it is resized. Resizing is only done at GC scavenges. Example: TLAB new size: thread: 0x001eac00 [id: 19] refills 50 alloc: 0.402570 size: 19684 -> 18996 New size 18996. Previous size was: 19684. refills: Number of tlab refills for this thread. alloc: The expected fraction of eden this thread will use.