Introduction

The Oracle TimesTen 18.1 In-Memory Database uses Shared Memory to store its data.

Shared Memory resources such as shared memory segments and sempahores are operating system kernel resources and need to be configured.

Linux Configuration

On supported Linux platforms, you need to configure the following values in the /etc/sysctl.conf file:

For example, on a Linux machine with a 4KB page size and 64 GB RAM, a 48 GB shared memory segment, SHMALL as 56 GB and supporting 20K connections would require:

  • kernel.shmmax = 51539607552  # 1024 * 1024 * 1024 * 48
  • kernel.shmall    = 14680064        # 1024 * 1024 * 1024 * 56 = 60129542144

                                                            # 60129542144 / 4096 = 14680064 [4K pages]

  • kernel.sem        = 21000    53760000    2000    2560   

                                                           # SEMMSL = 21000 [20K connections + some extras]

                                                           # SEMMNS = (SEMMSL * SEMMNI)

                                                           # SEMOPM = 2000

                                                           # SEMMNI = 2560

Using Linux HugePages has three benefits:

  • Generally faster memory access as fewer 2MB pages need to be accessed in the TLB rather than more 4KB pages
  • The HugePages are automatically locked into memory and hence cannot be paged out
  • Required if your database shared memory segment is larger than 256 GB

 

Configure HugePages on Linux

To configure HugePages, you need to know:

  • The Linux Group ID [GID] of the OS user who installed TimesTen:
    • $ id
    • uid=2179 (foo) gid=900 (bar) …
    • In this case, the GID is 900
  • The max size of the shared memory segment for the database
    • eg 6G is kernel.shmmax=6442450944
  • The HugePages size on your Linux machine
    • $ cat /proc/meminfo | grep Hugepagesize
      Hugepagesize:       2048 kB

The number of HugePages = shmmax [in MB] / page size [in MB].

eg, for a small DEV machine with 8 GB RAM:

  • shmmax = 6 GB = 6,144 MB
  • 6,144 / 2 = 3,072 pages

Add the following to the /etc/systcl.conf file:

  • vm.nr_hugepages=3072
  • vm.hugetlb_shm_group=900

Then persist those values in the Linux kernel:

# sudo /sbin/sysctl -p

Then check the actual configured value of hugepages:

# cat /proc/meminfo|grep HugePages
HugePages_Total:     3072
HugePages_Free:     3072
HugePages_Rsvd:          0
HugePages_Surp:           0

You may need to free memory and/or reboot the machine to be able to allocate the desired number of HugePages.

 

AIX Configuration

On AIX, semaphores are configured dynamically by the kernel so you just need to focus on large pages.

 

Sizing Shared Memory

You cannot allocate all of a machine’s RAM to be shared memory:

  • The Operating System Kernel needs some RAM for itself
  • The Operating System file buffer cache likes to use lots of RAM
  • Application processes use their own RAM

The following table is an estimate of possible shared memory sizes given a fixed amount of RAM:

Shared memory and RAM

TimesTen cannot use all of the shared memory for data, it also needs to use shared memory for:

  • Indexes
  • Sequences
  • TempSize [like the Oracle Sort_Area_Size used for things like ORDER BY and GROUP BY statements]
  • The Transaction Log Buffer
  • The PLSQL SGA
  • Database connections

There is a formula to estimate how much memory will be available for data:

  • PermSize + TempSize + LogBufMB + 1 + (0.042 * Connections) + PLSQL_MEMORY_SIZE
  • For example:
    • 32 GB RAM
    • 24 GB Shared Memory
    • 1 GB for the LogBufferMB
    • 256 MB for TempSize
    • 128 MB for PLSQL_MEMORY_SIZE
    • 2000 connections
      • PermSize [MB] = 24576 – (256 + 1024 + 1 + (0.042 * 2000) + 128 )
      • PermSize [MB] = 24576 – 1493 = 23083 = 22.5 GB
    • The larger the RAM on a machine, the larger the effective shared memory can be
    • The larger the shared memory, the larger the effective data [PermSize] that can be stored in the TimesTen In-Memory Database

Disclaimer: These are my personal thoughts and do not represent Oracle’s official viewpoint in any way, shape, or form.