A Solaris tmpfs uses real memory

That title may sound a little self explanatory and obvious, but over the last two weeks I have had two customers tell me flat out that /tmp uses swap and that I should still continue to investigate where their memory is being used.

This is likely because when you define /tmp in /etc/vfstab, you list the device being used as swap.

In the context of a tmpfs, swap means physical memory + physical swap. A tmpfs uses pageable kernel memory. This means that it will use kernel memory, but if required these pages can be paged to the swap device. Indeed if you put more data onto a tmpfs than you have physical memory, this is pretty much guaranteed.

If you are still not convinced try the following.

  1. In one window start up the command
    $ vmstat 2
  2. In another window make a 1gb file in /tmp.
    $ mkfile 1g /tmp/testfile
  3. Watch what happens in the free memory column in the vmstat.

There seems to be a misconception amongst some that a tmpfs is a way of stealing some of the disk we have allocated as swap to use as a filesystem without impacting memory. I'm sorry, this is not the case.

Comments:

Yes and swap command can also lead to confusion because "swap -l" gives the size of physical swap device on disk and "swap -s" gives virtual memory (RAM+SWAP) allocation. You can have a swap device of 50 MB and have more than 500 MB of "SWAP" (in fact virtual mem) reserved ...

bash-3.00# swap -l
swapfile dev swaplo blocks free
/dev/zvol/dsk/rpool/swap 181,3 8 102392 102392

( you know that disk swap is being used when the free column in swap -l command begins to lower )

bash-3.00# zfs get volsize rpool/swap
NAME PROPERTY VALUE SOURCE
rpool/swap2 volsize 50M local

bash-3.00# swap -s
total: 393668k bytes allocated + 528080k reserved = 921748k used, 120040k available

because memalloc (below) is a process doing a very large malloc (to reserve 500 MB of virtual memory) but does not really use it in RAM (low RSS) ...

bash-3.00# ps -o user,pid,vsz,rss,args -p 1749
USER PID VSZ RSS COMMAND
root 1749 410952 696 ./memalloc_x86

With greedy processes like that you need to have big-enough swap devices to fully use your physical RAM and not exhaust virtual memory with unused reservations because Solaris does not overcommit memory allocation like Linux or AIX.

Posted by Pascal R. on March 05, 2013 at 01:04 AM EST #

This is so cool virtual memory allocation and swap allocation is so complicated to understand on Solaris that client needs experts consulting.

Posted by Pascal R. on March 05, 2013 at 01:27 AM EST #

Pascal, first off, I'm sorry that the blog software thought that first submission (and the copies of it) were spam. As you can see I've made sure one of the got pushed out.

What you've written about memalloc() and swap is not quite right. The reason that RSS does not go down is that the pages are not actually allocated until they have something other than 0x0 in them. It still lists under VSZ, but until you use them, there is no need for them to be actually allocated.

On the issue of a tmpfs using physical memory and physical swap, I don't think that that is something that requires an "expert". It's actually discussed quite clearly in tmpfs(7FS).

All that you need to remember is that on Solaris Virtual memory is the sum of physical memory and swap space.

OK, there are a few exceptions to that in the kernel, but the kernel has to treat memory specially anyway. Most of it can't be paged out.

In general "user memory" will either be in one or the other so there is not a need for reservation in swap. The one exception to this rule is Dynamic Intimate Shared Memory (DISM). As part of the design of DISM requires that it can be completely paged out in order to support dynamic reconfiguration, we reserve swap for it.

Getting back to the topic, I wrote this as I found it surprising that I had two customers in the space of two weeks who honestly believed that tmpfs *only* used swap and not physical memory.

Posted by Alan on March 05, 2013 at 10:27 AM EST #

Post a Comment:
Comments are closed for this entry.
About

* - Solaris and Network Domain, Technical Support Centre


Alan is a kernel and performance engineer based in Australia who tends to have the nasty calls gravitate towards him

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today
Links
Blogroll

No bookmarks in folder

Sun Folk

No bookmarks in folder

Non-Sun Folk
Non-Sun Folks

No bookmarks in folder