Performance Improvements for the File System Cache in Solaris 10

One of the major improvements in Solaris 10 was the improvement of the file system cache. Here is a quick summary of the perf changes that I grabbed recently using Filebench.

The file system caching mechanism was overhauled to eliminate the majority of the MMU overhead associated with the page cache. The new segkpm facility providesfast mappings for physical pages within the kernel's address space. It is used by the file systems to provide a virtual addres that can be used to copy data to and from the user's address space for file system I/O.

Since the available virtual address range in a 64 bit kernel is always larger than physical memory size, the entire physical memory can be mapped into the kernel. This eliminates the need to have to map/unmap pages everytime they are accessed via segmap, significantly reducing code path and the need for TLB shoot downs. In addition, segkpm can use large TLB mappings to minimize TLB miss overhead.

Some quick measurements on Solaris 8 and 10 show the code path reduction. There are three important cases:

  • Small In-Memory: When the file fits in memory, and is smaller than the size of the segmap cache
  • Large In-Memory: When the file fits in memory, and is larger than the size of the segmap cache
  • Large Non-Cached: When the file does not fit in memory

Measurements show that the most significant gain is a reduction of CPU per system call when a file is in memory (fits in the page cache), and is larger than segmap (which is 12% of physmem on SPARC, and 64MB on x64). Importantly, this is the most common case I've seen with real applications, too.

Random Read of 8kSolaris 8Solaris 10
Small In-Memory (Less than segmap)89us83us
Large In-Memory (Greater than segmap but less then physmem)181us98us
Large Non-Cached236us153us

The throughput gains for a CPU constrained system are shown here:

With Solaris 10 you should expect significant improvements for applications which are I/O intensive into the file system cache. The actual improvement will vary, and will be greater on systems with higher CPU counts. You should also expect to see the cross call count drop (see xcal in mpstat), and a significantly reduced amount of system time.

Technorati Tag: OpenSolaris

Technorati Tag: Solaris

Comments:

Nice Blog. What's your recomendation for maxphys when accessing rdbms data files, such as Oracle, when using fibre connected disk system for batch apps? Appreciate some comments on this parameter. Regards,John.

Posted by john on October 24, 2005 at 09:42 AM PDT #

Hi John,

I posted a followup to your question here. Please let me know if this was helpful.

Richard.

Posted by Richard on November 11, 2005 at 04:17 AM PST #

<a href="http://vclosets.com">closet organizers</a>

Posted by closet organizers on November 29, 2008 at 03:55 AM PST #

Post a Comment:
  • HTML Syntax: NOT allowed
About

rmc

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today