Making SAMBA Go Faster.....

I do a lot of work with the CIFS server that we now provide as part of OpenSolaris, but I also still do work with SAMBA as well.

I have been experimenting with a workload where I am accessing 100 files from a Windows Server 2003 CIFS client (Sun Fire V40z). The server is a Sun Fire X4500 running SAMBA. I am doing sequential I/O with a  workload generating tool, and at any time 75 files are open for read and 25 open for write. There are 1million files in the file system comprising 4 TB of data.

I was not seeing great scalability or performance, then some research by a colleague (supported by some folk at samba.org) led me to try enabling Async IO (AIO) on the SAMBA server. This is a standard feature of SAMBA, and has been available in Sun's SAMBA server build since Solaris 10 8/07. From what I have been told , AIO specifically helps the case of client workload scaling w/ threads Vs scaling with many connections; the workload generating tool I am using (vdbench) scales with threads.

To enable AIO add these lines to the global part of your smb.conf :

aio read size = 1
aio write size =1

Then restart SAMBA.

This parameter is defined in bytes. This setting means that any I/O over 1 byte will be handled asynchronously by smbd. There may be reasons for this to be a bigger value in some cases, I don't know.

Without AIO, my previous best result was 42 MB/sec reads and 14 MB/sec writes; with AIO the client could read at 64.4 MB/sec reads and write a 22.1 MB/sec. CPU utilization on the X4500 running SAMBA went up from 15% to 50%.

Your mileage will vary depending on your workload and application, but that is quite a nice boost for just adding two lines to a configuration file :-)

Note: This work was done running the Solaris Express Community Edition snv_89 X86 (aka Nevada) on the Sun Fire X4500. The underlying file system was a ZFS file system provisioned from a RAID-Z storage pool configured as described here. I was reading/writing 8KB blocks.

Comments:

Thank you Tim for your evaluation and benchmarking. Your work helping me much because people oftenly asking for performance issues where I can simply reference them for your testing results.

There are 2 more performance improvements in samba 3.2.x :

- reworked buffer allocation management

- support of the kernel call which is reciprocal to sendfile().

Currently the Sun legals preparing framework for GPLv3 adoption under which license the samba 3.0.x is released.

Adding the new syscall into the SunOS requires the process which is "a bit more" complicated than in Linux kernel but this feature fortunately affects the server uploads only where the server uploads are counted by 25% only on the whole network volume traffic in average.

Posted by jurasek on May 28, 2008 at 01:08 AM BST #

I am having a serious issue with CIFS in the newest version of solaris (developer edition). When I am on a mac or a PC (XP) using Adobe Bridge or Bibble (photo content managers - basically they are probably opening lots of large (~20MB) files at once) the programs work fine and then hang which can sometimes bring the whole system down. But if I just use explorer or finder and copy the whole directory to the local machine then everything is fine (although once on XP while copying a directory the computer hung and then said resource not located (or something similar))

This is a serious issue because I am trying to serve all of my files from this NAS. It is also a serious issue because I transfered all my files over this medium and I dont want to have unexpectedly lost data.

Any thoughts??

I am not an idiot, although I know this post may seem to prove otherwise. If there are a specific set of tests I should run just tell me and I will try to stress test this stuff to figure out what the problem is.

Or should I just wait until the next update? ( i know the CIFS is in BETA)

Any suggestions?

Thanks

-David

Posted by David on June 04, 2008 at 03:53 PM BST #

Hi David. So, I am reading this as an issue with the OpenSolaris CIFS server, not SAMBA. The best support forum for the OpenSolaris CIFS server is cifs-discuss on opensolaris.org. I recommend that you join the forum and ask the question there. Sun CIFS engineers monitor that alias and are very responsive - that is where I ask my questions! HTH, Tim

Posted by Tim Thomas on June 05, 2008 at 08:34 AM BST #

Yes I know this isnt with SAMBA, but it appeared you are doing various sabma/cifs related stress testing and had wondered if you had run into a similar issue as I had with these applications using the same test you describe in this blog post. (but using the CIFS setup you discuss in other blogs you have made)

-Reaves

I have asked this question on cifs-discuss and haven't received any response. Perhaps it is too nebulous of a question.

Posted by David on June 06, 2008 at 08:49 AM BST #

Hi David, I wish that I could be more helpful but I have not seen a problem like this and cifs-discuss is the only "support" route currently. I am disappointed that you have not received any advice...maybe try wording your question more generically. The next release is always worth a try of course. Rgds, Tim

Posted by Tim Thomas on June 09, 2008 at 06:32 AM BST #

Post a Comment:
Comments are closed for this entry.
About

Tim Thomas

Search

Archives
« April 2014
MonTueWedThuFriSatSun
 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
    
       
Today