X

Jeff Taylor's Weblog

  • May 18, 2017

Enabling High Performance NAS Data Transfers for RAC on NAS using Solaris 11

On a Solaris 11 server, I found messages like this in an Oracle Real Application Cluster (RAC) node's alert.log:

Tue May 16 10:22:54 2017
Direct NFS: Failed to set socket buffer size.wtmax=[1056768] rtmax=[1056768], errno=-1


The messages applied to the second Oracle RAC node, but similar messages were NOT in the alert.log for the first node. This indicated that there was a problem on the second node: Oracle Direct NFS was not able to set optimal TCP buffer sizes. Setting the larger TCP buffer size is necessary in order to optimize the communication with the Network Attached Storage (NAS), in this case, an Oracle ZFS Storage ZS5-2. I had probably originally set the TCP parameters with ndd and therefore the values didn't survive a reboot.The second node had been rebooted more recently than the first.

Here is a fix that survives reboot.
 
root@first-node:~# ipadm show-prop -p max_buf,recv_buf,send_buf tcp
PROTO PROPERTY              PERM CURRENT      PERSISTENT   DEFAULT      POSSIBLE
tcp   max_buf               rw   2097152      --           1048576      1056768-1073741824
tcp   recv_buf              rw   1056768      --           256000       2048-2097152
tcp   send_buf              rw   1056768      --           49152        4096-2097152

root@
first-node:~# ipadm set-prop -p max_buf=2097152 tcp

root@
first-node:~# ipadm set-prop -p recv_buf=1056768 tcp

root@
first-node:~# ipadm set-prop -p send_buf=1056768 tcp

root@
first-node:~# ipadm show-prop -p max_buf,recv_buf,send_buf tcp
PROTO PROPERTY              PERM CURRENT      PERSISTENT   DEFAULT      POSSIBLE
tcp   max_buf               rw   2097152      2097152      1048576      1056768-1073741824
tcp   recv_buf              rw   1056768      1056768      256000       2048-2097152
tcp   send_buf              rw   1056768      1056768      49152        4096-2097152

 
 
root@second-node:~# ipadm show-prop -p max_buf,recv_buf,send_buf tcp
PROTO PROPERTY              PERM CURRENT      PERSISTENT   DEFAULT      POSSIBLE
tcp   max_buf               rw   1048576      --           1048576      1048576-1073741824
tcp   recv_buf              rw   256000       --           256000       2048-1048576
tcp   send_buf              rw   49152        --           49152        4096-1048576

root@
second-node:~# ipadm set-prop -p max_buf=2097152 tcp

root@
second-node:~# ipadm set-prop -p recv_buf=1056768 tcp

root@
second-node:~# ipadm set-prop -p send_buf=1056768 tcp
 
root@
second-node:~# ipadm show-prop -p max_buf,recv_buf,send_buf tcp
PROTO PROPERTY              PERM CURRENT      PERSISTENT   DEFAULT      POSSIBLE
tcp   max_buf               rw   2097152      2097152      1048576      1056768-1073741824
tcp   recv_buf              rw   1056768      1056768      256000       2048-2097152
tcp   send_buf              rw   1056768      1056768      49152        4096-2097152

 


References:

Be the first to comment

Comments ( 0 )
Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.