Networking wierdness between two S10 boxes
By tdw on Nov 24, 2006
The problem manifested itself as extremely poor NFS performance and 'scp' - where the SPARC box was the client and the x86 box the server, but only in that direction. scp to the x86 box was seemingly fine. Strangely enough, ftp worked okay in both directions.
What I found was that when using scp to transfer a file from the x86 box to the SPARC box, it would transfer a few KB, then wait a few seconds, transfer a few KB more, wait a bit longer, ... The waits got longer and longer (roughly doubling every time) making it a pain to transfer a file anything larger than a few KB.
At first, I thought it might be a crypto problem, since ftp appeared to be working fine, but NFS was slow when configured as V3 or V4, so I pretty much ruled that out.
In fact, I reached a dead-end on diagnosis and just tolerated it for the past 6 months - up until 2 days ago.
I happened to mention the problem to a colleague and after some discussion he suggested that maybe the Ethernet NIC on my x86 box was running in half-duplex mode, and simply couldn't keep up with ack's from the SPARC box when transferring lots of data. Sure enough, a little experimentation identified that this was a strong possibility. A quick Google located a discussion on db forums which highlighted my 3COM card as being a little problematic. FTP apparently doesn't wait for ack's, it just pours data down the pipe, which would explain why it apparently worked ok and scp ddidn't.
The new card arrived yesterday, and after some PCI slot reorganisation and a fistful of reboots, the card sprang into life with Garrett's superb 'afe' driver. File transfers are now far, far faster than they were before, in both directions - a transfer of a 320MB file taking just 30s (it took 3.5 minutes yesterday to do the same operation: SPARC -> x86).
I'm happy now and the 3COM NIC is destined for the garbage or maybe eBay....