Alejandro Vargas' Blog

  • March 13, 2007

The importance of configuring ASM storage with redundant HBA's

Alejandro Vargas
Technical Leader, ACS Global Delivery, Infrastructure & BigData

About a week ago a developer brought to our attention a dilemma, a small 1.5 GB table was executing FTS much faster on his test instance than on the development instance.

Test is a 10g single instance on vxfs, on a 4 cpu v40 with 16GB of memory, running RH4 32 bits.
Dev is a 10g RAC with ASM and each instance is also on a 4 cpu v40 with 16GB of memory, running RH4, in this case 64 bits.

We devoted a lot of time to benchmark performance of FTS on both systems, and also comparing performance of ASM vs FS on the same RAC.

Small to medium sized tables were finishing FTS faster on FS than on ASM. Large to very large tables finished faster on ASM.
Insert was faster on ASM.

But we expected to have faster FTS on ASM in all tests. The investigation led us to a report that stated  'FTS on FS faster than ASM'. The main issue described on the report was a problem of configuration, ASM was configured with only one fiber card and with block devices instead of raw devices.
Once this was fixed FTS on ASM performed as well or faster than on FS.

Based on this information we started an end to end review of our configuration, we also had a single fiber card on the nodes of this RAC.

Take a look at the RAC Detail Design and Best Practices

Production and Development systems should not compromise on best practices, they can be, as in this case, critical to performance and stability.

Join the discussion

Comments ( 2 )
  • Alex Gorbachev Friday, March 16, 2007
    Actually, raw devices interface is being depreciated in Linux as far as I understand. Block devices can provide unbuffered IO if opened with DirectIO option so having filesystemio_options=setall should do the trick.
  • Ofir Manor Sunday, March 18, 2007
    Just to clarify it (I'm reading between the lines):
    I assume that the TEST system with VxFS were using dual HBA and that the DEV system with ASM was using a single HBA.
    I didn't get the bit about block device vs "regular" raw device. Do you mean character device? can you show it?
    ps - maybe you should update the forum...
Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.