Multipathing in Solaris
By stracy on Mar 15, 2007
I’ve had the unique opportunity at Sun to work on items that were not in the mainstream, For several years we had a team of engineers working on multipathing (MP) drivers on non-Sun OS’s to support our storage. A key to storage sales is the ability to sell your storage with Solaris, of course, but also on other host platforms too – customers want storage to plug and play with any OS they may have.
The experience was great for us, we pushed all sorts of process boundaries inside of Sun several years ago to provide Sun MP drivers on Linux, Windows, HPUX and AIX, but just when all things came together, we began to really see what the industry was doing with MP – each OS for the most part was including a framework for MP embedded in the OS. Windows has the DSM framework, HPUX uses PVLinks, etc. Even Linux had mdadm which provides some basic failover. No cost for these embedded drivers either (at least not above and beyond the OS/HW).
But to add storage still requires intimate knowledge of the storage device. This means code changes and typically kernel code changes, especially related to asymmetric arrays - real code development and a several months of test and bug-fix. A measure of success in the industry in terms of MP support is also how many arrays, especially the popular ones are supported by any MP driver. In addition, the ability to do unique things with an array such as load-balancing and performance improvements are key.
A few years back, we took the Solaris multipathing driver and began a Open initiative supporting 3rd party arrays. As we went through the grind of supporting these arrays – writing specific failover operations, testing and releasing support it became very clear that to support the industry this was going to become a full-time job and tie up a significant amount of developer time.
The industry noticed this too and developed a standard called Target Portal Group access States ( TPGS – section 5.8). This is a wonderful specification that allows automatic pickup and support of any array (asymmetric or symmetric) for multipathing. It requires an investment in your multipathing driver one time.
In Solaris, we’ve invested a great deal into this standard and recently have started working with vendors that are implementing the standard as well as our own Sun devices (you’ll note that the iSCSI target includes this support). The specification is much more explicit on determining the actual state of each path and requires no guesswork on the host side. The array can let the host know which paths are optimal for traffic and which are not, which paths are on standby and which are unavailable. More information than the host has known in the past to provide better decision-making when error occurs (fail-over) or when load-balancing decisions need to be made.
So, you want to get your array supported quickly on Solaris? Implement TPGS and no changes are needed. Shoot. Score!!