Thursday Jan 17, 2008

Don't Forget That Storage Plumbing

Few can say that storage is not one very hot topic in the IT sector today.  The internet has created quite a medium for content delivery of podcasts, blogs, webcasts, webinars, etc.  This blog itself is an example.  Storage solutions are abundant, but don't forget that some storage vendors charge you for every neat feature.  Yes you pay for that special hardware that does compression to save storage space.  You also pay for every one of those protocols that you need (iSCSI, CIFS, NFS, etc.) and let's not forget about those important data services for protection (replication, clustering, anti-virus engine, etc.)  Some vendors even charge big money when you grow out of your storage pool and want expansion.  Can you say fork lift upgrade?  Well not all storage vendors want to charge you twice.  As the storage market approaches commodity some of us are getting ahead of the curve.  It is true that all storage companies want to make money... but the difference is leading the way versus fighting against something that will happen anyway.  Even new fast storage hardware will become a commodity as others join with similar offerings. It is important to understand that storage today is heavily reliant on low level software such as device drivers, frameworks and protocols that enable the higher level software in the storage stack to simply work.  If a company can expose and open up the storage stack it has a good possibility to attract not only customers but developers as well... which is what I'll call a community.

At there is a project being done by the community called COMSTAR (Common Multiprotocol SCSI Target).  It is a clever framework which enables protocol plug-ins which speak differnent flavors of storage like Fibre Channel, iSCSI, etc.  For me it is analogous to  the old port and class driver model of my youth.  For more info on COMSTAR click here

Also see what this community member has to say. Some other notables about the storage plumbing at OpenSolaris.  The COMSTAR effort puts code into the kernel for optimization.  It will improve upon the current iSCSI Target already available and in good use today.

Open sourcing the entire storage stack implies the storage plumbing too.  For example it enables this storage stack to work for you rather than you working to pay for that expensive storage solution.

Remember Sun's OS, middleware, database and infrastructure products have the following in common-- they are models of open sourced software that more and more customers are demanding from all vendors and providers.

Thursday Dec 06, 2007

NDMP The Protocol, It's All About The Storage...

Why talk about an industry standard protocol called NDMP (Network Data Management Protocol).  I agree by itself it is a low level medium that is a check box item in the world of storage.  However it is an important piece of plumbing that enables your "house" to function.  Numerous software storage applications, particularly backup software and tape products, utilize this protocol for providing services.  While it is analogous to other storage protocols such as iSCSI and FCoE it is "the" protocol when it comes to basic backup.  Pretty sophisticated backup software choices are out there all driven by this basic protocol.  This includes multiple configuration choices such as local backup, 3-way and DAR. 

  • SUN is a working member of the SNIA community to advance the adoption of industry standards. We will be contributing source code to the SNIA effort to help update the SNIA software from NDMP v3 to NDMP v4.

  • SUN is fostering its own OpenSolaris Storage Platform community in cooperation with SNIA and other industry standards bodies to implement and enhance storage industry standards.

  • SUN plans to incorporate the SNIA software being created by the NDMP software TWG into OpenSolaris when it is approved by the SNIA membership.

The storage stack of OpenSolaris keeps getting stronger.  While NDMP is a robust service for backup, when it comes to management of massive amounts of data your mileage may vary.









While the above picture shows the NDMP protocol in application it does not speak to the entire pyramid structure on the right.  Backup can help you with the performance/cost curve of storage classes, but true storage archive management (SAM) is the one storage application that can automatically manage multiple tiers of storage according to policies crafted by the environment.   Policies are usually driven of of the metadata (the data on the data that you store) which allows you to move data between classes of storage (e.g. expensive FC disks, cheaper SATA disks and magnetic tape) based on file size, access frequency, creation date, etc.  OpenSolaris has released initial SAM/QFS code sources and will continue to do so until the entire SAM/QFS code base is available to the community.  Equally exciting are the OpenSolaris projects ADM and MMS which are bringing storage archive management to the ZFS file system.  Couple all these activities together make one extremely busy storage community working with OpenSolaris.  Given all the technologies are open source code bases you have the ability to leverage with your own code even if it is proprietary.  Fueling the opensource community benefits not only better innovation but more customer choice as well.




The blog of Bob Porras - Vice President, Data, Availability, Scalability & HPC for Sun Microsystems, Inc.


« April 2014