Thursday Mar 05, 2009

What Is Open Storage Anyway?

Over the past few months I have been talking to a lot of people about Open Storage. So far I haven't talked to anyone outside of Sun who doesn't first ask the question "What is Open Storage anyway?" or "When you say Open Storage what does that mean exactly?". But defining a concept like Open Storage can be difficult. Here is one definition ...

Open Storage: As a general term, open storage refers to storage systems built with an open architecture using industry-standard hardware and open-source software.

This might be obvious to some, but lets break this down just a little.

Open Source - Using Open Source software is a key differentiator. Most storage vendors use proprietary software for their storage systems. Some might argue that using Open Source is more prone to vulnerabilities since the code is easily available. But in actuality, Open Source software is inherently more secure and more flexible since it is widely tested and can be easily updated to accommodate changes in the industry. Also, in the case of Sun's Open Storage products we fully support them with warranty and support contracts so if there ever is a problem we will step in and fix it.

Open Standards -  Using open standards is important so that you can always get to your data no matter what might happen to the company who makes your storage system. By using standards and protocols that have been widely accepted you can make sure that you can communicate with your storage system even if someone else's proprietary communications method becomes obsolete.

Open Architecture - This refers to using industry standard components instead of proprietary hardware that can only be sourced from a single vendor. Building a storage system this way reduces costs dramatically and allows the system to be upgraded to take advantage of new components and functionality as they become available.

Also, keep in mind that according to the above definition this concept is not Sun specific. Anyone could create an Open Storage solution or product using this definition (take a look at OpenFiler and FreeNAS for example). However, this definition still leaves a lot of ambiguity. It explains the concept, but does not give any detail about the various pieces needed to put together an Open Storage solution.

So, I came up with a framework to define the different areas that comprise Open Storage so that when I talk about it I can relate a given product or component to it's place in the framework. This is how my brain works (your mileage may vary). It is not official Sun marketing speak, although I hope it is adopted since it makes sense to me.

Open Storage Framework

Since it is a framework, I tried to keep it as simple as possible. The framework is divided into two layers: a software layer and a hardware layer. Within each of those layers are three different categories or "buckets" which components can fit into.

It is not a traditional layer model or anything like that. Think of it more like an organizational tool or a recipe.

Now let's take a look at each category and figure out what each one means.

Applications This is the thing you are running. It could be Oracle, Drupal, Sendmail, or any combination of applications. The FISHWorks user interface on Sun's Unified Storage Systems also fits into this category.
Open Utilities This category includes the operating system and all the utilities that the application depends on. Open Solaris, Linux, Gnu Tools, ZFS and Dtrace would all go here.
Open Protocols Open Protocols are free to use and have a specification that is open for all to see. Examples would be NFS, LDAP and iSCSI.
Compute This is where the processing gets done. Not only tradition CPUs go in this category, but also embedded devices. The key here is that they are "off the shelf" components that can be upgraded or changed as technologies change.
Memory/Cache Memory devices or "cache" speed up the communication between the compute and storage layers. This could be as simple as the RAM in a server or could include other products like the new Solid State Drives (SSDs) that are part of Sun's Unified Storage System's Hybrid Storage Pools.
Storage Of course the framework would not be complete without good old traditional storage components like hard disks and tape drives. However, depending on how they are used SSDs could fit into this category as well.

Leaders in the fatest growing and most innovative companies are really getting this.

Here is what Don MacAskill of SmugMug said on his blog ... "A storage device should be just a server with some open-source software and lots of disks. (The “open source” part is important. I’m sick of relying on closed-source RAID firmware). The amount of flexibility, performance, reliability and operational cost savings you can achieve with software RAID rather than hardware is enormous. With real datacenter-grade flash storage devices just around the corner, this becomes even more vital. ZFS makes all of this stuff Just Work, including properly adjusting the write caches on the disk, eliminating the RAID-5 write hole, etc."

Another example is Joyent, Inc. Since the founding of the company four years ago, Joyent has participated in the OpenSolaris storage community to serve its customers better by tapping the community's knowledge and engineering expertise. The company relies on one of the largest OpenSolaris installations in the world to provide a highly scalable, on-demand infrastructure for Web sites. “We've scaled clients up to over 1 billion page views a month using Joyent Accelerators built on ZFS, D-Trace and Containers in OpenSolaris,” says Rod Boothby, vice president of Platform Evangelism at Joyent.

These examples and many others like them seem to bolster the idea that the Open Storage concept is not only viable, but a better way going forward to implement storage systems. Clearly there is room here for interpretation and this framework will probably change over time. Perhaps your ideas about Open Storage are different from mine. I would love to hear them. Leave a comment and let me know what you think.


"Just what is Open Storage anyway?" This is the question I am trying to answer on this site. How am I doing?


« July 2016