How to build a private IaaS cloud platform? - Part IV, Shared Storage
By Karoly Vegh on Feb 27, 2013
This is Part IV of the series "How to build a private IaaS cloud platform?", elaborating about the platform requirements to face when building a multipurpose Infrastructure as a Service platform.
In this post we are looking into the requirements the chosen shared storage has to fulfill.
See the Main Summary Post with links to the other sections (requirements analysis for serverplatform, OS, virtualization, etc) here.
Part IV: Shared Storage
So, what are the most important requirements a shared storage has to fulfill to be chosen as an IaaS cloud building block?Requirements list:
- Snapshots/Cloning: Running hundreds of virtual machines in a shared environment quickly triggers the thought of how convenient it is to snapshot a state of an installed VM on the storage and keep its state where the admins can roll back to any time. Also, having snapshots allows one to clone VMs, hence quickly roll out or duplicate new environments. Having differential clones of a 'golden copy' allows one to spare a significant amount of storage
The reason we see snapshotting and cloning as hard requirements in IaaS platforms is that they enable administration, speed up deployment and provide a safety net in a quickly changing environment.
- Deduplication: Another way of keeping storage utilization high is having deduplication on the storage level among the virtual machines installation images. These VMs - most of them sharing probably large amounts of the same binaries - have lots of storage blocks used to store duplicate records. If the storage can deduplicate these, one can practically oversubscribe the storage utilization.
- Replication: Today's shared storage products all offer availability features. Nonetheless, complete outages happen, wether taking loss of power, fire, quakes, vogon spacefleets into account, customers prefer to have disaster recovery sites, where they replicate their productive data - including their current VM snapshots - to.
Replication provides availability of VMs in disastrous events.
- Storage Tiering: Again, costs. Most customers prefer not to have to fill up their storage arrays with solid state disks (SSDs) for those - while delivering staggering performance - are usually bound with higher costs. They prefer to mix SSDs, high throughput and high capacity disks and in some cases Tape in their shared storage implementation and choose to rely on the intelligence of their setup to shuffle data around on demand. That is, they expect storage autotiering within the box for costefficience.
Storage Tiering saves costs, hence it is a usual requirement for an IaaS storage backend.
- Application Support: The ever growing, unsatisfiable demand for I/O performance of today's applications cannot simply be provided by raw amounts of read/written blocks. The storage shall be closer integrated with the applications supporting their features, enabling administration and performance.
- Network Fabric Bandwidth: Often not the disk backend, not the computing frontend, but the connection between them, the network fabric is the performance bottleneck in a platform.
The bandwidth of this shall be high, with as low latency as possible.
- Observability: Complex storage setups run several workloads of different characteristics parellelly. Analyzing storage-behaviour in the storm of I/O requests can quickly get pretty challenging, although manageability, observability is one of the main factors of Storageadministration in terms of efficiense.
Alright, we got this far. We have some important requirements to fulfill. Let me introduce you to the ZFS Storage Appliance in case you were not familiar with it - and also, allow me to show you how it fulfills these requirements:
Sun ZFS Storage Appliance:
How the ZFS Storage Appliance fulfills these requirements is:
- Snapshots/Clones: The ZFS SA, as you probably have guessed, leverages the features of ZFS. In the ZFS world clones and snapshots are self-evident. That is, snapshotting VMs (for later rollback of for cloning) is supported as a matter of course on the ZFSSA.
- Deduplication: The ZFS SA, as you probably have guessed, leverages the features of ZFS. In the ZFS world deduplication is self evident. That is, running hundreds of VMs that are built on more or less the same binaries allows administrators to save storage capacity by deduplicating the storage below them.
- Replication: The ZFS SA, as you probably have guessed, leverages the features of ZFS. In the ZFS world replication is self evident. Systemadministrators will talk about ZFS's send/receive capabilities, that allows them to replicate data from one ZFS pool to another one. The replication framework of the ZFSSA is based on this capability too, enabling customers to replicate important VMs to a DisasterRecovery site.
- Storage Tiering: The ZFS SA, as you probably... you know the drill. ZFS has the Hybrid Storage Pool capability, which allows it to utilize DRAM, SSDs and HDDs in the same pool, caching both read and write data, selftuning the storage performance.
- Application Support: Using the application example databases:
- The Oracle database can connect to NAS storages with Direct NFS, shortcutting the I/O path between database and storage, improving performance
- The ZFS SA is certified for HCC (Hybrid Columnar Compression), a very efficient DB compression method to spare data storage.
- OISP: The Oracle Intelligent Storage Protocol that is going to tag I/O operations with additional information instructing the Storage Appliance to handle diverse I/Os differently
- Network Fabric Bandwidth: Next to the traditional FiberChannel and Ethernet connections the ZFS SA supports connections over InfiniBand too, that as of now provides very low latency with 40gbps raw throughput per cable. Eliminate the network bottleneck.
Yes of course you can run IP over IB. Also, with Oracle Virtual Network (former XSigo) you can virtualize FC connections over IB too.
- Observability: The ZFS SA provides an we-based management interface that contains DTrace Analytics - a live storage-operations/performance analytics framework generating graphs based on filtered events on-the-fly. Have you ever tried to get statistics out of your storage broken down to IP/user/protocol/disk basis? With the ZFS SA's DTrace Analytics it's easy.
Reduce maintenance efforts by enabling your administrators with storage observability!
The ZFS Storage Appliance fulfills the most common requirements by IaaS platforms by providing VM-management features like deduplication, cloning and replication, providing performance enhancement capabilities by Hybrid Storage Pools and InfiniBand support and keeping maintenance efforts by the live observability framework DTrace Analytics.
In the next post we will look into the requirements for a central management platform for IaaS platforms.If questions arise, do not hesitate to comment.-- charlie