Tuesday May 29, 2012

Is Linear Tape File System (LTFS) Best For Transportable Storage?

Those of us in tape storage engineering take a lot of pride in what we do, but understand that tape is the right answer to a storage problem only some of the time. And, unfortunately for a storage medium with such a long history, it has built up a few preconceived notions that are no longer valid.

When I hear customers debate whether to implement tape vs. disk, one of the common strikes against tape is its perceived lack of usability. If you could go back a few generations of corporate acquisitions, you would discover that StorageTek engineers recognized this problem and started developing a solution where a tape drive could look just like a memory stick to a user. The goal was to not have to care about where files were on the cartridge, but to simply see the list of files that were on the tape, and click on them to open them up. Eventually, our friends in tape over at IBM built upon our work at StorageTek and Sun Microsystems and released the Linear Tape File System (LTFS) feature for the current LTO5 generation of tape drives as an open specification.

LTFS is really a wonderful feature and we’re proud to have taken part in its beginnings and, as you’ll soon read, its future. Today we offer LTFS-Open Edition, which is free for you to use in your in Oracle Enterprise Linux 5.5 environment - not only on your LTO5 drives, but also on your Oracle StorageTek T10000C drives. You can download it free from Oracle and try it out.

LTFS does exactly what its forefathers imagined. Now you can see immediately which files are on a cartridge. LTFS does this by splitting a cartridge into two partitions. The first holds all of the necessary metadata to create a directory structure for you to easily view the contents of the cartridge. The second partition holds all of the files themselves. When tape media is loaded onto a drive, a complete file system image is presented to the user. Adding files to a cartridge can be as simple as a drag-and-drop just as you do today on your laptop when transferring files from your hard drive to a thumb drive or with standard POSIX file operations.

You may be thinking all of this sounds nice, but asking, “when will I actually use it?” As I mentioned at the beginning, tape is not the right solution all of the time. However, if you ever need to physically move data between locations, tape storage with LTFS should be your most cost-effective and reliable answer. I will give you a few use cases examples of when LTFS can be utilized.

Media and Entertainment (M&E), Oil and Gas (O&G), and other industries have a strong need for their storage to be transportable. For example, an O&G company hunting for new oil deposits in remote locations takes very large underground seismic images which need to be shipped back to a central data center. M&E operations conduct similar activities when shooting video for productions. M&E companies also often transfers files to third-parties for editing and other activities.

These companies have three highly flawed options for transporting data: electronic transfer, disk storage transport, or tape storage transport. The first option, electronic transfer, is impractical because of the expense of the bandwidth required to transfer multi-terabyte files reliably and efficiently. If there’s one place that has bandwidth, it’s your local post office so many companies revert to physically shipping storage media. Typically, M&E companies rely on transporting disk storage between sites even though it, too, is expensive.

Tape storage should be the preferred format because as IDC points out, “Tape is more suitable for physical transportation of large amounts of data as it is less vulnerable to mechanical damage during transportation compared with disk" (See note 1, below). However, tape storage has not been used in the past because of the restrictions created by proprietary formats. A tape may only be readable if both the sender and receiver have the same proprietary application used to write the file. In addition, the workflows may be slowed by the need to read the entire tape cartridge during recall.

LTFS solves both of these problems, clearing the way for tape to become the standard platform for transferring large files. LTFS is open and, as long as you’ve downloaded the free reader from our website or that of anyone in the LTO consortium, you can read the data. So if a movie studio ships a scene to a third-party partner to add, for example, sounds effects or a music score, it doesn’t have to care what technology the third-party has. If it’s written back to an LTFS-formatted tape cartridge, it can be read.

Some tape vendors like to claim LTFS is a “standard,” but beauty is in the eye of the beholder. It’s a specification at this point, not a standard. That said, we’re already seeing application vendors create functionality to write in an LTFS format based on the specification. And it’s my belief that both customers and the tape storage industry will see the most benefit if we all follow the same path. As such, we have volunteered to lead the way in making LTFS a standard first with the Storage Network Industry Association (SNIA), and eventually through to standard bodies such as American National Standards Institute (ANSI). Expect to hear good news soon about our efforts.

So, if storage transportability is one of your requirements, I recommend giving LTFS a look. It makes tape much more user-friendly and it’s free, which allows tape to maintain all of its cost advantages over disk!

Note 1 - IDC Report. April, 2011. “IDC’s Archival Storage Solutions Taxonomy, 2011”

- Brian Zents

Website Newsletter Facebook Twitter

Thursday Sep 15, 2011

How Content Management Makes Tape Drives More Efficient

Guest blog by Steve Aaker and Jamie Giovanetto

Oracle StorageTek's Enterprise Library Software (ELS) is the storage management software available for these Oracle StorageTek Tape Storage mainframe products:

In addition to its hardware enablement functions, ELS provides a rules-based content manager that can significantly increase the efficiency of your tape drives and systems. It's called the Library Content Manager or LCM for short, and it is described in the documentation for the ELS software.

For physical tapes, which can be native data or scratch cartridges, Multiple Volume Cartridges (MVCs), or cleaning cartridges and empty (free) storage cells, LCM places the tapes in the best location to:

  • Maximize the efficiency of the enabling software selections
  • Minimize the robotics activity at job mount time.

In the virtual tape environment, LCM controls whether a volume is in Oracle's StorageTek Virtual Storage Manager System buffer or only on an MVC at the appropriate time.

This content management capability can dramatically increase the efficiency of your tape storage.

An Example

Let's take the case where a StorageTek SL8500 modular library system from Oracle is totally full of cartridges and has no content management. All mounts and dismounts for the tapes in the library occur when required, but the question is whether that is sufficient. It is the minimum expected action, to be sure, but is it being accomplished in an efficient manner? If the cartridge to be mounted is in Library Storage Module (LSM) 03 and the drive where it is to be mounted is in LSM 00 (because that is the only location for that type of drive), the mount will take five major robotics actions. At dismount time, because there are no free cells, it will take another five major robotics actions to return the cartridge back to its source cell. In both directions, at least one of these robotic actions will be a move through the entire length of the library. When complete, the cartridge is back in LSM 03 and the drives it can be mounted on are still in LSM 00. Each time the volume is mounted, this scenario is repeated.

Now, let's look at what the same mount would look like in a content-managed StorageTek SL8500 modular library system. First, the cartridge to be mounted would have been placed in LSM 00 by a prior management run, which would have placed it in the LSM where the drives on which it could be mounted are located. Because of this management activity, the mount would require only two robotics actions, and the dismount would require the same. The reduction of three robotics actions, including that move through the entire length of the library, significantly reduces overall robotics time and results in much faster mount time. The cumulative mount times directly affect overall production time, so proper placement of volumes has a significant impact on production performance.

However, you might ask, "Didn't the cartridge have to be moved to the right place at some time? Didn't that take robotics activity?" Of course, it did. The difference is that content management activity would have been done outside the production job's execution time. In addition, the cartridge would have been placed where it can be mounted many times without requiring the many robotics actions required in the unmanaged example.

Note that this is just one of many examples where significant performance improvements can be obtained by active, rules-based content management. At its fullest implementation level, StorageTek LCM can bring scheduled production mounts down to sub-second levels in the virtual environment and, in some cases, to an average mount time that is approximately half the average mount time for an Oracle StorageTek SL8500 modular library system in the physical environment. Combined with its other capabilities and the underlying hardware and enabling software environment, Oracle's StorageTek Library Content Manager offers great enhancements to the automated tape environment.

- Steve Aaker and Jamie Giovanetto


Logan Rosenstein
and members of the OTN community


« July 2016
Blogs We Like