Monday Jan 14, 2008

Using the Sun Fire X4500 as Network Attached Archival Storage

A paper about a project I completed recently has been published on BigAdmin. The paper is "Configuring the Sun Fire X4500 Server as Network Attached Archival Storage for Symantec Enterprise Vault". The aim of the project was to see if the Sun Fire X4500 could be used as NAS in a Symantec Enterprise Vault environment. The findings were that this worked very well, and performance was excellent!

Software used on the Sun Fire X4500 was SAMBA, Sun StorageTek Storage Archive Manager, and Solaris ZFS.The project involved functional and performance testing, both of which are part of Symantec's comprehensive Self Certification program for Partners.

I went the extra mile, and wrote a configuration tool for the solution called x4500samconfig. The use of the tool is discussed in the paper, and it is available for download here. If you take an X4500 with Solaris 10 8/07 and the unconfigured Storage Archive Manager 4.6 packages installed, this script will have you up and running in minutes. The script configures ZFS, Storage Archive Manager and SAMBA. Some of what the script does "under the hood" is discussed here.

I am going to take the opportunity here to thank Sun's BigAdmin team: the process of publishing an article like this is quite lengthy. Initially, a paper is written by an Engineer (me in this case) for Sun Internal consumption only...this is pretty straightforward, requiring peer review only. Once the internal version is complete, the paper can be submitted for external publication on BigAdmin, and at this point the document must be legally reviewed and professionally edited which can take some time...and it is the BigAdmin team who do, Thanks BigAdmin Team!

Monday Sep 10, 2007

What Does Sun StorageTek Storage Archive Manager Do ?

You can get a nice marketing overview of the value of SAM here.

Digging a little deeper: from an application's perspective a SAM file system behaves like a standard UNIX file system, but under-the-hood it has Hierarchical Storage Management (HSM) features quietly securing your data; moving it between storage tiers based on a custom rule set.....

Life Cycle of a File in a SAM File System

Create: File is written into the SAM file system.

Archive: A file is archived (copied) after a period of time, called its archive age, to archive devices. These can be a directory in a local file system (called a disk archive) or removable media (typically tape devices). Up to 4 copies of a file can be made and they can be made at different times to different archive destinations. e.g. make Copy 1 after the file has been in the file system for 60 minutes, Copy 2 after 120 minutes. If a file is modified after it has been archived then we make new copies of it. As well as age, there are a number of other parameters which can form the Archiving Policies which determine when a file or files are archived; but essentially archiving is a continuous process, there is no backup window for a SAM file system, it takes care of itself.

Release: If the % full of a SAM file system passes a high watermark (e.g. 80% full) the data blocks of the oldest/least recently accessed files are automatically released from the cache until the low watermark (e.g. 70% full) is reached. SAM is often configured to only partially-release files i.e. leave the first N KB behind as a stub, this is so that applications like file managers can scan the first few KB of a file to find out what it is. SAM Policies can be configured so that a file cannot be released (ever), or not release until all of the required copies of the file have been made. Once a file has been released it is often referred to as being off-line.

Stage: The process of bringing a released file back from the archives to the file system is called staging. Disk archives normally stage the file back very quickly, but if the data blocks are coming from a tape archive a worst case delay in the order of minutes can be expected and the application reading the file must be tolerant of that, or its files should never be released.

Unarchive: Copies of a file can have an unarchive age set. e.g. unarchive Copy 1 of a file if it has not been accessed for 6 months. This means that if the file was later opened, Copy 2 would have to be read. This feature can be used to initially archive a file on to disk or fast tape and to later free the space on that media leaving a copy or copies on slower/cheaper media.

Re-cycle: Optional step to re-cycle space from the archives by removing deleted and unarchived files and old versions of files.

Sharing a SAM File System over NFS and CIFS

As well as accessing it locally, you can share a SAM file system using NFS and CIFS (via SAMBA)....

SAM Overview

The only requirement for NFS is that the NFS Client supports correct handling of the NFS3ERR_JUKEBOX error code: if it does not then problems can occur when off-line files are opened as the NFS client will not understand that the file is off-line, and if it is slow to come online (i.e. the file is on tape) NFS may timeout.

SAMBA is a local application so far as Solaris and SAM are concerned; as such, SAMBA will wait for off-line files to come back online just as any other application running locally to SAM would and so, indirectly, CIFS clients must wait for the file to come online also.

SAM Management

Managing SAM used to require a lot of work on the command line: now the File System Manager ships as standard with SAM and provides an easy to use UI from which you can manage all aspects of your SAM environment...but you can still use the command line if you prefer it :-).

File System Manager

SAM: Easy Access to Tiered Storage

Given the features of SAM and its ability to be shared to network clients, we have an easy way for applications to access a Tiered Storage Environment.

SAM elegantly solves the problem of backing up file systems storing very large numbers of files, continuously archiving the file system's content as it changes..and in event of hardware failure, SAM file system recovery processes are very quick and efficient also.

Once a file is archived, we are done...we don't make copy after copy of the unchanged file as a conventional backup product would as it works through a cycle of daily/weekly/monthly we use less tape and secondary storage.

These features makes SAM of particular interest when working with Archiving & Enterprise Content Management (ECM) applications. These create huge volumes of data in the form of files which are probably not going to change once written, but need to be stored safely for long periods of time. I will be writing about work that I have done in this area with SAM and the Sun Fire X4500 in a future posting.

Thursday Jun 07, 2007

Sun StorageTek Storage Archive Manager (aka SAM-FS or just SAM) now on Symantec Enterprise Vault Support Matrix

I am pleased to see that Sun StorageTek Storage Archive Manager 4.5 (aka SAM-FS or just SAM) is now listed as supported with Symantec Enterprise Vault, see page 21 of the Symantec Enterprise Vault Compatibility Matrix.

My next project in this space will be to certify the WORM features of Sun StorageTek Storage Archive Manager 4.6 with Enterprise Vault. Release 4.6 of SAM supports the same WORM interfaces as the Sun StorageTek Compliance Archiving Software does for the Sun StorageTek 5000 NAS Appliances.


Tim Thomas


« August 2016