Tuesday Mar 18, 2014

Configuring COMSTAR to Provide Local iSCSI Storage

Oracle Solaris 11 introduced two storage capabilities that I wasn't aware of until Oracle ACE Alexandre Borges brought them to my attention.

A Solaris 11 system can serve as an iSCSI target that offers storage to other machines, or as an iSCSI initiator to access the storage offered by another iSCSI target. This capability is a real advantage, because any storage offered through the iSCSI protocol is available to an iSCSI initiator as local storage, without the need to use expensive technologies such as Fibre Channel (FC).

Solaris provides this service through a framework named Common Multiprotocol SCSI TARget (COMSTAR). Alexandre Borges shows you how to use it:

Tech Article: Using COMSTAR and ZFS to Configure a Virtualized Storage Environment

How to use COMSTAR to provide local iSCSI storage for any service that runs in Windows, Linux, or Mac OS. It also shows you how to configure authentication using the Challenge Handshake Authentication Protocol (CHAP) to secure the iSCSI storage against forbidden access. Part 1 of a series about ZFS.

About Alexandre Borges

Alexandre Borges is an Oracle ACE who worked as an employee and contracted instructor at Sun Microsystems from 2001 to 2010 teaching Oracle Solaris, Oracle Solaris Cluster, Oracle Solaris security, Java EE, Sun hardware, and MySQL courses. Nowadays, he teaches classes for Symantec, Oracle partners, and EC-Council, and he teaches several very specialized classes about information security. In addition, he is a regular writer and columnist at Linux Magazine Brazil.

More content from Alexandre:

Exploring Installation Options and User Roles in Oracle Solaris 11

Part 1 of a two-part series that describes how Alexandre installed Oracle Solaris 11 and explored its new packaging system and the way it handles roles, networking, and services. This article focuses first on exploring Oracle Solaris 11 without the need to install it, and then actually installing it on your system.

Exploring Networking, Services, and the New Image Packaging System in Oracle Solaris 11

Alexandre walks you through the new way Oracle Solaris 11 manages networking, services, and packages, compared to the way it managed them in Solaris 10.

Articles in Linux Brazil Magazine (Portuguese)

Columns in Linux Brazil Magazine (Portuguese)

More About ZFS and COMSTAR

About the Photograph

Photograph of San Rafael Swell taken in Utah by Rick Ramsey on the way to Java One.

- Rick

Follow me on:
Blog | Facebook | Twitter | YouTube | The Great Peruvian Novel

Wednesday Jan 22, 2014

Cache Algorithms and Other Storage Tricks

This morning I was reading about something called a cache algorithm. According to our friends at TechTarget, a cache algorithm specifies rules for discarding items from a computer's cache. So I started rooting through some of our tech articles to see if I could find any more about it.

I failed.

But I did find a couple of articles about storage, plus a video, that although not specifically about cache algorithms, do talk about storage efficiency. The video interview is new. I recorded it at Oracle OpenWorld, but had not edited until now. The articles have both been published before.

Video Interview: Simplifying the Way You Manage Data

with Art Licht

Simplification, efficiency, and data protection. That's what you get when you manage your data with Oracle Enterprise Manager. Art Licht explains. Recorded at Oracle OpenWorld 2013.

Tech Article: How to Multiply the Capacity of Oracle Storage Systems

by Tom Luckenbach
Hybrid columnar compression can effectively multiply your storage capacity by 10-15 times, increasing performance and reducing your hardware costs. It's available on Oracle's Pillar Axiom storage systems. Tom Luckenbach walks you through the steps to set it up.

Tech Article: How We Improved SAN and NAS Performance with Hybrid Columnar Compression

by Art Licht

A case study showing how Oracle reduced storage space requirements by a multiple of 19 while getting a six-fold increase in database query performance. All thanks to the Hybrid Columnar Compression capability of Oracle Database 11g Release 2. Plus how you can achieve similar results. By Art Licht.

About the Photograph

The photo is real stretch ...
from left to right, my riding partners
JimBob, El Jefe Con Queso, and Da Don.
Snapped at Luckenbach, Texas a couple of years ago,
on the almost famous Ghost Rider Dog Run.

- Rick

Follow me on:
Blog | Facebook | Twitter | YouTube | The Great Peruvian Novel

Thursday Oct 03, 2013

How Does Oracle Make Storage So Freaky Fast?

The sound quality of these videos is not very good because I taped them while people around me were watching the America's Cup, but the content is worth your time. Jason Schaffer, from Oracle Storage Engineering, explains ...

How the ZS3 Storage System is Engineered

by Jason Schaffer (3 minutes)

The ZS3 is the fastest storage system "on the planet." Jason Schaffer explains what makes it so fast, how it was engineered, and what you can do with it.

How the ZS3 Storage Appliance Tunes Itself

by Jason Schaffer (2 minutes)

Jason Schaffer, from Oracle Storage Engineering, explains how the ZS3 Storage System uses the Oracle Intelligent Storage Protocol (OISP) to automatically tune its I/O patterns to make Oracle Database 12c run faster.

How Oracle Makes the ZS3 Storage System Go Fast

by Jason Schaffer (4 minutes)

Jason Schaffer explains how the ZS3 Storage Appliance uses DRAM to get its crazy fast performance. Taped at Oracle OpenWorld 2013.

More Resources About the ZS3 Storage Appliance

Friday Sep 13, 2013

About LTFS - Library Edition

Oracle just launched the T10000D tape drive with its incredible 8.5 TB of native capacity and LTFS-Library Edition (LTFS-LE), which expands the LTFS concept to an entire library. The Oracle T10000D has some neat features that I would like to address in the future, but today I’d like to talk about LTFS-LE since it really is a new concept.

About LFTS-LE

LTFS is an open source specification for writing data to tape on single tape drives. It is supported by Oracle and other tape vendors. The version you can download from Oracle is called StorageTek LTFS, Open Edition (LTFS-OE).

When an LTFS-compatible T10000 or LTO tape is formatted for LTFS, it is split into two partitions. The first partition holds the metadata that tells the user which files are on the tape and where they are located. The second partition holds the files themselves.

Benefits of Using LTFS-LE

There are a few nice benefits for those who utilize LTFS. Most important is the peace of mind that you will always be able to recover your data regardless of your backup application or any other proprietary software because it’s based on an open source specification. It also improves the portability of tape because two parties don’t both need the same application to read a tape. In fact, LTFS has seen tremendous adoption in industries that require the ability to transport large amounts of data.

The limitation with the open source version of LTFS is that it’s limited to just a single drive. Users with even the smallest archives would like to have their entire environment to be LTFS-based. That’s the impetus for StorageTek LTFS, Library Edition (LTFS-LE), but it also serves as a backup application eliminator because of how it’s architected. With LTFS-OE, after you download the driver, a tape looks like a giant thumb drive. LTFS-LE makes the tape library look like a shared drive with each tape appearing as a sub-folder. It’s like having a bucket full of thumb drives that are all accessible simultaneously!

Just as before, you don’t need any additional applications to access files. And end users are almost completely abstracted from the nuances of managing tape. All they need is a Samba or CIFS connection and they have access to the tape library. LTFS-LE is agnostic to corporate security architectures so a system administrator could make some folders (tapes) available to some users while restricting others based on corporate security guidelines.

Security and Performance Considerations

However, security is arguably one of the more straightforward considerations when deciding how to integrate an LTFS-LE implementation into your environment. An additional consideration is to ensure that LTFS-LE can meet your performance expectations. Tape drives are remarkably faster than they are given credit for (the Oracle T10000D can write at 252 MB/sec.), but sometimes networks aren’t designed to handle that much traffic so performance requirements need to be considered accordingly. In addition, it may take some time before a read operation actually starts as the library needs time to mount a tape. As a result, system administrators need to be cognizant of how end user applications will accept response times from any tape storage-based solution.

A final performance consideration is to be aware of how many tape drives are in your library relative to how many users may be accessing files directly from tape. If you have a disproportionately large number of users you may want to consider a more traditional enterprise-level archiving solution such as StorageTek Archive Manager (SAM), which writes files based on the Tape Archive Record (TAR) open source standard.

Ultimately, LTFS-LE provides exciting new opportunities for system administrators looking to preserve files with a format that isn’t dependent on proprietary solutions. It also makes it easy for users who need access to large amounts of storage without a lot of management difficulties. Support for LTFS continues to grow. Oracle is actually one of the co-chairs of the SNIA committee that’s working towards standardizing LTFS. And this is just the start for LTFS-LE as well, as Oracle will continue expanding its capabilities in the near future.

picture of 2008 Harley Davidson FXSTC taken by Rick Ramsey
- Brian Zents

Follow OTN Garage on:
Blog | Facebook | Twitter | YouTube |

Monday Jul 29, 2013

How to Bend Bare Metal to Your Will

photo copyright 2013 by Rick Ramsey

The fins on this 1957 DeSoto were shaped during a time when Americans weren't afraid of offending anyone with their opinions, right or wrong. We have, perhaps, grown a little more introspective, a little more considerate, but our cars have paid the price. They all look alike. Their edges have been worn away by focus groups. They have no personality. They cringe at the sight of their own shadows.

I weep for my adopted country.

Well, if you like classic American cars as much as I do, you may on occasion feel the need to bend bare metal to your will. Here's your chance.

Tech Article: How to Get Best Performance From the ZFS Storage Appliance

Disk storage. Clustering. CPU and L1/L2 caching size. Networking. And file systems. Just some of the components of Oracle ZFS Storage Appliance that you can shape for optimum performance. Anderson Souza shows you how. Go ahead. Give your appliance a pair of tail fins. (Link is in the title.)

Psst:
You can see more unique cars from the Golden Age of American Automobile at the Gateway Automobile Museum. If you can't get to the border between Utah and Colorado to appreciate them in person, like I was fortunate enough to do, you can enjoy them through your browser at http://www.gatewayautomuseum.com/cars-and-galleries/.

- Rick

Follow me on:
Blog | Facebook | Twitter | YouTube | The Great Peruvian Novel

Tuesday Apr 30, 2013

How to Build A Cloud for Family and Friends Using Oracle Solaris 11

image copyright 2013 by Rick Ramsey

When we talk about cloud, we tend to focus on The Cloud. Enterprise. Government. Scalable. Fast. Big. Bigger. Fastest. That's all wicked impressive, but it's not something I can do on a rainy Saturday afternoon. Now, I like and use Dropbox. There are other easy-to-use cloud services out there similar to Dropbox. But my Inner Geek wants his own cloud. Something modest and unassuming. Itty bitty, even. Just for fun. Kinda like putting a race cam on my Ducati 748: I don't need one, but I want to see if I can do it. Turns out it's nowhere near as involved as installing a race cam on a Ducati. And you don't need to get your hands greasy. Suk Kim, Oracle ACE Director, shows how.

How to Build a Web-Based Storage Solution Using Oracle Solaris 11

by Suk Kim, Oracle ACE Director

Combine AjaXplorer, Oracle Solaris 11.1, and Apache Web server to build a cloud-based storage service that is similar to Dropbox. These are the main tasks ... Install Oracle Solaris 11.1. Configure ZFS storage. Install the Apache and PHP packages. Set up Security. Connect to the client. Check ZFS compression and deduplication. That's all it takes. Suk Kim provides the instructions.

(In case it's not clear that the link is in the heading, Laura, you can also click here)

Suk Kim is an Oracle Ace Director for Oracle Solaris in South Korea. He is also chairman of the Korea Oracle Solaris User Network, manager of Oracle Solaris TechNet, manager of the Solaris School community, an adjunct professor at Ansan University, and a senior system and security consultant at NoBreak Co., LTD.

Follow Suk Kim here:

About the Cloud Picture

I took it from my house in Colorado in the summer of 2011 with a cheap Sony camera. 2013 has brought a snowy Spring to Colorado (next storm, on May 1, will drop 6 inches of snow on us), so it's likely we'll see a lot more of these storms in May, June, and July. I need to spring for a better camera so you can see how spectacular these storms are in the high country.

- Rick

Follow me on:
Blog | Facebook | Twitter | YouTube | The Great Peruvian Novel

Friday Apr 26, 2013

Three Goodies About the ZFS Storage Appliance

Today we have three goodies about the ZFS Storage Appliance to share (image removed from blog):

Video Interview: The Top Capabilities of ZFS Storage Appliance Explained

Nancy Hart describes her favorite capabilities about the ZFS Storage Appliance, and Jeff Wright explains how each of them works. They cover Hybrid Columnar Compression, Direct NFS (makes data transfer more efficient), Remote Direct Memory Access, Oracle Intelligent Storage Protocol (database aware of the storage and vice versa), DTrace Analytics to optimize deployments, and more.

Blog: My Personal ZFS Storage Appliance Crib Sheet

We recently published some articles about really cool ways to use the ZFS Storage Appliance, so I spent a little time looking into the darned thing. It's easy to find out what the ZFS Storage Appliance does, but more difficult to find out what its components are. What can I yank out and replace? What can I connect it to? And what buttons and levers can I push? Or pull. So I put together this crib sheet. If you didn't grow up in The Bronx, see wikipedia's definition of crib sheet.

3D Demo

Pop the doors open, pull out the disk shelves, find out what's inside each one. Great demo, and you're at the controls.

Additional Resources

For more technical resources about the ZFS Storage appliance, use any of the four tabs on OTN's Technical Resources Center. And, to see other blogs about Oracle's storage products, select the "Storage" tab under Categories in the right margin, or click here.

- Rick

Follow me on:
Blog | Facebook | Twitter | YouTube | The Great Peruvian Novel

Thursday Mar 28, 2013

Is Tape Storage Still Harder to Manage Than Disk Storage?

source

-guest post by Brian Zents-

Historically, there has been a perception that tape is more difficult to manage than disk, but why is that? Fundamentally there are differences between disk and tape. Tape is a removable storage medium and disk is always powered on and spinning. With a removable storage one piece of tape media has the opportunity to interact with many tape drives, so when there is an error, customers historically wondered whether the drive or the media was at fault. With a disk system there is no removable media, if there is an error you know exactly which disk platter was at risk and you know what corrective action to take.

However, times have changed. With the release of Oracle’s StorageTek Tape Analytics (STA) you are no longer left wondering if the drive or the media is at risk, because this system does the analysis for you, leaving you with proactive recommendations and resulting corrective actions … just like disk.

For those unfamiliar with STA, it’s an intelligent monitoring application for Oracle tape libraries. Part of the purpose of STA is to allow users to make informed decisions about future tape storage investments based on current realities, but it also is used to monitor the health of your tape library environment. Its functionality can be utilized regardless of the drive and media types within the library, or whether the libraries are in an open system or mainframe environment.

STA utilizes a browser-based user interface that can display a variety of screens. To start understanding errors and whether there is a correlation between drive and media errors, you would click on the Drives screen to understand the health of drives in a library. Screens in STA display both tables and graphs that can be sorted or filtered.

In this screen ...

... it is clear that one specific drive has many more errors relative to the system average.

Next, you would click on the Media screen:

The Media screen helps you quickly identify problematic media. But how do you know if there’s a relationship between the two different types of errors? STA tracks library exchanges, which is convenient because each exchange involves just one drive and one piece of media. So, as shown below, you can easily filter the screen results to just focus in on exchanges involving the problematic drive.

You can sort the corresponding table based on whether the exchange was successful or not. You can then review the errors to see if there is a relationship between the problematic media and drive. You may also want to review the drive’s exchanges to see if media that’s having issues has any similarities to other media that’s having problems. For example, a purchased pack of media could all be having similar problems.

What if there doesn’t appear to be a relationship between media and drive errors? Part of the ingenuity of STA is that just about everything is linked, so root causes are easy to find. First, you can look at an individual drive to see its recent behavior, as show on this screen:

From the table you can see that this particular drive was healthy until recently. The drive indicated it needed a cleaning, and somebody performed that cleaning. However, just a few exchanges later, it started reporting errors. In this case, it’s clear that the drive has an issue that goes beyond the relationship with a specific piece of media and should be taken offline. On the other hand, if the issue appears to be related to the media itself, you should identify a method to transfer the data off of the media, and replace the media.

- Brian Zents

Follow OTN Garage on:
Blog | Facebook | Twitter | YouTube

Tuesday Feb 05, 2013

Do YOU Know Where Your Data Has Been?

When you get change at the grocery store, you just don’t know where it’s been. (Image removed from blog.) And frankly, I don’t want to know, but wherever it’s been, it’s been in different environments with different wear-and-tear. If you try to re-use those dollar bills in a vending machine, you might get your candy bar. Or you might not, if the vending machine says your money is unreadable.

You get a less icky feeling about where your transportable storage has been, that is, until data you were expecting is as unreadable as that old dollar bill. Unfortunately, there is no native data integrity checking as data moves across storage landscapes. However the Oracle T10000C Data Integrity Validation (DIV) feature uses hardware-assisted CRC checks to not only help ensure the data is written correctly the first time, but also does so much more efficiently.

Data at rest is generally not an issue for any storage platform. In tape drives, data is protected with read after write verification as it is written, and Error Correction Code (ECC) is added to ensure data recovery once it is on the medium. In addition, a typical tape drive adds Cyclic Redundancy Code (CRC) protection, as soon as a record is received. This ensures the record does not get corrupted while moving between internal memories. Checking the CRC, though, is a time-consuming process that moves through the following steps:

  1. File pulled from disk to be stored on tape
  2. 256-bit CRC generated and stored in a catalog on a server
  3. File sent to tape drive without the CRC and written to a tape cartridge
  4. Upon recall, the file is called from a tape and sent to a server via the tape drive
  5. 256-bit CRC recreated and compared to catalog in the server

This process takes a minimum of 25 seconds to check the CRC on a 4 GB file, assuming a 2:1 compression ratio and a reasonable server workload. If the tape drives were allowed to assist in some of this workload, the processing time could be dramatically reduced. That’s the premise of the Oracle T10000C DIV feature’s hardware-assisted CRC check. The amount of reduction is simply dependent on the amount of trust the user places in the tape drive itself. While a basic model produces a slightly quicker process, the Oracle T10000C DIV process guarantees it will be done efficiently as shown in the table below.

Steps CRC Verification Model #1 Oracle T10000C Verification Model
1 File pulled from disk to be stored on tape File system sends SCSI Verify Command from server
2 32 bit CRC generated and stored with each record on server Tape drive receives command
3 file sent to tape drive - drive checks CRC File and CRC written to tape
4 File and CRC written to tape Upon recall, file and CRC called from tape to be read
5 Upon recall, file and CRC called from tape to be read Tape drive checks the 32-bit CRC
6 File and CRC checked in tape drive SCSI Verify command and status returned to server
7 32 bit CRC re-created and checked in hardware (Intel)  
Time MINIMUM 14 seconds to check the CRC on 4 GB file (2:1 compression ration) MAXIMUM 9 seconds to verify the CRC on 4 GB file (2:1 compression ratio) independent of server workload

Obviously, built-in-the-drive, end-to-end integrity checking can be much less resource intensive than having to read an entire file to verify that it is still good. Any 32-bit CRC check can be done as specified in ANSI X3.139. This is the same CRC used in the Fibre Channel Protocol and the Fiber Distributed Data Interface (FDDI) for optical transmissions. As a result, the generation polynomial is readily available. While this is a standard interface CRC, it is important to note that this check can be performed outside the interface protocol. In addition, the drive also can generate and use a CRC in the Intel CRC32c format.

Supporting hardware-assisted CRC checking can be as simple as sending a specified SCSI mode select command to turn on the checking. When the Oracle T10000C drive is in its DIV mode, the last 32 bits of any record are treated as a CRC and used to check the integrity of each record. If the CRC check fails, a write error is reported to allow the application to resend the record. A bad record will never be written to tape. If the CRC is correct, that CRC is stored with the record on tape and checked every time the record is read. All of this is done with zero performance loss on the tape drive. If a deferred write error has been reported to the application, the application can determine which record was in error using multiple methods. The recovery is completed when the application resends the previously failed record and the remainder of the data records.

If the drive is being utilized with CRC checks during a subsequent read operation, the CRC will be appended to the record. Verification of the file’s data integrity then is completed with a read verification. In other words, when a drive reads data having a CRC stored along with a record, it will output the CRC appended to the record. This allows the application or driver to perform its own data integrity checks to ensure, months or even years after recording, that the data has not been corrupted. The Intel CRC32c format allows very fast CRC processing and checking by the application. The user application, or driver, can use hardware-assisted CRC checks as follows:

  • Write with hardware-assisted CRC checks and read with hardware-assisted CRC checks
  • Write with hardware-assisted CRC checks and read in normal mode
  • Write in normal mode and read in hardware-assisted CRC checks mode (Note: In this case, the read CRC, which is generated by the drive on the fly, was not stored on tape.)

Another advantage of writing a tape in hardware-assisted CRC mode is the ability of the tape drive to use the Verify command to check an individual record, one file, multiple files, or the entire tape, without having to send all the data to the application to verify the validity of that data. This can be done because the hardware-assisted CRC is recorded on the tape with each record, and the tape drive has the ability to verify each record with that CRC. Because it is only 32 bits, checking only the CRC saves valuable processing resources and time. Ultimately, hardware-assisted CRC checking can have the following options:

  • Verify any record (up to 2MB)
  • Verify entire file (collection of 2MB records)
  • Verify N number of files
  • Verify N number of files of variable record size
  • Verify entire tape with one command
  • Verify mixed mode tape (hardware-assisted CRC check records and non-hardware-assisted CRC check records)
    • A hardware-assisted CRC check check is not made on non-hardware-assisted CRC check records
    • The drive must be in the correct DIV mode for the records it is verifying

- Brian Zents

Follow the OTN Garage:
Blog | Facebook | Twitter | YouTube

Wednesday Jan 09, 2013

How to Treat an NFS File As a Block Storage Device

source

Wim actually beat me in blogging about this feature while I was on vacation, but I'd like to add a little more background about dm-nfs, which I gathered from our kernel developers:

What is dm-nfs?

The dm-nfs kernel module provides a device-mapper target that allows you to treat an NFS file as a block device. It provides loopback-style emulation of a block device using a regular file as backing storage. The backing file resides on a remote system and is accessed via the NFS protocol.

The general idea is to have a more-efficient-than-loop access to files on NFS. The device mapper module directly converts requests to the dm device into NFS RPC calls.

dm-nfs is used transparently by Oracle VM's Dom0 when mounting NFS-backed virtual disks. It essentially allows for asynchronous and direct I/O to an NFS-backed block device, which is a lot faster than normal NFS for virtual disks. The Xen block hotplug script has been modified on OVM to look for files which are on NFS filesystems. If the file is on NFS, OVM uses dm-nfs automatically, otherwise it falls back to using the regular (but slower) loop mount method.

The original dm-nfs module was written by Chuck Lever. It has been supported and used by Oracle VM since version 2.2 and is also included in the Unbreakable Enterprise Kernel for Oracle Linux.

Why this feature matters

This feature creates virtual disk devices (LUNs) where the data is stored in an NFS file instead of on local storage. Managed networked storage has many benefits over keeping virtual devices on a disk local to the physical host.

A sample use case is the fast migration of guest VMs for load balancing or if a physical host requires maintenance. This functionality is also possible using iSCSI LUNs, but the advantage of dm-nfs is that you can manage new virtual drives on a local host system, rather than requiring a storage administrator to initialize new LUNs on the storage subsystem. Host administrators can handle their own virtual disk provisioning.

For durability and performance, dm-nfs uses asynchronous and direct I/O so all I/O operations are performed efficiently and coherently. Guest disk data is not double cached on the underlying host. If the underlying host crashes, there's a lower probability of data corruption. If the guest is frozen, a clean backup can be taken of the virtual disk, as you can be certain that its data has been fully written out.

How to use it

You use dm-nfs by first loading the kernel module, then using dmsetup to create a device mapper device on your file. The syntax is very similar to the dm-linear module.

The following sample code demonstrates how to use dmsetup to create a mapped device (/dev/mapper/$dm_nfsdev) for the file $filename that is accessible on a mounted NFS file system:

nblks=`stat -c '%s' $filename`
echo -n "0 $nblks nfs $filename 0" | dmsetup create $dm_nfsdev

Now you can mount /dev/mapper/$dm_nfsdev like any other filesystem image.

- Lenz Grimmer (Oracle Linux Blog)

Website Newsletter Facebook Twitter

Thursday Jul 12, 2012

Smaller/faster: what's not to like?

200 MB DiskpacksOne might think that things like disk space and even computer speed had become irrelevant. After all, our cell phones have more computing power and storage than million dollar computers of fifteen years ago. (Stop me if I’ve told this story too many times: 25 years ago we sold a terabyte of disk using the big 200 MB disk platters. So, that was a total of over 4,000 disks. Since this is ancient history, the details are hazy, but I do remember that it involved a sizable facility for the storage alone and an even more sizeable commission for the sales rep. Those were the good ol’ days!)

The truth is, there is always an opportunity to take advantage of more resources. Indeed, we are in the era of big data and it would seem that our big limitation now is the speed of light. Rather than brute force, clever engineers continually come up with better ways of doing things. The RDBMS world has tended to think in terms of rows, but there is a new trend to organize it in columns instead. Wikipedia has a great summary of the pros and cons worth taking a look at, if this is new to you. In a nutshell, columnar databases can provide real performance advantages for data warehouses.

Oracle’s Hybrid Columnar Compression technology is nicely described in this paper. Long time storage specialist Art Licht has written a paper about a study he did, explaining How We Improved SAN and NAS Performance with Hybrid Columnar Compression with some remarkable results: 19x reduction in required storage space and an average 6x increase in database query performance.

Art provides specifics on how to do this using the Pillar Axiom Storage System and Sun ZFS Storage Appliance, with detailed test results. This is an article you don’t want to miss: a real hands-on description that quickly brings you up to speed with the technology and its application in the real world. Cache Accesses

–Kemer

Friday Mar 09, 2012

My Personal Crib Sheet for the ZFS Storage Appliance

Question: What do the F22 Raptor and the ZFS Storage Appliance have in common?

Answer: They bend time. They compress distance. And they both come with their own simulator.

We recently published some articles about really cool ways to use the ZFS Storage Appliance (see below), so I spent a little time looking into the darned thing. It's easy to find out what the ZFS Storage Appliance does, but more difficult to find out what its components are. What can I yank out and replace? What can I connect it to? And what buttons and levers can I push? Or pull.

So I put together this crib sheet. If you didn't grow up in The Bronx, see wikipedia's definition of crib sheet.

What Have We Published Recently?

What the Heck Is It?

It is Oracle's main NAS system for enterprise environments. In case you don't already know, NAS (Network Attached Storage) is simply a storage system designed to be shared by several servers on a network. Instead of each server having its own storage, which would make sharing files wicked slow, you put all your storage on your NAS system, and let all the servers access it fast. Plus, it's much easier to manage. Shoot, you can even store your boot environments on your NAS system so that if one of your servers dumps core, you can reboot it from the NAS system.

It comes in three variations:

  • 7120 - for small and medium size installations - 3.3 TB to 177 TB raw capacity
  • 7320 - mid-range storage for the enterprise - cluster option - up to 288 TB raw capacity - Hybrid Storage Pools with up to 4 TB of optimized cache
  • 7420 - For virtualized environments requiring multiple data services and heterogeneous file sharing - single or cluster - up to 1.7 PB of raw capacity

What Makes It Special?

  • It's wicked fast (see F22 Raptor, above).
  • It's got management software that makes it easy to administer.
  • Its Hybrid Storage Pool Design recognizes I/O patterns and places data in the storage media that will provide best performance for that data, whether DRAM, flash, or disk.
  • Hybrid Columnar Compression reduces storage footprints for NAS-based databases from three to five times.
  • DTrace analytics help you diagnose performance and networking bottlenecks
  • Fault Management Architecure (FMA) identifies faults and automatically re-routes traffic around them.
  • When you need more capacity, you can add:
    • DRAM, cache, or I/O ports for more resources
    • Disk shelf units for greater total capacity
    • Flash drives for faster performance.
  • You can get it in a dual-cluster configuration for high availability.
  • It provides a variety of RAID protections to balance capacity, protection, and performance requirements of your applications.
  • It's waaaaay cheaper than an F22 and doesn't require all that cryptic back and forth with those moody Air Traffic Control people.

Details here.

What's In The Box?

When I asked, I got the usual "Well, I could tell you what's in the box, but then I'd have to shoot you." Turns out they don't want me messing with it. Or you messing with it. The darn thing is built from off-the-shelf components, but the value-add comes from the way they're tuned to work together. So if you, Mister Curiosity, decide to pop open a terminal and run ssh into Solaris, you'll see a message notifying you that if you continue with your wayward ways you'll void your warranty. Ack! Like the good ol' boys from the Georgia Satellites like to put it...

She said, "No huggee, no kissee
Until you make me a wife."
Oh, my honey, my baby
Don't put my love upon no shelf
She said, "Don't hand me no lines
And keep your hands to yourself."

Here's what you really need to know: It's a specialized server with a processor, memory, and disk drives. Loaded with a highly tuned version of Oracle Solaris and other software goodies. But don't think of it that way. Think of it as remote storage. That's all. A box with:

  • Two types of storage:
    1. Filesystem, such as CIFS, NFS, ZFS, etc.
    2. Block, allocated as a Logical Unit (LUN)
  • Connections for a wide variety of network protocols
  • Two sysadmin toolkits:
    1. BUI (boo! boo!)
    2. CLI (yay! yay!)
  • Analytics to help you monitor its performance.

Connections? What Can I Connect It To?

For starters, you can connect it to the other servers on the network, through the stock Infiniband HCA's. That's part of what makes it wicked fast. But you can also connect it to other devices through industry-standard network protocols, including:

  • Infiniband
  • Fibre Channel
  • NFS
  • Common Internet File System (CIFS)
  • Internet Small Computer System Interface (iSCSI)
  • NDMP (Network Data management Protocol so it can participate in remotely-coordinated automatic backups
  • A Virus Scan Service
  • NIS naming, LDAP directory, and Microsoft Active Directory services for centralized management of users, groups, hostnames, etc.

What Administration Tasks Does It Require?

Details vary by model and your needs, but basic administration consists of:

  • Defining the storage allocated to each server
  • Making it available to the servers (sharing)
  • Migrating data
  • Integrating it with other applications
  • Taking snapshots
  • Monitoring performance with DTrace Analytics
  • The usual backups, diagnostics, and housecleaning tasks for any server or storage system

Any Examples of What To Use It For?

Turns out you can do lots of cool things with the ZFS Storage Appliance. A partial listing:

For More Information

The Best American Country Song of All Time?

Is it the best? That's debatable. But it's certainly one of my favorite renditions of a country song, from one of my favorite movies of all time.

- Rick Ramsey (with special thanks to Andrew Ness)

Website

Newsletter

Facebook

Twitter

Tuesday Jan 31, 2012

How Much Do YOU Know About Your Server?

Before you run off half-cocked and purchase one of Oracle's Sun servers from some guy selling them out the back of his Escalade, you should be armed with three types of information:

What Servers Are Available?

This handy-dandy tool reveals just the right amount of information about Sun servers, letting you drill down at a pace that the average sysadmin with an IQ of 160 can absorb. Three clicks and you'll have all the info you need.

Which Version of What Software Runs on Whose Server?

Well, almost all the info you need. OTN's Software Stacks list the versions of the OS, firmware, systems management tools, virtualization technology, diagnostic tools, and middleware that runs on each of Oracle SPARC T-series and M-series servers, as well as Sun Blade and Sun Carrier-Grade servers.

What Support Is Offered for Each Server and for How Long?

Even if you are buying your servers from Hugo in the Escalade, you will want to know what type of support Oracle offers and for long we'll offer that support. This page spells that out for not just our servers, but all our hardware. You can also get to this page from the leftmost column of the Systems Community Home Page.

- Rick
Website
Newsletter
Facebook
Twitter

Tuesday Oct 18, 2011

Solving the Tape Storage Space Problem

Because I came of age professionally in the shorts-and-tshirt smarter-than-you culture of Silicon Valley, I always assumed tape storage was used only by retired British spies with names such as Baratheon and Brewster and Cameron who lived in dank mansions on the rocky coast of Scotland and still dressed in tweed jackets for dinner. They spent their days engrossed in the struggle to keep their Dunhill pipes lit and their hair piece in place against fury of the North Wind. Every few months a carrier pigeon would arrive from MI5, and Baratheon or Brewster or Cameron would slowly descend the stairs to a back room. A week later he would return with a name written in code, and hand it to a man in a dripping wetsuit and spear gun who would jump off the cliff without looking and swim it back to a submarine waiting off the coast.

Turns out I was behind the times. In fact, tape for archiving has several advantages that make it economically feasible in today's digital pack-rat economy. Such as durability. And much, much lower power consumption. You can read about them in this paper by Horison information strategies:

Tape: The Digital Curator of the Information Age (registration required)

If you're a storage admin or IT manager considering tape, there's another paper that may interest you more. Published on OTN in July, it describes very clearly the limitations of data that is written in a stream to tape, and how Oracle technologies overcome them. For instance, once you write a block of data to a stream of tape, that particular bit of data not only becomes inefficient to target for access, but updates to the data become clumsy and cumbersome. And as tape cartridges grow to store a terabyte of data, the problem becomes even more pronounced.

Oracle's StorageTek In-Drive Reclaim Accelerator avoids this problem by simply breaking up the serial data on a tape into smaller, more manageable chunks that are grouped together and managed as logical volumes. Find out how in this well-written white paper:

How it Works: StorageTek Reclaim Accelerator

For more information about Oracle tape drive products, visit OTN's Tape Storage product page.

- Rick
Website
Newsletter
Facebook
Twitter

Thursday Sep 15, 2011

How Content Management Makes Tape Drives More Efficient

Guest blog by Steve Aaker and Jamie Giovanetto

Oracle StorageTek's Enterprise Library Software (ELS) is the storage management software available for these Oracle StorageTek Tape Storage mainframe products:

In addition to its hardware enablement functions, ELS provides a rules-based content manager that can significantly increase the efficiency of your tape drives and systems. It's called the Library Content Manager or LCM for short, and it is described in the documentation for the ELS software.

For physical tapes, which can be native data or scratch cartridges, Multiple Volume Cartridges (MVCs), or cleaning cartridges and empty (free) storage cells, LCM places the tapes in the best location to:

  • Maximize the efficiency of the enabling software selections
  • Minimize the robotics activity at job mount time.

In the virtual tape environment, LCM controls whether a volume is in Oracle's StorageTek Virtual Storage Manager System buffer or only on an MVC at the appropriate time.

This content management capability can dramatically increase the efficiency of your tape storage.

An Example

Let's take the case where a StorageTek SL8500 modular library system from Oracle is totally full of cartridges and has no content management. All mounts and dismounts for the tapes in the library occur when required, but the question is whether that is sufficient. It is the minimum expected action, to be sure, but is it being accomplished in an efficient manner? If the cartridge to be mounted is in Library Storage Module (LSM) 03 and the drive where it is to be mounted is in LSM 00 (because that is the only location for that type of drive), the mount will take five major robotics actions. At dismount time, because there are no free cells, it will take another five major robotics actions to return the cartridge back to its source cell. In both directions, at least one of these robotic actions will be a move through the entire length of the library. When complete, the cartridge is back in LSM 03 and the drives it can be mounted on are still in LSM 00. Each time the volume is mounted, this scenario is repeated.

Now, let's look at what the same mount would look like in a content-managed StorageTek SL8500 modular library system. First, the cartridge to be mounted would have been placed in LSM 00 by a prior management run, which would have placed it in the LSM where the drives on which it could be mounted are located. Because of this management activity, the mount would require only two robotics actions, and the dismount would require the same. The reduction of three robotics actions, including that move through the entire length of the library, significantly reduces overall robotics time and results in much faster mount time. The cumulative mount times directly affect overall production time, so proper placement of volumes has a significant impact on production performance.

However, you might ask, "Didn't the cartridge have to be moved to the right place at some time? Didn't that take robotics activity?" Of course, it did. The difference is that content management activity would have been done outside the production job's execution time. In addition, the cartridge would have been placed where it can be mounted many times without requiring the many robotics actions required in the unmanaged example.

Note that this is just one of many examples where significant performance improvements can be obtained by active, rules-based content management. At its fullest implementation level, StorageTek LCM can bring scheduled production mounts down to sub-second levels in the virtual environment and, in some cases, to an average mount time that is approximately half the average mount time for an Oracle StorageTek SL8500 modular library system in the physical environment. Combined with its other capabilities and the underlying hardware and enabling software environment, Oracle's StorageTek Library Content Manager offers great enhancements to the automated tape environment.

- Steve Aaker and Jamie Giovanetto

Wednesday Aug 10, 2011

Join OTN Or ...

...The Lemur Gets It!


Turns out Oracle wants more sysadmins (Linux, Solaris, systems, storage, or network) to become official, bonafide, full-blown OTN members. I explained that sysadmins aren't really the "joining" type, but I lost. Oracle wants more sysadmins to join OTN. Period.

So I've been wondering how I could convince the more reluctant among you to become official, bonafide, full-blown OTN members. After all you, your bookie, and your bookie's mother-in-law can read our technical articles, view our OTN videos (may take time to load), and visit just about every part of the OTN Systems website without signing up for anything.

But there is a bunch of very cool stuff you can't do unless you're a member. This month I'll tell you about one.


Download Software For Free

You can't download our software for free unless you're an OTN member.

I know Oracle's license terms are not the same as Sun's were, but you still get to download and horse around with world-class software for free. If you're anywhere within a decade of your mid-life crisis, you'll clearly remember when you had to actually pay a lot of money for good software. All we ask is that you be honest about when you deploy our software. That's only fair.

For all the details, read the OTN developer license.

You can read about other benefits of membership here.

So, if you really want that lemur to have a future, sign up here, check the "Oracle Technology Network" box under "My Community Memberships," and identify yourself as a sysadmin.

And while you're at it, sign up for our newsletter. It'll highlight the best content we've published over the previous month, in case you weren't paying attention.

- Rick
Website
Newsletter
Facebook
Twitter

Thursday Apr 07, 2011

Cloning for Dummies

Clone TrooperIf it seems like we have published a lot of articles around the Sun ZFS Storage Appliance, it is because we have. Busy Oracle engineer and frequent contributor Sridhar Ranganathan has handed us another jewel: Oracle Database Cloning Solution Using Oracle Recovery Manager and Sun ZFS Storage Appliance. What a great pair of topics!

Let's forget for a minute that I work for Oracle, the leading database company in the World: that is a relatively new phenomenon. The truth is I've always known that databases dominate the solutions that computer hardware enables. Indeed, I vividly remember the early 1980s, when the emerging personal computer market was propelled largely by two primitive database applications: for balancing checkbooks and saving recipes. When Apple announced their new soft-sectored floppy drive, we had to get onto a waiting list to buy 360 KB of random access disk for our puny databases. The CEO of General Dynamics, where I worked at the time, discovered this power and the company was transformed almost overnight from analog to digital.

There are many important things DBAs do to secure their hefty compensation (had I anticipated that trend, I would have started off as a DBA, rather than as an actuary. Wait! I'm afraid there wasn't even the acronym "DBA" way back then...) One of these activities is that of cloning databases, which is done for a variety of reasons, including development, testing, and training without disrupting the actual database itself. As Sridhar points out, the Sun ZFS Storage Appliance provides an ideal platform for performing database cloning. It comes with a user-friendly interface for ease of management, a full set of data services for business continuity and disaster recovery purposes, multi-protocol support to cater to any infrastructure, analytics for monitoring and resolution purposes, and a hybrid storage pool for faster response time for test, development, and QA activities. With unlimited snapshots and cloning possibilities, many concurrent database instances can be launched for various purposes without impacting the production database.

This paper gives you a good look at the power of the graphical front end, includes examples, recommendations, best practices, and sizing considerations. As I said, this is another jewel, not to be missed.

- Kemer

Tuesday Mar 15, 2011

Long Title, Quick Start!

Linux RAC ClusterClustered applications are the keystone to highly available environments. Where you have clustering, you usually have databases – and where you have clustered databases, it is hard to avoid Oracle Real Application Clusters (Oracle RAC).

Clustered databases aren't generally known for ease of setup. The Sun BluePrints Program was orginally created over ten years ago to focus on this topic specifically and numerous books and white papers were a mainstay. We once tried to write a book on this topic in six months and this was so complicated that it couldn't be done. However, technology has progressed and it is now much easier to create powerful solutions without all the pain-and-suffering.

As witnessed by Sridhar Ranganathan's and Jeffrey Wright's latest opus: Oracle9i Real Application Clusters Database Quick Start Install Guide for Sun ZFS Storage Appliance Using Oracle Linux. When we say "Quick Start," we mean it: long title, quick start! In the fewest pages possible they take you through everything you need, from downloading and installing the software, to setting up the disk, and creating the database itself.

No reason to shy away from clustering any longer.

- Kemer

Monday Mar 14, 2011

Did I Repeat Myself? Did I Repeat Myself?

RepeatThere are many aspects to optimizing storage utilization. We usually think in terms of compression: packing the bits into the minimal space. However, have you ever considered how often we save the same data multiple times? Like that amusing picture that everyone in the office saves a personal copy of. It all adds up.

Deduplication – one of those geeky terms that is efficiently self-descriptive – solves the problem by removing duplicated data. Frequent contributor Jeff Wright gives us the lowdown in Sun ZFS Storage Appliance Deduplication Design and Implementation Guidelines. Approaches to deduplication vary in both when and how: the when can be synchronous or asynchronous, the how can be block or file level.

The data deduplication feature provided in the Sun ZFS Storage Appliance is available with Software Release 2010.Q1. This feature is implemented to provide synchronous block-level deduplication and is designed to be applicable to any data stored on the appliance. Jeff's article provides practical application and performance guidelines, along with a list of known issues and limitations.

The Sun ZFS Storage Appliance is one powerful and nifty device. You can tell from the number of interesting articles we are publishing on it that there is a lot under the hood.

- Kemer

Wednesday Mar 02, 2011

Sharing the Breadbox

A Better BreadboxIf the Oracle Solaris ZFS is the greatest file system since sliced bread, then, well, the Sun ZFS Storage Appliance is the ultimate breadbox. (I had to work long and hard on that analogy, so stay with me...) The Oracle Solaris ZFS Datasheet summarizes it nicely:

Anyone who has ever lost important files, run out of space on a partition, spent weekends adding new storage to servers, tried to grow or shrink a file system, or experienced data corruption knows that there is room for improvement in file systems and volume managers. Oracle Solaris ZFS is designed from the ground up to meet the emerging needs of a general-purpose local file system that spans the desktop to the datacenter

The Oxford American Dictionary defines an appliance as 'a device or piece of equipment designed to perform a specific task.' We have all the pieces to build powerful storage appliances that marry this powerful file system with an optimal combination of both disk and flash storage. Implied in the term appliance is generally ease-of-use, and we also have an incredibly slick graphical front-end (formerly known as Fishworks) that ties it all together. It's not a surprise that the Sun ZFS Storage Appliances have been a hit.

Of course, to be a true network appliance, you have to manage competing protocols and file services. While some of us have lived in the world of NFS for seemingly most of our lives, one must not ignore Microsoft's Active Directory, common to the Windows environment. Supporting this, too, doesn't even break a sweat with ZFS Storage Appliances

Sun ZFS Storage Appliance Rule-based Identity Mapping Between Active Directory and NIS Implementation Guide, by Art Larkin, tells you in detail how you can configure it to support multiple file services. Art explains in detail how the Sun ZFS Storage Appliance identity mapping service manages users of both Active Directory services and Network Information Services (NIS) by associating the Windows and UNIX identities of each user. This allows shares, such as directories or files to which access is controlled by a password, to be exported and accessed by clients using either Common Internet File System (CIFS)/Server Message Block (SMB) or Network File system (NFS) protocols. Even if you aren't concerned about this specific configuration, you will find the paper is an excellent introduction into how painless it is to do non-trivial management through the graphical front end: this 30-page paper is full of illustrations.

- Kemer

Friday Jan 14, 2011

Where'd the Docs Go?

image and preamble removed per request so doc info is easier to spot.

The new home page for all Oracle documentation is the Oracle Technology Network's Documentation Home Page.

If you scroll down a bit, you'll find the old Sun Microsystems documentation under these categories:

  • Java
  • Servers, Systems Management, Workstations, and Related Hardware
  • Storage
  • Systems Software (including Solaris)
  • Enterprise Management (including Ops Center)
  • Virtualization
  • Legacy Sun Products
To help you get where you want to go while you're still getting familiar with the new arrangement, Janice Critchlow and the old Sun doc folks put together this cheat sheet. It's followed by a picture of "something cute" that Janice insisted I include in the blog.

- Rick, Janice, and the Doc Folks

Thursday Dec 30, 2010

Backing up the Toast

ZFS Storage Appliance

All hardware is vulnerable. You can live in a happy place, dreaming that disaster visits only others, but the day will come when it knocks on your door: usually at the most inopportune time. Think "quarter-end financial accounting!" This is why we back up everything we can. Or should...

Oracle's Sun ZFS Storage Appliances take care of many details for you, but they don't magically save your bacon: you have to work at it. Fortunately, they support the industry standard Network Data Management Protocol (NDMP) to ensure efficient data protection for backup and recovery. The Sun ZFS Storage appliance works with a variety of industry-leading backup applications (such as Oracle Secure Backup, Symantec NetBackup, and EMC Legato Networker) to offer a complete solution for data protection needs.

We have just published a paper that lays it all out for you: what NDMP is, how to configure your Sun ZFS Storage Appliance, and features and limitations in using Oracle Secure Backup, Symantec NetBackup, and EMC Legato Networker. So, before you burn the toast and start a kitchen fire with bacon, take a look at NDMP Implementation Guide for the Sun ZFS Storage Appliance!

- Kemer

Tuesday Aug 03, 2010

Recipe for a Systems Monday

If you're not sure which sessions to attend at Oracle OpenWorld on Monday, Sep 20, see this entry in the Oracle Develop Conference Blog.

 - Rick


About

Contributors:
Rick Ramsey
Kemer Thomson
and members of the OTN community

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
12
13
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today
Blogs We Like