Monday Mar 12, 2012

Good papers and links for the ZFSSA

So I have a pretty good collection of links and papers for the ZFSSA, and instead of giving them out one-at-a-time when asked, I thought it may be easier to do it this way. Many of the links from my old blog last May no longer work, so here is an updated list of some good spots to check out.

These are for ZFS, in general, not the ZFSSA, but it gives one good insight to how ZFS functions:

Tuesday Jan 10, 2012

Even more ZFSSA announcements

The new announcements for the ZFSSA just keep on coming.

Oracle has released today the 3TB drives for the 7420 and 7320 disk trays. So you now can choose 2TB and 3TB 7,200 RPM drives and 300GB and 600GB 15,000 RPM drives in your 7420 and 7320 systems.

Now, the 2TB drive have a last order date of May 31, 2012, so after that it will be 3TB only for the slower-speed drives.

Also, has anyone checked out the new local replication feature that just came out in the 2011.1.1 software release? I'm going to play with it this week and I'll do a write up on it soon.


Thursday Oct 20, 2011

Shadow Migration

Still not talking about VDEVs? I know, I know, but hey, there's only so many hours in a day, folks, and I do have a life... So something came up this week and I want to talk about Shadow Migration, instead.

Now, built-into the ZFSSA you have both Replication and Shadow Migration. Be sure to use the right one for the right job. Replication is used from one 7000 family system to a different 7000 system. This is important: It can NOT be used on two clustered controllers of the same system. That will mess you up. It is only for other 7000's, and can not replicate to anything other than another ZFSSA. ***UPDATE- This is no longer the case. Replication inside the same system between two clustered controllers has been supported since October 2012.

Shadow Migration, on the other hand, is really handy for both migrating the data from any, non-ZFSSA, NFS source (think from a filer made by someone other than Oracle), or even from a different pool between controllers on the SAME clustered ZFSSA system. This can be very cool when you have an important share on one pool, and you want to move it (and the data inside it) to a different pool. Maybe it's because you want it on your RAIDz2 pool instead of your Mirrored pool. Maybe it's because you want ControllerA in charge of the share but it got made months ago by mistake in the pool owned by ControllerB. I don't care, you just want data from some share, either local to the system or from a NFS share on a different system, to come over into a brand-new share in some pool. Maybe you want to suck in the data from an older, non-Oracle filer, but you know it will take a while, and you want people to be able to still get to the data while the migration is taking place.

Great. That's Shadow Migration. It can get data from both a local source (another share of the same system) or from any NFS mount from anywhere. While the migration is taking place, the original source turns read-only, and users start to mount and use your new share being created. If the data being requested by a user has not been migrated over yet, the ZFSSA will go get it, while continuing to migrate in the background.

Here's how to do a Local Shadow Migration, moving data from a share in one pool to another pool on the same system.

1. Check out the Shadow Migration Service. Under Services, one can change how many threads the background service will use to do the migration. Make sure the green light is on here, while you're at it. **Update: I have been told that our internal team took this down from 8 to 2 for our large (13PB) migrations from various older filers to new 7000s for our Oracle data center. Oracle IT and our Oracle DC is now 100% ZFSSA. 

2. I have a share called Share1A, inside Pool1, which is a mirrored pool. Note that I have about 85MB of stuff in it.
Be careful NOT to choose the replication area from here, or at all, from anywhere. You're not doing replication, remember? 
Do not confuse replication with shadow migration.

3. Now, I don't want that data inside pool1, I really want it in Pool2, which is a RAIDz1 pool. So, switch to Pool2, and create a brand-new share, just like normal.
Change pools with the Pools drop-down in the upper left, then click the plus sign.

 4. Now, in the new Share box, first choose the pool you want the new share to be in, and then be sure to choose "LOCAL" as your data migration source.
Instead of typing in the path to some external NFS share, you will type in the local path of another share on the same system, in this case it's "/export/Share1A"

5. Now it gets cool. Check out my new Shadow1 share. As the migration begins (right away), you will see the progress bar here on the left. You can actually stop it, and even change the source from here, mid-stream (although that would be strange and I don't think I would recommend that).  ***Update: To be fair, it was explained to me that this process may take a while to start. The process may have to read a large amount of metadata before you see the bar move. If you have very large directories in the share, especially at the top, then be patient.

6. When the migration is done (The Local version should go quite quickly), the Shadow Migration section goes away, and you will get an alert message on the top of the screen like this:

7. Also, you can view some Shadow Migration specific Analytics while it's running:

8. Now that it's done, I have 2 shares. My original Share1A, and my new Shadow1 in a different pool with the same data copied over.
I could now delete the first share or pool in order to rebuild the pool a different way. Or, if this was a migration from an older filer, I could re-purpose that filer as a nice planter in my garden.

Monday Sep 12, 2011

New Q3.4.1 code out now for the 7000 ZFSSA family

Code release Q3.4.1 is available now on MOS for download.

This is very exciting news, even if you don't need to upgrade. Why? Because Q3.4.1 contains NO bug fixes. The only thing it does is allows your system to use the new drive trays with 15K speed drives.

You can mix-n-match drive trays in the same system, and create a new, faster ZFS pool using the 15K drives in the same system you already have. You do not want to mix the high-capacity, slower drives with these in the same pool, however. Make sure they are different pools to be most effective. Not everyone needs these faster drives to drive their performance, as the ZFSSA really tries to drive performance with it's huge amount of L1 and L2 cache. However, some people run workloads that either fill up the cache, or by-pass the Logzillas during writes altogether, and the faster, 15K speed spindles will matter a lot to them.


Tuesday Aug 02, 2011

Important - Clean your OS drives. Before you update your systems.

Some folks out there are not reading the readme file before they update, and are running into trouble. Remember the "Important Safety Tip" line in Ghostbusters about not crossing the streams? This is kind of like that.

Have you cleaned up your OS drives lately?

It's important. You should do this every now and then, but especially before you do a big update, such as the update to Q3.4 (which may have you running multiple updates, as explained in my previous post)

You want to remove system updates that are getting old. You may want to keep the previous system software in order to roll back, but do you really need the one before that and the one before that? Are you really going to roll back your 7000 to the system code you used in September 2010? I doubt it. Don't be a hoarder. Delete it. Call Dr. Drew if you need to.
Then, let's check your analytic datasets. These can get big before you know it. If you leave them running all the time, even when you don't need to collect data, you're asking for trouble. Either only run them when you need to collect data (you can do this manually or via a script or alert), or for goodness sake export them as a .CSV file at the end of each month and delete them off the system so they don't get too large. There have been issues reported of problems with the systems once these files got much too large. Part of those issues have been addressed in Q3.4, but you still need to keep it clean.

Very bad things will happen if you let your OS drives fill up. Think of all the atoms of your body moving away from each other at the speed of light. Furthermore, when you do an update, the update will go much more smoothly if you clean up the OS drives, first.
See the readme section below, which I'm sure you forgot to read...

The following section comes from the Q3.4 readme file:

1.4 System Disk Cleanup

Remove any old unused Analytics datasets especially any over 2G in size. The following command can be used to list the datasets.

node:> analytics datasets show

dataset-000 active 745K 2.19M arc.accesses[hit/miss]
dataset-001 active 316K 1.31G arc.l2_accesses[hit/miss]
dataset-002 active 238K 428K arc.l2_size
dataset-003 active 238K 428K arc.size
dataset-004 active 1.05M 2.80M arc.size[component]
dataset-005 active 238K 428K cpu.utilization

Also remove old support bundles or old software updates. It's important that the system disk has at least 20 to 25 percent free space. The following commands can be used to compare the amount of free space to the system disk size. In this case, 289G free / 466G size = 0.62 or 62% free space, which is reasonable. If you have trouble getting enough system disk free space call Oracle Support.

node:> maintenance system disks show
                       profile = mirror
                          root = 4.44G
                           var = 14.3G
                        update = 1.12G
                         stash = 1.38G
                          dump = 131G
                         cores = 29.7M
                       unknown = 15.0G
                          free = 289G


DISK        LABEL      STATE
disk-000    HDD 1      healthy
disk-001    HDD 0      healthy

node:> maintenance hardware select chassis-000 select disk select disk-000 show
                         label = HDD 0
                       present = true
                       faulted = false
                  manufacturer = SEAGATE
                         model = ST95000NSSUN500G
                        serial = 9SP123EQ
                      revision = SF03
                          size = 466G
                          type = data
                           use = system
                        device = c2t0d0
                     interface = SATA
                        locate = false
                       offline = false

Monday Jul 25, 2011

Are you looking for the software updates?

For those of you still not used to finding your way around MOS (My Oracle Support), here are the patch numbers you can use to search for the 7000 ZFSSA patch you are looking for:

12780572 - This is the latest ZFSSA release, version 2010.Q3.4 - from July 22, 2011.
You want this.

11887647 - This is version 2010.Q3.2.1 - This is the minimum version you need to be on in order to then upgrade to Q3.4

12727815 - This is the free 7000 plug-in for OGC (Oracle Grid Control) that allows one to monitor multiple 7000s from OGC. It's small enough that I have provided it here.

12736304 - This is the 7000 NFS plug-in patch for Solaris Cluster version 1. You need this if you have a cluster server running Oracle Solaris and you want to NFS to a 7000. You load this on the cluster, not on the 7000. It's small enough that I have provided it here.

Saturday Jul 23, 2011

2010.Q3.4 software release- July 2011

The newest software release for the ZFSSA, 2010.Q3.4, has been released. You can download it at the My Oracle Support patches & Downloads section.

Check out the release notes of what this update has here:

This is a pretty important update. It fixes an issue with the SIM card on the disk shelves. (The SIM card is the card that connects the disk shelf to the controller via the SAS2 port. It is a FRU and needs this update to stop the "white light special" issue that stops the SIM after a set number of hours. We have been un-seating and re-seating the cards to get around this up until this fix).

Two things to consider:
1. You have to be at code level 2010.Q3.2.1 or higher
2. The SIM firmware update takes about 4 minutes per disk shelf. So a 4-tray system will take about 12 minutes to update the SIM firmware on your system.

Plan your upgrades accordingly. If you are many versions back, you may have to do more than one upgrade to be at the proper level.


Friday Jul 08, 2011

Oracle DB high availability paper for ZFSSA

I read a great whitepaper today on best practices for snapshots and clones of an Oracle database on a ZFSSA.

If you use Oracle software, and are looking at the ZFSSA to store it, you want to check out this paper.  

**Sorry, I had the wrong link... Fixed... Thanks to whomever pointed that out**


Tuesday Jun 21, 2011

New price reduction on some ZFSSA parts

For those of you keeping track, you may have noticed a price reduction today on the ZFSSA Readzillas, Logzillas, and spinning drives. Ask your friendly neighborhood Sales Consultant for a new quote.



Wednesday Jun 08, 2011

Upgrade to Q3.3.1 notes -

Ok, so there is a good reason why you folks want to upgrade. These upgrades fix some great bugs that other clients may have found, and you just have been lucky not to have had effect you yet. Another reason is that at some point when you DO want to upgrade, you may be too far behind to upgrade directly to the newest version. Check out this screenshot. In trying to upgrade to Q3.3.1, the update informs me that I won't be able to do this until I upgrade to Q3.2.0.

Just something to be aware of, so you can plan for additional time if you need to upgrade twice during your maintenance window.



Monday Jun 06, 2011

New Q3.3.1 code- DID YOU KNOW?

Did you know you can also upgrade your 7000 Simulator to the new 2010.Q3.3.1 code that came out last week?
Remember, your 7000 simulator is the real software of a ZFSSA. The only thing being simulated is the hardware.

If you're having a hard time finding the download in MOS, when you get into MOS, just click on the "Patches & Updates" tab, and type 12622199 in the "Patch Name or Number" search box. That will take you right to it. Thanks to Jon B for suggesting this tip.


Friday Jun 03, 2011

New ZFSSA 7000 code release today

The new ZFSSA code was released today, 6-3-11.

  1. Sign in to my oracle support at
  2. Select the "Patches & Updates" tab.
  3. Search by Sun ZFS Storage Appliance product family or by Patch ID (see table below for patch IDs).
  4. Download the zip file to your local system and unzip.
  5. The ak-nas-2010-08-17-3-1-1-1-28-nd.pkg.gz update file and license files are expanded into the All_Supported_Platforms directory.

You can see the release notes here:


This micro release contains significant bug fixes for all supported platforms. Please carefully review the list of CRs that have been addressed and all release notes below prior to upgrading.

This release contains SAS-2 HBA and SAS-2 Disk Shelf Log Device firmware updates for SAS-2 based 7310, 7310C, 7410 and 7410C appliances and all 7120, 7320, 7320C, 7420, 7420C and 7720 appliances. The SAS-2 HBA and the SAS-2 Disk Shelf Log Device firmware will be eligible for firmware updates during the first boot of the appliance after upgrading to this release. The firmware updates are summarized in the following table:

Device Vendor Product ID Description Old Firmware Version New Firmware Version
SAS-2 HBA Sun Microsystems, Inc. Dual 4x6Gb External SAS-2 HBA Dual Port SAS-2 HBA 1.09.02 1.09.03
SAS-2 Log Device STEC ZeusIOPs 18GB SAS-2 Log Device 9002 9004

The SAS-2 HBA firmware update will take approximately 1 minute per SAS-2 HBA. Each SAS-2 Disk Shelf Log Device will take approximately 1 minute to update the firmware. The total firmware update time is dependent on the number of devices being updated. For example, an appliance with 12 SAS-2 Disk Shelves and 16 Log Devices may take 16 minutes to upgrade the firmware. Upgrades may also take longer when the appliance is under load. It is important that customers postpone administrative operations such as cluster failback, reboot, or power down until the system has updated all device firmware. For more information on firmware updates and information on how to monitor them following the first boot after upgrade, refer to the Maintenance:System:Updates Hardware Firmware Updates section of the Customer Service Manual or online help.

This release requires appliances to be running the 2010.Q3.2.1 micro release prior to upgrading to this release. In addition, this release includes upgrade healthchecks that are performed automatically when an upgrade is started prior to the actual upgrade from the prerequisite 2010.Q3.2.1 micro release. If an upgrade healthcheck fails it can cause an upgrade to abort. The upgrade healthchecks help to ensure component issues that may impact an upgrade are addressed. Release specific documentation is provided below to help with upgrades and upgrade healthchecks. Please carefully review it prior to performing an upgrade to 2010.Q3.3.1. It is important to resolve all hardware component issues prior to performing an upgrade.

Tuesday May 31, 2011

7000 Software Update Matrix

7000 Software updates. You can find these both in MOS,
or at
You should be on 2010.Q3.2.1 right now.
New ones coming in next few weeks, so pay attention.

Oracle ZFSSA 7000 Series

These patches can be found in MOS.
Don't even start with me, MOS is easy once you get the hang of it.

 Date  Patch #  Firmware  Type  Code in BUI  
 12/23/2010  10379795  2010.Q3.1.1  Minor  2010-08-17-1-1-1-1-16  
 12/28/2010  10435589  2010.Q3.2  Minor  2010-08-17-1-1-1-1-18  
 2/9/2011  10378803  2010.Q3  Plug-ins  NA- extra software for VSS and OEM  
 3/23/2011  11887647  2010.Q3.2.1  Minor  2010-08-17-2-1-1-1-21 - Current release  
 June 2011  unk  2010.Q3.2.2  Minor  various bug fixes  
 June 2011  unk  2011.Q1  Major  Major release coming  

Some of the new features in 2011.Q1 will include: Active Directory domain controller hot failover, SMB level 2 Oplocks, ZFS enhancements, Replication enhancements, iSCSI and FC enhancements, Datalink configuration enhancements







7000 ZFS Storage Appliance - General info and websites

The ZFSSA is the current name of the oracle 7000 family of products. You can see the public page for them here:

There are just a huge amount of websites, white papers, and blogs regarding this product and the technology that makes it tick, Solaris 10, 11 and ZFS. The following is just a few of them to help get you started.

Software updates:
You need to check here, as the ZFSSA appliance kit gets updated with a minor update about every 3 months and a major update about twice a year. Believe me, you want these. They will either fix bugs that you may have seen (or have not seen yet and don't want to), or give you access to fantastic new features. These will be at no cost to you, as with all software on the ZFSSA, there are no licenses or costs for the software. Very cool.

BigAdmin is a great site to find and download whitepapers on lots of topics, including many for the 7000. Like best practices for setting up SharePoint Server with the 7000.

Power Calculator:
This link takes you to the 7420 calculator, but you can walk the links up at the top to take you to many others

Oracle Enterprise Manager Plug-in:
Did you know you could monitor and get alerts from multiple 7000s using OEM? Oh, and did you know the plug-in was free? Oh, and did you know there is no cost to run OEM and the Oracle database needed to run it, as long as you only use it for OEM???? DID YOU? DID YOU??? I bet you didn't....

How to boot from SAN over FC from a 7000:
Yep, of course you can do that.

Workflow and script depot:
I'm afraid this one is Oracle Internal only, but important for my fellow storage consultants to have. This is Chris' Script Depot.

7000 documentation and training guide :
Sorry, this is another Oracle-internal website

7000 Simulator download -
This is really not a simulator, but the actual, real software for a 7000, just like you bought the real box. The only thing being simulated in VirtualBox (free) is the hardware that the software is running on. It thinks it's a real 7000, it's just running on your PC so it's slow. It can do anything the real box can, even replicate to another 7000. The only parts you won't be able to play with will be clustering and the Hybrid storage pool, as I doubt your laptop has a mix of Readzillas, Logzillas, and SAS2 drives. After you install and set this up, be sure to go get the latest software upgrade and upgrade it, just like its a real box. Because remember, it thinks it is.


How to reset passwords on your 7000 if you want to start from scratch.

How to reset passwords on your 7000 if you want to start from scratch.[Read More]

This blog is a way for Steve to send out his tips, ideas, links, and general sarcasm. Almost all related to the Oracle 7000, code named ZFSSA, or Amber Road, or Open Storage, or Unified Storage. You are welcome to contact with any comments or questions


« July 2016