X

An Oracle blog about ZFS Storage

Recent Posts

Cloud Infrastructure Services

Introducing Oracle ZFS Storage ZS7-2

Today, we are introducing the Oracle ZFS Storage ZS7-2, available in two configurations: Mid-range - a mid-range unified storage system ideal for use with performance-intensive, dynamic workloads at an attractive price point. High-end - a high-end enterprise unified storage system for workloads demanding extreme performance and scalability at a price point that rivals competitive midrange and high-end systems. Both models are economically compelling enterprise storage solutions for workloads that demand extreme performance, from Oracle Database environments to large-scale virtualized environments to traditional server environments. Oracle ZFS Storage ZS7-2 runs software version OS8.8.0 (2013.1.8.0), with the latest technical advancements from the Oracle ZFS Storage engineering team. This release introduces the following new features: Administrator AD Credentials Average Latency Statistics Customized Banner in BUI/CLI Custom Alerts through Workflows Data Deduplication & Storage Pool Usage I/O Throttling for Filesystems LDAP Server Preference Order REST 2.0 Rest Login Tokens SAS Cabling Fault Diagnosis Scrub Scheduling SNMP MIB Enhancements SNMP AES Privacy Algorithm Storage Pool Encryption Support LDAP STARTTLS on POR389 Username-based Default Directory for FTP Service The OS8.8.0 release is required for the ZS7-2 controllers, but is also available for download for the following Oracle ZFS Storage Appliance platforms: ZS5-4, ZS5-2, ZS4-4, ZS3-4, ZS3-2, 7420 and 7320. OS8.8 feature descriptions and details about the ZS7-2 platform are covered in the Oracle ZFS Storage Appliance: Software Updates (Doc ID 2021771.1) that is available on MOS.  

Today, we are introducing the Oracle ZFS Storage ZS7-2, available in two configurations: Mid-range - a mid-range unified storage system ideal for use with performance-intensive, dynamic workloads at...

The Wonders of ZFS Storage

Group hug and a strong handshake

Are you a DBA? Your job is hard. Are you a Storage Admin? Your job is hard too. How much easier would a group hug and a strong handshake make your life? What if that handshake was between the technologies you manage rather than with your colleagues? That is concept behind deep co-engineering between Oracle 12c Database and Oracle ZFS Storage Appliance...and it will definitely make your life—DBA or Storage Admin—so much easier. Oracle Intelligent Storage Protocol (OISP) lets the Oracle 12c database to tag each and every I/O and allows the Oracle ZFS Storage Appliance to dynamically tune to those hints. With each version OISP has just gotten better and better. OISP 1.0 was introduced to help out DBA’s and Storage Admins reduce time and effort to manually configure and tune storage shares for database deployments. OISP 1.1 took it a step further and the hints carried information on the DBid /PDBid and additional drill down information. This was especially important for troubleshooting issues in Multi-tenant environments. Newly introduced OISP 2.0 now has features designed to optimize critical resource usage (CPU, DRAM) on the appliance. Fair Share Scheduling (FSS) eliminates first-in/first-out (FIFO) method of allocating storage resources on the appliance. This eliminates scenarios where large bandwidth I/O workloads that arrived first would take up resources and force critical I/O operations like cluster heartbeat and synchronous database operations like LGWR to wait causing performance degradations at the application layer. OISP 2.0 introduces separate thread pools for READ, WRITE and Synchronous operations and isolates heavy write I/O operations to its own pool and the does the same with critical synchronous operations. The result is a far more consistent database performance. Not only that: OISP 2.0 introduces negative cache hints through which large block asynchronous I/O are evicted from DRAM freeing up that resource for critical data blocks. Operations like RMAN backups or archive log I/O do not take up blocks in DRAM anymore. This allows operations like backups during the middle of batch schedule without affecting database I/O performance at the appliance. Finally, OISP 2.0 extended analytics lets Storage Admins have deep visibility into database operations right from the storage management interface. This additional information correlates well with AWR reports that DBAs are well versed at. OISP operations can be broken down by database name, database operation , database filetype to help tie out the I/O operations with the DB function and DB that issued it.All very valuable information to do a deep dive on the analytics to troubleshoot performance issues related to I/O performance. Now finally DBAs finally have a storage device that understands them and speaks their language ; ) To see all this co-engineering in action take a a look at this demo. Go give your colleagues a group hug...with OISP, everything is about to get much better!

Are you a DBA? Your job is hard. Are you a Storage Admin? Your job is hard too. How much easier would a group hug and a strong handshake make your life? What if that handshake was between the...

Tips and Tricks for best RMAN Backup and Recovery Performance. The new and spectacular 60TB/h backup and the 62TB/h recovery rates with the all new ZS5-4.

Are you interested in improving your RMAN Backup and Recovery? Specifically,the Level 0 backups plus daily incrementals with your ZFS Storage Appliance? Isyour current vendor not providing the floor space/performance/price/raw capacitythat you need? Are you hungry? Then keep on reading! Just as you would make the perfect finger licking Mexican Taco (Yes, Iam from Mexico!), there is a way to ensure maximum RMAN performance with an OracleZFS Storage and an Oracle Engineered system(X6-2 server). Of course, the level of complexity is probablynot the same but you get the idea ;). BIG NETWORK PIPES! That’s right, make sure your clients and your storagearray are connected with InfiniBand. InfiniBand provides the biggest and mostuser friendly bandwidth pipes when it comes to RMAN backup and recovery (QDR IB 40Gbit). Each of thesecards can allow up to 5.5GB/sec! Additionally, IB is supremely integrated with OracleExadata. Likewise, when preparing yourtaco you must make sure that the tortilla is big enough to fit all of youringredients! High bandwidth capable Servers! What good is it if your servers can’trecover at the speed of your storage array and network bandwidth? At a 80:20split between Data and Reco, for example, most of our customers are hittingserver bandwidth limitation or bad network integration issues (aka notsupporting InfiniBand connectivity). Always make sure that your servers arecompatible with InfiniBand and that they are InfiniBand friendly. Always buy tortillas from your trustedMexican supermarket! Aka..not the commercial ones. Choose your right data profile for you ZFS Pool! Raidz2 provides thebest sequential/streaming IO for this type of workload. With its large vdevs ,this type of ZFS data profile allows for the fastest and largest IO performance. Similar to Flour tortillas whichsupport the heaviest ingredients but are less flexible than corn J . Finally, make sure you add the right ingredients! ZFS software level: recommended minimum of OS8.6.8,required minimum of OS8.6.5 (Avocado and Melted Mozzarella Cheese) ·         Exadata version: recommended minimum of 12.1.2.3.4,required minimum of 12.1.2.2.3 (Freshly made green salsa) ·         Write flash: not utilized (Do not add Nacho Cheese) ·         Read flash: not utilized (Do not add Ghost Pepper) ·         Storage profile: Use Raidz2 data profile, if yourworkload is not 100% streaming or sequential, use Mirror data profile (Flourtortillas) ·         Network: InfiniBand (Big tortillas) ·         Protocol: Oracle Direct NFS(8 hour slow cookedBarbacoa) Awesome marketing performance graph showing our 62TB/h restore and 60TB/h backup rates! Unbelievably, this number was achieved by using HDDs. Thank you for reading and I hope I did not make you that hungry! To learn more about the configuration details on how to achieve thesenumbers, please visit:  Using an Oracle RMAN Recovery Catalog on Oracle ZFS Storage Appliance White Paper  it can be found at: https://community.oracle.com/docs/DOC-1007521 *Result disclosure: The performance information that you see on thisblog is not ffficial or abided by Oracle.   Make sure you join the ZFS Storage User Group: Become an OZUG member!

Are you interested in improving your RMAN Backup and Recovery? Specifically, the Level 0 backups plus daily incrementals with your ZFS Storage Appliance? Isyour current vendor not providing the...

DeDupe 2.0

In the new OS 8.7, I mentioned that Oracle has greatly improved the DeDupe feature on the ZFSSA (ZFS Storage Appliance). Here is a small excerpt from our latest documentation regarding using our DeDupe  with various backup types on a ZFSSA. You can find this whitepaper here: https://community.oracle.com/docs/DOC-1012600***************************************** The ZFS implementation is inline, block-level deduplication, which means that redundant data can be reduced when it is written to disk. The way it works is that a hash is calculated for each block written to storage, whichis then compared to entries in a deduplication table (DDT) of previously written blocks. When an incomingblock matches, it is not written to storage. Instead, a reference pointer to the existing block is used, resultingin reduced storage space for large amounts of backup data.Deduplication 2.0 is targeted specifically for backup use cases. Full or level 0 backups yield the highest rateof deduplicated data.BenefitsThe new design provides that the DDT table is available in memory, and if the size of the table increases, it isavailable on DDT solid-state drives (SSDs), leveraging the existing Hybrid Storage Pool model to scaleperformance. Hybrid Storage Pool is a feature of Oracle ZFS Storage Appliance Overall deduplication 2.0 benefits include the following: Provides excellent data reduction yields for backup use cases Leverages powerful Hybrid Storage Pool for scalable deduplication performance No additional licensing costs for deduplication or compressionData Reduction Feature SummaryReview the following use cases and recommended data reduction features for Oracle ZFS Storage Appliance: Oracle ZFS Storage Appliance RequirementsOracle ZFS Storage Appliance deduplication 2.0 is supported with the following hardware and softwarerequirements: OS8.7 software is required but no separate deduplication license is required. The following platforms are supported and include recommended memory requirements:o Oracle ZFS Storage Appliance ZFS4-4 with at least 1 TB of memory per controllero Oracle ZFS Storage Appliance ZS5-2 with at least 384 GB of memory per controllero Oracle ZFS Storage Appliance ZS5-4 with at least 1.50 TB of memory per controller 3.2 TB SSDs for meta devices (DDT SSDs) and two for every four trays are recommended Oracle Storage Drive Enclosure DE3-24C storage trays for meta devices (DDT SSDs) are required. **************************************** As you can see in the above screenshot, there is now a 3rd column for SSDs. This one is called "META" and is for the drives you dedicate for the DeDupe table. You must have 2 in order for DeDupe to even be able to activate, and you need another 2 for every 4 disk trays in the pool you wish to use DeDupe on.  Note: You do NOT need any dedicated META drives when you're using an all-flash pool. It will just use the all-flash pool for the DDT. 

In the new OS 8.7, I mentioned that Oracle has greatly improved the DeDupe feature on the ZFSSA (ZFS Storage Appliance). Here is a small excerpt from our latest documentation regarding using our DeDupe...

OS 8.7 still coming, and what's up with the OS numbers?

OK, so 8.7 is not quite out yet. That's OK, because I am still on medical leave, too. I hope to be back the same time 8.7 comes. I am now being told the first week of April. There's a good reason. One of the best new features of 8.7 was still being worked on, and they had a choice of releasing 8.7 without it, or just waiting a few weeks and releasing it fully loaded. I think they made the right choice. Now, in my last post, there was a comment from Matt regarding the stupidity of our version numbers, and he was 100% correct that Oracle made it difficult. Well, to be fair, Sun made it difficult, and Oracle is now TRYING to make it better. Take a look at the table below, showing the current minimum supported software versions for the ZFSSA. Note that there are two very different numbering schemes. There is the old, original Sun numbers that start with the year of the major release, followed by numbers showing minor and micro releases after that.  Such as 2013.1.6.8, which is the version you should all be on right now.  Now the team knew this number scheme was difficult and confusing, especially when it's now 2017. It makes it seem that they haven't updated the software in years, and that's not the case at all. So now they are using the much preferred number scheme showing the number of the major release, followed by the minor release number, such as 8.6. OS 8.6 is the 8th major release of ZFSSA, and the .6 means there has been 6 minor releases since we introduced version 8.  You should now only worry about the "8.x" numbers, and forget about the older number scheme. Just be patient for 8.7. This is the most exciting minor release I have ever seen for the ZFSSA. It could almost be called a major release due to the new features it's going to introduce. Why isn't it? Well, because the basic code isn't changing, just new features and bug fixes, so it's technically a minor release, not a major one. That's good news, as it means it shouldn't take too long to upgrade your heads. By the way, Matt, you also mentioned support for your older controllers? Remember, Oracle is committed to support your controllers for LIFE as long as you maintain your support contract. There is no expiration dates for support on any of our hardware. Now, that does not mean they have to upgrade the software for it forever. If you have an old 7110, it still works great, but we're not going to upgrade it's OS anymore. I think that's fair. If you still have a 10-series, it's time to upgrade your hardware! Minimum Fully Supported Software Version per Product vers The following are the minimum and recommended versions of software fully supported on each product. If your software is older than the most recent supported version, please review the release notes from this and previous software releases to determine whether you may be affected by any of the issues addressed or can benefit from newly added features. If so, we recommend updating to the most recent supported software version listed for your product. Some products are supported by more than one current software release. Product Minimum FullySupportedVersion RecommendedVersion LatestVersion ZS5-2 OS8.6.5 (2013.1.6.5) OS8.6.8 (2013.1.6.8) OS8.6.8 (2013.1.6.8) ZS5-4 OS8.6.2 (2013.1.6.2) OS8.6.8 (2013.1.6.8) OS8.6.8 (2013.1.6.8) ZS4-4 2013.1.3.0 (OS8.3.0) OS8.6.8 (2013.1.6.8) OS8.6.8 (2013.1.6.8) ZS3-4, ZS3-2 2013.1.2.13 (OS8.2.13) OS8.6.8 (2013.1.6.8) OS8.6.8 (2013.1.6.8) 7420, 7320, 7120 2013.1.2.13 (OS8.2.13) OS8.6.8 (2013.1.6.8) OS8.6.8 (2013.1.6.8) 7410, 7310, 7210, 7110 2011.1.9.4 2011.1.9.4 2011.1.9.4

OK, so 8.7 is not quite out yet. That's OK, because I am still on medical leave, too. I hope to be back the same time 8.7 comes. I am now being told the first week of April. There's a good reason. One...

Cool new status area

There's lots of reasons to update your ZFSSA code. You should all be doing this. Yes, I have heard all of the excuses why people don't. Yes, I know some of you don't like it, or you're busy, or you don't want to take the 15 minutes of stress to fail-over one controller to another when it has to reboot. Sorry, but I can't agree with you. Oracle ZFSSA engineers push out these updates for a REASON. Often that reason is that the 600 PETABYTES of ZFSSA inside our Oracle customer Cloud services saw some kind of bug or issue, and they want to make sure YOU don't hit the same bug. Also, as in this example, they roll out some very cool and welcome features. Such as the ability we now have to remove a Readzilla or Logzilla disk from a pool! This was much harder to do last year, and now it's a piece of cake (the cake is a lie).Here's another one. Take a look at my new ARC Ratio section on my Status Dashboard :  I can now see my current ARC (Adaptive Responsive Cache) hit ratio, a very important metric that I love to keep my eye on. We are shooting for 99%, or at least keeping it over 85% for a vast majority of the time. If it falls below 80% for any given length of time, then you don't have enough DRAM for the workloads you are running. Talk to your local storage SE about how to improve this.You can change the layout of these status charts, and remove ones you don't need, by going to Settings, then Layout.

There's lots of reasons to update your ZFSSA code. You should all be doing this. Yes, I have heard all of the excuses why people don't. Yes, I know some of you don't like it, or you're busy, or you...

ZFSSA update 2013.1.5.0 is now available

The newest version of the ZFS Stprage Appliance code was released yesterday. Version 2013.1.5.0. This is a minor release, as opposed to a micro release, which means it has a few feature enhancements, as opposed to bug fixes (which are in the micros). Since this release does include all previous micros, its a very good idea to check your systems and upgrade to this latest release if you are behind. You can download it from the "Patches" tab of MOS.Here is the read me: https://updates.oracle.com/Orion/Services/download?type=readme&aru=19853187Here is the "What's New" section from the readme file: LDAP Security ConfigurationIn the user interface, the security-related parameters for LDAP service configuration have been redesigned to only present supported combinations of authentication methods, credential levels, and SSL/TLS options. The "anonymous" credential level does not use an authorization method. The "self" credential level uses Kerberos encryption and the SASL/GSSAPI authentication method. NFS Reserved Ports for SecurityNFS clients must use low-numbered (reserved) ports when the new NFS "reserved ports" property is set in conjunction with the AUTH_SYS authentication mode. Some NFS clients, such as Solaris and Linux, use low-numbered TCP ports by default. Other clients, such as Windows, may require configuration.NFSv4 Numeric String IDsA new property allows NFSv4 clients to use numeric strings for user and group IDs when used in conjunction with authentication type AUTH_SYS. If you do not set this property, user and group IDs are exchanged in the default form, which is user@domain.

The newest version of the ZFS Stprage Appliance code was released yesterday. Version 2013.1.5.0. This is a minor release, as opposed to a micro release, which means it has a few feature enhancements,...

Disk Scrub - Why and When?

What is a "Disk Scrub" and why should you do it?To begin with, It's a good idea to do this about once a month. Have you ever done one, ever? Maybe you don't need to. We will talk about what this is, why you would or would not do it, and how you can automate it if you wish. We're talking about this button, seen here in the Configuration-->Storage page of your ZFSSA: ZFS will automatically protect your data from "Bit rot", something that can happen to ALL forms of storage. Every time ZFS reads a block, it compares it to it's checksum, and automatically fixes it. That's great if you read your data a lot. However, what if you write the data, and then don't look at it again for years? Is it protected from bit rot?Nope.That's where a disk scrub comes in. Disk scrub will read all the VDEVs in the pool, therefor fixing any and all bit rot errors. If you don't normally read your data in the pool, Oracle recommends a disk scrub about every month. This is a very low-priority background process, and I doubt you'll even notice it's happening. Because it's low-priority, it can take anywhere from 1 second to many weeks to complete, all depending on how much data and how busy your ZFSSA is. I can give you some examples. These both came from the Oracle Cloud, which uses many ZFS systems. A ZFSSA with 192 4TB drives, configured as a single RAIDz1 pool, with only 1TB of data currently in it, finished a disk scrub in less than 90 seconds. On the other extreme end, and older 7410 system, with only 256GB of DRAM, and 192 2TB drives, completely full and running a high IO, write-intensive workload around the clock, took many months to complete a scrub!On his excellent blog, Matt Barnson gives some tips on who should use a scrub, and how often:Should I do this?Is the pool formatted with either RAIDZ or Mirror2 configuration? Although these two options offer higher performance than RAIDZ2 or Mirror3, redundancy is lower. (No, I'm not going to talk about Stripe. That should only ever be used on a simulator; I don't even know why it exists on a ZFS appliance.)Are unable to absolutely 100% guarantee that every byte of data in the pool is read frequently?  Note that even databases that the DBAs think of as "very busy" often have blocks of data that go un-read for years and are at risk of bit rot. Ask me how I know...Do you run restore tests of your data less frequently than once per year?Do you back up every byte of data in your pool less frequently than once per quarter?If you answer "Yes" to any of the above questions, then you probably want to scrub your pools from time to time to guarantee data consistency. How often should I do this?  This question is challenging for Support to answer, because as always the true answer is "It Depends".  So before I offer a general guideline, here are a few tips to help you create an answer more tailored to your use pattern.What is the expiration of your oldest backup? You should probably scrub your data at least as often as your oldest tapes expire so that you have a known-good restore point.How often are you experiencing disk failures? While the recruitment of a hot-spare disk invokes a "resilver" -- a targeted scrub of just the VDEV which lost a disk -- you should probably scrub at least as often as you experience disk failures on average in your specific environment.How often is the oldest piece of data on your disk read? You should scrub occasionally to prevent very old, very stale data from experiencing bit-rot and dying without you knowing it.If any of your answers to the above are "I don't know", I'll provide a general guideline: you should probably be scrubbing your zpool at least once per month. It's a schedule that works well for most use cases, provides enough time for scrubs to complete before starting up again on all but the busiest & most heavily-loaded systems, and even on very large zpools (192+ disks) should complete fairly often between disk failures.How can I automate this?There is no easy button to automate this, but you can make a script to do it for you from the CLI. I don't want to put the script commands here and get in trouble with anyone, but you can ask your friendly, neighborhood SE for some help in making one. In the meantime, just push the button in the BUI. Easy. 

What is a "Disk Scrub" and why should you do it? To begin with, It's a good idea to do this about once a month. Have you ever done one, ever? Maybe you don't need to. We will talk about what this is,...

Managing Storage for your 12c Pluggable Databases

A new versionof the Oracle Enterprise Manager plug-in for Oracle ZFS Storage Appliance isout! The latest release brings in deep visibility, all the way down to thestorage system for each pluggable database in Oracle Database 12c installations. Sounds incredibleright? Owning the complete stack – from application to storage layer has helpedus create product enhancements that provide unique value to the end users. Withthe Oracle Enterprise Manager plug-in for Oracle ZFSSA we have done just that, byallowing DBAs to monitor and manage their database storage through a singleglass pane and through an interface they are most familiar with. Oracle ZFSStorage comes with a built-in analytics tool that provides powerful diagnosticcapability to the storage administrator. With the Enterprise Manager plug-in,we have created the ability to pull these rich sets of storage analytics into acentral repository within Enterprise Manager and present them through the EnterpriseManager console. This way the administrator has to monitor only a singleinterface to get a comprehensive view of their environment. The latestrevision of the plug-in adds storage performance monitoring to an individualpluggable database level, enabling a database administrator to easily managecomplex multitenant Oracle Database 12c environments.Let me give you an example. Imagine you are a database administrator managing alarge Oracle Multitenant Oracle Database 12cinstallation. When LGWR writes to disk, all pluggable databases’ workloads arewritten at once, because LGWR works at the container level. If you see a spikein the IOPS to the disk, traditionally, you have little visibility into theeffect of an individual pluggable database on the overall storage performance. Withthe latest revision of the plug-in, you can break down I/O traffic by eachpluggable database within the container and observe the contribution of eachindividual database on the performance of the system. The per-database (or pluggable database) leveldrill downs help a DBA to identify database-related storage issues, with 67percent fewer steps. This new functionality combined with the ability to self-administerstorage resources for the database such as provisioning filesystems, LUNs, changingshare settings for optimal performance, all from the Enterprise Manager enablesthe DBA to take back control of their environment. The Enterprise Manager plug-in for Oracle ZFSSA isone of the outcomes of the deep co-engineering efforts between Oracle Databaseand ZFSSA. With Oracle Intelligent Storage Protocol, Automatic DataOptimization and Hybrid Columnar Compression you can unlock unparalleledperformance and efficiency gains in your 12cenvironment. To learn more about the plug-in, visit https://www.oracle.com/storage/nas/em-plug-in/index.html.

A new version of the Oracle Enterprise Manager plug-in for Oracle ZFS Storage Appliance is out! The latest release brings in deep visibility, all the way down to thestorage system for each pluggable...

This Is Our Time

Storage is at a moment of discontinuity. To put it mildly. The market-leading vendors build reliable storage machines that work really well when attached to one or even several applications.  But they do not, will not and cannot solve the coming challenges.  They are like Richard Gere standing in front a row of EMC Symmetrix boxes. Still handsome? Yes.  Even venerable. But getting old and gray. So, what's happened since Richard Gere first carried Debra Winger out of the factory in Officer and a Gentleman*? VIRTUALIZATION (especially VMware)!!  And its even less predictable cousin CLOUD!! They break the relationship between applications and storage.  Anything that relies on conventional spindles for performance is already dead in the water. In the aftermath, vendors and users alike are scrambling for machines that marry the speed of flash with the affordability of conventional drives.  They want machines that automate data location -- in real time -- because tuning has become a fool's errand.  Lucky for us: We have the EXACT RIGHT MACHINE.  So I'm throwing down the gauntlet: If you are building out a large storage infrastructure in 2015, and aren't at least kicking the tires on our Oracle ZFS Storage Appliance, you are making a fundamental mistake that might take a half-decade to recover from. Yes, my job involves convincing people to implement Oracle ZFS Storage Appliances in virtualization and cloud environments.  I get paid to make you believe what I just said. But I believe it. Because I've seen eyes light up time and time and time again when people try our systems for the first time. We're living in the era of hybrid storage, and we have the best Hybrid Storage Array you can buy. This Is Our Time. For the past few several years EMC and NetApp in particular have been doing a fine job defending current technology. In the past year or so the dam has started bursting -- spectacularly in some areas. Virtualization and cloud workloads break NetApp FAS systems.  And they break EMC VNX. And they break EMC VMAX. And yep, they break EMC Isilon.  And HP 3Par. (etc., etc., for most everything you have.) Anything built to provide IOPS by bundling conventional spindles is NOT designed to live underneath a hypervisor.  There's too much risk that too many virtual machines will try to get to the same 150-350 IOPS device at the same time. So what's showing up to replace them? Hybrid Storage and well-designed all flash systems (and sure, you can put all flash drives in your EMCs and NetApps, but that feels like a bit of a kludge, doesn't it?) But all-flash is the next thing.  It will have its time soon enough. The simple fact that flash vendors today justify their TCO by saying they dedupe better...  Hmmm.  So a flash drive costs less than a SAS drive because they make the data so much smaller?  Maybe I'm missing something, but couldn't you just dedupe the SAS drives, and won't they still be cheaper per gigabyte?  So, let's put silliness aside and simply acknowledge that all-flash has a place today, and may be the default tomorrow, but for now, it remains a very, very expensive place to keep cold data. And my linked clones will reduce data better than your dedupe anyways. And sure, if you shop around a bit you can find an all-flash array that's price competitive with a smallish Oracle ZFS Storage device. But before you buy it, find out how much the same system would cost with 300 TB+ raw.  Or a petabyte.  If you only have 30 TB of raw data,  I'd say take the discount and buy the artificially priced all-flash system.  Just don't grow, and you'll be fine... My point is: Until the cost per gigabyte of flash gets closer to the cost per gigabyte of good ol' spinning rust, this is the era of hybrid storage.  It automates the all of hardest parts of managing storage in a public or private cloud.Seriously, wouldn't either EMC and NetApp kill for a system that: 1.  Eliminates bottlenecks instantly by caching to DRAM everything that gets accessed.  It's precisely the sort of automation that compute clouds require.  Not after action analysis that caches yesterday's hot data. Cache what's hot, and cache it NOW. And if the data's cold, just leave it on SAS drives.  Better AND cheaper!! 2.  Has an embedded OS that knows how to deploy workloads efficiently across ALL of the available processor cores.  Imagine the consolidation you could achieve if you can keep your systems from getting processor bound! (Ponder this: Our little box can handle several thousand typical VMs without breaking sweat). 3.  Finds the noisy neighbor more effectively than any other device that's ever been built (using our storage analytics).  This makes the consolidation safe, and helps keep the systems up.  Absolutely critical for the sort of hyperconsolidation that the other attributes enable. Boom! Boom. And BOOM! So far we've sold a ton, and we're growing at a time when others are declining, and we've formed the backbone of the world's most successful SaaS cloud.  It's the right box at the right moment. We can try to explain it, but almost every time we sell a system in a non-Oracle use case it's because the customer tested their workload -- and then felt their jaw drop to the floor. No need to trust us.  We'll be happy to show you. * Symmetrix came out 8 years later, but I stand by my point.

Storage is at a moment of discontinuity. To put it mildly. The market-leading vendors build reliable storage machines that work really well when attached to one or even several applications.  But they...

Racked Systems and IaaS Acquisition Model Enable Cloud

We've been touting Oracle ZFS Storage as a great solution for cloud and virtualization a lot this year.  One thing that we've learned from our customers in the process (and from our own Oracle cloud folks) is that it's not enough to simply be a good technical solution for the workload.  You also need to deliver the product in a way that simplifies deployment (enterprise customers want this, too, naturally, but cloud providers seem particularly focused on repeatability). Enter factory-racked systems and an IaaS acquisition model. With the introduction of the ZFS Storage Appliance Racked System we enable the public and private cloud by offering IaaS acquisition model. The ZFS Storage Appliance IaaS offering allows you to acquire ZFS Storage by a monthly subscription instead ofactually purchasing the hardware.  The IaaS offering is a low costsubscription based model for ZFS Storage for deployment on-premiseor in Oracle data centers. It provides our customers with an OpEx acquisitionmodel, not CapEX.  Not a lease.  More like paying rent, right down to the fact that we take the hardware back when you're done with it. Nothing new, of course.  You can get these sorts of things from other storage providers.  We are simply continuing to address the needs of the on- and off-premise cloud customer by marrying the perfect storage architecture with better ways to acquire and deploy it. Platinum Services is included. Huge.  As we do with all Oracle Engineered Systems, we patch systems for you, provide 24x7 offsite monitoring, and give you access to our premium support "hotline".  And like we said, it's included (Platinum Services are expected to be available by H2FY2016 for the ZFS Storage Appliance Racked System when attached to an Exadata, SuperCluster or Exalogic).  These days we invest a lot in making Oracle Applications work better.  But the fact that Oracle ZFS Storage has been such an enormous asset in our SaaS and PaaS clouds simply can't be ignored.  And now it's easier than ever to bring all of that home.

We've been touting Oracle ZFS Storage as a great solution for cloud and virtualization a lot this year.  One thing that we've learned from our customers in the process (and from our own Oracle cloud...

Oracle ZFS Storage...An efficient storage system for media workloads

Life as a studiosystem administrator is full of twists and turns, like the plot of the movie projectyou are currently supporting! It’s a typical Monday morning, as you walk in tothe studio. The digital artists are working on the visual effects for theupcoming blockbuster movie, scheduled for a Spring release.  It’s business as usual, until you get calledin for a meeting at noon. The release schedule for the current movie projecthas changed. Instead of a Springrelease, the producer and the director now wish to bring the movie to thescreens three months in advance, right in time for Christmas. To meet theshortened delivery window, they are going to double the number of artists workingon the project (from 100 to 200 artists), with people working overtime to hitthe new deadline. The implication of this decision on your environment is thatnow you need a storage infrastructure that can support about 200 artists. Requestingfor proposals, evaluating vendors and procuring new storage system is timeconsuming. Imagine having astorage system that can easily scale to support large number of users in yourpost-production environment while still delivering consistently highperformance. Oracle’s ZFS Storage Appliance, with its massive number of CPUcores (120 cores in Oracle ZFS Storage ZS4-4), symmetric multi processing OSand DRAM-centric architecture is designed to deliver the user scalability andperformance density that media companies desire, without creating controller ordisk sprawl. A clusteredOracle ZFS Storage ZS4-4 system has the compute power to support 6000+ servernodes in the render farm, while still delivering low latency reads tointeractive digital artists. The massive horsepower built into Oracle ZFSStorage Appliance enables it to deliver equal if not better performance, whencompared to popular scale-out storage systems. The high end enterprise classOracle ZFS Storage has at least 78% higher throughput and 41% better overallresponse time than 14-node EMC Isilon S210 on the SPECsfs2008 NFSv3 benchmark. WithOracle ZFS Storage Appliance, you not only simplify the process of acquiringstorage systems but also achieve higher ROI, with no additional licensing feeas your storage requirements grow. We are at IBC thisyear. If you are a studio system administrator and want to learn more about thesuperior performance and the unmatched cost and operational efficiencies ofOracle ZFS Storage Appliance, please visit Oracle booth in Hall 7 at RAIConference Center, Amsterdam, September 11-16, 2015.

Life as a studio system administrator is full of twists and turns, like the plot of the movie project you are currently supporting! It’s a typical Monday morning, as you walk in tothe studio. The...

Oracle ZFS and OpenStack: We’re Ready … and Waiting

After Day 1 of the OpenStack Summit, the ongoing debate rages as it does will all newish things:  Is OpenStack ready for prime time? The heat is certainly there (the OpenStack-savvy folks will see what I did there), but is the light? Like anything in tech, it depends.  The one thing that is clearly true is that the movement itself is on fire. The "yes" side says "OpenStack is ready, but YOU aren't!!!"  The case being made on that side is: "OBVIOUSLY if you simply throw OpenStack over a bunch of existing infrastructure and "Platform 2" applications, you will fail.  If instead, you build an OpenStack-proof infrastructure, and then run OpenStack on top of it, you can succeed." The "No" side says "That sounds hard! And isn't at least part of the idea here to get to a single, simple, consolidated dashboard? I want THAT because THAT sounds easier." Who is right?  Both, of course.  But the "yes" side essentially admits that the answer is still sort of "no", because the "no" side is right that OpenStack probably still too hard for shrink-wrapped, make-life-in-my-data-center-easier use.  What the "yes" side is really saying is that some of the issues OpenStack solves for today are worth solving for despite the fact that they are hard.  Walmart's e-commerce site is a big example. Here in Oracle ZFS Storage land, we get asked to explain this "yes" or "no" problem to our customers every day (several of whom have presence at the summit), and we tell most of them that the answer is "not yet".  But keep an eye on it, because "yes" will be a very useful thing when it arrives.  For our part, we came here to the Vancouver summit saying the following: We're in! We plan to be full-on implementers of the stack, and have Cinder, Manila, and Swift integrations available already. We have a great architecture for consolidated cloud infrastructure, so as you kick the tires on OpenStack, think about trying it with us. In the language of the Gartner hype cycle, I think OpenStack is entering the notorious "trough of disillusionment" (Gartner doesn't have a specific mention of OpenStack on their curve). That's fine.  All great technological advances must necessarily pass through this stage.  Our plan is to keep developing while the world figures it all out, and to be there with the right answers when we all get to the other side.

After Day 1 of the OpenStack Summit, the ongoing debate rages as it does will all newish things:  Is OpenStack ready for prime time? The heat is certainly there (the OpenStack-savvy folks will see...

Oracle ZFS Storage Intelligent Replication Compression

IntelligentIs Better .... Remotereplication ensures robust business continuity and disaster recovery protectionby keeping data securely in multiple locations. It allows your business to run uninterrupted and provides quick recovery in case of adisaster such as a fire, flood, hurricane or earthquake. Unfortunately, longdistance replication can often be limited by poor network performance, varyingCPU workloads and WAN costs. What’s needed isintelligent replication that understands your environment and automatically optimizesfor performance and efficiency, on-the-fly, before sending data over the wire. Intelligentreplication that constantly monitors your network speeds, workloads and overallsystem performance and dynamically tunes itself for best throughput and costwith minimum impact to your production environment. Oracle ZFS Storage Appliance IntelligentReplication Compression Oracle’s ZFS Storage Replication with IntelligentCompression does just that. It increases replication performance byintelligently compressing data sent over the wire for optimum speed, efficiencyand cost. It monitors ongoing network speeds, CPU utilization and networkthroughput and then dynamically adjusts its compression algorithms andreplication stream thread counts to deliver best replication performance andefficiency, even with varying network speeds and changing workloads. This adaptive intelligence and dynamicauto-tuning allows ZFS Storage Appliance Replication to run on any network withincreased speed and efficiency, while minimizing overall system impact and WAN costs. Oracle’s Intelligent Replication Compression utilizesOracle’s unique Adaptive Multi-Threading and Dynamic Algorithm Selection forreplication compression and replication streams that continuously monitors(every 1MB) network throughput, CPU utilization and ongoing replicationperformance. Intelligent algorithms then automatically adjust the compressionlevels and multi-stream thread counts to optimize network throughput andefficiency. It dynamically auto-tunes compression levels and thread counts tofit changing network speeds and storage workloads, such as high compression forslow/bottlenecked networks and fast compression for fast networks or high CPUutilization workloads. It offers performance benefits in both slow- andhigh-speed networks when replicating various types of data, while optimizingoverall system performance and reducing network costs. IntelligentReplication Compression can lead to significant gains in replicationperformance and better bandwidth utilization in scenarios where customers havelimited bandwidth connections between multiple ZFS Storage sites and the WANequipment (such as WAN accelerator) does not provide compression. Up to 300%increases in replication speeds are possible, depending on network speeds, CPUutilization and data compressibility. Best of all, Intelligent ReplicationCompression comes free with the ZFS OS8.4 software release. WhatAbout Offline Seeding? Oracle’s ZFS Storage Replication is based onvery efficient snapshot technology, with only delta changes sent over the wire.This can be done continuously, scheduled, or on-demand. Intelligent ReplicationCompression makes this very fast and efficient, but, what about the initial orfull replica that could involve sending a very large amount of data to a remotesite? Transmitting very large amounts ofdata long distances over the WAN can be both costly and time consuming. Toaddress this, the ZFS Storage Appliance allows you to “seed” or send a fullreplication update off-line. You can do this by either doing a local copy toanother ZFS Storage Appliance and then shipping it to a remote site, or byusing an NFS server (JBOD/Disk sets) as a transport medium to send to anotherexisting remote ZFS Storage Appliance. Incremental replicas can then be done fast and inexpensivelyover the WAN. This saves both time and money when setting up a remote ZFS DRsite or needing to move large amounts of data efficiently.  Summary Superiorremote replication is all about speed, efficiency and intelligence. Speed, soyou can doit fast. Efficiency, so it doesn’t cost you an arm and a leg in WANcosts. Intelligence, so it dynamically optimizes itself for your ever-changingenvironment to achieve the highest performance at the lowest cost. Oracle ZFS Storage Replication withIntelligent Compression does all of that, and more.

Intelligent Is Better .... Remote replication ensures robust business continuity and disaster recovery protection by keeping data securely in multiple locations. It allows your business to run uninterru...

Oracle ZFS Storage Powers the Oracle SaaS Cloud

On the plane to OpenStack Summit, I was thinking about what we on Oracle ZFS Storage team have been saying about cloud storage, and how Oracle's cloud strategy internally (building the world's most successful Software-as-a-Service company) maps to our thinking. If you haven't followed the SaaS trends, Oracle's cloud has grown well beyond the recreational stage.  We're killing it, frankly, and it's built on Oracle ZFS Storage. The cliche is that there's no clear definition for cloud (or maybe it's that there are a bunch of them). I disagree.  I think that as typically happens, people have done their best to twist the definition to match whatever they already do.  Watch Larry Ellison's CloudWorld Tokyo keynote (there's a highlights video/but watch the whole thing).  At at 22 minutes in, he walks you through how real cloud applications work.   What I'm thinking about relative to storage architecture is this notion that next-generation "cloud" storage can just be a bunch of commodity disks (think Ceph, for example), where you copy the data three times and are done with it.  OpenStack Swift works this way. In the Hadoop/Big Data world, this is conventional wisdom.  But as the amount of data people are moving grows, it's simply hasn't turned out to be the case.  In the cloud, we're seeing the same bottlenecks that plague hyperconsolidation in the enterprise:  Too many apps trying to get to the same spindle at the same time, leading to huge latencies and unpredictable performance.  People are deploying flash, in response but I'd argue that's the blunt force solution.  We've learned at Oracle, and have demonstrated to our customers, that super fast, super intelligent caching is the answer.  Likewise, our friends at Adurant Technologies have shown that once your map reduce operations hit a certain scale point, Hadoop runs faster on external storage than it does on local disk. Turns out that you can't just jump to commodity hardware and expect optimal storage efficiency. EMC and NetApp simply aren't going to explain all of this to you.  From afar, they are hitting the right beats publicly, but look like they are flopping around looking for a real answer.  Their respective core storage businesses (FAS and VNX specifically) are flagging in the face of cloud.  Their customers are going where they can't. And indirectly, they are coming to us.  Whether they are buying Oracle Exadata and Exalogic with Oracle ZFS Storage to turbocharge their core applications, moving to Oracle's massively expanding IaaS/PaaS/SaaS clouds, or discovering how they can get 10x efficiency by putting Oracle ZFS Storage in their own data center, they are moving away from stuff that just doesn't work right for modern workloads. So, we're here at OpenStack, partly to embrace what are customers are hoping will be the long-sought Holy Grail of the Data Center (a single, consolidated cloud nerve center), and we're feeling rather confident.  We have the the right answer, and we know we're getting to critical mass in the market. If you happen to be in Vancouver this week, drop by Booth #P9 and we'll tell you all about it.

On the plane to OpenStack Summit, I was thinking about what we on Oracle ZFS Storage team have been saying about cloud storage, and how Oracle's cloud strategy internally (building the world's most...

Is that purpose-built backup appliance really the best for your database backup and restore?

Dataprotection is a critical component of the job if you have anything to do withIT operations. Regardless of your jobtitle, database administrators and storage administrators alike, know how criticalit is to plan for any type of systems outage. Backup and restore are key components of a data protection and preparingfor a disaster recovery event. Thestorage architecture is a key element in the overall enterprise IT environment,and when focusing on the implementation of disk storage for backup andrecovery, there isn’t just one platform to do the task. And if you are looking at database backups,only Oracle can provide a better database backup platform than storage behemothEMC. You can hearall about the benefits of the Oracle ZFS Storage Appliance from an Oracle salesrep, but having an outside, independent analyst break down the appliance iseven better. In a recent CompetitiveAdvantage report Jerome Wendt, lead analyst and president at DCIG comparedOracle ZFS Storage ZS4-4 to EMC Data Domain 990. His conclusion: “By usingOracle’s ZS4-4, enterprises for the first time have a highly-availability,highly scalable solution that is specifically co-engineered with OracleDatabase to accelerate backups while minimizing storage costs and capacity.” The reportcites a number of advantages that ZS4-4 has over EMC DD990: Higher backup and restore throughput: ZS4-4 can achieve native backup throughputspeeds of up to 30TB/hour and restore to 40TB/hour. EMC DD990 only achieves amaximum 15TB/hour backup rate with no published restore rates available. Amplecapacity to store enterprise Oracle Database backups: ZS4-4 scalesup to 3.5PB. DD990 only scales to a maximum of 570TB. Higherdata reduction rates than deduplication can achieve: The ZS4-4supports Hybrid Columnar Compression (HCC) so it can achieve compres­sion ratesof up to 50x and overall storage capacity reduction of up to 40%. HCC is notavailable with non-Oracle storage. DD990 cannot deliver these high OracleDatabase data reduction rates.. Acceleratesbackup and recovery throughput: OracleIntelligent Storage Protocol reduces manual tuning, expedites backups withZS4-4 and is not available with the EMC DD990. Higheravailability: ZS4-4 isavailable in an optional dual-controller configuration for high avail­ability.The EMC DD990 is only available in a single controller configuration. Moreflexible: ZS4-4 is amulti-function system as opposed to DD990, a single purpose deduplicatingbackup appliance. Inconclusion: “The OracleZFS Storage ZS4-4 provides enterprises the flexibility they need to quickly andefficiently protect their Oracle databases. Shipping in a highly availableconfiguration, scaling up to 3.5PBs and providing multiple storage networkingprotocols with high throughput rates, it combines these features with OracleDatabase’s native HCC and OISP features to provide enterprises with a solutionthat is not depen­dent upon a “one size fits all,” single-function appliancesuch as the EMC DD990.” To find outmore about how DCIG came about their conclusions, take a look for yourself: www.oracle.com/us/corporate/analystreports/dcig-zs44-odb-vs-emc-dd990-2504515.pdf. If you haveany responsibility for the backup and restore of an Oracle database, you owe itto yourself to learn more about how the Oracle ZS Storage Appliance deliversbetter performance to ensure your critical databases can be restored wheneverything is on the line. Worried aboutmanaging your own storage? Oracle hasmade managing the ZFS Storage Appliance easy for the DBA by integrating appliancemanagement into Oracle Enterprise Manager. Learn more on storage management in this short video: https://www.youtube.com/watch?v=JzmGKKXsuSw.

Data protection is a critical component of the job if you have anything to do with IT operations. Regardless of your jobtitle, database administrators and storage administrators alike, know how...

Lights, Camera, and… Action with Oracle ZFS Storage Appliance!

Inspired by therecent Oscars award ceremony, I decided to try my hand at directing and here ismy work in progress script, featuring our star – Oracle ZFS Storage Appliance. Location: EditingStudio Scene1: The on-location camera capturedsome amazing shots! We have rows and rows of creative minds sifting through rawmedia files and trying to make sense of the content. Where do these media fileslive – Oracle ZFS Storage Appliance! While they finalize the shots, inanother part of the studio, VFX artists create stunning visual effects by connectingto the render farm with thousands of servers, which are again supported by our storagestar! A ZS4-4 system delivers low latency read performance - easily handlingmillions of DPX files, while accessed by thousands of render farm servers and seamlesslydelivering content to interactive digital artists. The ZS4-4 posted an SPC-2 resultof 31,486.23 SPC-2 MBPSTM throughput, with streaming mediaperformance at 36,175.91 SPC-2 MBPSTM. Scene2: Meanwhile, in a futuristiclooking IT center, the studio system administrators are engaged in deepdiscussion on how efficiently they can configure the ZFS shares to deliveroptimal performance. They just received information that the on-going projecthas a variety of workload characteristics – the movie was filmed in 4K , butthe VFX will be done in 2K and later scaled up to conform to the originalformat. Not to worry, they can continue planning their summer vacation as OracleZFS Storage is there to save the day. ZFS storage supports adjustable record size (8KB to 1MB), enabling VFXdesigners to choose large block sizes for efficiently handling 2K, 4K and highframe rate (3D) digital media workloads. Scene3: A studio system administrator isabout to break for lunch, when he receives a call from the VFX division. A VFXartist suddenly observes a drop in performance, affecting the animationsequence she was working on. The studio system admin opens up the ZFS Storage’sGUI and goes to Analytics section. Within minutes the hidden problem isdiscovered and necessary action is taken to address the performance issue.Another employee of the month award to add to his office wall! Advanced Storage Analytics in ZFS Storage provide deep insights intoworkload characteristics, offering tremendous manageability advantages and 73%faster troubleshooting vs. NetApp (Strategic Focus Group Usability Comparison Study). Scene4: The IT bigwigs of the studio decidein a fiscal planning meeting: we need a storage solution that delivers thehighest efficiency and performance for the next big movie project. Who do we call? Oracle ZFS Storage delivers high performance without a high price tag. Oracle’shigh-end model, Oracle ZFS Storage ZS4-4 posted an SPC-2 result of 31,486.23SPC-2 MBPSTM throughput at $17.09 SPC-2 Price-PerformanceTMdelivering “million dollar performance at half the price,” while Oracle’smidrange model, Oracle ZFS Storage ZS3-2, achieved 16,212.66 SPC-2 MBPSTM withoverall #1 ranking in price-performance at $12.08 SPC-2 Price-PerformanceTM(as of March 17, 2015). Scene5: It is almost close of business atthe editing studio. The VFX artists want to backup their files and protect thecreative assets they have been working on. Oracle ZFS Storage Appliance to therescue again! Low latency write performance enables fast archiving while strong AES 256-bit data-at-rest encryption combined with a two-tier keyarchitecture offer highly secure, granular and easy to implement storage levelencryption for rich media assets. THE END Watch out for the Oracle booth at NAB show this year. If youwant an up close and personal look at our star, Oracle ZFS Storage Appliance and knowmore about its super powers, visit Oracle Stand: SU2202 (South Upper Hall), LasVegas Convention Center, April 13-16, 2015. 

Inspired by the recent Oscars award ceremony, I decided to try my hand at directing and here is my work in progress script, featuring our star – Oracle ZFS Storage Appliance. Location: Editing Studio Sce...

Oracle Expands Storage Efficiency Leadership

On March 17, 2015, a new SPC-2 result was posted for theOracle ZFS Storage ZS4-4 at 31,486.23 SPC-2 MBPSTM. (The SPC-2 benchmark, for those unfamiliar,is an independent, industry-standard performance benchmark test for sequentialworkload that simulates large file operations, database queries, and videostreaming.) This is the best throughputnumber ever posted for a ~$1/2 million USD system, and its on-par with the bestin terms of raw sequential performance. (Note that the SPC-2 Total PriceTM metric includes threeyears of support costs.) While achievinga raw performance result of this level is impressive (and it is fast enough toput us in the #3 overall performance spot, with Oracle ZFS Storage Appliancesnow holding 3 of the top 5 SPC-2 MBPSTM benchmark results), it iseven more impressive when looked at within the context of the “Top Ten” SPC-2results. System SPC-2 MBPS $/SPC-2 MBPS TSC Price Results Identifier HP XP7 storage 43,012.52 $28.30 $1,217,462 B00070 Kaminario K2 33,477.03 $29.79 $997,348.00 B00068 Oracle ZFS Storage ZS4-4 31,486.22 $17.09 $538,050 B00072 Oracle ZFS Storage ZS3-4 17,244.22 $22.53 $388,472 B00067 Oracle ZFS Storage ZS3-2 16,212.66 $12.08 $195,915 BE00002 Fujitsu ETERNUS DX8870 S2 16,038.74 $79.51 $1,275,163 B00063 IBM System Storage DS8870 15,423.66 $131.21 $2,023,742 B00062 IBM SAN VC v6.4 14,581.03 $129.14 $1,883,037 B00061 Hitachi Virtual Storage Platform (VSP) 13,147.87 $95.38 $1,254,093 B00060 HP StorageWorks P9500 XP Storage Array 13,147.87 $88.34 $1,161,504 B00056 Source: “Top Ten”SPC-2 Results, http://www.storageperformance.org/results/benchmark_results_spc2_top-ten SPC-2MBPS = the Performance Metric$/SPC-2 MBPS = the Price-Performance MetricTSC Price = Total Cost of Ownership MetricResults Identifier = A unique identification of the result Metric CompleteSPC-2 benchmark results may be found at http://www.storageperformance.org/results/benchmark_results_spc2. SPC-2,SPC-2/E, SPC-2 MBPS, SPC-2 Price-Performance, and SPC-2 TSC are trademarks ofStorage Performance Council (SPC). Results as of March 16, 2015, for more information see http://www.storageperformance.org Perhaps the better way to look at this top ten list is witha graphical depiction. When you lay itout with a reverse axis for SPC-2 Total PriceTM, you can get an“up-and-to-the-right” is good effect, with a “fast and cheap” quadrant. Looking at it this way, a couple of things are clear. Firstly, Oracle ZFS Storage ZS4-4 is far andaway the fastest result anywhere near its price point. Sure, there are two faster systems, but theyare way more expensive, about a million USD or more. So you could almost buy two ZS4-4 systems forthe same money. A second point thisbrings up is that the ZS3-2 model is the cheapest system in the top ten, andhas performance on par with or better than some of the very expensive systemsin the lower-left quadrant. In fact, the ZS3-2 model has for some time now held the #1position in SPC-2 price-performanceTM, with a score of $12.08 perSPC-2 MBPSTM. So we alreadyhave the performance efficiency leader in terms of the overall SPC-2price-performanceTM metric, as well. Of course, the efficiency story doesn’t stop withperformance. There’s also operationalefficiency to consider. Many others haveblogged about our D Trace storage analytics features, which provide deepinsight and faster time to resolution than about anything else out there, andalso our simple yet powerful browser user interface (BUI) and command lineinterface (CLI) tools, so I won’t go deeply into all that. But, suffice it to say that we can get jobsdone faster, saving operational costs over time. Strategic Focus Group did a usability studyversus NetApp and found the following results: Source: Strategic Focus Group Usability Comparison: https://go.oracle.com/LP=4206/?elqCampaignId=6667 All of this goes to show that the ZFS Storage Applianceoffers superior storage efficiency from performance, capex, and opexperspectives. This is true in anyapplication environment. In fact all theabove metrics are storage-specific and agnostic as to who your platform orapplication vendors are. Of course, asgood as this general storage efficiency is, it’s even better when you arelooking at an Oracle Database storage use case, where our unique co-engineeredfeatures like Oracle Intelligent Storage Protocol, our deep snapshot andEnterprise Manager integration, and exclusive options like Hybrid ColumnarCompression come into play to make the Oracle ZFS Storage Appliance even moreefficient. The Oracle ZFS Storage Appliance is “efficient” by manydifferent metrics. All things taken intotal, we think it’s the most efficient storage system you can buy.

On March 17, 2015, a new SPC-2 result was posted for the Oracle ZFS Storage ZS4-4 at 31,486.23 SPC-2 MBPSTM. (The SPC-2 benchmark, for those unfamiliar,is an independent, industry-standard performance...

Thin Cloning of PDBs is a Snap with Oracle Storage

I thought I’d bring up another integration point between OracleZFS Storage Appliance and Oracle Database 12c with Oracle Multitenant.  Specifically,I want to discuss how we integrate our ZFS snapshot technology with OracleMultitenant to reduce database storage footprint and speed databaseprovisioning straight from the SQL Plus command line.Oracle Multitenant option for Oracle Database 12c introducesthe concept of the pluggable database (PDB), which exist within a containerdatabase (CDB).  This is the basic construct that is used to facilitatedatabase creation in multitenant environments from either “seed” PDBs(essentially a template) or a “source” PDB (an existing, running, fulldatabase).  In either case, a simple SQL Plus command “create pluggabledatabase…” is used to easily create the new PDB.  Under the covers, the waythis normally works is by duplicating the relevant files associated with thesource or seed PDB, copying them to a new location to support the new PDB.But the Oracle ZFS Storage Appliance has thin cloningcapability based upon ZFS snapshots and ZFS clones.  This leverages the uniquecopy-on-write architecture of the ZFS filesystem to enable administrators tocreate a nearly infinite number of snaps / clones in a manner that occupiesalmost no initial storage space, takes negligible amount of time to create, andcauses negligible performance impact to existing workloads.  Is it possible toleverage this technology for deploying PDBs in a Multitenant environment?  Yes!We offer ZFS snapshot integration straight from the SQL Pluscommand line.  This integration allows a DBA to utilize the same “createpluggable database…” to easily create a new PDB from an existing source or seedPDB.  But, the twist is you no longer actually have to copy files.  By addingthe “snapshot copy” suffix to the same SQL Plus command, you invoke the ZFSsnapshot and cloning functionality behind the scenes, transparently andautomatically.  This inserts copy-on-write pointers to the existing PDB ratherthan copying the actual files.  The upshot is that you provision the same newPDB just as you would have using the original method, but the new PDB willoccupy almost no incremental storage space initially after clone creation. Also, creation of the new PDB happens in seconds because no data is actuallybeing copied from place to place.How does this all work?  Check out thisarticle on using snapshot cloning in an example infrastructure with OracleMultitenant and SPARC servers.So, with Oracle ZFS Storage Appliance and Oracle Multitenant,creating new PDBs from existing PDBs is extremely simple, carries no initialstorage footprint, and very happens very fast.  Which means reduced disk costsand also reduced management time related cost.  This is yet another example ofdeep integration between Oracle products that are designed to work together.See my related video at: https://www.youtube.com/watch?v=1E2KIbaBPRs

I thought I’d bring up another integration point between Oracle ZFS Storage Appliance and Oracle Database 12c with Oracle Multitenant.  Specifically,I want to discuss how we integrate our ZFS snapshot...

Oracle Storage Takes a Cue from Oracle Database

Oracle Intelligent Storage Protocol (OISP) version 1.1 wasrecently announced along with Oracle ZFS Storage Appliance’s new OS 8.3.  So,what’s the scoop on this, and what’s the strategy it represents?At the most basic level, OISP is a mechanism by which OracleDatabase 12c can pass cues to Oracle ZFS Storage Appliance.  We use the DirectNFS client within the Database to do this.  What sort of cues do we transmit,you ask?  In principal, pretty much whatever we want!  Well, within reason, ofcourse.  But the point is that Oracle owns both products – Oracle Database 12cand Oracle ZFS Storage Appliance.  We can figure out what cues are helpful andhow to use them, and then program them into the code for both products.  Now –that’s pretty cool from an engineering perspective.But… Passing cues?  Sound a little vague?  Let me make itmore concrete…There’s two ways we use these cues, at present.  Theoriginal version automatically and dynamically tunes storage share settings forspecific I/O from Oracle Database.  Without OISP, the best practice is tocreate multiple shares on the storage appliance for each of the major databasefile types (data files, control files, online redo log files, etc).  The reasonfor this is so each share can be optimally tuned with different record size andlogbias settings for the workload profile of each file type.  Performance canbe greatly enhanced by tuning, but it is a manual effort.  With OISP, however, theshares can be consolidated and the storage adjusts share parametersappropriately, on-the-fly, for each of the different file types.  In this way,optimal tuning can be achieved without wasted time and guesswork.OISP version 1.1 adds per-database analytics functionalityto the picture.  Many are already familiar with our advanced D Trace analyticspackage that is included with the Oracle ZFS Storage Appliance, which providesthe industry’s best end-to-end visibility into the storage workload, includingdrill-down capabilities.  Now, as an enhancement to our analytics, OISP version1.1 uses cues to provide drill-downs on a per-database basis.  This evenworks at the pluggable database (PDB) level in Oracle Multitenantenvironments.  This is significant because, even with PDBs, which share OnlineRedo Logs and Control Files at the container database level, you canimmediately see the effect of a particular database on overall storage resourceutilization.  So, for example, if one database falls victim to a rogue query ina multitenant environment, the administrator can immediately see which databasethat is and quickly act to solve the issue causing the problem.    So, OISP significantly speeds and simplifies databasestorage provisioning.  It also provides deep visibility, so admins canimmediately know, rather than just guess, the linkages between storage workloadsand individual databases.  And this is only the beginning.  Imagine the opportunitiesahead of us to bring further game-changing customer value to Oracle Database12c.See my related video at https://www.youtube.com/watch?v=onz5T2Q3i6k

Oracle Intelligent Storage Protocol (OISP) version 1.1 was recently announced along with Oracle ZFS Storage Appliance’s new OS 8.3.  So,what’s the scoop on this, and what’s the strategy...

Data Encryption ... Software vs Hardware

Software vsHardware Encryption,  What’s Better and Why People often ask me, when it comes to storage (or data-at-rest)encryption, what’s better, File System Encryption (FSE) which is done insoftware by the storage controller, or Full Disk Encryption (FDE) which is donein hardware via specialized Self Encrypting Drives (SEDs). Both methods are very effective in providing security protectionagainst data breaches and theft, but differ in their granularity, flexibility and cost. Agood example of this is to compare Oracle ZFS Storage Appliance that uses very granularFileSystem Encryption versus NetApp storage that uses Self Encrypting Drives(SEDs). Granularity and Flexibility With ZFS Storage you can encrypt at a file system level, providing muchmore granularity and security controls. For example, you can encrypt aproject, share, or a LUN, assigning different access and security levelsfor different users, groups, or applications depending on the sensitivity ofthe data and the security/business requirements of a particular group oran organization. NetApp, using Full Disk Encryption (FDE) does not have this granularityor flexibility. As the name implies, the encryption is done at the full disk-level,by the SED drive. So if you have only a small file to encrypt on a 4TB SED drive,you’re stuck with 4TB granularity of that whole drive. To make things worse,since NetApp does not support mixing SEDs/HDDs in the same disk shelf, yourgranularity might be as bad as 96TB—just to encrypt a small file!. Furthermore, FDE requires specialized self-encryption drives (SEDs)which are not only expensive, but come only in certain capacities andperformance classes. ZFS Storage encryption, on the other hand, works with yourstandard disk drives (including SSDs), independent of capacity, performance orcost. Cost Self Encrypting Drives (SEDs) can be very expensive. NetApp chargesanywhere from a 40% to 60% price premium for their SEDs. For example, the price of their DS4246 disk-shelffor FAS8000 with 24 x 4TB 7.2K encrypted drives is $51,720, whereas the same drive shelf with non-encrypted drives is $32,400 (source:Gartner). That’s a $19,320 price difference, or a 60% price premium for encrypted drives. For thesame tray with 24 x 800GB SSDs, it’s $289,320 for encrypted SSDs vs $188,040 for non-encrypted SSDs - a $101,280, or 54%, price difference. Scaling it out to something like a petabyteof storage, this extra cost can add up to hundreds of thousands of dollars, ormore. Comparing this to Oracle ZFS Storage Appliance Encryption, which usesFile System Encryption and standard disk drives, the cost saving is huge. For adual controller (HA cluster), ZFS Encryption software is only $10,000, and thatincludes local key management. It’s also capacity independent so you can scaleit to a petabyte of encrypted data or more at no extra cost. How does thiscompares to NetApp? Well, if we look at 1PB of encrypted data and the aboveHDD cost structure,it would be $201,250 for NetApp and only $10,000 for ZFS Storage. For SSD’s, it would be $5.28M for NetApp andstill only $10K for ZFS Storage. That’s over 528X more for NetApp, if you’rekeeping score. . —quitea hefty cost difference. Other factors ....Some might argue that hardware encryptionis faster than software encryption. Yes,today this might be true, especially with large block sequential workloads asencryption in general is a pretty CPU intensive process. This difference isless with small block random workloads, and hardly any with cached reads. The ZFS Storage Appliance offers verypowerful multi-core CPUs and large amounts of DRAM to minimize these encryptionperformance costs. It also offers fine granularity, so one can manage whatshares/projects to encrypt and at what level (128/192/256-bit) so as to bettermanage and control both security and overall system performance. In the future,as more and more CPUs adapt advanced encryption acceleration in their chips, Iexpect this performance difference between software and hardware encryption to disappear, but not the cost, granularity, flexibility or ease of scale.

Software vs Hardware Encryption,  What’s Better and Why People often ask me, when it comes to storage (or data-at-rest) encryption, what’s better, File System Encryption (FSE) which is done insoftware...

ZFS Data Encryption ... Secure, Flexible and Cost-Effective

Part 2: How to Prevent the Next Data Breach with OracleZFS Storage Appliance Encryption and Key Management As we discussed in Part 1, storage system (ordata-at-rest) encryption provides the best level of protection against databreaches and theft as it secures all your stored data independent of thenetwork or the application environment. The Oracle ZFS Storage Appliance Encryption is one of the best storage encryptionsolutions out there. It provides highly secure, most flexible and cost-effectivestorage encryption and key management on the market. Here are some highlights … Highly Secure • AES 256-bit encryption, the most secureencryption standard available today. • Two-tier encryption architecture, with minimalkey latency - First level encrypts the data volume (project, shareor a LUN) - Second level, also 256-bit, encrypts the keys inthe key management system • Customized authorization and access controls fordefining and managing admin, user and role-basedpermissions. • Local or Centralized Key Management, - Local key manager built-into the Oracle ZFSStorage Appliance, or - Centralized key manager, such as Oracle’senterprise Key Management System,which also supports other encryption devices, including Oracle’s StorageTek tapedrives, Oracle Databases, Java, and Solaris OS Simple, Efficient and Flexible • GUI-based (or CLI) interface for ease of set up,use and management • Granular encryption at project, share or LUNlevel for better management and security controls. This way encrypted and unencryptedvolumes can co-exist securely within the same storage system using standarddrives. You only need to encrypt the volumes that contain sensitive data, helpingoptimize performance, storage efficiency, security levels, group controls, and cost. Highly Reliable and Available • Oracle ZFS Storage Appliance Encryption uses a HighAvailability dual-controller design with the encryption keys residing on bothcontroller nodes within a cluster for redundancy and availability. It’s thesame for key management. • Keys are further protected via Backup and DR capabilities,because if you lose the key, you will not be able to access the data. It’s likea secure erase of that data volume. ZFS dual cluster design, redundancy, backupand DR capability protect against that. Summary Cyber attacks and databreaches are increasing at an alarming rate with very costly consequences. Data encryption providesvery effective protection these dangers and is fast becoming a requirement for manybusinesses and government agencies. Storage (or data-at-rest) encryption is thebest option against data breaches and theft, as it protects all yourdata independent of the network or the application environment. Oracle ZFSStorage Encryption and Key Management with its highly secure two-tier 256-bit encryptionarchitecture, granular flexibility, efficiency and reliability offers the data securityprotection needed for today’s environments. It can not only save millions of dollars in possible data breach costs, but company’s business and reputation. Remember... It’s better to be safe than sorry! For additional information, check out: · BestPractices for Deploying Encryption and Managing Its Keys on the Oracle ZFSStorage Appliance white paper: White paper: OracleZFS Storage Appliance Encryption (PDF)· FiveMinutes to Insight video on Data Encryption and the ZFS Storage Appliance: Watch the video(7:02)

Part 2: How to Prevent the Next Data Breach with Oracle ZFS Storage Appliance Encryption and Key Management As we discussed in Part 1, storage system (ordata-at-rest) encryption provides the best level...

Data Encryption ... Be Safe or Be Sorry

How to Prevent theNext Data Breach with Encryption (Part 1) Another day, another costly data breach headline. This time, it was Sony Pictures as the victimof a devastating cyber attack. Newspapersreport that North Korean hackers broke into Sony’s computer systems stealingand exposing a trove of sensitive documents. The studio’s reputation is in tatters as embarrassing revelations spillfrom tens of thousands of leaked emails, private documents and other companysensitive materials. Millions of dollarsin lawsuits are expected from ex-employees over leaked Social Security numberand other personal information. Sony is not alone. Security breaches and cyber attacks arerising at an alarming rate. Verizon DataBreach Investigation Reports indicated that between 2002 and 2012 over 1 billionrecords were compromised. Some 97% could have been prevented with basic securitycontrols and data encryption. According to the Ponemon Institute Study, theaverage cost to a breached company to investigate, notify and respond to theseattacks was $3.5 million in 2014—not including legal liabilities, regulatoryfines, and intangibles such as loss of customer confidence and brand reputation.Data encryption is a much less expensive option. What You Can Do to SecureYour Data Data encryption provides very effective protection against maliciouscyber attacks, unauthorized access, use and theft of sensitive information. Itis not only a good business practice, but is fast becoming a compliancerequirement by many businesses and government agencies. Data encryption uses very sophisticated algorithms to transformplaintext into cyphertext that’s impossible to read without highly secureencryption keys, such as those based on the 256-bit Advanced EncryptionStandards. Once encrypted, data can only be accessed and read with the correspondingencryption keys, which need to be managed appropriately based on establishedbusiness security policies. What and Where to Encrypt To protect against breaches, you want to encrypt all your sensitiveinformation, both current and archived. There are several encryption options, includingencryption at the application, network, or storage levels. · Application-LevelEncryption: Data encryption at theapplication level, like Oracle Database encryption, is performed by theapplication at time of the data’s creation. This is good as encryption is doneat the source, but it’s application specific, and often puts a heavy load onthe application server negatively impacting application performance andresponse times. · Network-LevelEncryption: Data is encrypted at thenetwork level (aka in-line encryption) either via a dedicated encryptionappliance or in a switch. While this might seem convenient, it does introduceadditional hardware, management complexity, and cost. It can also impactnetwork performance and increase pipe costs as encrypted data can’t becompressed or de-duped, which is often a common practice when moving largeamounts of data over the network, like in the case of Business Continuity,Replication and DR. · Storage-LevelEncryption: Storage-level, ordata-at-rest encryption, is performed by the storage system itself, either bythe controller or special self-encrypting drives (SEDs). While both are effective, controller-basedencryption is more desirable as it’s more flexible, scalable and often lessexpensive than the SED type. Controller-based encryption can be applied to all yourstorage on standard drives and not just that on the specialized and much moreexpensive SEDs. Data in-flight information, or network data, is often protectedby numerous existing network security and encryption standards and protocols, suchas SSL/TLS. Storage-level (or data-at-rest) encryption isthe best option against data breaches and theft, potentially saving companiesmillions of dollars in cost and reputation. It providesthe optimal data security by protecting all of your sensitive data across allenvironments. Storage encryption is application, host, and transportindependent offering better security, higher performance, and lower cost thanapplication-level or network-level encryption without burdening the applicationservers or increasing network loads. In Part 2, I will review the Oracle ZFS Storage Appliance Encryption and KeyManagement.

How to Prevent the Next Data Breach with Encryption (Part 1) Another day, another costly data breach headline. This time, it was Sony Pictures as the victim of a devastating cyber attack. Newspapersrepo...

Using the Right Tool for the Job

You wouldn’t hammer a screw or screw a nail, you would use theright tool for the job. That’s the elegance of the ZFS Storage Appliance, it simplyuses the right tools for the jobs it needs to do. The Hybrid StoragePool within the ZFS Storage Appliance has three different caches, using threedifferent technologies to perform three different jobs. I’ll explain each in alittle detail below: The Write Cache useswrite optimised SSDs (aka Logzillas). These get hammered (excuse the pun) and thereforeneeds to have high levels of endurance and low latency write response. They don’trequire large capacity (only a few GB are ever used at any given time) and theyare almost never read. As such, we use a specific type of SSD for this job. Thebenefit the write cache provides is twofold: Low latency response times for synchronous writerequests (using write optimised SSDs) Aggregation of small I/O into transaction groupsresulting in fewer but larger (and therefore more efficient) physical I/O tothe HDD. The 2nd cache is the Adaptive Replacement Cache (ARC).The ARC is held in DRAM which in a ZFS Storage Appliance is comparativelypretty large (up to 3TB on a ZS4-4 controller pair). DRAM of course is thefastest form of storage. It serves up I/O measured in nanoseconds compared tothe much slower microseconds of flash drives and milliseconds of hard drives.We consider ZFS to be a DRAM first storage architecture and it is one key partof the blinding performance ZFS can deliver. The benefit of the ARC is threefold: Provides the lowest latency reads possible (using large amounts of DRAM) Stores a large amount of current hot data forreuse Reduces the read I/O demand on the hard drives The 3rd cache is the Layer 2 Adaptive Replacement Cache (L2ARC) which uses large read optimisedSSDs (aka Readzillas) which have large capacity (up to 12.8TB per controllerpair) and great read performance, with more modest write and endurancecharacteristics. The role of the L2ARC is to collect smaller pieces of datathat fall out of the ARC. This typically consists of data that is <32K insize and has a reasonable level of re-use. The L2ARC is like a bench player, typicallynot receiving much fanfare, but invaluable when called into action. The benefitof the L2ARC is threefold: Provides low latency reads (at flash storagespeeds) Stores a large amount of warm small I/O data Protects the HDD from having to do (low productivity)small random IOPS Using these three different technologies to act as threedifferent forms of cache has the effect that a final technology (being theHDDs) is left to do what it does well. That is, storing large qualities of datain an efficient manner using large physical I/O, therefore making the most ofevery single rotation of the spindle. Pulling this all together you not only get the fasteststorage, but also the most efficient, by simply using the right tool for thejob!

You wouldn’t hammer a screw or screw a nail, you would use the right tool for the job. That’s the elegance of the ZFS Storage Appliance, it simply uses the right tools for the jobs it needs to do. The H...

I Like Big Blocks and I Can Not Lie (1992 vs 2014)

A relatively under discussed feature of the ZFS Storage Appliance is it's ability to use a variable block size for the storing of files. The "record size" on ZFS can be set anywhere from 512 bytes all the way up to 1 megabyte. This differs from the likes of NetApp and Isilon which respectively use 4K and 8K fixed block sizes.  When NetApp launched back in 1992 it was a good approach to base things around 4K as IOPS/GB were excellent. In fact, 1992 saw the release of the industry's first 7,200 RPM drive, Seagate's Barracuda ATA, coming in at a whopping 2.5GB with approximately with 40 IOPS/GB. However, put this in contrast to today's 4TB 7,200 RPM drives at 0.025 IOPS/GB and a 4K fixed block size starts to make a lot less sense.Though IOPS/GB have dropped with each generation of HDD, what is great with modern HDD is that with increasing areal density throughput speeds have steadily improved, but to make use of that speed you really need to be making the most of those limited IOPS. With ZFS, using a large record size of 1MB does exactly that, as shown by the incredible SPC-2 throughput benchmarks results provide by both the Oracle ZS3-2 & ZS3-4 Storage Appliances. If you've got a lot of data to process, such a geological analysis, VFX rendering, driving a data warehouse or simply doing backups you're going to love working with a large block size. Let me finish by paraphrasing Sir Mix-a-Lot's hit song "I... like... big blocks and I can not lie, you other vendors can't deny"... if you end up with that tune stuck in your head for the rest of the day, I apologise. However, it's somewhat apt that this single was also first released in 1992 ;-)

A relatively under discussed feature of the ZFS Storage Appliance is it's ability to use a variable block size for the storing of files. The "record size" on ZFS can be setanywhere from 512 bytes all...

Using Analytics (Dtrace) to Troubleshoot a Performance Problem, Realtime

I was recently on a call with one of the largest financial retirement investment firms in the USA.  They were using a very small 7320 ZFS Storage appliance (ZFSSA) with 24GB of dram and 18 x 3TB HDDs.  This system was nothing in terms of specifications compared to the latest shipping systems, the  ZS3-2 and ZS3-4 storage systems with 512GB and 2TB of DRAM respectively.  Nevertheless I was more then happy to see what help myself and mostly the ZFSSA Analytics (Dtrace) could offer.   The Problem: Umm, Performance aka Storage IO Latency is less then acceptable...This customer was/is using this very small ZFSSA as an additional Data Guard target from their production Exadata.  In this case it was causing issues and pain.  Below you can see a small sample screen shot we got from their Enterprise Manager Console. Discovery: Lets see what is going on right now! Its time to take a little Dtrace tour.. The next step was to fire up the browser and take a look at the real-time analytic's.    Our first stop on the dtrace train is to look at IOPS by protocol.  In this case the protocol is NFSv3 but we could easily see the same ting from nfsv4, smb, http, ftp, fc, iscsi etc... Quickly we see that this little box with 24GB of DRAM and 18 x 7200RPM HDD's was being pounded! Averaging 13,000 IOPS with 18 drives isn't bad.  I should note that this box had zero Read SSD drives and 2 x Write SSD drives.   Doing a quick little calculation we see that this little system is doing about 650 IOPS per drive per second. Holy Cow SON! Wait, is that even possible? There is something else at play here, could it be the large ZFSSA cache (a measly 24GB in our case) at work?  Hold your horses, IOPS are not everything in fact you could argue that they really don't matter at all, what really matters is latency.  This is how we truly measure performance, how fast does it take to do one thing versus how many things can I do at the same time regardless of how fast they each get done.  To best understand how much effect DRAM has on latency see our World Record SPECSFS Benchmark information here.  Here you see that sure enough that some of the read latency is pathetic, for just about any application in the world.   There are many ways to solve this problem, the average SAN/NAS vendor would tell you to simply add more disk, With dtrace we can get more granular and ask many other questions and see if there is perhaps a better or more efficient way(s) to solve this problem.This leads us to our next stop on the dtrace discovery train, ARC accesses (Adaptive Replacement Cache).  Here we quickly find that even with our lackluster performance in terms of read latency.  We have an amazing read cache hit ratio.  Roughly about 60% of our IO is coming from our 24GB of DRAM. On this particular system the DRAM is upgradable to 144GB of DRAM.  Do ya think that would make a small dent in those 6033 data misses below? This nicely leads into the next stop on the dtrace train which is to ask dtrace for all those 6033 data misses how many of them would be eligible to be read from L2ARC (READ SSD Drives in the Hybrid Storage Pool).  We quickly noticed that indeed they would have made a huge difference.  Sometimes 100% of the misses were eligible.  This means that after missing the soon to be upgraded 6x dram based cache the rest of the read IO's of this workload would likely be served from high performance MLC Flash SSD right in the controller itself. Conclusion: Analytics on the ZFSSA are amazing, The Hybrid Storage Pool of the ZFSSA is amazing, the large DRAM based cache on the ZFSSA is very amazing...At this point I recommend that they take a two phase approach to the workload.  First they upgrade the DRAM Cache 6x and add 2 x L2ARC Read SSD drives.  After that they could evaluate if they still needed to add more disk or not. Extra Credit Stop:  One last stop I made  was to look at their NFS share settings and see if they had compression turned on like I have recommended in a previous post.  I noticed that they did not have it enabled and that CPU was very low at less then 10% AVG utilization.  I then explained to the customer how they would benefit even now without any upgrades by enabling compression and I asked if I could enable it that second at 12pm in the afternoon during a busy work day.  They trusted me and we enabled it on the fly and then for giggles I decided to see if it made a difference on disk IO and sure enough it made an immediate impact and disk IO lowered because now every new write they had was taking less disk space and less IO to move through the subsystem since we compress real-time in memory.  Remember that this system was a Data guard target so it was constantly being written too. You can clearly see in the image below how compression lowered the amount of  IO on the actual drives.  Trying doing that with your average storage array compression. Special Thanks goes out to my teammates Michael Dautle and Hugh Shannon who helped with this adventure and capturing of the screenshots.

I was recently on a call with one of the largest financial retirement investment firms in the USA.  They were using a very small 7320 ZFSStorage appliance (ZFSSA) with 24GB of dram and 18 x 3TB HDDs. ...

Scream Safely! Introducing ZFS Storage ZS4-4 and OS8.3

Turbocharged storage!!!That's the best way to describe our Oracle ZFS Storage ZS4-4 (released today). It's our 4th generation high-end Oracle ZFS Storage Appliance, and it comes with our just-released OS8.3 software. This new platform uses the latest hardware technology, and combined with OS8.3, doubles the performance of the record-setting previous generation, and also offers new innovations in security and Oracle Database 12c analytics.In short, it's a beast.(sounding now like a guy describing a new car on a game show) The 2-node high-availability configuration has eight 15-core Intel Ivy Bridge-EX processors, for a total of 120 cpu cores (that's 50% higher than the ZS3-4 it replaces). It also boasts 3TB of DRAM (more DRAM than most NAS appliances have in flash). Here are the highlights:The design feature that lets us take advantage of all of that DRAM is called hybrid storage pools (HSPs) - a DRAM-centric, multi-layer caching architecture that speeds up I/O ('cuz what good is a turbo charger that's not coupled to the engine?), significantly increasing performance without sacrificing capacity. The ZS4-4 design lets us pack 10x to 20x more DRAM than competitive platforms could even use, and can deliver up to 90% I/O from DRAM in some environments, easily out-pacing all-flash NAS competitive solutions. The New OS8.3 SoftwareOS8.3 (a.k.a. 2013.1.3) is the latest version of the ZFS Storage Software. In addition to powering ZS4-4, it adds new features like Encryption (a separately priced option) and Oracle Database 12c analytics. EncryptionNew to the software lineup is Oracle ZFS Storage Appliance inline encryption, which offers highly secure, granular and easy to implement storage-level encryption. Strong AES 256-bit data-at-rest encryption combined with a two-tier key architecture protects against security breaches, legal exposures, and meets regulatory requirements (check the release notes for compatibility with older ZFS Storage models). Database Integration (as we engineer our stuff to work together)Oracle Intelligent Storage Protocol is a unique communication technique that sends "hints" from the Oracle Database 12c to the ZFS Storage appliance. It dynamically tunes Oracle ZFS Storage for optimal database performance: Reducing manual activities, simplifying deployment, and reducing risk. The new version (Oracle Intelligent Storage Protocol v1.1) provides support for database and pluggable database level analytics, broken down by database or pluggable database names. If a specific pluggable database is affecting overall storage system performance, OISP v1.1 can easily identify the culprit. Competitive systems do not provide this level of visibility. Try it today by using the ZFS Storage Appliance Simulator software.Bottom Line: Why Should I Care?For starters, the new platform doubles the performance of the previous generation (isn't that enough?), which can be critical in avoiding bottlenecks and accelerating your applications. It also helps you sleep at night because now you can encrypt your data, at the project, share or LUN level, making sure your data is secure. If you are using Oracle Database 12c (or will be soon), now you can simplify and accelerate storage performance tuning and troubleshooting at the pluggable database level with our Oracle Intelligent Storage Protocol v1.1 feature. For More InformationWebcast: New Oracle NAS Storage InnovationsOracle ZFS Storage ZS4-4Datasheet: Oracle ZFS Storage ZS4-4Connect with Oracle Hardware on Facebook and TwitterHat Tip to Carlos A. Ortiz and Nasim Tanner, who provided the provided the steak here. I just added the sizzle. - Bob

Turbocharged storage!!! That's the best way to describe our Oracle ZFS Storage ZS4-4 (released today). It's our 4th generation high-end Oracle ZFS Storage Appliance, and it comes with our just-released...

New Logzilla Drives for your ZFSSA

Yes, the new, larger Logzilla SSD drives for your ZFSSA systems are now out. They are 200GB usable, up from the 73GB usable drives.  Yes, you will sometimes see them referred to in some marketing literature as 400GB. This is because there is extra room in enterprise SSD chips to allow for cell burnout and keep their 5 years lifetime. Make no mistake, they will give you 200GB of actually capacity in the ZFSSA systems. Yes, they are compatible with the current 73GB version. You can mix and match. The one thing to look out for is in a 'mirrored' log profile. If you mix a new one with an old one in a mirrored log profile, then the new one will size down to 73GB to match it. In a striped profile, it doesn't matter, nor will it matter if you have 2 or more of each.One last thing-- They are almost twice as fast as the older 73GB version. If you mix them, you will get faster, but not as fast as if you had all 200GB versions. Diminishing returns. Talk to your local SC on whether your Lozgilla workload is so great that either adding some new ones or even changing out your old ones would help your performance. Not every workload needs Logzillas, but there are built-in analytics that can tell us if yours is a good fit. Enjoy! 

Yes, the new, larger Logzilla SSD drives for your ZFSSA systems are now out. They are 200GB usable, up from the 73GB usable drives.  Yes, you will sometimes see them referred to in some...

Cluster tricks & tips

Most of us have clustered ZFSSAs, and have been frustrated at one time or another with getting the proper resource to be owned by the proper controller. I feel your pain, and believe me, I have to deal with it as much or even more than you do. There are, however, some cool things you can do here and it will make your life easier if you fully understand how this screen works.  First, understand this- You almost never want to push the 'Takeover' button. The 'Takeover' button actually sends a signal to instantly reboot the OTHER controller, in a non-graceful way. More on that below. We have two heads in this picture and they're both in the "Active" state as you see here. This means you can not click the "Failback" button which is how we move resources to the head you wish to own them. You are only allowed ONE Failback when a head is in the "Ready for Failback" state, as it is when it first comes up. We have already hit Failback on this system, so both heads are now Active. That's it. You're done until one reboots.  Do NOT hit the 'Takeover' button. That button should be labeled "Ungracefully shutdown the other controller". Those were just too many words to fit on the button, so they called it Takeover. Sure, that means that since the other head is now being instantly rebooted, this head will now takeover all of the resources and the other head will now reboot. This is one of the worse ways to reboot the other head. It's not nice. It does not flush the cache first. It's actually slower then the other way. When and why would you ever hit it? There's a few reasons. Perhaps the other head is in a failed state that is not allowing you to log in and shut it down correctly. Perhaps you are just setting the controls up on day one, you know there's no workload at all, and you really don't care how the other head gets rebooted. If that's the case, then go for it.  Instead, for a clean and faster reboot, log into the controller you want to reboot, and click the power button: This allows you to reboot is gracefully, flushing the cache first, and it actually comes up faster than the 'takeover' way, almost always. Now that it has rebooted, which may take 5-15 minutes, the good controller's cluster screen should show that it's "Ready for Failback". Be certain all of your resources are set to the proper owner, and then hit the "Failback" button to move the resources and change both controllers to the "Active" state. REMEMBER--- You only get to hit the Failback button ONCE!!! So take your time and do all of your config and setup and get the ownership right before you hit it. Otherwise, you will be rebooting one of your controllers again. Not a huge deal, but another 15 minutes of your life, and perhaps a production slowdown for your clients. Now for a trick. There's nothing I can do to help you with the network resources. If they are on the wrong controller, you may have to reboot one and fix it and do a failback. However, if you have a storage pool on the wrong controller, I may be able to show you something cool.  The best thing to remember and do is this: Create the resource (network or pool) ON the controller you wish to be the owner in the first place!!! Then, it will already be owned by the proper one, and you don't have to do a failback at all. However, what if, for whatever reason, you need to move a pool to the other controller and you MUST NOT reboot a controller in order to move it using the Failback process? In other words, you have an Active-Active setup, the Failback button is grayed out, and it's very important that you change the ownership of a storage pool but you are not allowed to reboot one of the controllers? Bummer, right? Not so fast, check this out.  So here I have a system with two pools, Rotation and Bob, both on controller A. The Bob pool is supposed to be on controller B. They are both Active, so I can not click Failback. I would normally have to reboot head B to fix this. But I don't want to. So I'm going to unconfigure the Bob pool here on controller A. That's right, unconfigure. This does NOT hurt your data. Your data is safe as long as you do NOT create a new pool in that space. We're not going to create a new pool. We're going to IMPORT the Bob pool on controller B. All of your shares, LUNs, and their properties will be perfectly fine. There is only one hiccup, which we will talk about. Go to Configuration-->Storage, select the correct pool (Bob), and then click "Unconfig". But first, I want you to look carefully at the info below the pie chart here. Note that Bob currently has 2 Readzilla cache drives in it. This is important. You will get this screen. Take a deep breath and hit apply. No more Bob. Bob gone. Not really. It's still there and can be imported into another controller. This is how we safely move disk trays to new controllers, anyway. No big deal. So, now go log into the OTHER controller. Don't do this on the same one or else you'll have to start all over again. Here we are on B. DO NOT click the Plus Sign!!!! That will destroy your data!!!!Click the IMPORT button. The Import button will go out and scan your disk trays for any valid ZFS pools not already listed. Here, it finds one called "bob".  Select it and hit "Commit". There, Bob Pool is back. All of it's shares and LUNs will be there too. The "Rotation" pool shows Exported because it's owned by the "A" controller, and the Bob Pool is owned here on B.  We can go to Configuration-->Cluster and see all is well and Bob Pool is indeed owned by the controller we wanted, and we never had to reboot! However, we have one big problem.... Did you notice when you Imported the Bob Pool  into controller B, the Cache drives did NOT come over?It now has zero cache drives. What did you expect? The cache drives are the readzillas inside the controller, itself. They can't move over just because you changed the owner.No problem.I have 2 extra Readzillas in my B controller not being used. So All I have to do is Add them to the Bob Pool. Go back to Configuration-->Storage on the B controller. Select the Bob pool and click "ADD". Do NOT click the plus sign. This is different. I can now add any extra drives to the Bob pool. In this case, I don't have anything I could possibly add other then these two readzillas inside controller B. So pretty easy. Once added, I'm all good. I now have the Bob pool, with cache drives, being serviced on controller B with no reboot necessary. That's it. By the way, you know you can not remove drives from a pool, right? We can only add. This includes SSDs like Logzillas and Readzillas.Well, I kind of just showed you a way you CAN remove readzillas from a pool, didn't I? Hmmmmmm.....

Most of us have clustered ZFSSAs, and have been frustrated at one time or another with getting the proper resource to be owned by the proper controller. I feel your pain, and believe me, I have to deal...

New code out today- AK8.1 or 2013.1.1.0

The first minor release of code 2013.1 is now out and can be downloaded in MOS.It is 2013.1.1.0, or AK8.1Along with some bug fixes, it has three main new features:Support for 2-port 16Gbs Fibre Channel HBA Target and Initiator (backup)The drivers for supporting the 8300 Series adapters are available in 2013.1.1.0. In ZFS Storage Appliance, it supports SAN traffic at line rate, 16Gbps Fibre-Channel speeds at extremely low CPU usage with full hardware offloads.This extreme performance eliminates potential I/O bottlenecks in today's powerful multiprocessor, multicore servers. Large Block Size (1M) SupportEnable support for block/record sizes bigger than 128k (256k, 512k and 1M) for filesystems or LUNS. The implementation for this includes a deferred update detailed in the Deferred Updates and Remote Replication Compatibility with Large Block/Recordsize Update sections below. SPA Sync ConcurrencyIn order to better utilize the performance of high-speed storage devices, such as SSDs, improvements have been made to ZFS's algorithm for committing transaction groups. Specifically, the Storage Pool Allocator (SPA) sync process has been improved to parallelize some operations so that it spends a larger percentage of time writing data to the pool devices.

The first minor release of code 2013.1 is now out and can be downloaded in MOS. It is 2013.1.1.0, or AK8.1 Along with some bug fixes, it has three main new features: Support for 2-port 16Gbs Fibre...

VNIC - New feature of AK8 - Working with VNICs

One of the important new features of the AK8 code is the ability to use multiple IP addresses on the same physical network port. This feature is called VNICs, or Virtual NICs. This allows us to no longer "burn" a whole port in a cluster when one cluster peer owns a network port. Traditionally, we have had to leave Net0 empty on controller 2, because it was used for managing controller 1. Vise-versa for Net1 on Controller 1. Then, if you have data going over 10GigE ports, you probably only had half of your ports running at any given time, and the partner 10GigE port on the other controller just sat there, doing nothing, unless the first controller went down. What a waste. Those days are over.  I want to thank and give a big shout-out to our good partner, OnX Enterprise Solutions, for allowing me to come into their lab and play around with their 7320 to do this demo. They let me make a big mess of their lab for the day as I played around with VNICs. If you're looking for a partner who knows Oracle well and can also piece together a solution from multiple vendors to get you what you need, OnX is a good choice. If you would like to talk to your local OnX rep, you can contact Scott Gill at Scott.Gill@Onx.com and he can point you in the right direction for your area.  Here we go: Here is what your Datalinks window looks like BEFORE you upgrade to AK8. Here's what the same screen looks like after you upgrade. See the new box? So here is my current network setup. I have my 4 physical interfaces setup each with an IP address. If I ping them, no problems.  So I can ping 180, 181, 251, and 252. However, if I try to ping 240, it does not work, as the 240 address is not being used by any of these interfaces, right?Let's change that. Here, I'm going to make a new Datalink by clicking the Datalink "Plus sign" button. I will check the VNIC box and tell it to use igb2, even though another interface is already using it. Now, I will create a new Interface, and choose "v_dl2" for it's datalink. My new network screen looks like this. A few things to take note of here. First, when I click the "igb2" device, it only highlights dl2 and int2. It does not highlight v_dl2 or v_int2.I think it should, but OK, it looks like VNICs don't highlight when you click the device. Second, note how the underscore character in v_dl2 and v_int2 do not seem to show on this screen. You can see it plainly if you go in and edit them, but from here it looks like a space instead of an underscore. Just a cosmetic bug, but something to be aware of. Now, if I click the VNIC datalink "v_dl2", on the other hand, it DOES highlight the device it belongs to, as it should. Seen here: Note that it did not, however, highlight int2 with it, even though int2 is connected to igb2. That's because we clicked v_dl2, which int2 has nothing to do with. So I'm OK with that. So let's try pinging 240 now. Of course, it works great.  So I now make another VNIC, and call it v_dl3 using igb3, and v_int3 with an address of 241. I then setup three shares, using ports 251, 240, and 241.Remember that IP 251 and 240 both are using the same physical port of igb2, and IP 241 is using port igb3. Next, I copy a folder full of stuff over to all three shares at the same time. I have analytics going so I can see the traffic. My top chart is showing the logical interfaces, and the bottom chart is showing the physical ports.Sure enough, look at the igb2 and vnic1 interfaces. They equal the traffic going over the igb2 physical port on the second chart. VNIC2, on the other hand, gets igb3 all to itself. This would work the same way with 10Gig or Infiniband ports. You can now have multiple IP addresses and even completely different subnets sharing the same physical ports. You may need to make route table entries for that. This allows us to use all of the ports you paid for with no more waste.  Very, very cool.  One small "bug" I found when doing this. It's really not a bug, it was designed to do this when VNICs were not around. But now that we have NVIC capability, they should probably change this. I've alerted the engineering team about this and they're looking into it, so perhaps it will be fixed in a later code. Here it is. Remember when we made the new VNIC datalink, I specifically said to click on the "Plus Sign" button to create it? I don't always do that. I really like to use the drag-and-drop method to create my datalinks in the network screen.HOWEVER, if you were to do that for building a VNIC, it will mess you up a little. Watch this. Here, I'm dragging igb3 over to make a new datalink. igb3 is already being used by dl3, but I'm going to make this a VNIC, so who cares, right? Well, the ZFSSA does not KNOW you are going to make it a VNIC, now does it? So... it works as designed and REMOVES the igb3 device from the current dl3 datalink in the background. See how it's now missing? At the same time, the dl3 datalink choice is missing from my list of possible VNICs for me to choose from!!!! Hey!!! I wanted to pick dl3. Why isn't it on the list??? Well, it can't be on this list because dl3 no longer has a device associated with it. Bummer for you. When you click cancel, the device is still missing from dl3. The fix is easy. Just edit dl3 by clicking the pencil button, do absolutely nothing, and click "Apply". The device will magically come back. Now, make the VNIC datalink by clicking the "Plus Sign" button. Sure enough, once you check the VNIC box, dl3 is a valid choice. No problem.  That's it for now. Have fun with VNICs.

One of the important new features of the AK8 code is the ability to use multiple IP addresses on the same physical network port. This feature is called VNICs, or Virtual NICs. This allows us to...

OS8- AK8- The bad news...

Ok I told you I would give you the bad news of AK8 to go along with all the cool new stuff, so here it is. It's not that bad, really, just things you need to be aware of.First, the 2013.1 code is being called OS8, AK8 and 2013.1 by different people. I mean different people INSIDE Oracle!! It was supposed to be easy, but it never is. So for the rest of this blog entry, I'm calling it AK8. AK8 is not compatible with the 7x10 series. Ever. The 7x10 series is not supported with AK8, and if you try to upgrade one, it will fail at the healthcheck.All 7x20 series, all of them regardless of age, are supported with AK8.Drive trays. Let's talk about drive trays and SAS cards. The older drive trays for the 7x20 series were called the "Riverwalk 2" or "DS2" trays. They were technically the "J4410" series JBODs that Sun used to sell a la carte before we stopped selling JBODs. Don't get me started on that, it still makes me mad. We used these for many years, and you can still buy them right now until December 15th, 2013, when they will no longer be sold. The DS2 tray only came as a 4u, 24 drive shelf. It held 3.5" drives, and you had a choice of 2TB, 3TB, 300GB or 600GB drives. The SAS HBA in the 7x20 series was called a "Thebe" card, with a part # of 7105394. The 7420, for example, came standard with two of these "Thebe" cards for connecting to the disk trays. Two Thebe cards could handle up to 12 trays, so one would add two more cards to go to 24 trays, or have up to six Thebe cards to handle 36 trays. This card was for external SAS only. It did not connect to the internal OS drives or the Readzillas, both of which used the internal SCSI controller of the server.These Riverwalk 2 trays ARE supported with AK8. You can upgrade your older 7420 or 7320, no problem, as-is. The much older Riverwalk 1 trays or J4400 trays are NOT supported by AK8. However, they were only used by the 7x10 series, and we already said that the 7x10 series was not supported.Here's where it gets tricky. Since last January, we have been selling the new style disk trays. We call them the "DE2-24P" and the "DE2-24C" trays. The "C" tray is for capacity drives, which are 3.5" 3TB or 4TB drives. The "P" trays are for performance drives, which are 2.5" 300GB and 900GB drives. These trays are NOT Riverwalk 2 trays, even though the "C" series may kind of look like it. Different manufacturer and different firmware. They are not new. Like I said, we've been selling them with the 7x20 series since last January. They are the only disk trays we will be selling going forward. Of course, AK8 supports them.So what's the problem? The problem is going to be for people who have to mix drive trays. Remember, your older 7x20 series has Thebe SAS2 HBAs. These have 2 SAS ports per card.  The new ZS3-2 and ZS3-4 systems, however, have the new "Thebe2" SAS2 HBAs. These Thebe2 cards have 4 ports per card. This is very cool, as we can now do more SAS channels with less cards. Instead of needing 4 SAS cards to grow to 24 trays like we did with the old Thebe cards, I can now do 24 trays with only 2 Thebe2 cards. This means more IO slots for fun things like Infiniband and 10G. So far, so good, right? These Thebe2 cards work with any disk tray. You can even mix older DS2 trays with the newer DE2 trays in the same system, as long as you have Thebe2 cards.Ah, there's your problem. You don't have Thebe2 cards in your old 7420, do you? Well, I told you the bad news wasn't that bad, right? We can take out your Thebe cards and replace them with Thebe2. You can then plug your older DS2 trays right back in, and also now get newer DE2 trays going forward. However, it's important that the trays are on different SAS channels. You can mix them in the same system, but not on the same channel. Ask your local SC if you need help with the new cable layout. By the way, the new ZS3-2 and ZS3-4 systems also include a new IO card called "Erie" cards. These are for INTERNAL SAS to the OS drives and the Readzillas. So those are now SAS2 instead of SATA like the older models. Yes, the Erie card uses an IO slot, but that's OK, because the Thebe2 cards allow us to use less SAS HBAs to grow the system, right?That's it. Not too much bad news and really not that bad. AK8 does not support the 7x10 series, and you may need new Thebe2 cards in your older systems if you want to add on newer DE2 trays. I think we can all agree that there are worse things out there. Like our Congress.  Next up.... More good news and cool AK8 tricks. Such as virtual NICS. 

Ok I told you I would give you the bad news of AK8 to go along with all the cool new stuff, so here it is. It's not that bad, really, just things you need to be aware of. First, the 2013.1 code is...

Do you want to upgrade to AK8 (2013.1) right now?

Ok, so you will hear some great stuff about AK8, but are you going to upgrade your production system to a new major release right after it comes out? Probably not. If you have a test system or a lab system you can play with, then I highly recommend upgrading it so you can start to see the new performance features that AK8 can give you. If you only have one system, or they're all in production, then of course you're going to wait for the first minor release of the new code, aren't you? I would too. I'm told the first minor is coming out in just a few weeks. It is the release they used for the public benchmark performance testing. So you can feel more confident in that release. You may also be able to talk to your local sales team about getting a demo unit. Then, you can play with the new code in a safe lab area before upgrading your production system. Next up... The negative aspects of upgrading to AK8. It's not too bad, but you will need to know which older systems can't do it, how to work with older disk trays, and whether or not you can replicate newer systems with older systems.  Hey, I told you I wasn't just going to blow sunshine on you all the time, right? I can spit out the kool-aid as well as drink it!  :)

Ok, so you will hear some great stuff about AK8, but are you going to upgrade your production system to a new major release right after it comes out? Probably not. If you have a test system or a lab...

Upgrading to OS8 - AK8- 2013.1

The upgrade to OS8, AK8 or whatever we are calling it this week was pretty straightforward. It will take some extra time, as it has to perform some one-time jobs the first time it reboots, but it wasn't more than 15 minutes. Your mileage may vary, it's possible on larger systems that it takes longer. There is also a deferred update I will show you down below that you can choose to do right away or later. Once you do that deferred update, you do NOT want to roll back to the previous version, so be warned.  It's been over 1.5 years since the last major update, so many of you probably have never done one before. The process is just like a minor update, it just takes longer.   Get the update from MOS and unzip it to a folder. Go ahead and upload it and unpack it like normal from your Maintenance-->System screen. I did like how it tried to tell me how much time was left, but the numbers were all over the place, and it was over by the time it was correct. Now, when you click the arrow to apply the update, the normal health check window appears, but you will notice something extra. That's the 'Deferred Update' choice. You can make it apply as soon as it reboots, or you can manually apply it later. Remember, you do NOT want to rollback after this is applied. I did "Upon Request", click the "Check" button, and if all is well, click "Apply"  After it installs and reboots, you can look at the command line via serial port or SSH. You will notice a few things are different during this boot-up. Right after the "Updating ####" section you can see it actually upgrading various services and the SMF repository. This can take around 3 minutes, but if you have a lot of aggragations or IPMP then it could take longer. So relax. You can see mine, below, which went 290 seconds, and then continued upgrading other stuff.  The upgrade continues, and the screen is pretty obvious.  When you see it configuring network devices, you're almost done. You can see the new code level, and it's about to go to the login prompt. At that point, you should be able to log back into the BUI.  Log back into the BUI, and you will see the new version is the current version in Maintenance-->System Now, let's do the deferred update on the same screen. You can read about the deferred updates here, and click apply when ready to add them. In this case, it's for the ability to associate multiple initiator groups with a LUN, something we have wanted for some time now, so very cool. Note that ANY other deferred updates you have not applied yet will also apply, as there is no way to pick and choose. Either they all apply or none do. Remember I said not to roll-back to a previous version of the code after you do this? It will let you, but if you do, your LUN operations will fail. No bueno. Don't do it. The deferred upgrades are one-way. Note that the deferred update does NOT force a reboot.  Once you apply the deferred updates, the whole deferred update area goes away, and the screen now looks like this.  Do you want to see something cool right away now in OS8 that you could not do before? There's a lot I will talk about later, but for now, since you're so excited, go to Configuration-->Alerts, and create a new Threshold Alert. Notice the new Capacity threshold alerts, where you can now get emails or create an action when a pool, and project, or a share goes over, say, 80% full. Sweet.

The upgrade to OS8, AK8 or whatever we are calling it this week was pretty straightforward. It will take some extra time, as it has to perform some one-time jobs the first time it reboots, but...

ZS3 is #1 on Storage Performance Council benchmark site

This is pretty cool. It seems the ZS3-4 just became the number 1 system in performance on Storage Performance Council's benchmark site.The email below went out today to all SPC members. I would like to point out that we are also the LEAST EXPENSIVE system per SPC-2 performance. Check out our Price/Performance numbers. So we came in at 17,244 for a $388,472 system, for a price/performance of $22.53.Now compare that to the 2nd place system on the site, which is HP's P9500. It came in at  13,147 for a huge price of $1,161,503 and a price/performance of $88.34We KILLED it....****************************SPC Members:Oracle Corporation has submitted the SPC-2 Result™ listed below. The Executive Summary and Full Disclosure Report (FDR) are posted in the Benchmark Results section of the website. The documents may be accessed by using the URL listed below:http://www.storageperformance.org/results/benchmark_results_spc2#b00067 Oracle ZFS Storage ZS3-4 (2-node cluster):   SPC-2 Submission Identifier .... B00067 SPC-2 MBPS™ …………………….... 17,244.22   SPC-2 Price-Performance™ …… $22.53/SPC-2 MBPS™   Total ASU Capacity ………….…..  31,610.959 GB   Data Protection Level ………..…. Protected 2 (Mirroring)   Total Price ………………………….... $388,472.03Congratulations to Oracle for an outstanding SPC-2 Result, which established a new #1 for SPC-2 performance (17,244.22 SPC-2 MBPS™).Regards,SPC AdministratorStorage Performance Council (SPC) 

This is pretty cool. It seems the ZS3-4 just became the number 1 system in performance on Storage Performance Council's benchmark site. The email below went out today to all SPC members.  I would like...

New ZS3 ZFS Storage family announced TODAY! Finally!

It's official and we can finally be excited about the new ZS3 family. We can start talking about it now and start ordering it on Thursday. It won't actully ship, however, until next month on October 8th. I know, I know... but hey I CAN give you something TODAY... How about the 4TB drives, available right now??? Also, the new 1.6TB Readzillas and the 16-port 4x4 SAS HBAs are all here this Thursday, Sept 12th. Not bad, right?So we now have three new systems:1- The new ZS3-22- The new ZS3-43- The updated 7420M2 with internal SASThe first two will ship with the new OS8 code. The 3rd one will ship with the older OS7 (2011.1.7) code, but can be updated to OS8 at anytime. (see my last blog entry about the new OS names) Ok, here is the low-down. The new 4TB drives are for the DE2-C trays which have been out since last December. I do NOT YET KNOW if they will also be available for the older DS2 trays, but I will tell you when I find out. This is important--- The new 1.6TB Readzillas are SAS, not SATA, so they will only work in the new ZS3 series and the new 7420M2 box. Your older 7420 and 7320 use internal SATA, not SAS, for their Readzillas and system OS drives. The new 900GB OS drives and the new 1.6TB Readzillas are SAS, so you need the newer versions to work with them.The LAST order date for the current 7420 is September 30, 2013, and the LOD for the 7320 and 7120 is November 30, 2013. You can get a new product datasheet or the product announcement from your local storage SC.  

It's official and we can finally be excited about the new ZS3 family. We can start talking about it now and start ordering it on Thursday. It won't actully ship, however, until next month on October...

Some info about our ZFSSA codes

As you now know, version 2011.1.7.0 is the current shipping code for our ZFSSA. You really want to be running this code, no matter what ZFSSA system you have. This code will work all they way back to 7x10 systems. There are found bugs in even the last code, 2011.1.6.0, that this newer code will fix, so get on it.Let's talk for a moment about code names and numbers, as it's going to change from your point of view very soon. Many years ago, Sun Microsystems created the "Fishworks" team to create this code that we now run on the ZFS Storage appliance. You can still see Fishworks and the original team names if you "Shift-Click" the Oracle/Sun logo in the top left corner of your ZFSSA. (There are MANY secret Shift-Click operations in the ZFSSA. I told you about some back in my blog on analytics here: https://blogs.oracle.com/7000tips/entry/fun_tips_with_analytics) By the way, FISH stands for "Fully Integrated Software & Hardware.So the code that Fishworks created is a layer between you, the user, and the special version of Solaris and ZFS underneath. This is called an "Appliance Kit", and you will see all sorts of system names with an "AK" on them, which are directly linked to the Appliance Kit, which is basically the code for the interface, both the GUI and the CLI, which you all know and love. Internally at Oracle the Fishworks team, now a much larger team that Oracle has grown far beyond the original, calls the code levels for the ZFSSA "AK#####". For example, the code level you are all running right now is called AK7. It has minor updates to it, but the major code is AK7, with a minor now of 04.24.7, so really the last code level released is AK7.04.24.7. You have all been calling it "2011.04.24.7", because in the past they used the year the major release came out as it's name. For obvious reasons, this no longer makes sense. People think the current code they're running was made two years ago in 2011, but that's just not the case. This last release bears almost no resemblance to the original AK7 code. So much has changed in it.So, to make things simpler, Oracle is dropping the year on the code, and will now call it AK#.#.#, starting with the upcoming release of AK8. In all likelihood,  there will still be one more minor release of AK7 coming first, so don't wait to upgrade like AK8 is just around the corner. It's still going to be a few months, and you don't want to hit a bug before that, so upgrade when you can to AK7-7 (my nickname for the current release). AK8 will be a game-changer. I'm not allowed to talk about it too much, but speak with your local storage SC and maybe they can give you a heads-up. HUGE stuff coming folks. Just the performance enhancements are going to be a world-changing event in the storage industry. If you have a 7x20 series system, going to AK8, without doing anything else, is going to make your system better and faster.You can see all of the software release history here: https://wikis.oracle.com/display/FishWorks/Software+Updates

As you now know, version 2011.1.7.0 is the current shipping code for our ZFSSA. You really want to be running this code, no matter what ZFSSA system you have. This code will work all they way back to...

Wikibon has a new article giving nice praise to the ZFSSA

It seems Wikibon has done some research and interviews and has written a very nice article on the awesome cost savings of the ZFSSA. Check it out here:http://wikibon.org/wiki/v/Oracle_ZFS_Hybrid_Storage_Appliance_Reads_for_Show_but_Writes_for_DoughHere are some of my favorite quotes from the article:“The high-end ZFS storage array is the highest performinghybrid storage device that has been analyzed by Wikibon, and in a class of itsown when it comes to high write-IO environments.” “Wikibon analyzed the architecture and performance of theZFS Appliance in depth, and compared it to "traditional storagearrays" (e.g. EMC, NetApp, HDS, HP, etc. mainstream mid-market arrays) inhigh write environments.” “For an environment with 100 terabytes, 1,000,000 IOPS and20% writes, the additional cost of the traditional system (NetApp) is 194%higher than the hybrid system (ZFS). “ “Wikibon members should consider the ZFS Appliance in moredemanding workloads where sustained write performance and IO requirements arehigher. Examples include high performance environments such as specific backupapplications and core transaction-intensive database workloads. In thesesituations, because of the hybrid design of the ZFS Appliance, customers will findsignificant savings relative to traditional disk arrays that don't scale aswell.” “CIOs, CTOs and senior storage executives should positionthe Oracle ZFS appliance as an ideal strategic fit for high streamingenvironments such as database backups. As well, the product can be successfullyintegrated into high-performance Oracle database workloads. In write-intensiveand heavy IO workloads, the ZFS appliance will likely prove the best-of-breed,lowest cost solution.” “The general feedback from the ZFS appliance practitionerswas positive” “Praise for the performance of the ZFS, particularly inbackup (high-write) environment;” “7 gigabytes/second write rates achieved in a benchmarks;” “11 terabytes/hour sustained over 2.5 hours for backup,compared with 1 terabytes/hour for a traditional storage device;” “ZFS snapshots and clones universally praised;” “DTrace was praised for the quality and completeness of theperformance analytic tool;” “Compression performance was strongly praised (up to 16xcompression), especially for reads;” “None of the respondents needed to tune the ZFS read orwrite caching - performance maintenance was minimal;” “No problems with availability.” 

It seems Wikibon has done some research and interviews and has written a very nice article on the awesome cost savings of the ZFSSA.  Check it out here: http://wikibon.org/wiki/v/Oracle_ZFS_Hybrid_Stora...

SnapManager for Oracle DB for ZFSSA is out and ready

A few weeks ago, Oracle announced the Oracle database SnapManager software for ZFSSA. It is a license just like the Clone or the Replication license. It's just a one-time, yes-or-no, on-or-off license per controller. Better yet, you can go ahead and get the software and try it out for free for 30 days. Go check it out with the link below. TheSnap Management Utility combines the underlying snapshot, clone, and rollbackcapabilities of the Oracle ZFS Storage Appliance with standard host-sideprocessing so all operations are consistent. Downloading the Oracle Snap Management Utility forOracle Database Software A. Customers who purchased the license need to downloadthe software from eDelivery (see instructions below) B. Customers who wish to evaluate for 30-days prior topurchase may download from the same site. The license allows a 30-day evaluation period.Follow instructions below. Instructions to download software: 1. Go to eDelivery link: https://edelivery.oracle.com/EPD/Search/handle_go 2. Login 3. Accept Terms and Restrictions 4. In the “MediaPack Search” window: a. Under ProductPack, select “Sun Products” b. UnderPlatform, select “Generic” c. Click “Go” 5. From the results, select the “Oracle Snap ManagementUtility for Oracle Database” 6. There are two files for download: a. The “OracleSnap Management Utility for Oracle Database, Client v 1.1.0” is required b. The “Sun ZFSStorage Software 2011.1.5.0” is the latest version of the ZFS Storage Appliance SW provided for customers who need to upgradetheir software. UPDATE-7-27-13- Just found out that if you buy the SMU license, you do NOT need to buy the clone license. The cloning is included in SMU, so that's cool.

A few weeks ago, Oracle announced the Oracle database SnapManager software for ZFSSA. It is a license just like the Clone or the Replication license. It's just a one-time, yes-or-no, on-or-off license...

Oracle Iaas now includes the ZFS Backup Appliance

Ok, so this is pretty cool. If you didn't know, Oracle has this great program called Iaas, which is Infrastructure As A Service. You can go check it out here: http://www.oracle.com/us/products/engineered-systems/iaas/overview/index.html What this means it that someone who really wants an Oracle engineered system, such as an Exadata, but can't come up with the up-front cost, can do Iaas and put it in their datacenter for a low monthly fee. This can be really cool. Some people can now change their entire budget from a Cap-ex to an Op-ex, save a bunch of up-front costs, and still get the hardware they need and want. As of this week, the ZFSBA is now included in the Iaas offering. So one can get the ZFS Backup Appliance and use it to backup their engineered system (Exadata, Exalogic, or SuperCluster) over infiniband. They can also use it to then make snaps and clones of that data for their testing and development, as well as use it for general-purpose storage over 10Gig, 1Gig or FC. Pretty sweet way to get the ZFS Storage system in your site without the up-front costs. You can get the ZFSBA in a Iaas all by itself if you want, without the engineered system at all, just to get the ZFS storage. Now, some of you may be asking, "What the heck is the ZFSBA and how is it different than the ZFSSA?" I haven't talked about the ZFSBA before. The ZFS Backup appliance. I probably should have. You can get more info on it here: http://www.oracle.com/us/products/servers-storage/storage/nas/zfs-backup-appliance/overview/index.htmlHere is the low-down. It's a 7420 cluster with drive trays, all pre-cabled and in a rack, ready-to-go. The 7420 has IB cards in place and the whole system is a single line-item to make it easy for the sales team to have a single line-item part number to use as an easy way to add a ZFSSA to an engineered system deal for backing up the engineered system. There are two versions, one with high-capacity drives and the other with high-performance drives. Either one you get can add additional trays of either type later. Unlike the other engineered systems, the ZFSBA does allow one to use the extra space in the rack, which is nice.  So, if you want a 7420 cluster and a rack, is there a downside to always using the ZFSBA to order a 7420? Not many. Same price, easier to order with less part numbers. You can still customize it and add more stuff. There is one downside, and that's the fact that the ZFSBA does use the 32-core version of the 7420, not the 40-core version. The backup of an Exadata does not require more cores, so they went with the smaller of the two. If you need more power and more DRAM for faster workloads, however, you may want to build a 7420 ZFSSA the normal way. If this doesn't make sense, please add a comment below or just email me.   Steve 

Ok, so this is pretty cool. If you didn't know, Oracle has this great program called Iaas, which is Infrastructure As A Service. You can go check it out here: http://www.oracle.com/us/products/engineer...

New code and new disk trays !!!

Hey everybody, happy new year and some great news for the ZFSSA... The new 2u disk trays have come out early. I was not expecting them until later this quarter, but was surprised yesterday that Oracle announced them ready for sale. Sweet. So we now have a 4u capacity tray for 3TB drives (soon to be 4TB drives), and a 2u high-performance tray with either 300GB or 900GB 10K speed drives. These new 900GB 10K speed drives have the same IOPS as our current 600GB 15K speed drives, since the form factor went from 3.5" to 2.5". So you now can have 24 drives in a 2u tray. Very cool. These new trays require OS 2011.1.5, and right now you can NOT mix them with the older DS2 trays. Being able to mix them will be supported later, however. To go along with that, the new 2011.1.5 code has been released. you can download it right now in MOS. It fixes a ridiculous amount of  issues, as well as supports these new 2u drive trays. You can read all about the new code here: https://updates.oracle.com/Orion/Services/download?type=readme&aru=15826899 Enjoy!  **Update 1-18-13 - I need to correct myself, and I'm adding this note instead of changing what I wrote up above and trying to hide that I messed up... Hey it happens...At first I was lead to believe the the smaller size platter made up for the slower speed on the new 2.5" drives. This is not the case. It does help, but the 10K speed drives do get slightly less IOPS and throughput then the 3.5" 15K speed drives.  Not that this matters too much for us, since we pride ourselves on the fact we drive performance with the ZFSSA via our cache, not our spindle speed, but it's important to point out.  Now, the power savings and space savings are real, and very much worth using the smaller form factor. Also, you do understand that Oracle does not have a whole lot to do with this? This is the way drive manufacturers are going. They just don't make 2.5" drives at 15K speed. So this is the way it is. Now, at some point sooner rather than later, we will also be putting out an all SSD tray. So if you need fast IOP speeds on the spindles, we will have you covered there, too.

Hey everybody, happy new year and some great news for the ZFSSA... The new 2u disk trays have come out early. I was not expecting them until later this quarter, but was surprised yesterday that Oracle...

My error with upgrading 4.0 to 4.2- What NOT to do...

Last week, I was helping a client upgrade from the 2011.1.4.0 code to the newest 2011.1.4.2 code. We downloaded the 4.2 update from MOS, upload and unpacked it on both controllers, and upgraded one of the controllers in the cluster with no issues at all. As this was a brand-new system with no networking or pools made on it yet, there were not any resources to fail back and forth between the controllers. Each controller had it's own, private, management interface (igb0 and igb1) and that's it. So we took controller 1 as the passive controller and upgraded it first. The first controller came back up with no issues and was now on the 4.2 code. Great. We then did a takeover on controller 1, making it the active head (although there were no resources for it to take), and then proceeded to upgrade controller 2. Upon upgrading the second controller, we ran the health check with no issues. We then ran the update and it ran and rebooted normally. However, something strange then happened. It took longer than normal to come back up, and when it did, we got the "cluster controllers on different code" error message that one gets when the two controllers of a cluster are running different code. But we just upgraded the second controller to 4.2, so they should have been the same, right??? Going into the Maintenance-->System screen of controller 2, we saw something very strange. The "current version" was still on 4.0, and the 4.2 code was there but was in the "previous" state with the rollback icon, as if it was the OLDER code and not the newer code. I have never seen this happen before. I would have thought it was a bad 4.2 code file, but it worked just fine with controller 1, so I don't think that was it. Other than the fact the code did not update, there was nothing else going on with this system. It had no yellow lights, no errors in the Problems section, and no errors in any of the logs. It was just out of the box a few hours ago, and didn't even have a storage pool yet. So.... We deleted the 4.2 code, uploaded it from scratch, ran the health check, and ran the upgrade again. once again, it seemed to go great, rebooted, and came back up to the same issue, where it came to 4.0 instead of 4.2. See the picture below.... HERE IS WHERE I MADE A BIG MISTAKE.... I SHOULD have instantly called support and opened a Sev 2 ticket. They could have done a shared shell and gotten the correct Fishwork engineer to look at the files and the code and determine what file was messed up and fixed it. The system was up and working just fine, it was just on an older code version, not really a huge problem at all. Instead, I went ahead and clicked the "Rollback" icon, thinking that the system would rollback to the 4.2 code.   Ouch... What happened was that the system said, "Fine, I will delete the 4.0 code and boot to your 4.2 code"... Which was stupid on my part because something was wrong with the 4.2 code file here and the 4.0 was just fine.  So now the system could not boot at all, and the 4.0 code was completely missing from the system, and even a high-level Fishworks engineer could not help us. I had messed it up good. We could only get to the ILOM, and I had to re-image the system from scratch using a hard-to-get-and-use FishStick USB drive. These are tightly controlled and difficult to get, almost always handcuffed to an engineer who will drive out to re-image a system. This took another day of my client's time.  So.... If you see a "previous version" of your system code which is actually a version higher than the current version... DO NOT ROLL IT BACK.... It did not upgrade for a very good reason. In my case, after the system was re-imaged to a code level just 3 back, we once again tried the same 4.2 code update and it worked perfectly the first time and is now great and stable.  Lesson learned. By the way, our buddy Ryan Matthews wanted to point out the best practice and supported way of performing an upgrade of an active/active ZFSSA, where both controllers are doing some of the work. These steps would not have helpped me for the above issue, but it's important to follow the correct proceedure when doing an upgrade.1) Upload software to both controllers and wait for it to unpack 2) On controller "A" navigate to configuration/cluster and click "takeover" 3) Wait for controller "B" to finish restarting, then login to it, navigate to maintenance/system, and roll forward to the new software. 4) Wait for controller "B" to apply the update and finish rebooting 5) Login to controller "B", navigate to configuration/cluster and click "takeover" 6) Wait for controller "A" to finish restarting, then login to it, navigate to maintenance/system, and roll forward to the new software. 7) Wait for controller "A" to apply the update and finish rebooting 8) Login to controller "B", navigate to configuration/cluster and click "failback"

Last week, I was helping a client upgrade from the 2011.1.4.0 code to the newest 2011.1.4.2 code. We downloaded the 4.2 update from MOS, upload and unpacked it on both controllers, and upgraded one of...

Quotas - Using quotas on ZFSSA shares and projects and users

So you don't want your users to fill up your entire storage pool with their MP3 files, right? Good idea to make some quotas. There's some good tips and tricks here, including a helpful workflow (a script) that will allow you to set a default quota on all of the users of a share at once. Let's start with some basics. I mad a project called "small" and inside it I made a share called "Share1". You can set quotas on the project level, which will affect all of the shares in it, or you can do it on the share level like I am here. Go the the share's General property page. First, I'm using a Windows client, so I need to make sure I have my SMB mountpoint. Do you know this trick yet? Go to the Protocol page of the share. See the SMB section? It needs a resource name to make the UNC path for the SMB (Windows) users. You do NOT have to type this name in for every share you make! Do this at the Project level. Before you make any shares, go to the Protocol properties of the Project, and set the SMB Resource name to "On". This special code will automatically make the SMB resource name of every share in the project the same as the share name. Note the UNC path name I got below. Since I did this at the Project level, I didn't have to lift a finger for it to work on every share I make in this project. Simple. So I have now mapped my Windows "Z:" drive to this Share1. I logged in as the user "Joe". Note that my computer shows my Z: drive as 34GB, which is the entire size of my Pool that this share is in. Right now, Joe could fill this drive up and it would fill up my pool.  Now, go back to the General properties of Share1. In the "Space Usage" area, over on the right, click on the "Show All" text under the Users & Groups section. Sure enough, Joe and some other users are in here and have some data. Note this is also a handy window to use just to see how much space your users are using in any given share.  Ok, Joe owes us money from lunch last week, so we want to give him a quota of 100MB. Type his name in the Users box. Notice how it now shows you how much data he's currently using. Go ahead and give him a 100M quota and hit the Apply button. If I go back to "Show All", I can see that Joe now has a quota, and no one else does. Sure enough, as soon as I refresh my screen back on Joe's client, he sees that his Z: drive is now only 100MB, and he's more than half way full.  That was easy enough, but what if you wanted to make the whole share have a quota, so that the share itself, no matter who uses it, can only grow to a certain size? That's even easier. Just use the Quota box on the left hand side. Here, I use a Quota on the share of 300MB.  So now I log off as Joe, and log in as Steve. Even though Steve does NOT have a quota, it is showing my Z: drive as 300MB. This would effect anyone, INCLUDING the ROOT user, becuase you specified the Quota to be on the SHARE, not on a person.  Note that back in the Share, if you click the "Show All" text, the window does NOT show Steve, or anyone else, to have a quota of 300MB. Yet we do, because it's on the share itself, not on any user, so this panel does not see that. Ok, here is where it gets FUN.... Let's say you do NOT want a quota on the SHARE, because you want SOME people, like Root and yourself, to have FULL access to it and you want the ability to fill the whole thing up if you darn well feel like it. HOWEVER, you want to give the other users a quota. HOWEVER you have, say, 200 users, and you do NOT feel like typing in each of their names and giving them each a quota, and they are not all members of a AD global group you could use or anything like that.  Hmmmmmm.... No worries, mate. We have a handy-dandy script that can do this for us. Now, this script was written a few years back by Tim Graves, one of our ZFSSA engineers out of the UK. This is not my script. It is NOT supported by Oracle support in any way. It does work fine with the 2011.1.4 code as best as I can tell, but Oracle, and I, are NOT responsible for ANYTHING that you do with this script. Furthermore, I will NOT give you this script, so do not ask me for it. You need to get this from your local Oracle storage SC. I will give it to them. I want this only going to my fellow SCs, who can then work with you to have it and show you how it works.  Here's what it does...Once you add this workflow to the Maintenance-->Workflows section, you click it once to run it. Nothing seems to happen at this point, but something did.   Go back to any share or project. You will see that you now have four new, custom properties on the bottom.  Do NOT touch the bottom two properties, EVER. Only touch the top two. Here, I'm going to give my users a default quota of about 40MB each. The beauty of this script is that it will only effect users that do NOT already have any kind of personal quota. It will only change people who have no quota at all. It does not effect the Root user.  After I hit Apply on the Share screen. Nothing will happen until I go back and run the script again. The first time you run it, it creates the custom properties. The second and all subsequent times you run it, it checks the shares for any users, and applies your quota number to each one of them, UNLESS they already have one set. Notice in the readout below how it did NOT apply to my Joe user, since Joe had a quota set.  Sure enough, when I go back to the "Show All" in the share properties, all of the users who did not have a quota, now have one for 39.1MB. Hmmm... I did my math wrong, didn't I?    That's OK, I'll just change the number of the Custom Default quota again. Here, I am adding a zero on the end.  After I click Apply, and then run the script again, all of my users, except Joe, now have a quota of 391MB  You can customize a person at any time. Here, I took the Steve user, and specifically gave him a Quota of zero. Now when I run the script again, he is different from the rest, so he is no longer effected by the script. Under Show All, I see that Joe is at 100, and Steve has no Quota at all. I can do this all day long. es, you will have to re-run the script every time new users get added. The script only applies the default quota to users that are present at the time the script is ran. However, it would be a simple thing to schedule the script to run each night, or to make an alert to run the script when certain events occur.  For you power users, if you ever want to delete these custom properties and remove the script completely, you will find these properties under the "Schema" section under the Shares section. You can remove them here. There's no need to, however, they don't hurt a thing if you just don't use them.  I hope these tips have helped you out there. Quotas can be fun. 

So you don't want your users to fill up your entire storage pool with their MP3 files, right? Good idea to make some quotas. There's some good tips and tricks here, including a helpful workflow...

Our winners- and some BBQ for everyone

Please also see "Allen's Grilling Channel" over to the right in my Bookmarks section...Congrats to our two winners for the first two comments on mylast entry. Steve from Australia and John Lemon. Steve won since he was thefirst person over the International Date Line to see the post I made so lateafter a workday on Friday. So not only does he get to live in a country withthe 2nd most beautiful women in the world, but now he gets some coolOracle Swag, too. (Yes, I live on the beach in southern California, so you canguess where 1st place is for that other contest…Now if Steve happensto live in Manly, we may actually have a tie going…) OK, ok, for everyone else, you can be winners, too. How youask? I will make you the envy of every guy and gal in your neighborhood orcampsite. What follows is the way to smoke the best ribs you or anyone you knowhave ever tasted. Follow my instructions and give it a try. People at yourparty/cookout/campsite will tell you that they’re the best ribs they’ve everhad, and I will let you take all the credit. Yes, I fully realize this post isgoing to be longer than any post I’ve done yet. But let’s get serious here.Smoking meat is much more important, agreed? JIn all honesty, this is a repeat of another blog I did, so I’m just copying andpasting. Step 1. Get some ribs. I actually really like Costco’s pack.They have both St. Louis and Baby Back. (They are the same ribs, but cut inhalf down the sides. St. Louis style is the ‘front’ of the ribs closest to thestomach, and ‘Baby back’ is the part of the ribs where is connects to thebackbone). I like them both, so here you see I got one pack of each. About 4racks to a pack. So these two packs for $25 each will feed about 16-20 of myguests. So around 3 bucks a person is a pretty good deal for the best ribsyou’ll ever have. Step 2. Prep the ribs the night before you’re going tosmoke. You need to trim them to fit your smoker racks, and also take off themembrane and add your rub. Then cover and set in fridge overnight. Here’s howto take off the membrane, which will not break down with heat and smoke likethe rest of the meat, so must be removed. Use a butter knife to work in a waysbetween the membrane and the white bone. Just enough to make room for yourfinger. Try really hard not to poke through the membrane, you want to keep itwhole. See how my gloved fingers can now start to lift up and pulloff the membrane? This is what you are trying to do. It’s awesome when thewhole thing can come off at once. This one is going great, maybe the best oneI’ve ever done. Sometime, it falls apart and doesn't come off in one nicepiece. I hate when that happens. Now, add your rub and pat it down once into the meat withyour other hand. My rub is not secret. I got it from my mentor, a BBQcompetitive chef who is currently ranked #1 in California and #3 in the nationon the BBQ circuit. He does full-day classes in southern California if anyoneis interested in taking his class. Go to www.slapyodaddybbq.comto check him out. I tweaked his run recipe a tad and made my own. It’s one partLawry’s, one part sugar, one part Montreal Steak Seasoning, one part garlicpowder, one-half part red chili powder, one-half part paprika, and then 1/20thpart cayenne. You can adjust that last ingredient, or leave it out. Real cheapstuff you can get at Costco. This lets you make enough rub to last about a yearor two. Don’t make it all at once, make a shaker’s worth and use it up beforeyou make more. Place it all in a bowl, mix well, and then add to a shaker likeyou see here. You can get a shaker with medium sized holes on it at anyrestaurant supply store or Smart & Final. The kind you see at pizza placesfor their red pepper flakes works best. Now cover and place in fridge overnight. Step 3. The next day. Ok, I’m ready to go. Get your stufftogether. You will need your smoker, some good foil, a can of peach nectar, abottle of Agave syrup, and a package of brown sugar. You will need this stufflater. I also use a clean spray bottle, and apple juice. Step 4. Make your fire, or turn on your electric smoker. Inthis example I’m using my portable charcoal smoker. I got this for only $40. Ithen modified it to be useful. Once modified, these guys actually work verywell. Trust me, your food DOES NOT KNOW how expensive your smoker is. Someonewho tells you that you need to spend a bunch of money on a smoker is an idiot.I also have an electric smoker that stays in my backyard. It’s cleaner andlarger so I can smoke more food. But this little $40 one works great for goingcamping. Here is what my fire-bowl looks like. I leave a space in the middle open,and place cold charcoal and wood chucks in a circle going outwards. This makesit so when I dump the hot coals down the middle, they will slowly burnoutwards, hitting different wood chucks at different times, allowing me to go4-5 hours without having to even touch my fire. For ribs, I use apple and pecanwood. Pecan works for anything. Apple or any fruit wood is excellent for pork. So now I make my hot charcoal with a chimney only abouthalf-full. I found a great use for that side-burner on my grill that I neveruse. It makes a fantastic chimney starter. You never use fluids of any kind,nor ever use that stupid charcoal that has lighter fluid built into it. Never,ever, ever. Step 5. Smoke. Add your ribs in the racks and stack them upin your smoker. I have a digital thermometer on a probe that I use to keeptrack of the temp in the smoker. I just lay the probe on the top rack and shutthe lid. This cheap guy is a little harder to maintain the right temperature ofaround 225 F, so I do have to keep my eye on it more than my electric one or amore expensive charcoal one with the cool gadgets that regulate your temp foryou. Every hour, spray apple juice all over your ribs using thatspray bottle. After about 3 hours, you should have a very good crust (calledthe Bark) on your ribs. Once you have the Bark where you want it, carefullyremove your ribs and place them in a tray. We are now ready for a veryimportant part to make the flavor. Get a large piece of foil and place one rib section on it.Splash some of the peach nectar on it, and then a drizzle of the Agave syrup.Then, use your gloved hand to pack on some brown sugar. Do this on BOTH sides,and then completely wrap it up TIGHT in the foil. Do this for each rib section,and then place all the wrapped sections back into the smoker for another 4 to 6hours. This is where the meat will get tender and flavorful. The first threehours is only to make the smoke bark. You don’t need smoke anymore, since theribs are wrapped, you only need to keep the heat around 225 for the next 4-6hours. Obviously you don’t spray anymore. Just time and slow heat. Be patient.It’s actually really hard to overdo it. You can let them go longer, and allthat will happen is they will get even MORE tender!!! If you take them out toosoon, they will be tough. How do you know? Take out one package (use long tongs) andopen it up. If you grab a bone with your tongs and it just falls apart andbreaks away from the rest of the meat, you are done!!! Enjoy!!! Step 6. Eat. It pulls apart like this when it’s done. By the way, smoking tri-tip is way easier. Just rub it withthe same rub, and put in your smoker for about 2.5 hours at 250 F. That’s it.Low-maintenance. It comes out like this, with a fantastic smoke ring andamazing flavor. Thanks, and I will put up another good tip, about the ZFSSA, around the end of November. Steve 

Please also see "Allen's Grilling Channel" over to the right in my Bookmarks section... Congrats to our two winners for the first two comments on mylast entry. Steve from Australia and John Lemon....

Replication - between pools in the same system

OK, I fully understand that's it's been a LONG time since I've blogged with any tips or tricks on the ZFSSA, and I'm way behind. Hey, I just wrote TWO BLOGS ON THE SAME DAY!!! Make sure you keep scrolling down to see the next one too, or you may have missed it. To celebrate, for the one or two of you out there who are still reading this, I got something for you. The first TWO people who make any comment below, with your real name and email so I can contact you, will get some cool Oracle SWAG that I have to give away. Don't get excited, it's not an iPad, but it pretty good stuff. Only the first two, so if you already see two below, then settle down. Now, let's talk about Replication and Migration.  I have talked before about Shadow Migration here: https://blogs.oracle.com/7000tips/entry/shadow_migrationShadow Migration lets one take a NFS or CIFS share in one pool on a system and migrate that data over to another pool in the same system. That's handy, but right now it's only for file systems like NFS and CIFS. It will not work for LUNs. LUN shadow migration is a roadmap item, however. So.... What if you have a ZFSSA cluster with multiple pools, and you have a LUN in one pool but later you decide it's best if it was in the other pool? No problem. Replication to the rescue. What's that? Replication is only for replicating data between two different systems? Who told you that? We've been able to replicate to the same system now for a few code updates back. These instructions below will also work just fine if you're setting up replication between two different systems. After replication is complete, you can easily break replication, change the new LUN into a primary LUN and then delete the source LUN. Bam. Step 1- setup a target system. In our case, the target system is ourself, but you still have to set it up like it's far away. Go to Configuration-->Services-->Remote Replication. Click the plus sign and setup the target, which is the ZFSSA you're on now. Step 2. Now you can go to the LUN you want to replicate. Take note which Pool and Project you're in. In my case, I have a LUN in Pool2 called LUNp2 that I wish to replicate to Pool1.  Step 3. In my case, I made a Project called "Luns" and it has LUNp2 inside of it. I am going to replicate the Project, which will automatically replicate all of the LUNs and/or Filesystems inside of it.  Now, you can also replicate from the Share level instead of the Project. That will only replicate the share, and not all the other shares of a project. If someone tells you that if you replicate a share, it always replicates all the other shares also in that Project, don't listen to them.Note below how I can choose not only the Target (which is myself), but I can also choose which Pool to replicate it to. So I choose Pool1.  Step 4. I did not choose a schedule or pick the "Continuous" button, which means my replication will be manual only. I can now push the Manual Replicate button on my Actions list and you will see it start. You will see both a barber pole animation and also an update in the status bar on the top of the screen that a replication event has begun. This also goes into the event log.  Step 5. The status bar will also log an event when it's done. Step 6. If you go back to Configuration-->Services-->Remote Replication, you will see your event. Step 7. Done. To see your new replica, go to the other Pool (Pool1 for me), and click the "Replica" area below the words "Filesystems | LUNs" Here, you will see any replicas that have come in from any of your sources. It's a simple matter from here to break the replication, which will change this to a "Local" LUN, and then delete the original LUN back in Pool2. Ok, that's all for now, but I promise to give out more tricks sometime in November !!! There's very exciting stuff coming down the pipe for the ZFSSA. Both new hardware and new software features that I'm just drooling over. That's all I can say, but contact your local sales SC to get a NDA roadmap talk if you want to hear more.   Happy Halloween,Steve 

OK, I fully understand that's it's been a LONG time since I've blogged with any tips or tricks on the ZFSSA, and I'm way behind. Hey, I just wrote TWO BLOGS ON THE SAME DAY!!! Make sure you...

New Write Flash SSDs and more disk trays

In case you haven't heard, the Write SSDs the ZFSSA have been updated. Much faster now for the same price. Sweet. The new write-flash SSDs have a new part number of 7105026 , so make sure you order the right ones. It's important to note that you MUST be on code level 2011.1.4.0 or higher to use these. They have increased in IOPS from 6,000 to 11,000, and increased throughput from 200MB/s to 350MB/s.    Also, you can now add six SAS HBAs (up from 4) to the 7420, allowing one to have three SAS channels with 12 disk trays each, for a new total of 36 disk trays. With 3TB drives, that's 2.5 Petabytes. Is that enough for you? Make sure you add new cards to the correct slots. I've talked about this before, but here is the handy-dandy matrix again so you don't have to go find it. Remember the rules: You can have 6 of any one kind of card (like six 10GigE cards), except IB which is still four max. You only really get 8 slots, since you have two SAS cards no matter what. If you want more than 12 disk trays, you need two more SAS cards, so think about expansion later, too. In fact, if you're going to have two different speeds of drives (in other words you want to mix 15K speed and 7,200 speed drives in the same system), I would highly recommend two different SAS channels. So I would want four SAS cards in that system, no matter how many trays you have. 

In case you haven't heard, the Write SSDs the ZFSSA have been updated. Much faster now for the same price. Sweet. The new write-flash SSDs have a new part number of 7105026 , so make sure you order the...

Phone Home- just like E.T.

Hmmm, still no update, so they have changed the ETA from "July" to "It will be out when it's ready". I have not heard of a new ETA, so please don't ask. In the meantime, there are plenty of you that do not have the Automated Service Request (ASR) feature turned on for your ZFSSA systems. This is better known as "Phone Home". It's not only extremely handy and free, but it could possibly save your job and your company lots of time and money. You really, really want to turn it on. The Phone Home feature on the ZFSSA does two things. It obviously creates a support ticket with Oracle support in the event of some failure on the ZFSSA. That's good. You will see an email that this happened, and can then go track it with your account on the MOS (My Oracle Support) website. If you're wondering what issues will force an ASR, then go back and read my blog entry from last year here: https://blogs.oracle.com/7000tips/entry/asr_automated_service_request_aka You need to make sure your ZFSSA system can get to the internet and has access through your firewalls to the following sites and ports:1. inv-cs.oracle.com  on port 4432. asr-services.oracle.com  on port 443 The other thing it does is send heartbeats to the Oracle ASR phone home database. As a pre-sales engineer, I find this very handy for my clients. I'll show you a few screenshots below of what I can see in the phone home database for my clients. This lets me keep track of minor, major, and critical issues my customers are having with their ZFSSA systems. I can see how their storage pools are setup and if they are becoming too full. This is something I like to track and keep my eye on for my customers. I will offer to create summary reports for them on a monthly basis to help them keep track of their systems, as many of them don't look at them very often or have too many other things to worry about. Your local Storage SC can also access this database and show you what he or she sees for your systems on Phone Home. Here is where you setup Phone Home in the ZFSSA BUI interface: Here is an example of what I can see in the Phone Home database.  In this example, just lower down on the same screen, you can see the list of issues this system has had over the years, and a click will get you more detail. This system had some hard drives replaced in January, and one in May, but nothing of note since then. Many of the other 'Major' alerts you see below were actually just cluster peer takeovers done for testing and upgrades.

Hmmm, still no update, so they have changed the ETA from "July" to "It will be out when it's ready". I have not heard of a new ETA, so please don't ask. In the meantime, there are plenty of you that do...

New Analytic settings for the new code

If you have upgraded to the new 2011.1.3.0 code, you may find some very useful settings for the Analytics. If you didn't already know, the analytic datasets have the potential to fill up your OS hard drives. The more datasets you use and create, that faster this can happen. Since they take a measurement every second, forever, some of these metrics can get in the multiple GB size in a matter of weeks. The traditional 'fix' was that you had to go into Analytics -> Datasets about once a month and clean up the largest datasets. You did this by deleting them. Ouch. Now you lost all of that historical data that you might have wanted to check out many months from now. Or, you had to export each metric individually to a CSV file first. Not very easy or fun. You could also suspend a dataset, and have it not collect data at all. Well, that fixed the problem, didn't it? of course you now had no data to go look at. Hmmmm.... All of this is no longer a concern. Check out the new Settings tab under Analytics... Now, I can tell the ZFSSA to keep every second of data for, say, 2 weeks, and then average those 60 seconds of each minute into a single 'minute' value. I can go even further and ask it to average those 60 minutes of data into a single 'hour' value.  This allows me to effectively shrink my older datasets by a factor of 1/3600 !!! Very cool. I can now allow my datasets to go forever, and really never have to worry about them filling up my OS drives. That's great going forward, but what about those huge datasets you already have? No problem. Another new feature in 2011.1.3.0 is the ability to shrink the older datasets in the same way. Check this out. I have here a dataset called "Disk: I/O opps per second" that is about 6.32M on disk (You need not worry so much about the "In Core" value, as that is in RAM, and it fluctuates all the time. Once you stop viewing a particular metric, you will see that shrink over time, just relax).  When one clicks on the trash can icon to the right of the dataset, it used to delete the whole thing, and you would have to re-create it from scratch to get the data collecting again. Now, however, it gives you this prompt: As you can see, this allows you to once again shrink the dataset by averaging the second data into minutes or hours. Here is my new dataset size after I do this. So it shrank from 6.32MB down to 2.87MB, but i can still see my metrics going back to the time I began the dataset. Now, you do understand that once you do this, as you look back in time to the minute or hour data metrics, that you are going to see much larger time values, right? You will need to decide what size of granularity you can live with, and for how long. Check this out.Here is my Disk: Percent utilized from 5-21-2012 2:42 pm to 4:22 pm: After I went through the delete process to change everything older than 1 week to "Minutes", the same date and time looks like this: Just understand what this will do and how you want to use it. Right now, I'm thinking of keeping the last 6 weeks of data as "seconds", and then the last 3 months as "Minutes", and then "Hours" forever after that. I'll check back in six months and see how the sizes look.Steve 

If you have upgraded to the new 2011.1.3.0 code, you may find some very useful settings for the Analytics. If you didn't already know, the analytic datasets have the potential to fill up your OS hard...

New code is out- Version 2011.1.3.0

The newest version of the ZFSSA code, 2011.1.3.0, is now out and availiable on MOS.I will be writing more about one of it's many new, useful features coming early next week. It's very cool, and has to do with how you can now change the size of your analytic datasets.Steve ak-2011.04.24.3.0 Release NotesAdded by 10197, last edited by jwalker on May 15, 2012  (view change)2011.1.3.0This minor release of the Sun ZFS Storage Appliance software contains significant bug fixes for all supported platforms. Please carefully review the list of CRs that have been addressed and all known issues prior to updating.Among other issues, this release fixes some memory fragmentation issues (CRs 7092116 and 7105404), includes improvements to DTrace Analytics, and failover improvements to DNS, LDAP, and the SMB Domain Controller.This release requires appliances to be running the 2010.Q3.2.1 micro release or higher prior to updating to this release. In addition, this release includes update health checks that are performed automatically when an update is started prior to the actual update from the prerequisite 2010.Q3.2.1 micro release or higher. If an update health check fails, it can cause an update to abort. The update health checks help ensure component issues that may impact an update are addressed. It is important to resolve all hardware component issues prior to performing an update.Deferred UpdatesWhen updating from a 2010.Q3 release to a 2011.1 release, the following deferred updates are available and may be reviewed in the Maintenance System BUI screen. See the "Maintenance:System:Updates#Deferred_Updates" section in the online help for important information on deferred updates before applying them.1. RAIDZ/Mirror Deferred Update (Improved RAID performance)This deferred update improves both latency and throughput on several important workloads. These improvements rely on a ZFS pool upgrade provided by this update. Applying this update is equivalent to upgrading the on-disk ZFS pool to version 29.2. Optional Child Directory Deferred Update (Improved snapshot performance)This deferred update improves list retrieval performance and replication deletion performance by improving dataset rename speed. These improvements rely on a ZFS pool upgrade provided by this update. Before this update has been applied, the system will be able to retrieve lists and delete replications, but will do so using the old, much slower, recursive rename code. Applying this update is equivalent to upgrading the on-disk ZFS pool to version 31.Supported PlatformsSun Storage 7110Sun Storage 7210Sun Storage 7310Sun Storage 7410Sun ZFS Storage 7120Sun ZFS Storage 7320Sun ZFS Storage 7420Sun ZFS Backup ApplianceSun ZFS 7000 Storage Appliance SimulatorIssues AddressedThe following CRs have been fixed in this release:4325892performance decrease if 1st nameserver is down4377911RFE for improved DNS resolver failover performance6822262Windows Media Service/SQL server cannot connect to cifs share6822586SMB should try UDP first to get LDAP SRV record and retry with TCP if truncated UDP response6941854idmap_getwinnamebygid() and idmap_getwinnamebyuid() need to work for builtin names6953716Exception 'Native message: datasets is undefined' while append dataset to current worksheet6973870Specify retention time for Analytics data6991949panic will happen during some error injection stress6996698The SRAO may terminate irrelevant memory copy6997450gcpu_mca_process() doesn't return a right disp for poisoned error7023548replacement failed for faulted readzilla7040757smb_com_write_andx NULL pointer dereference panic in mbc_marshal_get_uio7044065Replay records within a dataset in parallel7047976zil replay assertion failure with full pool7048780I/O sent from iSCSI initiator embedded in VirtualBox completes with a status TASK SET FULL7052406NFS Server shouldn't take zero copy path on READ if no write chunk list provided7052703zl_replay_lock needs to be initialised and destroyed7066080s11n code generated by fcc should include strings.h7066138fcc must define _INT64_TYPE7066170configurable max- and min-units for akInputDuration7066552NFS Server Fails READS with NFS4ERR_INVAL when using krbi or krb5p7071147DC failover improvements7071628ldap_cachemgr exits after failure in profile refresh, only when using sasl/GSSAPI authentication7071916ztest/ds_3 missing log records: replayed X < committed Y7074722dataspan can be marked read-only even if it has dirty subspans7080443LDAP client failover doesn't work7080790ZIL: Assertion failed: zh->zh_replay_seq < *replayed_seq (0x1a < 0x1a)7084762ztest/ds_3 missing log records: replayed X < committed Y - Part 27089422ldap client uses bindTimeLimit instead of searchTimeLimit when searching for entries7090133Large READs are broken with krb5i or krb5p with NFS Zero-Copy turned on7090153table-free akInputRadio7090166ak_dataspan_stashed needs a reality check7091223command to prune datasets7092116Extremely sluggish 7420 node due to heap fragmentation7093687LDAP client/ldap_cachemgr: long delays in failover to secondary Directory Server7098553deadlock when recursive zfs_inactive collides with zfs_unmount7099848Phone Home logs still refer to SUN support7102888akShow() can clobber CSS7103620akInputRadio consumers must be explicit in their use of subinputs as labels7104363Influx of snapshots can stall resilvering7105404appliance unavailable due to zio_arena fragmentation7107750SMB kernel door client times out too early on authentication requests7108243ldap_cachemgr spins on configuration error7114579Operations per second broken down by share reports "Datum not present" for most time periods7114890Ak Build tools should accommodate double-slashes in paths7117823RPC: Can't decode result after READ of zero bytes7118230Need to deliver CMOS images for Lynxplus SW 1.57121760failure to post an alert causes a umem double-free7122403akCreateLabel(): helper function for creating LABEL elements7122405several Analytics CLI commands do not check for extra arguments7122426akParseDateTime() could be a bit more flexible7123096panic: LU is done with the task but LPORT is not done, itask ffffff9f59c540a0 itask_flags 32047125626fmtopo shows duplicate target-path for both sims in a tray starting in 2010.q3.4 and 2011.1.17126842NTLMSSP negotiation fails with 0xC00000BB (NT_STATUS_NOT_SUPPORTED)7128218uio_to_mblk() doesn't check for esballoca() failure7129787status could be uninitialized in netlogon_logon function7130441CPU is pegging out at 98%7131965SMB stops serving data (still running) Need to reset smb to fix issue7133069smbserver locks up on 7320 running 2011.1. Can't kill the service7133619Need to deliver CMOS image for SW 1.3 for Otoro7133643Need to deliver CMOS image for SW 1.2 for Otoro+7142320Enable DNS defer-on-fail functionality7144155idmap kernel module has lock contention calling zone_getspecific()7144745Online help Application Integration - MOS should be replaced with OTN7145938Add maximum cards for 7420 10GbE, FC, Quad GbE, and InfiniBand7146346Online Help: Document Sun ZFS Backup Appliance7149992Update doc to include 7320 DRAM7152262double-digit firefox version number throws off appliance version checks7153789ldapcachemgr lint warnings7154895Remove CMOS images for Lynx+ SW 1.57155512lint warning in usr/src/cmd/ldapcachemgr/cachemgr_change.c7158091BUI Alert Banner and Wait Dialog are not functioning correctly7158094Still not ready for 90057158519dataset class authorization doesn't work as expected7158522pruning an unsaved dataset does nothing but looks like it working continuously7160553NMI does not panic appliance platforms using apix7161060system hang due to physical memory exhaustion seen when major shift in workload7165883arc data shrinks continuously after arc grew to reach its steady state

The newest version of the ZFSSA code, 2011.1.3.0, is now out and availiable on MOS. I will be writing more about one of it's many new, useful features coming early next week. It's very cool, and has to...

Analytics & Threshold Alerts

Alerts are great for not only letting you know when there's some kind of hardware event, but they can also be pro-active and let you know there's a bottleneck coming BEFORE it happens. Check these out. There are two kinds of Alerts in the ZFSSA. When you go to Configuration-->Alerts, you fist see the plus sign by the "Alert Actions" section. These are pretty self-explanatory and not what I'm talking about today. Click on the "Threshold Alerts", and then click the plus sign by those. This is what I'm talking about. The default one that comes up, "CPU: Percent Utilization" is a good one to start with. I don't mind if my CPUs go to 100% utilized for a short time. After all, we bought them to be used, right? If they go over 90% for over 10 minutes, however, something is up, and maybe we have workloads on this machine it was not designed for, or we don't have enough CPUs in the system and need more. So we can setup an alert that will keep an eye on this for us and send us an email if this were to occur. Now I don't have to keep watching it all the time. For an even better example, keep reading... What if you want to keep your eyes on whether your Readzillas or Logzillas are being over-utilized? In other words, do you have enough of them? Perhaps you only have 2 Logzillas, and you think you may be better off with 4, but how do you prove it? No problem. Here in Threshold Alerts, click on the Threshold drop-down box, and choose your "Disk: Percent Utilization for Disk: Jxxxxx 013" choice, which is my Logzilla drive in the Jxxxxx tray.Wait. What's that? You don't have a choice in your drop-down for the Threshold item you are looking for, such as an individual disk? Well, we will have to fix that. Leave Alerts for now, and join me over in Analytics. Start with a worksheet with "Disk: Percent utilization broken down by Disk" chart. You do have this, as it's already one of your built-in datasets.Now, expand it so you can see all of your disks, and find one of your Readzilla or Logzilla drives. (Hint: It will NOT be disk 13 like my example here. Logzillas are always in the 20, 21, 22, or 23 slots of a disk tray. Go to your Configuration-->Hardware screens and you can easily find out which drives are which for your system).Now, click on that drive to highlight it, like this:   Click on the Drill Button, and choose to drill down on that drive as a raw statistic. You will now have a whole new data chart, just for that one drive.  Don't go away yet. You now need to save that chart as a new dataset, which will keep it in your ZFSSA analytic metrics forever. Well, until you delete it.Click on the "Save" button, the second to last button on that chart. It looks like a circle with white dots on it (it's supposed to look like a reel-to-reel tape spindle). Now go to your "Analytics-->Datasets", and you will see a new dataset in there for it.   Go back to your Threshold Alerts, and you will now be able to make an alert that will tell you if this specific drive goes over 90% for more than 10 minutes. If this happens a lot, you probably need more Readzillas or Logzillas. I hope you like these Alerts. They may take some time to setup at first, but in the long run you may thank yourself. It might not be a bad idea to send the email alerts to a mail distribution list, instead of a single person who may be on vacation when the alert is hit.  Enjoy. 

Alerts are great for not only letting you know when there's some kind of hardware event, but they can also be pro-active and let you know there's a bottleneck coming BEFORE it happens. Check...

Route Table Stuff

Let's talk about your Routing Table. I have never installed a ZFSSA, ever, without having to edit this table. If you believe that you do not need to edit your routing table, then you are wrong.:)  Ok, maybe not. Maybe you only have your ZFSSA connected to one network with only a few systems on it. I guess it's possible. Even in my simulator, however, I had to edit the routing table so I could use it no matter how I had my laptop connected, at home over a VPN or at work or using a public Wifi. So I'm going to bet a nice dinner that you, or someone, should be checking this out. First things first. I'm going to assume you have a cluster. I try really hard to only sell clusters, but yes I know there are plenty of single-nodes out there too. Single-node people can skip these first two paragraphs. It's very important in your cluster to have a 1GigE management interface to each of the two controllers. You really want to be able to manage each controller, even when one of them is down, right? So best practice is to use the 'igb0' port for controller 1 management and to use the 'igb1' port for controller 2 management. It's important to make these ports 'Private' in the cluster configuration screen, so they do NOT failover to the other controller when a cluster takeover takes place for whatever reason. Igb0 and igb1 are two of the four built-in 1GigE ports. You can still use igb2 and igb3 for data, either alone or as an aggregate, and don't make them private, so they DO failover in a cluster takeover event. Now go to your remote workstation, which may be over a different subnet, and you should be able to ping and connect to Controller 1 using igb0.Now, back to the routing table. You have probably noticed that you can not ping or connect to the other controller, and you think something is wrong. Not to worry, everything is fine. You just need to tell your routing table, which is shared between the heads, how to talk to that other port, igb1. You see, you have a default route setup already for port igb0, that's why it works. Your new, private, igb1 however, does not know how to speak back to your remote system you are now using to manage via the BUI from a different subnet. So, make a new default route for igb1 and point it to the default gateway, which is the router it needs to use in order to cross subnets. See the picture below. Note how I have a default route for "ZFS1-MGMT" for port igb0. This shows a green light because I'm currently on ZFS1, and it sees this port just fine. I also have a default route for "ZFS2-MGMT" from port igb1. This route has a blue light, showing it as inactive. That's because this controller, ZFS1, has nothing plugged into it's igb1 port. That's perfect. Hit "Apply". Now count to 10. Now from your remote host, go ahead and ping or connect to Controller 2, and it works!!! This is because your controllers share a routing table, and when you added that igb1 route, it propagated over to the other controller, where igb1 is plugged in, and that route has a green light over there and it works fine. You will see from Controller 2's point of view that igb1 has a green light and igb0 has a blue light.  (continued below the picture) Now it's time to setup any static routes you may need. If you have different subnets for your 1GigE management and your IB or 10GigE data (a very good idea), then you will need to make these. It's important to have routes for this, as you do not want data coming in over the 10GigE pipe, but then returning over the 1GigE pipe, right? That will happen if this is not setup correctly. Make your routes, as the picture example shows with a 10Gig aggragate here we called "Front-end-IP". Any traffic coming in from subnet 172.20.69 will use this pipe. Lastly, check your multi-homing model button up top. I like 'Adaptive'. Loose is the default, and makes it so your packets can traverse your routes, even though they may go over the wrong route, so it seems like your system is working. This can very well be an illusion. Your ping may work, but it may be coming from the wrong interface, as "Loose" basically means the ZFSSA just doesn't care or enforce any rules. "Strict", on the other hand, is great if you want total enforcement. If you are very good with your routes, and are positive you have it right, and want to ensure that a packet never goes the wrong way, even if that means dropping the packet, then use Strict. I'm using Adaptive here, which is a happy medium.  From the help file: The "Adaptive" choice will prefer routes with a gateway address on the same subnet as the packet's source IP address: 1) An IP packet will be accepted on an IP interface so long as its destination IP address is up on the appliance. 2) An IP packet will be transmitted over the IP interface tied to the route that most specifically matches an IP packet's destination address. If multiple routes are equally specific, prefer routes that have a gateway address on the same subnet as the packet's source address. If no eligible routes exist, drop the packet.Update 4/23/12- My colleague, Darius (https://blogs.oracle.com/si/), rightfully wanted me to point out how important it was to setup a static route for replication. You do not want replication to go over a private management port by mistake, as this will cause it to fail when one controller or the other goes down for maintenance. I hope this helps. Routing can be fun. 

Let's talk about your Routing Table. I have never installed a ZFSSA, ever, without having to edit this table. If you believe that you do not need to edit your routing table, then you are wrong.:)  Ok,...

New SPC2 benchmark- The 7420 KILLS it !!!

This is pretty sweet. The new SPC2 benchmark came out last week, and the 7420 not only came in 2nd of ALL speed scores, but came in #1 for price per MBPS.Check out this table. The 7420 score of 10,704 makes it really fast, but that's not the best part. The price one would have to pay in order to beat it is ridiculous. You can go see for yourself at http://www.storageperformance.org/results/benchmark_results_spc2The only system on the whole page that beats it was over twice the price per MBPS. Very sweet for Oracle.So let's see, the 7420 is the fastest per $. The 7420 is the cheapest per MBPS. The 7420 has incredible, built-in features, management services, analytics, and protocols. It's extremely stable and as a cluster has no single point of failure. It won the Storage Magazine award for best NAS system this year. So how long will it be before it's the number 1 NAS system in the market? What are the biggest hurdles still stopping the widespread adoption of the ZFSSA? From what I see, it's three things: 1. Administrator's comfort level with older legacy systems. 2. Politics 3. Past issues with Oracle Support.  I see all of these issues crop up regularly. Number 1 just takes time and education. Number 3 takes time with our new, better, and growing support team. many of them came from Oracle and there were growing pains when they went from a straight software-model to having to also support hardware. Number 2 is tricky, but it's the job of the sales teams to break through the internal politics and help their clients see the value in oracle hardware systems. Benchmarks like this will help.

This is pretty sweet. The new SPC2 benchmark came out last week, and the 7420 not only came in 2nd of ALL speed scores, but came in #1 for price per MBPS. Check out this table. The 7420 score of 10,704...

Fun tips with Analytics

If you read this blog, I am assuming you are at leastfamiliar with the Analytic functions in the ZFSSA. They are basically amazing,very powerful and deep. However, you may not be aware of some great, hiddenfunctions inside the Analytic screen. Once you open a metric, the toolbar looks like this: Now, I’m not going over every tool, as we have done thatbefore, and you can hover your mouse over them and they will tell you what theydo. But…. Check this out.Open a metric (CPU Percent Utilization works fine), and click on the “Hour”button, which is the 2nd clock icon. That’s easy, you are nowlooking at the last hour of data. Now, hold down your ‘Shift’ key, and click itagain. Now you are looking at 2 hours of data. Hold down Shift and click itagain, and you are looking at 3 hours of data. Are you catching on yet? You can do this with not only the ‘Hour’ button, but also with the ‘Minute’, ‘Day’,‘Week’, and the ‘Month’ buttons. Very cool. It also works with the ‘ShowMinimum’ and ‘Show Maximum’ buttons, allowing you to go to the next iterationof either of those. One last button you can Shift-click is the handy ‘Drill’button. This button usually drills down on one specific aspect of your metric.If you Shift-click it, it will display a “Rainbow Highlight” of the currentmetric. This works best if this metric has many ‘Range Average’ items in theleft-hand window. Give it a shot. Also, one will sometimes click on a certain second of data in the graph, like this:  In this case, I clicked 4:57 and 21 seconds, and the 'Range Average' on the left went away, and was replaced by the time stamp. It seems at this point to some people that you are now stuck, and can not get back to an average for the whole chart. However, you can actually click on the actual time stamp of "4:57:21" right above the chart. Even though your mouse does not change into the typical browser finger that most links look like, you can click it, and it will change your range back to the full metric. Another trick you may like is to save a certain view or look of a group of graphs. Most of you know you can save a worksheet, but did you know you could Sync them, Pause them, and then Save it? This will save the paused state, allowing you to view it forever the way you see it now.  Heatmaps. Heatmaps are cool, and look like this:  Some metrics use them and some don't. If you have one, and wish to zoom it vertically, try this. Open a heatmap metric like my example above (I believe every metric that deals with latency will show as a heatmap). Select one or two of the ranges on the left. Click the "Change Outlier Elimination" button. Click it again and check out what it does.  Enjoy. Perhaps my next blog entry will be the best Analytic metrics to keep your eyes on, and how you can use the Alerts feature to watch them for you. Steve 

If you read this blog, I am assuming you are at least familiar with the Analytic functions in the ZFSSA. They are basically amazing, very powerful and deep. However, you may not be aware of some great,...

Using all Ten IO slots on a 7420

So I had the opportunity recently to actually use up all ten slots in a clustered 7420 system. This actually uses 20 slots, or 22 if you count the clusteron card. I thought it was interesting enough to share here. This is at one of my clients here in southern California. You can see the picture below. We have four SAS HBAs instead of the usual two. This is becuase we wanted to split up the back-end taffic for different workloads. We have a set of disk trays coming from two SAS cards for nothing but Exadata backups. Then, we have a different set of disk trays coming off of the other two SAS cards for non-Exadata workloads, such as regular user file storage. We have 2 Infiniband cards which allow us to do a full mesh directly into the back of the nearby, production Exadata, specifically for fast backups and restores over IB. You can see a 3rd IB card here, which is going to be connected to a non-production Exadata for slower backups and restores from it.The 10Gig card is for client connectivity, allowing other, non-Exadata Oracle databases to make use of the many snapshots and clones that can now be created using the RMAN copies from the original production database coming off the Exadata. This allows for a good number of test and development Oracle databases to use these clones without effecting performance of the Exadata at all.We also have a couple FC HBAs, both for NDMP backups to an Oracle/StorageTek tape library and also for FC clients to come in and use some storage on the 7420.  Now, if you are adding more cards to your 7420, be aware of which cards you can place in which slots. See the bottom graphic just below the photo.  Note that the slots are numbered 0-4 for the first 5 cards, then the "C" slots which is the dedicated Cluster card (called the Clustron), and then another 5 slots numbered 5-9. Some rules for the slots: Slots 1 & 8 are automatically populated with the two default SAS cards. The only other slots you can add SAS cards to are 2 & 7. Slots 0 and 9 can only hold FC cards. Nothing else. So if you have four SAS cards, you are now down to only four more slots for your 10Gig and IB cards. Be sure not to waste one of these slots on a FC card, which can go into 0 or 9, instead.  If at all possible, slots should be populated in this order: 9, 0, 7, 2, 6, 3, 5, 4

So I had the opportunity recently to actually use up all ten slots in a clustered 7420 system. This actually uses 20 slots, or 22 if you count the clusteron card. I thought it was interesting enough...

Good papers and links for the ZFSSA

So I have a pretty good collection of links and papers for the ZFSSA, and instead of giving them out one-at-a-time when asked, I thought it may be easier to do it this way. Many of the links from my old blog last May no longer work, so here is an updated list of some good spots to check out. How to Manage the ZFS Storage Appliance with JavaScript Remote Replication Using Oracle ZFS Storage Appliance Systems Database Cloning using Oracle Sun ZFS Storage Appliance and Oracle Data Guard Using Sun ZFS Storage Appliance iSCSI LUNs in an Oracle Linux Environment Using Sun ZFS Storage Appliance iSCSI LUNs in a Microsoft Windows Server 2008 R2 Environment How to Make Fibre Channel Storage Available to Oracle Solaris With the Sun ZFS Storage Appliance ZFS Storage Appliance Software Update Wiki  Oracle ZFSSA Software Plug-ins (for OEM, VSS, VMWare, etc)  ZFSSA 7420 Power calculator  Oracle Database Cloning Solution Using Oracle Recovery Manager and Sun ZFS Storage Appliance  Identity Mapping Between Active Directory and NIS (Network Information Services) Implementation Guide  ZFS Storage Appliance Oracle Enterprise Manager 10g & 11g Grid Controller Plug-in  ZFS Storage 7420 Appliance 32,000 Mailbox Resiliency Exchange 2010 Storage Solution  Using ZFS to Fight Data Rot, the Silent Killer Microsoft SharePoint Server on the Sun ZFS Storage Appliance Microsoft SQL Server 2008 Database on the Sun ZFS Storage Appliance  Implementing Fibre Channel SAN Boot with Oracle's Sun ZFS Storage Appliance How to Deploy a Sun ZFS Storage Appliance in an Oracle Virtual Desktop Infrastructure Installing and Configuring the Sun ZFS Storage Appliance Provider for Volume Shadow Copy Service Software NDMP Implementation Guide for the Sun ZFS Storage Appliance  Implementing Microsoft Exchange with the Oracle ZFS Storage 7420  Configuring the Sun ZFS Storage Appliance to Use IDMU to Map Identities Between Active Directory and NIS  ZFS Storage Appliance Deduplication Design and Implementation Guidelines The new ZFSSA Datasheet  These are for ZFS, in general, not the ZFSSA, but it gives one good insight to how ZFS functions: Oracle Solaris ZFS Storage Management  Lab: Introduction to Oracle Solaris 11 ZFS File System

So I have a pretty good collection of links and papers for the ZFSSA, and instead of giving them out one-at-a-time when asked, I thought it may be easier to do it this way. Many of the links from my...

New 7420 hardware released today

Some great new upgrades to the 7420 were announced and released today. You can now get 10-core CPUs in your 7420, allowing you to have 40 cores in each controller. Even better, you can now also go to a huge 1TB of DRAM for your L1ARC in each controller, using the new 16GB DRAM modules.So your new choices for the new 7420 hardware are 4 x 8-core or 4 x 10-core models. Oracle is no longer going to sell the 2 x CPU models, and they are also going to stop selling the 6-core CPUs, both as of May 31st. Also, you can now order 8GB or 16GB modules, meaning that the minimum amount of memory is now 128GB, and can go to 1TB in each controller. No more 64GB, as the 4GB module has also been phased out (starting today, actually).Now before you get upset that you can no longer get the 2-CPU model, be aware that there was also a price drop, so that the 4 x 8-core CPU model is a tad LESS then the old 2 x 8-core CPU model. So stop complaining.It's the DRAM that I'm most excited about. I don't have a single ZFSSA client that I know of that has a CPU bottleneck. So the extra cores are great, but not amazing. What I really like is that my L1ARC can now be a whole 1TB. That's crazy, and will be able to drive some fantastic workloads. I can now place your whole, say 800GB, database entirely in DRAM cache, and not even have to go to the L2ARC on SSDs in order to hit 99% of your reads. That's sweet. 

Some great new upgrades to the 7420 were announced and released today. You can now get 10-core CPUs in your 7420, allowing you to have 40 cores in each controller. Even better, you can now also go to...

New ZFSSA code release today

The first minor release of the 2011.1.1 major release for the ZFSSA came out yesterday. You can get the code via MOS, under the "Patches and updates" tab. Just click the "Product or Family (advanced)" link, and then type "ZFS" in the search window and it really takes you right to it. Or search on it's patch ID, which is 13772123 Along with some other fixes, the most important piece of this update is the RPC flow control fix, which will greatly help those using the ZFSSA to backup an Exadata over Infiniband.  If you're not already on the major release of 2011.1.1, I urge you to update to it as soon as you can. You can jump right to this new 2011.1.1.1 code, as long as you are already on 2010.Q3.2.1 or higher. You don't need to go to 2011.1.1 first, just jump to 2011.1.1.1. If you are using your ZFSSA to backup an Exadata, I urge you to get on 2011.1.1.1 ASAP, even if it means staying late and scheduling special time to do it. It's also important to note that if you have a much older ZFSSA (one of the 7x10 models that are using the older SAS1 HBAs, and not the SAS2 HBAs), that you do NOT upgrade to 2011.1 code. The latest code that supports your SAS1 systems is 2010Q3.4.2. **Update 2-26-12:  I noted a few folks saying the link was down, however that may have been a burp in the system, as I just went into MOS and was able to get 2011.1.1.1 just fine. So delete your cookies and try again. - Steve

The first minor release of the 2011.1.1 major release for the ZFSSA came out yesterday. You can get the code via MOS, under the "Patches and updates" tab. Justclick the "Product or Family (advanced)"...