X

An Oracle blog about ZFS Storage

  • April 25, 2017

DeDupe 2.0

Steve Tunstall
Principle Storage Engineer

In the new OS 8.7, I mentioned that Oracle has greatly improved the DeDupe feature on the ZFSSA (ZFS Storage Appliance).

Here is a small excerpt from our latest documentation regarding using our DeDupe  with various backup types on a ZFSSA. You can find this whitepaper here: https://community.oracle.com/docs/DOC-1012600
*****************************************

The ZFS implementation is inline, block-level deduplication, which means that redundant data can be reduced

when it is written to disk. The way it works is that a hash is calculated for each block written to storage, which

is then compared to entries in a deduplication table (DDT) of previously written blocks. When an incoming
block matches, it is not written to storage. Instead, a reference pointer to the existing block is used, resulting
in reduced storage space for large amounts of backup data.
Deduplication 2.0 is targeted specifically for backup use cases. Full or level 0 backups yield the highest rate
of deduplicated data.
Benefits
The new design provides that the DDT table is available in memory, and if the size of the table increases, it is
available on DDT solid-state drives (SSDs), leveraging the existing Hybrid Storage Pool model to scale
performance. Hybrid Storage Pool is a feature of Oracle ZFS Storage Appliance

//cdn.app.compendium.com/uploads/user/e7c690e8-6ff9-102a-ac6d-e4aebca50425/f4a5b21d-66fa-4885-92bf-c4e81c06d916/Image/4193a92c3708e871c6c323284e77e3c0/dedupe1.jpg

Overall deduplication 2.0 benefits include the following:
Provides excellent data reduction yields for backup use cases
Leverages powerful Hybrid Storage Pool for scalable deduplication performance
No additional licensing costs for deduplication or compression
Data Reduction Feature Summary
Review the following use cases and recommended data reduction features for Oracle ZFS Storage Appliance:

//cdn.app.compendium.com/uploads/user/e7c690e8-6ff9-102a-ac6d-e4aebca50425/f4a5b21d-66fa-4885-92bf-c4e81c06d916/Image/ec61fe886f9c3f9d0c81a97b135c2d3e/dedupe2.jpg

Oracle ZFS Storage Appliance Requirements
Oracle ZFS Storage Appliance deduplication 2.0 is supported with the following hardware and software
requirements:
OS8.7 software is required but no separate deduplication license is required.
The following platforms are supported and include recommended memory requirements:
o Oracle ZFS Storage Appliance ZFS4-4 with at least 1 TB of memory per controller
o Oracle ZFS Storage Appliance ZS5-2 with at least 384 GB of memory per controller
o Oracle ZFS Storage Appliance ZS5-4 with at least 1.50 TB of memory per controller
3.2 TB SSDs for meta devices (DDT SSDs) and two for every four trays are recommended
Oracle Storage Drive Enclosure DE3-24C storage trays for meta devices (DDT SSDs) are required.


//cdn.app.compendium.com/uploads/user/e7c690e8-6ff9-102a-ac6d-e4aebca50425/f4a5b21d-66fa-4885-92bf-c4e81c06d916/Image/79603134ea3b14a9eded868ef86244fb/dedupe5.jpg

****************************************

As you can see in the above screenshot, there is now a 3rd column for SSDs. This one is called "META" and is for the drives you dedicate for the DeDupe table. You must have 2 in order for DeDupe to even be able to activate, and you need another 2 for every 4 disk trays in the pool you wish to use DeDupe on. 

Note: You do NOT need any dedicated META drives when you're using an all-flash pool. It will just use the all-flash pool for the DDT. 

Be the first to comment

Comments ( 0 )
Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.