X

Welcome to All Things Data Integration: Announcements, Insights, Best Practices, Tips & Tricks, and Trend Related...

Oracle GoldenGate Veridata Automatic Row Partitioning feature

Nisharahmed Soneji
Senior Principal Product Manager, Oracle GoldenGate

I am happy to announce that we have released the latest Oracle GoldenGate Veridata Bundle patch(12.2.1.4.200714) in this week. The GoldenGate Veridata Bundle patch contains the most awaited feature on comparing huge data in a table automatically. We call it Automatic Row partitioning. I would recommend you to read my latest blog on how to capture the data from Amazon Aurora PostgreSQL using GoldenGate PostgreSQL. The GoldenGate PostgreSQL capture allows you to capture the data from Amazon Aurora PostgreSQL & Amazon RDS PostgreSQL Database in real-time and helps you deliver the data in Autonomous Database Services.

The GoldenGate Veridata has row partitioning feature in the product since quite sometime. We renamed the existing row partitioning feature to “Manual Row Partitioning”, which helps users to create multiple row partitions on a given table and then user can select any one partition in the comparison pair. Although it served a purpose for user who has huge data in a table, provided a flexibility to create partitions manually, and choose the partition while comparing the data. Since data is growing day by day (in Tera bytes to Peta bytes, and so on), the demand for partitioning table data increases and user needs more of automation in the partitioning process. User wants to create partitions automatically and creates multiple compare pairs for those partitions. We learn the requirements and provided a great solution for users who wants to compare the huge table data in parallel.

In this feature, the existing row partitioning functionality, which is renamed as Manual partitioning is intact, and we have not changed it. The new “Auto row partitioning” functionality allows user to create row partition for a selected table and creates the compare pairs for each partition automatically. It has configurable parameters like “Automatic Row partition” the number value ranging 2 to 100. The number of automatic row partitions would be created based on the number.

We know that Database users creates their own partitions in the Database tables. Until now, there was no mechanism to utilize these partitions into Veridata product for comparison. With Auto partitioning feature, we have also allowed user to fetch the existing Database table partitions into Veridata. When user selects the table and if table has existing partitions, it will be shown on the screen. The user needs to map existing partitions for comparison. The user can select multiple partitions if needed.

Image: Existing Table Partition selection

Image: Existing Table Partition selection

The existing database table partitions feature can be used with Manual row partition and Automatic Row partition. You can select Table.Partition and create your own manual partition as you used to do earlier. 

Image: Manual Raw Partition

Image: Manual Row Partition

Automatic Row Partitioning - example 1, the user selects the table with having 4 existing partitions, and selects the Automatic row partition value as 2, the Veridata will create 8 compare pairs automatically. The two partitions will be generated for each existing table partitions.

Image: Example 1 Auto Partitioning

Automatic Row Partitioning - example 2, the user selects the table having no existing partitions, and selects the automatic row portions value as 10, the Veridata will create 10 compare pairs for each newly created partitions.

Image: Example 2 Auto Partitioning

You can use the both the features through Veridata command line utilities like Veridata Import and Export, Vericom. Please see example commands below.

Image: Command line utilities

The automatic row partitioning and Existing Database table partition fetching features are for Oracle database only. I am sure, you would love the feature and use it most in the Veridata product. You may find more details about the feature in the Veridata documentation.

Be the first to comment

Comments ( 0 )
Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.