Oracle GoldenGate Veridata Automatic Row Partitioning feature

July 28, 2020 | 4 minute read
Nisharahmed Soneji
Distinguished Data Management Specialist
Text Size 100%:

I am happy to announce that we have released the latest Oracle GoldenGate Veridata Bundle patch(12.2.1.4.200714) in this week. The GoldenGate Veridata Bundle patch contains the most awaited feature on comparing huge data in a table automatically. We call it Automatic Row partitioning. I would recommend you to read my latest blog on how to capture the data from Amazon Aurora PostgreSQL using GoldenGate PostgreSQL. The GoldenGate PostgreSQL capture allows you to capture the data from Amazon Aurora PostgreSQL & Amazon RDS PostgreSQL Database in real-time and helps you deliver the data in Autonomous Database Services.

The GoldenGate Veridata has row partitioning feature in the product since quite sometime. We renamed the existing row partitioning feature to “Manual Row Partitioning”, which helps users to create multiple row partitions on a given table and then user can select any one partition in the comparison pair. Although it served a purpose for user who has huge data in a table, provided a flexibility to create partitions manually, and choose the partition while comparing the data. Since data is growing day by day (in Tera bytes to Peta bytes, and so on), the demand for partitioning table data increases and user needs more of automation in the partitioning process. User wants to create partitions automatically and creates multiple compare pairs for those partitions. We learn the requirements and provided a great solution for users who wants to compare the huge table data in parallel.

In this feature, the existing row partitioning functionality, which is renamed as Manual partitioning is intact, and we have not changed it. The new “Auto row partitioning” functionality allows user to create row partition for a selected table and creates the compare pairs for each partition automatically. It has configurable parameters like “Automatic Row partition” the number value ranging 2 to 100. The number of automatic row partitions would be created based on the number.

We know that Database users creates their own partitions in the Database tables. Until now, there was no mechanism to utilize these partitions into Veridata product for comparison. With Auto partitioning feature, we have also allowed user to fetch the existing Database table partitions into Veridata. When user selects the table and if table has existing partitions, it will be shown on the screen. The user needs to map existing partitions for comparison. The user can select multiple partitions if needed.

Image: Existing Table Partition selection

Image: Existing Table Partition selection

The existing database table partitions feature can be used with Manual row partition and Automatic Row partition. You can select Table.Partition and create your own manual partition as you used to do earlier. 

Image: Manual Raw Partition

Image: Manual Row Partition

Automatic Row Partitioning - example 1, the user selects the table with having 4 existing partitions, and selects the Automatic row partition value as 2, the Veridata will create 8 compare pairs automatically. The two partitions will be generated for each existing table partitions.

Image: Example 1 Auto Partitioning

Automatic Row Partitioning - example 2, the user selects the table having no existing partitions, and selects the automatic row portions value as 10, the Veridata will create 10 compare pairs for each newly created partitions.

Image: Example 2 Auto Partitioning

You can use the both the features through Veridata command line utilities like Veridata Import and Export, Vericom. Please see example commands below.

Image: Command line utilities

The automatic row partitioning and Existing Database table partition fetching features are for Oracle database only. I am sure, you would love the feature and use it most in the Veridata product. You may find more details about the feature in the Veridata documentation.

Nisharahmed Soneji

Distinguished Data Management Specialist

Nishar is the "Distinguished Data Management Specialist", leading the sales opportunity for Data Integration products, OCI GoldenGate & GoldenGate On-premises in the EMEA region. The GoldenGate is the best database replication technology available in the market. He has been advocating for the customers on new-age technology and best practices and helping them achieve strategic goals using Oracle solutions and Products. He has been evangelizing GoldenGate Products, Solutions, and new-age architects(Data Mesh, Data Fabric) in Conferences, enabling and coaching the Tech hub engineers on the Products' functionality & insights. The family of GoldenGate products includes OCI GoldenGate, GoldenGate for Oracle, GoldenGate for Non-Oracle, GoldenGate for Big Data, GoldenGate Stream Analytics, GoldenGate Foundation Suite(Veridata, GoldenGate OEM PlugIn), and OCI Data Integration.

Before joining the sales team, he was a Senior Principal Product Manager at Oracle. He led the OCI GoldenGate Cloud Service, GoldenGate Foundation Suite Products, and GoldenGate core products globally.

Nishar has been a key leadership team member who created new cloud services from inception to growing Oracle Data Integration business services. He has also spearheaded multiple Oracle internal innovations, resulting in top-line and bottom-line growth in Database Replication.

Nishar has over 22+ years of experience in technology, specializing in enterprise data replication and integration products, with a demonstrated track record of creating and scaling world-class businesses. For the last 10+ years, He has been focused on driving revenue through sales and product management, making the business strategy, executing the product execution, and leading the cross-functional teams and products.

Before joining the Sales and Product Management role, he was the software development architect; he led various complex technology projects for several companies, including Endress+Hauser, Cummins Inc, and GoldenGate Inc.

Show more

Previous Post

Data Loader Task in Oracle Cloud Infrastructure (OCI) Data Integration

Aditya Duvuri | 6 min read

Next Post


Applications in Oracle Cloud Infrastructure (OCI) Data Integration

Aditya Duvuri | 4 min read
Oracle Chatbot
Disconnected