In series of blogs around Oracle GoldenGate Foundation Suite (OGFS) products, I mentioned the new features for OGG Monitor in my previous blog. Today, I am pleased to announce that we have released the latest OGG Veridata Bundle patch(220.127.116.11.171215) end of December. In this Veridata Bundle patch, we have provided support for Hive (Big data), and automatic key column mapping.
Let me first write about automatic key column mapping.
In the absence of unique identifiers, such as the primary keys and unique indexes, which can uniquely identify each row, you need to manually define any column or columns to be used as a unique identifier. If you have hundreds of such tables, then it is an excessive work to select PKey for each table in the user interface (UI). However, there was a workaround to use the Veridata GoldenGate Parameter Processing (VGPP) and create the compare pair. It was still requires some work to be done from your side. Now with this new bundle patch, Veridata will select key columns automatically for you based on specific inputs provided while creating compare pairs or connections. If you have enabled automatic key column mapping at the Connection Level, then any group using this Connection has the feature enabled. You can override automatic key column mapping at the Compare Pair Generation stage.
The connection level has mainly two new options.
Use Source or Target Columns as Key Columns When Generating Compare Pairs: If the source table has primary keys or unique index defined, but the target does not, then Oracle GoldenGate Veridata uses the same columns (as the source) as unique identifiers for the target. Similarly, if the source does not have the primary key or index, but the target has them, then key columns of target are used for source. If either (source or target) of the columns of the Primary Key or index is not present on the target side, then the primary key or index is not considered.
Use All Columns as Key Columns When Generating Compare Pairs: Enables Automatic mapping for the source and target connections. If this option is disabled either at the source or the target connection, then the automatic mapping is also disabled for that group. If you enable this option to map all columns from source and target, then the mapping is considered only when both source and target table do not have primary or unique keys.
If you do not want to specify "automatic key selection" options at the connection level, you can do so while configuring compare pair as well. The Manual and Pattern compare pair configuration has following new options:-
The other important feature was released in the bundle patch is supporting Big Data Hive. We know that you must be having your data replicated in big data targets. However, how will you make sure that the replicated data is correct? Veridata has always provided validation support for Databases. Now we have started and provided support for our first Big data target Hive. You can compare data between your source relational databases (Oracle, MSSQL, DB2 LUW, Sybase, Informix, etc.) and Big Data hive (2.1.1+). Even you can do Hive to Hive comparison too.
The Hive comparison need not require any different options to be selected; you can make the Hive comparison just like you do Oracle to Oracle comparison. You would need to create the Hive connection (for your source or target) and create groups which will contain compare pairs from the source database and Hive database. Once all the compare pairs and options for primary key selection is selected, you can run the job. The Hive feature supports Delta comparison and raw partition. However, it does not support Hive authentication and Repair feature as of now.