Thursday Nov 12, 2015

Oracle Data Integrator 12.2.1 - Parallel Development using Subversion Branches

This is the second article in the series of three articles covering Oracle Data Integrator (ODI) 12.2.1 lifecycle management features. In the previous article, Managing Versions in Apache Subversion, I discussed ODI Studio features for creating, viewing, comparing and restoring versions in Apache Subversion. In this article I will cover the Branch and Tag management capabilities that are required for parallel development.

Parallel Development Using Subversion Branches

First, let’s take a look at the high level setup of the Subversion branches in ODI for any parallel development. As shown in the below diagram, you can configure a Subversion branch/trunk with one master and work repository combination. In this example, the Subversion trunk is used by user set 1 as the main code line and contains all ODI objects. Branch 01 is configured for user set 2, who are working on a subset of ODI objects. User set 3 is working on a release branch and fixing bugs on the release code line. Each of these user set work in parallel and leverages Subversion branches as the channel to merge their changes across different branches.


Creating Tags

A Tag is a text name applied to the current versions of a set of objects to take a snapshot of their current state. It freezes the current state of objects and can be referred back with the Tag name at some future point in time. You can create a Tag directly from the ODI Studio and you can create it for all the repository objects or for a selected set of objects.

ODI takes following actions on the objects present in the Tag to ensure consistency.

  1. Add objects to Subversion if they are not yet versioned.
  2. Create versions for the objects if their latest changes are not yet versioned.
  3. Automatically include all the dependencies of the selected objects so that all the relevant objects can be found in the Tag when you refer back to it.

Thus, a Tag creation is a good way of ensuring all the relevant objects are being version controlled and it is recommended to create Tags at different logical points during your development cycle.

There are two type of Tags created in ODI

  • Full Tag – It is created with all the objects in the repository. It will first sync all the repository object state to Subversion versions before applying the Tag.

  • Partial Tag – It is created for a selected set of objects and user drags and drops an object into the Partial Tag creation wizard. Al the dependencies for the selected object are automatically added to the Tag.

In the example, a mapping was added to the Partial Tag and all the required model and topology objects are automatically added by ODI Studio.

Creating a Subversion Branch

You can create a Subversion branch from a Tag. The branch creation screen lists all the relevant Subversion Tags and their respective comments to easily locate a Tag for branch creation. The objects versions of the selected Tag becomes the branch point and may not necessarily be the latest version of the objects.


Populating ODI Repository from a Branch or Trunk Objects

Once you configure your ODI repository with a Subversion branch/trunk, you can populate it with the branch/trunks objects through a couple of ODI Studio menu options.


  1. Populating repository – It is used when you want to populate a fresh repository with the branch content. For example when you create a project branch and want to setup an ODI repository to the branch.
  2. Populating restored repository – This option is used when your current repository gets corrupted and you restore an older repository backup. It syncs the objects present in the restored repository with the latest contents from the Subversion branch/trunk. It will pull all the objects from the Subversion branch/trunk and imports them in a special Import Mode that only inserts or updates the objects presented in the repository while leaves behind any extra objects in the repository.

Branch Merge

Merging changes from one development branch to another is an essential need for any parallel development. ODI Studio allows you to merge changes from a Branch, Trunk or Tag to the repository it is connected.


Merge Summary

The merge process automatically merges the object changes that do not have any conflicts. The Merge summary details all the affected objects in the merge. It also lists the objects that could not be automatically merged and their conflicts should me manually resolved. The summary report can be saved in the file system in multiple formats so that it can be shared with the team.


Merge Results

The merge result window assists you in resolving conflicts from a branch merge. There are a couple of tabs available in this window.

Merge Object Selection Tab


It lists the objects affected in the merge and provides different search and filtering options. It allows following operations on the conflicted objects

  1. Mark the conflict as resolved for the selected object.
  2. Open Merge Conflict Resolution window for the selected object.
  3. Assign ownership of the conflict resolution to a particular user responsible for conflict resolution of that object. By default the merge process assigns conflict ownership to the user who has last modified it.

Merge Conflict Resolution Tab


It assists you in manually resolving an object’s conflict by presenting the object properties side by side.

Conclusion

A number of branch management features introduced in ODI 12.2.1 are vital for organizing objects across branches and allow parallel development between functionally and geographically distributed teams.

Stay tuned for the third and final article on lifecycle management, which will cover the release management capabilities using newly introduced Deployment Archives.

Friday Nov 06, 2015

Oracle Data Integrator 12.2.1 - Managing Versions in Apache Subversion

The latest Oracle Data Integrator (ODI) release, 12.2.1, came out with brand new lifecycle management features that allows you to use Apache Subversion as the external Version Control System (VCS) for creating and managing ODI objects versions. It does not just stop there but also facilitates the creation of tags, manage branches for parallel development, and create Deployment Archives for promoting ODI objects from Development to Production.

In this article I will talk about various version management operations available in ODI Studio for maintaining, viewing, comparing and restoring versions from Apache Subversion.

How does it work?

ODI Studio remains the sole User Interface for performing any such versioning operations. It directly communicates with the external Version Control System and presents all the needed information within ODI Studio user interface. For example to create a version, an object is exported into XML file which is then checked in into Version Control System. Conversely to restore an object, its XML file is first checked out from Version Control System and then imported back into ODI repository.


Configuring Apache Subversion

In order to use Subversion, we first need to setup its connectivity in ODI Studio in just a couple of steps.


  • Edit Connection - This screen allows to setup subversion URL and user credentials as per the selected authentication types. There are a number of authentication types supported for setting up such connectivity – HTTP, SSH, SSL etc.

  • Configure - This screen allows you select the subversion project and the Trunk or Branch under it for maintaining ODI objects. If you don’t have the subversion project or branch already created then you can create them right from this screen at the time of configuration.

There are a couple of other useful options configured here

  1. Auto version – Set this if you want to make ODI to automatically create version of a versioned object whenever it is saved. By enabling you can make sure that the subversion copy is always in sync with your repository copy of a versioned object. But it may impact your save performance due to underlying export and checkin.
  2. VCS Key – Configure the VCS key that is used to encrypt the sensitive content while exporting the object in to XML file. For example the passwords in a Data Server definition should not be exported in plain text when the Data Server is exported into XML. Such passwords are encrypted and decrypted using the VCS Key configured here.

Managing Versions

The operations for managing versions are available at two levels.

  • At Studio menu level - This menu lists the subversion operations that are not specific to a particular object and can be performed on a group of objects.

  • In the object context menu (Right click menu) - These options allow you to perform various subversion operations on that particular object.

Adding objects to Apache Subversion

Once you decide what objects need to be version controlled, then you can add them to subversion individually or in group.

Adding Single object

 It is performed through object's context menu


There are couple of convenience features available here

  1. All the necessary parent hierarchy is automatically added with child object so that developer need not do it individually. For example when you add a mapping for the first time the corresponding folder and project that contains this mapping are also added automatically.
  2. You can add selected or all children objects along with the parent object. For example when you add a folder to subversion then the packages, procedures, mappings etc present under the folder can also be added along with it in the same operation.

Adding group of objects

The “Add Non-versioned Objects to VCS” option in ODI Studio and object context menu presents you the list of objects that are not yet added to subversion and allows you to add them to subversion in group. At studio menu level it list all such objects in the entire repository whereas at object context menu level it lists all the non-versioned objects under that object and is applicable for only the container objects such as projects, folder etc.


Indicator for Version Controlled Objects

A version controlled object can be easily identified by the subversion indicator presence next to its icon. Notice the indicator next to “Version Control” folder in below screenshot indicating that folder is version controlled whereas the other folder is not yet added to subversion.


Creating Object Versions

New versions of a version controlled object can be created through the context menu option – "Create VCS Version"


When a version is created at the container object level – such as project, folder, model etc – then following operations are also performed along with creating version of the the container object.

  • Versions are created for all version controlled children that are modified since last version. Note: It does not affect any non-versioned child.
  • Any child object deletion, rename or move is also pushed to subversion.

Viewing Version History

You can view version history for an object within the ODI Studio itself. It pulls object's version history from subversion and presents it in a couple of formats. Each of the formats provides different filtering/search options to easily locate a particular version.

  • Tabular format – through context menu option “Version History”
  • Tree format – through context menu option “Version Tree”

Comparing Versions

You can compare a version in object’s version history with another version or with the current copy present in repository. The difference between the two is presented side by side in a special window. There are toolbar buttons provided for easy navigation through the changes. The changes are color coded to easily differentiate the added, deleted or modified properties.