Monday Dec 14, 2015

Oracle Data Integrator Webcast Recap: Special Topic - Cloud Application Adapter

If you missed our Oracle Data Integrator Webcast two weeks ago about the simplicity of leveraging Oracle Data Integrator (ODI) with Cloud technologies – take a look here: Oracle Data Integrator Special Topic:  Salesforce.com & Universal Cloud.


With the explosion of the number of Cloud Applications today, the topic is top of mind for many organizations. ODI coupled with Progress DataDirect’s JDBC drivers can be the perfect fit – find out by viewing the webcast! Thanks again to our guest speakers: Sumit Sarkar, Chief Data Evangelist at Progress Software and Christophe Dupupet, Director – A-Team, Cloud Solutions Architects at Oracle.

For additional posts about ODI and the Cloud, browse the A-Team Chronicles – one of the posts you will find: A Universal Cloud Applications Adapter for ODI.

For other ODI webcasts, please view the Oracle Data Integrator Demo Webcast Archives. Our next one will cover ODI’s 12.2.1 New Features and is scheduled for January 14, 2016 at 1pm ET/10am PT. Join us!

Friday Dec 11, 2015

Oracle Data Integrator 12.2.1 - Release Management with Deployment Archives

In earlier articles on ODI Lifecycle Management features we covered Managing Versions in Apache Subversion and Parallel Development using Subversion Branches. Now let’s explore the Release Management aspects addressed in ODI 12.2.1.

Deployment Archive

The release management use cases are addressed in ODI 12.2.1 through the newly introduced object called Deployment Archive. It is an archive file containing a set of ODI objects along an XML file containing archive details. It is the vehicle to promote objects from a development environment to the testing or production environment as depicted in below diagram. In a typical release process a Deployment Archive is first deployed in testing environment where the validations can be performed and after testing is done it is applied to a production environment.


Deployment Archive Types

There are four types of Deployment Archives

  1. Initial Deployment Archive – It is used to initialize a fresh ODI repository. It is created with all the objects in your development project.
  2. Patch Deployment Archive – As the name suggest it is used to apply patch containing a bug fix or enhancement to an ODI repository that is already initialized with Initial Deployment Archive. It usually contains only the object having the bug fix or enhancement and all its dependencies.
  3. Rollback Deployment Archive – A Rollback Archive facilitates restoring a pre-patch state of deployment environment. It is created when a Patch Deployment Archive is applied to an ODI Repository and contains the artifacts from the repository that are being updated by the patch.
  4. Execution Deployment Archive – It holds only the runtime executable objects and gets created along with Initial or Patch Deployment Archive. It is used to promote artifacts to ODI execution repository, which holds only executable objects.

Creating Deployment Archive

All Deployment archive related operations can be invoked from the ODI Studio menu. There are three options available for creating a Deployment Archive, which differs in the way the objects to be included in the Archive are selected.


 Each of these options allows you to
  • Select the type of Deployment Archive to create (if applicable)
  • Provide Archive Name
  • Specify the file location where the archive is created
  • Specify an Export key which is used to encrypt sensitive data in the exported object XML
  • Provide an option to exclude physical topology containing connectivity details.

Create from a VCS Label (Tag)

The objects from a Subversion Tag are included in the Deployment Archive.

Create from Repository > Full Repository

All the objects from the entire repository are included in the Deployment Archive. Since all the objects are included here, with this option you can create only the Initial Deployment Archive.


Create from Repository >Selected Objects

You can selectively add the objects you wish to include in the Deployment Archive. All dependencies of an object is also automatically included along with the object so that the Deployment Archive always contains a consistent set of objects that can be applied in any environment.

Viewing Deployment Archives

You can list and view the details of Deployment Archives from ODI Studio by specifying the folder location where all the Deployment Archives are created.


You can double click on any entry to the list of objects present under that archive.


Applying an Deployment Archive

An empty repository can be initialized with the Initial Deployment Archive.


If the repository is already initialized then you can apply a Patch deployment Archive to it.


While applying patch archive you can also specify the file location where the corresponding Rollback Deployment Archive should be created. A Rollback Archive is used to restore the pre-patch state of the repository when the Patch application result in some problems. 

As shown below, a typical Production environment is first initialized with the released artifacts from a Initial Deployment Archive and then at some later point in time the Patch Deployment Archive is applied for some bug fix or enhancement.


Auditing Deployment Archives

ODI maintains the history of all the Deployment Archives applied to a repository and it can be easily view in ODI Studio.


The Inventory table for the applied patches has a number of columns as shown above providing important information.
  • It provides details about the Deployment Archive- its type, how it was created, who created it and when it was created.
  • It shows in what order a Deployment Archive was applied, by whom and when.
  • It provides the statistics - how many object got added, updated or deleted as part of it.
  • It shows if the Deployment Archive can be rolled back and the path where the corresponding Rollback Deployment Archive was created.

Rolling Back a Patch Deployment Archive

The patches for which a Rollback Deployment Archive was created can be rolled back from studio.


Note that not all Deployment Archives can be rolled back. In order to be rolled back it should satisfy following conditions.
  1. It should be a Patch Deployment Archive
  2. The corresponding Rollback Archive should have been created during patch application
  3. The corresponding Rollback Archive should be present in the specific file location
  4. The patches can be rolled back in the sequence they are applied. It means that the patches can be rolled back in Last In First Out (LIFO) order only. 

Programming Interface for Automation

There is Software Development Kit (SDK) Application Programming Interface (API) available for all the Release management activities described in this article. The deployment archive management related classes are available in oracle.odi.core.service.deployment package. Using these APIs you can write programs to automate your release process.

Conclusion

ODI 12.2.1 has come out with a number of features addressing different Lifecycle Management needs. It provides seamless integration with Apache Subversion to manage ODI objects versions in external version control system. It provides Branch and Tag management capabilities to facilitate parallel development between functionally or geographically distributed teams. It provides comprehensive release management capabilities to create Deployment Archives with artifacts from development environment which can then be tested in isolation, applied to production environment and can be rolled back if required. With all these capabilities users can easily manage an objects lifecycle in ODI 12.2.1, which is now generally available to download from OTN page.

Monday Dec 07, 2015

Big Data Preparation Cloud Service

Preparing Complex Data For Use

Even in the most well designed big data projects, a larger portion of the effort and cost is spent in preparing complex data ready for analysis. Big Data Preparation Cloud Service (BDPCS) flips that equation. Minimizing the data preparation time helps unlock the value of complex data quickly. However, data preparation is not just a series of sorting, prepping and standardization operations. When done right, it is a crucial step to enrich data. To unearth insights and prepare a data enrichment pipeline that can then be operationalized to deliver higher quality and vastly insightful data to downstream applications.

Oracle Big Data Preparation Cloud Service is part of the comprehensive Cloud Platform for Big Data that Oracle announced during Open World.

Built for Analysts, Data Scientists and Data Experts

BDPCS utilizes a non technical, web based interface to guide the data preparation effort. This is keeping in view that data preparation should not be dependent on technical skills but business expertise. This brings data science closer to the business experts, data analysts and data scientists. Natural Language Processing based algorithms surfaces recommendations that can be then applied as business and enrichment rules to the data set. These recommendations are presented as "plain english" texts that can then be modified and updated.

Build on Apache Spark, Big Data Preparation Cloud Service also includes a responsive visual element that profiles the data sets for even further analysis and standardizations. Using Big Data Preparation Cloud Service, experts can now pull data, mash them up, enrich the data and publish it quickly. This allows them to iterate through many analysis that generally take weeks or sometime months to co ordinate with the IT and technology teams.

Differentiators

Data preparation and data wrangling is attracting a lot of investment as Data Science enters the next stage of refinement. Big Data Preparation Cloud Service brings certain critical elements that puts it ahead of its competition.

It brings recommendation driven user experience

It utilizes Natural Language Processing classifications and enrichment, and

It has extensive automation capabilities.

For more details about Oracle Big Data Preparation Cloud Services visit our homepage here.