Thursday Jan 29, 2015

Oracle GoldenGate 12c for Oracle Database - Integrated Capture sharing capture session

Oracle GoldenGate for Oracle Database has introduced several features in Release 12.1.2.1.0. In this blog post I would like to explain one of the interesting features:  “Integrated Capture (a.k.a. Integrated Extract) sharing capture session”. This feature allows making the creation of additional Integrated Extracts faster by leveraging existing LogMiner dictionaries. As Integrated Extract requires registering the Extract let’s first see what is ‘Registering the Extract’?

REGISTER EXTRACT EAMER DATABASE

The above command registers the Extract process with the database for what is called “Integrated Capture Mode”. In this mode the Extract interacts directly with the database LogMining server to receive data changes in the form of logical change records (LCRs).

When you create Integrated Extract prior to release Oracle GoldenGate 12.1.2.1.0, you might have seen the delay in registering the Extract with database. It is mainly because the creation of Integrated Extract involves dumping the dictionary and then processing that dictionary to populate LogMiner tables for each session, which causes overhead to online systems and hence it requires extra time to startup. The same process is being followed when you create additional Integrated Extract.

What if you could use the existing LogMiner dictionaries to create the additional Integrated Extract? This is what it has been done in this release. Additional Integrated Extract creation can be made faster significantly by leveraging existing LogMiner dictionaries which have been mined already. Hence no more separate copy of the LogMiner dictionaries to be dumped with each Integrated Extract. As a result, it will make the creation of additional Integrated Extracts much faster and helps avoid significant database overhead caused by dumping and processing the dictionary.

In order to use the feature, you should have Oracle DB version 12.1.0.2 or higher, and Oracle GoldenGate for Oracle version 12.1.2.1.0 or higher. The feature is currently supported for non-CDB databases only.

Command Syntax:

REGISTER EXTRACT group_mame DATABASE

..

{SHARE [AUTOMATIC | extract | NONE]}

It has primarily three options to select with; NONE is default if you don’t specify anything.

AUTOMATIC option will clone/share the LogMiner dictionary from the existing closest capture. If no suitable clone candidate is found, then a new LogMiner dictionary is created.

Extract option will clone/share from the capture session associated for the specified Extract. If this is not possible, then an error occurs the register does not complete.

NONE option does not clone or create a new LogMiner dictionary; this is the default.

While you use the feature, the SHARE options should be followed by SCN and specified SCN must be greater than or equal to at least one of the first SCN of existing captures and specified SCN should be less than current SCN.

Let’s see few behaviors prior to 12.1.2.1.0 release and with SHARE options. 'Current SCN’ indicates the current SCN value when register Extract command was executed in following example scenario.

Capture Name

LogMiner ID

First SCN

Start SCN

LogMiner Dictionary ID (LM-DID)

EXT1

1

60000

60000

1

EXT2

2

65000

65000

2

EXT3

3

60000

60000

3

EXT4

4

65000

66000

2

EXT5

5

60000

68000

1

EXT6

6

70000

70000

4

EXT7

7

60000

61000

1

EXT8

8

65000

68000

2

Behavior Prior to 12.1.2.1.0 – No Sharing

Register extract EXT1 with database (current SCN: 60000)

Register extract EXT2 with database (current SCN: 65000)

Register extract EXT3 with database SCN 60000 (current SCN: 65555)

Register extract EXT4 with database SCN 61000   à Error!!

Registration of Integrated Extract EXT1, EXT2 and EXT3 happened successfully where as EXT4 fails because the LogMiner server does not exist at SCN 61000.

Also take a note that all Integrated Extract (EXT1 – EXT3) created dictionaries separately (LogMiner Dictionary IDs are different, now onwards I’ll call them LM-DID).

New behavior with different SHARE options

  • Register extract EXT4 with database SHARE AUTOMATIC (current SCN: 66000)

EXT4 automatically chose the capture session EXT2 as it has Start SCN 65000 which is nearer to current SCN 66000. Hence EXT4 & EXT3 capture sessions would share the same LM-DID 2.

  • Register extract EXT5 with database SHARE EXT1 (current SCN: 68000)

EXT5 is sharing the capture session EXT1. Since EXT1 is up and running, it doesn’t give any error. LM-DID 1 would be shared across EXT 5 and EXT1 capture sessions.

  • Register extract EXT6 with database SHARE NONE (current SCN: 70000)

EXT6 is being registered with SHARE NONE option; hence the new LogMiner dictionary will be created or dumped. Please see LM-DID column for EXT6 in above table. It contains LM-DID value 4.

  • Register extract EXT7 with database SCN 61000 SHARE NONE (current SCN: 70000)

It would generate an error as similar to EXT4 @SCN61000. The LogMiner Server doesn’t exist at SCN 61000 and since the SHARE option is NONE, it won’t share the existing LogMiner dictionaries as well. This is same behavior as prior to 12.1.2.1.0 release.

  • Register extract EXT7 with database SCN 61000 SHARE AUTOMATIC (current SCN: 72000)

EXT7 is sharing the capture session EXT1 as it is the closest for SCN61000. You must have noticed that the EXT7 @SCN61000 scenario has passed with SHARE AUTOMATIC option, which was not the case earlier (EXT4 @61000).

  • Register extract EXT8 with database SCN 68000 SHARE EXT2 (current SCN: 76000)

EXT8 extract is sharing EXT2 capture session. Hence sharing of LogMiner dictionaries happens between EXT8 & EXT2

This feature is not only providing you faster start up for additional Integrated Extract, but also resolves few scenarios which wasn’t possible earlier. If you are using this feature and had questions or comments, please let me know by leaving your comments below. I’ll reply to you as soon as possible.

Monday Dec 29, 2014

Oracle Data Enrichment Cloud Service (ODECS) - Coming Soon

What are your plans around Big Data and Cloud?

If your organization has already begun to explore these topics, you might be interested a new offering from Oracle that will dramatically simplify how you use your data in Hadoop and the Cloud:

Oracle Data Enrichment Cloud Service (ODECS)

There is a perception that most of the time spent in Big Data projects is dedicated to harvesting value. The reality is that 90% of the time in Big Data projects is really spent on data preparation. Data may be structured, but more often it will be semi-structured such as weblogs, or fully unstructured such as free form text. The content is vast, inconsistent, and incomplete, often off topic, and from multiple differing formats and sources. In this environment each new dataset takes weeks or months of effort to process, frequently requiring programmers writing custom scripts. Minimizing data preparation time is the key to unlocking the potential of Big Data.

Oracle Data Enrichment Cloud Service (ODECS) addresses this very reality. ODECS is a non-technical, web-based tool that sets out to minimize data preparation time in an effort to quickly unlock the potential of your data. The ODECS tool provides an interactive set of services that automate, streamline, and guide the process of data ingestion, preparation, enrichment, and governance without costly manual intervention.

The technology behind this service is amazing; it intuitively guides the user with a machine learning driven recommendation engine based on semantic data classification and natural language processing algorithms. But the best part is that non-technical staff can use this tool as easily as they use Excel, resulting in a significant cost advantage for data intensive projects by reducing the amount of time and resources required to ingest and prepare new datasets for downstream IT processes.

Curious to find out more? We invite you to view a short demonstration of ODECS below:


Let us know what you think!

Stay tuned as we write more about this offering…

Wednesday Dec 17, 2014

Oracle Partition Exchange Blog from the ODI A-Team

More great information from the ODI A-Team!

Check out the A-Team’s most recent blog about the Oracle Partition Exchange – it does come in two parts:

Using Oracle Partition Exchange with ODI

Configuring ODI with Oracle Partition Exchange

The knowledge module is on Java.Net, and it is called “IKM Oracle Partition Exchange Load”.  To search for it, enter “PEL” or “exchange” in the Search option of Java.Net.

A sample ODI 12.1.3 Repository is available as well.  The ODI sample repository has great examples of how to perform both initial and incremental data upload operations with Oracle Partition Exchange.  This repository will help users to understand how to use Oracle Partition Exchange with ODI.

Happy reading!

Wednesday Dec 10, 2014

Oracle Enterprise Metadata Management 12.1.3.0.1 is now available!

As a quick refresher, Metadata Management is essential to solve a wide variety of critical business and technical challenges which include how report figures are calculated, understanding the impact of changes to data upstream, providing reports in a business friendly way in the browser and providing reporting capabilities on the entire metadata of an enterprise for analysis and improvement. Oracle Enterprise Metadata Management is built to solve all these pressing needs for customers in a lightweight browser-based interface. Today, we announce the availability of Oracle Enterprise Metadata Management 12.1.3.0.1 as we continue to enhance this offering.

With Oracle Enterprise Metadata Management 12.1.3.0.1, you will find business glossary updates, updates for a better experience to the user interface as well as improved and new metadata harvesting bridges including Oracle SQL Server Data Modeler, Microsoft SQL Server Integration Services, SAP Sybase PowerDesigner, Tableau and more. There are also new dedicated web pages for tracing data lineage and impact! At a more granular level you will also find new customizable action menus per repository object type for more personalization. For a full read on new features, please read here. Additionally, view here for the certification matrix details.

Download Oracle Enterprise Metadata Management 12.1.3.0.1!

Thursday Nov 20, 2014

Let Oracle GoldenGate 12c Take You to the Cloud

If your organization is in the ~80% of the global business community, you are most likely working on a cloud computing strategy for your organization, or actively implementing. The cloud computing growth rate is 5X more than the overall IT growth rate because of the clear and already proven cost savings, agility, and  scalability benefits of cloud architectures.

When organizations decide to embark on their cloud journey, they notice there are several questions and challenges to be addressed, involving data accessibility, security, availability, system management, performance etc. Oracle GoldenGate's real-time data integration and bi-directional transactional replication technology addresses critical challenges such as:

  • How to move my systems to the cloud without interrupting operations?
  • How to enable timely data synchronization between the systems on the cloud and on-premises to ensure access to consistent data for all end users?
  • How do I run operational reports with the data I have in cloud environments, or feed my analytical systems in cloud solutions?
  • In managed or private clouds, how do I keep the cloud platform highly available when I need to do maintenance, upgrades?

 On Tuesday,  December 2nd we will tackle these questions in a free webcast:

Live Webcast: Oracle GoldenGate 12c for the Enterprise and the Cloud

Tuesday, December 2nd, 2014 10am PT/ 1pm ET 

In this webcast, you will not only hear about Oracle GoldenGate's strong solutions for cloud environments, but also the latest features that strengthen its offering. The new features we will discuss include:

  • Support for Informix, SQL Server 2014, MySQL Community Edition, and big data environments
  • Real-time data integration between on premises and cloud with SOCKS5 compliance
  • New data repair functionality to help ensure database consistency across heterogeneous systems
  • Moving from Oracle Streams to GoldenGate with the new migration utility

 I would like to invite you to join me and my colleague Chai Pydimukkala, Senior Director of Product Management for Oracle GoldenGate in this session to learn the latest on GoldenGate 12c and ask your questions in a live Q&A.

Hope to see you there!

Tuesday Nov 18, 2014

Oracle GoldenGate for Informix is Released

Oracle GoldenGate for Informix 12.1.2.1.0 is available on OTN and Oracle eDelivery. This is a new addition to Oracle GoldenGate's heterogeneous database support.[Read More]

Monday Oct 27, 2014

Raymond James Financial Leverages Oracle Data Integration

Hot off the press! 

Raymond James Financial shares how it uses Oracle Data Integrator and Oracle GoldenGate in establishing an enterprise information platform that integrates data from multiple heterogeneous sources such as HP NonStop, Microsoft SQL Server, etc and provides a consolidated company view.  This solution provides quicker access to actionable, timely data that helps operational efficiency for its 6,000 financial advisers by enabling repeatable processes in data migration and loading.

Read the press release.

For more information on this solution, this blog may be of interest also.

Wednesday Oct 08, 2014

Meet the Data Integration Winners of 2014 Oracle Excellence Awards for Fusion Middleware Innovation

Last Tuesday evening we had a special celebration at Oracle OpenWorld. In an Oscars-like ceremony we announced the winners of the 2014 Oracle Excellence Awards for Fusion Middleware Innovation. You can find an overview of the Fusion Middleware Innovation Awards Ceremony and the full list of winners across all 12 categories in the following blog post: And The Winners Are....

In this blog post I would like to introduce you the 2014 Fusion Middleware Innovation Award winners in the Data Integration category: NET Serviços and Griffith University.

NET Serviços is Latin America’s largest multi-service cable company and enjoyed a steep growth in its business. However the existing system architecture was not supporting this business growth, and the new product areas and business models they were expanding to.  Net Serviços team, led by Robin Michael Gray, built a single corporate real-time integration platform in a standards-based private cloud architecture with high level of maturity of standardization and architecture documentation. They implemented the private cloud environment on Oracle Exadata and Oracle Exalogic, and for enterprise-wide real-time integration they are using Oracle Data Integrator (ODI), Oracle GoldenGate and Oracle SOA Suite. This platform enabled:

  • 60% reduction in development time
  • 50% reduction in management costs via native solutions for monitoring and error handling
  • 80% reduction in data quality problems via data validation as part of integration
  • Increased agility in identifying and fixing problems due to very innovative ODI data quality firewall and Oracle GoldenGate "error hospital" 

Griffith University is based in Australia, and is one of the most influential universities in the Asia-Pacific region. Bruce Callow, chief technology officer for Griffith University, and his team saw the opportunity to improve student experience and attract more enrollment by building a modern web and mobile interface. Griffith University deployed Oracle Data IntegratorOracle SOA Suite and Oracle Service Bus with the help of their partner Integral to build a real-time data hub that supports key applications such as Oracle RightNow Service Cloud and allows a new web and mobile interface for students. The solution also reduced the load on the core Oracle PeopleSoft system during peak load windows. With this solution Griffith University:

• Achieved unified view of students by using Oracle Data Integrator, Oracle SOA Suite, and Oracle Service Bus

• Improved visibility on the state of demand for courses and to optimize enrollments and course scheduling in real time to best meet that demand

• Increased student satisfaction in enrollment process from 20% to 86%.

You can learn more about Griffith University's solution here: Griffith University Manages Enrollment Peaks 4x Faster—Boosts Student Satisfaction and Retention Rate

Congratulations to Net Serviços and Griffith University with their partner Integral for winning this year's Oracle Excellence Awards for usion Middleware Innovation in Data Integration category! 

Wednesday Oct 01, 2014

Are you at Oracle OpenWorld?

The week isn’t over just yet!

Oracle Data Integration has nine pods located throughout Moscone South where you can meet the product management and development teams. You will find us in the Fusion Middleware, Database, Application and Big Data areas this year.

For additional details on the Oracle Data Integration Solutions track at OpenWorld, make sure to check us out!  There are still many more sessions to attend before the end of the conference!

Monday Sep 08, 2014

Big Data Reservoirs: Getting from Big Data to Valuable Data

Big data’s potential has been widely exclaimed. But many organizations are challenged by the need to collect, store, and analyze the ever-growing amounts and varieties of data that constitute big data.

Big data reservoirs—large holding areas where all kinds of data can be stored and analyzed—are a logical first step in building a next-generation information architecture. In the world of big data, we don’t often know what the value of the data will be. A big data reservoir allows for ad hoc discovery, organization, and enrichment of this data before it traverses to more refined sets of analytics tools.

Because the data volumes are large and the variety of data types unpredictable, big data reservoirs have to be cost-effective and flexible. Not surprisingly, Hadoop and NoSQL databases are the preferred choices for creating big data reservoirs.

But moving data from a big data reservoir into actionable systems requires complex manipulation, comparison, and real-time updates to keep data accurate and systems in synch. Oracle Data Integrator fulfills the critical need of performing complex data manipulations and moving data in and out of big data reservoirs. The latest release, Oracle Data Integrator 12.1.3, is replete with new features designed specifically for the challenges of handling big data. And Oracle GoldenGate works in perfect harmony with Oracle Data Integrator to keep data reservoirs updated with the latest real-time information.

Oracle Data Integrator also delivers leading capabilities for dealing with yet another big data challenge: heterogeneous data. Big data comes in many different forms, from transactional data to social media logs and more, and Oracle Data Integrator can handle all of it. Oracle Data Integrator is the standard for extract, load, and transform operations for Oracle Applications. And it now supports data loading from relational databases into Apache Hadoop Distributed File System (HDFS), Hive, and HBase environments, and from HDFS and Hive into relational databases. In addition, Oracle Data Integrator now supports JavaScript Object Notation and can use Sqoop to integrate Hadoop from non-Hadoop or relational data.

Register for the upcoming Oracle webcast to learn more about best practices using big data reservoirs to unlock the value in your enterprise big data.

Thursday Sep 04, 2014

Oracle GoldenGate is Certified on IBM z/OS 2.1

Starting in Oracle GoldenGate 11.2.1.0.23, Oracle GoldenGate for DB2 z/OS is certified on z/OS 2.1. For more information about Oracle GoldenGate for DB2 z/OS, please refer to the following documentation:

To run on z/OS 2.1, the minimum version of Oracle GoldenGate patch release is as follows:

  • 11.2..1.0.23 18794269 ORACLE GOLDENGATE V11.2.1.0.23 FOR IBM DB2 9.1 ON Z/OS (Patch)


Wednesday Jul 09, 2014

ODI 11g - HDFS Files to Oracle, community post from ToadWorld

There is a new tutorial on using ODI to move HDFS files into an Oracle database using OLH-OSCH from Deepak Vohra on the ToadWorld blog. This article covers all the setup required in great detail and will be very helpful if you're planning on integrating with HDFS files.

http://www.toadworld.com/platforms/oracle/w/wiki/10957.integrating-hdfs-file-data-with-oracle-database-11g-in-oracle-data-integrator-11g.aspx

Thursday Jul 03, 2014

ODI 11g - Hive to Oracle with OLH-OSCH, community post from ToadWorld

There is a new blog post on using ODI to move data from Hive to Oracle using OLH-OSCH from Deepak Vohra on the ToadWorld blog. It covers everything from install to all the configuration of paths and configurations files. So if you are going down this route it's worth checking it out, he goes into great detail into everything that needs done and setup.

http://www.toadworld.com/platforms/oracle/w/wiki/10955.integrating-apache-hive-table-data-with-oracle-database-11g-in-oracle-data-integrator-11g.aspx

Big thanks to Deepak for sharing his experiences and providing the blog to get folk up and running. 

Saturday Jun 21, 2014

Oracle Data Integration Continues to Shine on Oracle Exadata

In previous posts I have shared with you how Oracle offers the best data integration for Oracle Exadata. Oracle’s key differentiation in providing solutions that are engineered to work together, applies to the critical add-on technologies such as data integration and replication as well. Our customers’ successes with using Oracle GoldenGate and Oracle Data Integrator is a continuous flow of confirmation that Oracle’s differentiated offering brings real results, simplifies IT, and enables innovation.

Today, I have two brand new customer examples to share with you. 11ST uses Oracle GoldenGate for real-time data integration into the data warehouse, and Abu Dhabi National Oil Company Distribution uses Oracle GoldenGate for achieving high availability for its private cloud environment running on Exadata.

  • 11ST is Korea’s largest online marketplace where customers and sellers can freely trade retail goods, ranging from clothing and food, to electronic equipment. With over 10x turnover growth over the last 4 years, the company faced challenges in delivering high speed transaction processing and good customer experience. So the company decided to implement Oracle Exadata for core database platform to accelerate data-processing speed for online transactions, and for the data warehouse to improve analytical capabilities with a solution that can handle intensive analytical workloads in a scalable fashion.With Exadata, the company increased storage capacity by up to 5x and decreased storage costs, while improving customer satisfaction by accelerating data-processing speed for online transactions.
11ST chose Oracle GoldenGate as the real-time data integration solution for the data warehouse running on Exadata. Their goal is to enable staff access to more timely data throughout the day such as online sales tracking and earnings by period. Oracle GoldenGate supports fast decision-making by completing daily batch loads 4.5x faster—in two hours instead of nine hours.

The key business transformation is seen in their customers’ experience. With the availability of real-time data synchronization between the enterprise data warehouse and marketing systems, the company can act fast to changing customer tastes, such as for women’s clothing and jewelry, to improve its offering and customer service. Using timely data and advanced analytics, the company runs personalized marketing programs, such as cosmetics or shoe promotions, through mobile, private-brand services, which was not possible with the legacy system. You can read more about 11ST story here.

Especially for data warehousing solutions many customers leverage the comprehensive and integrated Oracle Data Integration platform for Exadata. Below is a diagram that shows how customers can achieve an end-to-end solution.

Avea,and Paychex are ywo other good examples of using Oracle Data Integration for Exadata environments for improved analytical capabilities.

  • Abu Dhabi National Oil Company Distribution (ADNOC Distribution) is a United Arab Emirates (UAE) government-owned company that specializes in marketing and distributing petroleum products within the country. The organization decided to implement Oracle Exadata Database Machine to consolidate five Oracle Databases—supporting enterprise resource planning (ERP), online transaction processing (OLTP) applications, and business intelligence (BI) applications—onto a single database platform within a private database cloud.

As this private cloud environment supports the backbone of a business with 250 million financial transactions annually, ADNOC Distribution cannot afford even a minute's downtime. To ensure 24/7 uptime and meet service-level agreements, the company uses Oracle Exadata with Oracle Real Application Clusters and Oracle Automatic Storage Management. Using Oracle GoldenGate ADNOC maximized system availability and improved its ability to integrate Oracle Exadata with nonOracle platforms, such as Microsoft SQL or IBM DB2. You can read more about 11ST story here.

 In a recent webcast we discussed Oracle GoldenGate's offering for consolidation into private cloud. I also recommend watching the webcast: Zero Downtime Consolidation to Oracle Database 12c with Oracle GoldenGate 12c on demand to learn how to use Oracle GoldenGate for maximizing availability in private cloud environments.

Friday May 30, 2014

Looking for Cutting-Edge Data Integration: 2014 Excellence Awards

2014 Oracle Excellence Awards Data Integration

It is nomination time!!!

This year's Oracle Fusion Middleware Excellence Awards will honor customers and partners who are creatively using various products across Oracle Fusion Middleware. Think you have something unique and innovative with one or a few of our Oracle Data Integration products?

We would love to hear from you! Please submit today.

The deadline for the nomination is June 20, 2014.

What you win:

  • An Oracle Fusion Middleware Innovation trophy

  • One free pass to Oracle OpenWorld 2014

  • Priority consideration for placement in Profit magazine, Oracle Magazine, or other Oracle publications & press release

  • Oracle Fusion Middleware Innovation logo for inclusion on your own Website and/or press release

Let us reminisce a little…

For details on the 2013 Data Integration Winners:

Royal Bank of Scotland’s Market and International Banking and The Yalumba Wine Company, check out this blog post: 2013 Oracle Excellence Awards for Fusion Middleware Innovation… and the Winners for Data Integration are…

and for details on the 2012 Data Integration Winners:

Raymond James and Morrisons, check out this blog post: And the Winners of Fusion Middleware Innovation Awards in Data Integration are… 

Now to view the 2013 Winners (for all categories).

We hope to honor you!

Here's what you need to do: 

Click here to submit your nomination today.  And just a reminder: the deadline to submit a nomination is 5pm Pacific Time on June 20, 2014.

About

Learn the latest trends, use cases, product updates, and customer success examples for Oracle's data integration products-- including Oracle Data Integrator, Oracle GoldenGate and Oracle Enterprise Data Quality

Search

Archives
« March 2015
SunMonTueWedThuFriSat
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
    
       
Today