Friday Jan 11, 2013

Customer Spotlight:Raymond James

Back in October we announced that Raymond James is a winner of 2012 Fusion Middleware Innovation Awards in the Data Integration category. As I mentioned on this site before, Raymond James used Oracle Data Integrator and Oracle GoldenGate in an integrated solution for its operational data store that supports the application services across the enterprise. By using Oracle's real-time data integration solution from heterogeneous systems into a centralized ODS, the team achieved higher application performance, reduced complexity and cost, and up to 50 percent faster time to market. 

I had a pleasure meeting the team, Ryan Fornett, Tim Garrod, and Michael Wells, from Raymond James at Oracle OpenWorld 2012. In addition to hearing from them during their presentations, we had a chance to record interviews with Tim and Ryan while at OpenWorld. You can find the links to their videos below:

 Yesterday, with their permission we posted a detailed blog on the FMW blog site about their implementation:

Data Integration Customer Spotlight: Raymond James Financial

I highly recommend reading it to see Oracle Data Integration products in action.

Many thanks to Ryan, Tim, and Mike for sharing the details of their innovative solution with us.


Tuesday Jun 19, 2012

ODI 11g - Faster Files

Deep in the trenches of ODI development I raised my head above the parapet to read a few odds and ends and then think why don’t they know this? Such as this article here – in the past customers (see forum) were told to use a staging route which has a big overhead for large files. This KM is an example of the great extensibility capabilities of ODI, its quite simple, just a new KM that;

  1. improves the out of the box experience – just build the mapping and the appropriate KM is used
  2. improves out of the box performance for file to file data movement.

This improvement for out of the box handling for File to File data integration cases (from the 11.1.1.5.2 companion CD and on) dramatically speeds up the file integration handling. In the past I had seem some consultants write perl versions of the file to file integration case, now Oracle ships this KM to fill the gap. You can find the documentation for the IKM here. The KM uses pure java to perform the integration, using java.io classes to read and write the file in a pipe – it uses java threading in order to super-charge the file processing, and can process several source files at once when the datastore's resource name contains a wildcard. This is a big step for regular file processing on the way to super-charging big data files using Hadoop – the KM works with the lightweight agent and regular filesystems.

So in my design below transforming a bunch of files, by default the IKM File to File (Java) knowledge module was assigned. I pointed the KM at my JDK (since the KM generates and compiles java), and I also increased the thread count to 2, to take advantage of my 2 processors.

For my illustration I transformed (can also filter if desired) and moved about 1.3Gb with 2 threads in 140 seconds (with a single thread it took 220 seconds) - by no means was this on any super computer by the way. The great thing here is that it worked well out of the box from the design to the execution without any funky configuration, plus, and a big plus it was much faster than before,

So if you are doing any file to file transformations, check it out!

About

Learn the latest trends, use cases, product updates, and customer success examples for Oracle's data integration products-- including Oracle Data Integrator, Oracle GoldenGate and Oracle Enterprise Data Quality

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
2
3
5
6
7
8
9
10
12
13
14
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today