ODI - Integrating social data from Facebook Graph API in 0 to 60

I'm always on the lookout for enabling technology to push the envelope. Here is a viewlet illustrating how to integrate Facebook  data into your flow. It uses a preprocessor that will help you integrate JSON in general into your data integration world.

There are a few pieces to the jigsaw. ODI provides an XML driver that can be used to process XML data from the filesystem, http or ftp for example. ODI 11.1.1.7 has improved this greatly by leveraging Oracle standard XML schema and parsing components. Transforming JSON to XML is one approach which is convenient for when you already have XML based tools or utilities. With this utility you can load JSON data into your data integration flows and in the process understand more about the data (when we reverse engineer the JSON we get a schema). Using this we can integrate arbitrary JSON data such as social data feeds from Twitter, Facebook, Google, and aggregators like Datasift among many others.

Watch the viewlet to see it in action.
Comments:

Hi David,

Great example to get people thinking of different ways to connect and gather data from various external sources. And the viewlet approach to sharing is great!

Regards,
Michael Rainey

Posted by Michael Rainey on April 17, 2013 at 07:44 AM PDT #

Thanks Michael, its simple but powerful when the pieces are used together.

Cheers
David

Posted by David on April 17, 2013 at 08:20 AM PDT #

Great stuff David!

Failed at the comment authentication first time round. 9+59 = 68 and not 67 :-(

Cheers
uli

Posted by guest on April 17, 2013 at 02:35 PM PDT #

Hi Uli

Yep, 68! You should see what some people answered;-) Just kidding

Cheers
David

Posted by David on April 17, 2013 at 03:54 PM PDT #

Hi David,

Do we have the jar file available with ODI 11.1.1.7 or we have to download it seperately as ODIFromJSON is not recongnised by ODI and gives error.

I know we can write the code ourself to get the data from social websites but I am interested if there is something new in ODI 11.1.1.7

Thanks in advance
Cheers
Nishant

Posted by Nishant on April 29, 2013 at 04:26 AM PDT #

Hi Nishant

I'll post the material on the ODI java.net site today and post a blog entry pointing to it. It will work with
11.1.1.7 also.

Cheers
David

Posted by David on April 29, 2013 at 07:46 AM PDT #

Hi Nishant

I have uploaded a user function into the OracleDI java.net sharing site. You should be able to access this with your OTN account. The ODI user function wrappers the Java, read the readme.txt in the zip to get install instructions.

https://java.net/projects/oracledi/downloads/download/ODI/User%20Functions/odi_json_utils.zip

Cheers
David

Posted by guest on April 29, 2013 at 10:22 AM PDT #

Hi David,

Thanks for this. This jar is able to convert JSON into XML. However, when we do reverse engineering in ODI, i got an below error while generating its DTD. But when I removed some data from the XML file and reverse it then it was working. It look like some issue with the XML file structure

oracle.xml.parser.v2.XMLParseException" exception occurred saying: Expected name instead of 0.

Regards
Nishant

Posted by Nishant Jain on April 30, 2013 at 02:59 AM PDT #

I got the message which was creating an issue. In below "0" was converted to <0>, which was not get parsed by ODI XML driver

"message": "Oracle - Ironman 3",
"message_tags": {
"0": [
{
"id": "113897213765",
"name": "Oracle",
"type": "page",
"offset": 0,
"length": 6
}
]
},

Posted by Nishant Jain on April 30, 2013 at 03:46 AM PDT #

Hi Nishant

The Facebook JSON returned now has these names generated which are invalid XML element names. Let me see if I can get the converter to change them. I came across a similar problem with google restful apis that had JSON tags that were invalid XML element names.

Cheers
David

Posted by David on May 01, 2013 at 08:21 AM PDT #

Hi Nishant

I've made a change to help with this case and updated the zip on java.net. You will have to remove the old JAR and update with the new one (version 1.1) in this zip.

Cheers
David

Posted by David on May 03, 2013 at 09:12 AM PDT #

Outstanding David!! We are going to fetch social networking sites information and this post gave me a starting point to work on this. Is there a way I can subscribe for your posts to get notification whenever you post something new ?

Posted by Bhabani on August 06, 2013 at 09:09 AM PDT #

Hi Bhabani

I'm glad it gave you some ideas, there was a follow up blog post below with the utility if that helps also;
https://blogs.oracle.com/dataintegration/entry/odi_integrating_more_social_data

I was posting on the OWB blog, but now primarily post ODI related articles to this blog.
Cheers
David

Posted by David on August 06, 2013 at 09:15 AM PDT #

Post a Comment:
  • HTML Syntax: NOT allowed
About

Learn the latest trends, use cases, product updates, and customer success examples for Oracle's data integration products-- including Oracle Data Integrator, Oracle GoldenGate and Oracle Enterprise Data Quality

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
2
3
5
6
7
8
9
10
12
13
14
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today