OBIA Fusion Cloud Adaptor - View extracted files in UCM

Authors: "Sireesha Mandava - Oracle"

When using Cloud Adaptor, the files will get uploaded to UCM/Storage Service. This blog explains how to view the files in UCM.

Every extract that is run creates what is referred to as a Manifest file. This file contains the list of files that make up that extract. The file name is MANIFEST.MF which gets renamed to <timestamp>_MANIFEST_<TIMESTAMP>.MF post successful download via the ODI Load Plan.

At times you want to verify the files that got extracted before running the load plan. This blog explains the steps to do so.

Locate the Manifest File:

From the UCM console, search for MANIFEST.MF.

Note: If files are already downloaded, the manifest would have renamed as explained above.Change your search criteria accordingly in such a case.

You will see the list of Manifest files in the search results as below:

Click on the hyperlink to view the contents of the manifest file.

How to use MANIFEST

Extractor uploads a MANIFEST.MF file along with all the extracted VO's csv/mdcsv/pecsv in compressed/zip form. MANIFEST file will have the list of files that are uploaded in the current batch run. MANIFEST file is NOT compressed/zip. There can be more than one MANIFEST.MF file; one per extract run.

.csv files are VO data files.

.mdcsv files are metadata files which has column and its data type definition for Flex VOs.

.pecsv files are data files having only primary key column values, which are used to identify deleted records in warehouse.

File name format "file_{VONAME}-batch{number}-{TIMESTAMP}". Sample file_crmanalyticsam_agreementam_agreementitempvo-batch1813107209-20140712_123812

All files will be uploaded with file extension .zip irrespective of the above mentioned file types. BatchDetails files can be ignored as its no more used and will not be generated/uploaded in future.

Sample UCM MANIFEST.MF file content -

First row of the MANIFEST has information about fusion apps source version used by BI Apps warehouse integration which can be ignored for custom warehouse integration. Further lines will have information about the uploaded files in the form vo_name;ucm_document_id;md5_check_sum_value. For example

crmanalyticsam_partiesanalyticsam_customer;9526;b2af2bf486366e2c2cb7598849f0df2e

Here '9526' is the 'ucm_document_id' of uploaded file, 'b2af2bf486366e2c2cb7598849f0df2e' is 'md5_check_sum_value'.So the file corresponding to the entry above would be named something like

file_crmanalyticsam_partiesanalyticsam_customer-batch1813107209-20140712_123812.csv

While downloading content from UCM, search for DOCTITLE "MANIFEST.MF" with sort by DOCDATE in DESC order. This will provide all the MANIFEST.Mf UCM
docIds in order. Download each MANIFEST file using docId. Read its content to download data files using their respective 'ucm_document_id' by parsing the above mentioned form. May use MD5 check sum to verify downloaded file content.

After unzipping, you can process files based on their file extension like .csv, .mdcsv, .pecsv

Once the data files are processed, the corresponding MANIFEST.MF file in UCM is stamped with TIMESTAMP prefix like '{TIMESTAMP}_MANIFEST.MF' so that its not picked in the next download.

View csv or mdscv files in UCM

Once you have identified the manifest file corresponding to the extract you are interested in, you can get the name of the csv/mdsv/pecsv file from the manifest. Then search by that file name to locate the file in UCM and then view it (similar to steps mentioned under "Locate the Manifest file" above).