Thursday Dec 08, 2011

OWB – SQLLoader, big data files and logging

OWB’s flat file loading support is rich and there are a lot of configuration properties for mappings in general and the properties exposed for SQLLoader mappings are expansive. One of the things that OWB does is load the resultant logs from SQLLoader into the runtime audit tables through the runtime service and also scrapes audit information (number of rows etc) from the file.

The thing to be wary of is whether a verbose output is used for feedback – combined with the rows per commit property (default is 200) the SQLLoader mapping will write a feedback message every 200 rows committed. Imagine…big data files and filling a log file with this kind of message every n rows can be quite large!

With this setting your log will have a lot of ‘Commit point reached’ messages in the log.

 

The ‘Supress’ properties (a typo I know) can be used to hide this information and make the logs compact, so switch on the Supress Feedback property as follows and the log shrinks, with no verbose output;

 

 

This equates to the SQLLoader SILENT option, if we look at the control file generated by OWB, we see the SILENT=(FEEDBACK) option is now added;

 

The tooltip for the misspelled ‘Supress Feedback’ is ‘Suppresses the “commit point reach” messages that normally appear on the screen’. Others have come across this on the forum also.

About

ETL, CDC, Real-Time DI and Data Quality for the Oracle Database from the inside.

Search

Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today