« Getting your feet wet with data profiling: freeware tool | Main | Want to learn more about the HP Oracle Database Machine? »

Good Data Quality Read on Rittman-Mead's Blog, and validating input with data rules

There's a good read over on the Rittman-Mead Consulting blog on End-to-End data quality. OWB would be a good tool to implement parts of the solution they have in mind-- to do an initial DQ assessment, and to do cleansing during load. And DQ reporting on both the sources and the target system is certainly doable, using data auditors and your favorite BI application, to prove to the powers above that the problem isn't yours.

Input validation is where OWB's answer is a bit less obvious, though. Enforcing the OWB DQ rules during input validation is doable, I think, but you have to be creative. Here's how I'd go about it...

OWB creates PL/SQL packages for each PL/SQL mapping, including mappings with data rules. One way of coming at this would be to create a mapping for each set of rules you need to enforce at input time, and that returns the required validation status as an output. Then expose the PL/SQL package for the mapping as a database web service as described in the Warehouse Builder SDK.

Invoke this web service from your source application at the time of input, and reject bad entries and report problems based on your output.

Note that this also gives you consistency between your data rules as used in ETL w/DQ and your input validation at the front end; if you change your data rules, then re-generate and re-deploy, your validation logic is updated as well.


TrackBack

TrackBack URL for this entry:
http://blogs.oracle.com/mt/mt-tb.cgi/8264

Post a comment

(If you haven't left a comment here before, you may need to be approved by the site owner before your comment will appear. Until then, it won't appear on the entry. Thanks for waiting.)