Welcome to All Things Warehouse Builder

  • ETL
    August 25, 2008

Sizing an OWB system...

Jean-Pierre Dijcks
Master Product Manager

Should I even go here...? Maybe not, but one thing that I thought would be interesting to mention here is an avenue to take when you are struggling with sizing. Yes, this will have loads of disclaimers but might be a better place to start than just out of the blue...

So, lets say we are trying to set up an OWB system, which may be a DW or Data mart. What hardware do we run this on so that we get some half decent performance? Good question. Ask the hardware guy, and you'll get lots of cool stuff that you may not need (or may, but find to expensive). Ask the software guys, and you'll get puzzled looks. Or a pointer back to the hardware guys.

Now what do you do. Well for starters, look at this page: http://www.oracle.com/solutions/business_intelligence/optimized-warehouse-initiative.html

Oracle has addressed the issue with pre-defined configurations that create a BALANCED system. It is not a system that will give you infinitely great performance for the smallest $$$ amount... it is a balanced system. This means you will get a good throughput in a normal data warehouse like situation. It does this balancing because it specifies all components. So not just CPU and memory, but disks, HBAs (connect your CPU to your disky things and is kind of required to actually load data and process it - yet often forgotten).

Back to sizing for OWB, these systems will do a good job because they are balanced, and we would expect them to be a good starting point for any DW system.

Since OWB does the ETL in the database, you would not need any extra hardware to host the ETL server, it will go into this single system. Due to that, sizing is simpler and hardware utilization is better.

For data quality sizing is a bit more complex, but even here, if you look at the basic systems, sizing a profiling application is roughly equal. Look at the total data size, determine complexity (much less users on the profile run, a few doing simple queries after it is profiled) and these configs would work as a good starting point as well.

Looking at that optimized warehouse page, note that most vendors are present, so you can kind of pick the one you like. Now I know the temptation is there to skimp on stuff, but if you start with this, and don't skimp on items, you might just have sized your system about right...

Join the discussion

Comments ( 1 )
  • Greg Partenach Wednesday, August 27, 2008
    I do think a good discussion on sizing is necessary. Not only this, but knowing when you should not need to size up any larger than 'X', even when having performance issues. Often, there are nice techniques in the ETL and even bits of tuning on the repository that can be done to increase performance quite a bit.
Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.