Information, tips, tricks and sample code for Big Data Warehousing in an autonomous, cloud-driven world

Data Model or Knowledge Model

Jean-Pierre Dijcks
Master Product Manager

Data models (like Oracle's Retail Data Model and the newly released Communications Data Model) have some interesting benefits - we think. The most commonly mentioned one is that these models offer shorter time to value for a data warehouse or BI project. But is this really the only (ok, as if that is not enough) value that comes from a data model? I would hope the answer is "NO - there is soooo much more you get from using such a model".

Data Models

I do think there is much more to the models, but only if you get the right model with the right implementation. Let's look into data models a little bit more.

A data model - and let's assume that the logical model is cute but the real cheese is in the physical model - delivers a set of predefined entities translated into tables. This means that you as the implementer of the data warehouse does not have to model all of this. Therefore you just saved time. If the model is good, you now have saved time and added value by delivering a data warehouse in a short(er) time.

Knowledge Models

Ok. Sounds good, but I think that the value in BI and Data Warehousing increasingly comes from looking forward. If the data model is restricted to the above, does that really help? I would argue that the data model should go a step further and should grow into a knowledge model.

A knowledge model offers pre-defined solutions delivering knowledge about what is going on in a business and are tailored to that business. So it is aimed at a particular vertical (most data models are by the way) and it provides a window into the detailed information needed (again, most models provide this). It should however also deliver knowledge to drive the business forward, in the form of advanced analytics tailored to specific problems. It should democratize knowledge by embedding advanced analytics. By embedding real value - things your end users have never even dreamed of knowing - time to value becomes about delivering real value in a real short time.

The important question - at least in my mind - is to find a data model that delivers knowledge in a pre-built form. And to deliver that, the data model must go beyond tables and BI reports. It must even go beyond dashboards. It should attempt to predict the future or at least allow you  to play with the future. That means that a knowledge model must include advanced analytics in the form of OLAP like possibilities and must include data mining models available to the masses. A knowledge model is not just a logical model, but a fully implemented solution providing that knowledge-out-of-the-box.

The Multi-Engine Problem

The assessment that a knowledge model needs real OLAP and real data mining capabilities creates an issue for any solution that relies on pieces of the whole being delivered by separate engines. Ideally the model has all options embedded and leverages the same base engine to compute all the cool stuff.  The reason to keep it all together in the same engine is twofold. One reason is that with the increase in data stored moving data around becomes becomes too hard. The second reason is that you want to avoid building ETL routines from the nice schema you just bought to any outside engines. If you need to do all this intra-solution ETL the time to deliver real value dramatically increases. 

I do think visualizations can come from anywhere as long as all the underlying knowledge production can be fed into the upper visualization layers. This is the obvious infrastructure talks to upper layers via a set of known and public APIs or languages.

Embedding things like data mining into the data model, than surfacing the information via SQL in the generic BI solution provides a nice and clean architecture and provides flexibility to find solutions that fit your business.

Dealing with Change

A competitive edge rarely comes from something that has been around for years, competitors move fast as the general business does. The communications business has changed dramatically over the past few years and is more and more driven by content, SMS messaging and images. I mean, who talks on a phone these days?

So do start with a model that delivers smart stuff from the get go to business users. They will appreciate what they get and then figure out how to use the smarts to make the business more successful. Starting with just a data model is not really going anywhere as it does not tickle the imagination. But you will have to take a risk and use some new ideas once in a while.

Data Model or Knowledge Model?

To get to a knowledge model you should look at more than just the data model. Consider where you are implementing the parts and pieces and how this gets all provided to the end user. True value comes from knowledge models, not data models. True value comes from a smart infrastructure use, not from sticking with the old and known.

Be the first to comment

Comments ( 0 )
Please enter your name.Please provide a valid email address.Please enter a comment.CAPTCHA challenge response provided was incorrect. Please try again.