Measuring reality is much easier than reconstructing it

The title of this entry says it all. When it comes to collecting data for any analytic work, it is much easier to measure the current data than attempting to reconstruct it from historical databases.

For example, assume you need to analyze the factors that affect cross selling success in the call center and you want to include data like the wait time in the queue or the number of calls the agent answered in the current shift before the call where cross selling was attempted. Collecting this data from history is very complex because:

  1. Not all data is collected all the time
  2. Data from different systems may end up in very disparate historical databases
  3. Different data may have different retention periods and granularity
  4. Different systems may have uncoordinated clocks
  5. Queries become very complex when trying to pinpoint the state of a data record at a specific time
  6. Queries become complex in order to include only events that happened before the point in time in question
For all these reasons and more, it is much easier to perform analytics in Real-Time, when reality can be measured by directly connecting to other systems. For example, it does not matter if the clocks in the different systems are totally unccordinated or work in a different time zone, all I need to worry about is to retrieve the latest data. Similarly, if I need to know the city a person lives in I just retrieve it from the DB, there is no need to go through the list of address changes.

This is one of the reasons I believe that even if you can hand-craft very accurate models, the real time models automatically generated by a self learning system can, in many cases, end up being much more accurate because they can take advantage of more data that is also more accurate.
Comments:

Post a Comment:
  • HTML Syntax: NOT allowed
About

Issues related to Oracle Real-Time Decisions (RTD). Entries include implementation tips, technology descriptions and items of general interest to the RTD community.

Search

Categories
Archives
« April 2014
SunMonTueWedThuFriSat
  
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
   
       
Today