This week’s guest blog post is contributed by Jonathan Seidner, Senior Director, Engineering, Oracle Data Cloud.
An article in our Cross-Device Learning Center, Device Map Accuracy, Precision and Recall, comprehensively explains the technical differences between the three different measurements used to describe device maps, and particularly the inherent trade-off between precision and recall.
However, that article doesn’t go into much detail on when a company should focus more on precision and more on recall. In this post, I will present some of the practical implications of this trade-off.
If you haven’t read that article, I’ll summarize the bottom line here: there will always be a trade-off between precision and recall in a device map.
While achieving the optimum balance between the two is one of the main goals of the data science behind probabilistic device matching, there are situations in which a focus on one or the other is necessary for achieving the best business performance of using the device map. A refresher of the definitions of these terms:
For some use cases, precision (correctness) is more important, whereas in others, recall (market coverage) is what counts. Here are some examples of each.
For cross-device analytics and cross-device conversion attribution, it is critical to accurately measure the effectiveness of advertising campaigns across devices. Incorrect data in this case can lead to erroneous marketing decisions that reduce advertising effectiveness and decrease the ROI of an advertiser’s advertising budget.
For these applications, total market coverage is less important than a high degree of accuracy, as even an accurate representative sample can provide the necessary intelligence to make the right marketing decisions.
Another case in which precision is more important than recall is for optimizing the ROI of programmatic retargeting campaigns based on campaign performance/results over time.
The programmatic retargeter does not want to waste CPM dollars on showing the retargeting ads to the incorrectly-matched additional devices of known visitors, so it is important to limit retargeting ad impressions to those linked devices with high confidence only.
When running large-scale targeted advertising campaigns where high volume is the goal, it makes sense to try extend the target audience segments via all devices connected with the core segment.
This will be true, even if the lack of 100% precision means that some devices matched to the core audience at lower confidence intervals will not actually belong to the audience.
Likewise, for an advertiser (as opposed to a vendor) running CPC cross-device retargeting campaigns, market reach is more important than precision.
This is because even if the retargeting ads are displayed on some devices which are not, in fact, used by individuals who previously visited the website, there is little or no harm in the “mistake.”
This is because the advertiser is only paying for clicks in this scenario. Yet, to maximize the effectiveness of the campaigns, it is very important to cover as much of the total set of recent website visitors as possible.
Precision and recall are two parameters of device maps that “compete” with each other.
While probabilistic device matching vendors attempt to find the ideal balance between the two, there are particular use cases in which focusing on one or the other is necessary for achieving the best business performance of using a device map.
Image: ESB Basic/Shutterstock