Confidence in Recommendation
By user12610620 on Jun 02, 2010
I heard an interesting talk during lunch today from the Oracle folks who do retail science. They create forecasts about how products will sell given different factors. To oversimplify, "you'll sell N units of this product if you discount it by 25% starting on this date". One of the non-technical problems they face is building confidence in their forecasts. Sometimes a prediction (i.e. a recommendation) makes sense to the user - have a sale on flowers before Mother's Day and you'll sell more flowers. But sometimes the model predicts something that doesn't have an obvious explanation even through the historical data that is collected indicates that it should be true. The user has to make a judgment call. This, I think, is very similar to the kind of trust a user builds in a recommender system. If the first thing the retail system predicts doesn't make any sense to me (even though it is based on solid facts), my first impression will be that the system isn't going to work. What I really want is for the system to start out by demonstrating that it can predict the "obvious" things (like flowers for Mother's Day, or in our music recommender's case, that I might like Coldplay). Once I'm confident that it can do that, then I'll feel more confident in it when it predicts something less obvious.
On a somewhat unrelated note, one of the speakers observed that a product that has only a few items on the shelf isn't going to sell well. Picture a grocery store where there's only a few boxes of couscous left. Maybe you'll pass on those and get something else. You think "maybe those are the rejects that nobody else wanted". I think it's an interesting look at human behavior that when you turn that into an online store, you can motivate people to buy those boxes by saying "Only 3 boxes left".