AbstractConsumer reviews and ratings of products and services have become ubiquitous on the internet. This paper analyzes the implications of the sequential nature of reports on the accuracy of their representation of the true underlying quality of the service they rate. We consider a sequence of consumers arriving sequentially over time and reporting a grade for some service. Upon arrival to the system, the consumer develops a sincere rating and also observes the average of past ratings. She then provides a report based on these two pieces of information. We analyze how different behavioral models with regard to how consumers account for existing reviews impact the gap one may observe between reports' statistics and those of the true underlying quality. We show that biases arise: in general, the long-run average might be different from the the true average quality of the service. We establish that the worst-case gap can be (arbitrarily) large for herding reporting mechanisms while it is always bounded for compensating mechanisms. Despite such distortions, we show that structure exists with regard to the relative ordering of alternatives: the long-run averages are monotone in the stochastic order of the true rating. We then analyze the potential for manipulation as a function of the reporting behavior.
View Ideas at Work: Research Brief
Besbes, Omar, and Marco Scarsini. "On Information Distortions in Online Ratings." Columbia University, May 16, 2013.