Nan Hu
Abstract
Introduction
While product review systems that collect and disseminate opinions about products from recent buyers (Table 1) are valuable forms of word-of-mouth communication, evidence suggests that they are overwhelmingly positive. Kadet notes that most products receive almost five stars. Chevalier and Mayzlin also show that book reviews on Amazon and Barnes & Noble are overwhelmingly positive. Is this because all products are simply outstanding? However, a graphical representation of product reviews reveals a J-shaped distribution (Figure 1) with mostly 5-star ratings, some 1-star ratings, and hardly any ratings in between. What explains this J-shaped distribution? If products are indeed outstanding, why do we also see many 1-star ratings? Why aren't there any product ratings in between? Is it because there are no "average" products? Or, is it because there are biases in product review systems? If so, how can we overcome them?
The J-shaped distribution also creates some fundamental statistical problems. Conventional wisdom assumes that the average of the product ratings is a sufficient proxy of product quality and product sales. Many studies used the average of product ratings to predict sales. However, these studies showed inconsistent results: some found product reviews to influence product sales, while others did not. The average is statistically meaningful only when it is based on a unimodal distribution, or when it is based on a symmetric bimodal distribution. However, since product review systems have an asymmetric bimodal (J-shaped) distribution, the average is a poor proxy of product quality.
This report aims to first demonstrate the existence of a J-shaped distribution, second to identify the sources of bias that cause the J-shaped distribution, third to propose ways to overcome these biases, and finally to show that overcoming these biases helps product review systems better predict future product sales.
We tested the distribution of product ratings for three product categories (books, DVDs, videos) with data from Amazon collected between February--July 2005: 78%, 73%, and 72% of the product ratings for books, DVDs, and videos are greater or equal to four stars (Figure 1), confirming our proposition that product reviews are overwhelmingly positive.
Figure 1 (left graph) shows a J-shaped distribution of all products. This contradicts the law of "large numbers" that would imply a normal distribution. Figure 1 (middle graph) shows the distribution of three randomly-selected products in each category with over 2,000 reviews. The results show that these reviews still have a J-shaped distribution, implying that the J-shaped distribution is not due to a "small number" problem. Figure 1 (right graph) shows that even products with a median average review (around 3-stars) follow the same pattern.
- Admati, A. R. and Pfleiderer, P. Broadcasting opinions with an overconfident sender. International Economic Review 45, 2, (2004), 467--498.Google Scholar
Cross Ref
- Chatterjee, P. Online reviews: Do consumers use them. Advances in Consumer Research 28, 1, (2001), 129--133.Google Scholar
- Chevalier, J. and Goolsbee, A. Measuring prices and price competition online: Amazon and Barnes and Noble. Quantitative Marketing and Economics 1, 2, (2003), 203--222.Google Scholar
- Chevalier, J. and Mayzlin, D. The effect of word of mouth on sales: Online book reviews. J. of Marketing Research 43, 3, (2006).Google ScholarCross Ref
- Clemons, E. K., Gao, G., and Hitt, L. M. When online reviews meet hyper differentiation: A study of craft beer industry. J. of Management Information Systems 23, 2, (2006), 149--171. Google ScholarDigital Library
- Dellarocas, C. The digitization of word-of-mouth: Promise and challenges of online reputation mechanisms. Management Science 49, 10, (2003), 1407--1424. Google ScholarDigital Library
- Dellarocas, C., Awad, N., and Zhang, X. Exploring the value of online reviews to organizations: Implications for revenue forecasting and planning. Proceedings of the 24th International Conference on Information Systems, 2004, Washington, D.C.Google Scholar
- Godes, D. and Mayzlin, D. Using online conversations to study word of mouth communication. Marketing Science 23, 4, (2004), 545--560. Google ScholarDigital Library
- Kadet, A. Rah-Rah ratings. SmartMoney Magazine, (Feb. 23, 2007) 116.Google Scholar
- Li, X. and Hitt, L. M. Self selection and information role of online product reviews. Workshop on Information Systems and Economics, 2004, Washington, DC.Google Scholar
- Liu, Y. Word-of-mouth for movies: Its dynamics and impact on box office revenue. J. of Marketing 70, 3, (2006) 74--89.Google ScholarCross Ref
- Reichheld, F. The one number you need to grow. Harvard Business Review 81, 12, (2003), 46--54.Google Scholar
Index Terms
- Overcoming the J-shaped distribution of product reviews
Recommendations
On self-selection biases in online product reviews
Online product reviews help consumers infer product quality, and the mean (average) rating is often used as a proxy for product quality. However, two self-selection biases, acquisition bias (mostly consumers with a favorable predisposition acquire a ...
Personalized Recommendation Algorithm Based on Product Reviews
Under the background of leap-forward development for the internet, e-commerce has played an important role in people's daily life, but huge data sizes have also brought problems, such as information overload which can be solved by using a recommendation ...
The Impact of Online Product Reviews on Product Returns
Although many researchers in information systems and marketing have studied the effect of product reviews on sales, few have looked at their effect on product returns. We hypothesize that, by reducing product uncertainty, product reviews affect the ...
Comments