skip to main content
10.1145/1282100.1282158acmotherconferencesArticle/Chapter ViewAbstractPublication PagesicecConference Proceedingsconference-collections
Article

Designing novel review ranking systems: predicting the usefulness and impact of reviews

Published:19 August 2007Publication History

ABSTRACT

With the rapid growth of the Internet, users' ability to publish content has created active electronic communities that provide a wealth of product information. Consumers naturally gravitate to reading reviews in order to decide whether to buy a product. However, the high volume of reviews that are typically published for a single product makes it harder for individuals to locate the best reviews and understand the true underlying quality of a product based on the reviews. Similarly, the manufacturer of a product needs to identify the reviews that influence the customer base, and examine the content of these reviews. In this paper, we propose two ranking mechanisms for ranking product reviews: a consumer-oriented ranking mechanism ranks the reviews according to their expected helpfulness, and a manufacturer-oriented ranking mechanism ranks the reviews according to their expected effect on sales. Our ranking mechanism combines econometric analysis with text mining techniques in general, and with subjectivity analysis in particular. We show that subjectivity analysis can give useful clues about the helpfulness of a review and about its impact on sales. Our results can have several implications for the market design of online opinion forums.

References

  1. Archak, N., Ghose, A., and Ipeirotis, PG. Show me the money! Deriving the pricing power of product features by mining consumer reviews. In Proceedings of the Thirteenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD-2007) (2007). Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Berndt, E. R. The Practice of Econometrics: Classic and Contemporary. Addison-Wesley, 1996.Google ScholarGoogle Scholar
  3. Chevalier, J. A., and Mayzlin, D. The effect of word of mouth on sales: Online book reviews. Journal of Marketing Research 43, 3 (Aug. 2006), 345--354.Google ScholarGoogle ScholarCross RefCross Ref
  4. Cui, H., Mittal, V., and Datar, M. Comparative experiments on sentiment classification for online product reviews. In Proceedings of the Twenty-First National Conference on Artificial Intelligence (AAAI-2006) (2006). Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. Das, S. R., and Chen, M. Yahoo! for Amazon: Sentiment extraction from small talk on the web. Working Paper, Santa Clara University.Google ScholarGoogle Scholar
  6. Dave, K., Lawrence, S., and Pennock, D. M. Mining the peanut gallery: Opinion extraction and semantic classification of product reviews. In Proceedings of the 12th International World Wide Web Conference (WWW12) (2003), pp. 519--528. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. Dellarocas, C., Awady, N. F., and Zhangz, X. M. Exploring the value of online product ratings in revenue forecasting: The case of motion pictures. Working Paper, Robert H. Smith School Research Paper, 2007.Google ScholarGoogle Scholar
  8. Forman, C., Ghose, A., and Wiesenfeld, B. A multi-level examination of the impact of social identities on economic transactions in electronic markets. Working Paper CeDER-06-09, Stern School of Business, New York University. Available at http://hdl.handle.net/2451/14809, 2006.Google ScholarGoogle Scholar
  9. Ghose, A., Ipeirotis, P. G., and Sundararajan, A. Opinion mining using econometrics: A case study on reputation systems. In Proceedings of the 44th Annual Meeting of the Association for Computational Linguistics (ACL 2007) (2007).Google ScholarGoogle Scholar
  10. Gruhl, D., Guha, R., Kumar, R., Novak, J., and Tomkins, A. The predictive power of online chatter. In Proceedings of the Eleventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD-2005) (2005), pp78--87. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. Hatzivassiloglou, V., and McKeown, K. R. Predicting the semantic orientation of adjectives. In Proceedings of the 38th Annual Meeting of the Association for Computational Linguistics (ACL'97) (1997), pp. 174--181. Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. Hu, M., and Liu, B. Mining and summarizing customer reviews. In Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD-2004) (2004), pp. 168--177. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Hu, N., Pavlou, P. A., and Zhang, J. Can online reviews reveal a product's true quality? Empirical findings and analytical modeling of online word-of-mouth communication. In Proceedings of the 7th ACM conference on Electronic commerce (EC'06) (2006), pp. 324--330. Google ScholarGoogle ScholarDigital LibraryDigital Library
  14. Kim, S.-M., and Hovy, E. Determining the sentiment of opinions. In Proceedings of the 20th International Conference on Computational Linguistics (COLING 2004) (2004), pp. 1367--1373. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Lee, T. Y., and Bradlow, E. T. Automatic construction of conjoint attributes and levels from online customer reviews. University of Pennsylvania, The Wharton School Working Paper OPIM WP 06-08-01, 2006.Google ScholarGoogle Scholar
  16. Lewitt, S., and Syverson, C. Market distortions when agents are better informed: The value of information in real estate transactions. Working Paper, University of Chicago, 2005.Google ScholarGoogle Scholar
  17. Liu, B., Hu, M., and Cheng, J. Opinion observer: Analyzing and comparing opinions on the Web. In Proceedings of the 14th International World Wide Web Conference (WWW 2005) (2005), pp. 342--351. Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Nigam, K., and Hurst, M. Towards a robust metric of opinion. In AAAI Spring Symposium on Exploring Attitude and Affect in Text (2004), pp. 598--603.Google ScholarGoogle Scholar
  19. Pang, B., and Lee, L. A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts. In Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL 2004) (2004), pp. 271--278. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Pang, B., and Lee, L. Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales. In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics (ACL 2005) (2005). Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Pang, B., Lee, L., and Vaithyanathan, S. Thumbs up? Sentiment classification using machine learning techniques. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP 2002) (2002). Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Snyder, B., and Barzilay, R. Multiple aspect ranking using the good grief algorithm. In Proceedings of the Human Language Technology Conference of the North American Chapter of the Association of Computational Linguistics (HLT-NAACL 2007) (2007).Google ScholarGoogle Scholar
  23. Turney, P. D. Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (ACL 2002) (2002), pp417--424. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Wilson, T., Wiebe, J., and Hwa, R. Recognizing strong and weak opinion clauses. Computational Intelligence 22, 2 (May 2006), 73--99.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Designing novel review ranking systems: predicting the usefulness and impact of reviews

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in
        • Published in

          cover image ACM Other conferences
          ICEC '07: Proceedings of the ninth international conference on Electronic commerce
          August 2007
          482 pages
          ISBN:9781595937001
          DOI:10.1145/1282100

          Copyright © 2007 ACM

          Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 19 August 2007

          Permissions

          Request permissions about this article.

          Request Permissions

          Check for updates

          Qualifiers

          • Article

          Acceptance Rates

          Overall Acceptance Rate150of244submissions,61%

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader