Georg Buscher
Ryen W. White
Susan Dumais
ABSTRACT
Understanding the impact of individual and task differences on search result page examination strategies is important in developing improved search engines. Characterizing these effects using query and click data alone is common but insufficient since they provide an incomplete picture of result examination behavior. Cursor- or gaze-tracking studies reveal richer interaction patterns but are often done in small-scale laboratory settings. In this paper we leverage large-scale rich behavioral log data in a naturalistic setting. We examine queries, clicks, cursor movements, scrolling, and text highlighting for millions of queries on the Bing commercial search engine to better understand the impact of user, task, and user-task interactions on user behavior on search result pages (SERPs). By clustering users based on cursor features, we identify individual, task, and user-task differences in how users examine results which are similar to those observed in small-scale studies. Our findings have implications for developing search support for behaviorally-similar searcher cohorts, modeling search behavior, and designing search systems that leverage implicit feedback.
- Allen, B. (2000). Individual differences and conundrums of user-centered design. JASIS, 51(6): 508--520. Google Scholar
Digital Library
- Arroyo, E., Selker, T., and Wei, W. (2006). Usability tool for analysis of web designs using mouse tracks. Ext. Abstracts CHI, 484--489. Google ScholarDigital Library
- Aula, A., Majaranta, P., and Raiha, K.-J. (2005). Eye-tracking reveals personal styles for search result evaluation. INTER-ACT, 1058--1061. Google ScholarDigital Library
- Bhavnani, S. (2001). Important cognitive components of domain-specific search knowledge. TREC, 571--578.Google Scholar
- Buscher, G., Dumais, S., and Cutrell, E. (2010). The good, the bad and the random: An eye-tracking study of ad quality in Web search. SIGIR, 42--49. Google ScholarDigital Library
- Byström, K. and Järvelin, K. (1995). Task complexity affects information seeking and use. IP&M, 31(2), 191--213. Google ScholarDigital Library
- Chapelle, O. and Zhang, Y. (2009). A dynamic Bayesian click network click model for Web search ranking. WWW, 1--10. Google ScholarDigital Library
- Cole, M., Gwizdka, J., Liu, C., Bierig, R., Belkin, N.J., and Zhang, X. (2011). Task and user effects on reading patterns in information search. Inter. with Comp., 23: 346--362.Google ScholarCross Ref
- Cutrell, E. and Guan, Z. (2007). What are you looking for? An eye-tracking study of information usage in Web search. CHI, 407--416. Google ScholarDigital Library
- Dou, Z., Song, R., and Wen, J. R. (2007). A large-scale evaluation and analysis of personalized search strategies. WWW, 581--590. Google ScholarDigital Library
- Downey, D., Dumais, S., Liebling, D., and Horvitz, E. (2008). Understanding the relationship between searchers' queries and information goals. CIKM, 449--458. Google ScholarDigital Library
- Dumais, S., Buscher, G., and Cutrell, E. (2010). Individual differences in gaze patterns for Web search. IIiX, 185--194. Google ScholarDigital Library
- Ford, N., Miller, D., and Moss, N. (2005). Web search strategies and human individual differences. JASIST, 56(7): 741--756. Google ScholarDigital Library
- Fox, S., Karnawat, K., Mydland, M., Dumais, S., and White, T. (2005). Evaluating implicit measures to improve the search experience. ACM TOIS, 23(2): 147--168. Google ScholarDigital Library
- Guan, Z. and Cutrell, E. (2007). An eye tracking study of the effect of target rank on web search. CHI, 417--420. Google ScholarDigital Library
- Guo, Q. and Agichtein, E. (2008). Exploring mouse movements for inferring query intent. SIGIR, 707--708. Google ScholarDigital Library
- Guo, Q. and Agichtein, E. (2010). Towards predicting Web searcher gaze position from mouse movements. Ext. Abstracts CHI, 3601--3606. Google ScholarDigital Library
- Huang, J., White, R. W., and Dumais, S. T. (2011). No clicks, no problem: Using cursor movements to understand and improve search. CHI, 1225--1234. Google ScholarDigital Library
- Joachims, T., Granka, L., Pan, B., Hembrooke, H., and Gay, G. (2005). Accurately interpreting clickthrough data as implicit feedback. SIGIR, 154--161. Google ScholarDigital Library
- Karypis, G. Cluto: A clustering toolkit. www.cs.umn.edu/~cluto. Retrieved July 2011.Google Scholar
- Kellar, M., Watters, C., and Shepherd, M. (2007). A field study characterizing Web-based information-seeking tasks. JASIST, 58(7): 999--1018. Google ScholarDigital Library
- Kohavi, R., Longbotham, R., Sommerfield, D., and Henne, R. M. (2009). Controlled experiments on the Web: Survey and practical guide. Data Min. Know. Disc., 18(1): 140--181. Google ScholarDigital Library
- Lagun, D. and Agichtein, E. (2011). ViewSer: Enabling large-scale remote user studies of Web search examination and interaction. SIGIR, 365--374. Google ScholarDigital Library
- Li, J., Huffman, S.B., and Tokuda, A. (2009). Good aban-donment in mobile and PC internet search. SIGIR, 43--50. Google ScholarDigital Library
- Liu, C. and Chung, C. (2007). Detecting mouse movement with repeated visit patterns for retrieving noticed knowledge components on web pages. IEICE Trans. Inform. & Syst., E90-D(10): 1687--1696. Google ScholarDigital Library
- Liu., J., Cole., M. J., Liu, C., Bierig, R., Gwizdka, J., Belkin, N. J., Zhang, J., and Zhang, X. (2010). Search behaviors in different task types. JCDL, 69--78. Google ScholarDigital Library
- Lorigo, L., Pan, B., Hembrooke, H., Joachims, T., Granka, L., and Gay, G. (2006). The influence of task and gender on search evaluation and behavior using Google. IP&M, 42(4): 1123--1131. Google ScholarDigital Library
- Rodden, K. and Fu, X. (2007). Exploring how mouse movements relate to eye movements on web search results pages. SIGIR Workshop on Web Info. Seek. and Interact., 29--32.Google Scholar
- Rodden, K., Fu, X., Aula, A., and Spiro, I. (2008). Eye-mouse coordination patterns on web search results pages. Ext. Abstracts CHI, 2997--3002. Google ScholarDigital Library
- Saracevic, T. (1991). Individual differences in organizing, searching and retrieving information. ASIS, 82--86.Google Scholar
- Teevan, J., Dumais, S. T., and Liebling, D. (2008). To personalize or not to personalize: Modeling queries with variation in user intent. SIGIR, 163--170. Google ScholarDigital Library
- Teevan, J., Liebling, D. J., and Geetha, G. R. (2011). Understanding and predicting personal navigation. WSDM, 85--94. Google ScholarDigital Library
- Thatcher, A. (2008). Web search strategies: The influence of Web experience and task type. IP&M, 44(3): 1308--1329. Google ScholarDigital Library
- White, R. and Morris, D. (2007). Investigating the querying and browsing behavior of advanced search engine users. SIGIR, 255--262. Google ScholarDigital Library
- White, R. and Drucker, S. (2007). Investigating behavioral variability in Web search. WWW, 21--30. Google ScholarDigital Library
- White, R., Dumais, S. T., and Teevan, J. (2009). Characterizing the influence of domains expertise on Web search behavior. WSDM, 132--141. Google ScholarDigital Library
Index Terms
- Large-scale analysis of individual and task differences in search result page examination strategies
Recommendations
Re-examining search result snippet examination time for relevance estimation
SIGIR '12: Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrievalPrevious studies of web search result examination have provided valuable insights in understanding and modelling searcher behavior. Yet, recent work (e.g., [3]) has been developed based on the assumption that the time a searcher spends examining a ...
Supporting orientation during search result examination
CHI '13: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsSearch engines help their users decide which results to visit using captions comprising titles, URLs, and snippets containing the query keywords and proximal text from landing pages (the search results linked from the result page). Although caption ...
Influence of Vertical Result in Web Search Examination
SIGIR '15: Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information RetrievalResearch in how users examine results on search engine result pages (SERPs) helps improve result ranking, advertisement placement, performance evaluation and search UI design. Although examination behavior on organic search results (also known as "ten ...
Comments