Ashlee Edwards
ABSTRACT
One of the primary ways researchers have characterized engagement during information search is by increases in search behaviors, such as queries and clicks. However, studies have shown that frustration is also characterized by increases in these same behaviors. This research examines the differences in the search behaviors and physiologies of people who are engaged or frustrated during search. A 2x2 within-subject laboratory experiment was conducted with 40 participants. Engagement was induced by manipulating task interest and frustration was induced by manipulating the quality of the search results. Participants' interactions and physiological responses were recorded, and after they searched, they evaluated their levels of engagement, frustration and stress. Participants reported significantly greater levels of engagement when completing tasks that interested them and significantly less engagement during searches with poor results quality. For all search behaviors measured, only two significant differences were found according to task interest: participants had more scrolls and longer query intervals when searching for interesting tasks, suggesting greater interaction with content. Significant differences were found for nine behaviors according to results quality, including queries issued, number of SERPs displayed and number of SERP clicks, suggesting these are potentially better indicators of frustration rather than engagement. When presented with poor quality results, participants had significantly higher heart rates than when presented with normal quality results. Finally, participants had lower heart rates and greater skin conductance responses when conducting interesting tasks than when conducting uninteresting tasks. This research provides insight into the differences in search behaviors and physiologies of participants when they are engaged versus frustrated and presents techniques that can be used by those wishing to induce engagement and frustration during laboratory IIR studies.
- Abram Amsel. 1958. The role of frustrative nonreward in noncontinuous reward situations. Psychological bulletin, 55, 102--119. Google Scholar
Cross Ref
- Ioannis Arapakis, Joemon Jose, and Phillip D. Gray. 2008. Affective feedback: an investigation into the role of emotions in the information seeking process. In Proceedings SIGIR, 395--402. Google ScholarDigital Library
- Ioannis Arapakis, Mounia Lalmas, George Valkanas. 2014. Understanding within-content engagement through pattern analysis of mouse gestures. In Proceedings CIKM, 1439--1448. Google ScholarDigital Library
- Ioannis Arapakis, Ioannis Konstas, and Joemon Jose. 2009. Using facial expressions and peripheral physiological signals as implicit indicators of topical relevance. In Proceedings of ACM Multimedia, 461--470. Google ScholarDigital Library
- Anne Aula, R.M. Khan, Zhiwei Guan. 2010. How does search behavior change as search becomes more difficult? In Proceedings of SIGCHI, 35--44.Google ScholarDigital Library
- Miguel Barreda-#193;ngeles, Ioannis Arapakis, I., Xiao Bai, B. Barla Cambazoglu, and Alexandre Pereda-Baños. 2015. Unconscious Physiological Effects of Search Latency on Users and Their Click Behaviour. In Proceedings of SIGIR, 203--212.Google Scholar
- Wolfram Boucsein. 2012. Electrodermal activity. Springer Science and Business Media. Google ScholarCross Ref
- Jason J. Braithwaite, Derrick G. Watson, Robert Jones, Mickey Rowe. 2013. A Guide for Analysing Electrodermal Activity (EDA) & Skin Conductance Responses (SCRs) for Psychological Experiments. In Psychophysiology, 49, 1017--1034.Google Scholar
- Mihaly Csikszentmihalyi. 2014. Flow. In Flow and the Foundations of Positive Psychology, 227--238.Google Scholar
- Georges Dupret and Mounia Lalmas. 2013. Absence time and user engagement: evaluating ranking functions. In Proceedings of WSDM, 173--182. Google ScholarDigital Library
- Ashlee Edwards and Diane Kelly. How does interest in a work task impact search behavior and engagement? 2016. In CHIIR 2016, 249--252. Google ScholarDigital Library
- Ashlee Edwards, Diane Kelly, and Leif Azzopardi 2015. The impact of query interface design on stress, workload and performance. In Advances in Information Retrieval, 691--702. Google ScholarCross Ref
- Henry A. Field, James Allan, and Rosie Jones. 2010. Predicting searcher frustration. In Proceedings of SIGIR, 34--41. Google ScholarDigital Library
- Joseph F. Grafsgaard, Joseph B. Wiggins, Kristy Elizabeth Boyer, Eric N. Wiebe, and James Lester. 2013. Automatically recognizing facial expression: Predicting engagement and frustration. In Educational Data Mining 2013, 8 pages.Google Scholar
- Ahmed Hassan, Ryen W. White, Susan Dumais, and Yi-Min Wang. 2014. Struggling or exploring? disambiguating long search sessions. In Proceedings of WSDM, 53--62. Google ScholarDigital Library
- William S. Helton. 2004. Validation of a short stress state questionnaire. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 1238--1242. Google ScholarCross Ref
- Diane Kelly, Jaime Arguello, Ashlee Edwards, and Wan-Ching Wu. 2015. Development and evaluation of search tasks for IIR experiments using a cognitive complexity framework. In Proceedings of ICTIR, 101--110. Google ScholarDigital Library
- Richard S. Lazarus and Susan Folkman. 1984. Stress, appraisal, and coping. New York: Springer Pub. Co.Google Scholar
- Janette Lehman, Mounia Lalmas, Ricardo Baeza-Yates, and Elad Yom-Tov. 2013. Networked user engagement. In Proceedings of the 1st Workshop on User Engagement Optimization, 7--10. Google ScholarDigital Library
- Janette Lehmann, Mounia Lalmas, Georges Dupret, and Ricardo Baeza-Yates. 2013. Online multitasking and user engagement. In Proceedings of CIKM, 519--528. Google ScholarDigital Library
- Janette Lehmann, Mounias Lalmas, Elad Yom-Tov, and Georges Dupret. 2012. Models of user engagement. In User Modeling, Adaptation, and Personalization, 164--175. Google ScholarDigital Library
- Yashar Moshfeghi, and Joemon Jose. 2013. An effective implicit relevance feedback technique using affective, physiological and behavioural features. In Proceedings of SIGIR, 133--142. Google ScholarDigital Library
- Lise Solberg Nes, Suzanne C. Segerstrom, and Sandra E. Sephton. 2005. Engagement and arousal: Optimism's effects during a brief stressor. In Personality and Social Psychology Bulletin, 31, 111--120. Google ScholarCross Ref
- Heather L. O'Brien and Maria Lebow. 2013. Mixed-methods approach to measuring user experience in online news interactions. In JASIST, 64, 1543--1556. Google ScholarCross Ref
- Heather L. O'Brien and Elaine G. Toms. 2008. What is user engagement? A conceptual framework for defining user engagement with technology. In JASIST, 59, 938--955.Google ScholarDigital Library
- Heather L. O'Brien and Elaine G. Toms. 2013. Examining the generalizability of the User Engagement Scale (UES) in exploratory search. In IP&M, 49, 1092--1107.Google Scholar
- Adan Ortiz-Cordova and Bernard J. Jansen. 2012. Classifying web search queries to identify high revenue generating customers. In JASIST, 63, 1426--1441. Google ScholarCross Ref
- Lawrence H. Peters and Edward J O'Connor. 1980. Situational constraints and work outcomes: The influences of a frequently overlooked construct. In Academy of Management Review, 5, 391--397.Google ScholarCross Ref
- Jocelyn Scheirer, Raul Fernandez, Jonathan Klein, Rosalind W. Picard. 2002. Frustrating the user on purpose: a step toward building an affective computer. In Interacting with Computers, 14, 93--118. Google ScholarCross Ref
Index Terms
- Engaged or Frustrated?: Disambiguating Emotional State in Search
Recommendations
Engaged or Frustrated?: Disambiguating engagement and frustration in search
One of the primary ways researchers have characterized engagement is by an increase in search actions. Another possibility is that instead of experiencing increased engagement, people who click and query frequently are actually frustrated; several ...
A Study of Snippet Length and Informativeness: Behaviour, Performance and User Experience
SIGIR '17: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information RetrievalThe design and presentation of a Search Engine Results Page (SERP) has been subject to much research. With many contemporary aspects of the SERP now under scrutiny, work still remains in investigating more traditional SERP components, such as the result ...
Exploring the Use of Query Auto Completion: Search Behavior and Query Entry Profiles
CHIIR '16: Proceedings of the 2016 ACM on Conference on Human Information Interaction and RetrievalQuery auto completion (QAC) is nearly ubiquitous in modern search systems, however, there are few published studies on how searchers use QAC query suggestions. This study describes the use of QAC by 29 searchers working on eight assigned search topics ...
Comments