Does gamification improve student learning outcome? Evidence from a meta-analysis and synthesis of qualitative data in educational contexts

https://doi.org/10.1016/j.edurev.2020.100322Get rights and content

Highlights

  • Gamification can increase student learning performance (effect size of 0.504).

  • Shorter gamified interventions have greater average effect sizes.

  • Participants report four main reasons why they enjoy gamification.

  • Participants report two main reasons why they dislike gamification.

  • Two main unresolved questions in gamification research are highlighted.

Abstract

Despite the buzz around gamification as an exciting new method to engage students, evidence of its ability to enhance learning is mixed. In fact, gamification has attracted considerable controversy (“gamification is bullshit”) and some derogatory labels such as “exploitationware.” Therefore, in order to make the case for or against gamification in education, it is important to examine the effects (if any) of gamification on student learning achievements. This study is a meta-analysis of 30 independent interventions (3,202 participants) drawn from 24 quantitative studies that have examined the effects of gamification on student academic performance in various educational settings. The results show an overall significant medium effect size in favor of gamification over learning without gamification (Hedges' g = 0.504, 95% CI [0.284–0.723], p < 0.001). No publication bias is detected. An analysis of 32 qualitative studies reveals four reasons for learners' enjoyment of gamification: (a) gamification can foster enthusiasm; (b) gamification can provide feedback on performance; (c) gamification can fulfill learners’ needs for recognition; and (d) gamification can promote goal setting, and two reasons for their dislike of gamification: (a) gamification does not bring additional utility and (b) gamification can cause anxiety or jealousy. We conclude by highlighting two unresolved questions, and suggesting several future research directions concerning gamification in educational contexts.

Section snippets

Background

In recent years, there has been a considerable amount of buzz about gamification across various academic disciplines and industries. It has been proposed that gamified practices will become a key element in motivating people, particularly those born in the digital age of online games, to engage in certain tasks such as improving their fitness (Nike+), generating brand loyalty (Starbucks), or answering other people's questions (e.g., Yahoo! Answers). The potential of gamification, however,

Prior reviews

This section briefly reviews some of the relevant literature on gamification (Borges, Durelli, Reis, & Isotani, 2014; Dichev & Dicheva, 2017; Falkner & Falkner, 2014; Hamari, Koivisto, & Sarsa, 2014; Looyestyn et al., 2017) and discusses its limitations. None of the previous reviews have examined the effects of gamification on learner academic performance. Several have reviewed studies that examined a mixture of gamification and games-based learning implementations or studies conducted in a

Databases searched

The process for selecting relevant studies was guided by the preferred-reporting of items for systematic reviews and meta-analyses (PRISMA) statement (Moher, Liberati, Tetzlaff, & Altman, 2009). Leading academic databases were searched, as they are more likely to yield relevant and high quality papers (Webster & Watson, 2002). The recommended databases are ACM Digital Library, EBSCO host research databases (including Academic Search Premier, British Education Index, ERIC, TOC Premier), Emerald

Data extraction

The article selection process identified 24 articles (Aşıksoy, 2018; Cadavid and Corcho, 2018, de-Marcos et al., 2014, 2017; Gafni et al., 2018; Haruna et al., 2018; Hew et al., 2016; Homer et al., 2016; Huang & Hew, 2018; Huang et al., 2018; Jong et al., 2018; Ketyi, 2016; Lam et al., 2018; Marín et al., 2018; Ozdener, 2018; Rachels et al., 2018; Su & Cheng, 2015; Sun-Lin & Chiou, 2017; Tan & Hew, 2016; Tsay et al., 2018; Turan et al., 2016; Van Nuland et al., 2015; Yildirim, 2017; Zainuddin,

General characteristics of the interventions

Nine of the interventions were conducted with elementary school students (e.g., Rachels & Rockinson-Szapkiw, 2018), and five interventions were conducted at the high school level (e.g., Jong, Chan, Hue, & Tam, 2018). One involved a mixture of high school, undergraduate, and postgraduate students (Gafni, Achituv, Eidelman, & Chatsky, 2018). Ten were carried out with undergraduate students (e.g., Marín et al., 2018). Three studies were conducted with postgraduate students (e.g., Tan & Hew, 2016),

Discussion

Many pundits have promoted the potential of gamification for motivating students to perform desired learning tasks. Yet despite the popularity of gamification, there is little consensus on whether it contributes to improved academic performance. Previous studies have reported mixed findings. This review provides a useful overview of quantitative studies on the use of gamification in educational settings. More specifically, we conduct a meta-analysis of studies of gamification and its impact on

Conclusion

Despite the aforementioned limitations, this study contributes to the literature in three ways. First, it augments our understanding of gamification in educational contexts. Put another way, it goes beyond the unqualified assertion that gamification is beneficial by quantifying the effect of gamification on learners' academic performance. The second contribution is the identification of factors that may moderate the effect sizes of gamification. For example, this study provides empirical

References (158)

  • R. Fu et al.

    Conducting quantitative synthesis when comparing medical interventions: AHRQ and the effective health care program

    Journal of Clinical Epidemiology

    (2011)
  • K.F. Hew et al.

    Engaging Asian students through game mechanics: Findings from two experiment studies

    Computers & Education

    (2016)
  • B. Huang et al.

    Implementing a theory-driven gamification model in higher education flipped courses: Effects on out-of-class activity completion and quality of artifacts

    Computers & Education

    (2018)
  • T. Jagušt et al.

    Examining competitive, collaborative and adaptive gamification in young learners' math learning

    Computers & Education

    (2018)
  • E.A. Locke

    Motivation through conscious goal setting

    Applied and Preventive Psychology

    (1996)
  • D.G. Altman et al.

    Statistics notes: Standard deviations and standard errors

    BMJ British Medical Journal

    (2005)
  • L.A. Annetta

    The “I's” have it: A framework for serious educational game design

    Review of General Psychology

    (2010)
  • S. Arkün Kocadere et al.

    Gamification from player type perspective: A case study

    (2018)
  • G. Aşıksoy

    The effects of the gamified flipped classroom environment (GFCE) on students' motivation, learning achievements and perception in a physics course

    Quality and Quantity

    (2018)
  • C.H. Aydin

    MOOCs as change agents

  • M. Baldwin et al.

    The culture of social comparison

    Proceedings of the National Academy of Sciences of the United States of America

    (2018)
  • A. Bandura

    Self-efficacy mechanism in human agency

    American Psychologist

    (1982)
  • F.C. Blumberg et al.

    Serious games: What are they? What do they do? Why should we play them?

  • I. Bogost

    Gamification is bullshit

    (2011)
  • M. Borenstein

    Software for publication bias

  • M. Borenstein et al.

    Introduction to meta-analysis

    (2009)
  • S.d.S. Borges et al.

    A systematic mapping on gamification applied to education

  • O. Borras-Gene et al.

    New challenges for the motivation and learning in engineering education using gamification in MOOC

    International Journal of Engineering Education

    (2016)
  • V. Braun et al.

    Using thematic analysis in psychology

    Qualitative Research in Psychology

    (2006)
  • P. Buckley et al.

    Game on! Students' perceptions of gamified learning

    Educational Technology & Society

    (2017)
  • C. Butler

    The effect of leaderboard ranking on players' perception of gaming fun

  • J.n.M. Cadavid et al.

    Competitive programming and gamification as strategy to engage students in computer science courses

    Revista ESPACIOS

    (2018)
  • I. Caponetto et al.

    Gamification and education: A literature review

  • K.P. Carson et al.

    The usefulness of the "fail-safe" statistic in meta-analysis

    Educational and Psychological Measurement

    (1990)
  • C.C. Cavalcanti et al.

    Gamification design for tutor education in an online course

    ETD: Educação Temática Digital

    (2018)
  • J. Chen et al.

    The role of collaboration, computer use, learning environments, and supporting strategies in CSCL: A meta-analysis

    (2018)
  • C. Cheong et al.

    Towards the gamification of learning: Investigating student perceptions of game elements

    Journal of Information Systems Education

    (2014)
  • K.F. Chiu et al.

    Factors influencing peer learning and performance in MOOC asynchronous online discussion forum

    Australasian Journal of Educational Technology

    (2018)
  • R. Coe

    It's the effect size, stupid: What effect size is and why it is important

  • J. Cohen
    (1988)
  • D. Cook et al.

    Method and reporting quality in health professions education research: A systematic review

    Medical Education

    (2011)
  • K. Corti

    Games-based learning: A serious business application

    (2006)
  • M. Csikszentmihalyi

    Flow: The psychology of optimal experience

    (1990)
  • L. de-Marcos et al.

    Towards the social gamification of e-learning: A practical experiment

    International Journal of Engineering Education

    (2017)
  • E. Deci et al.
    (1985)
  • L. Deslauriers et al.

    Measuring actual learning versus feeling of learning in response to being actively engaged in the classroom

    Proceedings of the National Academy of Sciences of the United States of America

    (2019)
  • S. Deterding et al.

    From game design elements to gamefulness: Defining "gamification"

  • Cited by (257)

    • Humans-in-the-loop: Gamifying activity label repair in process event logs

      2024, Engineering Applications of Artificial Intelligence
    View all citing articles on Scopus
    View full text