Gamification through leaderboards: An empirical study in engineering education

Computer Applications in Engineering Education

Volumn 27, Issue 4, 2019, Pages 777-788

  Ortiz Rojas, Margarita ^a,b   Chiluiza, Katherine ^b   Valcke, Martin ^a  

^a GHENT UNIVERSITY   (Belgium)

^b ESCUELA SUPERIOR POLITÉCNICA DEL LITORAL   (Ecuador)

Author keywords

gamification; intrinsic motivation (IM); leaderboards; learning performance (LP); STEM

Indexed keywords

COMPUTER PROGRAMMING; CURRICULA; MOTIVATION;

EMPIRICAL STUDIES; GAMIFICATION; GAMING EXPERIENCES; INTERACTION EFFECT; INTRINSIC MOTIVATION; INTRODUCTORY COMPUTER PROGRAMMING; LEADERBOARDS; LEARNING PERFORMANCE;

STEM (SCIENCE, TECHNOLOGY, ENGINEERING AND MATHEMATICS);

EID: 85068970778     PISSN: 10613773     EISSN: 10990542     Source Type: Journal    
DOI: 10.1002/cae.12116     Document Type: Article

References (75)

1. Morse A. Delivering STEM (Science, Technology, Engineering and Mathematics) Skills for the Economy, 2018. https://www.nao.org.uk/wp-content/uploads/2018/01/Delivering-STEM-Science-technology-engineering-and-mathematics-skills-for-the-economy.pdf
2. European Commission. Horizon 2020 Work Programme 2014 – 2015. Science with and for Society Revised. https://ec.europa.eu/research/participants/data/ref/h2020/wp/2014_2015/main/h2020-wp1415-swfs_en.pdf Published 2015. Accessed December 4, 2018.
3. Chen X STEM Attrition: College Students’ Paths Into and Out of STEM Fields Statistical Analysis Report. 2013. https://nces.ed.gov/pubs2014/2014001rev.pdf
4. Marginson S, Tytler R, Freeman B, Roberts K STEM: Country Comparisons. 2013. https://acola.org.au/wp/PDF/SAF02Con-sultants/SAF02_STEM_FINAL.pdf. Accessed May 15, 2018.
5. OECD. PISA 2015 Results (Volume I). 2016. https://doi.org/.org/10.1787/9789264266490-en
6. Huyer S Is the Gender Gap Narrowing in Science and Engineering? Paris; 2015. http://unesdoc.unesco.org/Ulis/cgi-bin/ulis.pl?catno=235447&set=005AC3693D_1_28&gp=1&lin=1&ll=1
7. Aluwihare PSD, Manoshika R. Addressing the issues of low student enrollment — the case of the Kandy Regional Centre of the Open University of Sri Lanka. Asian Assoc Open Univ J. 2013;8(1):103-115. https://doi.org/10.1108/AAOUJ-08-01-2013-B010
8. Paura L, Arhipova I Student dropout rate in engineering education study program. In: Proceedings of 15th International Scientific Conference Engineering for Rural Development; 2016:1-6. http://www.tf.llu.lv/conference/proceedings2016/Papers/N118.pdf
9. National Research Council. In: Kober N, ed. Reaching Students: What Research Says About Effective Instruction in Undergraduate Science and Engineering. Washington, DC: The National Academies Press; 2015. https://doi.org/10.17226/18687
10. de Sousa Borges S, Durelli VHS, Reis HM, Isotani S. A Systematic Mapping on Gamification Applied to Education. SAC ’14 Proceedings of the 29th Annual ACM Symposium on Applied Computing. New York, NY: ACM; 2014:pp.216-222. https://doi.org/10.1145/2554850.2554956
11. Landers RN, Bauer KN, Callan RC, Armstrong MB. Psychological Theory and the Gamification of Learning. In: Reiners T, Wood LC, eds. Gamification in Education and Business. Cham: Springer International Publishing; 2015:pp. 165-186. https://doi.org/10.1007/978-3-319-10208-5_9
12. Ortiz-Rojas, M., Chiluiza, K., Valcke, M. Gamification and Learning Performance: A Systematic Review of the Literature. European Conference on Games Based Learning. Graz, Austria; 2017.
13. Ortiz, M., Chiluiza, K., Valcke, M. Gamification in Higher Education and STEM: A Systematic Review of Literature. 8th Annual International Conference on Education and New Learning Technologies - Edulearn16. Barcelona, Spain; 2016: pp. 6548-6558. DOI: 10.21125/edulearn.2016.0422
14. Deterding S., Dixon, D, Khaled R, Nacke L From Game Design Elements to Gamefulness: Defining “Gamification.” Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments. Tampere, Finland; 2011: pp. 9-15. DOI: 10.1145/2181037.2181040
15. Werbach K, Hunter D. For the Win: How Game Thinking Can Revolutionize Your Business. Philadelphia, PA: Wharton Digital Press; 2012.
16. Landers R. Developing a theory of gamified learning: linking serious games and gamification of learning. Simul Gaming. 2014;45(6):752-768. https://doi.org/10.1177/1046878114563660
17. Landers R, Landers A. An empirical test of the theory of gamified learning: the effect of leaderboards on time-on-task and academic performance. Simul Gaming. 2014;45(6):769-785. https://doi.org/10.1177/1046878114563662
18. Denny P, McDonald F, Empson R, Kelly P, Petersen A. Empirical support for a causal relationship between gamification and learning outcomes. CHI '18 Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. New York, NY: ACM; 2018. https://doi.org/10.1145/3173574.3173885
19. Christy KR, Fox J. Leaderboards in a virtual classroom: a test of stereotype threat and social comparison explanations for women's math performance. Comput Educ. 2014;78:66-77. https://doi.org/10.1016/j.compedu.2014.05.005
20. Butler C. The Effect of Leaderboard Ranking on Players’ Perception of Gaming Fun. Berlin, Heidelberg: Springer; 2013:pp. 129-136. In:. https://doi.org/10.1007/978-3-642-39371-6_15
21. Marczewski A Even Ninja Monkeys like to Play: Gamification, Game Thinking & Motivational Design. Gamified, UK; 2015. https://books.google.com.ec/books/about/Even_Ninja_Monkeys_Like_to_Play.html?id=RoHpjgEACAAJ&redir_esc=y. Accessed May 15, 2018.
22. Zichermann G, Cunningham C. Gamification by Design: Implementing Game Mechanics in Web and Mobile Apps. Sebastopol, CA, USA: O'Reilly Media; 2011.
23. Costa JP, Wehbe RR, Robb J, Nacke LE. Time's up. Proceedings of the First International Conference on Gameful Design, Research, and Applications - Gamification ’13. New York, NY: ACM Press; 2013:pp. 26-33. https://doi.org/10.1145/2583008.2583012
24. Landers RN, Bauer KN, Callan RC. Gamification of task performance with leaderboards: a goal setting experiment. Comput Human Behav. 2017;71:508-515. https://doi.org/10.1016/J.CHB.2015.08.008
25. Bowey JT, Birk MV, Mandryk RL. Manipulating Leaderboards to Induce Player Experience. CHI PLAY ’15 Proceedings of the 2015 Annual Symposium on Computer-Human Interaction in Play. New York, NY, USA: ACM; 2015:pp. 115-120. https://doi.org/10.1145/2793107.2793138
26. Landers RN, Landers AK. An empirical test of the theory of gamified learning. Simul Gaming. 2014;45(6):769-785. https://doi.org/10.1177/1046878114563662
27. Seaborn K, Pennefather P, Fels DI. Reimagining leaderboards: towards gamifying competency models through social game mechanics. Proc First Int Conf Gameful Des Res Appl. 2013:107-110. https://doi.org/10.1145/2583008.2583026
28. Chou Y-K. Actionable Gamification: Beyond Points, Badges and Leaderboards. Octalysis Media. Fremont, CA: Octalysis Media; 2015.
29. Niemiec CP, Ryan RM. Autonomy, competence, and relatedness in the classroom: applying self-determination theory to educational practice. Theory Res Educ. 2009;7(2):133-144. https://doi.org/10.1177/1477878509104318
30. Ryan R, Deci E. Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. Am Psychol. 2000;55(1):68-78. https://doi.org/10.1037/0003-066X.55.1.68
31. Shi L, Cristea AI. Motivational Gamification Strategies Rooted in Self-Determination Theory for Social Adaptive E-Learning. In: Micarelli A, Stamper J, Panourgia K, eds. Intelligent Tutoring Systems. Cham: Springer International Publishing; 2016:pp. 294-300.
32. Ryan R, Deci E. Self-Determination Theory An Introduction and Overview. Self-Determination Theory Basic Psychological Needs in Motivation, Development, and Wellness. New York: Guilford Publications; 2017:pp. 3-25.
33. Aparicio AF, Vela FLG, Sánchez JLG, Montes JLI. Analysis and Application of Gamification. Proceedings of the 13th International Conference on InteracciÓN Persona-Ordenador. INTERACCION ’12. New York, NY, USA: ACM; 2012. https://doi.org/10.1145/2379636.2379653
34. Sailer M, Hense JU, Mayr SK, Mandl H. How gamification motivates: an experimental study of the effects of specific game design elements on psychological need satisfaction. Comput Human Behav. 2017;69:371-380. https://doi.org/10.1016/j.chb.2016.12.033
35. Sailer M, Hense J, Mandl H, Klevers M. Psychological perspectives on motivation through gamification. Interact Des Archit J - IxD&A. 2013;19:28-37. https://forum.poppy-project.org/uploads/default/original/2X/4/41fd4c996c7427bdb0994d948b599fd75217aab5.pdf
36. Deterding S Situated motivational affordances of game elements: A conceptual model. In: CHI Gamification Workshop 2011; 2011:pp. 3-6.
37. Reeve J, Deci EL. Elements of the competitive situation that affect intrinsic motivation. Personal Soc Psychol Bull. 1996;22(1):24-33. https://doi.org/10.1177/0146167296221003
38. Deterding S. Eudaimonic Design, or: Six Invitations to Rethink Gamification. In: Fuchs M, Fizek S, Ruffino P, Schrape N, eds. Rethinking Gamification. Germany: Meson Press; 2014:pp. pp. 305-331.
39. Bandura A. Self-efficacy: toward a unifying theory of behavioral change. Adv Behav Res Ther. 1978;1(4):139-161. https://doi.org/10.1016/0146-6402(78)90002-4
40. Bandura A. Self-efficacy mechanism in human agency. Am Psychol. 1982;37(2):122-147. https://doi.org/10.1037/0003-066X.37.2.122
41. Kapp KM. The Gamification of Learning and Instruction: Game-Based Methods and Strategies for Training and Education. San Francisco, CA: Pfeiffer & Company; 2012.
42. Gunuc S, Kuzu A. Student engagement scale: development, reliability and validity. Assess Eval High Educ. 2015;40(4):587-610. https://doi.org/10.1080/02602938.2014.938019
43. Ibanez M-B, Di-Serio A, Delgado-Kloos C. Gamification for engaging computer science students in learning activities: a case study. IEEE Trans Learn Technol. 2014;7(3):291-301. https://doi.org/10.1109/TLT.2014.2329293
44. Looyestyn J, Kernot J, Boshoff K, Ryan J, Edney S, Maher C. Does gamification increase engagement with online programs? A systematic review. PLoS One. 2017;12(3):e0173403. https://doi.org/10.1371/journal.pone.0173403
45. Pedro LZ, Lopes AMZ, Prates BG, Vassileva J, Isotani S. Does gamification work for boys and girls? Proceedings of the 30th Annual ACM Symposium on Applied Computing - SAC ’15. New York, NY, USA: ACM Press; 2015:pp. 214-219. https://doi.org/10.1145/2695664.2695752
46. Landers RN, Armstrong MB. Enhancing instructional outcomes with gamification: an empirical test of the Technology-enhanced Training Effectiveness Model. Comput Human Behav. 2017;71:499-507. https://doi.org/10.1016/J.CHB.2015.07.031
47. Amoako PYO, Sarpong KA, Arthur JK, Adjetey C. Article: performance of students in computer programming: background, field of study and learning approach paradigm. Int J Comput Appl. 2013;77(12):17-21. https://www.ijcaonline.org/archives/volume77/number12/13446-1221
48. Pioro BT. Introductory computer programming: gender, major, discrete mathematics, and calculus. J Comput Sci Coll. 2006;21(5):123-129. http://dl.acm.org/citation.cfm?id=1127351.1127373
49. Gorard S. Research Design: Creating Robust Approaches for the Social Sciences. 55 City Road, London: SAGE; 2013.
50. Yang FM, Kao ST. Item response theory for measurement validity. Shanghai Arch Psychiatry. 2014;26(3):171-177. https://doi.org/10.3969/j.issn.1002-0829.2014.03.010
51. Ryan RM, Mims V, Koestner R. Relation of reward contingency and interpersonal context to intrinsic motivation: a review and test using cognitive evaluation theory. J Pers Soc Psychol. 1983;45(4):736-750. https://doi.org/10.1037/0022-3514.45.4.736
52. Ryan R, Deci R selfdeterminationtheory.org – Intrinsic Motivation Inventory (IMI).
53. McAuley E, Duncan T, Tammen VV. Psychometric properties of the Intrinsic Motivation Inventory in a competitive sport setting: a confirmatory factor analysis. Res Q Exerc Sport. 1989;60(1):48-58. https://doi.org/10.1080/02701367.1989.10607413
54. Bandura A Guide for Constructing Self-efficacy Scales. Self-Efficacy Beliefs of Adolescents.; 2006: pp. 307-337. https://www.uky.edu/~eushe2/Bandura/BanduraGuide2006.pdf
55. Pólya G. How to Solve It: A New Aspect of Mathematical Method. New York: Princeton University Press; 2014.
56. Pennington N, Grabowski B. The Tasks of Programming. Psychology of Programming. London: Academic Press; 1990:pp. 45-62.
57. Baguley T. Standardized or simple effect size: what should be reported? Br J Psychol. 2009;100(3):603-617. https://doi.org/10.1348/000712608X377117
58. Byrne BM. Structural Equation Modeling with EQS and EQS/Windows: Basic Concepts, Applications, and Programming. Thousand Oaks, CA: Sage Publications; 1994.
59. MacCallum RC, Browne MW, Sugawara HM. Power analysis and determination of sample size for covariance structure modeling. Psychol Methods. 1996;1(2):130-149. https://doi.org/10.1037/1082-989X.1.2.130
60. Hu L, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct Equ Model A Multidiscip J. 1999;6(1):1-55. https://doi.org/10.1080/10705519909540118
61. Hanus MD, Fox J. Assessing the effects of gamification in the classroom: a longitudinal study on intrinsic motivation, social comparison, satisfaction, effort, and academic performance. Comput Educ. 2014;80:152-161. https://doi.org/10.1016/j.compedu.2014.08.019
62. Jurgelaitis M, Čeponienė L, Čeponis J, Drungilas V. Implementing gamification in a university-level UML modeling course: a case study. Comput Appl Eng Educ. 2018;0(0):332-343. https://doi.org/10.1002/cae.22077
63. Fernández-Ballesteros R. Self-report questionnaires. Comprehensive Handbook of Psychological Assessment, Behavioral Assessment. vol.3. Hoboken, NJ, USA: John Wiley & Sons Inc; 2004:pp. 194-221.
64. Chan Mow ITV. Investigative Study on NUS Computer programming Students’ Perceptions of their computer programming experience: some preliminary findings. Int J Inf Syst Comput Sci. 2012;1(1):12-21. https://pdfs.semanticscholar.org/1fb4/5955510c33faed997a695dbee31d8334f4f6.pdf
65. Chilana PK, Alcock C, Dembla S, et al. Perceptions of Non-CS Majors in Intro Programming: The Rise of the Conversational Programmer. 2015 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC). Atlanta, GA, USA: IEEE; 2015:251-259. https://doi.org/10.1109/VLHCC.2015.7357224
66. Teague D. Pedagogy of introductory computer programming: a people-first approach. 2011:1-150. Queensland University of Technology (Master thesis)
67. Guay F, Ratelle CF, Roy A, Litalien D. Academic self-concept, autonomous academic motivation, and academic achievement: mediating and additive effects. Learn Individ Differ. 2010;20(6):644-653. https://doi.org/10.1016/J.LINDIF.2010.08.001
68. Mekler ED, Brühlmann F, Tuch AN, Opwis K. Towards understanding the effects of individual gamification elements on intrinsic motivation and performance. Comput Human Behav. 2017;71:525-534. https://doi.org/10.1016/j.chb.2015.08.048
69. Szeto A, Haines J, Buchholz AC. Impact of an optional experiential learning opportunity on student engagement and performance in undergraduate nutrition courses. Can J Diet Pract Res a Publ Dietitians Canada=Rev Can la Prat la Rech en Diet une Publ des Diet du Canada. 2016;77(2):84-88. https://doi.org/10.3148/cjdpr-2015-038
70. Verkade H, Lim SH. Uptake of optional activities leads to improved performance in a biomedical sciences class. J Scholarsh Teach Learn. 2015;15(6):48-62. https://doi.org/10.14434/josotl.v15i6.18988
71. Ortiz-Rojas M, Chiluiza K, Valcke M Gamification in Computer Programming: Effects on learning, engagement, self-efficacy and intrinsic motivation. European Conference on Games Based Learning; 2017.
72. Auvinen T, Hakulinen L, Malmi L. Increasing students’ awareness of their behavior in online learning environments with visualizations and achievement badges. IEEE Trans Learn Technol. 2015;8(3):261-273. https://doi.org/10.1109/TLT.2015.2441718
73. Barata G, Gama S, Jorge J, Goncalves D. Engaging Engineering Students with Gamification. 2013 5th International Conference on Games and Virtual Worlds for Serious Applications (VS-GAMES). Poole, Uk: IEEE; 2013:pp. 1-8. https://doi.org/10.1109/VS-GAMES.2013.6624228
74. Labrador Ruiz de la Hermosa E, Villegas E. Unir Gamificación y Experiencia de Usuario para mejorar la experiencia docente. RIED Rev Iberoam Educ a distancia. 2016;19(2):125-142.
75. Mandernach BJ. Assessment of student engagement in higher education: a synthesis of literature and assessment tools. Int J Learn Teach Educ Res. 2015;12(2):1-14. https://www.ijlter.org/index.php/ijlter/article/view/367/167