A trace-based approach to identifying users’ engagement and qualifying their engaged-behaviours in interactive systems: application to a social game

User Modeling and User-Adapted Interaction

Volumn 24, Issue 5, 2014, Pages 413-451

  Bouvier, Patrice ^a   Sehaba, Karim ^b   Lavoué, Élise ^c  

^a UNIVERSITÉ LYON 1   (France)

^b UNIVERSITÉ LUMIÈRE LYON 2   (France)

^c UNIVERSITÉ JEAN MOULIN LYON 3   (France)

Author keywords

Activity Theory; Engaged behaviours; Engagement assessment; Interaction traces; Qualitative approach; Self Determination Theory; Social game; User behaviour analysis

Indexed keywords

ACTIVITY COEFFICIENTS;

ACTIVITY THEORY; ENGAGED BEHAVIOURS; ENGAGEMENT ASSESSMENT; INTERACTION TRACES; QUALITATIVE APPROACH; SELF-DETERMINATION THEORIES; SOCIAL GAMES; USER BEHAVIOUR;

BEHAVIORAL RESEARCH;

EID: 84910139222     PISSN: 09241868     EISSN: 15731391     Source Type: Journal    
DOI: 10.1007/s11257-014-9150-2     Document Type: Article

References (75)

1. Andersen, E., O’Rourke, E., Liu, Y.E., Snider, R., Lowdermilk, J., Truong, D., Cooper, S., Popovic, Z.: The impact of tutorials on games of varying complexity. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 59–68. ACM, Austin, Texas, USA, CHI 2012 (2012)
2. Arroyo, I., Cooper, D.G., Burleson, W., Woolf, B.P., Muldner, K., Christopherson, R.: Emotion sensors go to school. In: Proceedings of the 2009 Conference on Artificial Intelligence in Education: Building Learning Systems that Care: From Knowledge Representation to Affective Modelling, pp. 17–24. IOS Press, Brighton, UK (2009)
3. Attfield, S., Kazai, G., Lalmas, M., Piwowarski, B.: Towards a science of user engagement (position paper). In: WSDM Workshop on User Modelling for Web Applications, Hong Kong, China (2011)
4. Barak, A., Hen, L., Boniel-Nissim, M., Shapira, N.: A comprehensive review and a meta-analysis of the effectiveness of internet-based psychotherapeutic interventions. J. Technol. Human Serv. 26(2–4), 109–160 (2008)
5. Baranowski, T., Baranowski, J., Cullen, K.W., Marsh, T., Islam, N., Zakeri, I., Honess-Morreale, L., deMoor, C.: Squire’s quest! Dietary outcome evaluation of a multimedia game. Am. J. Prev. Med. 24(1), 52–61 (2003)
6. Baranowski, T., Buday, R., Thompson, D.I., Baranowski, J.: Playing for real: video games and stories for health-related behavior change. Am. J. Prev. Med. 34(1), 74–82 (2008)
7. Bardram, J.,E.: Plans as situated action: an activity theory approach to workflow systems. In: Proceedings of the Fifth Conference on European Conference on Computer-Supported Cooperative Work, ECSCW’97, pp. 17–32. Kluwer, Lancaster, UK (1997)
8. Bauckhage, C., Kersting, K., Sifa, R., Thurau, C., Drachen, A., Canossa, A.: How players lose interest in playing a game: an empirical study based on distributions of total playing times. In: Lucas, S., Cho, S.-B., M.S.E. (eds.) Proceedings of the IEEE Conference on Computational Intelligence and Games, CIG 2012, pp. 139–146. Granada, Spain (2012)
9. Beal, C.R., Qu, L., Lee, H.: Classifying learner engagement through integration of multiple data sources. In: Proceedings of the 21st National Conference on Artificial Intelligence—Volume 1, AAAI’06. Boston, Massachusetts, pp. 151–156 (2006)
10. Bianchi-Berthouze, N.: Understanding the role of body movement in player engagement. Human Comput. Interact. 28(1), 40–75 (2013)
11. Bouvier, P., Lavoué, E., Sehaba, K., George, S.: Identifying learner’s engagement in learning games—a qualitative approach based on learner’s traces of interaction. In: Foley, O., Restivo, M.T., Uhomoibhi, J.O., Helfert, M. (eds.) Proceedings of the 5th International Conference on Computer Supported Education, CSEDU 2013, pp. 339–350. SciTePress, Aachen, Germany (2013a)
12. Bouvier, P., Sehaba, K., Lavoué, E., George, S.: Using traces to qualify learner’s engagement in game-based learning. In: Proceedings of The 13th IEEE International Conference on Advanced Learning Technologies, ICALT 2013, pp. 432–436. IEEE Computer Society, Beijing, China (2013b)
13. Boyle, E.A., Connolly, T.M., Hainey, T., Boyle, J.M.: Engagement in digital entertainment games: a systematic review. Comput. Human Behav. 28(3), 771–780 (2012)
14. Brockmyer, J.H., Fox, C.M., Curtiss, K.A., McBroom, E., Burkhart, K.M., Pidruzny, J.N.: The development of the game engagement questionnaire: a measure of engagement in video game-playing. J. Exp. Soc. Psychol. 45(4), 624–634 (2009)
15. Brown, E., Cairns, P.:A grounded investigation of game immersion. In: CHI ’04 Extended Abstracts on Human Factors in Computing Systems, CHI EA 2004, pp. 1297–1300. Vienna, Austria (2004)
16. Buder, J., Bodemer, D.:Group awareness tools for controversial cscl discussions: dissociating rating effects and visualized feedback effects. In: 9th International Conference on Computer Supported Collaborative Learning, CSCL 2011, pp. 358–365, Hong Kong (2011)
17. Calvo, R., D’Mello, S.: Affect detection: an interdisciplinary review of models, methods, and their applications. IEEE Transactions on Affective Computing 1(1), 18–37 (2010)
18. Camerini, L., Giacobazzi, M., Boneschi, M., Schulz, P.J., Rubinelli, S.: Design and implementation of a web-based tailored gymnasium to enhance self-management of fibromyalgia. User Model User-Adapt Interact 21(4–5), 485–511 (2011)
19. Canossa, A., Drachen, A.: Patterns of play: play-personas in user-centred game development. In: Barry, A., Helen, K., Tanya, K. (eds.) Breaking New Ground: Innovation in Games, Play, Practice and Theory: Proceedings of the 2009 Digital Games Research Association Conference. Brunel University, London, UK (2009)
20. Champalle, O., Sehaba, K., Cosmas, D., Mille, A., Prié, Y.: Assistance to trainers for the observation and analysis activities of operators trainees on nuclear power plant full-scope simulator. In: International Conference on Intelligent Networking and Collaborative Systems (INCoS 2012), pp. 33–40. IEEE Computer Society, Bucharest, Romania (2012)
21. Chen, M., Kolko, B.E., Cuddihy, E., Medina, E.: Modeling but not measuring engagement in computer games. In: Proceedings of the 7th International Conference on Games Learning Society, GLS’11, pp. 55–61. ETC Press, Madison, Wisconsin, USA (2011)
22. Chen, M., Kolko, B.E., Cuddihy, E., Medina, E.: Modeling but not measuring engagement in computer games. In: Proceedings of the 7th International Conference on Games Learning Society, GLS’11, pp. 55–61. ETC Press, Madison, Wisconsin, USA (2011)
23. Chittaro, L., Zangrando, N.: The persuasive power of virtual reality: effects of simulated human distress on attitudes towards fire safety. In: Proceedings of the 5th International Conference on Persuasive Technology, PERSUASIVE 2010, pp. 58–69. Springer, Copenhagen, Denmark (2010)
24. Christensen, H., Griffiths, M.K., Farrer, L.: Adherence in internet interventions for anxiety and depression: systematic review. J. Med. Internet Res. 11(2), e13 (2009)
25. Clauzel, D., Sehaba, K., Prié, Y.: Modelling and visualising traces for reflexivity in synchronous collaborative systems. In: Proceedings of the 2009 International Conference on Intelligent Networking and Collaborative Systems, INCoS 2009, pp. 16–23. IEEE Computer Society, Barcelona, Spain (2009)
26. Clauzel, D., Sehaba, K., Prié, Y.: Enhancing synchronous collaboration by using interactive visualisation of modelled traces. Simul. Model. Pract. Theory 19(1), 84–97 (2011)
27. Cocea, M., Weibelzahl, S.: Log file analysis for disengagement detection in e-learning environments. User Model. User-Adapt. Interact. 19(4), 341–385 (2009)
28. Consolvo, S., Everitt, K., Smith, I., Landay, J.A.: Design requirements for technologies that encourage physical activity. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2006, pp. 457–466. ACM, Montréal, Québec, Canada (2006)
29. D’Mello, S.K., Craig, S.D., Sullins, J., Graesser, A.C.: Predicting affective states expressed through an emote-aloud procedure from autotutor’s mixed-initiative dialogue. Int. J. Artif. Intell. Educ. 16(1), 3–28 (2006)
30. Dietterich, T.G.: Machine learning for sequential data: a review. In: Proceedings of the Joint IAPR International Workshop on Structural, Syntactic, and Statistical Pattern Recognition, pp. 15–30. Springer, Windsor, ON, Canada (2002)
31. Doherty, G., Coyle, D., Sharry, J.: Engagement with online mental health interventions: an exploratory clinical study of a treatment for depression. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2012, pp. 1421–1430. ACM, Austin, TX, USA (2012)
32. Donkin, L., Christensen, H., Naismith, L.S., Neal, B., Hickie, B.I., Glozier, N.: A systematic review of the impact of adherence on the effectiveness of e-therapies. J. Med. Internet Res. 13(3), e52 (2011)
33. Engelmann, T., Dehler, J., Bodemer, D., Buder, J.: Knowledge awareness in cscl: a psychological perspective. Comput. Human Behav. 25(4), 949–960 (2009)
34. Fogg, B.J.: Persuasive Technology: Using Computers to Change What We Think and Do. Morgan Kaufmann, Amsterdam (2002)
35. Fogg, B.: A behavior model for persuasive design. In: Proceedings of the 4th International Conference on Persuasive Technology, Persuasive 2009, pp. 40:1–40:7. ACM, Claremont, CA, USA (2009)
36. Fredricks, J.A., Blumenfeld, P.C., Paris, A.H.: School engagement: potential of the concept, state of the evidence. Rev. Educ. Res. 74, 59–109 (2004)
37. Froehlich, J., Dillahunt, T., Klasnja, P., Mankoff, J., Consolvo, S., Harrison, B., Landay, J.A.: Ubigreen: investigating a mobile tool for tracking and supporting green transportation habits. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2009, pp. 1043–1052. ACM, Boston, Massachusetts, USA (2009)
38. Gagné, A.R., El-Nasr, M.S., Shaw, C.D.: A deeper look at the use of telemetry for analysis of player behavior in rts games. In: Proceedings of the 10th International Conference on Entertainment Computing, ICEC’11, pp. 247–257. Springer, Vancouver, Canada (2011)
39. Garris, R., Ahlers, R., Driskell, J.E.: Games, motivation, and learning: a research and practice model. Simul. Gaming 33(4), 441–467 (2002)
40. Hägni, K., Eng, K., Hepp-Reymond, M.C., Holper, L., Keisker, B., Siekierka, E., Kiper, D.C.: The effect of task and ownership on time estimation in virtual environments. In: Presence 2007: The 10th Annual International Workshop on Presence, pp. 145–150, Barcelona, Spain (2007)
41. Hassenzahl, M., Tractinsky, N.: User experience—a research agenda. Behav. Inf. Technol. 25(2), 91–97 (2006)
42. Honka, A., Kaipainen, K., Hietala, H., Saranummi, N.: Rethinking health: Ict-enabled services to empower people to manage their health. IEEE Rev. Biomed. Eng. 4, 119–139 (2011)
43. Chen, M., Kolko, B.E., Cuddihy, E., Medina, E.: Modeling but not measuring engagement in computer games. In: Proceedings of the 7th International Conference on Games Learning Society, GLS’11, pp. 55–61. ETC Press, Madison, Wisconsin, USA (2011)
44. Janssen, J., Erkens, G., Kirschner, P.A.: Group awareness tools: it’s what you do with it that matters. Comput. Hum. Behav. 27(3), 1046–1058 (2011)
45. Jennett, C., Cox, A.L., Cairns, P., Dhoparee, S., Epps, A., Tijs, T., Walton, A.: Measuring and defining the experience of immersion in games. Int. J. Hum.–Comput. Stud. 66(9), 641–661 (2008)
46. Kaptelinin, V.: The object of activity: making sense of the sense-maker. Mind Cult. Activity 12(1), 4–18 (2005)
47. Kaptelinin, V., Nardi, B.A.: Acting with Technology: Activity Theory and Interaction Design. MIT Press, Cambridge (2006)
48. Kim, J.H., Gunn, D.V., Schuh, E., Phillips, B., Pagulayan, R.J., Wixon, D.: Tracking real-time user experience (true): a comprehensive instrumentation solution for complex systems. In: Proceedings of the Twenty-Sixth Annual SIGCHI Conference on Human Factors in Computing Systems, CHI 2008, pp. 443–452. ACM, Florence, Italy (2008)
49. Kivikangas, J.M., Chanel, G., Cowley, B., Ekman, I., Salminen, M., Järvelä, S., Ravaja, N.: A review of the use of psychophysiological methods in game research. J. Gaming Virtual Worlds 3(3), 181–199 (2011)
50. Kjeldskov, J., Skov, M.B., Paay, J., Pathmanathan, R.: Using mobile phones to support sustainability: a field study of residential electricity consumption. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2012, pp. 2347–2356. ACM, Austin, TX, USA (2012)
51. Köck, M., Paramythis, A.: Activity sequence modelling and dynamic clustering for personalized e-learning. User Model. User-Adapt. Interact. 21(1–2), 51–97 (2011)
52. Law, E.L.C., Roto, V., Hassenzahl, M., Vermeeren, A.P., Kort, J.: Understanding, scoping and defining user experience: a survey approach. In: Proceedings of the 27th International Conference on Human Factors in Computing Systems, CHI 2009, pp. 719–728. ACM, Boston, MA, USA (2009)
53. Leontiev, A.N.: Activity, Consciousness and Personality. Prentice Hall, Englewood Cliffs (1978)
54. Linehan, C., Kirman, B., Lawson, S., Chan, G.: Practical, appropriate, empirically-validated guidelines for designing educational games. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2011, pp. 1979–1988. ACM, Vancouver, BC, Canada (2011)
55. Maitland, J., Chalmers, M.: Designing for peer involvement in weight management. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2011, pp. 315–324. ACM, Vancouver, BC, Canada (2011)
56. Mandryk, R.L., Atkins, M.S., Inkpen, K.M. A continuous and objective evaluation of emotional experience with interactive play environments. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI 2006, pp. 1027–1036. ACM, Montréal, QC, Canada (2006)
57. Michel, C., Lavoué, E., Pietrac, L.: A dashboard to regulate project-based learning. In: Proceedings of the 7th European Conference on Technology Enhanced Learning, EC-TEL 2012, pp. 250–263. Springer, Saarbrücken, Germany (2012)
58. Michie, S., Johnston, M., Francis, J., Hardeman, W., Eccles, M.: From theory to intervention: mapping theoretically derived behavioural determinants to behaviour change techniques. Appl. Psychol. 57(4), 660–680 (2008)
59. Mintz, J., Aagaard, M.: The application of persuasive technology to educational settings. Educ. Technol. Res. Dev. 60(3), 483–499 (2012)
60. O’Brien, H.L., Toms, E.G.: What is user engagement? A conceptual framework for defining user engagement with technology. J. Am. Soc. Inf. Sci. 59(6), 938–955 (2008)
61. Orji, R., Vassileva, J., Mandryk, R.L.: Modeling the efficacy of persuasive strategies for different gamer types in serious games for health. User Model. User-Adapt. Interact. (this issue) (2014)
62. Pagulayan, R.J., Keeker, K., Wixon, D., Romero, R.L., Fuller, T.: User-centered design in games. In: Jacko, J.A., Sears, A. (eds.) The Human–Computer Interaction Handbook, pp. 883–906. Lawrence Erlbaum, Hillsdale (2003)
63. Picard, R.W.: Affective Computing. MIT Press, Cambridge (2000)
64. Rich, C., Ponsleur, B., Holroyd, A., Sidner, C.L.: Recognizing engagement in human–robot interaction. In: Proceedings of the 5th ACM/IEEE International Conference on Human–Robot Interaction, HRI ’10, pp. 375–382. IEEE Press, Osaka, Japan (2010)
65. Ritterband, L., Thorndike, F., Cox, D., Kovatchev, B., Gonder-Frederick, L.: A behavior change model for internet interventions. Ann. Behav. Med. 38(1), 18–27 (2009)
66. Ryan, R.M., Deci, E.L.: Self-determination theory and the facilitation of intrinsic motivation, social development, and well-being. Am. Psychol. 55, 68–78 (2000)
67. Scheffel, M., Wolpers, M., Beer, F.: Analyzing contextualized attention metadata with rough set methodologies to support self-regulated learning. In: 2010 IEEE 10th International Conference on Advanced Learning Technologies (ICALT), ICALT 2010, pp. 125–129. IEEE Computer Society, Sousse, Tunisia (2010)
68. Schön, D.A.: The Reflective Practitioner: How Professionals Think in Action. Basic Books (1984)
69. Schubert, T., Friedmann, F., Regenbrecht, H.: The experience of presence: factor analytic insights. Presence: Teleoper. Virtual Environ. 10(3), 266–281 (2001)
70. Settouti, L.S., Prié, Y., Marty, J.C., Mille, A.: A trace-based system for technology-enhanced learning systems personalisation. In: Proceedings of the 2009 Ninth IEEE International Conference on Advanced Learning Technologies, ICALT 2009, pp. 93–97. IEEE Computer Society, Riga, Latvia (2009)
71. Sidner, C.L., Kidd, C.D., Lee, C., Lesh, N.: Where to look: a study of human–robot engagement. In: Proceedings of the 9th International Conference on Intelligent User Interfaces, IUI ’04, pp. 78–84. ACM, Funchal, Madeira, Portugal (2004)
72. Tychsen,A., Canossa, A: Defining personas in games using metrics. In: Proceedings of the 2008 Conference on Future Play: Research, Play, Share, Future Play 2008, pp. 73–80. ACM, Toronto, ON, Canada (2008)
73. Vassileva, J.: Motivating participation in social computing applications: a user modeling perspective. User Model. User-Adapt. Interact. 22(1–2), 177–201 (2012)
74. Vygotsky, L.: Mind in Society: The Development of Higher Psychological Processes. Harvard University Press, Cambridge (1978)
75. Zimmerman, B.J.: Attaining Self-Regulation: A Social Cognitive Perspective, pp. 13–40 (2000)