Going beyond what is visible: What multichannel data can reveal about interaction in the context of collaborative learning?

Computers in Human Behavior

Volumn 96, Issue , 2019, Pages 235-245

  Malmberg, Jonna ^a   Järvelä, Sanna ^a   Holappa, Jukka ^a   Haataja, Eetu ^a   Huang, Xiaohua ^a   Siipo, Antti ^a  

^a UNIVERSITY OF OULU   (Finland)

Author keywords

Arousal; Collaborative learning; Facial expression recognition; Regulated learning; Self regulated learning; Simultaneous arousal

Indexed keywords

FACE RECOGNITION;

AROUSAL; COLLABORATIVE LEARNING; FACIAL EXPRESSION RECOGNITION; REGULATED LEARNING; SELF-REGULATED LEARNING; SIMULTANEOUS AROUSAL;

DATA HANDLING;

EID: 85065032132     PISSN: 07475632     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.chb.2018.06.030     Document Type: Article

References (81)

1. Azevedo, R., Issues in dealing with sequential and temporal characteristics of self-and socially-regulated learning. Metacognition and Learning 9:2 (2014), 217–228, 10.1007/s11409-014-9123-1.
2. Azevedo, R., Defining and measuring engagement and learning in science: Conceptual, theoretical, methodological, and analytical issues. Educational Psychologist 50:1 (2015), 84–94 http://doi.org/10.1080/00461520.2015.1004069.
3. Azevedo, R., Johnson, A., Chauncey, A., Graesser, A., Use of hypermedia to convey and assess self-regulated learning. Zimmerman, B., Schunk, D., (eds.) Handbook of self-regulation of learning and performance, 2011, Routledge, New York, NY, 393–407.
4. Bandura, A., Self-efficacy mechanism in human agency. American Psychologist 37:2 (1982), 122–147 http://doi.org/10.1037/0003-066X.37.2.122.
5. Barron, B., When smart groups fail. The Journal of the Learning Sciences 12 (2003), 307–359.
6. Benedek, M., Kaernbach, C., Decomposition of skin conductance data by means of nonnegative deconvolution. Psychophysiology 47:4 (2010), 647–658 http://doi.org/10.1111/j.1469-8986.2009.00972.x.
7. Blascovich, J., Vanman, E., Mendes, W., Dickerson, S., Social psychophysiology for social and personality psychology. 2011, SAGE Publications, UK http://doi.org/10.4135/9781446287842.
8. Boekaerts, M., Niemivirta, M., Self-regulated learning. Finding a balance between learning goals and ego-protective goals. Boekaerts, M., Pintrich, P.R., Zeidner, M., (eds.) Handbook of self-regulation, 2000, Academic Press, San Diego, 417–450.
9. Boucsein, W., Fowles, D.C., Grimnes, S., Ben-Shakhar, G., Roth, W.T., Dawson, M.E., et al. Publication recommendations for electrodermal measurements. Psychophysiology 49:8 (2012), 1017–1034 http://doi.org/10.1111/j.1469-8986.2012.01384.x.
10. Buder, J., Bodemer, D., Supporting controversial CSCL discussions with augmented group awareness tools. International Journal of Computer-Supported Collaborative Learning 3 (2008), 123–139 http://doi.org/10.1007/s11412-008-9037-5.
11. Chanel, G., Mühl, C., Connecting brains and bodies: Applying physiological computing to support social interaction. Interacting with Computers 27:5 (2015), 534–550.
12. Cleary, T.J., Callan, G.L., Zimmerman, B.J., Assessing self-regulation as a cyclical, context-specific phenomenon: Overview and analysis of srl microanalytic protocols. Educational Research International 2012 (2012), 1–19 http://doi.org/10.1155/2012/428639.
13. Dillenbourg, P., What do you mean by collaborative learning?. Dillenbourg, P., (eds.) Collaborative learning: Cognitive and computational approaches, 1999, Elsevier, Oxford, 1–19.
14. D'Mello, S., Graesser, A., The half-life of cognitive-affective states during complex learning. Cognition & Emotion 25:7 (2011), 1299–1308.
15. D'Mello, S., Lehman, B., Pekrun, R., Graesser, A., Confusion can be beneficial for learning. Learning and Instruction 29 (2014), 153–170.
16. Ekman, P., Friesen, W.V., Facial action coding system: Investigator's guide. 1978, Consulting Psychologists Press, Palo Alto, CA.
17. Ekman, P., Friesen, W.V., Felt, false, and miserable smiles. Journal of Nonverbal Behavior 6:4 (1982), 238–252.
18. Eleftheriadis, S., Rudovic, O., Pantic, M., Discriminative shared Gaussian processes for multiview and view-invariant facial expression recognition. IEEE Transactions on Image Processing 24:1 (2015), 189–204.
19. Elkins, A.N., Muth, E.R., Hoover, A.W., Walker, A.D., Carpenter, T.L., Switzer, F.S., Physiological compliance and team performance. Applied Ergonomics 40:6 (2009), 997–1003 http://doi.org/10.1016/j.apergo.2009.02.002.
20. Erkens, G., Jaspers, J., Prangsma, M., Kanselaar, G., Coordination processes in computer supported collaborative writing. Computers in Human Behavior 21:3 (2005), 463–486 https://doi.org/10.1016/j.chb.2004.10.038.
21. Fairclough, S.H., Venables, L., Tattersall, A., The influence of task demand and learning on the psychophysiological response. International Journal of Psychophysiology 56:2 (2005), 171–184 http://doi.org/10.1016/j.ijpsycho.2004.11.003.
22. Fontaine, J.R., Scherer, K.R., Roesch, E.B., Ellsworth, P., The world of emotion is not two-dimensional. Psychological Science 18 (2007), 1050–1057.
23. Fransen, J., Kirschner, P.A., Erkens, G., Mediating team effectiveness in the context of collaborative learning: The importance of team and task awareness. Computers in Human Behavior 27:3 (2011), 1103–1113.
24. Gillies, R.M., Carroll, A., Cunnington, R., Rafter, M., Palghat, K., Bednark, J., et al. Multimodal representations during an inquiry problem-solving activity in a year 6 science class: A case study investigating cooperation, physiological arousal and belief states. Australian Journal of Education 60:2 (2016), 111–127 http://doi.org/10.1177/0004944116650701.
25. Giromini, L., Morese, R., Salatino, A., Di Girolamo, M., Viglione, D.J., Zennaro, A., Rorschach Performance Assessment System (R-PAS) and vulnerability to stress: A preliminary study on electrodermal activity during stress. Psychiatry Research 246 (2016), 166–172.
26. Hackman, R., Morris, C., Group tasks, group interaction process, and group performance effectiveness. Advances in Experimental Social Psychology 8 (1975), 45–99 https://doi.org/10.1016/S0065-2601(08)60248-8.
27. Hadwin, A.F., Järvelä, S., Miller, M., Self-regulation, co-regulation, and shared regulation in collaborative learning environments. Handbook of self-regulation of learning and performance, Routledge, 2017, 99–122.
28. Harley, J.M., Bouchet, F., Hussain, M.S., Azevedo, R., Calvo, R., Computers in human behavior: A multi-componential analysis of emotions during complex learning with an intelligent multi-agent system q. Computers in Human Behavior 48 (2015), 615–625 http://doi.org/10.1016/j.chb.2015.02.013.
29. Henriques, R., Paiva, A., Antunes, C., Accessing emotion patterns from affective interactions using electrodermal activity. 2013 humaine association conference on affective computing and intelligent interaction, 2013, 43–48 http://doi.org/10.1109/ACII.2013.14.
30. Hernández-García, Á., González-González, I., Jiménez-Zarco, A.I., Chaparro-Peláez, J., Applying social learning analytics to message boards in online distance learning: A case study. Computers in Human Behavior 47:2015 (2014), 68–80 http://doi.org/10.1016/j.chb.2014.10.038.
31. Hockey, R.J., Compensatory control in the regulation of human performance under stress and high workload: A cognitive-energetical framework. Biological Psychology 45:1–3 (1997), 73–93 http://doi.org/10.1016/S0301-0511(96)05223-4.
32. Huang, X., Kortelainen, J., Zhao, G., Li, X., Moilanen, A., Seppänen, T., et al. Multi-modal emotion analysis from facial expressions and electroencephalogram. Computer Vision and Image Understanding 147 (2016), 114–124.
33. Huang, X., Zhao, G., Hong, X., Zheng, W., Pietikäinen, M., Spontaneous facial micro-expression analysis using spatiotemporal completed local quantized patterns. Neurocomputing 175 (2016), 564–578.
34. Iiskala, T., Vauras, M., Lehtinen, E., Salonen, P., Socially shared metacognition of dyads of pupils in collaborative mathematical problem-solving processes. Learning and Instruction 21:3 (2011), 379–393 https://doi.org/10.1016/j.learninstruc.2010.05.002.
35. Immordino-Yang, M.H., Christodoulou, J.A., Pekrun, R., Linnenbrink-Garcia, L., Neuroscientific contributions to emotion measurement in educational contexts. Pekrun, R., Linnenbrink-Garcia, L., (eds.) Handbook of emotions in education, 2014, Routledge, New York, 607–624.
36. Isohätälä, J., Järvenoja, H., Järvelä, S., Socially shared regulation of learning and participation in social interaction in collaborative learning. 2016 (Submitted).
37. Jordan, B., Henderson, A., Interaction analysis: Foundations and practice. The Journal of the Learning Sciences 4:1 (1995), 39–103.
38. Järvelä, S., Hadwin, A., New frontiers: Regulating learning in CSCL. Educational Psychologist 48:1 (2013), 25–39, 10.1080/00461520.2012.750227.
39. Järvelä, S., Järvenoja, H., Malmberg, J., Hadwin, A.F., Exploring socially shared regulation in the context of collaboration. Journal of Cognitive Education and Psychology 12:3 (2013), 267–286.
40. Järvelä, S., Järvenoja, H., Malmberg, J., Isohätälä, J., Sobocinski, M., How do types of interaction and phases of self-regulated learning set a stage for collaborative engagement?. Learning and Instruction 43:2016 (2015), 39–51 http://doi.org/10.1016/j.learninstruc.2016.01.005.
41. Järvelä, S., Malmberg, J., Sobocinski, M., Haataja, E., Kirschner, P., What multimodal data can tell us about the self-regulated learning process. 2016 (Submitted).
42. Järvenoja, H., Järvelä, S., Malmberg, J., Supporting groups’ emotion and motivation regulation during collaborative learning. Learning and Instruction, 2017 (in press) http://doi.org/10.1016/j.learninstruc.2017.11.004.
43. Johnson, A.M., Azevedo, R., D'Mello, S.K., The temporal and dynamic nature of self-regulatory processes during independent and externally assisted hypermedia learning. Cognition and Instruction 29:4 (2011), 471–504 http://doi.org/10.1080/07370008.2011.610244.
44. Keskinarkaus, A., Huttunen, S., Siipo, A., Holappa, J., Laszlo, M., Juuso, I., Laukka, S., MORE—a multimodal observation and analysis system for social interaction research. Multimedia Tools and Applications 75:11 (2015), 1–25.
45. Kory, J., D'Mello, S., A review and meta-analysis of multimodal affect detection. ACM Computing Surveys, 47(3), 2015 http://doi.org/10.1145/2682899.
46. Kreijns, K., Kirschner, P.A., Jochems, W., Identifying the pitfalls for social interaction in computer-supported collaborative learning environments: A review of the research. Computers in Human Behavior 19:3 (2003), 335–353 http://doi.org/10.1016/S0747-5632(02)00057-2.
47. Ku, H.Y., Tseng, H.W., Akarasriworn, C., Collaboration factors, teamwork satisfaction, and student attitudes toward online collaborative learning. Computers in Human Behavior 29:3 (2013), 922–929, 10.1016/j.chb.2012.12.019.
48. Kwon, K., Liu, Y.H., Johnson, L.P., Group regulation and social-emotional interactions observed in computer supported collaborative learning: Comparison between good vs. poor collaborators. Computers & Education, 78, 2014 http://doi.org/10.1016/j.compedu.2014.06.004.
49. Linnenbrink-Garcia, L., Pekrun, R., Students’ emotions and academic engagement: Introduction to the special issue. Contemporary Educational Psychology 36:1 (2011), 1–3 http://doi.org/10.1016/j.cedpsych.2010.11.004.
50. Malmberg, J., Järvelä, S., Järvenoja, H., Capturing temporal and sequential patterns of self-, co-, and socially shared regulation in the context of collaborative learning. 2016 (Submitted).
51. Malmberg, J., Järvelä, S., Järvenoja, H., Panadero, E., Promoting socially shared regulation of learning in CSCL: Progress of socially shared regulation among high- and low-performing groups. Computers in Human Behavior 52 (2015), 562–572 http://doi.org/10.1016/j.chb.2015.03.082.
52. Mendes, W.B., Assessing autonomic nervous system activity. Harmon-Jones, E., Beer, J.S., (eds.) Methods in social neuroscience, 2009, Guilford Press, New York, 118–147.
53. Mikroyannidis, A., Okada, A., Scott, P., Rusman, E., Specht, M., Stefanov, K., et al. WeSPOT: A personal and social approach to inquiry-based learning. Journal of Universal Computer Science 19:14 (2013), 2093–2111 http://doi.org/10.3217/jucs-019-14-2093.
54. Miller, M., Hadwin, A., Scripting and awareness tools for regulating collaborative learning: Changing the landscape of support in CSCL. Computers in Human Behavior 52 (2015), 573–588 http://doi.org/10.1016/j.chb.2015.01.050.
55. Molenaar, I., Chiu, M.M., Dissecting sequences of regulation and cognition: Statistical discourse analysis of primary school children's collaborative learning. Metacognition and Learning, 2014 http://doi.org/10.1007/s11409-013-9105-8.
56. Morris, R., Church, H., Hadwin, A.F., Gress, C.L.Z., Winne, P.H., The use of roles, scripts, and prompts to support CSCL in gStudy. Computers in Human Behavior 26 (2010), 815–824.
57. Munro, L.L., Dawson, M.E., Schell, A.M., Sakai, L.M., Electrodermal lability and rapid performance decrement in a degraded stimulus continuous performance task. Journal of Psychophysiology 1 (1987), 249–257.
58. Nikula, R., Psychological correlates of nonspecific skin-conductance responses. Psychophysiology 28:1 (1991), 86–90 http://doi.org/10.1111/j.1469-8986.1991.tb03392.x.
59. Palumbo, R.V., Marraccini, M.E., Weyandt, L.L., Wilder-Smith, O., McGee, H.A., Liu, S., et al. Interpersonal autonomic physiology: A systematic review of the literature. Personality and Social Psychology Review 21:2 (2016), 99–141 http://doi.org/10.1177/1088868316628405.
60. Patton, M.Q., Qualitative research and evaluation methods. 3rd ed., 2002, Sage, Thousand Oaks, CA.
61. Pecchinenda, A., The affective significance of skin conductance activity during a difficult problem-solving task. Cognition & Emotion 10:5 (1996), 481–504 http://doi.org/10.1080/026999396380123.
62. Phielix, C., Prins, F.J., Kirschner, P. a., Erkens, G., Jaspers, J., Group awareness of social and cognitive performance in a CSCL environment: Effects of a peer feedback and reflection tool. Computers in Human Behavior 27:3 (2011), 1087–1102 http://doi.org/10.1016/j.chb.2010.06.024.
63. Pintrich, P.R., Smith, D.A., Garcia, T., McKeachie, W.J., Reliability and predictive validity of the motivated for learning strategies Questionnaire (MSLQ). Educational and Psychological Measurement 53:3 (1993), 801–814.
64. Reimann, P., Markauskaite, L., Bannert, M., e-Research and learning theory: What do sequence and process mining methods contribute?. British Journal of Educational Technology 45:3 (2014), 528–540 http://doi.org/10.1111/bjet.12146.
65. Schutz, P.A., Davis, H.A., Emotions and self-regulation during test taking. Educational Psychologist 35:4 (2000), 243–256 http://doi.org/10.1207/S15326985EP3504_03.
66. Smith, C.A., Dimensions of appraisal and physiological response in emotion. Journal of Personality and Social Psychology, 56(3), 1989, 339.
67. Teasley, S.D., Roschelle, J., Constructing a joint problem space: The computer as a tool for sharing knowledge. Lajoie, S.P., Derry, S.J., (eds.) Computers as cognitive tools, 1995, Lawrence Erlbaum, Hillsdale, NJ, 229–257.
68. Tomaka, J., Blascovich, J., Kelsey, R.M., Leitten, C.L., Subjective, physiological, and behavioral effects of threat and challenge appraisal. Journal of Personality and Social Psychology 65:2 (1993), 248–260.
69. Valstar, M., Pantic, M., Fully automatic facial action unit detection and temporal analysis. 2006 conference on computer vision and pattern recognition workshop (CVPRW'06), 2006, IEEE (pp. 149–149).
70. Van Boxtel, C., Studying peer interaction from three perspectives: The example of collaborative concept learning. van der Linden, J., Renshaw, P., (eds.) Dialogic learning, 2004, Kluwer:Academic Publishers, Dordrecht, 125–143.
71. Väyrynen, E., Kortelainen, J., Seppänen, T., Classifier-based learning of nonlinear feature manifold for visualization of emotional speech prosody. IEEE Transactions on Affective Computing 4:1 (2013), 47–56.
72. Volet, S., Summers, M., Thurman, J., High-level co-regulation in collaborative learning: How does it emerge and how is it sustained?. Learning and Instruction 19:2 (2009), 128–143 http://doi.org/10.1016/j.learninstruc.2008.03.001.
73. Volet, S., Vauras, M., Salonen, P., Self- and social regulation in learning contexts: An integrative perspective. Educational Psychologist 44:4 (2009), 215–226 http://doi.org/10.1080/00461520903213584.
74. Weinberger, A., Stegmann, K., Fischer, F., Learning to argue online: Scripted groups surpass individuals (unscripted groups do not). Computers in Human Behavior 26:4 (2010), 506–515.
75. Winne, P.H., Improving measurements of self-regulated learning. Educational Psychologist 45:4 (2010), 267–276 http://doi.org/10.1080/00461520.2010.517150.
76. Winne, P.H., Hadwin, A.F., Studying as self-regulated learning. Hacker, D.J., Dunlosky, J., Graesser, A.C., (eds.) Metacognition in educational theory and practice, 1998, Lawrence Erlbaum, Mahwah, 277–304.
77. Worsley, M., Blikstein, P., Leveraging multimodal learning analytics to differentiate student learning strategies. Proceedings of the fifth international conference on learning analytics and knowledge, 2015, March, ACM, 360–367.
78. Zhao, G., Pietikäinen, M., Dynamic texture recognition using volume local binary patterns. Dynamical vision, 2007, Springer, Berlin, Heidelberg, 165–177.
79. Zhao, K., Chan, C.K., Fostering collective and individual learning through knowledge building. International Journal of Computer-Supported Collaborative Learning 9:1 (2014), 63–95.
80. Zimmerman, B.J., Theories of self-regulated learning and academic achievement: An overview and analysis. Zimmerman, B.J., Schunk, D.H., (eds.) Self-regulated learning and academic achievement, 2001, Lawrence Erlbaum Associates, New York, NY, 1–38.
81. Zimmerman, B.J., Schunk, D.H., Reflections on theories of self-regulated learning and academic achievement. Self-regulated learning and academic achievement, 2013, Routledge, 282–301.