Hand Gesture and Mathematics Learning: Lessons From an Avatar

Cognitive Science

Volumn 41, Issue 2, 2017, Pages 518-535

  Cook, Susan Wagner ^a   Friedman, Howard S ^b   Duggan, Katherine A ^b   Cui, Jian ^c   Popescu, Voicu ^c  

^a University of Iowa   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

^c PURDUE UNIVERSITY   (United States)

Author keywords

Animated pedagogical agent; Cognitive development; Gesture; Instruction; Learning; Mathematics; Nonverbal behavior

Indexed keywords

CHILD; FEMALE; GESTURE; HUMAN; LEARNING; MALE; MATHEMATICS; PROBLEM SOLVING; TEACHING; TRANSFER OF LEARNING;

CHILD; FEMALE; GESTURES; HUMANS; LEARNING; MALE; MATHEMATICS; PROBLEM SOLVING; TEACHING; TRANSFER (PSYCHOLOGY);

EID: 85015420793     PISSN: 03640213     EISSN: 15516709     Source Type: Journal    
DOI: 10.1111/cogs.12344     Document Type: Article

References (67)

1. Alibali, M. W., & Nathan, M. J. (2012). Embodiment in mathematics teaching and learning: Evidence from learners’ and teachers’ gestures. Journal of the Learning Sciences, 21(2), 247–286. doi:10.1080/10508406.2011.611446
2. Arcavi, A. (2003). The role of visual representations in the learning of mathematics. Educational Studies in Mathematics, 52(3), 215–241.
3. Atkinson, R. (2002). Optimizing learning from examples using animated pedagogical agents. Journal of Educational Psychology, 94(2), 416–427. doi:10.1037//0022-0663.94.2.416
4. Ballard, D. H., Hayhoe, M., Pook, P., & Rao, R. (1997). Deictic codes for the embodiment of cognition. Behavioral and Brain Sciences, 20(4), 723–767.
5. Barlow, S., Converse, S., Kahler, S., Lester, J., & Stone, B. (1997). Animated pedagogical agents and problem solving effectiveness: A large scale empirical evaluation. In B. du Buolay, & R. Mizoguchi (Eds.), Artificial intelligence in education (p. 23). Amsterdam: IOS Press.
6. Barr, R. (2010). Transfer of learning between 2D and 3D sources during infancy: Informing theory and practice. Developmental Review, 30(2), 128–154 http://doi.org/10.1016/j.dr.2010.03.001.
7. Barr, D. J., Levy, R., Scheepers, C., & Tily, H. J. (2013). Random effects structure for confirmatory hypothesis testing: Keep it maximal. Journal of Memory and Language, 68(3), doi:10.1016/j.jml.2012.11.001
8. Bartholomé, T., & Bromme, R. (2009). Coherence formation when learning from text and pictures: What kind of support for whom? Journal of Educational Psychology, 101(2), 282–293. http://doi.org/10.1037/a0014312.
9. Baylor, A. L., & PALS Research Group. (2003). The impact of three pedagogical agent roles (pp. 928–929). Presented at the second international joint conference. New York: ACM Press. doi:10.1145/860575.860729
10. Baylor, A. L., & Kim, S. (2008). The effects of agent nonverbal communication on procedural and attitudinal learning outcomes. Intelligent Virtual Agents, doi:10.1007/978-3-540-85483-8_21
11. Baylor, A. L., & Kim, S. (2009). Designing nonverbal communication for pedagogical agents: When less is more. Computers in Human Behavior, 25(2), 450–457. doi:10.1016/j.chb.2008.10.008
12. Baylor, A. L., & Ryu, J. (2003). The effects of image and animation in enhancing pedagogical agent persona. Journal of Educational Computing Research, 28(4), 373–394. doi:10.2190/V0WQ-NWGN-JB54-FAT4
13. Bergmann, K., Kopp, S., & Eyssel, F. (2010). Individualized gesturing outperforms average gesturing — Evaluating gesture production in virtual humans. In Albeck et al. (Eds.), Proceedings of the 10th Conference on Intelligent Virtual Agents (Vol. 6356, pp. 104–117). Berlin: Springer. doi:10.1007/978-3-642-15892-6_11
14. Bergmann, K., & Macedonia, M. (2013). A virtual agent as vocabulary trainer: Iconic gestures help to improve learners’ memory performance. In R. Aylettet al. (Eds.), Proceedings of the 10th Conference on Intelligent Virtual Agents (Vol. 8108, pp. 139–148). Berlin: Springer. doi:10.1007/978-3-642-40415-3_12
15. Berthold, K., & Renkl, A. (2009). Instructional aids to support a conceptual understanding of multiple representations. Journal of Educational Psychology, 101(1), 70. doi:10.1037/a0013247
16. Black, M., Chang, J., Chang, J., & Narayanan, S. (2009). Comparison of child-human and child-computer interactions based on manual annotations. In Proceedings of the Workshop on Child, Computer and Interaction. Cambridge, USA.
17. Buisine, S., & Martin, J.-C. (2007). The effects of speech–gesture cooperation in animated agents’ behavior in multimedia presentations. Interacting with Computers, 19, 1–10. doi:10.1016/j.intcom.2007.04.002
18. Cassell, J., Pelachaud, C., Badler, N., Steedman, M., Achorn, B., Becket, T., Douville, B., Prevost, S., & Stone, M., et al. (1994). ANIMATED CONVERSATION: Rule-based generation of facial expression, gesture & spoken intonation for multiple conversational agents (pp. 413–420). Presented at the the 21st annual conference, New York: ACM Press. doi:10.1145/192161.192272
19. Cassell, J., & Thorisson, K. R. (1999). The power of a nod and a glance: Envelope vs. emotional feedback in animated conversational agents. Applied Artificial Intelligence, 13(4–5), 519–538. doi:10.1080/088395199117360
20. Chiu, C. C., & Marsella, S. (2011). How to train your avatar: A data driven approach to gesture generation. In Intelligent Virtual Agents, (pp. 127–140) Berlin: Springer.
21. Church, R., Ayman-Nolley, S., & Mahootian, S. (2004). The role of gesture in bilingual education: Does gesture enhance learning? International Journal of Bilingual Education and Bilingualism, 7(4), 303–319.
22. Cohen, J., MacWhinney, B., Flatt, M., & Provost, J. (1993). PsyScope: A new graphic interactive environment for designing psychology experiments. Behavioral Research Methods, Instruments, and Computers, 25(2), 257–271.
23. Cook, S. W., Duffy, R. G., & Fenn, K. M. (2013). Consolidation and transfer of learning after observing hand gesture. Child Development, 84(6), 1863–1871. doi:10.1111/cdev.12097
24. Cook, S., & Goldin-Meadow, S. (2006). The role of gesture in learning: Do children use their hands to change their minds. Journal of Cognition and Development, 7(2), 211–232. DOI: 10.1207/s15327647jcd0702_4.
25. Craig, S. D., Gholson, B., & Driscoll, D. M. (2002). Animated pedagogical agents in multimedia educational environments: Effects of agent properties, picture features and redundancy. Journal of Educational Psychology, 94(2), 428. doi:10.1037/0022-0663.94.2.428
26. Cutica, I., & Bucciarelli, M. (2008). The deep versus the shallow: Effects of co-speech gestures in learning from discourse. Cognitive Science, 32(5), 921–935. doi:10.1080/03640210802222039
27. De Koning, B. B., & Tabbers, H. K. (2013). Gestures in instructional animations: A helping hand to understanding non-human movements? Applied Cognitive Psychology, 27(5), 683–689. doi:10.1002/acp.2937
28. DePaulo, B. M., & Friedman, H. S. (1998). Nonverbal communication. In D. Gilbert, S. Fiske, & G. Lindzey (Eds.), Handbook of social psychology (Vol. II, pp. 3–40). Boston: McGraw Hill.
29. Devlin, B., McNeil, N., Carrazza, C., Byrd, C., & McKeever, M. O. (2015). Early understanding of math equivalence predicts future math achievement. Presented at the Society for Research in Child Development, Philidelphia, PA.
30. Dunsworth, Q., & Atkinson, R. K. (2007). Fostering multimedia learning of science: Exploring the role of an animated agent's image. Computers & Education, 49(3), 677–690. doi:10.1016/j.compedu.2005.11.010
31. Goldin-Meadow, S., Kim, S., & Singer, M. (1999). What the teacher”s hands tell the student”s mind about math. Journal of Educational Psychology, 91(4), 720–730.
32. Goldin-Meadow, S., Nusbaum, H., Kelly, S., & Wagner, S. (2001). Explaining math: Gesturing lightens the load. Psychological Science, 12, 516–522.
33. Heidig, S., & Clarebout, G. (2011). Do pedagogical agents make a difference to student motivation and learning? Educational Research Review, 6(1), 27–54. doi:10.1016/j.edurev.2010.07.004
34. Hostetter, A. B. (2011). When do gestures communicate? A meta-analysis Psychological Bulletin, 137(2), 297–315. doi:10.1037/a0022128
35. Huk, T., Steinke, M., & Floto, C. (2009). The educational value of visual cues and 3D-representational format in a computer animation under restricted and realistic conditions. Instructional Science, 38(5), 455–469. doi:10.1007/s11251-009-9116-7
36. Hyde, J., Kiesler, S., Hodgins, J. K., & Carter, E. J. (2014). Conversing with children (pp. 1787–1796). Presented at the the 32nd annual ACM conference. New York: ACM Press. doi:10.1145/2556288.2557280
37. Jamet, E., Gavota, M., & Quaireau, C. (2008). Attention guiding in multimedia learning. Learning and Instruction, 18(2), 135–145.
38. Kalyuga, S., Chandler, P., & Sweller, J. (1999). Managing split-attention and redundancy in multimedia instruction. Applied Cognitive Psychology, 13(4), 351–371.
39. Kirk, E., Pine, K. J., & Ryder, N. (2011). I hear what you say but I see what you mean: The role of gestures in children's pragmatic comprehension. Language and Cognitive Processes, 26(2), 149–170. doi:10.1080/01690961003752348
40. Koning, B. B., Tabbers, H. K., Rikers, R. M. J. P., & Paas, F. (2009). Towards a framework for attention cueing in instructional animations: Guidelines for research and design. Educational Psychology Review, 21(2), 113–140. doi:10.1007/s10648-009-9098-7
41. de Koning, B. B., Tabbers, H. K., Rikers, R. M. J. P., & Paas, F. (2010). Attention guidance in learning from a complex animation: Seeing is understanding? Learning and Instruction, 20(2), 111–122.
42. Lester, J. C., Converse, S. A., Kahler, S. E., Barlow, S. T., Stone, B. A., & Bhogal, R. S. (1997). The persona effect: Affective impact of animated pedagogical agents (pp. 359–366). Presented at the the SIGCHI conference. New York: ACM Press. doi:10.1145/258549.258797
43. Lin, L., & Atkinson, R. K. (2011). Using animations and visual cueing to support learning of scientific concepts and processes. Computers & Education, 53(3), 650–658.
44. Louwerse, M. M., & Bangerter, A. (2010). Effects of ambiguous gestures and language on the time course of reference resolution. Cognitive Science, 34(8), 1517–1529. doi:10.1111/j.1551-6709.2010.01135.x
45. Lusk, M. M., & Atkinson, R. K. (2007). Animated pedagogical agents: Does their degree of embodiment impact learning from static or animated worked examples? Applied Cognitive Psychology, 21(6), 747–764. doi:10.1002/acp.1347
46. Macedonia, M., Müller, K., & Friederici, A. D. (2010). The impact of iconic gestures on foreign language word learning and its neural substrate. Human Brain Mapping, 32(6), 982–998. doi:10.1002/hbm.21084
47. Matthews, P., Rittle-Johnson, B., McEldoon, K., & Taylor, R. (2012). Measure for measure: What combining diverse measures reveals about children's understanding of the equal sign as an indicator of mathematical equality. Journal for Research in Mathematics Education, 43(3), 316–350. doi:10.5951/jresematheduc.43.3.0316
48. Mayer, R. E. (2005). Principles of multimedia learning based on social cues: Personalization, voice, and image principles. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (pp. 201–212). New York: Cambridge University Press.
49. Mayer, R. E., & DaPra, C. S. (2012). An embodiment effect in computer-based learning with animated pedagogical agents. Journal of Experimental Psychology-Applied, 18(3), 239–252. doi:10.1037/a0028616
50. McNeil, N., & Alibali, M. W. (2004). You'll see what you mean: Students encode equations based on their knowledge of arithmetic. Cognitive Science, 28(3), 451–466.
51. Moreno, R., & Mayer, R. (2007). Interactive multimodal learning environments. Educational Psychology Review, 19(3), 309–326.
52. Moreno, R., Reislein, M., & Ozogul, G. (2010). Using virtual peers to guide visual attention during learning. Journal of Media Psychology: Theories, Methods, and Applications, 22(2), 52–60. doi:10.1027/1864-1105/a000008
53. Mori, M., MacDorman, K. F., & Kageki, N. (2012). The uncanny valley [from the field]. Robotics & Automation, 19(2), 98–100. doi:10.2307/25704807?ref=no-x-route:e46664d78673b8826934eedef4fc9d67
54. van Mulken, S., André, E., & Müller, J. (1998). The persona effect: How substantial is it? (pp. 53–66). London: Springer. doi:10.1007/978-1-4471-3605-7_4
55. Novack, M. A., Congdon, E. L., Hemani-Lopez, N., & Goldin-Meadow, S. (2014). From action to abstraction: Using the hands to learn math. Psychological Science: A Journal of the American Psychological Society/APS, 25(4), 903–910. doi:10.1177/0956797613518351
56. Ozcelik, E., Arslan-Ari, I., & Cagiltay, K. (2010). Why does signaling enhance multimedia learning? Evidence from eye movements. Computers in Human Behavior, 26(1), 110–117. doi:10.1016/j.chb.2009.09.001
57. Ping, R. M., & Goldin-Meadow, S. (2008). Hands in the air: Using ungrounded iconic gestures to teach children conservation of quantity. Developmental Psychology, 44(5), 1277–1287.
58. Richland, L. E., & McDonough, I. M. (2010). Learning by analogy: Discriminating between potential analogs. Contemporary Educational Psychology, 35(1), 28–43. doi:10.1016/j.cedpsych.2009.09.001
59. Rosenthal, R. (1976). Experimenter effects in behavioral research. New York: Halsted Press.
60. Rowe, M. L., Silverman, R. D., & Mullan, B. E. (2013). The role of pictures and gestures as nonverbal aids in preschoolers’ word learning in a novel language. Contemporary Educational Psychology, 38(2), 109–117. doi:10.1016/j.cedpsych.2012.12.001
61. Singer, M., & Goldin-Meadow, S. (2005). Children learn when their teacher's gestures and speech differ. Psychological Science, 16(2), 85–89.
62. Singer, M., Radinsky, J., & Goldman, S. R. (2008). The role of gesture in meaning construction. Discourse Processes, 45(4–5), 365–386. doi:10.1080/01638530802145601
63. Tabbers, H. K., Martens, R. L., & Merriënboer, J. J. G. (2010). Multimedia instructions and cognitive load theory: Effects of modality and cueing. British Journal of Educational Psychology, 74(1), 71–81. doi:10.1348/000709904322848824
64. Valenzeno, L., Alibali, M. W., & Klatzky, R. (2003). Teachers” gestures facilitate students” learning: A lesson in symmetry. Contemporary Educational Psychology, 28(2), 187–204. doi:10.1016/S0361-476X(02)00007-3
65. Wagner, S., Nusbaum, H., & Goldin-Meadow, S. (2004). Probing the mental representation of gesture: Is handwaving spatial? Journal of Memory and Language, 50(4), 395–407.
66. Wang, X., Bernas, R., & Eberhard, P. (2004). Engaging ADHD students in tasks with hand gestures: A pedagogical possibility for teachers. Educational Studies, 30(3), 217–229. doi:10.1080/0305569042000224189
67. Weismer, S. E., & Hesketh, L. J. (1993). The influence of prosodic and gestural cues on novel word acquisition by children with specific language impairment. Journal of Speech Language and Hearing Research, 36(5), 1013–1025. doi:10.1044/jshr.3605.1013