Music to my eyes: Cross-modal interactions in the perception of emotions in musical performance

Cognition

Volumn 118, Issue 2, 2011, Pages 157-170

  Vines, Bradley W ^a,c,d   Krumhansl, Carol L ^b   Wanderley, Marcelo M ^c   Dalca, Ioana M ^a   Levitin, Daniel J ^a,c  

^a MCGILL UNIVERSITY   (Canada)

^b Department of Obstetrics Gynecology   (United States)

^c MCGILL UNIVERSITY   (Canada)

^d UNIVERSITY OF BRITISH COLUMBIA   (Canada)

Author keywords

Cross modal interaction; Emotion; Expressivity; Gesture; Multi sensory integration; Music cognition; Non verbal communication; Performance

Indexed keywords

ADULT; ARTICLE; BEHAVIOR; BODY MOVEMENT; COGNITION; EMOTION; HUMAN; MUSIC; NONVERBAL COMMUNICATION; ORTHOGONAL ROTATION; PRIORITY JOURNAL; RATING SCALE; SENSORY STIMULATION; VERBAL BEHAVIOR;

ADOLESCENT; ADULT; COGNITION; EMOTIONS; HUMANS; MOVEMENT; MUSIC; NONVERBAL COMMUNICATION; PERCEPTION; YOUNG ADULT;

EID: 78651105536     PISSN: 00100277     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.cognition.2010.11.010     Document Type: Article

References (86)

1. Balkwill L.-L., Thompson W.F. A cross-cultural investigation of the perception of emotion in music: Psychophysical and cultural cues. Music Perception 1999, 17(1):43-64.
2. Bharucha J.J., Curtis M., Paroo K. Varieties of musical experience. Cognition 2006, 100:131-172.
3. Bigand E., Vieillard S., Madurell F., Marozeau J., Dacquet A. Multidimensional scaling of emotional responses to music: The effect of musical expertise and of the duration of the excerpts. Cognition & Emotion 2005, 19(8):1113-1139.
4. Blood A.J., Zatorre R.J. Intensely pleasurable responses to music correlate with activity in the brain regions implicated in reward and emotion. Proceedings of the National Academy of Sciences 2001, 98:11810-11823.
5. Broughton M., Stevens C., Malloch S. Music, movement and marimba: An investigation of the role of movement and gesture in communicating musical expression to an audience. Psychology of Music 2009, 37(2):137-153.
6. Brown S. The 'musilanguage' model of music evolution. The origins of music 2000, 271-300. MIT Press, Cambridge. N.L. Wallin, B. Merker, S. Brown (Eds.).
7. Cacioppo J.T., Berntson G.G. Relationship between attitudes and evaluative space: A critical review, with emphasis on the separability of positive and negative substrates. Psychological Bulletin 1994, 115:401-423.
8. Cacioppo J.T., Gardner W.L., Berntson G.G. Beyond bipolar conceptualizations and measures: The case of attitudes and evaluative space. Personality and Social Psychology Review 1997, 1:3-25.
9. Chapados C., Levitin D.J. Cross-modal interactions in the experience of musical performances: Physiological correlates. Cognition 2008, 108:639-651.
10. Cooper G.W., Meyer L.B. The rhythmic structure of music 1960, University of Chicago Press, Chicago.
11. Cross I. Music as a biocultural phenomenon. Annals of the New York Academy of Sciences 2003, 999:106-111.
12. Dahl S., Friberg A. Visual perception of expressiveness in musicians' body movements. Music Perception 2007, 24(5):433-454.
13. Davidson J.W. Visual perception of performance manner in the movements of solo musicians. Psychology of Music 1993, 21:103-113.
14. Davidson J.W. Which areas of a pianist's body convey information about expressive intention to an audience?. Journal of Human Movement Studies 1994, 26:279-301.
15. Davidson J.W., Correia J.S. Body movement. The science and psychology of music performance 2002, 237-253. Oxford University Press, New York. R. Parncutt, G.E. McPherson (Eds.).
16. Davies S. Musical meaning and expression 1994, Cornell University Press, Ithaca.
17. Delalande F. La gestique de Gould; éléments pour une sémiologie du geste musical. Glenn Gould Pluriel 1988, 83-111. Louise Courteau Éditrice Inc., Montréal. G. Guertin (Ed.).
18. Deliège I., El Ahmade A. Mechanisms of cue extraction in musical groupings: A study of perception on Sequenza VI for viola solo by Luciano Berio. Psychology of Music 1990, 18:18-44.
19. Di Carlo N.S., Guaitella I. Facial expressions of emotion in speech and singing. Semiotica 2004, 149(1/4):37-55.
20. Dittrich W.H., Troscianko T., Lea S.E.G., Morgan D. Perception of emotion from dynamic point-light displays represented in dance. Perception 1996, 25:727-738.
21. Eibl-Eibesfeldt I. Similarities and differences between cultures in expressive movements. Non-verbal communication 1972, 297-314. Cambridge University Press, Cambridge. R.A. Hinde (Ed.).
22. Ekman P. An argument for basic emotions. Cognition & Emotion 1992, 6:169-200.
23. Ekman P. Facial expressions. The handbook of cognition and emotion 1998, 301-320. John Wiley & Sons, Ltd., Sussex, UK. T. Dalgleish, T. Power (Eds.).
24. Faith M., Thayer J.F. A dynamical systems interpretation of a dimensional model of emotion. Scandinavian Journal of Psychology 2001, 42:121-133.
25. Fitch W.T. The biology and evolution of music: A comparative perspective. Cognition 2006, 100(1):173-215.
26. Fredrickson W.E. Perception of tension in music: Musicians versus nonmusicians. Journal of Music Therapy 2000, 37(1):40-50.
27. Gabrielsson A. The performance of music. The psychology of music 1999, 501-602. Academic Press, London. 2nd ed. D. Deutsch (Ed.).
28. Gabrielsson A. Emotions in strong experiences with music. Music and emotion: Theory and research 2001, 431-449. Oxford University Press, Oxford. P.N. Juslin, J.A. Sloboda (Eds.).
29. Gabrielsson A., Lindström S. Strong experiences related to music: A descriptive system. Musicae Scientiae 2003, 7(2):157-217.
30. Galati G., Committeri G., Spitoni G., Aprile T., Di Russo F., Pitzalis S., et al. A selective representation of the meaning of actions in the auditory mirror system. Neuroimage 2008, 40(3):1274-1286.
31. Gazzola V., Aziz-Zadeh L., Keysers C. Empathy and the somatotopic auditory mirror system in humans. Current Biology 2006, 16(18):1824-1829.
32. Gick B., Derrick D. Aero-tactile integration in speech perception. Nature 2009, 462:502-504.
33. Goldin-Meadow S. Hearing gesture: How our hands help us think 2003, Harvard University Press, Cambridge, MA.
34. Green D.P., Goldman S.L., Salovey P. Measurement error masks bipolarity in affect ratings. Journal of Personality and Social Psychology 1993, 64(6):1029-1041.
35. Green D.P., Salovey P. In what sense are positive and negative affect independent? A reply to Tellegen, Watson, and Clark. Psychological Science 1999, 10(4):304-306.
36. Hatfield E., Cacioppo J.T., Rapson R.L. Emotional contagion 1994, Cambridge University Press, New York.
37. Hauk O., Shtyrov Y., Pulvermuller F. The sound of actions as reflected by mismatch negativity: Rapid activation of cortical sensory-motor networks by sounds associated with finger and tongue movements. European Journal of Neuroscience 2006, 23(3):811-821.
38. Hunter P.G., Schellenberg E.G., Schimmack U. Mixed affective responses to music with conflicting cues. Cognition & Emotion 2008, 22(2):327-352.
39. Huron D. Is music evolutionary adaptation?. Annals of the New York Academy of Sciences 2001, 930:43-61.
40. Izard C.E. The face of emotion 1971, Appleton Century Crofts, New York.
41. Jackendoff R., Lerdahl F. The capacity for music: What is it, and what's special about it?. Cognition 2006, 100:33-72.
42. Johansson G. Visual perception of biological motion and a model for its analysis. Perception & Psychophysics 1973, 14:201-211.
43. Koelsch S., Kasper E., Sammler D., Schulze K., Gunter T., Friederici A.D. Music, language and meaning: Brain signatures of semantic processing. Nature Neuroscience 2004, 7(3):302-307.
44. Krumhansl C.L. Cognitive foundations of musical pitch 1990, Oxford University Press, New York.
45. Krumhansl C.L. A perceptual analysis of Mozart's Piano Sonata K. 282: Segmentation, tension, and musical ideas. Music Perception 1996, 13(3):401-432.
46. Krumhansl C.L. An exploratory study of musical emotions and psychophysiology. Canadian Journal of Experimental Psychology 1997, 51(4):336-352.
47. Krumhansl C.L. Music: A link between cognition and emotion. Current Directions in Psychological Science 2002, 11(2):45-50.
48. Krumhansl C.L., Schenck D.L. Can dance reflect the structural and expressive qualities of music? A perceptual experiment on Balanchine's choreography of Mozart's Di-vertimento No. 15. Musicae Scientiae 1997, 1:63-85.
49. Larsen, J. T., Norris, C. J., & Cacioppo, J. T. (2004). The evaluative space grid: A single-item measure of positive and negative evaluative reactions. Working paper, Department of Psychology, Texas Tech University, Lubbock.
50. Larsen J.T., McGraw A.P., Cacioppo J.T. Can people feel happy and sad at the same time?. Journal of Personality and Social Psychology 2001, 81:684-696.
51. Lasswell H.D. The structure and function of communication in society. The communication of ideas 1964, Cooper Square Publishers, New York. L. Bryson (Ed.).
52. Lerdahl F. Tonal pitch space 2001, Oxford University Press, New York.
53. Lerdahl F., Jackendoff R. A generative theory of tonal music 1983, MIT Press, Cambridge, MA.
54. Levinson J. What a musical work is. Journal of Philosophy 1980, 77:5-28.
55. Levitin D.J., Menon V. Musical structure is processed in " language" areas of the brain: A possible role for Brodmann Area 47 in temporal coherence. Neuroimage 2003, 20(4):2142-2152.
56. Levitin D.J., Menon V. The neural locus of temporal structure and expectancies in music: Evidence from functional neuroimaging at 3 Tesla. Music Perception 2005, 22(3):549-562.
57. Levitin D.J., Schaaf A., Goldberg L.R. Factor diagrammer (Version 1.1b) 2005, McGill University, Montreal, (computer software) .
58. Lopes D.M., Bergeron V. Hearing and seeing musical expression. Philosophy and Phenomenological Research 2009, 78:1-16.
59. McAdams S., Vines B.W., Vieillard S., Smith B.K., Reynolds R. Influences of large-scale form on continuous ratings in response to a contemporary piece in a live concert setting. Music Perception 2004, 22(2):297-350.
60. McGurk H., MacDonald J. Hearing lips and seeing voices. Nature 1976, 264:746-748.
61. McNeill D. Hand and mind: What gestures reveal about thought 1992, University of Chicago Press, Chicago.
62. McNeill D. Triangulating the growth point-arriving at consciousness. Gesture, speech, and sign 1999, 77-92. Oxford University Press, New York. L.S. Messing, R. Campbell (Eds.).
63. McNeill D. Gesture and thought 2005, University of Chicago Press, Chicago.
64. Menon V., Levitin D.J. The rewards of music listening: Response and physiological connectivity of the mesolimbic system. Neuroimage 2005, 28(1):175-184.
65. Meyer L.B. Emotion and meaning in music 1956, University of Chicago Press, Chicago.
66. Mithen S. The singing Neanderthals 2007, Harvard University Press, Cambridge, MA.
67. Ortony A., Turner T.J. What's basic about basic emotions?. Psychological Review 1990, 97(3):315-331.
68. Patel A.D. Language, music, syntax and the brain. Nature Neuroscience 2003, 6(7):674-681.
69. Repp B.H. Patterns of note onset asynchronies in expressive piano performance. Journal of the Acoustical Society of America 1996, 100:3917-3932.
70. Runeson S., Frykholm G. Kinematic specification of dynamics as an informational basis for person-and-action perception: Expectations, gender, recognition, and deceptive intention. Journal of Experimental Psychology: General 1983, 112:585-615.
71. Russell J.A. Affective space is bipolar. Journal of Personality and Social Psychology 1979, 37:345-356.
72. Russell J.A. A circumplex model of affect. Journal of Personality and Social Psychology 1980, 39:1161-1178.
73. Russell J.A. Core affect and the psychological construction of emotion. Psychological Review 2003, 110(1):145-172.
74. Russell J.A., Carroll J.M. On the bipolarity of positive and negative affect. Psychological Bulletin 1999, 125:3-30.
75. Schubert E. Measuring emotion continuously: Validity and reliability of the two dimensional emotion space. Australian Journal of Psychology 1999, 51:154-165.
76. Schubert E. Modeling perceived emotion with continuous musical features. Music Perception 2004, 21(4):561-585.
77. Schutz M. Seeing music? What musicians need to know about vision. Empirical Musicology Review 2008, 3(3):83-108.
78. Schutz M., Lipscomb S. Hearing gestures, seeing music: Vision influences perceived tone duration. Perception 2007, 36(6):888-897.
79. Schwartz J.-L., Berthommier F., Savariaux C. Seeing to hear better: Evidence for early audio-visual interactions in speech identification. Cognition 2004, 93(2):B69-B78.
80. Tellegen A., Watson D., Clark L.A. On the dimensional and hierarchical structure of affect. Psychological Science 1999, 10(4, Special Section):297-303.
81. Thompson W.F., Graham P., Russo F.A. Seeing music performance: Visual influences on perception and experience. Semiotica 2005, 156:203-227.
82. Thompson W.F., Russo F., Quinto L. Audio-visual integration of emotional cues in song. Cognition & Emotion 2008, 22(8):1457-1470.
83. Vines B.W., Krumhansl C.L., Wanderley M.M., Levitin D.J. Cross-modal interactions in the perception of musical performance. Cognition 2006, 101(1):80-103.
84. Vines B.W., Nuzzo R.L., Levitin D.J. Quantifying and analyzing musical dynamics: Differential calculus, physics and functional data techniques. Music Perception 2005, 23(2):137-152.
85. Wanderley M.M. Quantitative analysis of non-obvious performer gestures. Gesture and sign language in human-computer interaction 2002, 241-253. Springer Verlag, Berlin. I. Wachsmuth, T. Sowa (Eds.).
86. Watson D., Tellegen A. Toward a consensual structure of mood. Psychological Bulletin 1985, 98(2):219-235.