Automated analysis of body movement in emotionally expressive piano performances

Music Perception

Volumn 26, Issue 2, 2008, Pages 103-119

  Castellano, Ginevra ^a,c   Mortillaro, Marcello ^b   Camurri, Antonio ^a   Volpe, Gualtiero ^a   Scherer, Klaus ^b  

^a UNIVERSITY OF GENOA   (Italy)

^b UNIVERSITY OF GENEVA   (Switzerland)

^c QUEEN MARY UNIVERSITY OF LONDON   (United Kingdom)

Author keywords

Automated analysis; Bodily movements; Emotional discrimination; Emotional expression; Piano performance

Indexed keywords

EID: 57049146237     PISSN: 07307829     EISSN: 15338312     Source Type: Journal    
DOI: 10.1525/mp.2008.26.2.103     Document Type: Article

References (56)

1. ARGYLE, M. (1980). Bodily communication. London: Methuen & Co Ltd.
2. BALOMENOS, T., RAOUZAIOU, A., IOANNOU, S., DROSOPOULOS, A., KARPOUZIS, K., & KOLLIAS, S. (2005). Emotion analysis in man-machine interaction systems. In S. Bengio & H. Bourlard (Eds.), Lecture notes in computer science: Vol. 3361. Machine learning for multimodal interaction (pp. 318-328). Berlin: Springer-Verlag.
3. BERNHARDT, D., & ROBINSON, P. (2007). Detecting affect from non-stylised body motions. In A. Paiva, R. Prada, & R.W. Picard (Eds.), Lecture notes in computer science: Vol. 4738. Affective computing and intelligent interaction, second international conference (pp. 59-70). Berlin: Springer Verlag.
4. BOONE, R. T., & CUNNINGHAM, J. G. (1998). Children's decoding of emotion in expressive body movement: The development of cue attunement. Developmental Psychology, 34, 1007-1016.
5. BRUNSWIK, E. (1956). Perception and the representative design of psychological experiments. Berkeley: University of California Press.
6. CAMURRI, A., COLETTA, P., VARNI, G., & GHISIO, S. (2007). Developing multimodal interactive systems with EyesWeb XMI. Proceedings of the 2007 conference on new interfaces for musical expression (NIME07) (pp. 305-308). New York, USA.
7. CAMURRI A., DE POLI G., LEMAN M., & VOLPE G. (2005). Toward communicating expressiveness and affect in multimodal interactive systems for performing art and cultural applications. IEEE Multimedia Magazine, 12, 43-53.
8. CAMURRI, A., LAGERLÖF, I., & VOLPE, G. (2003). Recognizing emotion from dance movement: Comparison of spectator recognition and automated techniques. International Journal of Human-Computer Studies, 59, 213-225.
9. CAMURRI, A., MAZZARINO, B., RICCHETTI, M., TIMMERS, R., & VOLPE, G. (2004).Multimodal analysis of expressive gesture in music and dance performances. In A. Camurri & G.Volpe (Eds.), Lecture notes in artificial intelligence: Vol. 2915. Gesture-based communication in human-computer interaction (pp. 20-39). Berlin: Springer Verlag.
10. CAMURRI, A., MAZZARINO, B., & VOLPE, G. (2004). Analysis of expressive gesture: The Eyesweb expressive gesture processing library. In A. Camurri & G.Volpe (Eds.), Lecture notes in artificial intelligence: Vol. 2915. Gesture-based communication in human-computer interaction (pp. 460-467). Berlin: Springer Verlag.
11. CANAZZA S., DE POLI G., DRIOLI C., RODA A., & VIDOLIN A. (2000). Audio morphing different expressive intentions for multimedia systems. IEEE Multimedia Magazine, 7, 79-83.
12. CASTELLANO, G, VILLALBA, S. D., & CAMURRI, A. (2007). Recognising human emotions from body movement and gesture dynamics. In A. Paiva, R. Prada, & R.W. Picard (Eds.), Lecture notes in computer science: Vol. 4738. Affective computing and intelligent interaction, second international conference (pp. 71-82). Berlin: Springer Verlag.
13. CLARKE E. (1988). Generative principles in music performance. In J. A. Sloboda (Ed.), Generative processes in music: The psychology of performance, improvisation and composition, (pp. 1-26). New York: Oxford University Press.
14. COOKE, D. (1959). The language of music. London: Oxford University Press.
15. DAHL, S., & FRIBERG, A. (2007).Visual perception of expressiveness in musician's body movements. Music Perception, 24, 433-454.
16. DAVIDSON, J. W. (1994).What type of information is conveyed in the body movements of solo musician performers? Journal of Human Movement Studies, 6, 279-301.
17. DE MEIJER, M. (1989). The contribution of general features of body movement to the attribution of emotions. Journal of Nonverbal Behavior, 13, 247-268.
18. DE POLI G., RODA A., & VIDOLIN A. (1998). Note-by-note analysis of the influence of expressive intentions and musical structure in violin performance. Journal of New Music Research, 27, 293-321.
19. DI CARLO N. S., & GUAITELLA, I. (2004). Facial expressions of emotion in speech and singing. Semiotica, 149, 37-55.
20. EKMAN, P., & FRIESEN, W. V. (1974). Detecting deception from the body or face. Journal of Personality and Social Psychology, 29, 288-298.
21. FRIBERG, A. (1995). A quantitative rule system for musical performance. Unpublished doctoral dissertation, Royal Institute of Technology, Stockholm, Sweden.
22. GABRIELSSON, A., & JUSLIN, P. N. (1996). Emotional expression in music performance: Between the performer's intention and the listener's experience. Psychology of Music, 24, 68-91.
23. GABRIELSSON A., & JUSLIN P. (2003). Emotional expression in music. In R. Davidson, K. R. Scherer, & H. H. Goldsmith (Eds.), Handbook of affective sciences (pp. 503-534). New York: Oxford University Press.
24. GABRIELSSON, A., & LINDSTRÖM, E. (2001). The influence of musical structure on emotional expression. In P. N. Juslin & J. A. Sloboda (Eds.), Music and emotion: Theory and research (pp. 223-248). New York: Oxford University Press.
25. GUNES, H., & PICCARDI, M. (2007). Bi-modal emotion recognition from expressive face and body gestures. Journal of Network and Computer Applications, 30, 1334-1345.
26. JUSLIN, P. N. (1995). Emotional communication in music viewed through a Brunswikian lens. In G. Kleinen (Ed.), Musical expression. Proceedings of the conference of ESCOM and DGM 1995 (pp. 21-25). Bremen, Germany: University of Bremen.
27. JUSLIN, P. N. (1997a). Emotional communication in music performance: A functionalist perspective and some data. Music Perception, 14, 383-418.
28. JUSLIN, P. N. (1997b). Can results from studies of perceived expression in musical performances be generalized across response formats? Psychomusicology, 16, 77-101.
29. JUSLIN, P. N. (2000). Cue utilization in communication of emotion in music performance: Relating performance to perception. Journal of Experimental Psychology: Human Perception and Performance, 26, 1797-1813.
30. JUSLIN, P. N., & LAUKKA, P. (2004). Expression, perception, and induction of musical emotions: A review and a questionnaire study of everyday listening. Journal of New Music Research, 33, 217-238.
31. KAHOL, K., TRIPATHI, P., PANCHANATHAN, S., & RIKAKIS, T. (2003). Gesture segmentation in complex motion sequences. Proceedings IEEE International Conference in Image Processing (pp. 105-108), Barcelona, Spain.
32. LABAN, R. (1963). Modern educational dance. London: Macdonald & Evans Ltd.
33. LABAN, R., & LAWRENCE F. C. (1947). Effort. London: Macdonald & Evans Ltd.
34. LANGER, S. K. (1957). Philosophy in a new key (3rd ed.). Cambridge, MA: Harvard University Press.
35. LIVINGSTONE, S. R., & THOMPSON, W. F. (2006). Multimodal affective interaction. A comment on musical origin. Music Perception, 24, 89-94.
36. LUCK, G., & TOIVIAINEN, P. (2006). Ensemble musicians' synchronization with conductors' gestures: An automated feature-extraction analysis. Music Perception, 24, 189-199.
37. POLLICK, F. E., PATERSON, H., BRUDERLIN, A., & SANFORD, A. J. (2001). Perceiving affect from arm movement. Cognition, 82, B51-B61.
38. QUINLAN, R. (1993). C4.5: Programs for machine learning. San Mateo, CA: Morgan Kaufmann Publishers.
39. SCHERER, K. R. (1978). Personality inference from voice quality: The loud voice of extroversion. European Journal of Social Psychology, 8, 467-487.
40. SCHERER, K. R. (2001). Appraisal considered as a process of multi-level sequential checking. In K. R. Scherer, A. Schorr, & T. Johnstone (Eds.), Appraisal processes in emotion: Theory, methods, research (pp. 92-120). New York and Oxford: Oxford University Press.
41. SCHERER, K. R. (2003). Vocal communication of emotion: A review of research paradigms. Speech Communication, 40, 227-256.
42. SCHERER, K. R. (2004). Which emotions can be induced by music? What are the underlying mechanisms? And how can we measure them? Journal of New Music Research, 33, 239-251.
43. SCHERER, K. R., & WALLBOTT, H. G. (1985). Analysis of nonverbal behavior. In T. A. Van Dijk (Ed.), Handbook of discourse analysis (pp. 199-230). London: Academic Press.
44. SCHERER K. R., & ZENTNER M. R. (2001). Emotional effects of music: Production rules. In P. N. Juslin & J. A. Sloboda (Eds.), Music and emotion: Theory and research (pp. 361-392). Oxford: Oxford University Press.
45. SCHUBERT, E. (2002). Correlation analysis of continuous emotional response to music: Correcting for the effects of serial correlation. Musicae Scientiae, Special Issue 2001-2002, 213-236.
46. SLOBODA, J. A., & JUSLIN, P. N. (2001). Psychological perspectives on music and emotion. In P. N. Juslin & J. A. Sloboda (Eds.), Music and emotion: Theory and research (pp. 71-104). New York: Oxford University Press.
47. SUNDBERG, J., FRIBERG, A., & FRYDÉN, L. (1991). Common secrets of musicians and listeners: An analysis-by-synthesis study of musical performance. In P. Howell, R. West, & I. Cross (Eds.), Representing musical structure (pp. 161-197). London: Academic Press.
48. THOMPSON, W. F., SUNDBERG, J., FRIBERG, A., & FRYDÉN, L. (1989). The use of rules for expression in the performance of melodies. Psychology of Music, 17, 63-82.
49. THOMPSON W. F., GRAHAM P., & RUSSO F. A. (2005). Seeing music performance: Visual influences on perception and experience. Semiotica, 156, 203-227.
50. THOMPSON, W. F., RUSSO, F. A., & QUINTO, L. (2008). Audiovisual integration of emotional cues in song. Cognition and Emotion. Retrieved May 27, 2008, from http://www.informaworld.com/ smpp/ftinterface~content=a789794895~fulltext=713240930.
51. TIMMERS, R., MAROLT, M., CAMURRI, A., & VOLPE, G. (2006). Listeners' emotional engagement with performances of a skriabin etude: An explorative study. Psychology of Music, 34, 481-510.
52. VINES, B., WANDERLEY, M. M., NUZZO, R., LEVITIN, D., & KRUMHANSL, C. (2004). Performance gestures of musicians: What structural and emotional information do they convey? In A. Camurri & G.Volpe (Eds.), Lecture notes in artificial intelligence: Vol. 2915. Gesture-based communication in human-computer interaction (pp. 468-478). Berlin: Springer Verlag.
53. VINES, B., NUZZO, R. L., & LEVITIN, D.J. (2005). Analyzing temporal dynamics in music: Differential calculus, physics, and functional data analysis techniques. Music Perception, 23, 137-152.
54. WALLBOTT, H. G. (1998). Bodily expression of emotion. European Journal of Social Psychology, 28, 879-896.
55. WALLBOTT, H. G., & SCHERER, K. R. (1986). Cues and channels in emotion recognition. Journal of Personality and Social Psychology, 51 690-699.
56. WITTEN, I. H., & FRANK, E. (2005). Data mining: Practical machine learning tools and techniques (2nd ed.). San Francisco: Morgan Kaufmann.