The MEGA project: Analysis and synthesis of multisensory expressive gesture in performing art applications

Journal of New Music Research

Volumn 34, Issue 1, 2005, Pages 5-21

  Camurri, Antonio ^a   De Poli, Giovanni ^b   Friberg, Anders ^c   Leman, Marc ^d   Volpe, Gualtiero ^a  

^a UNIVERSITY OF GENOA   (Italy)

^b UNIVERSITY OF PADOVA   (Italy)

^c GHENT UNIVERSITY   (Belgium)

^d ROYAL INSTITUTE OF TECHNOLOGY   (Sweden)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 33646190755     PISSN: 09298215     EISSN: 17445027     Source Type: Journal    
DOI: 10.1080/09298210500123895     Document Type: Review

References (68)

1. Argyle, M. (1980). Bodily communication. London: Methuen.
2. Bonini, F. & Rodà, A. (2001). Expressive content analysis of musical gesture: An experiment on piano improvisation. Paper presented at the Workshop on Current Research Directions in Computer Music, Barcelona.
3. Boone, R.T. & Cunningham, J.G. (1998). Children's decoding of emotion in expressive body movement: The development of cue attunement. Developmental Psychology, 34, 1007-1016.
4. Bresin, R. & Battel, G.U. (2000). Articulation strategies in expressive piano performance: Analysis of legato, staccato and repeated notes in performances of the Andante movement of Mozart's sonata in G major (K 545). Journal of New Music Research, 29(3), 211-224.
5. Bresin, R. & Friberg, A. (2000). Emotional coloring of computer-controlled music performances. Computer Music Journal, 24(4), 44-63.
6. Bresin, R., De Poli, G. & Ghetta, R. (1995a). A fuzzy formulation to KTH performance rules system. Paper presented at the 2nd International Conference on Acoustic and Musical Research (CIARM 95), Ferrara.
7. Bresin, R., De Poli, G. & Ghetta, R. (1995b). A fuzzy approach to performance rules. Paper presented at the XI Conference on Musical Informatics, Bologna.
8. Bresin, R., De Poli, G. & Ghetta, R. (1995c). Fuzzy performance rules. Paper presented at the KTH Symposium on Grammars for Music Performance, Stockholm.
9. Camurri, A. & Coglio, A. (1998). An architecture for emotional agents. IEEE Multimedia, October-December, 24-33.
10. Camurri, A., Hashimoto, S., Ricchetti, M., Trocca, R., Suzuki, K. & Volpe, G. (2000). EyesWeb: Toward gesture and affect recognition in interactive dance and music systems. Computer Music Journal, 24(1), 57-69.
11. Camurri, A., Mazzarino, B., Volpe, G., Morasso, P., Priano, F. & Re, C. (2003a). Application of multimedia techniques in the physical rehabilitation of Parkinson's patients. Journal of Visualization and Computer Animation, 14(5), 269-278.
12. Camurri, A., Lagerlöf, I. & Volpe, G. (2003b). Recognizing emotion from dance movement: Comparison of spectator recognition and automated techniques. International Journal of Human-Computer Studies, 59(1-2), 213-225.
13. Camurri, A., Mazzarino, B. & Volpe, G. (2004a). Expressive interfaces. Cognition, Technology & Work, 6(1), 15-22.
14. Camurri, A., Mazzarino, B. & Volpe, G. (2004b). Analysis of expressive gesture: The EyesWeb expressive gesture processing library. In: A. Camurri & G. Volpe (Eds.), Gesture-based communication in human-computer interaction. Berlin: Springer Verlag.
15. Camurri, A., Mazzarino, B., Ricchetti, M., Timmers, R. & Volpe, G. (2004c). Multimodal analysis of expressive gesture in music and dance performances. In: A. Camurri & G. Volpe (Eds.), Gesture-based communication in human-computer interaction. Berlin: Springer Verlag.
16. Camurri, A., Krumhansl, C.L., Mazzarino, B. & Volpe, G. (2004d). An exploratory study of anticipating human movement in dance. Paper presented at the 2nd International Symposium on Measurement, Analysis and Modeling of Human Functions, Genova.
17. Camurri, A., Coletta, P., Massari, A., Mazzarino, B., Peri, M., Ricchetti, M., Ricci, A. & Volpe, G. (2004e). Toward real-time multimodal processing: EyesWeb 4.0. Paper presented at the AISB 2004 Convention: Motion, Emotion and Cognition, Leeds.
18. Camurri, A., De Poli, G., Leman, M. & Volpe, G. (2005). Toward communicating expressiveness and affect in multimodal interactive systems for performing art and cultural applications. IEEE Multimedia, 72(1), 43-53.
19. Canazza, S., De Poli, G., Drioli, C., Rodà, A. & Vidolin, A. (2000). Audio morphing different expressive intentions for multimedia systems. IEEE Multimedia, 7(3), 79-84.
20. Canazza, S., De Poli, G., Mion, L., Rodà, A., Vidolin, A. & Zanon, A. (2003a). Espressive classifier at CSC: An overview of the main research streams. Paper presented at the XIV Conference on Musical Informatics, Firenze.
21. Canazza, S., De Poli, G., Rodà, A. & Vidolin A. (2003b). An abstract control space for communication of sensory expressive intentions in music performance. Journal of New Music Research, 52(3), 281-294.
22. Canazza, S., Friberg, A., Roda, A. & Zanon, P. (2003c). Expressive Director: A system for the real-time control of music performance synthesis. Paper presented at the Stockholm Music Acoustics Conference (SMAC 03), Stockholm.
23. Canazza, S., De Poli, G., Drioli, C., Rodà, A. & Vidolin, A. (2004). Modeling and control of expressiveness in music performance. Proceedings of the IEEE, 92(4), 286-701.
24. Chi, D., Costa, M., Zhao, L. & Badler, N. (2000). The EMOTE model for effort and shape. Paper presented at ACM SIGGRAPH'00, New Orleans.
25. Cirotteau, D., De Poli, G., Mion, L., Vidolin, A. & Zanon, P. (2004). Recognition of musical gestures in known pieces and in improvisations. In: A. Camurri & G. Volpe (Eds.), Gesture-based communication in human-computer interaction. Berlin: Springer Verlag.
26. Cowie, R., Douglas-Cowie, E., Tsapatsoulis, N., Votsis, G., Kollias, S., Fellenz, W. & Taylor, J. (2001). Emotion recognition in human-computer interaction. IEEE Signal Processing Magazine, 18(1), 32-80.
27. Dahl, S. & Friberg, A. (2004). Expressiveness of musician's body movements in performances on marimba. In: A. Camurri & G. Volpe (Eds.), Gesture-based communication in human-computer interaction. Berlin: Springer Verlag.
28. De Gotzen, A. (2004). Enhancing engagement in multimodality environments by sound movements in a virtual space. IEEE Multimedia, 11(2), 4-8.
29. De Meijer, M. (1989). The contribution of general features of body movement to the attribution of emotions. Journal of Nonverbal Behavior, 13, 247-268.
30. De Poli, G. (2003). Analysis and modeling of expressive intentions in music performance. Annals of the New York Academy of Sciences, 999, 118-123.
31. Dittrich, W.H., Troscianko, T., Lea, S.E.G. & Morgan, D. (1996). Perception of emotion from dynamic point-light displays represented in dance. Perception, 25, 727-738.
32. Friberg, A. & Sundberg, J. (1999). Does music performance allude to locomotion? A model of final ritardandi derived from measurements of stopping runners. Journal of the Acoustical Society of America, 105, 1469-1484.
33. Friberg, A., Sundberg, J. & Frydén, L. (2000a). Music from motion: Sound level envelopes of tones expressing human locomotion. Journal of New Music Research, 29(3), 199 - 210.
34. Friberg, A., Colombo, V., Frydén, L. & Sundberg, J. (2000b). Generating musical performances with Director Musices. Computer Music Journal, 24(3), 23-29.
35. Gabrielsson, A. & Lindström, E. (2001). The influence of musical structure on emotion. In: P.N. Juslin & J.A. Sloboda (Eds.), Music and emotion: theory and research. New York: Oxford University Press.
36. Hashimoto, S. (1997). KANSEI as the third target of information processing and related topics in Japan. In A. Camurri (Ed.), Proceedings of the International Workshop on KANSEI: The Technology of Emotion. Geneva: AIMI (Italian Computer Music Association)/DIST-University of Geneva.
37. Hevner, K. (1936). Experimental studies of the elements of expression in music. American Journal of Psychology, 48, 246-268.
38. Johansson, G. (1973). Visual perception of biological motion and a model for its analysis. Perception and Psychophysics, 14, 201-211.
39. Juslin, P.N. (1997a). Emotional communication in music performance: A functionalist perspective and some data. Music Perception, 14, 383-418.
40. Juslin, P.N. (1997b). Perceived emotional expression in synthesized performances of a short melody: Capturing the listener's judgment policy. Musicae Scientiae, 1, 225-256.
41. Juslin, P.N. (2000). Cue utilization in communication of emotion in music performance: Relating performance to perception. Journal of Experimental Psychology: Human Perception and Performance, 26(1), 797-813.
42. Juslin, P.N. (2001). Communicating emotion in music performance: A review and a theoretical framework. In: Juslin, P.N. & Sloboda, J.A. (Eds.), Music and emotion: theory and research. New York: Oxford University Press.
43. Juslin, P.N. & Madison, G. (1999). The role of timing patterns in recognition of emotional expression from musical performance. Music Perception, 17, 197-221.
44. Juslin, P.N. & Laukka, J. (2003). Communication of emotions in vocal expression and music performance: Different channels, same code? Psychological Bulletin, 129(5), 770-814.
45. Kurtenbach, G. & Hulteen, E. (1990). Gestures in humancomputer communication. In: B. Laurel (Ed.), The art and science of interface design. New York: Addison-Wesley.
46. Laban, R. (1963). Modern educational dance. London: Macdonald & Evans.
47. Laban, R. & Lawrence, F.C. (1947). Effort. London: Macdonald & Evans.
48. Leman, M., Lesaffre, M. & Tanghe, K. (2001). A toolbox for perception-based music analysis. Ghent: Ghent University, IPEM-Department of Musicology. Available online at: www.ipem.ugent.be/toolbox.
49. Leman, M., Vermeulen, V., De Voogdt, L., Taelman, J., Moelants, D. & Lesaffre, M. (2004). Correlation of gestural musical audio cues and perceived expressive qualities. In: A. Camurri & G. Volpe (Eds.), Gesturebased communication in human-computer interaction. Berlin: Springer Verlag.
50. Lesaffre, M., Leman, M., Tanghe, K., De Baets, B., De Meyer, H. & Martens, J.-P. (2003). User-dependent taxonomy of musical features as a conceptual framework for musical audio-mining technology. Paper presented at the Stockholm Music Acoustics Conference, Stockholm.
51. Lindström, E. & Friberg, A. (2002). Perceived tension in music: A computational model. Paper presented at the 7th International Conference on Music Perception and Cognition, Sydney.
52. Lucas, B. & Kanade, T. (1981). An iterative image registration technique with an application to stereo vision. Paper presented at the International Joint Conference on Artificial Intelligence.
53. Mion, L. (2003). Application of Bayesian networks to automatic recognition of expressive content of piano improvisations. Paper presented at the Stockholm Music Acoustics Conference, Stockholm.
54. Picard, R. (1997). Affective computing. Cambridge, MA: MIT Press.
55. Pollick F.E. (2004). The features people use to recognize human movement style. In: A. Camurri & G. Volpe (Eds.), Gesture-based communication in human-computer interaction. Berlin: Springer Verlag.
56. Pollick, F.E., Paterson, H., Bruderlin, A. & Sanford, A.J. (2001). Perceiving affect from arm movement. Cognition, 82, B51-B61.
57. Rinman, M.-L., Friberg, A., Bendiksen, B., Cirotteau, D., Dahl, S., Kjellmo, I., Mazzarino, B. & Camurri, A. (2004). Ghost in the Cave: An interactive collaborative game using non-verbal communication. In: A. Camurri & G. Volpe (Eds.), Gesture-based communication in human-computer interaction. Berlin: Springer Verlag.
58. Rowe, R. (1993). Interactive music systems: machine listening and composition. Cambridge, MA: MIT Press.
59. Rowe, R. (2001). Machine musicianship. Cambridge, MA: MIT Press.
60. Russell, J.A. (1980). A circumplex model of affect. Journal of Personality and Social Psychology, 39, 1161 - 1178.
61. Schaeffer, P. (1977). Traité des objets musicaux, 2nd edn. Paris: Editions du Seuil.
62. Scherer, K.R. (2003). Why music does not produce basic emotions: Pleading for a new approach to measuring the emotional effects of music. Paper presented at the Stockholm Music Acoustics Conference, Stockholm.
63. Schoonderwaldt, E. & Friberg, A. (2001). Toward a rule-based model for violin vibrato. In: Proceedings of the Workshop on Current Research Directions in Computer Music. Barcelona: Pompeu Fabra University, Audiovisual Institute.
64. Sloboda, J.A. & Juslin, P.N. (Eds.) (2001). Music and emotion: theory and research. Oxford: Oxford University Press.
65. Walk, R.D. & Homan, C.P. (1984). Emotion and dance in dynamic light displays. Bulletin of the Psychonomic Society, 22, 437-440.
66. Wallbott, H.G. (1980). The measurement of human expressions. In: W. von Rallfer-Engel (Ed.), Aspects of communications. Wallbott: Swets & Zeitlinger.
67. Zanon, P. & De Poli, G. (2003). Time-varying estimation of parameters in rule systems for music performance. Journal of New Music Research, 52(3), 295-316.
68. Zhao, L. (2001). Synthesis and acquisition of Laban movement analysis qualitative parameters for communicative gestures. Doctoral dissertation, University of Pennsylvania.