Gesture in performance with traditional musical instruments and electronics: Use of embodied music cognition and multimodal motion capture to design gestural mapping strategies

ACM International Conference Proceeding Series

Volumn , Issue , 2014, Pages 100-105

  Visi, Federico ^a   Schramm, Rodrigo ^b   Miranda, Eduardo ^a  

^a UNIVERSITY OF PLYMOUTH   (United Kingdom)

^b FEDERAL UNIVERSITY OF RIO GRANDE   (Brazil)

Author keywords

Embodied music cognition; Expressiveness; Gesture; Guitar; Mapping; Motion capture; Multimodal

Indexed keywords

ARTS COMPUTING; MUSICAL INSTRUMENTS;

EMBODIED MUSIC COGNITIONS; EXPRESSIVENESS; GESTURE; GUITAR; MOTION CAPTURE; MULTI-MODAL;

MAPPING;

EID: 84903977619     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2617995.2618013     Document Type: Conference Paper

References (38)

1. A. Camurri, B. Mazzarino, and G. Volpe. Analysis of expressive gesture: The eyesweb expressive gesture processing library. In A. Camurri and G. Volpe, editors, Gesture Workshop, volume 2915 of Lecture Notes in Computer Science, pages 460-467. Springer, 2003.
2. A. Camurri, G. D. Poli, M. Leman, and G. Volpe. A multi-layered conceptual framework for expressive gesture applications. In Proceedings of MOSART: Workshop on Current Directions in Computer Music, pages 29-34, 2001.
3. A. Camurri, G. Volpe, G. De Poli, and M. Leman. Communicating expressiveness and affect in multimodal interactive systems. MultiMedia, IEEE, 12(1):43-53, Jan 2005.
4. L. Costalonga. Biomechanical Modeling of Musical Performace: A Case Study of the Guitar. PhD thesis, School of Computing, Communications and Electronics, University of Plymouth, 2009.
5. A. Cox. The mimetic hypothesis and embodied musical meaning. Musicæ Scientiæ, 5(2), 2001.
6. S. Dahl, F. Bevilacqua, R. Bresin, M. Clayton, L. Leante, I. Poggi, and N. Rasamimanana. Gesture in performance. In R. I. Godøy and M. Leman, editors, Musical Gestures: Sound, Movement, and Meaning. Routledge, 2010.
7. A. Di Scipio. 'Sound is the interface': from interactive to ecosystemic signal processing. Organised Sound, 8, 2003.
8. M. Elmezain, A. Al-Hamadi, and B. Michaelis. Hand gesture spotting based on 3d dynamic features using hidden markov models. In D. Ślezak, S. Pal, B.-H. Kang, J. Gu, H. Kuroda, and T.-h. Kim, editors, Signal Processing, Image Processing and Pattern Recognition, volume 61 of Communications in Computer and Information Science, pages 9-16. Springer Berlin Heidelberg, 2009.
9. D. Fenza, L. Mion, S. Canazza, and A. Rodà. Physical movement and musical gestures: A multilevel mapping strategy. In Proceedings of Sound and Music Computing Conference, Salerno, 2005.
10. G. A. Fink. Markov Models for Pattern Recognition: From Theory to Applications. Springer-Verlag, Berlin Heidelberg, 2008.
11. J. J. Gibson. The theory of affordances. In Perceiving, Acting, and Knowing. Erlbaum, 1977.
12. R. I. Godøy. Motor-Mimetic Music Cognition. Leonardo, 36(4):317-319, Aug. 2003.
13. R. I. Godøy. Gestural affordances of musical sound. In R. I. Godøy and M. Leman, editors, Musical gestures: Sound, movement, and meaning. Routledge, 2010.
14. R. I. Godøy, A. R. Jensenius, and K. Nymoen. Chunking in music by coarticulation. Acta Acoustica united with Acoustica, 96(4):690-700, 2010.
15. R. I. Godøy and M. Leman, editors. Musical Gestures: Sound, Movement and Meaning. Routledge, 2010.
16. A. Gritten and E. King. Music and Gesture. Ashgate, 2006.
17. A. Gritten and E. King. New Perspectives on Music and Gesture. SEMPRE studies in the psychology of music. Ashgate, 2011.
18. N. Hirose. An ecological approach to embodiment and cognition. Cognitive Systems Research, 3(3):289-299, 2002.
19. A. D. Hunt, M. M. Wanderley, and M. Paradis. The importance of Parameter Mapping in Electronic Instrument Design. In Proceedings of the International Conference on New Interfaces for Musical Expression, pages 88-93, 2002.
20. A. R. Jensenius, M. M. Wanderley, R. I. Godøy, and M. Leman. Musical gestures: Concepts and methods in research. In R. I. Godøy and M. Leman, editors, Musical gestures: Sound, movement, and meaning, pages 12-35. Routledge, 2010.
21. J. M. Kilner and R. N. Lemon. What we know currently about mirror neurons. Current Biology, 23(23):R1057-62, 2013.
22. O. Lähdeoja, M. M. Wanderley, and J. Malloch. Instrument augmentation using ancillary gestures for subtle sonic effects. In Proceedings of the SMC 2009 - 6th Sound and Music Computing Conference, 2009.
23. M. Leman. Embodied Music Cognition and Mediation Technology. MIT Press, 2008.
24. M. Leman. Music, gesture, and the formation of embodied meaning. In R. I. Godøy and M. Leman, editors, Musical Gestures: Sound, Movement and Meaning, pages 126-153. Routledge, 2010.
25. M. Leman. Musical gestures and embodied cognition. In T. Dutoit, T. Todoroff, and N. d'Alessandro, editors, Actes des Journées d'Informatique Musicale (JIM 2012), pages 5-7, Mons, Belgique, 9-11 mai 2012. UMONS/numediart.
26. M. Lesaffre, M. Leman, K. Tanghe, B. D. Baets, H. D. Meyer, and J.-P. Martens. User-dependent taxonomy of musical features as a conceptual framework for musical audio-mining technology. In Proceedings of the Stockholm Music Acoustics Conference, pages 635-638, 2003.
27. P.-J. Maes, M. Leman, K. Kochman, M. Lesaffre, and M. Demey. The "one-person choir": A multidisciplinary approach to the development of an embodied human-computer interface. Computer Music Journal, 35(2):22-35, 2011.
28. P.-J. Maes, C. Palmer, M. Leman, and M. Wanderley. Action-based effects on music perception. Frontiers in Psychology, 4, 2014.
29. J. R. Matyja and A. Schiavio. Enactive music cognition: Background and research themes. Constructivist Foundations, 8(3):351-357, 2013.
30. E. R. Miranda and M. Wanderley. New Digital Musical Instruments: Control And Interaction Beyond the Keyboard (Computer Music and Digital Audio Series). A-R Editions, Inc., Madison, WI, USA, 2006.
31. M. Müller. Information Retrieval for Music and Motion. Springer Verlag, 2007.
32. T. Murray-Browne, D. Mainstone, N. Bryan-Kinns, and M. D. Plumbley. The Medium is the Message: Composing Instruments and Performing Mappings. In Proceedings of the International Conference on New Interfaces for Musical Expression, pages 56-59, 2011.
33. L. Nijs, M. Lesaffre, and M. Leman. The musical instrument as a natural extension of the musician. In M. Castellango and H. Genevois, editors, Proceedings of the 5th Conference of Interdisciplinary Musicology, pages 132-133. LAM-Institut jean Le Rond d'Alembert, 2009.
34. B. Repp and Y.-H. Su. Sensorimotor synchronization: A review of recent research (2006-2012). Psychonomic Bulletin & Review, 20(3):403-452, 2013.
35. M. Schutz and S. Lipscomb. Hearing gestures, seeing music: vision influences perceived tone duration. Perception, 36(6):888-97, 2007.
36. H. Smith and R. Dean. Practice-Led Research, Research-Led Practice in the Creative Arts. Research Methods for the Arts and Humanities. Edinburgh University Press, 2009.
37. V. Verfaille, M. M. Wanderley, and P. Depalle. Mapping strategies for gestural and adaptive control of digital audio effects. Journal of New Music Research, 35(1):71-93, 2006.
38. M. Waisvisz. Manager or Musician? About virtuosity in live electronic music. Do we operate our electronic systems or do we play them? In Proceedings of the International Conference on New Interfaces for Musical Expression, page 415, 2006.