A multimodal system for gesture recognition in interactive music performance

Computer Music Journal

Volumn 33, Issue 4, 2009, Pages 69-82

  Overholt, Dan ^a   Thompson, John ^b   Putnam, Lance ^c   Bell, Bo ^c   Kleban, Jim ^d   Sturm, Bob ^e   Kuchera Morin, JoAnn ^c  

^a AALBORG UNIVERSITY   (Denmark)

^b GEORGIA SOUTHERN UNIVERSITY   (United States)

^c Santa Barbara   (United States)

^d University of California   (United States)

^e INSTITUT JEAN LE ROND D ALEMBERT   (France)

Author keywords

[No Author keywords available]

Indexed keywords

ANALYSIS SYSTEM; AUDIO ANALYSIS; ELECTRIC FIELD SENSORS; INTERACTIVE MUSIC; MULTI-MODAL; MULTIMODAL SYSTEM; MUSICAL PERFORMANCE; NON-CONTACT SENSORS; SPECTRAL TRANSFORMATIONS;

COMPUTER VISION; SENSORS;

GESTURE RECOGNITION;

EID: 73649119197     PISSN: 01489267     EISSN: 15315169     Source Type: Journal    
DOI: 10.1162/comj.2009.33.4.69     Document Type: Article

References (34)

1. Bell, B., et al. 2007. "The Multimodal Music Stand." Proceedings of the 2007 International Conference on New Interfaces for Musical Expression. New York: Association for Computing Machinery, pp. 62-65.
2. Borchers, J., A. Hadjakos, and M. Muhlhauser. 2006. "MICON: A Music Stand for Interactive Conducting." Proceedings of the 2006 International Conference on New Interfaces for Musical Expression. New York: Association for Computing Machinery, pp. 254-259.
3. Boulanger, R., and M. Mathews. 1997. "The 1997 Mathews Radio-Baton and Improvisation Modes." Proceedings of the 1997 International Computer Music Conference." San Francisco, California: International Computer Music Association, pp. 395-398.
4. Cadoz, C., and M. Wanderley. 2000. "Gesture-Music." In M. Wanderley and M. Battier, eds. Trends in Gestural Control of Music. Paris: IRCAM, pp. 1-55.
5. Camurri, A., et al. 2004. "Multimodal Analysis of Expressive Gesture in Music and Dance Performances." In A. Camurri and G. Volpe, eds. Gesture-Based Communication in Human-Computer Interaction: Fifth International Gesture Workshop. Berlin: Springer-Verlag, pp. 20-39.
6. Camurri, A., et al. 2005. "Communicating Expressiveness and Affect in Multimodal Interactive Systems." IEEE Multimedia Magazine 12(1):43-53.
7. Gumtau, S., et al. 2005. "MEDIATE: A Responsive Environment Designed for Children with Autism." Proceedings of Accessible Design in the Digital World Conference. Dundee, Scotland: BCS, pp. 1-8.
8. Hewitt, D., and I. Stevenson. 2003. "E-Mic: Extended Mic-Stand Interface Controller." Proceedings of the 2003 International Conference on New interfaces For Musical Expression. New York: Association for Computing Machinery, pp. 122-128.
9. Jehan, T. 2007. "Tristan Jehan's Max/MSP Stuff. Available online at web.media.mit.edu/~tristan/maxmsp.html. Accessed November 2007.
10. Kapoor, A., and R. W. Picard. 2001. "A Real-Time Head Nod and Shake Detector." Proceedings from the 2001 Workshop on Perspective User Interfaces. New York: Association for Computing Machinery, pp. 49-54.
11. Kuchera-Morin, J., et al. 2005. "Out of the Ether: A System for Interactive Control of Virtual Performers Using Video and Audio Streams." In NSF Interactive Digital Multimedia IGERT Annual Research Review. Santa Barbara: University of California, Santa Barbara, pp. 13-17.
12. Levenson, T. 1994. "Taming the Hypercello." The Sciences 34(4):15-23.
13. Mathews, M. 1989. "The Radio Drum as a Synthesizer Controller." Proceedings of the 1989 International Computer Music Conference. San Francisco, California: International Computer Music Association, pp. 42-45.
14. McCartney, J. 2002. "Rethinking the Computer Music Language: SuperCollider." Computer Music Journal 26(4):61-68.
15. McNutt, E. 2003. "Performing Electroacoustic Music: A Wider View of Interactivity." Organised Sound 8(3):297-304.
16. Modler, P., and T. Myatt. 2004. "A Video System for Recognizing Gestures by Artificial Neural Networks for Expressive Musical Control." In Antonio Camurri, Gualtiero Volpe, eds. Lecture Notes on Computer Science. New York: Springer-Verlag, pp. 541-548.
17. Orio, N., S. Lemouton, and D. Schwarz. 2003. "Score Following: State of the Art and New Developments." Proceedings of the 2003 International Conference on New Interfaces For Musical Expression. New York: Association for Computing Machinery, pp. 36-41.
18. Overholt, D. 2006. "Musical Interaction Design with the CREATE USB Interface: Teaching HCI with CUIs Instead of GUIs." Proceedings of the 2006 International Computer Music Conference. San Francisco, California: International Computer Music Association, pp. 425-430.
19. Overholt, D. 2007. "Musical Interface Technology: Multimodal Control of Multidimensional Parameter Spaces for Electroacoustic Music Performance." Ph.D. Dissertation, University of California, Santa Barbara, Department of Media Arts and Technology.
20. Palacio-Quintin, C. 2003. "The Hyper-Flute." Proceedings of the 2003 International Conference on New Interfaces for Musical Expression. New York: Association for Computing Machinery, pp. 206-207.
21. Paradiso, J., and N. Gershenfeld. 1997. "Musical Applications of Electric Field Sensing." Computer Music Journal 26(2):69-89.
22. Pousset, D. 1992. "La Flûte-MIDI, L'histoire et Quelques Applications." Ph.D. Dissertation, University of Paris-Sorbonne.
23. Putnam, L. 2009a. "Gamma - Generic Synthesis C++ Library." Available online at mat.ucsb.edu/gamma. Accessed January 2009.
24. Putnam, L. 2009b. "Graphics Library of Views." Available online at mat.ucsb.edu/glv. Accessed January 2009.
25. Qian, G., et al. 2004. "A Gesture-Driven Multimodal Interactive Dance System." Proceedings of the IEEE International Conference on Multimedia and Expo, ICME 2004. New York: Institute of Electrical and Electronics Engineers, pp. 1579-1582.
26. Rose, R. T., F. Quek, and Y. Shi. 2004. "Macvissta: A System for Multimodal Analysis." Proceedings of the 6th International Conference on Multimodal Interfaces. New York: Association for Computing Machinery, pp. 259-264.
27. Rowe, R. 1993. Interactive Music Systems: Machine Listening and Composing. Cambridge, Massachusetts: MIT Press.
28. Smirnov, A. 2000. "Music and Gesture: Sensor Technologies in Interactive Music and the THEREMIN Based Space Control Systems." Proceedings of the 2000 International Computer Music Conference. San Francisco, California: International Computer Music Association, pp. 511-514.
29. Sonami, L. 2007. "Lady's Glove." Available online at www.sonami.net/ladyglove2.htm. Accessed January 2007.
30. Tanaka, A. 2000. "Musical Performance Practice on Sensor-Based Instruments." In M. Wanderley and M. Battier, eds. Trends in Gestural Control of Music. Paris: IRCAM, pp. 389-406.
31. Viola, P., and M. Jones. 2001. "Robust Real-Time Object Detection." Technical Report February 2001/01. Cambridge, Massachusetts: Compaq CRL.
32. Waisvisz, M. 1985. "The Hands: A Set of Remote MIDI-Controllers. " Proceedings of the 1985 International Computer Music Conference. San Francisco, California: International Computer Music Association, pp. 86-89.
33. Wanderley, M. 2001. "Interaction Musicien-Instrument: Application au Contrôle Gestuel de la Synthèse Sonore." Ph.D. Thesis, University of Paris 6.
34. Ystad, S., and Voiner, T. 2001. "A Virtually Real Flute." Computer Music Journal 25(2):13-24.