Designing, playing, and performing with a vision-based mouth interface

Proceedings of the International Conference on New Interfaces for Musical Expression

Volumn , Issue , 2003, Pages 116-121

  Lyons, Michael J ^a   Haehnel, Michael ^b   Tetsutani, Nobuji ^a  

^a ADVANCED TELECOMMUNICATIONS RESEARCH INSTITUTE INTERNATIONAL   (Japan)

^b RWTH AACHEN UNIVERSITY   (Germany)

Author keywords

Alternative input devices; Mouth controller; Video based interface

Indexed keywords

COMPUTER CONTROL SYSTEMS; MUSIC;

ALTERNATIVE INPUT DEVICES; COMPUTER VISION ALGORITHMS; FACIAL ACTION; MINIATURE CAMERAS; MOUTH CONTROLLERS; MUSICAL SOUNDS; NON-VERBAL COMMUNICATIONS; SPEECH PRODUCTION; VIDEO-BASED INTERFACE; VISION BASED;

SPEECH COMMUNICATION;

EID: 85160343723     PISSN: 22204792     EISSN: 22204806     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (26)

1. Camurri, A. and Trocca, R., Movement and Gesture in Intelligent Interactive Music Systems, in Wanderley, M., and Baffier, M. (Eds.), Trends in Gestural Control of Music CD-ROM, IRCAM, Paris, 2000, pp. 450-474.
2. Card, S.K., Mackinlay, and Robertson, G.G., A Morphological Analysis of the Design Space of Input Devices, ACM Transactions on Information Systems 9(2): 99-122, 1991.
3. Cook, P. and Scavone, G. P., The Synthesis ToolKit (STK), in Proc. of the 1999 International Computer Music Conference, Beijing, International Computer Music Association.
4. Duda, R.O. and Hart, P.E., Pattern Classification and Scene Analysis. John Wiley & Sons, 1973.
5. Dudley, H., Remaking Speech. Journal of the Acoustical Society of America, 11: 169-177, 1939.
6. EyesWeb Project, http://www.eyesweb.org.
7. Fels, S.S. and Mase, K., Iamascope: A graphical musical instrument. Computers and Graphics, 2(23):277-286, 1999.
8. Gillespie, B., Haptics in Music, in Cognition, and Computerized Sound, P. Cook (Ed.). MIT Press, Cambridge, MA, 1999.
9. Hunt, A., Wanderley, M.M., and Paradis, M., The Importance of Parameter Mapping in Electronic Instrument Design, in Proc of the 2002 Conference on New Interfaces for Musical Expression (NIME-02), 149-154.
10. Keyence Japan CK-200B miniature CCD camera: http://www.keyence.co.jp/hobby/ccd/index.html.
11. Lyons, M.J., Budynek, J., and Akamatsu, S., Automatic Classification of Single Facial Images, IEEE PAMI 21(12): 1357-1362, 1999.
12. Lyons, M.J., Campbell, R. et al., The Noh Mask Effect: Vertical Viewpoint Dependence of Facial Expression Perception, Proc. Roy. Soc. Lond. B 267: 2239-2245, 2000.
13. Lyons, M. J., and Tetsutani, N., Facing the Music: A Facial Action Controlled Musical Interface. in CHI 2001 Extended Abstracts, ACM Press, 309-310.
14. McGurk, H. and MacDonald, J., Hearing Lips and Seeing Voices, Nature, 264:746-748, 1976.
15. Ng, K., Interactive Gesture Music Performance Interfaces, in Proc of the 2002 Conference on New Interfaces for Musical Expression (NIME-02), 183-184.
16. Orio, N., A Gesture Interface Controlled by the Oral Cavity, in Proc. Of the 1997 International Computer Music Conference, San Francisco, International Computer Music Association, 141-144.
17. Pantic, M. and Rothkrantz, L.J.M., Automatic Analysis of Facial Expressions: The State of the Art, IEEE PAMI 22: 1424-1445, 2000.
18. Paradiso, J. and Sparacino, F., Optical Tracking for Music and Dance Performance, in "Optical 3-D Measurement Techniques IV", A. Gruen, H. Kahmen eds., Herbert Wichmann Verlag, Heidelberg Germany, pp. 11-18 (1997).
19. Penfield, W. & Rasmussen, T., The cerebral cortex of man: aclinical study of localization of function, Macmillan, New York, 1950.
20. Poupyrev, I., Augmented Groove: Collaborative Jamming in Augmented Reality, in SIGGRAPH 2000 Conference Abstracts and Applications, ACM Press, p. 77.
21. Rizzolatti, G, Gadiga, L., Gallese, V, and Fogassi, L., Premotor Cortex and the Recognition of Motor Actions, Cognitive Brain Research, 3: 131-141, 1996.
22. David Rokeby, Very Nervous System: http://www.interlog.com/~drokeby/vns.html
23. STEIM, BigEye: http://www.steim.nl/bigeye.html
24. Tarabella, L., Gestural and Visual Approaches to Performance, in Wanderley, M., & Baffier, M. (Eds.), Trends in Gestural Control of Music CD-ROM, IRCAM, Paris, 2000, pp. 604-615.
25. Varela, F.J., Thompson, E., and Rosch, E., The Embodied Mind, MIT Press, Cambridge MA, 1991.
26. Vogt, F., McCaig, G., Ali, M.A., Fels, S., Tongue ‘n’ Groove: An Ultrasound based Music Controller, in Proc of the 2002 Conference on New Interfaces for Musical Expression (NIME-02), 60-6.