Sonification of facial actions for musical expression

Proceedings of the International Conference on New Interfaces for Musical Expression

Volumn , Issue , 2005, Pages 127-131

  Funk, Mathias ^a   Kuwabara, Kazuhiro ^a   Lyons, Michael J ^a  

^a ADVANCED TELECOMMUNICATIONS RESEARCH INSTITUTE INTERNATIONAL   (Japan)

Author keywords

Facial expression; Facial therapy interface; Gesture based interaction; Video based musical interface

Indexed keywords

MUSIC;

DESIGN IMPLEMENTATION; FACIAL ACTION; FACIAL EXPRESSIONS; FACIAL THERAPY INTERFACE; GESTURE-BASED INTERACTION; MUSICAL EXPRESSION; NON-VERBAL COMMUNICATIONS; SOCIAL INTERACTIONS; SONIFICATIONS; VIDEO-BASED MUSICAL INTERFACE;

FACE RECOGNITION;

EID: 79959595862     PISSN: 22204792     EISSN: 22204806     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (23)

1. [1] Barron, J. L., Fleet, D. J. and Beauchemin, S. S. Performance of optical flow techniques. The International Journal of Computer Vision, 12(1):43-77, 1994.
2. Beauchemin, S. S. and Barron, J. L. The computation of optical flow. ACM Computing Surveys, 27:433-467, 1995.
3. Blaine, T. and Fels, S. Contexts of Collaborative Musical Experiences. 3rd International Conference on New Interfaces for Musical Expression (NIME03). pp. 129-134.2003.
4. Cutler, R. and Turk, M. View-based interpretation of realtime optical flow for gesture recognition. In Proceedings, Face and Gesture Recognition, pp. 416-421, 1998.
5. Hjelmas, E. and Low, B. K. Face Detection: A survey. Computer Vision and Image Understanding vol. 83 no. 3, pp. 216-235 (2001).
6. Horn, B. K. P. and Schunk, B. G. Determining optical flow. Artificial Intelligence, 17:185-203, 1981.
7. Hornof, A. and Sato, L. EyeMusic: Making Music with the Eyes. In Proceedings of the 2004 Conference on New Interfaces for Musical Expression (NIME04), Hamamatsu, Japan, June 3-5, 185-188.
8. Kurenniemi, E. The Dawn of DIMI (DVD) Edited by Mika Taanila, Producer Kinotar, (http://www.kinotar.com/), Helsinki. Available from Mego (http://www.mego.at).
9. Lyons, M. J. Facial Gesture Interfaces for Expression and Communication. IEEE International Conference on Systems, Man and Cybernetics 2004).
10. th IEEE Conference on Advanced Learning Technologies (2004)
11. Lyons, M. J., Haehnel, M. and Tetsutani, N. Designing, Playing, and Performing with a Vision-Based Mouth Interface. In Proceedings of the Conference on New Interfaces for Musical Expression, pp. 116-121, 2003.
12. Lyons, M. J., and Tetsutani, N. Facing the Music: A Facial Action Controlled Musical Interface. In CHI 2001 Extended Abstracts, pp. 309-310.
13. Mase, K. Recognition of facial expression from optical flow. Institute of electronics information and communication engineers Transactions, vol. E74, pp. 3474--3483, 1991.
14. Merrill, D. Head-tracking for gestural and continuous control of parametrized audio effects. In Proceedings of the 2003 Conference on New Interfaces for Musical Expression (NIME-03), pp. 218-219.
15. Ng, K. Interactive Gesture Music Performance Interfaces. In Proceedings of the 2002 Conference on New Interfaces for Musical Expression (NIME-02), pp. 183-184.
16. Ng, K. Music via Motion: Transdomain Mapping of Motion and Sound for Interactive Performances. IEEE Proceedings vol. 92, no. 4: pp. 645 – 655, 2004.
17. Orio, N. A Gesture Interface Controlled by the Oral Cavity. In Proceedings of the 1997 International Computer Music Conference, San Francisco, International Computer Music Association, 141-144.
18. Porta, M. Vision-based user interface methods and applications. Int. J. Human-Computer Studies 57, 27–73, 2002.
19. Salem, C. and Zhai, S. An Isometric Tongue Pointing Device, In Proceedings of CHI'97, 538-539.
20. Shklovsky, V. Theory of Prose Dalkey Archive Press (1991).
21. Thompson, E. Empathy and Consciousness, Journal of Consciousness Studies vol. 8, no. 5-7: 1-32, 2001.
22. Viola, P. and Jones, M. Rapid object detection using a boosted cascade of simple features. In CVPR 2001.
23. Vogt, F., McCaig, G., Ali, M. A. and Fels, S. Tongue ‘n’ Groove: An Ultrasound based Music Controller. In Proceedings of the 2002 Conference on New Interfaces for Musical Expression (NIME-02), pp. 60-66