Sonify your face: Facial expressions for sound generation

MM'10 - Proceedings of the ACM Multimedia 2010 International Conference

Volumn , Issue , 2010, Pages 1363-1371

  Valenti, Roberto ^a   Jaimes, Alejandro ^b   Sebe, Nicu ^c  

^a UNIVERSITY OF AMSTERDAM   (Netherlands)

^b YAHOO RESEARCH   (Spain)

^c UNIVERSITY OF TRENTO   (Italy)

Author keywords

affective computing; facial expressions; facial therapy interface; gesture based interaction; multimodal interface; sonification

Indexed keywords

AFFECTIVE COMPUTING; FACIAL EXPRESSIONS; FACIAL THERAPY INTERFACE; GESTURE-BASED INTERACTION; MULTI-MODAL INTERFACES; SONIFICATIONS;

BAYESIAN NETWORKS; FEATURE EXTRACTION; GESTURE RECOGNITION; HUMAN COMPUTER INTERACTION; INFERENCE ENGINES; INSTALLATION; INTERACTIVE COMPUTER SYSTEMS;

FACE RECOGNITION;

EID: 78650973696     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1873951.1874219     Document Type: Conference Paper

References (18)

1. C. Dobrian, and D. Koppelman, "The 'E' in NIME: Musical Expression with New Computer Interfaces," NIME 2006, Paris, France.
2. P. Ekman and W. Friesen. Facial Action Coding System: Investigator's Guide. Consulting Psychologists Press, 1978.
3. B. Fasel and J. Luettin, "Automatic facial expression analysis: A survey," Pattern Recognition, 36:259-275, 2000.
4. M. Funk, K. Kuwabara, and M. J. Lyons, "Sonification of Facial Actions for Musical Expression", in proc., Intl. Conference on New Interfaces for Musical Expression (NIME05), Vancouver, BC, Canada, June, 2005.
5. M. Gurevich, J. Trevino, "Expression and Its Discontents: Toward and Ecology of Musical Creation," NIME 2007.
6. A.J. Hornof, T. Rogers, and T. Halverson, "EyeMusic: Performing live music and multimedia compositions with eye movements," NIME 2007: Conference on New Interfaces for Musical Expression. In proceedings on pp. 299-300.
7. J. Kim, G. Schiemer, and T. Narushima, "Coulog: Playing with Eye Movements", NIME 2007, New York City, USA.
8. M. J. Lyons, M. Haehnel, "Designing, Playing, and Performing with a Vision-based Mouth Interface," NIME 03, Montreal, Canada.
9. N. Moody, N. Fells, and N. Bailey, "Ashitaka: an audiovisual Instrument," NIME 2007.
10. S. Oore, "Learning Advanced Skills on New Instruments," NIME 2005.
11. M. Pantic and L. Rothkrantz. "Automatic analysis of facial expressions: The state of the art," IEEE Transactions on PAMI, 22(12):1424-1445, 2000.
12. Puredata (http://www.puredata.org)
13. G. C. de Silva, T. Smyth, M. J. Lyons, "A Novel Face-tracking Mouth Controller and its Application to Interacting with Bioacoustic Models," NIME 2004.
14. R. Valenti, A. Jaimes and N. Sebe, "Facial Expression Recognition As a Creative Interface". In ACM International Conference on Intelligent User Interfaces, 2008.
15. R. Valenti, A. Jaimes and N. Sebe, "Facial Expression Recognition As a Creative Interface". In ACM International Conference on Intelligent User Interfaces, 2008.
16. J. Nielsen, Usability Engineering. Academic Press, 1999.
17. Y. Visell and J.R. Cooperstock, "Modeling and Continuous Sonification of Affordances for Gesture-Based Interfaces," 13th Intl. Conf. on Auditory Display, Montreal, Canada, June 2007.
18. H. Tao and T.S. Huang, "Connected Vibrations: A Modal Analysis Approach to Non-rigid Motion Tracking," IEEE Conf. on Computer Vision and Pattern Recognition, 1998.