An assistive bi-modal user interface integrating multi-channel speech recognition and computer vision

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 6762 LNCS, Issue PART 2, 2011, Pages 454-463

  Karpov, Alexey ^a   Ronzhin, Andrey ^a   Kipyatkova, Irina ^a  

^a ST PETERSBURG INSTITUTE FOR INFORMATICS AND AUTOMATION   (Russian Federation)

Author keywords

assistive technology; cognitive experiments; computer vision; Multi modal user interface; speech recognition

Indexed keywords

ASSISTIVE; ASSISTIVE TECHNOLOGY; AUTOMATIC RECOGNITION; COGNITIVE EXPERIMENTS; COMPUTER INPUT DEVICES; CONTACT LESS; CONTROL COMMAND; HEAD DETECTION; HEAD POSITION; HUMAN BEING; MULTI-CHANNEL; MULTI-MODAL; MULTICHANNEL AUDIO; MULTIMODAL USER INTERFACE; NATURAL SPEECH; PHYSICAL DISABILITY; POINTING MOVEMENT; REFERENCE OBJECTS; SEMANTIC INFORMATION; SPATIAL LOCATION; SPEECH COMMANDS; SPEECH SIGNALS; VIRTUAL MOUSE; VISION BASED; VOICE COMMAND; POINTING INTERFACES;

AUDIO SIGNAL PROCESSING; COMPUTER VISION; HUMAN COMPUTER INTERACTION; KNOWLEDGE MANAGEMENT; MAMMALS; SEMANTICS; USER INTERFACES; VIRTUAL REALITY; SIGNAL PROCESSING; SPEECH;

SPEECH RECOGNITION;

EID: 79960298793     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/978-3-642-21605-3_50     Document Type: Conference Paper

References (25)

1. Oviatt, S.: Multimodal interfaces. In: Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies and Emerging Applications, pp. 286-304. Lawrence Erlbaum Assoc., Mahwah (2003)
2. Bolt, R.A.: Put-that-there: Voice and gesture at the graphics interface. Computer Graphics 14(3), 262-270 (1980)
3. Malkewitz, R.: Head Pointing and Speech Control as a Hands-Free Interface to Desktop Computing. In: 3rd International ACM Conference on Assistive Technologies ASSETS 1998, Marina del Rey, CA, USA, pp. 182-188. ACM Press, New York (1998)
4. Gorodnichy, D., Roth, G.: Nouse 'Use your nose as a mouse' perceptual vision technology for hands-free games and interfaces. Image and Vision Computing 22(12), 931-942 (2004)
5. Harada, S., Landay, J.A., Malkin, J., Li, X., Bilmes, J.A.: The Vocal Joystick: Evaluation of voice-based cursor control techniques. In: 8th International ACM SIGACCESS Conference on Computers & Accessibility ASSETS 2006, Portland, USA, pp. 197-204. ACM Press, New York (2006)
6. Eiichi, I.: Multi-modal interface with voice and head tracking for multiple home appliances. In: 8th IFIP International Conference on Human-Computer Interaction INTERACT 2001, Tokyo, Japan, pp. 727-728 (2001)
7. Karpov, A., Ronzhin, A.: ICANDO: Low Cost Multimodal Interface for Hand Disabled People. Journal on Multimodal User Interfaces 1(2), 21-29 (2007)
8. Ronzhin, A., Karpov, A.: Russian Voice Interface. Pattern Recognition and Image Analysis: Advances in Mathematical Theory and Applications 17(2), 321-336 (2007)
9. Rabiner, L., Juang, B.: Speech Recognition. In: Benesty, J., et al. (eds.) Springer Handbook of Speech Processing. Springer, New York (2008)
10. Krim, H., Viberg, M.: Two decades of array signal processing research: the parametric approach. Signal Processing Magazine 13(4), 67-94 (1996)
11. Ronzhin, A., Karpov, A., Kipyatkova, I., Železný, M.: Client and Speech Detection System for Intelligent Infokiosk. In: Sojka, P., Horák, A., Kopeček, I., Pala, K. (eds.) TSD 2010. LNCS, vol. 6231, pp. 560-567. Springer, Heidelberg (2010)
12. Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: IEEE International Conference on Computer Vision and Pattern Recognition Conference CVPR 2001, Kauai, HI, USA (2001)
13. Lienhart, R., Maydt, J.: An Extended Set of Haar-like Features for Rapid Object Detection. In: IEEE International Conference on Image Processing ICIP 2002, Rochester, New York, USA, pp. 900-903 (2002)
14. Lucas, B.D., Kanade, T.: An Iterative Image Registration Technique with an Application to Stereo Vision. In: 7th International Joint Conference on Artificial intelligence IJCAI, Vancouver, Canada, pp. 674-679 (1981)
15. Bouguet, J.-Y.: Pyramidal Implementation of the Lucas-Kanade Feature Tracker Description of the algorithm. Intel Corporation Microprocessor Research Labs, Report, New York, USA (2000)
16. Karpov, A., Ronzhin, A., Cadiou, A.: A Multi-Modal System ICANDO: Intellectual Computer AssistaNt for the Disabled Operators. In: INTERSPEECH International Conference, Pittsburgh, PA, USA, pp. 1998-2001. ISCA Association (2006)
17. ISO 9241-9:2000(E) Ergonomic Requirements for Office Work with Visual Display Terminals (VDTs), Part 9: Requirements for Non-Keyboard Input Devices, International Standards Organization (2000)
18. Soukoreff, R.W., MacKenzie, I.S.: Towards a standard for pointing device evaluation, perspectives on 27 years of Fitts' law research in HCI. International Journal of Human Computer Studies 61(6), 751-789 (2004)
19. Zhang, X., MacKenzie, I.S.: Evaluating Eye Tracking with ISO 9241 - Part 9. In: Jacko, J.A. (ed.) HCI 2007. LNCS, vol. 4552, pp. 779-788. Springer, Heidelberg (2007)
20. Carbini, S., Viallet, J.E.: Evaluation of contactless multimodal pointing devices. In: 2nd IASTED International Conference on Human-Computer Interaction, Chamonix, France, pp. 226-231 (2006)
21. De Silva, G.C., Lyons, M.J., Kawato, S., Tetsutani, N.: Human Factors Evaluation of a Vision-Based Facial Gesture Interface. In: International Workshop on Computer Vision and Pattern Recognition for Computer Human Interaction, Madison, USA (2003)
22. Wilson, A., Cutrell, E.: FlowMouse: A computer vision-based pointing and gesture input device. In: Costabile, M.F., Paternó, F. (eds.) INTERACT 2005. LNCS, vol. 3585, pp. 565-578. Springer, Heidelberg (2005)
23. Ward, D., Blackwell, A., MacKay, D.: Dasher: A data entry interface using continuous gestures and language models. In: ACM Symposium on User Interface Software and Technology UIST 2000, pp. 129-137. ACM Press, New York (2000)
24. SPIIRAS Speech and Multimodal Interfaces Web-site, TV demonstration, http://www.spiiras.nw.ru/speech/demo/ort.avi
25. SPIIRAS Speech and Multimodal Interfaces Web-site, demonstration 2, http://www.spiiras.nw.ru/speech/demo/demo-new.avi