Exploiting prosodie structuring of coverbal gesticulation

ICMI'04 - Sixth International Conference on Multimodal Interfaces

Volumn , Issue , 2004, Pages 105-112

  Kettebekov, Sanshzar ^a  

^a Advanced Interfaces Inc   (United States)

Author keywords

Gesture; Multimodal; Prosody; Speech

Indexed keywords

DATA PROCESSING; FACE RECOGNITION; HUMAN COMPUTER INTERACTION; IMAGE SEGMENTATION; MARKOV PROCESSES; MATHEMATICAL MODELS; PERTURBATION TECHNIQUES; SPEECH; SPEECH RECOGNITION;

GESTURES; HIDDEN MARKOV MODELS (HMM); MULTIMODAL; PROSODY;

GESTURE RECOGNITION;

EID: 14944348289     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (42)

1. G. Rigoll, A. Kosmala, and S. Eickeler, "High Performance Real-Time Gesture Recognition Using Hidden Markov Models," in Gesture and Sign Language in Human-Computer Interaction, vol. LNAI 1371, M. Fröhlich, Ed., 1997, pp. 69-80.
2. M. Assan and K. Grobel, "Video-Based Sign Language Recognition Using Hidden Markov Models," in Gesture and Sign Language in Human-Computer Interaction, vol. LNAI 1371, I. Wachsmuth and M. Fröhlich, Eds., 1997, pp. 97-111.
3. R. Sharma, J. Cai, S. Chakravarthy, I. Poddar, and Y. Sethi, "Exploiting Speech/Gesture Co-occurrence for Improving Continuous Gesture Recognition in Weather Narration," In proc. of International Conference on Face and Gesture Recognition (FG'2000), Grenoble, France, 2000.
4. M. Yeasin and S. Chaudhuri, "Visual understanding of dynamic hand gestures," Pattern Recognition, vol. 33, pp. 1805-1817, 2000.
5. A. D. Wilson and A. F. Bobick, "Hidden Markov Models for Modeling and Recognizing Gesture Under Variation," in Hidden Markov Models: Applications in Computer Vision., T. Caelli, Ed.: World Scientific, 2001, pp. 123-160.
6. H. Lee and J. H. Kim, "An HMM-based Threshold Model Approach for Gesture Recognition," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 21, pp. 961-973, 1999.
7. A. F. Bobick and A. D. Wilson, "A State-Based Approach to the Representation and Recognition of Gesture," IEEE Trans on Pattern Analysis and Machine Intelligence, vol. 19, pp. 1325-1327, 1997.
8. F. G. Hofmann, P. Heyer, and G. Hommel, "Velocity Profile Based Recognition of Dynamic Gestures with Discrete Hidden Markov Models," in Gesture and Sign Language in Human-Computer Interaction, vol. LNAI 1371, M. Fröhlich, Ed., 1997, pp. 81-96.
9. A. Kendon, "Gesticulation and speech: Two aspects of the process of the utterance," in The relation between verbal and non-verbal communication, M. R. Key, Ed. Hague: Mouton, 1980, pp. 207-227.
10. D. McNeill, Hand and Mind: The University of Chicago Press, Chicago IL, 1992.
11. S. Kettebekov and R. Sharma, "Toward Natural Gesture/Speech Control of a Large Display," in Engineering for Human Computer Interaction, vol. 2254, Lecture Notes in Computer Science, M. R. Little and L. Nigay, Eds. Berlin Heidelberg New York: Springer Verlag, 2001, pp. 133-146.
12. N. Krahnstoever, S. Kettebekov, M. Yeasin, and R. Sharma, "A Real-Time Framework for Natural Multimodal Interaction with Large Screen Displays," In proc. of International Conference on Multimodal Interfaces, Pittsburgh, USA, 2002, pp. 349-354.
13. I. Rauschert, P. Agrawal, S. Fuhrmann, I. Brewer, R. Sharma, G. Cai, and A. MacEachren, "Designing a Human-Centered, Multimodal GIS Interface to Support Emergency Management," In proc. of The 10th ACM International Symposium on Advances in Geographic Information Systems, McLean, VA, 2002.
14. R. Sharma, M. Yeasin, N. Krahnstoever, I. Rauschert, G. Cai, A. MacEachren, K. Sengupta, and I. Brewer, "Speech-Gesture Driven Multimodal Interfaces for Crisis Management," Proceedings of IEEE special issue on Multimodal Human-Computer Interface, 2003.
15. E. Kaiser, A. Olwal, D. McGee, H. Benko, A. Corradini, X. Li, P. Cohen, and S. Feiner, "Mutual Disambiguation of 3D Multimodal Interaction in Augmented and Virtual Reality," In proc. of International Conference on Multimodal Interfaces, Vancouver, Canada, 2003, pp. 12-19.
16. D. McNeill, "Gesture and Language Dialectic," Acta Linguistica Hafniesia, 2002.
17. M. W. Alibali, S. Kita, and A.J.Young, "Gesture and the process of speech production: We think, therefore we gesture," Language and cognitive processes, vol. 15, pp. 593-613, 2000.
18. D. F. Armstrong, W. C. Stokoe, and S. E. Wilcox, Gesture and the Nature of Language: Cambridge University Press, 1995.
19. A. Kendon, "Do gestures communicate?: A review," Research on Language and Social Interaction, vol. 27, pp. 175-200, 1994.
20. R. Coulston, K. Krout, and P. R. Cohen, "Multimodal Interaction During Multiparty Dialogues: Initial Results," In proc. of 4th IEEE International Conference on Multimodal Interfaces, Pittsburgh, PA, 2002.
21. S. Oviatt, R. Coulston, S. Tomko, B. Xiao, R. Linsford, R. M. Wesson, and L. Carmichael, "Toward a Theory of Organized Multimodal Integration Pattern During Human-Computer Interaction," In proc. of 5th International Conference on Multimodal Interfaces, Vancouver, Canada, 2003, pp. 44-51.
22. S. Kettebekov, M. Yeasin, and R. Sharma, "Improving Continuous Gesture Recognition with Spoken Prosody," In proc. of IEEE Conference on Computer Vision and Pattern Recognition (CVPR'03), Madison, Wisconsin, 2003, vol. 1, pp. 565-570.
23. R. Kompe, Prosody in speech understanding systems. Berlin ; New York: Springer, 1997.
24. M. E. Beckman, "The parsing of prosody," Language and Cognitive Processes, vol. 11, pp. 17-67, 1996.
25. B. Butterworth and U. Hadar, "Gesture, speech, and computational stages: A reply to McNeill," Psychological Review, vol. 96, pp. 168-174, 1989.
26. R. M. Krauss, Y. Chen, and P. Chawla, "Nonverbal behavior and nonverbal communication: What do conversational hand gestures tell us?," in Advances in experimental social psychology, M. Zanna, Ed. San Diego, CA: Academic Press, 1996, pp. 389-450.
27. R. M. Krauss, "Why do we gesture when we speak?," Current Directions in Psychological Science, vol. 7, pp. 54-59, 1998.
28. J.-P. de Ruiter, "Gesture and speech production," in Series in Psycholinguistics. Nijmegen, The Netherlands: MPI, 1998.
29. D. McNeill, "Gesture and Language Dialectic," Acta Linguistica Hafniensia, 2002.
30. Y. Xiong, F. Quek, and D. McNeill, "Hand Gesture Symmetric Behavior Detection in Natural Conversation," In proc. of International Conference Multimodal Interaction (ICMI 02), Pittsburg, PA, pp. 179-184.
31. F. Quek and Y. Xiong, "Oscillatory Gestures and Discourse," In proc. of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), Hong Kong, 2002.
32. S. Kettebekov, M. Yeasin, N. Krahnstoever, and R. Sharma, "Prosody Based Co-analysis of Deictic Gestures and Speech in Weather Narration Broadcast," In proc. of Workshop on Multimodal Resources and Multimodal System Evaluation. (LREC 2002), Las Palmas, Spain, 2002, pp. 57-62.
33. M. Fröhlich and I. Wachsmuth, "Gesture Recognition of the Upper Limbs - From Signal to Symbol," in Gesture and Sign Language in Human-Computer Interaction, vol. LNAI 1371, I. W. M. Fröhlich, Ed. Berlin: Springer, 1998, pp. 173-184.
34. Y. Azoz, L. Devi, and R. Sharma, "Reliable tracking of human arm dynamics by multiple cue integration and constraint fusion," In proc. of IEEE Conference on Computer Vision and Pattern Recognition, 1998.
35. P. Boersma and D. Weenink, "PRAAT," 4.0 ed. Amsterdam, NL: Institute of Phonetic Sciences. University of Amsterdam, NL, 2002.
36. J. N. Holmes, "Mechanisms and Models of Human Speech Production," in Speech Synthesis and Recognition Aspects of Information Technology. Berkshire: Van Nostrand Reinhold, 1988.
37. I. Lehiste, Suprasegmentals. Cambridge, Massachusetts: MIT Press, 1970.
38. J. E. Atkinson, "Correlation Analysis of the Physiological Factors Controlling Fundamental Voice Frequency," Journal of the Acoustical Society of America, vol. 63, pp. 211-222, 1978.
39. J. Godfrey and J. N. Brodsky, "Acoustic Correlates of Emphasis," Journal of the Acoustical Society of America, vol. 80, 1986.
40. I. Titze, Principles of Voice Production. Englewood Cliffs: Prentice-Hall, 1994.
41. A. Adami, "Modeling Prosodic Differences for Speaker and Language Recognition," OGI School of Science and Engineering: Oregon Health and Science University, Beaverton, OR, Doctorate Thesis, 2004.
42. J. D. Brand, J. S. D. Mason, and S. Colomb, "Visual Speech: Physiological or Behavioural Biometric?," Lecture Notes in Computer Science, vol. 2091, pp. 157-162, 2001.