Designing transition networks for multimodal VR-interactions using a markup language

Proceedings - 4th IEEE International Conference on Multimodal Interfaces, ICMI 2002

Volumn , Issue , 2002, Pages 411-416

  Latoschik, M E ^a  

^a BIELEFELD UNIVERSITY   (Germany)

Author keywords

multimodal integration; multimodal interaction; transition networks; Virtual Reality; XML interaction representation

Indexed keywords

COMPUTATIONAL LINGUISTICS; SEMANTICS; USER INTERFACES; VIRTUAL REALITY;

APPLICATION CONTEXTS; AUGMENTED TRANSITION NETWORKS; CORE COMPONENTS; MULTI-MODAL INTERACTIONS; MULTIMODAL INTEGRATION; SEMANTIC RELATIONS; SIMULATION ENGINE; TARGET STRUCTURE;

INTERACTIVE COMPUTER SYSTEMS;

EID: 84963799143     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICMI.2002.1167030     Document Type: Conference Paper

References (28)

1. K. Böhm, W. Broll, and M. Sokolewicz. Dynamic gesture recognition using neural networks; a fundament for advanced interaction construction. In S. Fisher, J. Merrit, and M. Bolan, editors, Stereoscopic Displays and Virtual Reality Systems, SPIE Conference Electronic Imaging Science & Technology, volume 2177, San Jose, USA, 1994.
2. K. Böhm, W. Hübner, and K. Väänänen. Given: Gesture driven interactions in virtual environments; a toolkit approach to 3D interactions. In Interfaces to Real and Virtual Worlds, 1992.
3. R. A. Bolt. Put-That-There: Voice and gesture at the graphics interface. In ACM SIGGRAPH - Computer Graphics, New York, 1980. ACM Press.
4. R. Carey, G. Bell, and C. Marrin. ISO/IEC 14772-1:1997 virtual reality modeling language (VRML). Technical report, The VRML Consortium Incorporated, 1997.
5. J. Cassell. A framework for gesture generation and interpretation. In R. R. Cipolla and A. Pentland, editors, Computer Vision for Human-Machine Interaction. Cambridge University Press, 1998.
6. M. Cavazza, X. Pouteau, and D. Pernel. Multimodal communication in virtual environments. In Symbiosis of Human and Artifact, pages 597-604. Elsevier Science B. V., 1995.
7. P. Figueroa, M. Green, and H. J. Hoover. 3dml: A language for 3d interaction techniques specification. In Proceedings of the EUROGRAPHICS 2001, 2001.
8. A. Hauptmann and P. McAvinney. Gestures with speech for graphic manipulation. International Journal of Man-Machine Studies, 38:231-249, 1993.
9. M. Johnston. Unification-based multimodal parsing. In Proceedings of the 17th International Conference on Computational Linguistics and the 36th Annual Meeting of the Association for Computational Linguistics COLING-ACL, 1998.
10. M. Johnston and S. Bangalore. Finite-state multimodal parsing and understanding. In Proceedings of COLING 2000, Saarbrücken, Gemany, 2000.
11. M. Johnston and S. Bangalore. Finite-state methods for multimodal parsing and integration. In Finite-state Methods Workshop, ESSLLI Summer School on Logic Language and Information,Helsinki, Finland, august 2001.
12. M. Johnston, P. R. Cohen, D. McGee, S. L. Oviatt, J. A. Pittman, and I. Smith. Unification-based multimodal integration. In 35th Annual Meeting of the Association for Computational Linguistics, Madrid, pages 281-288, 1997.
13. D. Koons, C. Sparrel, and K. Thorisson. Intergrating simultaneous input from speech, gaze and hand gestures. In Intelligent Multimedia Interfaces. AAAI Press, 1993.
14. M. E. Latoschik. A general framework for multimodal interaction in virtual reality systems: PrOSA. In W. Broll and L. Schäfer, editors, The Future of VR and AR Interfaces - Multimodal, Humanoid, Adaptive and Intelligent. Proceedings of the Workshop at IEEE Virtual Reality 2001, number 138 in GMD report, pages 21-25, 2001.
15. M. E. Latoschik. Multimodale Interaktion in Virtueller Realität am Beispiel der virtuellen Konstruktion. PhD thesis, Technische Fakultät, Universität Bielefeld, 2001.
16. B. Lenzmann. Benutzeradaptive und multimodale Interface-Agenten. PhD thesis, Technische Fakultät, Universität Bielefeld, 1998.
17. M. Lucente, G.-J. Zwart, and A. D. George. Visualization space: A testbed for deviceless multimodal user interface. In Intelligent Environments Symposium, American Assoc. for Artificial Intelligence Spring Symposium Series, Mar. 1998.
18. M. T. Maybury. Research in multimedia an multimodal parsing and generation. In P. McKevitt, editor, Journal of Artificial Intelligence Review: Special Issue on the Integration of Natural Language and Vision Processing, volume 9, pages 2-27. Kluwer Academic Publishers Group, 1993.
19. J. G. Neal and S. C. Shapiro. Intelligent User Interfaces, chapter Intelligent Multi-Media Interface Technology, pages 11-45. Addison-Wesley Publishing Company, 1991.
20. J.-L. Nespoulous and A. Lecours. Gestures: Nature and function. In J.-L. Nespoulous, P. Rerron, and A. Lecours, editors, The Biological Foundations of Gestures: Motor and Semiotic Aspects. Lawrence Erlbaum Associates, Hillsday N.J., 1986.
21. S. Oviatt, P. Cohen, L. Wu, J. Vergo, L. Duncan, B. Suhm, J. Bers, T. Holzman, T. Winograd, J. Landay, J. Larson, and D. Ferro. Designing the user interface for multimodal speech and pen-based gesture applications: State-of-the-art systems and future research directions. In Proceedings of the HCI2000, 2000.
22. C. J. Sparrell and D. B. Koons. Interpretation of coverbal depictive gestures. In AAAI Spring Symposium Series, pages 8-12. Stanford University, March 1994.
23. H. Tramberend. A distributed virtual reality framework. In Virtual Reality, 1999.
24. M. Vo and C. Wood. Building an application framework for speech and pen input integration in multimodal learning interfaces. In Proceedings of International Conference on Acoustics, Speech, and Signal Processing, 1996.
25. http://www.web3d.org. WWW.
26. D. Weimer and S. Ganapathy. Interaction techniques using hand tracking and speech recognition. In M. Blattner and R. Dannenberg, editors, Multimedia Interface Design, pages 109-126. ACM Press, 1992.
27. http://www.w3.org/tr/xtnd. WWW.
28. http://www.techfak.uni-bielefeld.de/techfak/ags/wbski/wbski-engl.html. WWW.