Multimodal human-computer interaction for immersive visualization: Integrating speech-gesture recognitions and augmented reality for indoor environments

Proceedings of the Seventh IASTED International Conference on Computer Graphics and Imaging

Volumn , Issue , 2004, Pages 171-176

  Malkawi, Ali M ^a   Srinivasan, Ravi S ^a  

^a UNIVERSITY OF PENNSYLVANIA   (United States)

Author keywords

Building simulation; CFD; Gesture recognition; Multimodal HCI; Speech recognition

Indexed keywords

COMPUTATIONAL FLUID DYNAMICS; COMPUTER SIMULATION; DATA HANDLING; GESTURE RECOGNITION; GRAPHICAL USER INTERFACES; SENSORS; SPEECH RECOGNITION; VIRTUAL REALITY; WIRELESS TELECOMMUNICATION SYSTEMS;

AUGMENTED REALITY (AR) VISUALIZATION; BUILDING SIMULATION; DATA MANIPULATION; MULTIMODAL HUMAN COMPUTER INTERACTION (HCI);

HUMAN COMPUTER INTERACTION;

EID: 10444230419     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (33)

1. R.Azuma, A Survey of Augmented Reality. Presence: Teleoperators and Virtual Environments 5(4), 1997, 355-385.
2. W. Ribarsky, E. Ayers, J. Eble & S. Mukerjee, Using Glyph-maker to create customized visualizations of complex data. Computer 27(7), 1994, 57-64.
3. Teylingen, W. Ribarsky & C. van Der Mast, Virtual Data Visualizer. IEEE Transactions on Visualization and Computer Graphics 3(1), 1997, 65-74.
4. nd Australasian Computer Science Conf., 1999, 122-133.
5. M. Pilgrim, D. Bouchlaghem, D. Loveday & M. Holmes, A mixed reality system for building form and data representation, IEEE, 2001, 369-375.
6. Van Dam, Andries, D.H. Laidlaw & R.M. Simpson, Experiments in immersive VR for Scientific Visualization. Computer & Graphics 26, 2002, 535-555.
7. T. Wilde, J.A. Kohl & R.E. Flanery Jr., Immersive and 3D viewers for CUMULVS: VTK/CAVE and AVS/Express. Future Generation Computer Systems 19, 2003, 701-719.
8. S. Vogt, A. Khamene, F. Sauer, A. Keil & H. Niemann, A high performance AR system for medical applications. Proceedings of second IEEE and ACM International Symposium on mixed and AR, 2003.
9. S. Bryson, Virtual Wind Tunnel: A high-performance virtual reality application. Proceedings of IEEE Annual Virtual Reality International Symposium, 1993, 20-25.
10. T. Impelluso, Physically based virtual reality in a distributed environment. Computer Graphics 30(4), 1996, 60-61.
11. T.M. Wasfy & A.K. Noor, Visualization of CFD results in immersive Virtual Environments. Advances in Engineering Software 32, 2001, 717-730.
12. A. Malkawi & R. Choudhary, Visualizing the sensed environment in the real world. Journal of Human-Environment Systems, 3(1), 1999, 61-69.
13. V. Shahnawaz, J. Vance & S. Kitti, Visualization of post-processed CFD data in a Virtual Environment. Proceedings of DETC99, ASME Design Engg. Technical Conf., 1999, 1-7.
14. E. Linden, J. Hellstrom, M. Cehlin & M. Sandberg, Virtual Reality presentation of temperature measurements on a diffuser for displacement ventilation. Proceedings of Fourth International Conf. Indoor Air Quality and Energy Conservation in Buildings, Changsha, Hunan V II, 2001, 849-856.
15. A. Malkawi, Immersive Building Simulation, chapter in (eds) A. Malkawi & G. Augenbroe, Advanced Building Simulation (Taylor Press, 2004).
16. Hewett, Baecker, Card, Carey, Gasen, Mantei, Perlman, Strong & Verplank, HCI. ACM SIGCHI curricula for HCI, 1996.
17. R. Cipolla & A. Pentland (eds), Computer Vision for Human-Machine Interaction. (Cambridge University Press, Cambridge, 1998).
18. V.K. Pavlovic, R. Sharma & T.S. Huang, Gestural interface to a visual computing environment for molecular biologist. Proceedings of International Conf. Auto. Face Gesture Recognition, 1996, 30-34.
19. W.T. Freeman, K. Tanakan, J. Ohta & K. Kyuma, Computer Vision for computer games. Proceedings of International Conf. Autom. Face Gesture Recognition, 1996, 100-105.
20. R. Kadodayshi, K. Nishimoto & K. Mase, Design and evaluation of an immersive walk-through application for exploring cyberspace. Proceedings of Third International Conf. Autom. Face Gesture Recognition, 1998, 534-539.
21. R. Liang & M. Ouhyoung, A real-time gesture recognition system for sign language. Proceedings of Third International Conf. Autom. Face Recognition, 1998, 558-567.
22. R. Kjeldsen & J. Kender. Finding skin in color images. Proceedings Intl. Conf. on Automatic Face and Gesture Recognition, 1996, 312-317.
23. Immersion Corporation, website accessed on September 8,2003. URL: http://www.immersion.com
24. C.W. Ng & S. Ranganath, Gesture recognition via pose classification. Proceedings of the International Conf. Pattern Recognition ICPR 2000.
25. L.R. Karl, M. Pettey, & B. Shneiderman, Speech versus mouse commands for word processing: an empirical evaluation. International Journal of Man-machine Studies 39, 1993, 667-687.
26. T.M. Wasfy & A.K. Noor, Rule-based natural-language interface for VE. Advances in Engineering Software 33, 2002, 155-168.
27. T. Sowa & I. Wachsmuth, Interpretation of Shape-related Iconic gestures in VEs. Gesture and Sign Language in HCI, International Gesture Workshop, GW 2001, 21-23.
28. R. Sharma, M. Yeasin, N. Krahnstoever, I. Rauschert, G. Cai, I. Brewer, A. MacEachren & K. Sengupta, Speech-Gesture driven multimodal interfaces for crisis Management. Speech-Gesture Driven Multimodal Interfaces for Crisis Management, 2000.
29. MicroStrain Inc., website accessed on August 8, 2003. URL: http://www.microstrain.com
30. Virtual Technologies Inc., website accessed on August 2003. URL: http://www.immersion.com
31. Fluent Inc., website accessed on June 10, 2003. URL: http://www.fluent.com
32. Ascension Technologies Inc., website accessed on August 10,2003. URL: http://www.ascension-tech.com
33. Nvision Inc., website accessed on June 24, 2003. URL: http://www.nvisionindustries.com