Mudra: A unified multimodal interaction framework

ICMI'11 - Proceedings of the 2011 ACM International Conference on Multimodal Interaction

Volumn , Issue , 2011, Pages 97-104

  Hoste, Lode ^a   Dumas, Bruno ^a   Signer, Beat ^a  

^a VRIJE UNIVERSITEIT BRUSSEL   (Belgium)

Author keywords

declarative programming; multimodal fusion; multimodal interaction; rule language

Indexed keywords

ABSTRACTION LEVEL; DATA STREAM; DECLARATIVE PROGRAMMING; HIGH LEVEL SEMANTICS; INTERACTION STYLES; LEVELS OF ABSTRACTION; MULTI-MODAL INTERACTIONS; MULTI-MODAL INTERFACES; MULTIMODAL FUSION; RULE LANGUAGE; RULE-BASED LANGUAGE; SOFTWARE ENGINEERING PRINCIPLES;

ABSTRACTING; COMPUTER PROGRAMMING; DATA COMMUNICATION SYSTEMS; INNOVATION; SEMANTICS; SOFTWARE ENGINEERING; USER INTERFACES;

INTERACTIVE COMPUTER SYSTEMS;

EID: 83455176812     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2070481.2070500     Document Type: Conference Paper

References (21)

1. R. A. Bolt. "Put-That-There": Voice and Gesture at the Graphics Interface. In Proc. of SIGGRAPH 1980, 7th Annual Conference on Computer Graphics and Interactive Techniques, pages 262-270, Seattle, USA, 1980.
2. J. Chai, P. Hong, and M. Zhou. A Probabilistic Approach to Reference Resolution in Multimodal User Interfaces. In Proc. of IUI 2004, 9th International Conference on Intelligent User Interfaces, pages 70-77, Funchal, Madeira, Portugal, 2004.
3. J. Coutaz, L. Nigay, D. Salber, A. Blandford, J. May, and R. Young. Four Easy Pieces for Assessing the Usability of Multimodal Interaction: The CARE Properties. In Proc. of Interact 1995, International Conference on Human-Computer Interaction, pages 115-120, Lillehammer, Norway, June 1995.
4. P. Dietz and D. Leigh. DiamondTouch: A Multi-User Touch Technology. In Proc. of UIST 2001, 14th Annual ACM Symposium on User Interface Software and Technology, pages 219-226, Orlando, USA, 2001.
5. B. Dumas, D. Lalanne, and S. Oviatt. Multimodal Interfaces: A Survey of Principles, Models and Frameworks. Human Machine Interaction: Research Results of the MMI Program, pages 3-26, March 2009.
6. F. Echtler, M. Huber, and G. Klinker. Hand Tracking for Enhanced Gesture Recognition on Interactive Multi-Touch Surfaces. Technical Report TUM-I0721, Technische Universität München, Department of Computer Science, November 2007.
7. C. L. Forgy. Rete: A Fast Algorithm for the Many Pattern/Many Object Pattern Match Problem. Artificial Intelligence, 19(1):17-37, 1982.
8. M. Johnston and S. Bangalore. Finite-State Methods for Multimodal Parsing and Integration. In Proc. of ESSLLI 2001, 13th European Summer School in Logic, Language and Information, Helsinki, Finland, August 2001.
9. M. Johnston, P. Cohen, D. McGee, S. Oviatt, J. Pittman, and I. Smith. Unification-Based Multimodal Integration. In Proc. of ACL 1997, 35th Annual Meeting of the Association for Computational Linguistics, pages 281-288, Madrid, Spain, July 1997.
10. W. König, R. Rädle, and H. Reiterer. Squidy: A Zoomable Design Environment for Natural User Interfaces. In Proc. of CHI 2009, ACM Conference on Human Factors in Computing Systems, pages 4561-4566, Boston, USA, 2009.
11. D. Lalanne, L. Nigay, P. Palanque, P. Robinson, J. Vanderdonckt, and J. Ladry. Fusion Engines for Multimodal Input: A Survey. In Proc. of ICMI-MLMI 2009, International Conference on Multimodal Interfaces, pages 153-160, Cambridge, Massachusetts, USA, September 2009.
12. S. Oviatt. Advances in Robust Multimodal Interface Design. IEEE Computer Graphics and Applications, 23(5):62-68, September 2003.
13. S. Oviatt. Multimodal Interfaces. In The Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies and Emerging Applications, Second Edition, pages 286-304. Lawrence Erlbaum Associates, 2007.
14. E. Petajan, B. Bischoff, D. Bodoff, and N. Brooke. An Improved Automatic Lipreading System to Enhance Speech Recognition. In Proc. of CHI 1988, ACM Conference on Human Factors in Computing Systems, pages 19-25, Washington, USA, June 1988.
15. C. Scholliers, L. Hoste, B. Signer, and W. D. Meuter. Midas: A Declarative Multi-Touch Interaction Framework. In Proc. of TEI 2011, 5th International Conference on Tangible, Embedded and Embodied Interaction, pages 49-56, Funchal, Portugal, January 2011.
16. M. Serrano, L. Nigay, J. Lawson, A. Ramsay, R. Murray-Smith, and S. Denef. The OpenInterface Framework: A Tool for Multimodal Interaction. In Proc. of CHI 2008, ACM Conference on Human Factors in Computing Systems, Florence, Italy, April 2008.
17. R. Sharma, V. Pavlovic, and T. Huang. Toward Multimodal Human-Computer Interface. Proceedings of the IEEE, 86(5):853-869, 1998. (Pubitemid 128720228)
18. T. Sowa, M. Fröhlich, and M. Latoschik. Temporal Symbolic Integration Applied to a Multimodal System Using Gestures and Speech. In Proc. of GW 1999, International Gesture Workshop on Gesture-Based Communication in Human-Computer Interaction, pages 291-302, Gif-sur-Yvette, France, March 1999.
19. M. Vo and C. Wood. Building an Application Framework for Speech and Pen Input Integration in Multimodal Learning Interfaces. In Proc. of ICASSP 1996, IEEE International Conference on Acoustics, Speech, and Signal Processing, pages 3545-3548, Atlanta, USA, May 1996.
20. G. Welch and G. Bishop. An Introduction to the Kalman Filter. Technical Report TR 95-041, Department of Computer Science, University of North Carolina at Chapel Hill, 2000.
21. L. Wu, S. Oviatt, and P. Cohen. From Members to Teams to Committee - A Robust Approach to Gestural and Multimodal Recognition. IEEE Transactions on Neural Networks, 13(4):972-982, 2002.