Speech and sketching: An empirical study of multimodal interaction

Sketch-Based Interfaces and Modeling 2007 - ACM SIGGRAPH/Eurographics Symposium Proceedings

Volumn , Issue , 2007, Pages 83-90

  Adler, A ^a   Davis, R ^a  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

DESIGN; INTERACTIVE COMPUTER SYSTEMS; QUERY LANGUAGES; USER INTERFACES;

DESCRIPTORS; EMPIRICAL STUDIES; EVALUATION/METHODOLOGY; HUMAN DESIGNS; INPUT DEVICES; INTERACTION STYLES; MULTI-MODAL INTERACTIONS; MULTI-MODAL INTERFACES; MULTIMODAL CONVERSATIONS; NATURAL LANGUAGES; SKETCH RECOGNITION; SYSTEM ARCHITECTURES; USER STUDIES; USER-CENTERED DESIGNS; WHITE BOARDS;

GRAPHICAL USER INTERFACES;

EID: 57549110104     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1384429.1384449     Document Type: Conference Paper

References (18)

1. ALVARADO C., DAVIS R.: Resolving ambiguities to create a natural sketch based interface. In IJCAI (2001). 7
2. ADLER A., DAVIS R.: Speech and sketching for multimodal design. In IUI (2004), ACM Press, pp. 214-216. 8
3. ANDERSON R. J., HOYER C., WOLFMAN S. A., ANDERSON R.: A study of digital ink in lecture presentation. In CHI'04 (New York, NY, USA, 2004), ACM Press, pp. 567-574. 4
4. COHEN P. R., JOHNSTON M., MCCGEE D. R., OVIATT S. L., PITTMAN J., SMITH I., CHEN L., CLOWIX J.: QUICKSET: MULTIMODAL INTERACTION FOR DISTRIBUTED APPLIC CATIONS. IN MULTIMEDIA (1997), ACM PRESS, PP. 31-40. 1
5. DAVIS R.: Sketch understanding in design: Overview of work at the MIT AI lab. AAAI Spring Symposium (2002), 24-31. 1
6. DEMIRDJIAN D., Ko T., DARRELL T.: Untethered gesture acquisition and recognition for virtual world manipulation. Virtual Reality 8, 4 (2005), 222-230. 1,7
7. HUANG X., ALLEVA F., HON H.-W., HWANG M.-Y., ROSENFELD R.: The SPHINX-II speech recognition system: an overview. Computer Speech and Language 7, 2 (1993), 137-148. 3
8. JOHNSTON M., EHLEN P., WALKER M., WHITTAKER S., MALOOR P.: MATCH: An architecture for multimodal dialogue systems. In ACL (2002), pp. 276-383. 2, 7
9. KAISER E. C.: Multimodal new vocabulary recognition through speech and handwriting in a whiteboard scheduling application. In IUI'05 (New York, NY, USA, January 2005), ACM Press, pp. 51-58. 1, 2, 4, 7
10. KAISER E. C.: Using redundant speech and handwriting for learning new vocabulary and understanding abbreviations. In ICMI (Banff, Canada, 2006). 7
11. KAISER E. C., BARTHELMESS P., ERDMANN C., COHEN P.: Multimodal redundancy across handwriting and speech during computer mediated human-human interactions. In CHI'07 (New York, NY, USA, April 2007), ACM Press, pp. 1009-1018. 5, 7
12. OVIATT S., COHEN P., FONG M., FRANK M.: A rapid semi-automatic simulation technique for investigating interactive speech and handwriting. In ICSLP (1992), University of Alberta, pp. 1351-1354. 7
13. OVIATT S., COHEN P., WU L., VERGO J., DUNCAN L., SUHM B., BERS J., HOLZMAN T., WINO- GRAD T., LANDAY J., LARSON J., FERRO D.: Designing the user interface for multimodal speech and pen-based gesture applications: State-of-the-art systems and future research directions. In HCI (2000), pp. 263-322. 2, 7
14. OLTMANS M., DAVIS R.: Naturally conveyed explanations of device behavior. In PUI (2001). 2, 7
15. OVIATT S., DEANGELI A., KUHN K.: Integration and synchronization of input modes during multimodal human-computer interaction. In Human Factors in Computing Systems (1997), ACM Press, pp. 415-422. 5, 6, 7
16. RICH C., SIDNER C: COLLAGEN: A collaboration manager for software interface agents. User Modeling and User-Adapter Interaction 8, 3-4 (1998), 315-350.
17. MIT SketchML Format. http://rationale. csail.mit.edu/ETCHASketches/ format/. 3
18. ULLMAN D. G., WOOD S., CRAIG D.: The importance of drawing in the mechanical design process. Computers and Graphics 14, 2 (1990), 263-274. 1