Multimodal interaction with mobile devices: Outline of a semiotic framework for theory and practice

WINSYS 2006 - International Conference on Wireless Information Networks and Systems, Proceedings

Volumn , Issue , 2006, Pages 276-283

  Öquist, Gustav ^a  

^a UPPSALA UNIVERSITY   (Sweden)

Author keywords

Framework; Interaction; Mobile devices; Multimodality; Semiotics

Indexed keywords

COMMUNICATION CIRCUITS; DESKTOP METAPHORS; INTERACTION; MAPPING INFORMATION; MOBILE INTERACTION; MULTI-MODAL INTERACTIONS; MULTI-MODALITY; RESEARCH PROTOTYPE; TACTILE SENSE; THEORY AND PRACTICE;

HUMAN COMPUTER INTERACTION; INFORMATION SERVICES; INTERACTIVE COMPUTER SYSTEMS; MOBILE DEVICES; MOBILE TELECOMMUNICATION SYSTEMS; PORTABLE EQUIPMENT; SEMIOTICS;

WIRELESS NETWORKS;

EID: 77954106036     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (29)

1. Allwood, J. (2002). Bodily Communication - Dimensions of expression and Content. In B. Granström, D. House, and I. Karlsson (Eds). Multimodality in Language and Speech Systems, 7-26. Kluwer Academic Publishers.
2. Bersen, O. (1994). Foundations of multimodal representations. A taxonomy of representational modalities, Interacting with Computers, 6(4), 347-371.
3. Brewster, S.A. (1998). Using non speech sounds to provide navigation cues. ACM Transactions on Computer-Human Interaction, 5(3), 224-259.
4. Chandler, D. (2001). Semiotics: The Basics. New York: Routledge.
5. Fishkin, K. P., Gujar, A., Harisson, B. L., Moran, T. P. and Want, R. (2000). Embodied user interfaces for really direct manipulation. Communications of the ACM, 43(9), 75-80.
6. Furnas, G.W. (1991). New graphical reasoning models for understanding graphical interfaces. In Proceedings of ACM CHI'91 Conference (New Orleans, LA), 71-78. New York, NY: ACM Press.
7. Gaver, W. (1986). Auditory icons: Using sound in computer interfaces. Human-Computer Interaction, 2, 167-177.
8. Gold, B., and Nelson, M. (1999). Speech and Audio Signal Processing: Processing and Perception of Speech and Music. New York, NY: John Wiley & Sons.
9. Holland, S., Morse, D.R., and Gedenryd, H. (2002). Direct Combination. A new user interaction principle for mobile and ubiquitous HCI. In Proceedings of Mobile HCI 2002 (Pisa, Italy), 108-122. Berlin: Springer.
10. Kjeldskov, J. (2002). "Just-in-Place" information for mobile device interfaces. In Proceedings of Mobile HCI 2002 (Pisa, Italy), 271-275. Berlin: Springer.
11. Lorho, G., Hiipakka, J., and Marila, J. (2002). Structured menu presentation using spatial sound separation. In Proceedings of Mobile HCI 2002 (Pisa, Italy), 419-424. Berlin: Springer.
12. MacKenzie, I. S., and Soukoreff, R. W. (2002). Text entry for mobile computing: Models and methods, theory and practice. Human-Computer Interaction, 17, 147-198.
13. MacKenzie, I. S., Kober, H., Smith, D., Jones, T., and Skepner, E. (2001). LetterWise: Prefix-based disambiguation for mobile text input. In Proceedings of UIST'01 (Orlando, FL), 111-120. New York, NY: ACM Press.
14. McGookin, D.K., and Brewster, S.A. (2001) Fishears - The design of a multimodal focus and context system. In Proceedings of IHM-HCI'01, Vol. II (Lille, France), 1-4. Toulouse: Cépaduès-Editions.
15. McTear, M.F. (2002). Spoken dialogue technology: enabling the conversational interface. ACM Computing Surveys, 34(1), 90 - 169.
16. Milic-Frayling, N., and Sommerer, R. (2002). SmartView: Flexible viewing of web page contents. In Proceedings of WWW'02, (Honolulu, USA).
17. Nigay, L., and Coutaz, J. (1993). A design space for multimodal interfaces: concurrent processing and data fusion. In Proceedings of InterCHI'93, (Amsterdam, The Netherlands), 172-178.
18. Mäntyjärvi, J., and Seppänen, T. (2002). Adapting applications in mobile terminals using fuzzy context information. In Proceedings of Mobile HCI 2002 (Pisa, Italy), 95-107. Berlin: Springer.
19. Oakley, I., Adams, A., Brewster, S.A., and Gray, P.D. Guidelines for the design of haptic widgets. In Proceedings of BCS HCI 2002 (London, UK), 195-212. London: Springer.
20. Öquist, G., Goldstein, M., and Björk, S. (2002). Utilizing gaze detection to stimulate the affordances of paper in the Rapid Serial Visual Presentation Format. In Proceedings of Mobile HCI 2002 (Pisa, Italy), 378-381. Berlin: Springer.
21. Öquist, G., and Goldstein, M. (2003). Towards an improved readability on mobile devices: Evaluating Adaptive Rapid Serial Visual Presentation. Interacting with Computers, 15(4), 539-558.
22. Pirhonen, A., Brewster, S.A., and Holguin, C. (2002). Gestural and audio metaphors as a means of control for mobile devices. In Proceedings of ACM CHI'02 (Minneapolis, MN), 291-298. New York, NY: ACM Press.
23. Prechelt, L., and Typke R. (2001). An interface for melody input. ACM Transactions on Computer-Human Interaction, 8(2), 133-194.
24. Rekimoto, J. (1987). Pick and Drop: A direct manipulation technique for multiple computer environments. In Proceedings of UIST'87, 31-39. New York, NY: ACM Press.
25. Sazawal, V., Want, R., and Borriello, G, (2002). The Unigesture approach. In Proceedings of Mobile HCI 2002 (Pisa, Italy), 256-270. Berlin: Springer.
26. Schneiderman, B. (1982). The Future of Interactive Systems and the Emergence of Direct Manipulation. Behaviour and Information Technology, 1, 237-256.
27. Sokoler, T., Nelson, L., and Pedersen, E.R. (2002). Low-resolution supplementary tactile cues for navigational assistance. In Proceedings of Mobile HCI 2002 (Pisa, Italy), 369-372. Berlin: Springer.
28. Ward, D. J., Blackwell, A. F., and MacKay, D. J. C. (2002). Dasher: A gesture-driven data entry interface for mobile computing. Human-Computer Interaction, 17, 199-228.
29. Williams, J. R. (1998). Guidelines for the use of multimedia in instruction. In Proceedings of the Human Factors and Ergonomics Society 42nd Annual Meeting (Chicago, IL), 1447-1451. Santa Monica, CA: HFES.