Developing multimodal interfaces: A theoretical framework and guided propagation networks

Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Volumn 1374, Issue , 1998, Pages 158-187

  Martin, J C ^a   Veldman, R ^a   Beroule D ^a  

^a CNRS   (France)

Author keywords

[No Author keywords available]

Indexed keywords

INTERACTIVE COMPUTER SYSTEMS; SPECIFICATION LANGUAGES; SPECIFICATIONS;

GUIDED PROPAGATION NETWORKS; MULTI-MODAL; MULTI-MODAL INTERFACES; TEST APPLICATIONS; THEORETICAL FRAMEWORK;

COOPERATIVE COMMUNICATION;

EID: 84957886536     PISSN: 03029743     EISSN: 16113349     Source Type: Book Series    
DOI: 10.1007/bfb0052318     Document Type: Conference Paper

References (61)

1. André, E. and Rist, T. (1995) Generating coherent presentations employing textual and visual material. Artificial Intelligence Review 9 (2-3), 147-165.
2. Baekgaard, A. (1995) Constraining of input media in a spoken dialog system. In Proc. 4th European Conference on Speech Communication and Technology (EUROSPEECH '95), 1181-1184.
3. Bellalem, N. and Romary, L. (1995) Reference interpretation in a multimodal environment combining speech and gesture. In: Lee, J. (1995) Pre-Proceedings First Int. Worskhop on Intelligence and Multimodality in Multimedia Interfaces: Research and Applications, University of Edinburgh.
4. Béroule, D. (1985) Un modelè de mémoire adaptative, dynamique et associative pour le traitement automatique de la parole. Thesis, University of Paris XI, Orsay.
5. Béroule, D. (1988) The never-ending learning. In R. Eckmiller and C. v.d. Malsburg (eds.), Neural Computers. NATO ASI Series F, vol 41. Berlin: Springer, 219-230.
6. Béroule, D. (1990) Guided propagation: current state of theory and application. In F. Fogelman Souliè and J. Hérault (eds.) Neurocomputing, NATO ASI Series, Vol. F 68, 241-260. Berlin: Springer.
7. Béroule, D., Von Hoe, R. and Ruellan, H. (1994) A Guided Propagation Model of Reading. Annual Progress Report 28, Instituut voor Perceptie Onderzoek IPO, Eindhoven, 21-29.
8. Blanchet, P. (1992) Une architecture connexionniste pour l'apprentissage par l'expérience et la représentation des connaissances. Thesis, University of Paris XI, Orsay.
9. Bolt, R.A. (1980) 'Put - That - There': Voice and Gesture at The Graphics Interface. Computer Graphics 14 (3), 262-270.
10. Bos, E. (1993) Easier said or done? Studies in multimodal human-computer interaction. NICI technical report 93-02, University of Nijmegen.
11. Bourdot, P., Krus, M., Gherbi, R. (1995) Management of non-standard devices for multimodal user interfaces under UNIX/Xll. This volume.
12. Bressolle, M.C, Pavard, B., Leroux, M. (1997) The role of multimodal communication in cooperation and intention recognition: the case of air traffic control. This volume.
13. Briffault, X. (1996) Une interface multimodale pour l'aide a la navigation. Working paper, LIMSI, Orsay. http://www.limsi.fr/Individu/xavier/index.html
14. Bunt, H., Beun, R. J., and Borghuis, T. (eds.) Proceedings of the International Conference on Cooperative Multimodal Communication CMC/95. Eindhoven, May 24-26.
15. Carbonnel, J.R. (1970) Mixed-Initiative Man-Computer Dialogues. Bolt, Beranek and Newman (BBN) Report N 1971, Cambridge, MA.
16. Catinis, L., Caelen, J. (1995) Analyse du comportement multimodal de l'usager humain dans une tache de dessin. Acres des 7. Journées sur l'Ingéniérie de l'Interaction Homme-Machine (IHM'95), 123-129.
17. Cheyer, A. and Julia, L. (1995) Multimodal maps: an agent- based approach. This volume.
18. Coutaz, J., Salber, D., Carraux, E. and Portolan, N. (1996) NEIMO, a multiworkstation usability lab for observing and analyzing multimodal interaction. To appear in CHI'96 Conference Proceedings Companion. Video.
19. Coutaz, J. and Nigay, L. (1994) Les propriétés CARE dans les interfaces multimodales. Acres des 6émes Journées sur l'Ingéniérie de l'Interaction Homme-Machine (IHM'94), Lille, p. 7-14.
20. Escande, P., Béroule, D. and Blanchet, P. (1991) Speech recognition experiments with Guided Propagation. Proc. of IJCNN'91.
21. Daniel, M.P., Carite, L. and Denis, M. (1994) Modes of linearization in the description of spatial configurations. In Portugali, J. (ed.), The construction of cognitive maps. Dordrecht: Kluwer, 297-318.
22. Dowell, J., Shmueli, Y., and Salter, I. (1995) Applying a cognitive model of the user to the design of a multimodal speech interface. In: Lee, J. (1995) Pre-Proceedings First Int. Worskhop on Intelligence and Multimodality in Multimedia Interfaces: Research and Applications, University of Edinburgh.
23. Faure, C. and Julia, L. (1994) An agent-based architecture for a multimodal interface. Working notes of the AAAI symposium on Intelligent Multi-Media Multi-Modal Systems. March 21-23, Stanford.
24. Foote, J.T., Brown, M.G., Jones, G.J.F., Sparck Jones, K., and Young, S.J. (1995) Video mail retrieval by voice: towards intelligent retrieval and browsing of multimedia documents. In: Lee, J. (1995) Pre-Proceedings First Int. Worskhop on Intelligence and Multimodality in Multimedia Interfaces: Research and Applications, University of Edinburgh
25. Frohlich, D.M. (1991) The design space of interfaces. In L. Kjelldahl (ed.) Multimedia: principles, systems and applications. Berlin: Springer.
26. Gonqalves, M.R. (1996) Working notes on itinerary descriptions. LIMSI, Orsay. http://www.limsi.fr/Individu/goncalve/index.html
27. Hare, M., Doubleday, A., Bennett, I., and Ryan, M. (1995) Intelligent presentation of information retrieved from heterogeneous multimedia databases. In: Lee, J. (1995) Pre-Proceedings First Int. Worskhop on Intelligence and Multimodality in Multimedia Interfaces: Research and Applications, University of Edinburgh
28. Han, Y. and Zukerman, I. (1997) A cooperative approach for multimodal presentation planning. This volume.
29. Huls, C. and Bos, E. (1997) Studies into full integration of language and action. This volume.
30. Hurault-Plantet and Briffault (1996) Atelier de génie linguistique et visualisation graphique, http://www.limsi.fr/Individu/gs/GroupeLC/Outils.html
31. Hutchins, E.L., Holland, J.D. and Norman, D.A. (1986) Direct manipulation interfaces. In Norman, D.A. and Draper, S.W. (eds.), User centred system design: new perspectives on human computer design. Hillsdale, N J: Lawrence Erlbaum.
32. Inder, R., Oberlander, J., and Tobin, R. (1995) Intelligent support for navigation in hypermedia: discourse structure and the Web. In: Lee, J. (1995) Pre-Proceedings First Int. Worskhop on Intelligence and Multimodality in Multimedia Interface: Research and Applications, University of Edinburgh.
33. Jackendoff, R. (1987) On beyond zebra: the relation between linguistic and visual information. Cognition 26 (2), 89-114.
34. Lee, J. (ed.) (1995) Pre-Proceedings First International Workshop on Intelligence and Multimodality in Multimedia Interfaces: Research and Applications. University of Edinburgh.
35. Mackinlay, J., Card, S.K. & Robertson, G.G. (1990) A Semantic Analysis of the Design Space of Input Devices. Human-Computer Interaction. Vol 5, no 2-3, pp. 145-190.
36. Martin, J.C. (1995) Cooédrations entre modalités et liage par synchronie dans les interfaces multimodales. Ph.D. Thesis, TELECOM Paris. http://www.limsi.fr/Individu.martin
37. Martin, J.C. (1996) Types et buts de coopération entre modalités dans les interfaces multimodales. Techniques et Science Inf ormatiques 15, 10/1996, 1367-1397.
38. Martin, J.C. (1997) Towards intelligent cooperation between modalities. The example of a system enabling multimodal interaction with a map. Proc. IJCAI'97 International Workshop on Intelligent Multimodal Systems, 63-69. http://www.limsi.fr:80/Individu/martin/ijcai/article.html
39. Martin, J.C. and Béroule, D. (1993) Types et buts de cooperations entre modalités. In Proc. 5 th Conf. on Human-Computer Interaction IHM'93, 17-22.
40. Martin, J.C. and Béroule, D. (1995) Temporal codes within a typology of cooperation between modalities. Artificial Intelligence Review 9, 1-8.
41. Maybury, M. (1991) Introduction. Intelligent multimedia interfaces. Cambridge, MA: AAAI Press.
42. Nigay, L. and Coutaz, J. (1993) A design space for multimodal systems: concurrent processing and data fusion. Proc. of Interchi'93, 172-178.
43. Nigay, L. and Coutaz, J. (1995) Multifeature systems: from HCI properties to software design. In: Lee, J. (1995) Pre-Proceedings First Int. Worskhop on Intelligence and Multimodality in Multimedia Interfaces: Research and Applications, University of Edinburgh.
44. O'Nuallain, S. and Smith, A.G. (1994) An investigation into the common semantics of language and vision. Artificial Intelligence Review 8 (2-3), 113-122.
45. Olivier, P. and Tsujii, J.I. (1994) Quantitative perceptual representation of prepositional semantics. Artificial Intelligence Review 8 (2-3).
46. Roques, M. (1994) Dynamic Grammatical Representations in Guided Propagation Networks. In R.C. Carrasco and J. Oncina (eds.) Grammatical Inference and Applications, Lecture Notes in Artificial Intelligence 862, 189-202. Berlin: Springer.
47. Salisbury, M.W., Hendrickson, J.H., Lammers, T.L., Fu, C., and Moody, S. A. (1990) Talk and draw: bundling speech and graphics. IEEE Computer 23 (8), 59-65.
48. Santana, S. and Pineda, L.A. (1995) Producing coordinated natural language and graphical explanations in the context of a geometric problem-solving task. In: Lee, J. (1995) Pre-Proceedings First Int. Worskhop on Intelligence and Multimodality in Multimedia Interfaces: Research and Applications, University of Edinburgh.
49. Shastri, L. and Ajjanagadde, V. (1993) From simple associations to systematic reasoning: a connectionist representation of rules, variables and dynamic bindings using temporal synchrony. Behavioural and Brain Sciences, 16, 417-494.
50. Sims, R. and Hedberg, J. (1995) Dimensions of learner control: a reappraisal of interactive multimedia instruction. In: Lee, J. (1995) Pre-Proceedings First Int. Worskhop on Intelligence and Multimodality in Multimedia Interfaces: Research and Applications, University of Edinburgh.
51. Siroux, J., Guyomard, M., Multon, F., and Remondeau, C. (1997) Modeling and processing of the oral and tactile activities in the Georal tactile system. This volume.
52. Sowa, J. (1983) Conceptual Structures: Information Processing in Mind and Machine. Reading, MA: Addison-Wesley.
53. Stern, R.M. (1995) Robust speech recognition. Section 14 in electronic book: Survey of the State of the Art in Human Language Technology. http://www.cse.ogi.edu/CSLU/HLTsurvey/chlnode6.html/
54. Vaananen, K. (1995) Four pillars for improving the quality of multimedia applications. In Proc. First Int. Workshop on Evaluation Methods and Quality Criteria for Multimedia Applications, San Francisco.
55. Vo, M. T. and Waibel, A. (1993) Multimodal Human-Computer Interaction. In Proc. International Symposium on Spoken Dialogue: New Directions in Human and Man-Machine Communication, Tokyo, 95-101.
56. Veldman, R. (1995) Experiments on robust parsing in a multimodal Guided Propagation Network. LIMSI (ERASMUS) Report 95-11, Orsay
57. Wahlster, W., Andre, E., Finkler, W., Profitlich, H.J., and Rist, T. (1991) Plan-based integration of natural language and graphics generation. AI Journal 63, 387-427.
58. Wang, E., Shahnvaz, H., Hedman, L., Papadopoulos, K., and Watkinson, N. (1993) A usability evaluation of text and speech redundant help messages on a reader interface. In G. Salvendy & M. Smith (eds.), Human-Computer Interaction: Software and Hardware Interfaces, 724-729.
59. Westerlund, P., Béroule, D. and Roques, M. (1994) Experiments of robust parsing using a Guided Propagation Network. In Proc. International Conference on New Methods in Language Processing (NEMLAP'94) Manchester.
60. Webber, B. (1997) Instructing Animated Agents: Viewing Language in Behavioural Terms. This volume.
61. Yankelovich, N., Levow, G., Marx, M. (1995) Designing Speech Acts: Issues in Speech User Interfaces. Proc. of CHI '95, Conference on Human Factors in Computing Systems.