Dynamically structuring, updating and interrelating representations of visual and linguistic discourse context

Artificial Intelligence

Volumn 167, Issue 1-2, 2005, Pages 62-102

  Kelleher, J ^a   Costello, F ^b   Van Genabith, J ^c  

^a GERMAN RESEARCH CENTER FOR ARTIFICIAL INTELLIGENCE DFKI   (Germany)

^b UNIVERSITY COLLEGE DUBLIN   (Ireland)

^c DUBLIN CITY UNIVERSITY   (Ireland)

Author keywords

Cross modal representations; Discourse context; Generating referring expressions; Reference resolution; Synthetic vision; Visual salience

Indexed keywords

COMPUTER SYSTEMS; MATHEMATICAL MODELS; SEMANTICS;

CROSS-MODAL REPRESENTATIONS; DISCOURSE CONTEXT; GENERATING REFERRING EXPRESSIONS; REFERENCE RESOLUTION; SYNTHETIC VISION; VISUAL SALIENCE;

ARTIFICIAL INTELLIGENCE;

EID: 24144453570     PISSN: 00043702     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.artint.2005.04.008     Document Type: Article

References (51)

1. M. Tanenhaus, M. Spivey-Knowlton, K. Eberhard, and J. Sedivy Integration of visual and linguistic information during language comprehension Science 268 1995 1632 1634
2. I. Duwe, H. Strohner, Towards a cognitive model of linguistic reference, Report: 97/1, Situierte Künstliche Kommunikatoren 97/1, Univeristät Bielefeld, 1997
3. M. Spivey-Knowlton, M. Tanenhaus, K. Eberhard, and J. Sedivy Integration of visuospatial and linguistic information: Language comprehension in real time and real space P. Olivier K. Gapp Representation and Processing of Spatial Expressions 1998 Lawrence Erlbaum Associates 201 214
4. D. Byron, Understanding referring expressions in situated language: Some challenges for real-world agents, in: Proceedings of the First International Workshop on Language Understanding and Agents for the Real World, Hokkaido University, 2003
5. A. Kehler Disourse D. Jurafsky J. Martin Speech and Language Processing. An Introduction to Natural Language Processing, Computational Linguistics and Speech Recognition international ed. 2000 Prentice Hall Englewood Cliffs, NJ 669 718
6. K. Linden Natural language generation D. Jurafsky J. Martin Speech and Language Processing. An Introduction to Natural Language Processing, Computational Linguistics and Speech Recognition international ed. 2000 Prentice Hall Englewood Cliffs, NJ 763 796
7. R. Dale, and E. Reiter Computational interpretations of the Gricean maxims in the generation of referring expressions Cognitive Science 19 2 1995 233 263
8. J. McCawley Everything That Linguists Have Always Wanted To Know About Logic* (but were ashamed to ask) second ed. 1993 University of Chicago Press
9. H. Grice Studies in the Way of Words 1989 Harvard University Press
10. T. Pechmann Incremental speech production and referential overspecification Linguistics 27 1989 98 110
11. T. Winograd A procedural model of language understanding R. Schank K. Colby Computer Models of Thought and Language 1973 W.H. Freeman and Company New York 152 186
12. P. McKevitt Integration of Natural Language and Vision Processing (vols. I-IV) 1995/1996 Kluwer Academic Dordrecht
13. M. Maybury W. Wahlster Readings in Intelligent User Interfaces 1998 Morgan Kaufman San Francisco, CA
14. S. Oviatt, P. Cohen, L. Wu, J. Vergo, L. Duncan, B. Suhm, J. Bers, T. Holzman, T. Winograd, J. Landay, J. Larson, and D. Ferro Designing the user interface for multimodal speech and gesture applications: State-of-the-art systems and research directions Human Computer Interaction 15 4 2000 263 322
15. L. Kievit, P. Piwek, R. Beun, and H. Bunt Multimodal cooperative resolution of referential expressions in the DENK system H. Bunt R. Beun Cooperative Multimodal Communication Lecture Notes in Artificial Intelligence vol. 2155 2001 Springer Berlin 197 214
16. T. Jording, and I. Wachsmuth An anthropomorphic agent for the use of spatial language K. Coventry P. Olivier Spatial Language: Cognitive and Computational Aspects 2002 Kluwer Academic Dordrecht 69 86
17. P. Gorniak, and D. Roy Grounded semantic composition for visual scenes J. Artificial Intelligence Res. 21 2004 429 470
18. F. Landragin, N. Bellalem, L. Romary, Visual salience and perceptual grouping in multimodal interactivity, in: Proceeding of the International Workshop on Information Presentation and Natural Multimodal Dialogue (IPNMD), Verona, Italy, 2001
19. R. Forgus, and L. Melamed Perception: A Cognitive Stage Approach 1976 McGraw-Hill New York
20. M. Chum, and J. Wolfe Visual attention E.B. Goldstein Blackwell Handbook of Perception Handbooks of Experimental Psychology 2001 Blackwell 272 310 Chapter 9
21. M.I. Posner, C.R. Snyder, and B.J. Davidson Attention and the detection of signals J. Experimental Psychol. Gen. 109 2 1980 160 174
22. C. Koch, and L. Itti Computational modelling of visual attention Nature Rev. Neurosci. 2 3 2001 194 203
23. D. Heinke, and G. Humphreys Computational models of visual selective attention: A review G. Houghton Connectionist Models in Psychology 2004 Psychology Press
24. X. Tu, D. Terzopoulos, Artificial fishes: Physics, locomotion, perception, behaviour, in: Proceedings of ACM SIGGRAPH, Orlando, FL, 1994, pp. 43-50
25. H. Noser, O. Renault, D. Thalmann, and N. Magnenat-Thalmann Navigation for digital actors based on synthetic vision, memory and learning Computer Graphics 19 1 1995 7 9
26. J. Kuffner, and J. Latombe Fast synthetic vision, memory, and learning models for virtual humans Proceedings of Computer Animation Conference (CA-99) 1999 IEEE Computer Society Geneva 118 127
27. C. Peter, C. O'Sullivan, A memory model for autonomous virtual humans, in: Proceedings of Eurographics Irish Chapter Workshop (EGIreland-02), Dublin, 2002, pp. 21-26
28. H. Kamp, and U. Reyle From Discourse to Logic 1993 Kluwer Academic Dordrecht
29. B. Grosz, and C. Sidner Attention, intentions, and the structure of discourse Computational Linguistics 12 3 1986 175 204
30. J. Kelleher, and J. van Genabith A computational model of the referential semantics of projective prepositions P. Saint-Dizier Computational Linguistics Dimensions of the Syntax and Semantics of Prepositions 2005 Kluwer Academic Dordrecht 200 215
31. J. Hobbs, On the coherence and structure of discourse, Technical Report CSLI-85-37, Center for the Study of Language and Information, 1985
32. W. Mann, and S. Thompson Rhetorical structure theory: Description and construction of text structures G. Kempen Natural Language Generation: New Results in Artificial Intelligence, Psychology and Linguistics 1987 Nijhoff Dordrecht 83 96
33. M. Walker Limited attention and discourse structure Computational Linguistics 22 2 1996
34. R. Langacker Foundations of Cognitive Grammar: Theoretical Prerequisites, vol. 1 1987 Standford University Press Standford, CA
35. S. Salmon-Alt, L. Romary, Reference resolution within the framework of cognitive grammar, in: Proceedings of the Seventh International Colloquium on Cognitive Science (ICCS-01), Donostia, Spain, 2001, pp. 284-299
36. M. Pinkal, Definite noun phrases and the semantics of discourse, in: Proceedings of the 11th International Conference on Computational Linguistics (COLING 86), Bonn, 1986, pp. pp. 368-373
37. M. Poesio, Discourse interpretation and the scope of operators, Ph.D. dissertation, University of Rochester, 1994
38. D.L. Bean, E. Riloff, Corpus-based identification of non-anaphoric noun phrases, in: Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics (ACL-99), University of Maryland, 1999, pp. 373-380
39. B. Grosz, A. Joshi, and W. Weinstein Centering: A framework for modelling local coherence of discourse Computational Linguistics 21 2 1995 203 255
40. D. Appelt Planning English referring expressions Artificial Intelligence 26 1 1985 1 33
41. R. Dale Generating Referring Expressions: Building Descriptions in a Domain of Objects and Processes 1992 MIT Press Cambridge, MA
42. W. Claassen Generating referring expressions in a multimodal environment R. Dale E. Hovy D. Rosner O. Stock Aspects of Automated Natural Language Generation 1992 Springer Berlin
43. J. Lester, J. Voerman, S. Towns, and C. Callaway Deictic believability: Coordinating gesture, locomotion, and speech in lifelike pedagogical agents Appl. Artificial Intelligence 13 4-5 1999 383 414
44. K. van Deemter, Generating vague descriptions, in: Proc. First International Conference on Natural Language Generation (INLG-00), Mitzpe Ramon, Israel, 2000
45. M. Stone, On identifying sets, in: Proc. International Conference on Natural Language Generation (INLG-00), Mitze Ramon, Israel, 2000
46. E. Krahmer, and M. Theune Efficient context-sensitive generation of referring expressions K. van Deemter R. Kibble Information Sharing: Reference and Presupposition in Language Generation and Interpretation 2002 CLSI Publications Standford, CA
47. C. Gardent, Generating minimal definite descriptions, in: Proceedings of the 40th International Conference of the Association of Computational Linguistics (ACL-02), Philadelphia, 2002, pp. 96-103
48. K. van Deemter Generating referring expressions: Boolean extensions of the incremental algorithm Computational Linguistics 28 1 2002 37 52
49. E. Krahmer, I. van der Sluis, A new model for generating multimodal referring expressions, in: 9th European Workshop on Natural Language Generation (ENLG-03), Budapest, Hungary, 2003, pp. 47-57
50. M. Theune, From data to speech: Language generation in context, Ph.D., Eindhoven University of Technology, 2000
51. E. Hajicová, Issues of sentence structure and discourse patterns, in: Theoretical and Computational Linguistics, vol. 2, Charles University, Prague, 1993