Generation of Referring Expressions: Assessing the Incremental Algorithm

Cognitive Science

Volumn 36, Issue 5, 2012, Pages 799-836

  Deemter, Kees van ^a   Gatt, Albert ^b   Sluis, Ielka van der ^c   Power, Richard ^d  

^a UNIVERSITY OF ABERDEEN   (United Kingdom)

^b UNIVERSITY OF MALTA   (Malta)

^c TRINITY COLLEGE DUBLIN   (Ireland)

^d OPEN UNIVERSITY   (United Kingdom)

Author keywords

Evaluation metrics for generation algorithms; Generation production of referring expression; Incremental algorithm; Psycholinguistics; Reference

Indexed keywords

ALGORITHM; ARTICLE; HUMAN; LANGUAGE; LANGUAGE TEST;

ALGORITHMS; HUMANS; LANGUAGE; LANGUAGE TESTS;

EID: 84863434179     PISSN: 03640213     EISSN: None     Source Type: Journal    
DOI: 10.1111/j.1551-6709.2011.01205.x     Document Type: Article

References (88)

1. Anderson, A., Bader, M., Bard, E., Boyle, E., Doherty, G. M., Garrod, S., Isard, S., Kowtko, J., McAllister, J., Miller, J., Sotillo, C., Thompson, H. S., & Weinert, R. (1991). The hcrc Map Task corpus. Language and Speech, 34, 351-366.
2. Appelt, D. (1985). Planning english referring expressions. Artificial Intelligence, 26(1), 1-33.
3. Appelt, D., & Kronfeld, A. (1987). A computational model of referringProceedings of the 10th International Joint Conference on Artificial Intelligence (IJCAI-87) (pp. 640-647). Milan: Morgan Kaufman.
4. Areces, C., Koller, A., & Striegnitz, K. (2008). Referring expressions as formulas of description logicProceedings of the 5th International Conference on Natural Language Generation (INLG-08).
5. Arnold, J. E. (2008). Reference production: Production internal and addressee-oriented processes. Language and Cognitive Processes, 23(4), 495-527.
6. Arts, A. (2004). Overspecification in instructive texts. PhD thesis, University of Tilburg.
7. Artstein, R., & Poesio, M. (2008). Inter-coder agreement for computational linguistics (survey article). Computational Linguistics, 34(4), 555-596.
8. Belke, E., & Meyer, A. (2002). Tracking the time course of multidimensional stimulus discrimination: Analysis of viewing patterns and processing times during same-different decisions. European Journal of Cognitive Psychology, 14(2), 237-266.
9. Belz, A., & Gatt, A. (2007). The attribute selection for gre challenge: Overview and evaluation resultsProceedings of UCNLG+MT: Language Generation and Machine Translation.
10. Belz, A., & Gatt, A. (2008). Intrinsic vs. extrinsic evaluation measures for referring expression generationProceedings of 46th Annual Meeting of the Association for Computational Linguistics (ACL-08).
11. Bohnet, B. (2007). is-fbn, is-fbs, is-iac: The adaptation of two classic algorithms for the generation of referring expressions in order to produce expressions like humans doProceedings of the Language Generation and Machine Translation Workshop (UCNLG+MT) at MT Summit XI.
12. Bohnet, B. (2008). The fingerprint of human referring expressions and their surface realization with graph transducersProceedings of the 5th International Conference on Natural Language Generation (INLG-08).
13. Brennan, S., & Clark, H. (1996). Conceptual pacts and lexical choice in conversation. Journal of Experimental Psychology, 22(6), 1482-1493.
14. Campana, E., Tanenhaus, M., Allen, J., & Remington, R. (2004). Evaluating cognitive load in spoken language interfaces using a dual-task paradigmProceedings of the 9th International Conference on Spoken Language Processing (ICSLP'04).
15. Carletta, J. (1996). Assessing agreement on classification tasks: The kappa statistic. Computational Linguistics, 22(2), 249-254.
16. Carletta, J., & Mellish, C. (1996). Risk-taking and recovery in task-oriented dialogues. Journal of Pragmatics, 26, 71-107.
17. Chafe, W. L. (Ed.) (1980). The pear stories: Cognitive, cultural, and linguistic aspects of narrative production. Norwood, NJ: Ablex.
18. Clark, H., & Wilkes-Gibbs, D. (1986). Referring as a collaborative process. Cognition, 22, 1-39.
19. Dale, R. (1989). Cooking up referring expressionsProceedings of 27th Annual Meeting of the Association for Computational Linguistics, ACL-89.
20. Dale, R., & Haddock, N. (1991). Generating referring expressions containing relationsProceedings of 5th Conference of the European Chapter of the Association for Computational Linguistics.
21. Dale, R., & Reiter, E. (1995). Computational interpretation of the Gricean maxims in the generation of referring expressions. Cognitive Science, 19(8), 233-263.
22. van Deemter, K. (2002). Generating referring expressions: Boolean extensions of the incremental algorithm. Computational Linguistics, 28(1), 37-52.
23. van Deemter, K. (2006). Generating referring expressions that involve gradable properties. Computational Linguistics, 32(2), 195-222.
24. van Deemter, K., van der Sluis, I., & Gatt, A. (2006). Building a semantically transparent corpus for the generation of referring expressionsProceedigs of 4th International Conference on Natural Language Generation (Special Session on Data Sharing and Evaluation), INLG-06.
25. Eikmeyer, H. J., & Ahlsèn, E. (1996). The cognitive process of referring to an object: A comparative study of german and swedishProceedings of 16th Scandinavian Conference on Linguistics.
26. Engelhardt, P. E., Bailey, K., & Ferreira, F. (2006). Do speakers and listeners observe the gricean maxim of quantity? Journal of Memory and Language, 54, 554-573.
27. di Fabbrizio, G. D., Stent, A. J., & Bangalore, S. (2008a). Referring expression generation using speaker-based attribute selection and trainable realization (att-reg)Proceedings of 5th International Conference on Natural Language Generation (INLG'08) (pp. 211-214).
28. di Fabbrizio, G. D., Stent, A. J., & Bangalore, S. (2008b). Trainable speaker-based referring expression generationProceedings of 12th Conference on Computational Natural Language Learning (CONLL'08) (pp. 151-158).
29. Ford, W., & Olson, D. (1975). The elaboration of the noun phrase in children's object descriptions. Journal of Experimental Child Psychology, 19, 371-382.
30. Gardent, C. (2002). Generating minimal definite descriptionsProceedings of 40th Annual Meeting of the Association for Computational Linguistics, ACL-02.
31. Gatt, A., & Belz, A. (2008). The tuna challenge 2008: Overview and evaluation resultsProceedings of the 5th International Conference on Natural Language Generation (INLG-08).
32. Gatt, A., & Belz, A. (2010). Introducing shared task evaluation to nlg: The tuna shared task evaluation challengesE. Krahmer & M. Theune (Eds.), Lecture Notes in Computer Science, Vol 14. Empirical methods in natural language generation (pp. 264-293). New York: Springer.
33. Gatt, A., Belz, A., & Kow, E. (2008a). The tuna challenge 2008: Overview and evaluation resultsProceedings of 5th International Conference on Natural Language Generation, INLG-08.
34. Gatt, A., & van Deemter, K. (2007). Incremental generation of plural descriptions: Similarity and partitioningProceedings of the Conference on Empirical Methods in Natural Language Processing, EMNLP-07.
35. Gatt, A., der Sluis, I., & van Deemter, K. (2007). Evaluating algorithms for the generation of referring expressions using a balanced corpusProceedings of the 11th European Workshop on Natural Language Generation, ENLG-07.
36. Gatt, A., van der Sluis, I., & van Deemter, K. (2008b). Xml format guidelines for the tuna corpus. Technical report, Computing Science, University of Aberdeen, Available at:
37. Goudbeek, M., Krahmer, E., & Swerts, M. (2009). Alignment of (dis)preferred properties during the production of referring expressionsProceedings of Workshop ''Production of Referring Expressions: Bridging the gap between computational and empirical approaches to reference'' (PRE-CogSci 2009).
38. Grice, H. (1975). Logic and conversationP. Cole & J. Morgan (Eds.), Syntax and semantics: Speech acts (Vol. III, pp. 41-58). New York: Academic Press.
39. Gupta, S., & Stent, A. J. (2005). Automatic evaluation of referring expression generation using corporaProceedings of 1st Workshop on Using Corpora in NLG, Birmingham, UK.
40. Hermann, T., & Deutsch, W. (1976). Psychologie der Objektbenennung. Bern: Huber.
41. Hervás, R., & Gervás, P. (2009). Evolutionary and case-based approaches to regProceedings of 12th European Workshop on Natural Language Generation (ENLG-09).
42. Horacek, H. (1997). An algorithm for generating referential descriptions with flexible interfacesProceedings of 35th Annual Meeting of the Association for Computational Linguistics, ACL-97, Madrid (pp. 206-213).
43. Horacek, H. (2004). On referring to sets of objects naturallyProceedings of 3rd International Conference on Natural Language Generation, INLG-04.
44. Jordan, P., & Walker, M. (2000). Learning attribute selections for non-pronominal expressionsProceedings of 38th Annual Meeting of the Association for Computational Linguistics.
45. Jordan, P. W. (2000a). Influences on attribute selection in redescriptions: A corpus studyProceedings of the Cognitive Science Conference.
46. Jordan, P. W. (2000b). Intentional influences on object redescriptions in dialogue: Evidnece from an empirical study. PhD thesis, University of Pittsburgh.
47. Jordan, P. W. (2002). Contextual influences on attribute selection for repeated descriptionsK. van Deemter & R. Kibble (Eds.), Information sharing: Reference and presupposition in natural language generation and understanding (pp. 295-328). Stanford, CA: CSLI Publications.
48. Jordan, P. W., & Walker, M. (2005). Learning content selection rules for generating object descriptions in dialogue. Journal of Artificial Intelligence Research, 24, 157-194.
49. Kelleher, J. D., & Kruijff, G.-J. (2006). Incremental generation of spatial referring expressions in situated dialogProceedings of joint 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, ACL/COLING-06.
50. Keysar, B., Lin, S., & Barr, D. J. (2003). Limits on theory of mind use in adults. Cognition, 89, 25-41.
51. King, J. (2008). OSU-GP: Attribute selection using genetic programmingProceedings of the 5th International Conference on Natural Language Generation (INLG-08).
52. Koolen, R., Gatt, A., Goudbeek, M., & Krahmer, E. (2009). Need I say more? On factors causing referential overspecificationProceedings of Workshop ''Production of Referring Expressions: Bridging Computational and Psycholinguistic Approaches'' (pre-cogsci'09).
53. Krahmer, E., & Theune, M. (2002). Efficient context-sensitive generation of referring expressionsK. van Deemter & R. Kibble (Eds.), Information sharing: Reference and presupposition in language generation and interpretation (pp. 223-264). Stanford, CA: CSLI.
54. Krahmer, E., & van Deemter, K. (2011). Computational generation of referring expressions: A survey, in preparation. Available at: Accessed September 23, 2011.
55. Krahmer, E., van Erk, S., & Verleg, A. (2003). Graph-based generation of referring expressions. Computational Linguistics, 29(1), 53-72.
56. Krauss, R., & Weinheimer, S. (1964). Changes in reference phrases as a function of frequency of usage in social interaction: A preliminary study. Psychonomic Science, 1, 113-114.
57. Krippendorf, K. (1980). Content analysis. Newbury Park, CA: Sage Publications.
58. Kronfeld, A. (1989). Conversationally relevant descriptionsProceedings of 27th Annual Meeting of the Association for Computational Linguistics, ACL-89.
59. Levelt, W. M. J. (1989). Speaking: From intention to articulation. Cambridge, MA: MIT Press.
60. de Lucena, D., & Paraboni, I. (2008). usp-each frequency-based greedy attribute selection for referring expressions generationProceedings of 5th International Conference on Natural Language Generation (INLG'08) (pp. 219-220).
61. Maes, A., Arts, A., & Noordman, L. (2004). Reference management in instructive discourse. Discourse Processes, 37(2), 117-144.
62. Mangold, R., & Pobel, R. (1988). Informativeness and instrumentality in referential communication. Journal of Language and Social Psychology, 7(3-4), 181-191.
63. Oberlander, J. (1998). Do the right thing ... but expect the unexpected. Computational Linguistics, 24(3), 501-507.
64. Olson, D. R. (1970). Language and thought: Aspects of a cognitive theory of semantics. Psychological Review, 77, 257-273.
65. Paraboni, I., van Deemter, K., & Masthoff, J. (2007). Generating referring expressions: making referents easy to identify. Computational Linguistics, 33(2), 229-254.
66. Passonneau, R. (2006). Measuring agreement on set-valued items (masi) for semantic and pragmatic annotationProceedings of 5th International Conference on Language Resources and Evaluation, LREC-2006.
67. Passonneau, R. J. (1995). Integrating Gricean and attentional constraintsProceedings of 14th International Joint Conference on Artificial Intelligence (IJCAI-95).
68. Pechmann, T. (1984). Accentuation and redundancy in children's and adults' referential communicationH. Bouma & D. G. Bouwhuis (Eds.), Attention and performance (Vol. 10, pp. 417-432). Hillsdale, NJ: Lawrence Erlbaum.
69. Pechmann, T. (1989). Incremental speech production and referential overspecification. Linguistics, 27, 89-110.
70. Piwek, P. (2009). Salience and pointing in multimodal referenceProceedings of Workshop ''Production of Referring Expressions: bridging the gap between computational and empirical approaches to generating reference'' (PRE-CogSci'09).
71. Reiter, E. (1990). The computational complexity of avoiding conversational implicaturesProceedings 28th Annual Meeting of the Association for Computational Linguistics.
72. Reiter, E., & Dale, R. (2000). Building natural language generation systems. Cambridge, UK: Cambridge University Press.
73. Schriefers, H., & Pechmann, T. (1988). Incremental production of referential noun phrases by human speakersM. Zock & G. Sabah (Eds.), Advances in natural language generation (Vol. 1, pp. 172-179). London: Pinter.
74. Sedivy, J., Tanenhaus, M., Chambers, C., & Carlson, G. (1999). Achieving incremental semantic interpretation through contextual representation. Cognition, 71, 109-147.
75. Siddharthan, A., & Copestake, A. (2004). Generating referring expressions in open domainsProceedings of 42nd Annual Meeting of the Association for Computational Linguistics, ACL-04.
76. van der Sluis, I., Gatt, A., & van Deemter, K. (2006). Manual for the tuna corpus: Referring expressions in two domains. Technical report, University of Aberdeen. Available at: Accessed January 18, 2011.
77. van der Sluis, I., Gatt, A., & van Deemter, K. (2007). Evaluating algorithms for the generation of referring expressions: Going beyond toy domains In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP-07). Borovets, Bulgaria.
78. Spanger, P., Kurosawa, T., & Tokunaga, T. (2008). TITCH: Attribute selection based on discrimination power and frequencyProceedings of 5th International Conference on Natural Language Generation (INLG-08).
79. Stone, M. (2000). On identifying setsProceedings of 1st International Conference on Natural Language Generation, INLG-00.
80. Stone, M., Doran, C., Webber, B., Bleam, T., & Palmer, M. (2003). Microplanning with communicative intentions: The spud system. Computational Intelligence, 19(4), 311-381.
81. von Stutterheim, C., Mangold-Allwinn, R., Barattelli, S., Kohlmann, U., & Kölbing, H.-G. (1993). Reference to objects in text production. Belgian Journal of Linguistics, 8, 99-125.
82. Theune, M., Touset, P., Viethen, J., & Krahmer, E. (2007). Cost-based attribute selection for generating referring expressions (graph-fp and graph-sc)Proceedings of UCNLG+MT: Language Generation and Machine Translation (pp. 95-97).
83. Turing, A. (1950). Computing machinery and intelligence. Mind, LIX(2236), 433-460.
84. Turner, R., Sripada, S., Reiter, E., & Davy, I. (2007). Selecting the content of textual descriptions of geographically located events in spatio-temporal weather dataProceedings of the Conference on Applications and Innovations in Intelligent Systems XV (AI-07), Cambridge, UK.
85. Van der Sluis, I., & Krahmer, E. (2004). The influence of target size and distance on the production of speech and gesture in multimodal referring expressionsProceedings of ICSLP-2004, October 4-8, Jeju, Korea.
86. Viethen, J., & Dale, R. (2006). Algorithms for generating referring expressions: Do they do what people do?Proceedings of 4th International Conference on Natural Language Generation, INLG-06.
87. Whitehurst, G., & Sonnenschein, S. (1978). The development of communication: Attribute variation leads to contrast failure. Journal of Experimental Child Psychology, 25, 490-504.
88. Whitehurst, G. J. (1976). The development of communication: Changes with age and modeling. Child Development Development, 47, 473-482.