Situated language understanding as filtering perceived affordances

Cognitive Science

Volumn 31, Issue 2, 2007, Pages 197-231

  Gorniak, Peter ^a   Roy, Deb ^a  

^a MIT MEDIA LAB   (United States)

Author keywords

Affordances; Computational model; Language understanding; Parsing games; Plan recognition; Situated language

Indexed keywords

EID: 34347327195     PISSN: 03640213     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (45)

1. Allen, J. and Perrault, R. (1980). Analyzing intention in utterances. Artificial Intelligence, 15, 143-178.
2. Barsalou, L. (1999). Perceptual symbol systems. Behavioural and Brain Sciences, 22(4), 577-609.
3. Bickhard, M. H. (2001). Function, anticipation and representation. In Dubois, D. M. (Ed.), Computing Anticipatory Systems. CASYS 2000 - Fourth International Conference (pp. 459-469). Melville, NY: American Institute of Physics.
4. Bobick, A. F. and Ivanov, Y. A. (1998). Action recognition using probabilistic parsing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Santa Barbara, CA.
5. Boutilier, C., Dean, T., and Hanks, S. (1999). Decision-theoretic planning: Structural assumptions and computational leverage. Journal of AI Research, 11, 1-94.
6. Bui, H. H., Venkatesh, S., and West, G. (2002). Policy recognition in the abstract hidden markov model. Journal of Artificial Intelligence Research, 17, 451-499.
7. Chapman, D. (1991). Vision, Instruction and Action. Cambridge, MA: MIT Press.
8. Clark, A. (1998). Being There: Putting Brain, Body and World Together Again. MIT Press, Cambridge, MA.
9. Collins, M. (2003). Head-driven statistical models for natural language parsing. Computational Linguistics, 29, 589-637.
10. Drescher, G. (1991). Made-up minds. Cambridge, MA: MIT Press.
11. Duranti, A. and Goodwin, C. (1992). Rethinking Context: Language as an Interactive Phenomenon. Cambridge, MA: Cambridge University Press.
12. Earley, J. (1970). An efficient context-free parsing algorithm. Communications of the ACM, 6(8), 451-455.
13. Erol, K., Hendler, J., and Nau, D. (1994). HTN planning: Complexity and expressivity. In Proceedings of the American Association for Artificial Intelligence. Seattle, WA.
14. Geib, C. and Goldman, R. (2005). Partial observability and probabilistic plan/goal recognition. In IJCAI-05 workshop on Modeling Others from Observations.
15. Gibson, J. (1977). The theory of affordances. In Shaw, R. and Bransford, J. (Eds.), Perceiving, Acting and Knowing (pp. 67-82). New York: Wiley.
16. Glenberg, A. M. (1997). What memory is for. Behavioural and Brain Sciences, 20, 1-55.
17. Glenberg, A. M. and Kaschak, M. P. (2002). Grounding language in action. Psychonomic Bulletin and Review, 9(3), 558-565.
18. Gorniak, P. and Roy, D. (2005a). Probabilistic grounding of situated speech using plan recognition and reference resolution. In Proceedings of the International Conference on Multimodal Interfaces, Trento, Italy.
19. Gorniak, P. and Roy, D. (2005b). Speaking with your Sidekick: Understanding situated speech in computer role playing games. In Proceedings of Artificial Intelligence and Digital Entertainment, Marina Del Ray.
20. Gorniak, P. J. and Roy, D. (2004). Grounded semantic composition for visual scenes. Journal of Artificial Intelligence Research, 21, 429-470.
21. Hsiao, K., Mavridis, N., and Roy, D. (2003). Coupling perception and simulation: Steps towards conversational robotics. In Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Las Vegas, NE.
22. Klein, D. and Manning, C. D. (2003). Accurate unlexicalized parsing. In Proceedings of the 41st Meeting of the Association of Computational Linguistics, Sapporo, Japan.
23. Laurence, S. and Margolis, E. (1999). Concepts and cognitive science. In Margolis, E. and Laurence, S. (Eds.), Concepts: Core Readings (pp. 3-81). Cambridge, MA: MIT Press.
24. Litman, D. J. and Allen, J. F. (1984). A plan recognition model for clarification subdialogues. In COLING (pp. 302-311).
25. Littman, M., Sutton, R., and Singh, S. (2001). Predictive representations of state. In Proceedings of the Neural Information Processing Systems Conference, Vancouver, Canada.
26. Mangu, L., Brill, E., and Stolcke, A. (1999). Finding consensus among words: Latticebased word error minimization. In Proceedings of EUROSPEECH'99, 1, 495-498.
27. Miller, G. A., Galanter, E., and Pribram, K. H. (1960). Plans and the Structure of Behavior. New York: Adams, Bannister, Cox.
28. Minsky, M. (1985). Society of Mind. New York: Simon and Schuster.
29. Nau, D., Au, T., Ilghami, O., Kuter, U., Murdock, W., and Wu, D. (2003). Shop2: An HTN planning system. Journal of Artificial Intelligence Research, 20, 379-404.
30. Prinz, J. (2002). Furnishing the Mind: Concepts and their Perceptual Basis. Cambridge, MA: MIT Press.
31. Pulvermüller, F., Härle, M., and Hummel, F. (2001). Walking or talking?: Behavioral and neurophysiological correlates of action verb processing. Brain and Language, 78, 143-168.
32. Putnam, H. (1975). The meaning of 'meaning.' In Philosophical Papers, Vol. 2: Mind, Language and Reality. Cambridge, MA: Cambridge University Press.
33. Pynadath, D. V. and Wellman, M. P. (2000). Probabilistic state-dependent grammars for plan recognition. In Proceedings of the Conference on Uncertainty in Artificial Intelligence, UA12000. Palo Alto, CA: Morgan Kaufmann Publishers.
34. Roy, D. (2005). Semiotic schemas: A framework for grounding language in action and perception. Artificial Intelligence, 167, 170-205.
35. Roy, D., Hsiao, K.-Y., and Mavridis, N. (2004). Mental imagery for a conversational robot. IEEE Transactions on Systems, Man, and Cybernetics, 34(3), 1374-1383.
36. Schank, R. C. and Abelson, R. P. (1977). Scripts, Plans, Goals and Understanding. Mahweh, NJ: Lawrence Erlbaum Associates.
37. Schuler, W. (2003). Using model-theoretic semantic interpretation to guide statistical parsing and word recognition in a spoken language interface. In Proceedings of the Association for Computational Linguistics, Sapporo, Japan.
38. Smith, B. C. (1996). On the Origin of Objects. Cambridge, MA: MIT Press.
39. Steedman, M. (2002). Formalizing affordance. In Proceedings of the 24th Annual Meeting of the Cognitive Science Society (pp. 834-839), Fairfax, VA.
40. Stolcke, A. (1995). An efficient probabilistic context-free parsing algorithm that computes prefix probabilities. Computational Linguistics, 21(2), 165-201.
41. Stone, M. (2001). Representing communicative intentions in collaborative conversational agents. In AAAI Fall Symposium on Intent Inference for Collaborative Tasks, North Falmouth, MA.
42. Stoytchev, A. (2005). Behavior-grounded representation of tool affordances. In Proceedings of IEEE International Conference on Robotics and Automation (ICRA), Barcelona, Spain.
43. Winograd, T. (1970). Procedures as a representation for data in a computer program for understanding natural language. Ph.D. thesis, Massachusetts Institute of Technology.
44. Witten, I. H. and Bell, T. C. (1991). The zero-frequency problem: Estimating the probabilities of novel events in adaptive text compression. IEEE Trans. Information Theory, 37(4), 1085-1094.
45. Zwaan, R. A. (2003). The immersed experiencer: Toward an embodied theory of language comprehension. The Psychology of Learning and Motivation, 44.