Approaching the symbol grounding problem with probabilistic graphical models

AI Magazine

Volumn 32, Issue 4, 2011, Pages 64-76

  Tellex, Stefanie ^a   Kollar, Thomas ^b   Dickerson, Steven ^a   Walter, Matthew R ^a   Banerjee, Ashis Gopal ^a   Teller, Seth ^c   Roy, Nicholas ^a  

^a MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^b MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

^c USA   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

GRAPHIC METHODS; LINGUISTICS;

IN-DOOR NAVIGATIONS; LINGUISTIC STRUCTURE; MOBILE MANIPULATION; NATURAL LANGUAGES; PROBABILISTIC GRAPHICAL MODELS; PROBABILISTIC INFERENCE; SEQUENTIAL STRUCTURE; SYMBOL GROUNDING PROBLEM;

ROBOTS;

EID: 84858436215     PISSN: 07384602     EISSN: None     Source Type: Journal    
DOI: 10.1609/aimag.v32i4.2384     Document Type: Article

References (32)

1. Andrew, G., and Gao, J. 2007. Scalable Training of L1-Regularized Log-Linear Models. In Proceedings of the Twenty-Fourth International Conference on Machine Learning (ICML). New York: Association for Computing Machinery.
2. Bailey, D. 1997. When Push Comes to Shove: A Computational Model of the Role of Motor Control in the Acquisition of Action Verbs. Ph.D. Dissertation, Department of Computer Science, University of California, Berkeley, CA.
3. Branavan, S. R. K.; Chen, H.; Zettlemoyer, L. S.; and Barzilay, R. 2009. Reinforcement Learning for Mapping Instructions to Actions. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics, 82-90. Stroudsberg, PA: Association for Computational Linguistics.
4. Branavan, S.; Silver, D.; and Barzilay, R. 2011. Learning to Win by Reading Manuals in a Monte-Carlo Framework. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics. Stroudsberg, PA: Association for Computational Linguistics.
5. Bugmann, G.; Klein, E.; Lauria, S.; and Kyriacou, T. 2004. Corpus-Based Robotics: A Route Instruction Example. Proceedings of the 8th Intelligent Autonomous Systems Conference, 96-103. Amsterdam: IOS Press.
6. de Marneffe, M.; MacCartney, B.; and Manning, C. 2006. Generating Typed Dependency Parses from Phrase Structure Parses. In Proceedings of the International Conference on Language Resources and Evaluation (LREC), 449-454. Paris: European Language Resources Association.
7. Dzifcak, J.; Scheutz, M.; Baral, C.; and Schermerhorn, P. 2009. What to Do and How to Do It: Translating Natural Language Directives into Temporal and Dynamic Logic Representation for Goal Management and Action Execution. In Proceedings of the 2009 IEEE International Conference on Robotics and Automation (ICRA), 4163-4168. Piscataway, NJ: The Institute of Electrical and Electronics Engineers.
8. Ge, R., and Mooney, R. J. 2005. A Statistical Semantic Parser That Integrates Syntax and Semantics. In Proceedings of the Ninth Conference on Computational Natural Language Learning, 9-16. Stroudsburg, PA: Association for Computational Linguistics.
9. Harnad, S. 1990. The Symbol Grounding Problem. Physica D 42(1-3): 335-346.
10. Hsiao, K.-Y.; Mavridis, N.; and Roy, D. 2003. Coupling Perception and Simulation: Steps Towards Conversational Robotics. In Proceedings of the 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems, volume 1, 928-933. Piscataway, NJ: The Institute of Electrical and Electronics Engineers.
11. Jackendoff, R. S. 1983. Semantics and Cognition, 161-187. Cambridge, MA: The MIT Press.
12. Kollar, T., and Roy, N. 2009. Utilizing Object-Object and Object-Scene Context When Planning to Find Things. In Proceedings of the 2009 IEEE International Conference on Robotics and Automation (ICRA), 4116-4121. Piscataway, NJ: The Institute of Electrical and Electronics Engineers.
13. Kollar, T.; Tellex, S.; Roy, D.; and Roy, N. 2010a. Grounding Verbs of Motion in Natural Language Commands to Robots. In Proceedings of the 12th International Symposium on Experimental Robotics (ISER). Berlin: Springer.
14. Kollar, T.; Tellex, S.; Roy, D.; and Roy, N. 2010b. Toward Understanding Natural Language Directions. In Proceedings of the 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI), 259-266. New York: Association for Computing Machinery.
15. Kollar, T.; Tellex, S.; and Roy, N. 2010. A Discriminative Model for Understanding Natural Language Route Directions. In Dialog with Robots: Papers from the AAAI Fall Symposium. Technical Report FS-10-05. Menlo Park, CA: AAAI Press.
16. Kress-Gazit, H., and Fainekos, G. E. 2008. Translating Structured English to Robot Controllers. Advanced Robotics 22(12): 1343-1359.
17. Kschischang, F. R.; Frey, B. J.; and Loeliger, H.-A. 2001. Factor Graphs and the Sum-Product Algorithm. IEEE Transactions on Information Theory 47(2): 498-519. (Pubitemid 32318087)
18. Lafferty, J. D.; McCallum, A.; and Pereira, F. C. N. 2001. Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data. In Proceedings of the International Conference on Machine Learning (ICML), 282-289. San Francisco: Morgan Kaufmann Publishers.
19. MacMahon, M.; Stankiewicz, B.; and Kuipers, B. 2006. Walk the Talk: Connecting Language, Knowledge, and Action in Route Instructions. In Proceedings of the Twenty-First National Conference on Artificial Intelligence (AAAI), 1475-1482. Menlo Park, CA: AAAI Press.
20. Marocco, D.; Cangelosi, A.; Fischer, K.; and Belpaeme, T. 2010. Grounding Action Words in the Sensorimotor Interaction with the World: Experiments with a Simulated iCub Humanoid Robot. Frontiers in Neurorobotics 4(7).
21. Matuszek, C.; Fox, D.; and Koscher, K. 2010. Following Directions Using Statistical Machine Translation. In Pro ceedings of the 5th ACM/IEEE International Conference on Human-Robot Interaction (HRI), 251-258. New York: Association for Computing Machinery.
22. Modayil, J., and Kuipers, B. 2007. Autonomous Development of a Grounded Object Ontology by a Learning Robot. In Proceedings of the Twenty-Second AAAI Conference on Artificial Intelligence, 1095-1101. Menlo Park, CA: AAAI Press. (Pubitemid 350149715)
23. Regier, T. P. 1992. The Acquisition of Lexical Semantics for Spatial Terms: A Connectionist Model of Perceptual Categorization. Ph.D. Dissertation, Department of Computer Science, University of California at Berkeley.
24. Roy, D. 2005. Semiotic Schemas: A Framework for Grounding Language in Action and Perception. Artificial Intelligence 167(1-2): 170-205. (Pubitemid 41240681)
25. Roy, D.; Hsiao, K.; and Mavridis, N. 2003. Conversational Robots: Building Blocks for Grounding Word Meanings. Proceedings of the HLT-NAACL03 Workshop on Learning Word Meaning from Non-Linguistic Data. Stroudsberg, PA: Association for Computational Linguistics.
26. Roy, D.; Patel, R.; DeCamp, P.; Kubat, R.; Fleischman, M.; Roy, B.; Mavridis, N.; Tellex, S.; Salata, A.; Guinness, J.; Levit, M.; and Gorniak, P. 2006. The Human Speechome Project. In Proceedings of the 28th Annual Cognitive Science Conference, 192-196. Wheat Ridge, CO: Cognitive Science Society, Inc. (Pubitemid 44617698)
27. Shimizu, N., and Haas, A. 2009. Learning to Follow Navigational Route Instructions. In Proceedings of the 21st International Joint Conference on Artifical Intelligence, 1488-1493. Menlo Park, CA: AAAI Press.
28. Sugita, Y., and Tani, J. 2005. Learning Semantic Combinatoriality from the Interaction Between Linguistic and Behavioral Processes. Adaptive Behavior -Animals, Animats, Software Agents, Robots, Adaptive Systems 13(1): 33-52. (Pubitemid 41419987)
29. Tellex, S.; Kollar, T.; Dickerson, S.; Walter, M.; Banerjee, A.; Teller, S.; and Roy, N. 2011. Understanding Natural Language Commands for Robotic Navigation and Mobile Manipulation. In Proceedings of the Twenty-Fifth AAAI Conference on Artificial Intelligence. Menlo Park, CA: AAAI Press.
30. Tellex, S.; Kollar, T.; Shaw, G.; Roy, N.; and Roy, D. 2010. Grounding Spatial Language for Video Search. In International Conference on Multimodal Interfaces and the Workshop on Machine Learning for Multimodal Interaction, ICMIMLMI '10, 31:1-31:8. New York: Association for Computing Machinery.
31. Vogel, A., and Jurafsky, D. 2010. Learning to Follow Navigational Directions. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics (ACL), 806-814. Stroudsburg, PA: Association for Computational Linguistics.
32. Winograd, T. 1970. Procedures as a Representation for Data in a Computer Program for Understanding Natural Language. Ph.D. Dissertation, Massachusetts Institute of Technology, Boston, MA.