Bringing Machine Learning and Compositional Semantics Together

Annual Review of Linguistics

Volumn 1, Issue 1, 2015, Pages 355-376

  Liang, Percy ^a   Potts, Christopher ^a  

^a STANFORD UNIVERSITY   (United States)

Author keywords

compositionality; discriminative learning; distributed representations; logical forms; recursive neural networks; semantic parsing

Indexed keywords

EID: 84987707828     PISSN: 23339683     EISSN: 23339691     Source Type: Journal    
DOI: 10.1146/annurev-linguist-030514-125312     Document Type: Review

References (108)

1. AnderBois S. Brasoveanu A, Henderson R. 2012. The pragmatics of quantifier scope: a corpus study. In Proceedings of Sinn und Bedeutung 16, ed. A Aguilar-Guevara, A Chernilovskaya, R Nouwen, 1:15-28. Cambridge, MA: MIT Work. Pap. Linguist.
2. Artzi Y, Zettlemoyer LS. 2011. Bootstrapping semantic parsers from conversations. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pp. 421-32. Stroudsburg, PA: Assoc. Comput. Linguist.
3. Artzi Y, Zettlemoyer LS. 2013. Weakly supervised learning of semantic parsers for mapping instructions to actions. Trans. Assoc. Comput. Linguist. 1:49-62
4. Baklr G, Hofmann T, Schölkopf B, Smola AJ, Taskar B, ed. 2010. Predicting Structured Data. Cambridge, MA: MIT Press
5. Baroni M, Bernardi R, Do NQ. Shan CC. 2012. Entailment above the word level in distributional semantics. In Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics, pp. 23-32. Stroudsburg, PA: Assoc. Comput. Linguist.
6. Berant J, Chou A, Frostig R, Liang P. 2013. Semantic parsing on Freebase from question-answer pairs. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 1533-44. Stroudsburg, PA: Assoc. Comput. Linguist.
7. Berant J, Liang P. 2014. Semantic parsing via paraphrasing. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pp. 1415-25. Stroudsburg, PA: Assoc. Comput. Linguist.
8. Bird S, Klein E, Loper E. 2009. Natural Language Processing with Python. Sebastopol, CA: O'Reilly
9. Blackburn P, Bos J. 2003. Computational semantics. Theoria 18:27-45
10. Blackburn P, Bos J. 2005. Representation and Inference for Natural Language: A First Course in Computational Semantics. Stanford, CA: Cent. Study Lang. Inf.
11. Bollacker K, Evans C, Paritosh P, Sturge T, Taylor J. 2008. Freebase: a collaboratively created graph database for structuring human knowledge. In Proceedings of the 2008 Assoc. Comput. Mach. SIGMOD International Conference on Management of Data, pp. 1247-50. New York: Assoc. Comput. Mach.
12. Bottou L. 2010. Large-scale machine learning with stochastic gradient descent. In Proceedings of the 19th International Conference on Computational Statistics, ed. Y Lechevallier, G Saporta, pp. 177-86. Berlin: Springer
13. Bowman SR. 2014. Can recursive neural tensor networks learn logical reasoning? Presented at 2nd Int. Conf. Learn. Represent., Banff, Can.
14. Boyd S, Vandenberghe L. 2004. Convex Optimization. Cambridge, UK: Cambridge Univ. Press
15. Branavan S, Silver D, Barzilay R. 2011. Learning to win by reading manuals in a Monte-Carlo framework. J. Artif. Intell. Res. 43:661-704
16. Branavan S, Zettlemoyer L, Barzilay R. 2010. Reading between the lines: learning to map high-level instructions to commands. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pp. 1268-77. Stroudsburg, PA: Assoc. Comput. Linguist.
17. Cai Q, Yates A. 2013. Large-scale semantic parsing via schema matching and lexicon extension. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, 1:423-33. Stroudsburg, PA: Assoc. Comput. Linguist.
18. Carpenter B. 1997. Type-Logical Semantics. Cambridge, MA: MIT Press
19. Chen D. 2012. Fast online lexicon learning for grounded language acquisition. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics, 1:430-39. Stroudsburg, PA: Assoc. Comput. Linguist.
20. Chierchia G, Turner R. 1988. Semantics and property theory. Linguist. Philos. 11:261-302
21. Clark S, Coecke B, Sadrzadeh M. 2011. Mathematical foundations for a compositional distributed model of meaning. Linguist. Anal. 36:345-84
22. Clarke J, Goldwasser D, Chang MW, Roth D. 2010. Driving semantic parsing from the world's response. In Proceedings of the 14th Conference on Computational Natural Language Learning, pp. 18-27. Stroudsburg, PA: Assoc. Comput. Linguist.
23. Collobert R, Weston J. 2008. A unified architecture for natural language processing: deep neural networks with multitask learning. In Proceedings of the 25th International Conference on Machine Learning, pp. 160-67. New York: Assoc. Comput. Mach.
24. Collobert R, Weston J, Bottou L, Karlen M, Kavukcuoglu K, Kuksa P. 2011. Natural language processing (almost) from scratch. J. Mach. Learn. Res. 12:2493-537
25. Cooper R. 2012. Book review: Computational semantics with functional programming, Jan van Eijck and Christina Unger. Comput. Linguist. 38:447-49
26. Dagan I, Glickman O, Magnini B. 2006. The PASCAL recognising textual entailment challenge. In Machine Learning Challenges, Vol. 2944: Lecture Notes in Computer Science, ed. J Quiñonero-Candela, I Dagan, B Magnini, F d'Alché Buc, pp. 177-90. Berlin: Springer
27. de Marneffe MC, MacCartney B, Manning CD. 2006. Generating typed dependency parses from phrase structure parses. In Proceedings of the Fifth International Conference on Language Resources and Evaluation, pp. 1-8. Stroudsburg, PA: Assoc. Comput. Linguist.
28. Domingos P. 2012. A few useful things to know about machine learning. Commun. Assoc. Comput. Mach. 55:78-87
29. Dowty D. 2007. Compositionality as an empirical problem. In Direct Compositionality, ed. C Barker, P Jacobson, pp. 23-101. Oxford, UK: Oxford Univ. Press
30. Firth JR. 1935. The technique of semantics. Trans. Philol. Soc. 34:36-73
31. Ge R, Mooney RJ. 2005. A statistical semantic parser that integrates syntax and semantics. In Proceedings of the Ninth Conference on Computational Natural Language Learning, pp. 9-16. Stroudsburg, PA: Assoc. Comput. Linguist.
32. Graff D. 2000. Shifting sands: an interest-relative theory of vagueness. Philos. Top. 28:45-81
33. Grefenstette E, Sadrzadeh M, Clark S, Coecke B, Pulman S. 2011. Concrete sentence spaces for compositional distributional models of meaning. In Proceedings of the 9th International Conference on Computational Semantics, pp. 125-34. Stroudsburg, PA: Assoc. Comput. Linguist.
34. Harris Z. 1954. Distributional structure. Word 10:146-62
35. Heim I, Kratzer A. 1998. Semantics in Generative Grammar. Oxford, UK: Blackwell
36. Higgins D, Sadock JM. 2003. A machine learning approach to modeling scope preferences. Comput. Linguist. 29:73-96
37. Jackendoff RS. 1992. Languages of the Mind. Cambridge, MA: MIT Press
38. Jackendoff RS. 1997. The Architecture of the Language Faculty. Cambridge, MA: MIT Press
39. Janssen TMV. 1997. Compositionality. In Handbook of Logic and Language, ed. J van Benthem, AterMeulen, pp. 417-73. Cambridge, MA/Amsterdam: MIT Press/North-Holland
40. Jurafsky D, Martin JH. 2009. Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition. Englewood Cliffs, NJ: Prentice-Hall. 2nd ed.
41. Kamp H, Partee BH. 1995. Prototype theory and compositionality. Cognition 57:129-91
42. Kamp H, Reyle U. 1993. From Discourse to Logic: Introduction to Modeltheoretic Semantics of Natural Language, Formal Logic and Discourse Representation Theory. Dordrecht, Neth.: Kluwer
43. Kate RJ, Mooney RJ. 2006. Using string-kernels for learning semantic parsers. In Proceedings of the 21st International Conference on Computational Linguistics and the 44th Annual Meeting of the Association for Computational Linguistics, pp. 913-20. Stroudsburg, PA: Assoc. Comput. Linguist.
44. Kate RJ, Wong YW, Mooney RJ. 2005. Learning to transform natural to formal languages. In Proceedings of the 20th National Conference on Artificial Intelligence, 3:1062-68. Palo Alto, CA: Assoc. Adv. Artif. Intell.
45. Katz JJ. 1972. Semantic Theory. New York: Harper & Row
46. Katz JJ. 1996. Semantics in linguistics and philosophy: an intensionalist perspective. In The Handbook of Contemporary Semantic Theory, ed. S Lappin, pp. 599-616. Oxford, UK: Blackwell
47. Katz JJ, Fodor JA. 1963. The structure of semantic theory. Language 39:170-210
48. Kennedy C. 2011. Ambiguity and vagueness. In Semantics: An International Handbook of Natural Language Meaning, ed. C Maienborn, K von Heusinger, P Portner, 1:507-35. Berlin: Mouton de Gruyter
49. Klein E, Sag IA. 1985. Type-driven translation. Linguist. Philos. 8:163-201
50. Kripke S. 1975. Outline of a theory of truth. J. Philos. 72:690-716
51. Krishnamurthy J, Kollar T. 2013. Jointly learning to parse and perceive: connecting natural language to the physical world. Trans. Assoc. Comput. Linguist. 1:193-206
52. Kushman N, Barzilay R. 2013. Using semantic unification to generate regular expressions from natural language. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 826-36. Stroudsburg, PA: Assoc. Comput. Linguist.
53. Kwiatkowksi T, Zettlemoyer L, Goldwater S, Steedman M. 2010. Inducing probabilistic CCG grammars from logical form with higher-order unification. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pp. 1223-33. Stroudsburg, PA: Assoc. Comput. Linguist.
54. Kwiatkowski T, Choi E, Artzi Y, Zettlemoyer L. 2013. Scaling semantic parsers with on-the-fly ontology matching. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 1545-56. Stroudsburg, PA: Assoc. Comput. Linguist.
55. Kwiatkowski T, Zettlemoyer LS, Goldwater S, Steedman M. 2011. Lexical generalization in CCG grammar induction for semantic parsing. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pp. 1512-23. Stroudsburg, PA: Assoc. Comput. Linguist.
56. Landauer TK, McNamara DS, Dennis S, Kintsch W 2007. Handbook of Latent Semantic Analysis. Hillsdale, NJ: Erlbaum
57. Lei T, Long F, Barzilay R, Rinard M. 2013. From natural language specifications to program input parsers. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, 1:1294-303. Stroudsburg, PA: Assoc. Comput. Linguist.
58. Lewis D. 1970. General semantics. Synthese 22:18-67
59. Lewis M, Steedman M. 2013. Combined distributional and logical semantics. Trans. Assoc. Comput. Linguist. 1:179-92
60. Liang P, Jordan M, Klein D. 2011. Learning dependency-based compositional semantics. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pp. 590-99. Stroudsburg, PA: Assoc. Comput. Linguist.
61. Liang P, Jordan MI, Klein D. 2013. Learning dependency-based compositional semantics. Comput. Linguist. 39:389-446
62. Lu W, Ng HT, Lee WS, Zettlemoyer LS. 2008. A generative model for parsing natural language to meaning representations. In Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing, pp. 783-92. Stroudsburg, PA: Assoc. Comput. Linguist.
63. MacCartney B, Manning CD. 2009. An extended model of natural logic. In Proceedings of the Eighth International Conference on Computational Semantics, pp. 140-56. Stroudsburg, PA: Assoc. Comput. Linguist.
64. Manning CD, Schütze H. 1999. Foundations of Statistical Natural Language Processing. Cambridge, MA: MIT Press
65. Matuszek C, FitzGerald N, Zettlemoyer LS, Bo L, Fox D. 2012a. A joint model of language and perception for grounded attribute learning. In Proceedings of the 29th International Conference on Machine Learning, ed. J Langford, J Pineau, pp. 1671-78. Madison, WI: Omnipress
66. Matuszek C, Herbst E, Zettlemoyer LS, Fox D. 2012b. Learning to parse natural language commands to a robot control system. In Proceedings of the 13th International Symposium on Experimental Robotics, pp. 403-15. Berlin: Springer
67. Mitchell J, Lapata M. 2010. Composition in distributional models of semantics. Cogn. Sci. 34:1388-429
68. Mitchell TM. 1997. Machine Learning. New York: McGraw Hill
69. Montague R. 1970. Universal grammar. Theoria 36:373-398
70. Montague R. 1974. Formal Philosophy: Selected Papers of Richard Montague. New Haven, CT: Yale Univ. Press
71. Moss LS. 2009. Natural logic and semantics. In Proceedings of the 18th Amsterdam Colloquium: Revised Selected Papers, ed. M Aloni, H Bastiaanse, T de Jager, P van Ormondt, K Schulz, pp. 71-80. Berlin: Springer
72. Ng HT, Zelle J. 1997. Corpus-based approaches to semantic interpretation in natural language processing. Artif. Intell. Mag. 18:45-64
73. Palmer M, Gildea D, Xue N. 2010. Semantic Role Labeling. San Rafael, CA: Morgan & Claypool
74. Partee BH. 1980. Semantics-mathematics or psychology? In Semantics from Different Points of View, ed. E Bäuerle, A von Stechow, pp. 1-14. Berlin: Springer
75. Partee BH. 1981. Montague grammar, mental representations, and reality. In Philosophy and Grammar, ed. S Kanger, S Öhman, pp. 59-78. Dordrecht, Neth.: Reidel
76. Partee BH. 1995. Lexical semantics and compositionality. In Invitation to Cognitive Science, Vol. 1, ed. LR Gleitman, M Liberman, pp. 311-60. Cambridge, MA: MIT Press
77. Partee BH. 2004 (1984). Compositionality. In Compositionality in Formal Semantics, pp. 153-81. Oxford, UK: Blackwell
78. Pereira FCN. 2000. Formal grammar and information theory: together again? Philos. Trans. R. Soc. 358:1239-53
79. Reinhart T. 1997. Quantifier scope: how labor is divided between QR and choice functions. Linguist. Philos. 20:335-97
80. Rumelhart DE, Hinton GE, Williams RJ. 1986a. Learning internal representations by error propagation. In Parallel Distributed Processing. Explorations in the Microstructure of Cognition, Vol. 1: Foundations, ed. DE Rumelhart, JL McClelland, pp. 318-62. Cambridge, MA: MIT Press
81. Rumelhart DE, Hinton GE, Williams RJ. 1986b. Learning representations by back-propagating errors. Nature 323:533-36
82. Samuel AL. 1959. Some studies in machine learning using the game of checkers. IBM J. Res. Dev. 3:211-29
83. Samuel AL. 1967. Some studies in machine learning using the game of checkers. II. Recent progress.IBMJ. Res. Dev. 11:601-17
84. Saurí R, Pustejovsky J. 2009. FactBank: a corpus annotated with event factuality. Lang. Resour. Eval. 43:227-68
85. Smith NA. 2011. Linguistic Structure Prediction. San Rafael, CA: Morgan & Claypool
86. Socher R, Bauer J, Manning CD, Andrew YN. 2013a. Parsing with compositional vector grammars. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, 1:455-65. Stroudsburg, PA: Assoc. Comput. Linguist.
87. Socher R, Huang EH, Pennin J, Manning CD, Ng AY. 2011a. Dynamic pooling and unfolding recursive autoencoders for paraphrase detection. In Advances in Neural Information Processing Systems 24, ed. J Shawe-Taylor, RS Zemel, PL Bartlett, FCN Pereira, KQWeinberger, pp. 801-9. Red Hook, NY: Curran
88. Socher R, Huval B, Manning CD, Ng AY. 2012. Semantic compositionality through recursive matrix-vector spaces. In Proceedings of the 2012 Conference on Empirical Methods in Natural Language Processing, pp. 1201-11. Stroudsburg, PA: Assoc. Comput. Linguist.
89. Socher R, Pennington J, Huang EH, Ng AY, Manning CD. 2011b. Semi-supervised recursive autoencoders for predicting sentiment distributions. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pp. 151-61. Stroudsburg, PA: Assoc. Comput. Linguist.
90. Socher R, Perelygin A, Wu J, Chuang J, Manning CD, et al. 2013b. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 1631-42. Stroudsburg, PA: Assoc. Comput. Linguist.
91. Szabó ZG. 2012. Compositionality. In The Stanford Encyclopedia of Philosophy, ed. EN Zalta. Stanford, CA: Cent. Study Lang. Inf. Winter ed. http://plato.stanford.edu/entries/compositionality/
92. Szabolcsi A. 2009. Quantification. Cambridge, UK: Cambridge Univ. Press
93. Tang LR, Mooney RJ. 2001. Using multiple clause constructors in inductive logic programming for semantic parsing. In Proceedings of the 12th European Conference on Machine Learning, pp. 466-77. Berlin: Springer
94. Taskar B, Guestrin C, Koller D. 2003. Max-margin Markov networks. In Advances in Neural Information Processing Systems 17, ed. S Thrun, LK Saul, B Schölkopf, pp. 25-32. Cambridge, MA: MIT Press
95. Tellex S, Kollar T, Dickerson S, Walter MR, Banerjee AG, et al. 2011. Understanding natural language commands for robotic navigation and mobile manipulation. In Proceedings of the 25th AAAI Conference on Artificial Intelligence, pp. 1507-14. Palo Alto, CA: Assoc. Adv. Artif. Intell.
96. Thompson CA, Mooney RJ. 2003. Acquiring word-meaning mappings for natural language interfaces. J. Artif. Intell. Res. 18:1-44
97. Turney PD, Pantel P. 2010. From frequency to meaning: vector space models of semantics. J. Artif. Intell. Res. 37:141-88
98. van Eijck J, Unger C. 2010. Computational Semantics with Functional Programming. Cambridge, UK: Cambridge Univ. Press
99. Vogel A, Jurafsky D. 2010. Learning to follow navigational directions. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pp. 806-14. Stroudsburg, PA: Assoc. Comput. Linguist.
100. Warren DHD, Pereira FCN. 1982. An efficient easily adaptable system for interpreting natural language queries. Am. J. Comput. Linguist. 8:110-22
101. Werning M, Hinzen W, Machery E. 2012. The Oxford Handbook of Compositionality. Oxford, UK: Oxford Univ. Press
102. Wong YW, Mooney RJ. 2006. Learning for semantic parsing with statistical machine translation. In Proceedings of the Human Language Technology Conference of the NAACL, Main Conference, pp. 439-46. Stroudsburg, PA: Assoc. Comput. Linguist.
103. Wong YW, Mooney RJ. 2007. Learning synchronous grammars for semantic parsing with lambda calculus. In Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics, pp. 960-67. Stroudsburg, PA: Assoc. Comput. Linguist.
104. Woods WA, Kaplan RM, Nash-Webber B. 1972. The lunar sciences natural language information system: final report. Tech. rep., BBN rep. 2378, BBN Technol., Cambridge, MA
105. Yu CNJ, Joachims T. 2009. Learning structural SVMs with latent variables. In Proceedings of the 26th International Conference on Machine Learning, pp. 1169-76. New York: Assoc. Comput. Mach.
106. Zelle JM, Mooney RJ. 1996. Learning to parse database queries using inductive logic programming. In Proceedings of the 13th National Conference on Artificial Intelligence, 2:1050-55. Palo Alto, CA: Assoc. Adv. Artif. Intell.
107. Zettlemoyer LS, Collins M. 2005. Learning to map sentences to logical form: Structured classification with probabilistic categorial grammars. In Proceedings of the 21st Conference on Uncertainty in Artificial Intelligence, pp. 658-66. Arlington, VA: Assoc. Uncertain. Artif. Intell.
108. Zettlemoyer LS, Collins M. 2007. Online learning of relaxed CCG grammars for parsing to logical form. In Proceedings of the 2007 Joint Conference on EmpiricalMethods in Natural Language Processing and Computational Natural Language Learning, pp. 678-87. Stroudsburg, PA: Assoc. Comput. Linguist.