Language learning from positive evidence, reconsidered: A simplicity-based approach

Topics in Cognitive Science

Volumn 5, Issue 1, 2013, Pages 35-55

  Hsu, Anne S ^a   Chater, Nick ^b   Vitányi, Paul ^c  

^a UNIVERSITY COLLEGE LONDON   (United Kingdom)

^b UNIVERSITY OF WARWICK   (United Kingdom)

^c CWI   (Netherlands)

Author keywords

Child language acquisition; Induction; Learning; Minimum description length; Natural language; No negative evidence; Probabilistic Bayesian models; Simplicity principle

Indexed keywords

ARTICLE; BAYES THEOREM; HUMAN; INFORMATION SCIENCE; LANGUAGE DEVELOPMENT; LEARNING; LINGUISTICS; PHYSIOLOGY; STATISTICAL MODEL; STATISTICS;

BAYES THEOREM; GENERALIZATION (PSYCHOLOGY); HUMANS; INFORMATION THEORY; LANGUAGE DEVELOPMENT; LEARNING; LINGUISTICS; MODELS, STATISTICAL; PROBABILITY LEARNING;

EID: 84872706418     PISSN: 17568757     EISSN: 17568765     Source Type: Journal    
DOI: 10.1111/tops.12005     Document Type: Article

References (67)

1. Angluin, D. (1980). Inductive inference of formal languages from positive data. Information and Control, 45, 117-135.
2. Angluin, D. (1988). Identifying languages from stochastic examples. Technical report YALEU/DCS/RR-614. New Haven, CT: Yale University.
3. Attneave, E., & Frost, R. (1969). The determination of perceived tridimensional orientation by minimum criteria. Perception & Psychophysics, 6, 391-396.
4. Baker, C. L., & McCarthy, J. J. (1981). The logical problem of language acquisition. Cambridge, MA: MIT Press.
5. Barendregt, H. P. (1984). The lambda calculus. Amsterdam: Elsevier.
6. Bloomfield, L. (1933). Language. New York: Henry Holt.
7. Bowerman, M. (1988). The 'No Negative Evidence' problem: How do children avoid constructing an overly general grammar? In J. Hawkins (Ed.), Explaining language universals (pp. 73-101). Oxford, England: Blackwell.
8. Brent, M. R., & Cartwright, T. A. (1996). Distributional regularity and phonotactic constraints are useful for segmentation. Cognition, 61, 93-126.
9. Brown, R., & Hanlon, C. (1970). Derivational complexity and order of acquisition in child speech. In J. R. Hayes (Ed.), Cognition and the development of language. New York: Wiley.
10. Carlucci, L., & Case, J. (2013). On the necessity of U-shaped learning. Topics in Cognitive Science (this issue), 5, 56-88.
11. Chater, N., & Vitányi, P. (2002). Simplicity: A unifying principle incognitive science? Trends in Cognitive Sciences, 7, 19-22.
12. Chater, N., & Vitányi, P. (2007). Ideal learning of natural language: Positive results about learning from positive evidence. Journal of Mathematical Psychology, 51, 135-163.
13. Chomsky, N. (1957). Syntactic structures. The Hague/Paris: Mouton.
14. Chomsky, N. (1980). Rules and representations. Cambridge, MA: MIT Press.
15. Christiansen, M. H., & Chater, N. (1994). Generalization and connectionist language learning. Mind & Language, 9, 273-287.
16. Christiansen, M. H., & Chater, N. (1999). Connectionist natural language processing: The state of the art. Cognitive Science, 23, 417-437.
17. Christiansen, M. H., & Chater, N. (2007). Generalization and connectionist language learning. Mind & Language, 9, 273-287.
18. Christiansen, M., & Chater, N. (2010). Language acquisition meets language evolution. Cognitive Science, 34, 1131-1157.
19. Clark, A., & Eyraud, R. (2007). Polynomial identification in the limit of substitutable context-free languages. Journal of Machine Learning Research, 8, 1725-1745.
20. Clark, A., & Lappin, S. (2013). Complexity in language acquisition. Topics in Cognitive Science (this issue), 5, 89-110.
21. Cover, T. M., & Thomas, J. A. (2006). Elements of information theory (2nd ed.). Hoboken, NJ: Wiley.
22. Crain, S., & Lillo-Martin, D. (1999). Linguistic theory and language acquisition. Oxford, England: Blackwell.
23. Demetras, M., Post, K., & Snow, C. (1986). Feedback to first language learners: The role of repetitions and clarification questions. Journal of Child Language, 13, 275-292.
24. Dowman, M. (2000). Addressing the learnability of verb subcategorizations with Bayesian inference. In L. R. Gleitman & A. K. Joshi (Eds.), Proceedings of the Twenty Second Annual Conference of the Cognitive Science Society (pp. 107-112). Mahwah, NJ: Erlbaum.
25. Dresher, B., & Hornstein, N. (1976). On some supposed contributions of artificial intelligence to the scientific study of language. Cognition, 4, 321-398.
26. Ellison, M. (1992). The machine learning of phonological structure. PhD Thesis, University of Western Australia, Crawley, Western Australia.
27. Elman, J. (1990). Finding structure in time. Cognitive Science, 14, 179-211.
28. Feldman, J. (1972). Some decidability results on grammatical inference and complexity. Information and Control, 20, 244-262.
29. Feldman, J. (2000). Minimization of Boolean complexity in human concept learning. Nature, 403, 630-633.
30. Gold, E. M. (1967). Language identification in the limit. Information and Control, 16, 447-474.
31. Goldsmith, J. (2001). Unsupervised learning of the morphology of a natural language. Computational Linguistics, 27, 153-198.
32. Hochberg, J., & McAlister, E. (1953). A quantitative approach to figure "goodness." Journal of Experimental Psychology, 46, 361-364.
33. Horning, J. J. (1969). A study of grammatical inference. Technical Report CS 139. Palo Alto, CA: Stanford University.
34. Hornstein, N., & Lightfoot, D. (1981). Explanation in linguistics: The logical problem of language acquisition. London: Longman.
35. Hsu, A., & Chater, N. (2010). The logical problem of language acquisition: A probabilistic perspective. Cognitive Science, 34, 972-1016.
36. Hsu, A., Chater, N., & Vitányi, P. (2011). The probabilistic analysis of language acquisition: Theoretical, computational, and experimental analysis. Cognition, 120, 380-390.
37. Jain, S., Osherson, D. N., Royer, J. S., & Kumar Sharma, A. (1999). Systems that learn (2nd ed.). Cambridge, MA: MIT Press.
38. Kemp, C., Perfors, A., & Tenenbaum, J. B. (2007). Learning over hypothesis with hierarchical Bayesian models. Developmental Science, 10, 307-321.
39. Klein, D., & Manning, C. (2005). Natural language grammar induction with a generative constituent-context model. Pattern Recognition, 38, 1407-1409.
40. Langley, P., & Stromsten, S. (2000). Learning context-free grammars with a simplicity bias. In Proceedings of the Eleventh European Conference on Machine Learning (pp. 220-228). London: Springer-Verlag.
41. Leeuwenberg, E. L. J. (1969). Quantitative specification of information in sequential patterns. Psychological Review, 76, 216-220.
42. Li, M., & Vitányi, P. (1997). An introduction to Kolmogorov complexity and its applications (2nd ed.). London: Springer.
43. Mach, E. (1959). The analysis of sensations and the relation of the physicalto the psychical. New York: Dover Publications (original work published in 1886).
44. MacWhinney, B. (1993). The (il)logical problem of language acquisition. In Proceedings of the 15th annual conference of the Cognitive Science Society (pp. 61-70). Mahwah, NJ: Erlbaum.
45. MacWhinney, B. (2004). A multiple process solution to the logical problem of language acquisition. Journal of Child Language, 31, 883-914.
46. Marcus, G. F. (1993). Negative evidence in language acquisition. Cognition, 46, 53-85.
47. Niyogi, P. (2006). The computational nature of language learning and evolution. Cambridge, MA: MIT Press.
48. Odifreddi, P. (1988). Classical recursion theory. North Holland: Elsevier.
49. Onnis, L., Roberts, M., & Chater, N. (2002). Simplicity: A cure for overgeneralizations in language acquisition? In D. Gray & C. D. Schunn (Eds.), Proceedings of the 24th Annual Conference of the Cognitive Science Society (pp. 720-725. Mahwah, NJ: Erlbaum.
50. Pereira, F. C. N., & Warren, D. H. D. (1980). Definite clause grammars for language analysis. Artificial Intelligence, 13, 231-278.
51. Perfors, A., Regier, T., & Tenenbaum, J. B. (2006). Poverty of the Stimulus? A rational approach. In R. Sun & N. Miyaki (Eds.), Proceedings of the Twenty-Eighth Annual Conference of the Cognitive Science Society (pp. 663-668.) Mahwah, NJ: Erlbaum.
52. Pinker, S. (1979). Formal models of language learning. Cognition, 7, 217-283.
53. Pinker, S. (1984). Language learnability and language development (7th ed.). Cambridge, MA: Harvard University Press.
54. Pinker, S. (1989). Learnability and cognition: The acquisition of argument structure. Cambridge, MA: MIT Press.
55. Pinker, S., & Bloom, P. (1990). Natural language and natural selection. Behavioral and Brain Sciences, 13, 707-784.
56. Pylyshyn, Z. W. (1984). Computation and cognition: Toward a foundation for cognitive science. Cambridge, MA: Bradford Books/MIT Press.
57. Rissanen, J. (1987). Stochastic complexity. Journal of the Royal Statistical Society, Series B, 49, 223-239.
58. Rohde, D. L. T., & Plaut, D. C. (1999). Language acquisition in the absence of explicit negative evidence: How important is starting small?. Cognition, 72, 68-109.
59. Shannon, C. E. (1951). Prediction and entropy of printed English. Bell Systems Technical Journal, 31, 64.
60. Solomonoff, R. J. (1978). Complexity-based induction systems: comparisons and convergence theorems. IEEE Transactions on Information Theory, IT, 24, 422-432.
61. Steyvers, M., Griffiths, T., & Dennis, S. (2006). Probabilistic inference in human semantic memory. Trends in Cognitive Sciences, 10, 309-318.
62. Stolcke, A. (1994). Bayesian learning of probabilistic language models. Berkeley: University of California.
63. Tomasello, M. (2004). Syntax or semantics? Response to Lidz et al. Cognition, 93, 139-140.
64. Vousden, J. I., Ellefson, M. R., Solity, J. E., & Chater, N. (2011). Simplifying reading: Applying the simplicity principle to reading. Cognitive Science, 35, 34-78.
65. Wallace, C. S., & Freeman, P. R. (1987). Estimation and inference by compact coding. Journal of the Royal Statistical Society, Series B, 49, 240-251.
66. Wharton, R. M. (1974). Approximate language identification. Information and Control, 26, 236-255.
67. Wolff, J. G. (1988). Learning syntax and meanings through optimisation and distributional analysis. In Y. Levy, I. M. Schlesinger, & M. D. S. Braine (Eds.), Categories and processes in language acquisition, (pp. 179-215). Hillsdale, NJ: LEA.