Learning indexed families of recursive languages from positive data: A survey

Theoretical Computer Science

Volumn 397, Issue 1-3, 2008, Pages 194-232

  Lange, Steffen ^a   Zeugmann, Thomas ^b   Zilles, Sandra ^c  

^a DARMSTADT UNIVERSITY OF APPLIED SCIENCES   (Germany)

^b HOKKAIDO UNIVERSITY   (Japan)

^c UNIVERSITY OF ALBERTA   (Canada)

Author keywords

Formal languages; Inductive inference; Query learning; Recursion theory

Indexed keywords

ARTIFICIAL INTELLIGENCE; LEARNING SYSTEMS; MATHEMATICAL MODELS; RECURSIVE FUNCTIONS;

INDUCTIVE INFERENCE; QUERY LEARNING; RECURSION THEORY;

FORMAL LANGUAGES;

EID: 43249126698     PISSN: 03043975     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.tcs.2008.02.030     Document Type: Article

References (104)

1. P.W. Adriaans, M. Trautwein, M. Vervoort, Towards high speed grammar induction on large text corpora. In: SOFSEM 2000: Theory and Practice of Informatics, 27th Conference on Current Trends in Theory and Practice of Informatics, November 25-December 2, 2000, Milovy, Czech Republic, Proceedings, 2000, pp. 173-186
2. Angluin D. Finding patterns common to a set of strings. J. Comput. Syst. Sci. 21 1 (1980) 46-62
3. Angluin D. Inductive inference of formal languages from positive data. Inform. Control 45 2 (1980) 117-135
4. Angluin D. Inference of reversible languages. J. ACM 29 3 (1982) 741-765
5. Angluin D. Learning regular sets from queries and counterexamples. Inform. Comput. 75 2 (1987) 87-106
6. Angluin D. Queries and concept learning. Machine Learning 2 4 (1988) 319-342
7. Angluin D. Queries revisited. Theoret. Comput. Sci. 313 2 (2004) 175-194
8. Baliga G.R., Case J., and Jain S. The synthesis of language learners. Inform. Comput. 152 1 (1999) 16-43
9. Barzdin J. On synthesizing programs given by examples. International Symposium on Theoretical Programming. Lecture Notes in Computer Science vol. 5 (1974), Springer-Verlag 53-63
10. Barzdin J. Inductive inference of automata, functions and programs. Amer. Math. Soc. Transl. (1977) 107-122
11. Barzdin J.M. Prognostication of automata and functions. In: Freiman C.V., Griffith J.E., and Rosenfeld J.L. (Eds). Information Processing 71, Proceedings of IFIP Congress 71, Volume 1 - Foundations and Systems, August 23-28, 1971, Ljubljana, Yugoslavia (1972), North-Holland 81-84
12. Barzdin J.M., and Freivald R.V. On the prediction of general recursive functions. Soviet Math. Dokl. 13 (1972) 1224-1228
13. Ya{cyrillic}.M{cyrillic}. B{cyrillic}a{cyrillic}r{cyrillic}z{cyrillic}d{cyrillic}i{cyrillic}n{cyri llic}{cyrillic small soft sign}, S{cyrillic}l{cyrillic}o{cyrillic}zh{cyrillic}n{cyrillic}o{cyrillic}s{cyr illic}t{cyrillic}{cyrillic small soft sign} i{cyrillic} ch{cyrillic}a{cyrillic}s{cyrillic}t{cyrillic}o{cyrillic}t{cyrillic}n{cyr illic}o{cyrillic}ye{cyrillic} r{cyrillic}ye{cyrillic}sh{cyrillic}ye{cyrillic}n{cyrillic}i{cyrillic}ye{ cyrillic} n{cyrillic}ye{cyrillic}k{cyrillic}o{cyrillic}t{cyrillic}o{cyrillic}r{cyr illic}y{cyrillic}kh{cyrillic} a{cyrillic}l{cyrillic}g{cyrillic}o{cyrillic}r{cyrillic}i{cyrillic}t{cyri llic}m{cyrillic}i{cyrillic}ch{cyrillic}ye{cyrillic}s{cyrillic}k{cyrillic }i{cyrillic} n{cyrillic}ye{cyrillic}r{cyrillic}a{cyrillic}z{cyrillic}r{cyrillic}ye{cy rillic}sh{cyrillic}i{cyrillic}m{cyrillic}y{cyrillic}kh{cyrillic} m{cyrillic}a{cyrillic}s{cyrillic}s{cyrillic}o{cyrillic}v{cyrillic}y{cyri llic}kh{cyrillic} p{cyrillic}r{cyrillic}o{cyrillic}b{cyrillic}l{cyrillic}ye{cyrillic}m{cyr illic}, D{cyrillic}o{cyrillic}k{cyrillic}t{cyrillic}. d{cyrillic}i{cyrillic}s{cyrillic}s{cyrillic}ye{cyrillic}r{cyrillic}t{cyr illic}a{cyrillic}ts{cyrillic}i{cyrillic}ya{cyrillic}, N{cyrillic}o{cyrillic}v{cyrillic}o{cyrillic}s{cyrillic}i{cyrillic}b{cyri llic}i{cyrillic}r{cyrillic}s{cyrillic}k{cyrillic}, 1971
14. Ya{cyrillic}.M{cyrillic}. B{cyrillic}a{cyrillic}r{cyrillic}z{cyrillic}d{cyrillic}i{cyrillic}n{cyri llic}{cyrillic small soft sign}, D{cyrillic}v{cyrillic}ye{cyrillic} t{cyrillic}ye{cyrillic}o{cyrillic}r{cyrillic}ye{cyrillic}m{cyrillic}y{cy rillic} o{cyrillic} p{cyrillic}r{cyrillic}ye{cyrillic}d{cyrillic}ye{cyrillic}l{cyrillic}{cyr illic small soft sign}n{cyrillic}o{cyrillic}m{cyrillic} s{cyrillic}i{cyrillic}n{cyrillic}t{cyrillic}ye{cyrillic}z{cyrillic}ye{cy rillic} f{cyrillic}u{cyrillic}n{cyrillic}k{cyrillic}ts{cyrillic}i{cyrillic}i{cyr illic, short}, in: J.M. Barzdin (Ed.), T{cyrillic}ye{cyrillic}o{cyrillic}r{cyrillic}i{cyrillic}ya{cyrillic} A{cyrillic}l{cyrillic}g{cyrillic}o{cyrillic}r{cyrillic}i{cyrillic}t{cyri llic}m{cyrillic}o{cyrillic}v{cyrillic} i{cyrillic} P{cyrillic}r{cyrillic}o{cyrillic}g{cyrillic}r{cyrillic}a{cyrillic}m{cyri llic}m{cyrillic}, vol. I, Latvian State University, 1974, pp. 82-88
15. Berwick R. The Acquisition of Syntatic Knowledge (1985), MIT Press, Cambridge, MA
16. Bishop C. Pattern Recognition and Machine Learning (2006), Springer
17. Blum L., and Blum M. Toward a mathematical theory of inductive inference. Inform. Control 28 2 (1975) 125-155
18. Blum M. A machine independent theory of the complexity of recursive functions. Journal of the ACM 14 2 (1967) 322-336
19. Case J., and Jain S. Synthesizing learners tolerating computable noisy data. Journal of Computer and System Sciences 62 3 (2001) 413-441
20. Case J., Jain S., Lange S., and Zeugmann T. Incremental concept learning for bounded data mining. Inform. Comput. 152 1 (1999) 74-110
21. Case J., Jain S., and Sharma A. Synthesizing noise tolerant language learners. Theoret. Comput. Sci. 261 1 (2001) 31-56
22. Case J., and Lynes C. Machine inductive inference and language identification. Automata, Languages and Programming, 9th Colloquium, Proceedings vol. 140 (1982), Springer-Verlag 107-115
23. Cicchello O., and Kremer S.C. Inducing grammars from sparse data sets: A survey of algorithms and results. J. Machine Learning Res. 4 (2003) 603-632
24. Clark A., and Eyraud R. Identification in the limit of substitutable context-free languages. Algorithmic Learning Theory, 16th International Conference, ALT 2005, Singapore, October 2005, Proceedings. Lecture Notes in Artificial Intelligence vol. 3734 (2005), Springer 283-296
25. Daley R.P., and Smith C.H. On the complexity of inductive inference. Inform. Control 69 1-3 (1986) 12-40
26. de Jongh D., and Kanazawa M. Angluin's theorem for indexed families of r.e. sets and applications. Proceedings of the Ninth Annual Conference on Computational Learning Theory (1996), ACM Press, New York, NY 193-204
27. de la Higuera C. Characterisitc sets for polynomial grammatical inference. Machine Learning 27 (1997) 125-138
28. de la Higuera C. A bibliographical study of grammatical inference. Pattern Recognition 38 9 (2005) 1332-1348
29. Erlebach T., Rossmanith P., Stadtherr H., Steger A., and Zeugmann T. Learning one-variable pattern languages very efficiently on average, in parallel, and by asking queries. Theoret. Comput. Sci. 261 1 (2001) 119-156
30. Feldman J. Some decidability results on grammatical inference and complexity. Inform. Control 20 3 (1972) 244-262
31. Freivalds R., Kinber E., and Wiehagen R. On the power of inductive inference from good examples. Theoret. Comput. Sci. 110 1 (1993) 131-144
32. Gold E.M. Limiting recursion. Journal of Symbolic Logic 30 (1965) 28-48
33. Gold E.M. Language identification in the limit. Inform. Control 10 5 (1967) 447-474
34. Haussler D., Kearns M., Littlestone N., and Warmuth M.K. Equivalence of models for polynomial learnability. Inform. Comput. 95 2 (1991) 129-161
35. Hopcroft J.E., and Ullman J.D. Formal Languages and their Relation to Automata (1969), Addison-Wesley, Boston, MA, USA
36. Jain S., Kinber E., Lange S., Wiehagen R., and Zeugmann T. Learning languages and functions by erasing. Theoret. Comput. Sci. 241 1-2 (2000) 143-189
37. Jain S., Kinber E., and Wiehagen R. Language learning from texts: Degrees of intrinsic complexity and their characterizations. J. Comput. Syst. Sci. 63 3 (2001) 305-354
38. Jain S., Lange S., and Nessel J. On the learnability of recursively enumerable languages from good examples. Theoret. Comput. Sci. 261 1 (2001) 3-29
39. Jain S., Lange S., and Zilles S. Towards a better understanding of incremental learning. Algorithmic Learning Theory, 17th International Conference, ALT 2006, Barcelona, Spain, October 2006, Proceedings. Lecture Notes in Artificial Intelligence vol. 4264 (2006), Springer 169-183
40. Jain S., Osherson D., Royer J.S., and Sharma A. Systems that Learn: An Introduction to Learning Theory. second edition (1999), MIT Press, Cambridge, MA
41. Jain S., and Sharma A. Generalization and specialization strategies for learning r.e. languages. Ann. Math. Artificial Intelligence 23 1-2 (1998) 1-26
42. Kaufmann S., and Stephan F. Resource bounded next value and explanatory identification: Learning automata, patterns and polynomials on-line. Proceedings of the Tenth Annual Conference on Computational Learning Theory, July 6th-9th, 1997, Nashville, Tennessee (1997), ACM Press, New York, NY 263-274
43. Kearns M., and Pitt L. A polynomial-time algorithm for learning k-variable pattern languages from examples. Proceedings of the Second Annual Workshop on Computational Learning Theory, Santa Cruz, CA (1989), Morgan Kaufmann, San Mateo, CA 57-71
44. Kearns M.J., and Vazirani U.V. An Introduction to Computational Learning Theory (1994), The MIT Press, Cambridge, MA
45. Kinber E., and Stephan F. Language learning from texts: Mindchanges, limited memory, and monotonicity. Inform. Comput. 123 2 (1995) 224-241
46. Ko K.-I., Marron A., and Tzeng W.-G. Learning string patterns and tree patterns from examples. Machine Learning: Proceedings of the Seventh International Conference on Machine Learning (1990), Morgan Kaufmann 384-391
47. S. Kobayashi, Approximate identification, finite elasticity and lattice structure of hypothesis spaces. Technical Report CSIM 96-04, Dept. of Compt. Sci. and Inform. Math., University of Electro-Communications, Tokyo, Japan, 1996
48. Kobayashi S., and Yokomori T. Identifiability of subspaces and homomorphic images of zero-reversible languages. Algorithmic Learning Theory, 8th International Workshop, ALT '97, Sendai, Japan, October 1997, Proceedings. Lecture Notes in Artificial Intelligence vol. 1316 (1997), Springer, Berlin 48-61
49. Koshiba T. Typed pattern languages and their learnability. Computational Learning Theory, Second European Conference, EuroCOLT '95, Barcelona, Spain, March 1995, Proceedings. Lecture Notes in Artificial Intelligence vol. 904 (1995), Springer 367-379
50. Lange S. Algorithmic Learning of Recursive Languages (2000), Mensch & Buch Verlag, Berlin
51. Lange S., and Grieser G. Variants of iterative learning. Theoret. Comput. Sci. 292 2 (2003) 359-376
52. Lange S., Nessel J., and Wiehagen R. Learning recursive languages from good examples. Ann. Math. Artificial Intelligence 23 1/2 (1998) 27-52
53. Lange S., and Wiehagen R. Polynomial-time inference of arbitrary pattern languages. New Generation Comput. 8 4 (1991) 361-370
54. Lange S., and Zeugmann T. Types of monotonic language learning and their characterization. Proceedings of the Fifth Annual ACM Workshop on Computational Learning Theory (1992), ACM Press, New York, NY 377-390
55. Lange S., and Zeugmann T. Language learning in dependence on the space of hypotheses. Proceedings of the Sixth Annual ACM Conference on Computational Learning Theory (1993), ACM Press, New York, NY 127-136
56. Lange S., and Zeugmann T. Incremental learning from positive data. J. Comput. Syst. Sci. 53 1 (1996) 88-103
57. Lange S., and Zeugmann T. Set-driven and rearrangement-independent learning of recursive languages. Math. Syst. Theory 29 6 (1996) 599-634
58. Lange S., Zeugmann T., and Kapur S. Monotonic and dual monotonic language learning. Theoret. Comput. Sci. 155 2 (1996) 365-410
59. Lange S., and Zilles S. On the learnability of erasing pattern languages in the query model. Algorithmic Learning Theory, 14th International Conference, ALT 2003, Sapporo, Japan, October 2003, Proceedings. Lecture Notes in Artificial Intelligence vol. 2842 (2003), Springer 129-143
60. Lange S., and Zilles S. Formal language identification: Query learning vs. Gold-style learning. Inform. Process. Lett. 91 6 (2004) 285-292
61. Lange S., and Zilles S. Replacing limit learners with equally powerful one-shot query learners. Learning Theory: 17th Annual Conference on Learning Theory, COLT 2004, Banff, Canada, July 1-4, 2004. Proceedings. Lecture Notes in Artificial Intelligence vol. 3120 (2004), Springer 155-169
62. Lange S., and Zilles S. Relations between Gold-style learning and query learning. Inform. Comput. 203 2 (2005) 211-237
63. Littlestone N. Learning quickly when irrelevant attributes abound: A new linear-threshold algorithm. Machine Learning 2 4 (1988) 285-318
64. Mitchell A.R. Learnability of a subclass of extended pattern languages. Proceedings of the Eleventh Annual Conference on Computational Learning Theory (1998), ACM Press, New York, NY 64-71
65. Mitchell T. Generalization as search. Art. Int. 18 (1982) 203-226
66. Mitchell T.M. Machine Learning (1997), WCB/McGraw-Hill, Boston, MA
67. Moriyama T., and Sato M. Properties of language classes with finite elasticity. IEICE Trans. Inform. Syst. E78-D 5 (1995) 532-538
68. Motoki T., Shinohara T., and Wright K. The correct definition of finite elasticity: Corrigendum to Identification of unions. Proceedings of the Fourth Annual Workshop on Computational Learning Theory (1991), Morgan Kaufmann, San Mateo, CA 375
69. Natarajan B. Machine Learning: A Theoretical Approach (1991), Morgan Kaufmann, San Mateo, CA
70. Nessel J., and Lange S. Learning erasing pattern languages with queries. Theoret. Comput. Sci. 348 1 (2005) 41-57
71. Ng Y.K., and Shinohara T. Characteristic sets for inferring the unions of the tree pattern languages by the most fitting hypotheses. Grammatical Inference: Algorithms and Applications, 8th International Colloquium, ICGI 2006, Tokyo, Japan, September 20-22, 2006, Proceedings. Lecture Notes in Artificial Intelligence vol. 4201 (2006), Springer, Berlin 307-319
72. Ng Y.K., and Shinohara T. Developments from enquiries into the learnability of the pattern languages from positive data. Theoret. Comput. Sci. 397 1-3 (2008) 150-165
73. Odifreddi P. Classical Recursion Theory (1989), North Holland, Amsterdam
74. Osherson D.N., Stob M., and Weinstein S. Systems that Learn: An Introduction to Learning Theory for Cognitive and Computer Scientists (1986), MIT Press, Cambridge, MA
75. Pitt L. Inductive inference, DFAs, and computational complexity. Analogical and Inductive Inference, International Workshop AII '89, Reinhardsbrunn Castle, GDR, October 1989, Proceedings. Lecture Notes in Artificial Intelligence vol. 397 (1989), Springer-Verlag 18-44
76. Reidenbach D. A negative result on inductive inference of extended pattern languages. Algorithmic Learning Theory, 13th International Conference, ALT 2002, Lübeck, Germany, November 2002, Proceedings. Lecture Notes in Artificial Intelligence vol. 2533 (2002), Springer 308-320
77. Reidenbach D. On the learnability of E-pattern languages over small alphabets. Learning Theory: 17th Annual Conference on Learning Theory, COLT 2004, Banff, Canada, July 1-4, 2004. Proceedings. Lecture Notes in Artificial Intelligence vol. 3120 (2004), Springer 140-154
78. Reidenbach D. A non-learnable class of E-pattern languages. Theoret. Comput. Sci. 350 1 (2006) 91-102
79. Reidenbach D. Discontinuities in pattern inference. Theoret. Comput. Sci. 397 1-3 (2008) 166-193
80. Reischuk R., and Zeugmann T. An average-case optimal one-variable pattern language learner. J. Comput. Syst. Sci. 60 2 (2000) 302-335
81. Rossmanith P., and Zeugmann T. Stochastic finite learning of the pattern languages. Machine Learning 44 1/2 (2001) 67-91
82. Russell S., and Norvig P. Artificial Intelligence: A Modern Approach (2002), Prentice Hall
83. Sakakibara Y. Recent advances of grammatical inference. Theoret. Comput. Sci. 185 1 (1997) 15-45
84. Sato M., Mukouchi Y., and Zheng D. Characteristic sets for unions of regular pattern languages and compactness. Algorithmic Learning Theory, 9th International Conference, ALT '98, Otzenhausen, Germany, October 1998, Proceedings. Lecture Notes in Artificial Intelligence vol. 1501 (1998), Springer, Berlin 220-233
85. Shinohara T. Polynomial time inference of extended regular pattern languages. RIMS Symposium on Software Science and Engineering, Kyoto, 1982, Proceedings. Lecture Notes in Computer Science vol. 147 (1982), Springer-Verlag 115-127
86. Shinohara T. Rich classes inferable from positive data: Length-bounded elementary formal systems. Inform. Comput. 108 2 (1994) 175-186
87. Stephan F. Noisy inference and oracles. Theoret. Comput. Sci. 185 1 (1997) 129-157
88. Trakhtenbrot B.A., and Barzdin Y.M. Finite Automata, Behavior and Synthesis (1973), North Holland, Amsterdam
89. Valiant L. A theory of the learnable. Commun. ACM 27 11 (1984) 1134-1142
90. Wexler K. The subset principle is an intensional principle. In: Reuland E., and Abraham W. (Eds). Knowledge and Language vol. 1 (1993), Kluwer Academic Publishers 217-239
91. Wexler K., and Culicover P. Formal Principles of Language Acquisition (1980), MIT Press, Cambridge, MA
92. Wiehagen R. Limes-Erkennung rekursiver Funktionen durch spezielle Strategien. Elektronische Informationsverarbeitung und Kybernetik 12 1-2 (1976) 93-99
93. Wiehagen R. Identification of formal languages. Mathematical Foundations of Computer Science 1977, Proceedings, 6th Symposium, Tatranská Lomnica, September 5-9, 1977. Lecture Notes in Computer Science vol. 53 (1977), Springer-Verlag 571-579
94. Wiehagen R. Characterization problems in the theory of inductive inference. Automata, Languages and Programming, Fifth Colloquium, July 17-21, 1978, Udine, Italy. Lecture Notes in Compute Science vol. 62 (1978), Springer-Verlag 494-508
95. Wiehagen R. A thesis in inductive inference. Nonmonotonic and Inductive Logic, 1st International Workshop, Karlsruhe, Germany, December 1990, Proceedings. Lecture Notes in Artificial Intelligence vol. 543 (1990), Springer-Verlag 184-207
96. Wiehagen R., and Zeugmann T. Ignoring data may be the only way to learn efficiently. J. Experimental Theoret. Artif. Intell. 6 1 (1994) 131-144
97. Wright K. Identification of unions of languages drawn from an identifiable class. Proceedings of the Second Annual Workshop on Computational Learning Theory, Santa Cruz, CA (1989), Morgan Kaufmann 328-333
98. Yokomori T. Polynomial-time identification of very simple grammars from positive data. Theoret. Comput. Sci. 298 1 (2003) 179-206
99. Yoshinaka R. Learning efficiency of very simple grammars from positive data. Algorithmic Learning Theory, 18th International Conference, ALT 2007, Sendai, Japan, October 2007, Proceedings. Lecture Notes in Artificial Intelligence vol. 4754 (2007), Springer, Berlin 227-241
100. Zeugmann T. Lange and Wiehagen's pattern language learning algorithm: An average-case analysis with respect to its total learning time. Ann. Math. Artificial Intelligence 23 (1998) 117-145
101. Zeugmann T. From learning in the limit to stochastic finite learning. Theoret. Comput. Sci. 364 1 (2006) 77-97
102. Zeugmann T., and Lange S. A guided tour across the boundaries of learning recursive languages. Algorithmic Learning for Knowledge-Based Systems. Lecture Notes in Artificial Intelligence vol. 961 (1995), Springer 190-258
103. Zeugmann T., and Zilles S. Learning recursive functions - a survey. Theoret. Comput. Sci. 397 1-3 (2008) 4-56
104. Zilles S. Uniform Learning of Recursive Functions. Dissertationen zur Künstlichen Intelligenz vol. 278 (2003), Akademische Verlagsgesellschaft, Aka GmbH, Berlin