Computation of distances for regular and context-free probabilistic languages

Theoretical Computer Science

Volumn 395, Issue 2-3, 2008, Pages 235-254

  Nederhof, Mark Jan ^a   Satta, Giorgio ^b  

^a UNIVERSITY OF ST ANDREWS   (United Kingdom)

^b UNIVERSITY OF PADOVA   (Italy)

Author keywords

Kullback Leibler divergence; Language entropy; Probabilistic context free languages; Probabilistic finite automata; Probabilistic language distances

Indexed keywords

ARTIFICIAL INTELLIGENCE; BIOINFORMATICS; COMPUTATIONAL LINGUISTICS; LEARNING SYSTEMS; LINGUISTICS; PATTERN MATCHING; PROBABILITY; QUERY LANGUAGES;

COMPUTATIONAL BIOLOGY; CONTEXT FREE; CONTEXT-FREE GRAMMAR (CFG); ELSEVIER (CO); LANGUAGE MODELING (LM); LANGUAGES (TRADITIONAL); MACHINE-LEARNING;

CONTEXT FREE LANGUAGES;

EID: 45849119469     PISSN: 03043975     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.tcs.2008.01.010     Document Type: Article

References (41)

1. S. Abney, D. McAllester, F. Pereira, Relating probabilistic grammars and automata, in: 37th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference, Maryland, USA, 1999
2. Aho A., Hopcroft J., and Ullman J. The Design and Analysis of Computer Algorithms (1974), Addison-Wesley
3. J. Baker, Trainable grammars for speech recognition, in: J. Wolf, D. Klatt (Eds.), Speech Communication Papers Presented at the 97th Meeting of the Acoustical Society of America, 1979
4. Bar-Hillel Y., Perles M., and Shamir E. On formal properties of simple phrase structure grammars. In: Bar-Hillel Y. (Ed). Language and Information: Selected Essays on their Theory and Application (1964), Addison-Wesley, Reading, MA 116-150 (Chapter 9)
5. Booth T., and Thompson R. Applying probabilistic measures to abstract languages. IEEE Transactions on Computers C-22 (1973) 442-450
6. Calera-Rubio J., and Carrasco R. Computing the relative entropy between regular tree languages. Information Processing Letters 68 6 (1998) 283-289
7. Carrasco R. Accurate computation of the relative entropy between stochastic regular grammars. RAIRO (Theoretical Informatics and Applications) 31 5 (1997) 437-444
8. Charniak E. Statistical Language Learning (1993), MIT Press
9. Chi Z. Statistical properties of probabilistic context-free grammars. Computational Linguistics 25 1 (1999) 131-160
10. Corazza A., De Mori R., Gretter R., and Satta G. Computation of probabilities for an island-driven parser. IEEE Transactions on Pattern Analysis and Machine Intelligence 13 9 (1991) 936-950
11. Corazza A., De Mori R., Gretter R., and Satta G. Optimal probabilistic evaluation functions for search controlled by stochastic context-free grammars. IEEE Transactions on Pattern Analysis and Machine Intelligence 16 10 (1994) 1018-1027
12. Cormen T., Leiserson C., and Rivest R. Introduction to Algorithms (1990), The MIT Press
13. Cortes C., Mohri M., and Rastogi A. On the computation of some standard distances between probabilistic automata. Proceedings of the 11th International Conference on Implementation and Application of Automata. CIAA 2006. Lecture Notes in Computer Science vol. 4094 (2006), Springer-Verlag, Berlin, Germany
14. C. Cortes, M. Mohri, A. Rastogi, M. Riley, Efficient computation of the relative entropy of probabilistic automata, in: LATIN 2006, 2006
15. Duda R., Hart P., and Stork D. Pattern Classification (2001), John Wiley and Sons, New York, NY
16. Durbin R., Eddy S., Krogh A., and Mitchison G. Biological Sequence Analysis: Probabilistic Models of Proteins and Nucleic Acids (1999), Cambridge University Press, Cambridge, UK
17. Etessami K., and Yannakakis M. Recursive Markov chains, stochastic grammars, and monotone systems of nonlinear equations. 22nd International Symposium on Theoretical Aspects of Computer Science. Lecture Notes in Computer Science vol. 3404 (2005), Springer-Verlag, Stuttgart, Germany
18. Gregory B., and Kaltofen E. Analysis of the binary complexity of asymptotically fast algorithms for linear system solving. SIGSAM Bullettin 22 2 (1988) 41-49
19. Grenander U. Lectures in Pattern Theory, Vol. I: Pattern Synthesis (1976), Springer-Verlag
20. Harris T.E. The Theory of Branching Processes (1963), Springer-Verlag, Berlin, Germany
21. Hopcroft J., and Ullman J. Introduction to Automata Theory, Languages, and Computation (1979), Addison-Wesley
22. Hutchins S. Moments of strings and derivation lengths of stochastic context-free grammars. Information Sciences 4 (1972) 179-191
23. Jelinek F. Statistical Methods for Speech Recognition (1997), MIT Press
24. Jelinek F., and Lafferty J. Computation of the probability of initial substring generation by stochastic context-free grammars. Computational Linguistics 17 3 (1991) 315-323
25. Jurafsky D., and Martin J. Speech and Language Processing (2000), Prentice-Hall
26. Kelley C. Iterative Methods for Linear and Nonlinear Equations (1995), SIAM, Philadelphia, PA
27. Lewis H., and Papadimitriou C. Elements of the Theory of Computation (1981), Prentice-Hall
28. Lyngso R., and Pedersen C.S. The consensus string problem and the complexity of comparing hidden Markov models. Journal of Computing and System Science 65 (2002) 545-569
29. Manning C., and Schütze H. Foundations of Statistical Natural Language Processing (1999), Massachusetts Institute of Technology
30. Nederhof M.-J. Weighted deductive parsing and Knuth's algorithm. Computational Linguistics 29 1 (2003) 135-143
31. Nederhof M.-J. A general technique to train language models on language models. Computational Linguistics 31 2 (2005) 173-185
32. M.-J. Nederhof, G. Satta, Probabilistic parsing as intersection, in: 8th International Workshop on Parsing Technologies, LORIA, Nancy, France, 2003
33. M.-J. Nederhof, G. Satta, Computing partition functions of PCFGs, 2008 (submitted for publication)
34. Paz A. Introduction to Probabilistic Automata (1971), Academic Press, New York
35. Santos E. Probabilistic grammars and automata. Information and Control 21 (1972) 27-47
36. Sippu S., and Soisalon-Soininen E. Parsing Theory, Vol. I: Languages and Parsing. EATCS Monographs on Theoretical Computer Science vol. 15 (1988), Springer-Verlag
37. Soule S. Entropies of probabilistic grammars. Information and Control 25 (1974) 57-74
38. Starke P. Abstract Automata (1972), North-Holland Publishing Company, Amsterdam
39. Stolcke A. An efficient probabilistic context-free parsing algorithm that computes prefix probabilities. Computational Linguistics 21 2 (1995) 167-201
40. Vidal E., Thollard F., de la Higuera C., Casacuberta F., and Carrasco R.C. Probabilistic finite-state machines - Part I. IEEE Transactions on Pattern Analysis and Machine Intelligence 27 7 (2005) 1013-1025
41. D. Wojtczak, K. Etessami, PReMO: An analyzer for Probabilistic Recursive Models, TACAS'07, Braga, Portugal, in: Lecture Notes in Computer Science, vol. 4424, Springer, 2007, pp. 66-71