A linear-time algorithm for computing the multinomial stochastic complexity

Information Processing Letters

Volumn 103, Issue 6, 2007, Pages 227-233

  Kontkanen, Petri ^a   Myllymäki, Petri ^a  

^a UNIVERSITY OF HELSINKI   (Finland)

Author keywords

Algorithms; Combinatorial problems; Computational complexity

Indexed keywords

COMBINATORIAL MATHEMATICS; COMPUTATIONAL COMPLEXITY; DATA STRUCTURES; MAXIMUM LIKELIHOOD; RANDOM PROCESSES;

DENSITY ESTIMATION; IMAGE DENOISING; STATISTICAL INFERENCE; STOCHASTIC COMPLEXITY;

CLUSTERING ALGORITHMS;

EID: 34347261665     PISSN: 00200190     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ipl.2007.04.003     Document Type: Article

References (26)

1. Balasubramanian V. MDL, Bayesian inference, and the geometry of the space of probability distributions. In: Grünwald P., Myung I.J., and Pitt M. (Eds). Advances in Minimum Description Length: Theory and Applications (2006), MIT Press 81-98
2. Barron A., Rissanen J., and Yu B. The minimum description principle in coding and modeling. IEEE Transactions on Information Theory 44 6 (1998) 2743-2760
3. Chauve C., Dulucq S., and Guibert O. Enumeration of some labelled trees. In: Krob D., Mikhalev A.A., and Mikhalev A.V. (Eds). Formal Power Series and Algebraic Combinatorics, FPSAC'00 (2000), Springer-Verlag 146-157
4. Corless R.M., Gonnet G.H., Hare D.E.G., Jeffrey D.J., and Knuth D.E. On the Lambert W function. Advances in Computational Mathematics 5 (1996) 329-359
5. Cover T., and Thomas J. Elements of Information Theory (1991), John Wiley & Sons, New York, NY
6. Graham R.L., Knuth D.E., and Patashnik O. Concrete Mathematics. second ed. (1994), Addison-Wesley
7. Grünwald P. Minimum description length tutorial. In: Grünwald P., Myung I.J., and Pitt M. (Eds). Advances in Minimum Description Length: Theory and Applications (2006), MIT Press 23-79
8. Knuth D.E., and Pittel B. A recurrence related to trees. Proceedings of the American Mathematical Society 105 2 (1989) 335-349
9. M. Koivisto, Sum-product algorithms for the analysis of genetic risks, PhD thesis, Report A-2004-1, Department of Computer Science, University of Helsinki, 2004
10. Kontkanen P., Buntine W., Myllymäki P., Rissanen J., and Tirri H. Efficient computation of stochastic complexity. In: Bishop C., and Frey B. (Eds). Proceedings of the Ninth International Conference on Artificial Intelligence and Statistics (2003), Society for Artificial Intelligence and Statistics 233-238
11. P. Kontkanen, P. Myllymäki, Analyzing the stochastic complexity via tree polynomials, Technical Report 2005-4, Helsinki Institute for Information Technology (HIIT), 2005
12. P. Kontkanen, P. Myllymäki, A fast normalized maximum likelihood algorithm for multinomial data, in: Proceedings of the Nineteenth International Joint Conference on Artificial Intelligence (IJCAI-05), 2005
13. P. Kontkanen, P. Myllymäki, MDL histogram density estimation, in: Proceedings of the Eleventh International Conference on Artificial Intelligence and Statistics, San Juan, Puerto Rico, March 2007
14. Kontkanen P., Myllymäki P., Buntine W., Rissanen J., and Tirri H. An MDL framework for data clustering. In: Grünwald P., Myung I.J., and Pitt M. (Eds). Advances in Minimum Description Length: Theory and Applications (2006), MIT Press
15. Pearl J. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference (1988), Morgan Kaufmann Publishers, San Mateo, CA
16. Rissanen J. Modeling by shortest data description. Automatica 14 (1978) 445-471
17. Rissanen J. Stochastic complexity. Journal of the Royal Statistical Society 49 3 (1987) 223-239 252-265
18. Rissanen J. Fisher information and stochastic complexity. IEEE Transactions on Information Theory 42 1 (1996) 40-47
19. Rissanen J. Strong optimality of the normalized ML models as universal codes and information in data. IEEE Transactions on Information Theory 47 5 (2001) 1712-1717
20. J. Rissanen, Lectures on statistical modeling theory, August 2005. Available online at www.mdl-research.org
21. T. Roos, P. Myllymäki, H. Tirri, On the behavior of MDL denoising, in: Proceedings of the 10th International Workshop on Artificial Intelligence and Statistics (AISTATS), 2005, pp. 309-316
22. Schwarz G. Estimating the dimension of a model. Annals of Statistics 6 (1978) 461-464
23. Shtarkov Yu.M. Universal sequential coding of single messages. Problems of Information Transmission 23 (1987) 3-17
24. Szpankowski W. Average Case Analysis of Algorithms on Sequences (2001), John Wiley & Sons
25. Wilf H.S. generatingfunctionology. second ed. (1994), Academic Press
26. Xie Q., and Barron A.R. Asymptotic minimax regret for data compression, gambling, and prediction. IEEE Transactions on Information Theory 46 2 (2000) 431-445