Model selection and testing by the MDL principle

Information Theory and Statistical Learning

Volumn , Issue , 2009, Pages 25-43

  Rissanen, Jorma ^a  

^a HELSINKI INSTITUTE FOR INFORMATION TECHNOLOGY   (Finland)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 84872823612     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1007/978-0-387-84816-7_2     Document Type: Chapter

References (29)

1. Balasubramanian, V. (1996), Statistical inference, Occam's razor and statistical mechanics on the space of probability distributions, Neural Comput., 9(2), 349-268
2. Barron, A.R., Rissanen, J., and Yu, B. (1998), The MDL principle in modeling and coding, IEEE Trans. Inform. Theory IT-44(6), 2743-2760 (special issue to commemorate 50 years of information theory)
3. Chaitin, G.J. (1969), On the length of programs for computing finite binary sequences: statistical considerations JACM, 16, 145-159
4. Clarke, B.C. and Barron, A.R. (1990), Information-theoretic asymptotics of Bayes methods, IEEE Trans. Inform. Theory, IT-36, 453-471
5. Dawid, A.P. (1984), Present position and potential developments: some personal views, statistical theory, the prequential approach, J. R. Stat. Soc. A, 147(Part 2), 278-292
6. Elias, P. (1975), Universal codeword sets and representations of the integers, IEEE Trans. Inform. Theory, IT-21(2), 194-203
7. Grünwald, Peter, D. (2007), The Minimum Description Length Principle, MIT, Cambridge, 701 pages
8. Hannan, E.J. and Quinn, B.G. (1979), The determination of the order of an autoregression, J. R. Stat. Soc. B, 41, 190-195
9. Korodi, G. and Tabus, I. (2005), An efficient normalized maximum likelihood algorithm for DNA sequence compression, ACM Trans. Inform. Syst., 23(1), 3-34
10. Korodi, G. and Tabus, I. (2007), Normalized maximum likelihood model of order-1 for the compression of DNA sequences, in Proc. IEEE Data Compression Conference, DCC'07, pp. 33-42, Snowbird, 27-29 March 2007.
11. Li, M. and Vitanyi, P.M.B. (1997), An Introduction to Kolmogorov Complexity and its Applications, Second Edition, Springer, New York, 637 pages
12. Razavi, S.A. and Giurcaneanu, C.D. (2008), Composite hypothesis testing by optimally distinguishable distributions, in Proceedings of 2008 IEEE International Conference on Acoustics, Speech, and Signal Processing, ICASSP, Las Vegas, Nevada, USA, 30 Mar-4Apr (accepted for publication)
13. Rissanen, J. (1978), Modeling by shortest data description, Automatica, 14, 465-471
14. Rissanen, J. (1983), A universal prior for integers and estimation by minimum description length, Ann. Stat., 11(2), 416-431
15. Rissanen, J. (1984), Universal coding, information, prediction, and estimation, IEEE Trans. Inform. Theory, IT-30(4), 629-636
16. Rissanen, J. (1986), Stochastic complexity and modeling, Ann. Stat., 14, 1080-1100
17. Rissanen, J. (1996), Fisher information and stochastic complexity, IEEE Trans. Inform. Theory, IT-42(1), 40-47
18. Rissanen, J. (2000), MDL Denoising, IEEE Trans. Inform. Theory, IT-46(7), 2537-2543
19. Rissanen, J. (2007), Information and Complexity in Statistical Modeling, Springer, Berlin, 142 pages
20. Rissanen, J., Roos, T., and Myllymaki, P. (2008), Sequentially Normalized Least Squares Models, (to appear), http://www.mdl-research.org/jorma.rissanen
21. Schwarz, G. (1978), Estimating the dimension of the model. Ann. Stat. 6, 461-464
22. Shtarkov, Y. (1987), Universal sequential coding of single messages, Prob. Inform. Transmission, 23(3), 175-186
23. Solomonoff, R.J. (1960), A preliminary report on a general theory of inductive inference, Report ZTB-135, Zator, Cambridge, November 1960
24. Tabus, I. and Korodi, G. (2008), Genome compression using normalized maximum likelihood models for constrained Markov sources, in IEEE Information Theory Workshop, Porto, Portugal, May 5-9, 2008
25. Tabus, I., Korodi, G., and Rissanen J. (2003), DNA sequence compression using the normalized maximum likelihood model for discrete regression, in Data Compression Conference 2003, DCC '03, pp. 253-262, 2003
26. Tabus, I. and Rissanen, J., (2006), Maximum likelihood model for logit regression, Festschrift for T. Pukkila, University of Tampere, June 2006
27. Tabus, I., Rissanen, J., and Astola, J. (2006), Classification and feature gene selection using the normalized maximum likelihood model for discrete regressions, Signal Process., Special issue on genomic signal processing; also available at http://sigwww.cs.tut.fi/tabus/GSP.pdf
28. Vereshchagin, N.K. and Vitanyi P.M.B. (2004), Kolmogorov's Structure functions and model selection, IEEE Trans. Inform. Theory, IT-50(12), 3265-3290
29. Wallace, C.S. and Boulton, D.M. (1968), An information measure for classification, Comput. J., 11(2), 185-195