Applying MDL to learn best model granularity

Artificial Intelligence

Volumn 121, Issue 1, 2000, Pages 1-29

  Gao, Qiong ^a   Li, Ming ^b   Vitányi, Paul ^c  

^a INSTITUTE OF BOTANY   (China)

^b Santa Barbara   (United States)

^c CWI   (Netherlands)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTATIONAL COMPLEXITY; DATA STRUCTURES; DEGREES OF FREEDOM (MECHANICS); FEATURE EXTRACTION; FEEDFORWARD NEURAL NETWORKS; INFERENCE ENGINES; LEARNING SYSTEMS; MATHEMATICAL MODELS; OPTICAL CHARACTER RECOGNITION; PATTERN MATCHING; ROBOTIC ARMS; THEOREM PROVING;

BAYES THEOREM; KOLMOGOROV COMPLEXITY; MINIMUM DESCRIPTION LENGTH (MDL) PRINCIPLE;

ARTIFICIAL INTELLIGENCE;

EID: 0034250304     PISSN: 00043702     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0004-3702(00)00034-5     Document Type: Article

References (29)

1. Barron A.R., Rissanen J., Yu B. The minimum description length principle in coding and modelling. IEEE Trans. Inform. Theory. Vol. IT-44:(6):1998;2743-2760.
2. Bishop C. Neural Networks for Pattern Recognition. 1995;Oxford University Press, Oxford.
3. Cover T.M., Thomas J.A. Elements of Information Theory. 1991;Wiley, New York.
4. Gao Q., Li M. An application of minimum description length principle to online recognition of handprinted alphanumerals. Proc. IJCAI-89, Detroit, MI. 1989;843-848.
5. Jeffreys H. Theory of Probability. 3rd edn. 1961;Clarendon Press, Oxford.
6. Kearns M.J., Vazirani U. An Introduction to Computational Learning Theory. 1994;MIT Press, Cambridge, MA.
7. Kirchherr W.W., Li M., Vitányi P.M.B. The miraculous universal distribution. Math. Intelligence. Vol. 19:(4):1997;7-15.
8. Kolmogorov A.N. Three approaches to the quantitative definition of information. Problems Inform. Transmission. Vol. 1:(1):1965;1-7.
9. Kurtzberg J. Feature analysis for symbol recognition by elastic matching. IBM J. Res. Develop. Vol. 31:(1):1987;91-95.
10. Li M., Vitányi P.M.B. Inductive reasoning and Kolmogorov complexity. J. Comput. Syst. Sci. Vol. 44:1992;343-384.
11. Li M., Vitányi P.M.B. An Introduction to Kolmogorov Complexity and Its Applications. 2nd edn. 1997;Springer, New York.
12. MacKay D.J.C. A practical Bayesian framework for backpropagation networks. Neural Computation. Vol. 4:(3):1992;448-472.
13. Martin-Löf P. The definition of random sequences. Inform. Contr. Vol. 9:1966;602-619.
14. Quinlan J., Rivest R. Inferring decision trees using the minimum description length principle. Inform. Computation. Vol. 80:1989;227-248.
15. Rissanen J. Modeling by the shortest data description. Automatica-J.IFAC. Vol. 14:1978;465-471.
16. Rissanen J. Universal coding, information, prediction and estimation. IEEE Trans. Inform. Theory. Vol. IT-30:1984;629-636.
17. Rissanen J. Stochastical Complexity and Statistical Inquiry. 1989;World Scientific, Singapore.
18. Rissanen J. Stochastic complexity. J. Royal Stat. Soc., Series B. Vol. 49:1987;223-239. Discussion: pp. 252-265.
19. Solomonoff R.J. A formal theory of inductive inference, Part 1 and Part 2. Inform. Contr. Vol. 7:1964;1-22. 224-254.
20. Solomonoff R.J. Complexity-based induction systems: Comparisons and convergence theorems. IEEE Trans. Inform. Theory. Vol. IT-24:1978;422-432.
21. Suen C.Y. Automatic recognition of handprinted characters - The state of art. Proc. IEEE. Vol. 68:(4):1980;469-487.
22. Tapper C.C. Cursive script recognition by elastic matching. IBM J. Res. Develop. Vol. 26:(6):1982;765-771.
23. te Brake G., Kok J.N., Vitányi P.M.B. Model selection for neural networks: Comparing MDL and NIC. Proc. European Symposium on Artificial Neural Networks, Brussels. 1994.
24. Valiant L.G. Deductive learning. Phil. Trans. Royal Soc. London A. Vol. 312:1984;441-446.
25. Valiant L.G. A theory of the learnable. Comm. ACM. Vol. 27:1984;1134-1142.
26. Vitányi P.M.B., Li M. Minimum description length induction, Bayesianism, and Kolmogorov complexity. IEEE Trans. Inform. Theory. Vol. IT-46:(2):2000;446-464.
27. von Mises R. Grundlagen der Wahrscheinlichkeitsrechnung. Mathemat. Zeitsch. Vol. 5:1919;52-99.
28. Wallace C.S., Boulton D.M. An information measure for classification. Computer J. Vol. 11:1968;185-195.
29. Wallace C.S., Freeman P.R. Estimation and inference by compact coding. J. Roy. Statist. Soc. Vol. 49:1987;240-251. discussion: 252-265.