On strong consistency of model selection in classification

IEEE Transactions on Information Theory

Volumn 52, Issue 11, 2006, Pages 4767-4774

  Suzuki, Joe ^a  

^a OSAKA UNIVERSITY   (Japan)

Author keywords

Error probability; Hannan and Quinn's procedure; Kullback Leibler divergence; Law of the iterated logarithm; Model selection; Strong consistency

Indexed keywords

ERRORS; MARKOV PROCESSES; MATHEMATICAL MODELS; PROBABILITY; STATISTICAL METHODS;

AUTOREGRESSIVE PROCESSES; BAYESIAN NETWORK; EMPIRICAL ENTROPY; ITERATED LOGARITHM; KULLBACK-LEIBLER DIVERGENCE;

CLASSIFICATION (OF INFORMATION);

EID: 33750580550     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2006.883611     Document Type: Article

References (25)

1. H. Akaike, "A new look at the statistical model identification," IEEE Trans. Automat. Contr., vol. AC-19, pp. 716-723, 1974.
2. A. C. Atkinson, "A method for discriminating between models," J. Roy. Statist., Soc. Ser., vol. B32, pp. 323-353, 1970.
3. P. Billingsley, Statistical Inference for Markov Processes. Chicago, IL: The University of Chicago Press, 1961.
4. H. Cramer, Mathematical Methods of Statistics. Princeton, NJ: Princeton Univ. Press, 1946.
5. T. Cover and J. Thomas, Elements of Information Theory. New York: Wiley, 1991.
6. G. F. Cooper and E. Herskovits, "A Bayesian method for the induction of probabilistic networks from data," Machine Learn., vol. 9, pp. 309-347, 1992.
7. I. Csiszar, "Information theoretical methods in statistics," in Class Notes. College Park, MD: Univ. Maryland, 1990.
8. I. Csiszar and P. C. Shields, "The consistency of the BIC Markov order estimation," Ann. Statist., vol. 28, no. 6, Dec. 2000.
9. L. Finesso, "Consistent Estimation of the Order for Markov and Hidden Markov Chains," Ph.D. dissertation, University of Maryland, College Park, MD, 1990.
10. E. J. Hannan and B. G. Quinn, "The determination of the order of an autoregression," J. Roy. Statist. Soc., ser. B, vol. 41, pp. 190-195, 1979.
11. E. J. Hannan, "The estimation of the order of an ARMA process," Ann. Statist., vol. 8, no. 5, pp. 1071-1081, 1980.
12. P. Hall and E. Hannan, "On stochastic complexity and nonparametric density estimation," Biometrika, vol. 75, pp. 705-714, 1988.
13. J. C. Kieffer, "Strongly consistent code-based identification and order estimation for constrained finite-state model classes," IEEE Trans. Inf. Theory, vol. 39, pp. 893-902, 1993.
14. C. Liu and P. Narayan, "Order estimation and sequential universal data compression of a hidden Markov source by the method of mixtures," IEEE Trans. Inf. Theory, vol. IT-40, pp. 1167-1180, 1994.
15. N. Merhav, Gutman, and J. Ziv, "On the estimation of the order of a Markov chain and universal data compression," IEEE Trans. Inf. Theory, vol. IT-35, pp. 1014-1019, 1989.
16. N. Merhav, "The estimation of model order in exponential families," IEEE Trans. Inf. Theory, vol. IT-35, pp. 1109-1113, 1989.
17. J. Rissanen, "Modeling by shortest data description," Automatica, vol. 14, pp. 465-471, 1978.
18. J. Rissanen, "Stochastic complexity and modeling," Ann. Statist., vol. 14, pp. 1080-1100, 1986.
19. G. Schwarz, "Estimating the dimension of a model," Ann. Statist., vol. 6, pp. 461-464, 1978.
20. R. Shibata, "Selection of the order of autoregressive model by Akaike's information criterion," Biometrika, vol. 63, pp. 117-126, 1976.
21. Y. M. Shtarkov, "Universal sequential coding of single messages," Probl. Inf. Trans., vol. 16, pp. 175-186, 1987.
22. W. F. Stout, Almost Sure Convergence. New York: Academic, 1974.
23. J. Suzuki, "A construction of Bayesian networks from databases based on the MDL principle," in Proc. 1993 Uncertainty in Artificial Intelligence Conf., 1993, pp. 266-273.
24. B. Yu and T. Speed, "Data compression and histograms," Prob. Theory Rel. Fields, vol. 92, pp. 195-229, 1992.
25. J. Ziv and N. Merhav, "Estimating the number of states of a finite state source," IEEE Trans. Inf. Theory, vol. IT-38, pp. 61-65, 1992.