Context tree estimation for not necessarily finite memory processes, Via BIC and MDL

IEEE Transactions on Information Theory

Volumn 52, Issue 3, 2006, Pages 1007-1016

  Csiszár, Imre ^a,b   Talata, Zsolt ^a,b,c  

^a IEEE   (Hungary)

^b INSTITUTE OF EXPERIMENTAL MEDICINE   (Hungary)

^c GEORGIA INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

Bayesian information criterion (BIG); Consistent estimation; Context tree; Context tree maximization (CTM); Infinite memory; Minimum description length (MDL); Model selection

Indexed keywords

BAYESIAN INFORMATION CRITERION (BIG); CONSISTENT ESTIMATION; CONTEXT TREE ESTIMATION; CONTEXT TREE MAXIMIZATION (CTM); INFINITE MEMORY; MINIMUM DESCRIPTION LENGTH (MDL); MODEL SELECTION;

COMPUTATIONAL COMPLEXITY; INFORMATION THEORY; MATHEMATICAL MODELS; OPTIMIZATION; PROBABILISTIC LOGICS;

TREES (MATHEMATICS);

EID: 33744799923     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2005.864431     Document Type: Article

References (18)

1. D. Baron and Y. Bresler, "An O(N) semipredictive universal encoder via the BWT," IEEE Trans. Inf. Theory, vol. 50, no. 5, pp. 928-937, May 2004.
2. A. Barron, J. Rissanen, and B. Yu, "The minimum description length principle in coding and modeling," IEEE Trans. Inf. Theory, vol. 44, no. 6. pp. 2743-2760, Oct. 1998.
3. P. Bühlmann and A. J. Wyner, "Variable length Markov chains," Ann. Statist., vol. 27, pp. 480-513, 1999.
4. I. Csiszár and P. C. Shields, "The consistency of the BIC Markov order estimator," Ann. Statist., vol. 28, pp. 1601-1619, 2000.
5. I. Csiszár, "Large-scale typicality of Markov sample paths and consistency of MDL order estimators," IEEE Trans. Inf. Theory, vol. 48, no. 6, pp. 1616-1628, Jun. 2002.
6. L. Finesse, "Estimation of the order of a finite Markov chain," in Recent Advances in Mathematical Theory of Systems, Control, Nebvorks and Signal Processing, I, H. Kimura and S. Kodama, Eds. Tokyo, Japan: Mita, 1992, pp. 643-645.
7. R. E. Krichevsky and V. K. Trofimov, "The performance of universal encoding," IEEE Trans. Inf. Theory, vol. IT-27, no. 2, pp. 199-207, Mar. 1981.
8. A. Martín, G. Seroussi, and M. J. Weinberger, "Linear time universal coding and time reversal of tree sources via FSM closure," IEEE Trans. Inf. Theory, vol. 50, no. 7, pp. 1442-1468, Jul. 2004.
9. R. Nohre, "Some Topics in Descriptive Complexity," Ph.D. dissertation, Elec. Eng. Dept., Linköping Univ., Linköping, Sweden, 1994.
10. J. Rissanen, "A universal data compression system," IEEE Trans. Inf. Theory, vol. IT-29, no. 5, pp. 656-664, Sep. 1983.
11. _, Stochastic Complexity in Statistical Inquiry, Singapore: World Scientific, 1989.
12. G. Schwarz, "Estimating the dimension of a model," Ann. Statist., vol. 6, pp. 461-464, 1978.
13. M. J. Weinberger, A. Lempel, and J. Ziv, "A sequentional algorithm for the universal coding of finite memory sources," IEEE Trans. Inf. Theory, vol. 38, no. 3, pp. 1002-1014, May 1992.
14. M. J. Weinberger, J. Rissanen, and M. Feder, "A universal finite memory source," IEEE Trans. Inf. Theory, vol. 41, no. 3, pp. 643-652, May 1995.
15. F. M. J. Willems, Y. M. Shtarkov, and T. J. Tjalkens, "The Context-Tree Weighting Method: Basic Properties," Elec. Eng. Dept., Eindhoven Univ., Eindhoven, The Netherlands, Tech. Rep., 1993. An earlier unabridged version of [16].
16. _, "The context-tree weighting method: Basic properties," IEEE Trans. Inf. Theory, vol. 41, no. 3, pp. 653-664, May 1995.
17. _, "Context-tree maximizing," in Proc. 2000 Conf. Information Sciences and Systems, Princeton, NJ, Mar. 2000, pp. TP6-7-TP6-12.
18. F. M. J. Willems, "The context-tree weighting method: Extensions," IEEE Trans. Inf. Theory, vol. 44, no. 2, pp. 792-798, Mar. 1998.