Context-tree modeling of observed symbolic dynamics

Physical Review E - Statistical Physics, Plasmas, Fluids, and Related Interdisciplinary Topics

Volumn 66, Issue 5, 2002, Pages 11-

  Kennel, Matthew B ^a   Mees, Alistair I ^b  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b UNIVERSITY OF WESTERN AUSTRALIA   (Australia)

Author keywords

[No Author keywords available]

Indexed keywords

CODING TECHNIQUES; CONTEXT-TREE MODELING; MINIMUM DESCRIPTION LENGTH (MDL); SYMBOLIC DYNAMICS;

ALGORITHMS; COMPUTER SIMULATION; DATA COMPRESSION; DEGREES OF FREEDOM (MECHANICS); MARKOV PROCESSES; MATHEMATICAL MODELS; MONTE CARLO METHODS; PROBABILITY DISTRIBUTIONS; TREES (MATHEMATICS);

COMPUTATIONAL COMPLEXITY;

ARTICLE;

EID: 41349103845     PISSN: 1063651X     EISSN: None     Source Type: Journal    
DOI: 10.1103/PhysRevE.66.056209     Document Type: Article

References (38)

1. J. Rissanen, Stochastic Complexity in Statistical Inquiry (World Scientific, Singapore, 1989);
2. A. Barron, J. Rissanen, and B. Yu, IEEE Trans. Inf. Theory 44, 2743 (1998).
3. K. Judd and A. I. Mees, Physica D 82, 426 (1995).
4. J. Rissanen and G. G. Langdon, IEEE Trans. Inf. Theory 27, 12 (1981);
5. I. H. Witten, R. Neal, and J. G. Cleary, Commun. ACM 30, 520 (1987).
6. R. E. Krichevsky and V. K. Trofimov, IEEE Trans. Inf. Theory 28, 199 (1981).
7. Y. M. Shtarkov, T. J. Tjalkens, and F. M. J. Willems, Probl. Inf. Transm. 33, 17 (1997). Although the Krischevsky-Trofimov (KT) estimator has nice and analytically computable asymptotic properties, other estimators may be superior for finite length observed data. We explored a number of ansatzes used by the text compression community but none were consistently superior the KT estimator (4) allowing (Formula presented) to vary as an optimization parameter. In empirical experimentation (Formula presented) gives results reasonably close to the minimum codelength.
8. J. Rissanen, IEEE Trans. Inf. Theory 30, 629 (1984).
9. F. M. Willems, Y. M. Shtarkov, and T. J. Tjalkens, IEEE Trans. Inf. Theory 41, 653 (1995).
10. F. M. Willems, IEEE Trans. Inf. Theory 44, 792 (1998).
11. T. J. Tjalkens (private communication).
12. D. P. Helmbold and R. E. Schapire, Mac. Learn. 27, 51(1997).
13. P. A. J. Volf and F. M. J. Willems, in Proceedings of the 1995 IEEE International Symposium on Information Theory (IEEE, New York, 1995).
14. J. Rissanen, IEEE Trans. Inf. Theory 29, 656 (1983);S. Bunton, in Proceedings of the Data Compression Conference 1997, edited by J. A. Storer and M. Cohn (IEEE Computer Society Press, Los Alamitos, 1997), pp. 32–41.
15. M. J. Weinberger, J. J. Rissanen, and M. Feder, IEEE Trans. Inf. Theory 41, 643 (1995).
16. T. Schurmann and P. Grassberger, Chaos 6, 414 (1996).
17. M. B. Kennel and A. I. Mees, Phys. Rev. E 61, 2563 (2000).
18. D. Ron, Y. Singer, and N. Tishby, Mach. Learn. 25 117(1996).
19. The computer science literature sometimes deems this a “deterministic” finite-state automaton, meaning that the identity of the subsequent state is determined completely given the previous state and emitted symbol.
20. C. E. Shannon, Bell Syst. Tech. J. 27, 379 (1948).
21. T. Cover and J. Thomas, Elements of Information Theory (Wiley Interscience, New York, 1991).
22. A. D. Wyner, J. Ziv, and A. J. Wyner, IEEE Trans. Inf. Theory 44, 2045 (1998).
23. Miller & Madow, Air Force Cambridge Research Center Technical Report No. 54-75, 1954 (unpublished); B. Harris, Colloquia Mathematica Societatis Janos Bolya, p. 323 (1975);
24. H. Herzel, Syste. Anal. Model Sim. 5, 435 (1988).W. E. Caswell and J. A. Yorke, in Dimensions and Entropies in Chaotic Systems, edited by G. Mayer-Kress (Springer-Verlag, Berlin 1986).
25. I. Kontoyiannis, P. H. Algoet, Y. M. Suhov, and A. J. Wyner, IEEE Trans. Inf. Theory 44, 1319 (1998).
26. J. Ziv and A. Lempel, IEEE Trans. Inf. Theory 23, 337 (1977).
27. P. Elias, IEEE Trans. Inf. Theory 21, 194 (1973).
28. We conjecture that the unique correspondence remains for (Formula presented) as long as (Formula presented)
29. E. N. Lorenz, Tellus, Ser. A 36A, 98 (1984). The model is (Formula presented) We recorded the x coordinate every (Formula presented) and discretized by histogramming.
30. J. P. Crutchfield and C. R. Shalizi, Phys. Rev. E 59, 275 (1999).
31. J. P. Crutchfield and K. Young, Phys. Rev. Lett. 63, 105 (1989).
32. R. C. Carassco and J. Oncina, in Proceedings of the 2nd International Colloques on Grammatical Inference and Applications (Springer, Berlin, 1994), pp. 139–152.
33. A. M. Fraser, IEEE Trans. Inf. Theory 35, 245 (1989);
34. A. M. Fraser and H. L. Swinney, Phys. Rev. A 33, 1134 (1986).
35. M. Lehrman, A. B. Rechester, and R. B. White, Phys. Rev. Lett. 78, 54 (1997).
36. C. S. Daw, C. E. A. Finney, and M. B. Kennel, Phys. Rev. E 62, 1912 (2000).
37. R. Steuer, W. Ebeling, D. F. Russell, S. Bahar, A. Neiman, and F. Moss, Phys. Rev. E 64, 061911 (2001).
38. S. Lesher, L. Guan, and A. H. Cohen, Neurocomputing 32-33, 1073(2000).