Linear time universal coding and time reversal of tree sources via FSM closure

IEEE Transactions on Information Theory

Volumn 50, Issue 7, 2004, Pages 1442-1468

  Martín, Alvaro ^a   Seroussi, Gadiel ^b   Weinberger, Marcelo J ^b  

^a UNIVERSIDAD DE LA REPÚBLICA   (Uruguay)

^b HEWLETT PACKARD LABORATORIES   (United States)

Author keywords

Context algorithm; Finite memory; Finite state machines (FSMs); Suffix trees; Tree sources; Universal coding

Indexed keywords

ALGORITHMS; DATA STRUCTURES; ESTIMATION; INFORMATION THEORY; MATHEMATICAL MODELS;

FINITE MEMORY PROCESS; UNIVERSAL CODING;

FINITE AUTOMATA;

EID: 3042656666     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2004.830763     Document Type: Article

References (43)

1. J. Rissanen, "Universal coding, information, prediction, and estimation," IEEE Trans. Inform. Theory, vol. IT-30, pp. 629-636, July 1984.
2. ____, "A universal data compression system," IEEE Trans. Inform. Theory, vol. IT-29, pp. 656-664, Sept. 1983.
3. M. J. Weinberger, J. Rissanen, and M. Feder, "A universal finite memory source," IEEE Trans. Inform. Theory, vol. 41, pp. 643-652, May 1995.
4. P. L. Buhlmann, and A. Wyner, "Variable length Markov chains," Ann. Statist., vol. 27, pp. 480-513, 1998.
5. R. Nohre, "Some topics in descriptive complexity," Ph.D. dissertation, Dept. Comput. Sci., Tech., Univ. Linköping, Linköping, Sweden, 1994.
6. F. M. J. Willems, Y. M. Shtarkov, and T. J. Tjalkens, "The context-tree weighting method: Basic properties," IEEE Trans. Inform. Theory, vol. 41, pp. 653-664, May 1995.
7. F. M. J. Willems, "The context-tree weighting method: Extensions," IEEE Trans. Inform. Theory, vol. 44, pp. 792-798, Mar. 1998.
8. B. Ryabko, "Prediction of random sequences and universal coding," Probl. Inform. Transm., vol. 24, pp. 87-96, Apr./June 1988.
9. M. J. Weinberger, A. Lempel, and J. Ziv, "A sequential algorithm for the universal coding of finite-memory sources," IEEE Trans. Inform. Theory, vol. 38, pp. 1002-1014, May 1992.
10. B. Ryabko, "Twice-universal coding," Probl. Inform. Transm., vol. 20, pp. 173-177, July/Sept. 1984.
11. M. Merhav, and M. Feder, "Universal prediction," IEEE Trans. Inform. Theory, vol. 44, pp. 2124-2147, Oct. 1998.
12. M. J. Weinberger, N. Merhav, and M. Feder, "Optimal sequential probability assignment for individual sequences," IEEE Trans. Inform. Theory, vol. 40, pp. 384-396, Mar. 1994.
13. J. Rissanen, "Stochastic complexity and modeling," Ann. Statist., vol. 14, pp. 1080-1100, Sept. 1986.
14. F. M. J. Willems, Y. M. Shtarkov, and T. J. Tjalkens, "Context-tree maximizing," in Proc. 2000 Conf. Information Sciences and Systems, Princeton, NJ, Mar. 2000, pp. TP6-7-TP6-12.
15. R. E. Krichevskii, and V. K. Trofimov, "The performance of universal encoding," IEEE Trans. Inform. Theory, vol. IT-27, pp. 199-207, Mar. 1981.
16. R. Giegerich and S. Kurtz, "From Ukkonen to McCreight and Weiner: A unifying view to linear-time suffix tree construction," Algorithmica, vol. 19, pp. 331-353, Nov. 1997.
17. D. Baron and Y. Bresler, "An O(n) semipredictive universal encoder via the BWT," IEEE Trans. Inform. Theory, vol 50, pp. 928-937, May 2004.
18. J. G. Cleary, and I. H. Witten, "Data compression using adaptive coding and partial string matching," IEEE Trans. Commun., vol. COM-32, pp. 396-402, Apr. 1984.
19. M. Burrows, and D. J. Wheeler, "A Block-Sorting Lossless Data Compression Algorithm," Digital Systems Research Center, Palo Alto, CA, Tech. Rep. SRC Res. Rep. 124, 1994.
20. M. Effros, "PPM performance with BWT complexity: A fast and effective data compression algorithm," Proc. IEEE, vol. 88, pp. 1703-1712, Nov. 2000.
21. N. J. Larsson, "Extended application of suffix trees to data compression," in Proc. 1996 Data Compression Conf., Snowbird, UT, Apr. 1996, pp. 190-199.
22. M. Effros, K. Visweswariah, S. Kulkarni, and S. Verdú, "Universal lossless source coding with the Burrows-Wheeler transform," IEEE Trans. Inform. Theory, vol. 48, pp. 1061-1081, May 2002.
23. N. J. Larsson, "The context trees of block sorting compression," in Proc. 1998 Data Compression Conf., Snowbird, UT, Mar. 1998, pp. 189-198.
24. T. M. Cover, and J. A. Thomas, Elements of Information Theory. New York: Wiley, 1991.
25. J. Rissanen, and G. G. Langdon, "Universal modeling and coding," IEEE Trans. Inform. Theory, vol. IT-27, 12-23, Jan. 1981.
26. R. B. Ash, Information Theory: Wiley, 1967.
27. W. Feller, Probability Theory and Its Applications, 3rd Ed. New York: Wiley, 1968, vol. 1.
28. D. E. Knuth, The Art of Computer Programming. Fundamental Algorithms, 3rd ed. Reading, MA: Addison-Wesley, 1997, vol. 1.
29. ____, The Art of Computer Programming. Sorting and Searching, 2rd ed. Reading, MA: Addison-Wesley, 1997, vol. 3.
30. D. R. Morrison, "Patricia - Practical algorithm to retrieve information coded in alphanumeric," J. Assoc. Comput. Mach., vol. 15, no. 4. pp. 514-534, 1968.
31. W. Szpankowski, Average Case Analysis of Algorithms on Sequences. New York: Wiley, 2001.
32. P. Weiner, "Linear pattern matching algorithms," in Proc. 14th IEEE Annu. Symp. Switching and Automata Theory, 1973, pp. 1-11.
33. E. McCreight, "A space-economical suffix tree construction algorithm," J. Assoc. Comput. Mach., vol. 23, no. 2, pp. 262-272, 1976.
34. M. Feder, N. Merhav, and M. Gutnam, "Universal prediction of individual sequences," IEEE Trans. Inform. Theory, vol. 38, pp. 1258-1270, July 1992.
35. M. J. Weinberger, and M. Feder, "Predictive stochastic complexity and model estimation for finite-state processes," J. Statist. Planning and Inference, vol. 39, pp. 353-372, 1994.
36. N. G. de Bruijn, "A combinatorial problem," Proc. Koninklijke Nederlands Akademie van Wetenschappen, vol. 49, pp. 758-764, Part 2, 1946.
37. S. W. Golomb, Shift Register Sequences. San Francisco, CA: Holden-Day, 1967.
38. J. M. Hall, Combinatorial Theory, 2nd ed. New York: Wiley, 1986.
39. A. Martín, G. Seroussi, and M. J. Weinberger, "Linear Time Universal Coding of Tree Sources," Hewlett-Packard Labaratories, Tech. Rep., 2003.
40. T. M. Cover, "Enumerative source encoding," IEEE Trans. Inform. Theory, vol. IT-19, pp. 73-77, Jan. 1973.
41. D. Preuss, "Two-dimensional facsimile source encoding based on a Markov model," Nachrichtentech. Z., vol. 28, 358-363, Oct. 1975.
42. D. Gillman and R. L. Rivest, "Complete variable-length fix-free codes," Des., Codes Cryptogr., vol. 5, no. 2 pp. 109-114, 1995.
43. G. Seroussi, and M. J. Weinberger, "On tree sources, finite state machines, and time reversal," in Proc. IEEE Int. Symp. Information Theory, Whistler, BC, Canada, Sept. 1995, p. 390.