Schemes for bidirectional modeling of discrete stationary sources

IEEE Transactions on Information Theory

Volumn 52, Issue 11, 2006, Pages 4789-4807

  Yu, Jiming ^a   Verdú, Sergio ^a  

^a Princeton University   (United States)

Author keywords

Bidirectional modeling; Discrete stationary sources; Universal algorithms; Universal discrete denoising

Indexed keywords

ALGORITHMS; MATHEMATICAL MODELS; STATISTICAL METHODS;

BIDIRECTIONAL MODELING; DISCRETE STATIONARY SOURCES; UNIVERSAL ALGORITHMS; UNIVERSAL DISCRETE DENOISING;

INFORMATION THEORY;

EID: 33750590426     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2006.883626     Document Type: Article

References (43)

1. T. Weissman, E. Ordentlich, G. Seroussi, S. Verdú, and M. Weinberger, "Universal discrete denoising: Known channel," IEEE Trans. Inf. Theory, vol. 51, no. 1, pp. 5-28, Jan. 2005.
2. F. M. J.Willems, Y. M. Shtarkov, and T. J. Tjalkens, "The context-tree weighting method: Basic properties," IEEE Trans. Inf. Theory, vol. 41, no. 3, pp. 653-664, May 1995.
3. F. M. J. Willems, "The context-tree weighting method: Extensions," IEEE Trans. Inf. Theory, vol. 44, no. 2, pp. 792-798, Mar. 1998.
4. T. Weissman and N. Merhav, "Universal prediction of individual binary sequences in the presence of noise," IEEE Trans. Inf. Theory, vol. 47, no. 6, pp. 2151-2173, Sep. 2001.
5. T. Weissman and N. Merhav, "Universal prediction of random binary sequences in a noisy environment," Ann. Appl. Probab., vol. 14, no. 1, pp. 54-89, 2004.
6. E. Ordentlich, T. Weissman, M. J. Weinberger, A. Somekh-Baruch, and N. Merhav, "Discrete universal filtering through incremental parsing," in Proc. IEEE Data Compression Conf., Snowbird, UT,Mar. 2004, pp. 352-361.
7. E. Ordentlich, G. Seroussi, S. Verdú, and K. Viswanathan, "Universal Algorithms for Channel Decoding of Uncompressed Sources", submitted for publication.
8. J. Rissanen, "A universal data compression system," IEEE Trans. Inf. Theory, vol. IT-29, no. 5, pp. 656-664, Sep. 1983.
9. M. J. Weinberger, J. J. Rissanen, and M. Feder, "A universal finite memory source," IEEE Trans. Inf. Theory, vol. 41, no. 3, pp. 643-652, May 1995.
10. P. Bühlmann and A. J. Wyner, "Variable length Markov chains," Ann. Statist., vol. 27, pp. 480-513, 1999.
11. F. Ferrari and A. J. Wyner, "Estimation of general stationary processes by variable length Markov chains," Scand. J. Statist., pp. 459-480, Sep. 2003.
12. M. Mächler and P. Bühlmann, "Variable length Markov chains: Methodology, computing and software," J. Comput. Graph. Statist., pp. 435-455, Jun. 2004.
13. R. E. Krichevsky and V. K. Trofimov, "The performance of universal encoding," IEEE Trans. Inf. Theory, vol. IT-27, no. 2, pp. 199-207, Mar. 1981.
14. I. Csiszár and Z. Talata, "Context tree estimation for not necessarily finite memory processes, via BIC and MDL," IEEE Trans. Inf. Theory, vol. 52, no. 3, pp. 1007-1016, Mar. 2006.
15. F. M. J. Willems, Y. M. Shtarkov, and T. J. Tjalkens, "Context-tree maximizing," in Proc. 2000 Conf. Information Sciences and Systems, Princeton, NJ, Mar. 2000.
16. E. Ordentlich, G. Seroussi, S. Verdú, K. Viswanathan, M. Weinberger, and T. Weissman, "Channel decoding of systematically encoded unknown redundant sources," in Proc. IEEE Int. Symp. Information Theory, Chicago, IL, Jun./Jul. 2004, p. 165.
17. G. M. Gemelos, S. Sigurjonsson, and T. Weissman, "Universal minimax discrete denoising under channel uncertainty," IEEE Trans. Inf. Theory, vol. 52, no. 8, pp. 3476-3497, Aug. 2006.
18. G. M. Gemelos, S. Sigurjonsson, and T. Weissman, "Algorithms for discrete denoising under channel uncertainty," IEEE Trans. Signal Process., vol. 54, no. 6, pp. 2263-2276, Jun. 2006.
19. L. R. Rabiner, "A tutorial on hidden Markov models and selected applications in speech recognition," Proc. IEEE, vol. 77, no. 2, pp. 257-286, Feb. 1989.
20. Y. Ephraim and N. Merhav, "Hidden Markov processes," IEEE Trans. Inf. Theory, vol. 48, no. 6, pp. 1518-1569, Jun. 2002.
21. T. Moon and T. Weissman, "Discrete universal filtering via hidden Markov modelling," in Proc. IEEE Int. Symp. Information Theory, Adelaide, Australia, Sep. 2005, pp. 1285-1289.
22. J. Yu and S. Verdú, "Schemes for bidirectional modeling of discrete stationary sources," in Proc. 39th Annu. Conf. Information Science and Systems, Baltimore, MD, Mar. 2005.
23. E. Ordentlich, M. J. Weinberger, and T. Weissman, "Multi-directional context sets with applications to universal denoising and compression," in Proc. IEEE Int. Symp. Information Theory, Adelaide, Australia, Sep. 2005, pp. 1270-1274.
24. E. Ordentlich, M. J. Weinberger, and T. Weissman, "Efficient pruning of bidirectional context trees with applications to universal denoising and compression," in Proc. IEEE Information Theory Workshop, San Antonio, TX, Oct. 2004, pp. 94-98.
25. D. Baron and Y. Bresler, "An O(N) semipredictive universal encoder via the BWT," IEEE Trans. Inf. Theory, vol. 50, no. 5, pp. 928-937, May 2004.
26. R. M. Gray, "Information rates of autoregressive processes," IEEE Trans. Inf. Theory, vol. IT-16, no. 4, pp. 412-421, Jul. 1970.
27. R. M. Gray, "Rate distortion functions for finite-state finite-alphabet Markov sources," IEEE Trans. Inf. Theory, vol. IT-17, no. 2, pp. 127-134, Mar. 1971.
28. F. Spitzer, "Markov random fields and Gibbs ensembles," Amer. Math. Monthly, vol. 78, no. 2, pp. 142-154, 1971.
29. F. Spitzer, Random Fields and Interacting Particle Systems. Williamstown, MA: Math. Assoc. America, 1971, Notes on lectures given at the 1971 MAA Summer Seminar, Williams College.
30. J. Besag, "Spatial interaction and the statistical analysis of lattice systems," J. Roy. Statist. Soc. Ser. B (Methodological), vol. 36, no. 2, pp. 192-236, 1974.
31. S. Geman and D. Geman, "Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images," IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-6, pp. 721-741, Nov. 1984.
32. H.-O. Georgii, Gibbs Measures and Phase Transitions. Berlin, Germany: Water de Gruyter, 1988, vol. 9, de Gruyter Studies in Mathematics.
33. H. Cai, S. R. Kulkarni, and S. Verdú, "Universal divergence estimation for finite-alphabet sources," IEEE Trans. Inf. Theory, vol. 52, no. 8, pp. 3456-3475, Aug. 2006.
34. S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge, U.K.: Cambridge Univ. Press, 2004.
35. F. M. J. Willems and T. J. Tjalkens, Complexity Reduction of the Context-Tree Weighting Algorithm: A Study for KPN Research, Tech. Univ. Eindhoven, Eindhoven, The Netherlands, 1997, EIDMA Rep. RS.97.01.
36. P. Volf, "Weighting Techniques in Data Compression: Theory and Algorithm," Ph.D. dissertation, Tech. Univ. Eindhoven, Eindhoven, The Netherlands, 2002.
37. J. Rissanen, "Modeling by shortest data description," Automatica, vol. 14, pp. 465-471, 1978.
38. J. Rissanen, "A universal prior for integers and estimation by minimum description length," Ann. Statist., vol. 11, no. 2, pp. 416-431, 1983.
39. J. Rissanen, "Universal coding, information, prediction, and estimation," IEEE Trans. Inf. Theory, vol. IT-30, no. 4, pp. 629-635, Jul. 1984.
40. I. Csiszár and Z. Talata, "Consistent estimation of the basic neighborhood of Markov random fields," Ann. Statist., vol. 34, pp. 123-145, Feb. 2006.
41. G. Schwarz, "Estimating the dimension of a model," Ann. Statist., vol. 6, pp. 461-464, Mar. 1978.
42. R. Giegerich and S. Kurtz, "From Ukkonen to McCreight and Weiner: A unifying view of linear-time suffix tree construction," Algorithmica, vol. 19, pp. 331-353, Nov. 1997.
43. A. Martín, G. Seroussi, and M. J. Weinberger, "Linear time universal coding and time reversal of tree sources via FSM closure," IEEE Trans. Inf. Theory, vol. 50, no. 7, pp. 1442-1468, Jul. 2004.