Universal filtering via prediction

IEEE Transactions on Information Theory

Volumn 53, Issue 4, 2007, Pages 1253-1264

  Weissman, Tsachy ^a   Ordentlich, Erik ^b   Weinberger, Marcelo J ^b   Somekh Baruch, Anelia ^c   Merhav, Neri ^d  

^a Stanford University   (United States)

^b HEWLETT PACKARD LABORATORIES   (United States)

^c Princeton University   (United States)

^d TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

Author keywords

Compound sequential decision problem; Discrete denoising; Expert advice; Filtering; Individual sequences; Prediction; Semistochastic setting

Indexed keywords

COMMUNICATION CHANNELS (INFORMATION THEORY); ELECTRIC FILTERS; SPURIOUS SIGNAL NOISE;

COMPOUND SEQUENTIAL DECISION PROBLEM; DISCRETE DENOISING; DISCRETE MEMORYLESS CHANNEL;

SIGNAL FILTERING AND PREDICTION;

EID: 34147139873     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2007.892782     Document Type: Article

References (43)

1. R. J. Ballard, "Extended rules for the sequence compound decision problem with m × n component," Ph.D. dissertation, Michigan State Univ., East Lansing, 1974.
2. -1/2) convergence to k-extended Bayes risk in the sequence compound decision problem with m × n component," in Statistics and Probability RM-333. East Lansing: Michigan State Univ., 1974.
3. A. Baruch, "Universal algorithms for sequential decision in the presence of noisy observations," Master's thesis, Dept. Elec. Eng., Technion, Haifa, Israel, Feb. 1999.
4. A. Baruch and N. Merhav, "Universal filtering and prediction of individual sequences corrupted by noise," in Proc. 37th Annu. Allerton Conf. Communication, Control, and Computing, Monticello, IL, Sep. 1999, pp. 470-479.
5. D. Blackwell, "An analog of the minmax theorem for vector payoffs," Pacific J. Math, vol. 6, pp. 1-8, 1956.
6. D. Blackwell, "Controlled random walks," in Proc. Int. Congr. Mathematicians. Amsterdam, The Netherlands: North Holland, 1956, vol. 3, pp. 336-338.
7. N. Cesa-Bianchi, Y. Freund, D. P. Helmbold, D. Haussler, R. Schapire, and M. K. Warmuth, "How to use expert advice," J. Assoc. Comp. Mach., vol. 44, no. 3, pp. 427-485, 1997.
8. N. Cesa-Bianchi and G. Lugosi, "On prediction of individual sequences," Ann. Statist., vol. 27, no. 6, pp. 1865-1895, 1999.
9. N. Cesa-Bianchi and G. Lugosi, Prediction, Learning, and Games. New York: Cambridge Univ. Press, 2006.
10. T. M. Cover, "Behavior of sequential predictors of binary sequences," in Trans. 4th Prague Conf. Information Theory, Prague, Czechoslovakia, Sep. 1966.
11. T. M. Cover and A. Shenhar, "Compound Bayes predictors for sequences with apparent Markov structure," IEEE Trans. Syst. Man Cybern., vol. SMC-7, no. 6, pp. 421-424, Jun. 1977.
12. T. M. Cover and J. A.Thomas, Elements of Information Theory. New York: Wiley, 1991.
13. L. Devroye, L. Györfi, and G. Lugosi, A Probabilistic Theory of Pattern Recognition. New York: Springer-Verlag, 1996.
14. R. L. Dobrushin and B. S. Tsybakov, "Information transmission with additional noise," IEEE Trans. Inf. Theory, vol. IT-8, no. 5, pp. 293-304, Sep. 1962.
15. R. Durret, Probability: Theory and Examples. Belmont, CA: Duxbury, 1991.
16. Y. Ephraim and R. M. Gray, "A unified approach for encoding clean and noisy sources by means of waveform and autoregressive model vector quantization," IEEE Trans. Inf. Theory, vol. 34, no. 4, pp. 826-834, Jul. 1988.
17. Y. Ephraim and N. Merhav, "Hidden Markov processes," IEEE Trans. Inf. Theory, vol. 48, no. 6, pp. 1518-1569, Jun. 2002.
18. M. Feder, N. Merhav, and M. Gutman, "Universal prediction of individual sequences," IEEE Trans. Inf. Theory, vol. 38, no. 4, pp. 1258-1270, Jul. 1992.
19. J. Hannan, "Approximation to Bayes risk in repeated play," in Contributions to the Theory of Games. Princeton, NJ: Princeton Univ. Press, 1957, vol. III, pp. 97-139.
20. J. F. Hannan and H. Robbins, "Asymptotic solutions of the compound decision problem for two completely specified distributions," Ann. Math. Statist., vol. 26, pp. 37-51, 1955.
21. W. Hoeffding, "Probability inequalities for sums of bounded random variables," Amer. Statist. Assoc., vol. 58, pp. 13-30, 1963.
22. P. Lancaster and M. Tismenetsky, The Theory of Matrices. Orlando, FL: Academic, 1985.
23. N. Merhav and M. Feder, "Universal schemes for sequential decision from individual data sequences," IEEE Trans. Inf. Theory, vol. 39, no. 4, pp. 1280-1292, Jul. 1993.
24. N. Merhav and M. Feder, "Universal prediction," IEEE Trans. Inf. Theory, vol. 44, no. 6, pp. 2124-2147, Oct. 1998.
25. E. Ordentlich, T. Weissman, M. J. Weinberger, A. Somekh-Baruch, and N. Merhav, "Discrete universal filtering through incremental parsing," in Proc. 2004 Data Compression Conf. (DCC'04), Snowbird, UT, Mar. 2004, pp. 352-361.
26. H. Robbins, "Asymptotically subminimax solutions of compound statistical decision problems," in Proc. 2nd Berkeley Symp. Math. Statis. Prob., Berkeley, CA, 1951, pp. 131-148.
27. J. Van Ryzin, "The compound decision problem with m × n finite loss matrix," Ann. Math. Statist., vol. 37, pp. 412-424, 1966.
28. J. Van Ryzin, "The sequential compound decision problem with m × n finite loss matrix," Ann. Math. Statist., vol. 37, pp. 954-975, 1966.
29. E. Samuel, "Asymptotic solutions of the sequential compound decision problem," Ann. Math. Statist., pp. 1079-1095, 1963.
30. C. E. Shannon, "Channels with side information at the transmitter," IBM J. Res. Devel., vol. 2, pp. 289-293, 1958.
31. S. B. Vardeman, "A note on the applicability of sequence compound decision schemes," Scand. J. Statist., no. 6, pp. 86-88, 1979.
32. S. B. Vardeman, "Admissible solutions of k-extended finite state set and the sequence compound decision problems," J. Multiv. Anal. vol. 10, pp. 426-441, 1980.
33. S. B. Vardeman, "Approximation to minimum k-extended Bayes risk in sequences of finite state decision problems and games," Bulletin of the Institute of Mathematics Academia Sinica, vol. 10, no. 1, pp. 35-52, Mar. 1982.
34. V. Vovk, "Aggregating strategies," in Proc. 3rd Annu. Workshop on Computational Learning Theory, San Mateo, CA, 1990, pp. 371-383.
35. M. J. Weinberger and E. Ordentlich, "On-line decision making for a class of loss functions via Lempel-Ziv parsing," in Proc. 2000 Data Compression Conf. (DCC'00), Snowbird, UT, Mar. 2000, pp. 163-172.
36. T. Weissman, "Compound sequential decisions against the well-informed antagonist," in Proc. 2006 IEEE Information Theory Workshop, Punta del Este, Uruguay, Mar. 2006, pp. 77-81.
37. T. Weissman and N. Merhav, "Universal prediction of individual binary sequences in the presence of noise," IEEE Trans. Inf. Theory, vol. 47, no. 6, pp. 2151-2173, Sep. 2001.
38. T. Weissman and N. Mehav, "On limited-delay lossy coding and filtering of individual sequences," IEEE Trans. Inf. Theory, vol. 48, no. 3, pp. 721-733, Mar. 2002.
39. T. Weissman, N. Merhav, and A. Somekh-Baruch, "Twofold universal prediction schemes for achieving the finite-state predictability of a noisy individual binary sequence," IEEE Trans. Inf. Theory, vol. 47, no. 5, pp. 1849-1866, Jul. 2001.
40. T. Weissman, E. Ordentlich, G. Seroussi, S. Verdú, and M. Weinberger, "Universal discrete denoising: Known channel," IEEE Trans. Inf. Theory, vol. 51, no. 1, pp. 5-28, Jan. 2005.
41. H. S. Witsenhausen, "Indirect rate distortion problems," IEEE Trans. Inf. Theory, vol. IT-26, no. 5, pp. 518-521, Sep. 1980.
42. C. H. Zhang, "Compound decision theory and empirical Bayes methods," Ann. Statist., vol. 31, no. 2, pp. 379-390, 2003.
43. J. Ziv and A. Lempel, "Compression of individual sequences via variable-rate coding," IEEE Trans. Inf. Theory, vol. IT-24, no. 5, pp. 530-536, Sep. 1978.