Tracking the best quantizer

IEEE Transactions on Information Theory

Volumn 54, Issue 4, 2008, Pages 1604-1625

  Gyorgy, András ^a   Linder, Tamás ^b   Lugosi, Gábor ^c  

^a INSTITUTE OF EXPERIMENTAL MEDICINE   (Hungary)

^b QUEEN S UNIVERSITY   (Canada)

^c UNIVERSITAT POMPEU FABRA   (Spain)

Author keywords

Algorithmic efficiency; Individual sequences; Lossy source coding; Multi resolution coding; Multiple description quantization; Nonstationary sources; Scalar quantization; Sequential coding; Sequential prediction

Indexed keywords

ALGORITHMS; ASYMPTOTIC ANALYSIS; GRAPH THEORY; INFORMATION THEORY; RANDOM PROCESSES;

LOSSY SOURCE CODING; MULTI RESOLUTION CODING; MULTIPLE DESCRIPTION QUANTIZATION; SCALAR QUANTIZATION; SEQUENTIAL CODING;

ENCODING (SYMBOLS);

EID: 41949135363     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2008.917651     Document Type: Article

References (43)

1. T. Linder and G. Lugosi, "A zero-delay sequential scheme for lossy coding of individual sequences," IEEE Trans. Inf. Theory, vol. 47, no. 6, pp. 2533-2538, Sep. 2001.
2. T. Weissman and N. Merhav, "On limited-delay lossy coding and filtering of individual sequences," IEEE Trans. Inf. Theory, vol. 48, no. 3, pp. 721-733, Mar. 2002.
3. J. Hannan, "Approximation to Bayes risk in repeated plays," in Contributions to the Theory of Games, M. Dresher, A. Tucker, and P. Wolfe, Eds. Princeton, NJ: Princeton Univ. Press, 1957, vol. 3, pp. 97-139.
4. D. Blackwell, "An analog of the minimax theorem for vector payoffs," Pacific J. Math., vol. 6, pp. 1-8, 1956.
5. V. Vovk, "Aggregating strategies," in Proc 3rd Annu. Workshop Computational Learning Theory (Rochester, NY, Aug. 1990). San Francisco, CA: Morgan Kaufmann, 1990, pp. 372-383.
6. N. Littlestone and M. K. Warmuth, "The weighted majority algorithm," Inf. Comput., vol. 108, pp. 212-261, 1994.
7. N. Cesa-Bianchi, Y. Freund, D. P. Helmbold, D. Haussler, R. Schapire, and M. K. Warmuth, "How to use expert advice," J. ACM, vol. 44, no. 3, pp. 427-485, 1997.
8. N. Cesa-Bianchi and G. Lugosi, Prediction, Learning, and Games. New York: Cambridge Univ. Press, 2006.
9. R. E. Schapire and D. P. Helmbold, "Predicting nearly as well as the best pruning of a decision tree," Mach. Learn., vol. 27, pp. 51-68, 1997.
10. F. Pereira and Y. Singer, "An efficient extension to mixture techniques for prediction and decision trees," Mach. Learn., vol. 36, pp. 183-199, 1999.
11. M. Mohri, General Algebraic Frameworks and Algorithms for Shortest Distance Problems AT&T Labs Research, Tech. Rep. 981219-10TM, 1998.
12. E. Takimoto and M. K. Warmuth, "Path kernels and multiplicative updates," in Proc. 15th Annu. Conf. Computational Learning Theory, COLT 2002, J. Kivinen and R. H. Sloan, Eds. Berlin, Heidelberg: Springer-Verlag, Jul. 2002, vol. LNAI 2375, pp. 74-89.
13. E. Takimoto and M. K. Warmuth, "Path kernels and multiplicative updates," J. Mach. Learn. Res., vol. 4, pp. 773-818, 2003.
14. A. György, T. Linder, and G. Lugosi, "Efficient algorithms and minimax bounds for zero-delay lossy source coding," IEEE Trans. Signal Process., vol. 52, no. 8, pp. 2337-2347, Aug. 2004.
15. A. Kalai and S. Vempala, "Efficient algorithms for the online decision problem," in Proc. 16th Annu. Conf. Learning Theory and the 7th Kernel Workshop, COLT-Kernel 2003, B. Schölkopf and M.Warmuth, Eds. New York: Springer-Verlag, Aug. 2003, pp. 26-40.
16. A. György, T. Linder, and G. Lugosi, "A "follow the perturbed leader"-type algorithm for zero-delay quantization of individual sequences," in Proc. Data Compression Conf., Snowbird, UT, Mar. 2004, pp. 342-351.
17. S. Matloub and T. Weissman, "Universal zero-delay joint source-channel coding," IEEE Trans. Inf. Theory, vol. 52, no. 12, pp. 5240-5250, Dec. 2006.
18. F. M. J. Willems, "Coding for a binary independent piecewise-identically-distributed source," IEEE Trans. Inf. Theory vol. 42, no. 6, pp. 2210-2217, Nov. 1996.
19. G. I. Shamir and N. Merhav, "Low-complexity sequential lossless coding for piecewise-stationary memoryless sources," IEEE Trans. Inf. Theory, vol. 45, no. 5, pp. 1498-1519, Jul. 1999.
20. G. I. Shamir and D. J. Costello, Jr., "Asymptotically optimal low-complexity sequential lossless coding for piecewise-stationary memoryless sources - Part I: The regular case," IEEE Trans. Inform. Theory, vol. 46, no. 7, pp. 2444-2467, Nov. 2000.
21. M. Herbster and M. K. Warmuth, "Tracking the best expert," Mach. Learn., vol. 32, no. 2, pp. 151-178, 1998.
22. P. Auer and M. K. Warmuth, "Tracking the best disjunction," Mach. Learn., vol. 32, no. 2, pp. 127-150, 1998.
23. V. Vovk, "Derandomizing stochastic prediction strategies," Mach. Learn., vol. 35, pp. 247-282, 1999.
24. O. Bousquet and M. K. Warmuth, "Tracking a small set of experts by mixing past posteriors," J. Mach. Learn. Res., vol. 3, pp. 363-396, Nov. 2002.
25. M. Herbster and M. K. Warmuth, "Tracking the best linear predictor," J. Mach. Learn. Res., vol. 1, pp. 281-309, 2001.
26. X. Wu and K. Zhang, "Quantizer monotonicities and globally optimal scalar quantizer design," IEEE Trans. Inf. Theory, vol. 39, no. 3, pp. 1049-1053, May 1993.
27. A. Aggarwal, B. Schieber, and T. Tokuyama, "Finding a minimum weight K-link path in graphs with Monge property and applications," in Proc. 9th Annu. Symp. Computational Geometry, San Diego, CA, May 1993, pp. 189-197.
28. D. Muresan and M. Effros, "Quantization as histogram segmentation: Globally optimal scalar quantizer design in network systems," in Proc. Data Compression Conf., Snowbird, UT, Apr. 2002, pp. 302-311.
29. A. György, T. Linder, and G. Lugosi, "Tracking the best of many experts," in Proc. 18th Annu. Conf. Learning Theory, COLT 2005 (Bertinoro, Italy, Jun. 2005). Berlin, Germany: Springer-Verlag, 2005, pp. 204-216.
30. L. H. Ozarow, "On the source coding problem with two channels and three receivers," Bell Syst. Tech. J., vol. 59, pp. 1909-1922, 1980.
31. A. El Gamal and T. M. Cover, "Achievable rates for multiple descriptions," IEEE Trans. Inf. Theory, vol. IT-28, no. 6, pp. 851-857, Nov. 1982.
32. V. K. Goyal, "Multiple description coding: Compression meets the network," IEEE Signal Process. Mag., vol. 18, no. 5, pp. 74-93, Sep. 2001.
33. W. E. Equitz and T. Cover, "Successive refinement of information," IEEE Trans. Inf. Theory, vol. 37, no. 2, pp. 269-275, Mar. 1991.
34. B. Rimoldi, "Successive refinement of information: Characterization of the achievable rates," IEEE Trans. Inf. Theory, vol. 40, no. 1, pp. 253-259, Jan. 1994.
35. M. Effros and D. Dugatkin, "Multiresolution vector quantization," IEEE Trans. Inf. Theory, vol. 50, no. 12, pp. 3130-3145, Dec. 2004.
36. V. A. Vaishampayan, "Design of multiple description scalar quantizers," IEEE Trans. Inf. Theory, vol. 39, no. 3, pp. 821-834, May 1993.
37. M. Effros and D. Muresan, "Codecell contiguity in optimal fixed-rate and entropy-constrained network scalar quantizers," in Proc. Data Compression Conf., Snowbird, UT, Apr. 2002, pp. 312-321.
38. S. Dumitrescu, X. Wu, and G. Bahl, "Fast algorithms for optimal two-description scalar quantizer design," in Proc. Data Compression Conf., Snowbird, UT, Mar. 2004, pp. 42-51.
39. S. Dumitrescu and X. Wu, "Algorithms for optimal multi-resolution quantization," J. Algorithms, vol. 50, pp. 1-22, Jan. 2004.
40. S. Dumitrescu and X. Wu, "Lagrangian global optimization of two-description scalar quantizers," in Proc. 2004 IEEE Int. Symp. Information Theory, Chicago, IL, Jun./Jul. 2004, p. 301.
41. C. Tiam and S. S. Hemami, "Universal multiple description scalar quantization: Analysis and design," IEEE Trans. Inf. Theory, vol. 51, no. 9, pp. 2089-2102, Sep. 2004.
42. W. Hoeffding, "Probability inequalities for sums of bounded random variables," J. Amer. Statist. Assoc., vol. 58, pp. 13-30, 1963.
43. N. Cesa-Bianchi and G. Lugosi, "On prediction of individual sequences," Ann. Statist., vol. 27, pp. 1865-1895, 1999.