Optimality of KLT for high-rate transform coding of Gaussian vector-scale mixtures: Application to reconstruction, estimation, and classification

IEEE Transactions on Information Theory

Volumn 52, Issue 9, 2006, Pages 4049-4067

  Jana, Soumya ^a   Moulin, Pierre ^a  

^a University of Illinois at Urbana Champaign   (United States)

Author keywords

Chernoff distance; Classification; Estimation; f divergence; Gaussian scale mixture; High resolution quantization; Karhunen Lo ve transform (KLT); Mean squared error (MSE); Multiplicative noise; Quadratic criterion

Indexed keywords

CLASSIFICATION (OF INFORMATION); ESTIMATION; OPTIMIZATION; SIGNAL ENCODING; SIGNAL RECONSTRUCTION; WAVELET TRANSFORMS; WHITE NOISE;

CHERNOFF DISTANCE; GAUSSIAN VECTOR-SCALE MIXTURES; HIGH RESOLUTION QUANTIZATION; KARHUNEN-LOEVE TRANSFORM (KLT); MEAN SQUARED ERROR; MULTIPLICATIVE NOISE; QUADRATIC CRITERION;

GAUSSIAN NOISE (ELECTRONIC);

EID: 33748521889     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIT.2006.880056     Document Type: Article

References (54)

1. J.-Y. Huang and P. Schultheiss, "Block quantization of correlated Gaussian random variables," IEEE Trans. Commun., vol. COM-11, no. 3, pp. 289-296, Sep. 1963.
2. A. Gersho and R. M. Gray, Vector Quantization and Signal Compression. Boston, MA: Kluwer, 1992.
3. S. Mallat, A Wavelet Tour of Signal Processing. San Diego, CA: Academic, 1999.
4. M. Effros, H. Feng, and K. Zeger, "Suboptimality of the Karhunen-Loéve transform for transform coding," IEEE Trans. Inf. Theory, vol. 50, no. 8, pp. 1605-1619, Aug. 2004.
5. M. J. Wainwright, E. P. Simoncelli, and A. S. Willsky, "Random cascades on wavelet trees and their use in analyzing and modeling natural images," Appl. Comput. Harmonic Anal., vol. 11, pp. 89-123, 2001.
6. I. J. Schoenberg, "Metric spaces and completely monotone functions," Ann. Math., ser. second, vol. 39, no. 4, pp. 811-841, Oct. 1938.
7. D. F. Andrews and C. L. Mallows, "Scale mixtures of normal distributions," J. Roy. Statist. Soc., vol. 36, pp. 99-102, 1974.
8. I. F. Blake and J. B. Thomas, "On a class of processes arising in linear estimation theory," IEEE Trans. Inf. Theory, vol. IT-14, no. 1, pp. 12-16, Jan. 1968.
9. H. Brehm and W. Stammler, "Description and generation of spherically invariant speech-model signals," Signal Process., vol. 12, pp. 119-141, 1987.
10. P. L. D. León II and D. M. Etter, "Experimental results of subband acoustic echo cancelers under spherically invariant random processes," in Proc. Int. Conf. Acoustics, Speech , and Signal Processing (ICASSP'96), Atlanta, GA, May 1996, pp. 961-964.
11. F. Müller, "On the motion compensated prediction error using true motion fields," in Proc. Int. Conf. Image Processing (ICIP'94 , Austin, TX, Nov. 1994, vol. 3, pp. 781-785.
12. E. Conte and M. Longo, "Characterization of radar clutter as a spherically invariant random process," Proc. Inst. Elec. Eng., Pt. F, Commun., Radar, Signal Processing, vol. 134, pp. 191-197, 1987.
13. M. Rangaswamy, "Spherically invariant random processes for modeling non-Gaussian radar clutter," in Conf. Rec. 27th Asilomar Conf. Signals, Systems, and Computers, Pacific Grove, CA, 1993, vol. 2, pp. 1106-1110.
14. A. Abdi, H. A. Barger, and M. Kaveh, "Signal modeling in wireless fading channels using spherically invariant processes," in Proc. Int. Conf. Acoustics, Speech, and Signal Processing (ICASSP'2000), Istanbul, Turkey, Jun. 2000, vol. 5, pp. 2997-3000.
15. J. K. Romberg, H. Choi, and R. G. Baraniuk, "Bayesian tree-structured image modeling using wavelet-domain hidden Markov models," IEEE Trans. Image Process., vol. 10, no. 7, pp. 1056-1068, Jul. 2001.
16. S. G. Mallat, "A theory for multiresolution signal decomposition: The wavelet representation," IEEE Trans. Pattern Anal. Machine Intell., vol. 11, no. 7, pp. 674-693, Jul. 1989.
17. S. M. LoPresto, K. Ramchandran, and M. T. Orchard, "Image coding based on mixture modeling of wavelet coefficients and a fast estimation-quantization framework," in Proc. Data Compression Conf., Snowbird, UT, Mar. 1997, pp. 221-230.
18. J. Liu and P. Moulin, "Information-theoretic analysis of interscale and intrascale dependencies between image wavelet coefficients," IEEE Trans. Image Process., vol. 10, no. 11, pp. 1647-1658, Nov. 2001.
19. K. Yao, "A representation theorem and its applications to spherically-invariant random processes," IEEE Trans. Inf. Theory, vol. IT-19, no. 5, pp. 600-608, Sep. 1973.
20. H. M. Leung and S. Cambanis, "On the rate distortion functions of spherically invariant vectors and sequences," IEEE Trans. Inf. Theory, vol. IT-24, no. 1, pp. 367-373, Jan. 1978.
21. M. Herbert, "Lattice quantization of spherically invariant speech-model signals," Archiv fúr Elektronik und Übertragungstechnik, vol. AEÜ-45, pp. 235-244, Jul. 1991.
22. F. Müller and F. Geischläger, "Vector quantization of spherically invariant random processes," in Proc. IEEE Int. Symp. Information Theory, Whistler, BC, Canada, Sep. 1995, p. 439.
23. S. G. Chang, B. Yu, and M. Vetterli, "Adaptive wavelet thresholding for image denoising and compression," IEEE Trans. Image Process., vol. 9, no. 9, pp. 1532-1546, Sep. 2000.
24. Y.-P. Tan, D. D. Saur, S. R. Kulkarni, and P. J. Ramadge, "Rapid estimation of camera motion from compressed video with application to video annotation," IEEE Trans. Circuits Syst. Video Technol., vol. 10, no. 1, pp. 133-146, Feb. 2000.
25. J.-W. Nahm and M. J. T. Smith, "Very low bit rate data compression using a quality measure based on target detection performance," Proc. SPIE, vol. 248, pp. 246-255, Jul. 1995.
26. S. R. F. Sims, "Data compression issues in automatic target recognition and the measurement of distortion," Opt. Eng., vol. 36, no. 10, pp. 2671-2674, Oct. 1997.
27. A. Jain, P. Moulin, M. I. Miller, and K. Ramchandran, "Information-theoretic bounds on target recognition performance based on degraded image data," IEEE Trans. Pattern Anal. Machine Intell., vol. 24, no. 9, pp. 1153-1166, Sep. 2002.
28. K. L. Oehler and R. M. Gray, "Combining image compression and classification using vector quantization," IEEE Trans. Pattern Anal. Machine Intell., vol. 17, no. 5, pp. 461-473, May 1995.
29. K. O. Perlmutter, S. M. Perlmutter, R. M. Gray, R. A. Olshen, and K. L. Oehler, "Bayes risk weighted vector quantization with posterior estimation for image compression and classification," IEEE Trans. Image Process., vol. 5, no. 2, pp. 347-360, Feb. 1996.
30. T. Fine, "Optimum mean-squared quantization of a noisy input," IEEE Trans. Inf. Theory, vol. IT-11, no. 2, pp. 293-294, Apr. 1965.
31. J. K. Wolf and J. Ziv, "Transmission of noisy information to a noisy receiver with minimum distortion," IEEE Trans. Inf. Theory, vol. IT-16, no. 4, pp. 406-441, Jul. 1970.
32. Y. Ephraim and R. M. Gray, "A unified approach for encoding clean and noisy sources by means of waveform and autoregressive model vector quantization," IEEE Trans. Inf. Theory, vol. 34, no. 4, pp. 826-834, Jul. 1988.
33. A. Rao, D. Miller, K. Rose, and A. Gersho, "A generalized VQ method for combined compression and estimation," in Proc. Int. Conf. Acoustics, Speech, and Signal Processing (ICASSP'96), Atlanta, GA, May 1996, pp. 2032-2035.
34. H. V. Poor and J. B. Thomas, "Applications of Ali-Silvey distance measures in the design of generalized quantizers," IEEE Trans. Commun., vol. COM-25, no. 9, pp. 893-900, Sep. 1977.
35. G. R. Benitz and J. A. Bucklew, "Asymptotically optimal quantizers for detection of i.i.d. data," IEEE Trans. Inf. Theory, vol. 35, no. 2, pp. 316-325, Mar. 1989.
36. D. Kazakos, "New error bounds and optimum quantization for multisensor distributed signal detection," IEEE Trans. Commun., vol. 40, no. 7, pp. 1144-1151, Jul. 1992.
37. H. V. Poor, "Fine quantization in signal detection and estimation," IEEE Trans. Inf. Theory, vol. 34, no. 5, pp. 960-972, Sep. 1988.
38. M. Goldburg, "Applications of Wavelets to Quantization and Random Process Representations," Ph. D. dissertation, Stanford Univ., Stanford, CA, May 1993.
39. V. K. Goyal, "Theoretical foundations of transform coding," IEEE Signal Process. Mag., vol. 28, no. 5, pp. 9-21, Sep. 2001.
40. R. M. Gray and D. L. Neuhoff, "Quantization," IEEE Trans. Inf. Theory, vol. 44, no. 6, pp. 2325-2383, Oct. 1998.
41. S. Jana and P. Moulin, "Optimal transform coding of Gaussian mixtures for joint classification/reconstruction," in Proc. Data Compression Conf. (DCC'03), Snowbird, UT, Mar. 2003, pp. 313-322.
42. G. Zhou and G. B. Giannakis, "Harmonics in Gaussian multiplicative and additive noise: Cramer-Rao bounds," IEEE Trans. Signal Process., vol. 43, no. 5, pp. 1217-1231, May 1995.
43. T. M. Cover and J. A. Thomas, Elements of Information Theory. New York: Wiley, 1991.
44. P. L. Zador, Topics in the Asymptotic Quantization of Continuous Random Variables 1966, Bell Laboratories Technical Memorandum.
45. V. N. Koshelev, "Quantization with minimal entropy," Probl. Pered. Inform., no. 14, pp. 151-156, 1963.
46. H. Gish and J. Pierce, "Asymptotically efficient quantizing," IEEE Trans. Inf. Theory, vol. IT-14, no. 5, pp. 676-683, Sep. 1968.
47. R. M. Gray, T. Linder, and J. Li, "A Lagrangian formulation of Zador's entropy-constrained quantization theorem," IEEE Trans. Inf. Theory, vol. 48, no. 3, pp. 695-707, Mar. 2002.
48. N. S. Jayant and P. Knoll, Digital Coding of Analog Waveforms. Englewood Cliffs, NJ: Prentice-Hall, 1988.
49. T. Linder, R. Zamir, and K. Zeger, "High-resolution source coding for nondifference distortion measures: Multidimensional companding," IEEE Trans. Inf. Theory, vol. 45, no. 2, pp. 548-561, Mar. 1999.
50. J. W. Dshalalow, Real Analysis: An Introduction to the Theory of Real Functions and Integration. London, U.K.: Chapman & Hall/CRC, 2001.
51. H. Avi-Itzhak and T. Diep, "Arbitrarily tight upper and lower bounds on the Bayesian probability of error," IEEE Trans. Pattern Anal. Machine Intell., vol. 18, no. 1, pp. 89-91, Jan. 1996.
52. H. L. Van, Trees, Detection, Estimation and Modulation Theory, Part I. New York: Wiley, 1968.
53. A. W. Marshall and I. Olkin, Inequalities: Theory of Majorization and Its Applications. San Diego, CA: Academic, 1979.
54. I. S. Gradshteyn and I. M. Ryzhik, Table of Integrals, Series and Products. New York: Academic, 1980.