Data compression and harmonic analysis

IEEE Transactions on Information Theory

Volumn 44, Issue 6, 1998, Pages 2435-2476

  Donoho, David L ^b   Vetterli, Martin ^a,c,d   Devore, R A ^e   Daubechies, Ingrid ^a,f  

^a IEEE   (United States)

^b STANFORD UNIVERSITY   (United States)

^c EPFL   (Switzerland)

^d UNIVERSITY OF CALIFORNIA   (United States)

^e UNIVERSITY OF SOUTH CAROLINA   (United States)

^f PRINCETON UNIVERSITY   (United States)

Author keywords

Besov spaces; Block coding; Cosine packets; Fourier transform; Gabor transform; Gaussian process; Karhunen lo ve transform; Littlewood paley theory; N widths; Nongaussian process; Rate distortion; Sampling theorem; Tentropy

Indexed keywords

ENCODING (SYMBOLS); FOURIER TRANSFORMS; HARMONIC ANALYSIS; IMAGE COMPRESSION; WAVELET TRANSFORMS;

BESOV SPACES; BLOCK CODING; GABOR TRANSFORM; GAUSSIAN PROCESS; KARHUNEN-LOEVE TRANSFORM; LITTLEWOOD-PALEY THEORY; NON GAUSSIAN PROCESS; RATE DISTORTION; SAMPLING THEOREM; SCALAR QUANTIZATION;

DATA COMPRESSION;

EID: 0032182894     PISSN: 00189448     EISSN: None     Source Type: Journal    
DOI: 10.1109/18.720544     Document Type: Article

References (106)

1. N. Ahmed, T. Natarajan, and K. R. Rao, "Discrete cosine transform," IEEE Trans. Comput., vol. C-23, pp. 88-93, Jan. 1974.
2. E. W. Aslaksen and J. R. Klauder, "Unitary representations of the affine group," J. Math. Phys., vol. 9, pp. 206-211, 1968.
3. P. Auscher, G. Weiss, and G. Wickerhauser, "Local sine and cosine bases of Coifman and Meyer and the construction of smooth wavelets," in Wavelets: A Tutorial in Theory and Applications, C. K. Chui, Ed. New York: Academic, 1992.
4. H. B. Barlow, "Possible principles underlying the transformation of sensory messages," in Sensory Communication, W. A. Rosenbluth, Ed. Cambrige, MA: MIT Press, 1961, pp. 217-234.
5. T. Berger, Rate Distortion Theory: A Mathematical Basis for Data Compression. Englewood Cliffs, NJ: Prentice Hall, 1971.
6. L. Birgé, "Approximation dans les espaces métriques et théorie de l'estimation," (in French), ("Approximation in metric spaces and the theory of estimation"), Z. Wahrsch. Verw. Gebiete, vol. 65, no. 2, pp. 181-237, 1983.
7. L. Birgé and P. Massart, "An adaptive compression algorithm in Besov spaces," Constr. Approx., to be published.
8. M. S. Birman and M. Z. Solomjak, "Piecewise-polynomial approximations of functions of the classes Wp"" Mat. Sbornik, vol. 73, pp. 295-317, 1967.
9. A. P. Calderön, "Intermediate spaces and interpolation, the complex method," Studio Math., vol. 24, pp. 113-190, 1964.
10. B. Carl and I. Stephani, Entropy, Compactness, and the Approximation of Operators. Cambridge, U.K.: Cambridge Univ. Press, 1990.
11. L. Carleson, "The work of Charles Fefferman," in Proc. Int. Congr. Mathematics (Helsinki, Finland, 1978). Helsinki, Finland: Acad. Sei. Fennica, 1980, pp. 53-56.
12. A. Cohen and J. P. D'Ales, "Non-linear approximation of random functions," S1AM J. Appl. Math., vol. 57, pp. 518-540, 1997.
13. A. Cohen, I. Daubechies, and P. Vial, "Orthonormal wavelets for the interval," Appl. Comput. Harmonic Anal., vol. 1, no. 1, 1993.
14. A. Cohen, I. Daubechies, O. Guleryuz, and M. Orchard, "On the importance of combining wavelet-based nonlinear approximation in coding strategies," unpublished manuscript, 1997.
15. A. Cohen, W. Dahmen, I. Daubechies, and R. A. DeVore, "Tree, Approximation and Encoding," preprint, 1998..
16. R. R. Coifman, "A real-variable characterization of Hp," Studio Math., vol. 51, pp. 269-274, 1974.
17. R. R. Coifman and M. V. Wickerhauser, "Entropy-based algorithms for best basis selection," vol. 38, no. 2, pp. 713-718, Mar. 1992.
18. R. R. Coifman and Y. Meyer, "Remarques sur l'analyze de Fourier à fenêtre, C. R. Acad Sei. Paris, vol. 312, pp. 259-261, 1991.
19. T. M. Cover and J. A. Thomas, Elements of Information Theory. New York: Wiley, 1991.
20. I. Daubechies, "Orthonormal bases of compactly supported wavelets," Commun. Pure Appl. Math., vol. 41, pp. 909-996, 1988.
21. "The wavelet transform, time-frequency localization, and signal analysis," IEEE Trans. Inform. Theory, vol. 36, pp. 961-1005, 1990.
22. I. Daubechies, S. Jaffard, and J. L. Journé, "A simple Wilson orthonormal basis with exponential decay," SIAM J. Math. Anal., vol. 24, pp. 520-527, 1990.
23. I. Daubechies, Ten Lectures on Wavelets. Philadelphia, PA: SIAM, 1992.
24. G. David and S. Semmes, Analysis of and on Uniformly Rectifiable Sets. (Amer. Math. Soc., Mathematical Surveys and Monographs, vol. 38), 1993.
25. K. DeLeeuw, J. P. Kahane, and Y. Katznelson, "Sur les coefficients de Fourier des fonctions continues," C. R. Acad. Sei. Paris, vol. 285, pp. 1001-1003, 1978.
26. R. A. DeVore, "Nonlinear approximation," Acta Numer., vol. 7, pp. 51-150, 1998.
27. R. A. DeVore, B. Jawerth, and B. Lucier, "Image compression through wavelet transform coding," IEEE Trans. Inform. Theory, vol. 38, pp. 719-746, 1992.
28. R. DeVore and X. M. Yu, "Degree of adaptive approximation," Math. Comput., vol. 55, pp. 625-635, 1990.
29. R. DeVore, B. Jawerth, and V. A. Popov, "Compression of wavelet decompositions," Amer. J. Math., vol. 114, pp. 737-785, 1992.
30. R. A. DeVore and G. G. Lorentz, Constructive Approximation. New York: Springer, 1993.
31. D. L. Donoho, "Unconditional bases are optimal bases for data compression and statistical estimation," Appl. Comput. Harmonic Anal., vol. 1, no. 1. pp. 100-105, 1993. [
32. "Unconditional bases and bit-level compression," Appl. Comput. Harmonic Anal, vol. 3, no. 4, pp. 388-392, 1996.
33. [ 33] "CART and best-ortho-basis: A connection," A. Statist., vol. 25, no. 5, pp. 1870-1911, 1997.
34. "Counting bits with Kolmogorov and Shannon," manuscript, 1998.
35. R. M. Dudley, "The sizes of compact subsets of Hubert spaces and continuity of Gaussian processes," J. Fund. Anal., vol. 1, pp. 290-330, 1967.
36. D. E. Edmunds and H. Triebel, Function Spaces, Entropy Numbers, and Differential Operators. Cambridge, U.K.: Cambridge Univ. Press, 1996.
37. D. J. Field, "The analysis of natural images and the response properties of cortical cells," J. Opt. Soc. Amer., 1987.
38. "What is the goal of sensory coding?" Neural Comput., vol. 6, no. 4, pp. 559-601, 1994.
39. G. Folland, Harmonic Analysis in Phase Space. Princeton, NJ: Princeton Univ. Press, 1989.
40. H. G. Feichtinger, "Atomic characterizations of modulation spaces through Gabor-type representations," Rocky Mount. J. Math., vol. 19, pp. 113-126, 1989.
41. C. Fefferman, "The uncertainty principle," Bull. Amer. Math. Soc., vol. 9, pp. 129-206, 1983.
42. M. Frazier and B. Jawerth, "The transform and applications to distribution spaces," in Function Spaces and Applications (Lecture Notes in Mathematics, vol. 1302), M. Cwikel, et al., Eds. Berlin, Germany: Springer-Verlag, 1988, pp. 223-246.
43. M. Frazier, B. Jawerth, and G. Weiss, Littlewood-Paley Theory and the Study of Function Spaces (CBMS Reg. Conf. Ser. in Mathematics, no. 79). Amer. Math. Soc., 1991.
44. D. Gabor, "Theory of communication," J. Inst. Elect. Eng., vol. 93, pp. 429-457, 1946.
45. J. Garnett, Bounded Analytic Functions. New York: Academic, 1981.
46. A. Gersho and R. M. Gray, Vector Quantization and Signal Compression. Boston, MA: Kluwer, 1992.
47. R. Godement, "Sur les relations d'orthogonalité de V. Bargmann," C. R. Acad. Sei. Paris, vol. 255, pp. 521-523, 657-659, 1947.
48. R. M. Gray and D. L. Neuhoff, "Quantization," IEEE Trans. Inform. Theory, this issue, pp. 2325-2383.
49. V. Goyal and M. Vetterli "Computation-distortion characteristics of block transform coding," in ICASSP-97 (Munich, Germay, Apr. 1997), vol. 4, pp. 2729-2732.
50. K. Gröchenig and S. Samarah, "Nonlinear approximation with local Fourier bases," unpublished manuscript, 1998.
51. K. Gröchenig and D. Walnut, "Wilson bases are unconditional bases for modulation spaces," unpublished manuscript, 1998.
52. A. Grossmann and J. Morlet, "Decomposition of Hardy functions into square-integrable wavelets of constant shape," SIAM J. Appl. Math., vol. 15, pp. 723-736, 1984.
53. G. H. Hardy, A Mathematician's Apology. Cambridge, U.K.: Cambridge Univ. Press, 1940.
54. R. Howe, "On the role of the Heisenberg group in hramonic analysis." Bull Amer. Math. Soc., vol. 3, pp. 821-843, 1980.
55. J. J. Y. Huang and P. M. Schultheiss, "Block quantization of correlated Gaussian random variables," IEEE Trans. Commun., vol. CUM-11, pp. 289-296, Sept. 1963.
56. P. Jones, "Rectifiable sets and the travelling salesman problem," Inven-tiones Mathematicae, vol. 102, pp. 1-15, 1990.
57. "Lennart Carleson's work in analysis," in Festschrift in Honor ofLennart Carleson and Yngve Domar, (Acta Universitatis Upsaliensis, vol. 58). Stockholm, Sweden: Almquist and Wiksell Int., 1994.
58. J. P. Kahane and P. G. Lemarié-Rieusset, Fourier Series and Wavelets. Luxemburg: Gordon and Breach, 1995.
59. J. Klauder and B. Skagerstam, Coherent States, Applications in Physics and Mathematical Physics. Singapore: World Scientific, 1985.
60. A. N. Kolmogorov, "On the Shannon theory of information transmission in the case of continuous signals," IRE Trans. Inform. Theory, vol. IT-2, pp. 102-108, 1956.
61. "Some fundamental problems in the approximate and exact representation of functions of one or several variables," in Proc III. Math Congress USSR, vol. 2. Moscow, USSR: MCU Press, 1956, pp. 28-29. Reprinted in Komogorov's
62. Selected Works, vol. I.
63. A. N. Kolmogorov and V. M. Tikhomirov, e-entropy and e-capacity. Usp. Mat. Nauk, vol. 14, pp. 3-86, 1959. (English transi.
64. Amer. Math. Soc. Transi., ser. 2, vol 17, pp. 277-364.)
65. H. P. Kramer and M. V. Mathews, "A linear coding for transmitting a set of correlated signals," IRE Trans. Inform. Theory, vol. IT-23, pp. 41-46, Sept. 1956.
66. P. G. Lemarié and Y. Meyer, "Ondelettes et Bases Hilbertiennes," Revista Mat. Iberomamericana, vol. 2, pp. 1-18, 1986.
67. S. LoPresto, K. Ramchandran, and M. T. Orchard, "Image coding based on mixture modeling of wavelet coefficients and a fast estimationquantization framework," in Data Compression Conf. '97 (Snowbird, UT, 1997), pp. 221-230.
68. G. G. Lorentz, "Metric entropy and approximation," Bull. Amer. Math. Soc., vol. 72, pp. 903-937, Nov. 1966.
69. L. Le Cam, "Convergence of estimates under dimensionality restrictions," Ann. Statist., vol. 1, pp. 38-53, 1973.
70. A. S. Lewis and G. Knowles, "Image compression using the 2-D wavelet transform," IEEE Trans. Image Processing, vol. 1, pp. 244-250, Apr. 1992.
71. S. Mallat, "A theory for multiresolution signal decomposition: The wavelet representation," IEEE Trans. Pattern Anal. Machine Intel!., vol. 11, pp. 674-693, July 1989.
72. S. Mallat and F. Falzon, "Analysis of low bit rate image transform coding," IEEE Trans. Signal Processing, vol. 46, pp. 1027-1042, Apr. 1998.
73. H. S. Malvar and D. H. Staelin, "The LOT: Transform coding without blocking effects," IEEE Trans. Acoust., Speech, Signal Processing, vol. 37, no. 4, pp. 553-559, 1989.
74. H. S. Malvar, "Extended lapped transforms: Properties, applications, and fast algorithms," IEEE Trans. Signal Processing, vol. 40, pp. 2703-2714, 1992.
75. Y. Meyer, "Review of 'An Introduction to Wavelets' and 'Ten Lectures on Wavelets'," Bull. Amer. Math. Soc., vol. 28, pp. 350-359, 1993.
76. Ondelettes et Operateurs. Paris, France: Hermann, 1990.
77. "Wavelets and applications" (Lecture at CIRM Luminy meeting, Luminy, France, Mar. 1992).
78. F. Mintzer, "Filters for distortion-free two-band multirate filter banks," IEEE Trans. Acoust., Speech, Signal Processing, vol. 33, pp. 626-630, June 1985.
79. A. Pinkus, n-Widths in Approximation Theory. New York: Springer, 1983.
80. G. Pisier, The Volume of Convex Bodies and Banach Space Geometry. Cambridge, U.K.: Cambridge Univ.Press, 1989.
81. J. O. Ramsay and B. W. Silverman, Functional Data Analysis. New York: Springer, 1997.
82. K. Ramchandran and M. Vetterli, "Best wavelet packet bases in a ratedistortion sense," IEEE Trans. Image Processing, vol. 2, pp. 160-175, Apr. 1993.
83. D. L. Ruderman, "Origins of scaling in natural images" VISION RES., vol. 37, no. 23, pp. 3385-3398, Dec. 1997.
84. A. Said and W. A. Pearlman, "A new, fast, and efficient image codec based on set partitioning in hierachical trees," IEEE Trans. Circuits Syst. Video Technol, vol. 6, pp. 243-250, June 1996.
85. C. E. Shannon, "A mathematical theory of communication," Bell Syst. Tech. J., vol. 27, pp. 379-23, 623-656, 1948.
86. "Communication in the presence of noise," P roc. IRE, vol. 37, pp. 10-21, 1949.
87. "The bandwagon" (1956 ), in Claude Elwood Shannon: Collected Papers, N. J. A. Sloane and A. D. Wyner, Eds. Piscataway, NJ: IEEE Press, 1993.
88. J. M. Shapiro, "Embedded image coding using zerotrees of wavelet coefficients," IEEE Trans. Signal Processing, vol. 41, pp. 3445-3462, Dec. 1993.
89. E. Simoncelli, "Statistical models for images: Compression, restoration and synthesis,"presented at the IEEE 31st Asilomar Conf. Signals, Systems, and Computers, Pacific Grove, CA, 1997.
90. M. J. T. Smith and T. P. Barnwell, III, "Exact reconstruction for tree-structured subband coders," IEEE Trans. Acoust., Speech, Signal Processing, vol. 35, pp. 431-441, 1986.
91. E. Stein, Singular Integrals and Differentiability Properties of Functions. Princeton, NJ: Princeton Univ. Press, 1970.
92. Harmonic Analysis: Real Variable Methods, Orthogonality, and Oscillatory Integrals. Princeton, NJ: Princeton Univ. Press, 1993.
93. J. O. Stromberg, Festschrift in Honor of Antoni Zygmund. Monterey, CA: Wadsworth, 1982.
94. "Maximal functions, Hardy Spaces, BMO, and Wavelets, from my point of view," in Festschrift in Honor of Lennart Carleson and Yngve Domar, (Acta Universitatis Upsaliensis vol. 58). Stockholm, Sweden: Almquist and Wiksell Int., 1994.
95. V. M. Tikhomirov, "Widths and entropy," Usp. Mat. Nauk, vol. 38, pp. 91-99, 1983 (in Russian).
96. English transi, in Russian Math Surveys, vol. 38, pp. 101-111.
97. "Commentary: e-entropy and e-capacity," in A. N. Kolmogorov: Selected Works. III. Information Theory and Theory of Algorithms, A. N. Shiryaev, Ed. Boston, MA: Kluwer, 1992.
98. B. Torrésani, "Time-frequency representations: Wavelet packets and optimal decomposition," Ann. l'Institut Henri Poincare (Physique Théorique), vol. 56, no. 2, pp. 215-34, 1992.
99. H. Triebel, Theory of Function Spaces. Basel, Switzerland: Birkhauser, 1983.
100. P. P. Vaidyanathan, "Quadrature mirror filter banks, M-band extensions and perfect reconstruction techniques," IEEE ASSP Mag., vol. 4, pp. 4-20, July 1987.
101. M. Vetterli, "Multi-dimensional subband coding: Some theory and algorithms," Signal Processing, vol. 6, no. 2, pp. 97-112, Apr. 1984.
102. "Filter banks allowing perfect reconstruction," Signal Processing, vol. 10, no. 3, pp. 219-244, Apr. 1986.
103. M. Vetterli and J. Kovacevic, Wavelets and Subband Coding. Engle-wood Cliffs, NJ: Prentice-Hall, 1995.
104. P. H. Westerink, J. Biemond, D. E. Boekee, and J. W. Woods, "Subband coding of images using vector quantization," IEEE Trans. Commun., vol. 36, pp. 713-719, June 1988.
105. J. W. Woods and S. D. O'Neil, "Sub-band coding of images," IEEE Trans. Acoust., Speech Siganl Processing, vol. 34, pp. 1278-1288, Oct. 1986.
106. A. Zygmund, Trigonometric Series, vols. I & II. Cambridge, U.K.: Cambridge Univ. Press, 1959.