Dictionaries for sparse representation modeling

Proceedings of the IEEE

Volumn 98, Issue 6, 2010, Pages 1045-1057

  Rubinstein, Ron ^a   Bruckstein, Alfred M ^a   Elad, Michael ^a  

^a TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

Author keywords

Dictionary learning; Harmonic analysis; Signal approximation; Signal representation; Sparse coding; Sparse representation

Indexed keywords

ELECTRICAL ENGINEERING; HARMONIC ANALYSIS;

DICTIONARY LEARNING; SIGNAL APPROXIMATION; SIGNAL REPRESENTATIONS; SPARSE CODING; SPARSE REPRESENTATION;

LEARNING SYSTEMS;

EID: 77952714665     PISSN: 00189219     EISSN: None     Source Type: Journal    
DOI: 10.1109/JPROC.2010.2040551     Document Type: Article

References (91)

1. P. J. Huber, Robust Statistics. New York: Wiley, 1981.
2. J. W. Cooley and J. W. Tukey, "An algorithm for the machine calculation of complex Fourier series," Math. Comput., vol.19, pp. 297-301, 1965.
3. D. J. Field, "What is the goal of sensory coding?" Neural Comput., vol. 6, no. 4, pp. 559-601, 1994.
4. A. K. Jain, Fundamentals of Digital Image Processing. Prentice-Hall, 1989.
5. S. Mallat, A Wavelet Tour of Signal Processing, 3rd ed. Academic Press, 2009.
6. I. T. Jolliffe, Principal Component Analysis, 2nd ed. New York: Springer, 2002.
7. R. A. DeVore, "Nonlinear approximation," Acta Numerica, vol.7, pp. 51-150, 1998.
8. J. B. Allen and L. R. Rabiner, "A unified approach to short-time Fourier analysis and synthesis," Proc. IEEE, vol.65, no.11, pp. 1558-1564, 1977. (Pubitemid 8601837)
9. W. B. Pennebaker and J. L. Mitchell, JPEG Still Image Data Compression Standard. New York: Springer, 1993.
10. D. Gabor, "Theory of communication," J. Inst. Electr. Eng., vol.93, no.26, pp. 429-457, 1946.
11. M. J. Bastiaans, "Gabor's expansion of a signal into Gaussian elementary signals," Proc. IEEE, vol.68, no.4, pp. 538-539, 1980. (Pubitemid 10468826)
12. A. Janssen, "Gabor representation of generalized functions," J. Math. Anal. and Applic., vol.83, no.2, pp. 377-394, 1981.
13. I. Daubechies, A. Grossmann, and Y. Meyer, "Painless nonorthogonal expansions," J. Math. Phys., vol.27, p. 1271, 1986.
14. I. Daubechies, "The wavelet transform, time-freuency localization and signal analysis," IEEE Trans. Inf. Theory, vol.36, no.5, pp. 961-1005, 1990. (Pubitemid 20738359)
15. H. G. Feichtinger and K. Gröchenig, "Banach spaces related to integrable group representations and their atomic decompositions, part I," J. Funct. Anal., vol.86, no.2, pp. 307-340, 1989.
16. H. G. Feichtinger and K. Gröchenig, "Gabor wavelets and the Heisenberg group: Gabor expansions and short time Fourier transform from the group theoretical point of view," Wavelets: A Tutorial in Theory and Applications, C. K. Chiu, Ed., pp. 359-397, 1992.
17. H. G. Feichtinger and K. Gröchenig, "Gabor frames and time-frequency analysis of distributions," J. Funct. Anal., vol.146, no.2, pp. 464-495, 1997. (Pubitemid 127168463)
18. J. Wexler and S. Raz, "Discrete gabor expansions," Signal Process., vol.21, no.3, pp. 207-221, 1990.
19. S. Qian and D. Chen, "Discrete gabor transform," IEEE Trans. Signal Process., vol.41, no.7, pp. 2429-2438, 1993.
20. J. G. Daugman, "Two-dimensional spectral analysis of cortical receptive field profiles," Vision Research, vol.20, no.10, pp. 847-856, 1980.
21. J. G. Daugman, "Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters," J. Opt. Soc. Amer. A, vol.2, no.7, pp. 1160-1169, 1985.
22. J. G. Daugman, "Complete discrete 2-D Gabor transforms by neural networks for image analysis and compression," IEEE Trans. Acoust. Speech Signal Process., vol.36, no.7, pp. 1169-1179, 1988.
23. M. Porat and Y. Y. Zeevi, "The generalized Gabor scheme of image representation in biological and machine vision," IEEE Trans. Patt. Anal. Machine Intell., vol.10, no.4, pp. 452-468, 1988. (Pubitemid 18665917)
24. P. Burt and E. Adelson, "The Laplacian pyramid as a compact image code," IEEE Trans. Commun., vol.31, no.4, pp. 532-540, 1983. (Pubitemid 13521088)
25. I. Daubechies, Ten Lectures on Wavelets. Society for Industrial Mathematics, 1992.
26. Y. Meyer and D. Salinger, Wavelets and Operators. Cambridge University Press, 1995.
27. J. Morlet and A. Grossman, "Decomposition of Hardy functions into square integrable wavelets of constant shape," SIAM J. Math. Anal., vol.15, pp. 723-736, 1984.
28. Y. Meyer, "Principe d'incertitude, bases hilbertiennes et alg̀ebres d'oṕerateurs," S éminaire Bourbaki, no. 662, 1985-1986.
29. P. G. Lemarie and Y. Meyer, "Ondelettes et bases hilbertiennes," Rev. Mat. Iberoamericana, vol.2, no.1-2, pp. 1-18, 1986.
30. I. Daubechies, "Orthonormal bases of compactly supported wavelets," Comm. Pure Appl. Math., vol.41, no.7, pp. 909-996, 1988.
31. S. G. Mallat, "A theory for multiresolution signal decomposition: The wavelet representation," IEEE Trans. Patt. Anal. Machine Intell., vol.11, no.7, pp. 674-693, 1989.
32. S. Mallat and W. L. Hwang, "Singularity detection and processing with wavelets," IEEE Trans. Inf. Theo., vol.38, no.2, pp. 617-643, 1992.
33. S. Mallat and S. Zhong, "Characterization of signals from multiscale edges," IEEE Trans. Patt. Anal. Machine Intell., vol.14, no.7, pp. 710-732, 1992.
34. B. A. Olshausen and D. J. Field, "Emergence of simple-cell receptive field properties by learning a sparse code for natural images," Nature, vol.381, no.6583, pp. 607-609, 1996.
35. R. R. Coifman, Y. Meyer, and V. Wickerhauser, "Wavelet analysis and signal processing," Wavelets and Their Applications, pp. 153-178, 1992.
36. E. P. Simoncelli, W. T. Freeman, E. H. Adelson, and D. J. Heeger, "Shiftable multiscale transforms," IEEE Trans. Inf. Theory, vol.38, no.2, pt. 2, pp. 587-607, 1992.
37. G. Beylkin, "On the representation of operators in bases of compactly supported wavelets," SIAM J. Numerical Anal., pp. 1716-1740, 1992. (Pubitemid 23606068)
38. G. P. Nason and B. W. Silverman, "The stationary wavelet transform and some statistical applications," Lecture Notes in Statistics 103: Wavelets and Statistics, A. Antoniadis and G. Oppenheim, Eds., pp. 281-299, 1995.
39. R. R. Coifman and D. L. Donoho, "Translation-invariant de-noising," Lecture Notes in Statistics 103: Wavelets and Statistics, A. Antoniadis and G. Oppenheim, Eds., pp. 125-150, 1995.
40. S. Mallat and Z. Zhang, "Matching pursuits with time-frequency dictionaries," IEEE Trans. Signal Process., vol.41, no.12, pp. 3397-3415, 1993.
41. S. S. Chen, D. L. Donoho, and M. A. Saunders, "Atomic Decomposition by Basis Pursuit, Stanford, 1995, Technical ReportVStatistics.
42. A. M. Bruckstein, D. L. Donoho, and M. Elad, "From sparse solutions of systems of equations to sparse modeling of signals and images," SIAM Review, vol.51, no.1, pp. 34-81, 2009.
43. E. J. Cand̀es and D. L. Donoho, "CurveletsVA surprisingly effective nonadaptive representation for objects with edges," Curves and Surfaces, 1999.
44. D. L. Donoho, "Wedgelets: Nearly minimax estimation of edges," Annals of Statistics, vol.27, no.3, pp. 859-897, 1999.
45. M. B. Wakin, J. K. Romberg, H. Choi, and R. G. Baraniuk, "Wavelet-domain approximation and compression of piecewise smooth images," IEEE Trans. Image Process., vol.15, no.5, pp. 1071-1087, 2006.
46. R. M. Willett and R. D. Nowak, "Platelets: A multiscale approach for recovering edges and surfaces in photon-limited medical imaging," IEEE Trans. Med. Imaging, vol.22, no.3, pp. 332-350, 2003.
47. V. Chandrasekaran, M. B. Wakin, D. Baron, and R. G. Baraniuk, "Representation and compression of multi-dimensional piecewise functions using surflets," IEEE Trans. Inf. Theory, vol.55, no.1, pp. 374-400, 2009.
48. E. J. Cand̀es and D. L. Donoho, "Ridgelets: A key to higher-dimensional intermittency?" Philosophical Transactions A: Mathematical, Physical and Engineering Sciences, vol. 357, no. 1760, pp. 2495-2509, 1999.
49. J. L. Starck, E. J. Cand̀es, and D. L. Donoho, "The curvelet transform for image denoising," IEEE Trans. Image Process., vol.11, no.6, pp. 670-684, 2002.
50. M. S. Lewicki and T. J. Sejnowski, "Learning overcomplete representations," Neural Comput., vol.12, no.2, pp. 337-365, 2000.
51. K. Kreutz-Delgado, J. F. Murray, B. D. Rao, K. Engan, T. W. Lee, and T. J. Sejnowski, "Dictionary learning algorithms for sparse representation," Neural Comput., vol. 15, no. 2, pp. 349-396, 2003. (Pubitemid 37049825)
52. I. Daubechies, M. Defrise, and C. De Mol, "An iterative thresholding algorithm for linear inverse problems with a sparsity constraint," Comm. Pure Appl. Math., vol.57, no.11, pp. 1413-1457.
53. E. J. Cand̀es, L. Demanet, D. L. Donoho, and L. Ying, "Fast discrete curvelet transforms," Multiscale Modeling & Simulation, vol.5, pp. 861-899, 2006. (Pubitemid 47381814)
54. L. Ying, L. Demanet, and E. J. Cand̀es, "3D discrete curvelet transform," in Proc. SPIE: Wavelets XI, 2005, vol.5914, pp. 351-361. (Pubitemid 43105318)
55. E. J. Cand̀es and D. L. Donoho, "Continuous curvelet transform: I. Resolution of the wavefront set," Appl. Comput. Harmon. Anal., vol.19, no.2, pp. 162-197, 2005. (Pubitemid 41214741)
56. A. Woiselle, J. L. Starck, M. J. Fadili, "New 3D Data Representations: Applications in Astrophysics," to be published.
57. M. N. Do and M. Vetterli, "Contourlets: A new directional multiresolution image representation," in Conf. Rec. 36th Asilomar Conf. Signals Syst. Comput., 2002, vol.1, pp. 497-501. (Pubitemid 35438363)
58. M. N. Do and M. Vetterli, "The contourlet transform: An efficient directional multiresolution image representation," IEEE Trans. Image Process., vol.14, no.12, pp. 2091-2106, 2005. (Pubitemid 41781516)
59. Y. Lu and M. N. Do, "A new contourlet transform with sharp frequency localization," in Proc. IEEE Int. Conf. Image Process., 2006, pp. 1629-1632.
60. Y. M. Lu and M. N. Do, "Multidimensional directional filter banks and surfacelets," IEEE Trans. Image Process., vol.16, no.4, pp. 918-931, 2007. (Pubitemid 46542383)
61. A. L. da Cunha, J. Zhou, and M. N. Do, "The nonsubsampled contourlet transform: Theory, design, and applications," IEEE Trans. Image Process., vol.15, no.10, pp. 3089-3101, 2006. (Pubitemid 44473649)
62. R. Eslami and H. Radha, "Translation-invariant contourlet transform and its application to image denoising," IEEE Trans. Image Process., vol.15, no.11, pp. 3362-3374, 2006. (Pubitemid 44615133)
63. E. LePennec and S. Mallat, "Sparse geometric image representations with bandelets," IEEE Trans. Image Process., vol.14, no.4, pp. 423-438, 2005. (Pubitemid 40552380)
64. G. Peyŕe and S. Mallat, "Surface compression with geometric bandelets," ACM Trans. Graph. (Proc. SIGGRAPH 05), vol.24, pp. 601-608, 2005.
65. N. Kingsbury, "Complex wavelets for shift invariant analysis and filtering of signals," Appl. Comput. Harmon. Anal., vol.10, no.3, pp. 234-253, 2001. (Pubitemid 33675919)
66. I. W. Selesnick, R. G. Baraniuk, and N. C. Kingsbury, "The dual-tree complex wavelet transform," IEEE Signal Process Mag., vol.22, no.6, pp. 123-151, 2005.
67. D. Labate, W. Lim, G. Kutyniok, and G. Weiss, "Sparse multidimensional representation using shearlets," in Proc. SPIE: Wavelets XI, 2005, vol.5914, pp. 254-262. (Pubitemid 43105309)
68. G. Kutyniok and D. Labate, "Resolution of the wavefront set using continuous shearlets," Trans. Amer. Math. Soc., vol.361, pp. 2719-2754, 2009.
69. G. Easley, D. Labate, and W. Lim, "Sparse directional image representations using the discrete shearlet transform," Appl. Comput. Harmon. Anal., vol.25, pp. 25-46, 2008. (Pubitemid 351713737)
70. V. Velisavljevic, B. Beferull-Lozano, M. Vetterli, and P. L. Dragotti, "Directionlets: Anisotropic multidirectional representation with separable filtering," IEEE Trans. Image Process., vol. 15, no. 7, pp. 1916-1933, 2006. (Pubitemid 43994851)
71. S. Mallat, "Geometrical grouplets," Appl. Comput. Harmon. Anal., vol.26, no.2, pp. 161-180, 2009.
72. Y. C. Pati, R. Rezaiifar, and P. S. Krishnaprasad, "Orthogonal matching pursuit: Recursive function approximation with applications to wavelet decomposition," in Conf. Rec. 27th Asilomar Conf. Signals, Syst. Comput., 1993, pp. 40-44.
73. I. F. Gorodnitsky and B. D. Rao, "Sparse signal reconstruction from limited data using FOCUSS: A re-weighted minimum norm algorithm," IEEE Trans. Signal Process., vol.45, no.3, pp. 600-616, 1997.
74. D. L. Donoho, Y. Tsaig, I. Drori, and J. L. Starck, Sparse Solution of Underdetermined Linear Equations by Stagewise Orthogonal Matching Pursuit, 2007, submitted for publication.
75. D. Needell and J. A. Tropp, "CoSaMP: Iterative signal recovery from incomplete and inaccurate samples," Appl. Comput. Harmon. Anal., vol.26, no.3, pp. 301-321, 2009.
76. J. Mairal, F. Bach, J. Ponce, and G. Sapiro, "Online learning for matrix factorization and sparse coding," J. Mach. Learn. Res., to be published.
77. K. Skretting and K. Engan, "Recursive least squares dictionary learning algorithm," IEEE Trans. Signal Process., to be published.
78. K. Engan, S. O. Aase, and J. Hakon Husoy, "Method of optimal directions for frame design," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., 1999, vol.5, pp. 2443-2446.
79. K. Engan, B. D. Rao, and K. Kreutz-Delgado, "Frame design using FOCUSS with method of optimal directions (MOD)," in Proc. Norwegian Signal Process. Symp., 1999, pp. 65-69.
80. S. Lesage, R. Gribonval, F. Bimbot, and L. Benaroya, "Learning unions of orthonormal bases with thresholded singular value decomposition," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., 2005, vol.5, pp. 293-296. (Pubitemid 43762401)
81. S. Sardy, A. G. Bruce, and P. Tseng, "Block coordinate relaxation methods for nonparametric wavelet denoising," J. Comput. Graph. Stat., vol.9, no.2, pp. 361-379, 2000.
82. R. Vidal, Y. Ma, and S. Sastry, "Generalized principal component analysis (GPCA)," IEEE Trans. Pattern Anal. Mach. Intell., vol.27, no.12, pp. 1945-1959, 2005. (Pubitemid 43051391)
83. M. Aharon, M. Elad, and A. M. Bruckstein, "The K-SVD: An algorithm for designing of overcomplete dictionaries for sparse representation," IEEE Trans. Signal Process., vol.54, no.11, pp. 4311-4322, 2006. (Pubitemid 44637761)
84. T. Blumensath and M. Davies, "Sparse and shift-invariant representations of music," IEEE Trans. Speech Audio Process., vol.14, no.1, p. 50, 2006. (Pubitemid 43863452)
85. P. Jost, P. Vandergheynst, S. Lesage, and R. Gribonval, "MoTIF: An efficient algorithm for learning translation invariant dictionaries," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process., 2006, vol.5. (Pubitemid 46501165)
86. K. Engan, K. Skretting, and J. H. Husøy, "Family of iterative LS-based dictionary learning algorithms, ILS-DLA, for sparse signal representation," Digital Signal Process., vol.17, no.1, pp. 32-49, 2007. (Pubitemid 44828547)
87. M. Aharon and M. Elad, "Sparse and redundant modeling of image content using an image-signature-dictionary," SIAM J. Imaging Sci., vol.1, no.3, pp. 228-247, 2008.
88. P. Sallee and B. A. Olshausen, "Learning sparse multiscale image representations," Adv. Neural Inf. Process. Syst., vol.15, pp. 1327-1334, 2003.
89. J. Mairal, G. Sapiro, and M. Elad, "Learning multiscale sparse representations for image and video restoration," SIAM Multiscale Modeling and Simulation, vol.7, no.1, pp. 214-241, 2008.
90. R. Rubinstein, M. Zibulevsky, and M. Elad, "Learning sparse dictionaries for sparse signal representation," IEEE Trans. Signal Process., to be published.
91. M. Yaghoobi, L. Daudet, and M. E. Davies, "Parametric dictionary design for sparse coding," IEEE Trans. Signal Process., to be published.