Double sparsity: Learning sparse dictionaries for sparse signal approximation

IEEE Transactions on Signal Processing

Volumn 58, Issue 3 PART 2, 2010, Pages 1553-1564

  Rubinstein, Ron ^a   Zibulevsky, Michael ^a   Elad, Michael ^a  

^a TECHNION ISRAEL INSTITUTE OF TECHNOLOGY   (Israel)

Author keywords

Computed tomography; Dictionary learning; K SVD; Signal denoising; Sparse coding; Sparse representation

Indexed keywords

COMPUTED TOMOGRAPHY; DICTIONARY LEARNING; K-SVD; SIGNAL DENOISING; SPARSE CODING; SPARSE REPRESENTATION;

ALGORITHMS; COMPUTERIZED TOMOGRAPHY; MATHEMATICAL OPERATORS; NOISE POLLUTION CONTROL; SIGNAL PROCESSING; TOMOGRAPHY;

THREE DIMENSIONAL;

EID: 78349258863     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2009.2036477     Document Type: Article

References (45)

1. S. S. Chen, D. L. Donoho, and M. A. Saunders, "Atomic decomposition by basis pursuit," SIAM Rev., vol. 43, no. 1, pp. 129-159, 2001. (Pubitemid 32406896)
2. S. Mallat and Z. Zhang, "Matching pursuits with time-frequency dictionaries," IEEE Trans. Signal Process., vol. 41, no. 12, pp. 3397-3415, Dec. 1993.
3. A. M. Bruckstein, D. L. Donoho, and M. Elad, "From sparse solutions of systems of equations to sparse modeling of signals and images," SIAM Rev., vol. 51, no. 1, pp. 34-81, 2009.
4. M. Elad, J. L. Starck, P. Querre, and D. L. Donoho, "Simultaneous cartoon and texture image inpainting using morphological component analysis (MCA)," Appl. Computational Harmonic Anal., vol. 19, pp. 340-358, 2005. (Pubitemid 41660098)
5. M. Elad and M. Aharon, "Image denoising via sparse and redundant representations over learned dictionaries," IEEE Trans. Image Process., vol. 15, no. 12, pp. 3736-3745, Dec. 2006. (Pubitemid 44811686)
6. J. Mairal, G. Sapiro, and M. Elad, "Learning multiscale sparse representations for image and video restoration," SIAM Multiscale Modeling Simulation, vol. 7, no. 1, pp. 214-241, 2008.
7. M. Protter and M. Elad, "Image sequence denoising via sparse and redundant representations," IEEE Trans. Image Process., vol. 18, no. 1, pp. 27-35, Jan. 2009.
8. O. Bryt and M. Elad, "Compression of facial images using the K-SVD algorithm," J. Visual Commun. Image Representation, vol. 19, no. 4, pp. 270-283, 2008.
9. M. Zibulevsky and B. A. Pearlmutter, "Blind source separation by sparse decomposition in a signal dictionary," Neural Comput., vol. 13, no. 4, pp. 863-882, 2001. (Pubitemid 33594308)
10. H. Y. Liao and G. Sapiro, "Sparse representations for limited data tomography," in Proc. 5th IEEE Int. Symp. Biomed. Imag.: Nano Macro (ISBI), 2008, pp. 1375-1378.
11. J. Mairal, F. Bach, J. Ponce, G. Sapiro, and A. Zisserman, "Non-local sparse models for image restoration," presented at the IEEE Int. Conf. Comput. Vision (ICCV), Kyoto, Japan, 2009.
12. G. Davis, S. Mallat, and M. Avellaneda, "Adaptive greedy approximations," Constructive Approximation, vol. 13, no. 1, pp. 57-98, 1997.
13. Y. C. Pati, R. Rezaiifar, and P. S. Krishnaprasad, "Orthogonal matching pursuit: Recursive function approximation with applications to wavelet decomposition," in Proc. Conf. Rec. 27th Asilomar Conf. Signals, Syst. Comput., 1993, pp. 40-44.
14. 1 minimization," in Proc. Nat. Academy Sci., 2003, vol. 100, pp. 2197-2202. (Pubitemid 36297476)
15. I. F. Gorodnitsky and B. D. Rao, "Sparse signal reconstruction from limited data using FOCUSS: A re-weighted minimum norm algorithm," IEEE Trans. Signal Process., vol. 45, no. 3, pp. 600-616, Mar. 1997. (Pubitemid 127765947)
16. D. L. Donoho, Y. Tsaig, I. Drori, and J. L. Starck, "Sparse solution of underdetermined linear equations by stagewise orthogonal matching pursuit," 2007.
17. S. Sardy, A. G. Bruce, and P. Tseng, "Block coordinate relaxation methods for nonparametric wavelet denoising," J. Computational Graph. Stat., vol. 9, no. 2, pp. 361-379, 2000.
18. M. Elad, "Why simple shrinkage is still relevant for redundant representations?," IEEE Trans. Inf. Theory, vol. 52, no. 12, pp. 5559-5569, Dec. 2006. (Pubitemid 46445399)
19. M. Elad, B. Matalon, and M. Zibulevsky, "Coordinate and subspace optimization methods for linear least squares with non-quadratic regularization," Appl. Computational Harmon. Anal., vol. 23, no. 3, pp. 346-367, 2007. (Pubitemid 47552874)
20. K. Schnass and P. Vandergheynst, "Dictionary preconditioning for greedy algorithms," IEEE Trans. Signal Process., vol. 56, no. 5, pp. 1994-2002, May 2008.
21. T. Blumensath and M. Davies, "Gradient pursuits," IEEE Trans. Signal Process., vol. 56, no. 6, pp. 2370-2382, Jun. 2008. (Pubitemid 351795890)
22. S. Mallat, A Wavelet Tour of Signal Processing, 3rd ed. New York: Academic, 2009.
23. E. J. Candès and D. L. Donoho, Curvelets-A Surprisingly Effective Nonadaptive Representation for Objects With Edges. Nashville, TN: Vanderbilt University Press, 1999, Curve and Surface Fitting: Saint-Malo, pp. 105-120.
24. M. N. Do and M. Vetterli, "The contourlet transform: An efficient directional multiresolution image representation," IEEE Trans. Image Process., vol. 14, no. 12, pp. 2091-2106, Dec. 2005. (Pubitemid 41781516)
25. D. Labate, W. Lim, G. Kutyniok, and G. Weiss, "Sparse multidimensional representation using shearlets," in Proc. SPIE: Wavelets XI, 2005, vol. 5914, pp. 254-262.
26. I. W. Selesnick, R. G. Baraniuk, and N. C. Kingsbury, "The dual-tree complex wavelet transform," IEEE Signal Process. Mag., vol. 22, no. 6, pp. 123-151, Nov. 2005. (Pubitemid 46404908)
27. E. LePennec and S. Mallat, "Sparse geometric image representations with bandelets," IEEE Trans. Image Process., vol. 14, no. 4, pp. 423-438, Apr. 2005. (Pubitemid 40552380)
28. I. T. Jolliffe, Principal Component Analysis, 2nd ed. New York: Springer, 2002.
29. R. Vidal, Y. Ma, and S. Sastry, "Generalized principal component analysis (GPCA)," IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 12, pp. 1945-1959, Dec. 2005. (Pubitemid 43051391)
30. K. Engan, S. O. Aase, and J. H. Husoy, "Method of optimal directions for frame design," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 1999, vol. 5, pp. 2443-2446.
31. M. Aharon, M. Elad, and A. M. Bruckstein, "The K-SVD: An algorithm for designing of overcomplete dictionaries for sparse representation," IEEE Trans. Signal Process., vol. 54, no. 11, pp. 4311-4322, Nov. 2006. (Pubitemid 44637761)
32. R. R. Coifman and M. V.Wickerhauser, "Entropy-based algorithms for best basis selection," IEEE Trans. Inf. Theory, vol. 38, no. 2, pt. 2, pp. 713-718, Mar. 1992.
33. H. H. Szu, B. A. Telfer, and S. L. Kadambe, "Neural network adaptive wavelets for signal representation and classification," Opt. Eng., vol. 31, p. 1907, 1992.
34. W. J. Jasper, S. J. Garnier, and H. Potlapalli, "Texture characterization and defect detection using adaptive wavelets," Opt. Eng., vol. 35, p. 3140, 1996.
35. M. Nielsen, E. N. Kamavuako, M. M. Andersen, M. F. Lucas, and D. Farina, "Optimal wavelets for biomedical signal compression," Med. Biological Eng. Comput., vol. 44, no. 7, pp. 561-568, 2006. (Pubitemid 44184468)
36. S. Lesage, R. Gribonval, F. Bimbot, and L. Benaroya, "Learning unions of orthonormal bases with thresholded singular value decomposition," in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2005, vol. 5, pp. 293-296.
37. M. Aharon and M. Elad, "Sparse and redundant modeling of image content using an image-signature-dictionary," SIAM J. Imag. Sci., vol. 1, no. 3, pp. 228-247, 2008.
38. R. Rubinstein, M. Zibulevsky, and M. Elad, "Efficient implementation of the K-SVD algorithm using batch orthogonal matching pursuit," CS Technion, 2008.
39. J. C. Bezdek and R. J. Hathaway, "Some notes on alternating optimization," Lecture Notes Comput. Sci., vol. 2275, pp. 187-195, 2002.
40. A. J. Smola and B. Scholkopf, "Sparse greedy matrix approximation for machine learning," in Proc. 17th Int. Conf. Mach. Learn., 2000, pp. 911-918.
41. R. A. Horn and C. R. Johnson, Topics in Matrix Analysis. Cambridge, U.K.: Cambridge University Press, 1991.
42. E. J. Im, "Optimizing the performance of sparse matrix-vector multiplication," Ph.D. dissertation, EECS Dept., Univ. California, Berkeley, 2000.
43. S. F. Cotter, R. Adler, R. D. Rao, and K. Kreutz-Delgado, "Forward sequential algorithms for best basis selection," IEEE Proc.-Vision, Image Signal Process., vol. 146, no. 5, pp. 235-244, 1999. (Pubitemid 30534103)
44. R. Rubinstein, M. Zibulevsky, and M. Elad, "Accelerating sparse-coding techniques using sparse dictionaries," CS Technion, 2009.
45. M. Aharon, M. Elad, and A. M. Bruckstein, "On the uniqueness of overcomplete dictionaries, and a practicalway to retrieve them," Linear Algebra Its Appl., vol. 416, no. 1, pp. 48-67, 2006. (Pubitemid 43737210)