l0 Sparsifying Transform Learning With Efficient Optimal Updates and Convergence Guarantees

IEEE Transactions on Signal Processing

Volumn 63, Issue 9, 2015, Pages 2389-2404

  Ravishankar, Saiprasad ^a   Bresler, Yoram ^a  

^a University of Illinois at Urbana Champaign   (United States)

Author keywords

Denoising; dictionary learning; fast algorithms; image representation; non convex; sparse representation; transform model

Indexed keywords

ALGORITHMS; GLOBAL OPTIMIZATION; IMAGE PROCESSING; IMAGE RECONSTRUCTION; ITERATIVE METHODS; LEARNING ALGORITHMS; MEDICAL IMAGE PROCESSING; MEDICAL IMAGING; SIGNAL PROCESSING;

DE-NOISING; DICTIONARY LEARNING; FAST ALGORITHMS; IMAGE REPRESENTATIONS; NON-CONVEX; SPARSE REPRESENTATION; TRANSFORM MODELS;

IMAGE DENOISING;

EID: 84927591259     PISSN: 1053587X     EISSN: None     Source Type: Journal    
DOI: 10.1109/TSP.2015.2405503     Document Type: Article

References (44)

1. M. Elad, P. Milanfar, and R. Rubinstein, "Analysis versus synthesis in signal priors," Inverse Probl., vol. 23, no. 3, pp. 947-968, 2007.
2. A. M. Bruckstein, D. L. Donoho, and M. Elad, "From sparse solutions of systems of equations to sparse modeling of signals and images,"SIAM Rev., vol. 51, no. 1, pp. 34-81, 2009.
3. R. Rubinstein, T. Peleg, and M. Elad, "Analysis K-SVD: A dictionary-learning algorithm for the analysis sparse model," IEEE Trans. Signal Process., vol. 61, no. 3, pp. 661-677, 2013.
4. W. K. Pratt, J. Kane, and H. C. Andrews, "Hadamard transform image coding," Proc. IEEE, vol. 57, no. 1, pp. 58-68, 1969.
5. S. Ravishankar and Y. Bresler, "Learning sparsifying transforms,"IEEE Trans. Signal Process., vol. 61, no. 5, pp. 1072-1086, 2013.
6. B. A. Olshausen and D. J. Field, "Emergence of simple-cell receptive field properties by learning a sparse code for natural images," Nature, vol. 381, no. 6583, pp. 607-609, 1996.
7. K. Engan, S. Aase, and J. Hakon-Husoy, "Method of optimal directions for frame design," in Proc. IEEE Int. Conf. Acoust. Speech, Signal Process., 1999, pp. 2443-2446.
8. K. Kreutz-Delgado, J. F. Murray, B. D. Rao, K. Engan, T.-W. Lee, and T. J. Sejnowski, "Dictionary learning algorithms for sparse representation,"Neural Comput., vol. 15, no. 2, pp. 349-396, 2003.
9. M. Aharon, M. Elad, and A. Bruckstein, "K-SVD: An algorithm for designing overcomplete dictionaries for sparse representation," IEEE Trans. Signal Process., vol. 54, no. 11, pp. 4311-4322, 2006.
10. M. Yaghoobi, T. Blumensath, and M. Davies, "Dictionary learning for sparse approximations with the majorization method," IEEE Trans. Signal Process., vol. 57, no. 6, pp. 2178-2191, 2009.
11. K. Skretting and K. Engan, "Recursive least squares dictionary learning algorithm," IEEE Trans. Signal Process., vol. 58, no. 4, pp. 2121-2130, 2010.
12. J. Mairal, F. Bach, J. Ponce, and G. Sapiro, "Online learning for matrix factorization and sparse coding," J. Mach. Learn. Res., vol. 11, pp. 19-60, 2010.
13. M. Elad and M. Aharon, "Image denoising via sparse and redundant representations over learned dictionaries," IEEE Trans. Image Process., vol. 15, no. 12, pp. 3736-3745, 2006.
14. J. Mairal, M. Elad, and G. Sapiro, "Sparse representation for color image restoration," IEEE Trans. Image Process., vol. 17, no. 1, pp. 53-69, 2008.
15. S. Ravishankar and Y. Bresler, "MR image reconstruction from highly undersampled k-space data by dictionary learning," IEEE Trans. Med. Imag., vol. 30, no. 5, pp. 1028-1041, 2011.
16. M. Yaghoobi, S. Nam, R. Gribonval, and M. Davies, "Analysis operator learning for overcomplete cosparse representations," in Proc. Eur. Signal Process. Conf., 2011, pp. 1470-1474.
17. B. Ophir, M. Elad, N. Bertin, and M. Plumbley, "Sequential minimal eigenvalues - An approach to analysis dictionary learning," in Proc. Eur. Signal Process. Conf., 2011, pp. 1465-1469.
18. M. Yaghoobi, S. Nam, R. Gribonval, and M. E. Davies, "Constrained overcomplete analysis operator learning for cosparse signal modelling,"IEEE Trans. Signal Process., vol. 61, no. 9, pp. 2341-2355, 2013.
19. S. Ravishankar and Y. Bresler, "Learning doubly sparse transforms for images," IEEE Trans. Image Process., vol. 22, no. 12, pp. 4598-4612, 2013.
20. M. Zibulevsky and B. A. Pearlmutter, "Blind source separation by sparse decomposition in a signal dictionary," Neural Computat., vol. 13, no. 4, pp. 863-882, 2001.
21. L. Pfister, "Tomographic Reconstruction With Adaptive Sparsifying Transforms," Master's thesis, Univ. Illinois at Urbana-Champaign, Urbana, USA, Aug. 2013.
22. L. Pfister and Y. Bresler, "Model-based iterative tomographic reconstruction with adaptive sparsifying transforms," in Proc. SPIE Int. Symp. Electron. Imag.: Computat. Imag. XII, 2014, vol. 9020, pp. 90 200H-1-90 200H-11.
23. L. Mirsky, "On the trace of matrix products," Math. Nachrichten, vol. 20, no. 3-6, pp. 171-174, 1959.
24. G. W. Stewart, Matrix Algorithms: Volume 1: Basic Decompositions. Philadelphia, PA, USA: SIAM, 1998.
25. P. Tseng, "Convergence of a block coordinate descent method for non-differentiable minimization," J. Optim. Theory Appl., vol. 109, no. 3, pp. 475-494, 2001.
26. T. Blumensath and M. E. Davies, "Iterative hard thresholding for compressed sensing," Applied and Computat. Harmon. Anal., vol. 27, no. 3, pp. 265-274, 2009.
27. T. Blumensath and M. E. Davies, "Iterative thresholding for sparse approximations,"J. Fourier Anal. Appl., vol. 14, no. 5-6, pp. 629-654, 2008.
28. S. Ravishankar and Y. Bresler, "Learning doubly sparse transforms for image representation," in IEEE Int. Conf. Image Process., 2012, pp. 685-688.
29. S. Ravishankar and Y. Bresler, "Sparsifying transform learning for compressed sensing MRI," in Proc. IEEE Int. Symp. Biomed. Imag., 2013, pp. 17-20.
30. M. Elad, Michael Elad Personal Page, [Online]. Available: http://www.cs.technion.ac.il/~elad/Various/KSVD-Matlab-ToolBox.zip, accessed 2014.
31. A. Hyvärinen and E. Oja, "A fast fixed-point algorithm for independent component analysis," Neural Comput., vol. 9, no. 7, pp. 1483-1492, 1997.
32. A. J. Bell and T. J. Sejnowski, "An information-maximization approach to blind separation and blind deconvolution," Neural Comput., vol. 7, no. 6, pp. 1129-1159, 1995.
33. A. J. Bell and T. J. Sejnowski, "The "independent components of natural scenes are edge filters," Vision Res., vol. 37, no. 23, pp. 3327-3338, 1997.
34. A. Hyvärinen, J. Karhunen, and E. Oja, Independent Component Analysis. New York, NY, USA: Wiley-Intersci., 2001.
35. J.-F. Cardoso, "High-order contrasts for independent component analysis,"Neural Comput., vol. 11, no. 1, pp. 157-192, 1999.
36. A. J. Ferreira and M. A. T. Figueiredo, "Class-adapted image compression using independent component analysis," in Proc. Int. Conf. Image Process., 2003, vol. 1, pp. I-625-I-628.
37. A. Hyvärinen, The Fastica Matlab Package [Online]. Available: http://research.ics.aalto.fi/ica/fastica/code/FastICA-2.5.zip, accessed August 2014.
38. Y. Pati, R. Rezaiifar, and P. Krishnaprasad, "Orthogonal matching pursuit: Recursive function approximation with applications to wavelet decomposition," in Proc. Asilomar Conf. Signals, Syst. Comput., 1993, vol. 1, pp. 40-44.
39. J.-F. Cardoso, Jade for Real-Valued Data, 2005 [Online]. Available: http://bsp.teithe.gr/members/downloads/Jade, accessed August 2014.
40. K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, "Image denoising by sparse 3D transform-domain collaborative filtering," IEEE Trans. Image Process., vol. 16, no. 8, pp. 2080-2095, 2007.
41. B. Wen, S. Ravishankar, and Y. Bresler, "Structured overcomplete sparsifying transform learning with convergence guarantees and applications,"Int. J. Comput. Vision pp. 1-31, 2014 [Online]. Available: http://dx.doi.org/10.1007/s11263-014-0761-1
42. S. Ravishankar and Y. Bresler, "Learning overcomplete sparsifying transforms for signal processing," in Proc. IEEE Int. Conf. Acoust., Speech Signal Process. (ICASSP), May 2013, pp. 3088-3092.
43. P. Schönemann, "A generalized solution of the orthogonal procrustes problem," Psychometrika, vol. 31, no. 1, pp. 1-10, 1966.
44. M. Zibulevsky, "Blind source separation with relative newton method,"in Proc. 4th Int. Symp. Independ. Compon. Anal. Blind Signal Separat., 2003, pp. 897-902.