Universal regularizers for robust sparse coding and modeling

IEEE Transactions on Image Processing

Volumn 21, Issue 9, 2012, Pages 3850-3864

  Ramírez, Ignacio ^a   Sapiro, Guillermo ^b  

^a UNIVERSIDAD DE LA REPÚBLICA   (Uruguay)

^b University of Minnesota   (United States)

Author keywords

Classification; denoising; dictionary learning; sparse coding; universal coding; zooming

Indexed keywords

DE-NOISING; DICTIONARY LEARNING; SPARSE CODING; UNIVERSAL CODING; ZOOMING;

CLASSIFICATION (OF INFORMATION); COMPUTER PROGRAMMING;

IMAGE PROCESSING;

EID: 84865443625     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2012.2197006     Document Type: Article

References (53)

1. M. Aharon, M. Elad, and A. Bruckstein, "The K-SVD: An algorithm for designing of overcomplete dictionaries for sparse representations," IEEE Trans. Signal Process., vol. 54, no. 11, pp. 4311-4322, Nov. 2006. (Pubitemid 44637761)
2. K. Engan, S. Aase, and J. Husoy, "Multi-frame compression: Theory and design," Signal Process., vol. 80, no. 10, pp. 2121-2140, Oct. 2000.
3. B. Olshausen and D. Field, "Sparse coding with an overcomplete basis set: A strategy employed by V1?" Vision Res., vol. 37, no. 23, pp. 3311-3325, 1997. (Pubitemid 27493805)
4. I. Daubechies, M. Defrise, and C. De Mol, "An iterative thresholding algorithm for linear inverse problems with a sparsity constraint," Commun. Pure Appl. Math., vol. 57, no. 11, pp. 1413-1457, 2004. (Pubitemid 39427442)
5. B. Efron, T. Hastie, I. Johnstone, and R. Tibshirani, "Least angle regression," Ann. Stat., vol. 32, no. 2, pp. 407-499, 2004. (Pubitemid 41250302)
6. S. Chen, D. Donoho, and M. Saunders, "Atomic decomposition by basis pursuit," SIAM J. Sci. Comput., vol. 20, no. 1, pp. 33-61, 1998. (Pubitemid 128689501)
7. B. Krishnapuram, L. Carin, M. Figueiredo, and A. Hartemink, "Sparse multinomial logistic regression: Fast algorithms and generalization bounds," IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 6, pp. 957-968, Jun. 2005. (Pubitemid 40889845)
8. J. Mairal, F. Bach, J. Ponce, G. Sapiro, and A. Zisserman, "Supervised dictionary learning," in Proc. Adv. Neural Inform. Process. Syst., 2008, pp. 1-8.
9. J. Mairal, G. Sapiro, and M. Elad, "Learning multiscale sparse representations for image and video restoration," Comput. Inform. Sci., vol. 7, no. 1, pp. 214-241, Apr. 2008.
10. R. Raina, A. Battle, H. Lee, B. Packer, and A. Ng, "Self-taught learning: Transfer learning from unlabeled data," in Proc. Int. Conf. Mach. Learn., Jun. 2007, pp. 759-766.
11. A. Bruckstein, D. Donoho, and M. Elad, "From sparse solutions of systems of equations to sparse modeling of signals and images," SIAM Rev., vol. 51, no. 1, pp. 34-81, Feb. 2009.
12. R. Giryes, Y. Eldar, and M. Elad, "Automatic parameter setting for iterative shrinkage methods," in Proc. IEEE 25th Convent. Electron. Elect. Eng., Dec. 2008, pp. 820-824.
13. M. Figueiredo, "Adaptive sparseness using Jeffreys prior," in Adv. NIPS, T. G. Dietterich, S. Becker, and Z. Ghahramani, Eds. Cambridge, MA: MIT Press, Dec. 2001, pp. 697-704.
14. H. Zou, "The adaptive LASSO and its oracle properties," J. Amer. Stat. Assoc., vol. 101, no. 476, pp. 1418-1429, 2006.
15. J. Rissanen, "Universal coding, information, prediction and estimation," IEEE Trans. Inform. Theory, vol. 30, no. 4, pp. 629-636, Jul. 1984. (Pubitemid 14632095)
16. R. Coifman and M. Wickenhauser, "Entropy-based algorithms for best basis selection," IEEE Trans. Inform. Theory, vol. 38, no. 2, pp. 713-718, Mar. 1992.
17. P. Moulin and J. Liu, "Analysis of multiresolution image denoising schemes using generalized-Gaussian and complexity priors," IEEE Trans. Inform. Theory, vol. 45, no. 3, pp. 909-919, Apr. 1999.
18. N. Saito, "Simultaneous noise suppression and signal compression using a library of orthonormal bases and the MDL criterion," in Wavelets in Geophysics, E. Foufoula-Georgiou and P. Kumar, Eds. New York: Academic, 1994.
19. E. J. Candès, "Compressive sampling," in Proc. Int. Cong. Math., Aug. 2006, pp. 1-20.
20. I. Ramírez and G. Sapiro, "An MDL framework for sparse coding and dictionary learning," IEEE Trans. Signal Process., vol. 60, no. 6, pp. 2913-2927, Jun. 2012.
21. S. Mallat and Z. Zhang, "Matching pursuit in a time-frequency dictionary," IEEE Trans. Signal Process., vol. 41, no. 12, pp. 3397-3415, Dec. 1993.
22. T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning: Data Mining, Inference and Prediction. New York: Springer-Verlag, Feb. 2009.
23. R. Tibshirani, "Regression shrinkage and selection via the LASSO," J. Royal Stat. Soc. Series B, vol. 58, no. 1, pp. 267-288, 1996.
24. T. Cover and J. Thomas, Elements of Information Theory. New York: Wiley, 2006.
25. J. Rissanen, Stochastic Complexity in Statistical Inquiry. Singapore: World Scientific, 1992.
26. E. Lam and J. Goodman, "A mathematical analysis of the DCT coefficient distributions for images," IEEE Trans. Image Process, vol. 9, no. 10, pp. 1661-1666, Oct. 2000.
27. S. Ji, Y. Xue, and L. Carin, "Bayesian compressive sensing," IEEE Trans. Signal Process., vol. 56, no. 6, pp. 2346-2356, Jun. 2008. (Pubitemid 351795888)
28. M. Tipping, "Sparse Bayesian learning and the relevance vector machine," J. Mach. Learn., vol. 1, pp. 211-244, Jan. 2001. (Pubitemid 33687203)
29. D. Wipf and B. Rao, "An empirical bayesian strategy for solving the simultaneous sparse approximation problem," IEEE Trans. Signal Process., vol. 55, no. 7, pp. 3704-3716, Jul. 2007. (Pubitemid 47018850)
30. D. Wipf, J. Palmer, and B. Rao, "Perspectives on sparse Bayesian learning," in Proc. Adv. Nopany Inst. Professional Studies, Dec. 2003, pp. 1-249.
31. M. Everingham, A. Zisserman, C. Williams, and L. Van Gool. The Pascal Visual Object Classes Challenge 2006 Voc2006 Results [Online]. Available: http://www.pascalnetwork. org/challenges/VOC/voc2006/results.pdf
32. N. Merhav and M. Feder, "Universal prediction," IEEE Trans. Image Theory, vol. 44, no. 6, pp. 2124-2147, Oct. 1998. (Pubitemid 128741886)
33. Y. Shtarkov, "Universal sequential coding of single messages," Probl. Inform. Trans., vol. 23, no. 3, pp. 3-17, Jul. 1987.
34. P. Grünwald, The Minimum Description Length Principle. Cambridge, MA: MIT Press, Jun. 2007.
35. G. Schwartz, "Estimating the dimension of a model," Ann. Stat., vol. 6, no. 2, pp. 461-464, 1978.
36. J. Fan and R. Li, "Variable selection via nonconcave penalized likelihood and its oracle properties," J. Amer. Stat. Assoc., vol. 96, no. 456, pp. 1348-1360, Dec. 2001. (Pubitemid 33695585)
37. E. J. Candès, M.Wakin, and S. Boyd, "Enhancing sparsity by reweighted 1 minimization," J. Fourier Anal. Appl., vol. 14, no. 5, pp. 877-905, Dec. 2008.
38. R. Saab, R. Chartrand, and O. Yilmaz, "Stable sparse approximation via nonconvex optimization," in Proc. IEEE Acoust. Speech Signal Process. Int. Conf., Apr. 2008, pp. 3885-3888.
39. S. Foucart and M. Lai, "Sparsest solutions of underdetermined linear systems via q -minimization for 0 < q ≤ 1," Appl. Comput. Harmonic Anal., vol. 3, no. 26, pp. 395-407, 2009.
40. J. Trzasko and A. Manduca, "Relaxed conditions for sparse signal recovery with general concave priors," IEEE Trans. Signal Process., vol. 57, no. 11, pp. 4347-4354, Nov. 2009.
41. R. Chartrand, "Fast algorithms for nonconvex compressive sensing: MRI reconstruction from very few data," in Proc. IEEE Biomed. Imag.: From Nano Macro Int. Symp., Jun. 2009, pp. 262-265.
42. J. Trzasko and A. Manduca, "Highly undersampled magnetic resonance image reconstruction via homotopic 0-minimization," IEEE Trans. Med. Imag., vol. 28, no. 1, pp. 106-121, Jan. 2009.
43. J. Bernardo and A. Smith, Bayesian Theory. New York: Wiley, 1994.
44. A. Barron, J. Rissanen, and B. Yu, "The minimum description length principle in coding and modeling," IEEE Trans. Inform. Theory, vol. 44, no. 6, pp. 2743-2760, Oct. 1998. (Pubitemid 128741906)
45. H. Zou and R. Li, "One-step sparse estimates in nonconcave penalized likelihood models," Ann. Stat., vol. 36, no. 4, pp. 1509-1533, 2008.
46. G. Gasso, A. Rakotomamonjy, and S. Canu, "Recovering sparse signals with non-convex penalties and DC programming," IEEE Trans. Signal Process., vol. 57, no. 12, pp. 4686-4698, Dec. 2009.
47. G. Motta, E. Ordentlich, I. Ramirez, G. Seroussi, and M. Weinberger, "The iDUDE framework for grayscale image denoising," IEEE Trans. Image Process., vol. 20, no. 1, pp. 1-21, Jan. 2011.
48. I. Ramirez, F. Lecumberry, and G. Sapiro, "Universal priors for sparse modeling," in Proc. IEEE Comput. Adv. Multi-Sensor Adaptive Process. Int. Workshop, Dec. 2009, pp. 197-200.
49. I. Ramírez, P. Sprechmann, and G. Sapiro, "Classification and clustering via dictionary learning with structured incoherence and shared features," in Proc. IEEE Comput. Vision Pattern Recogn. Conf., Jun. 2010, pp. 3501-3508.
50. J. Tropp, "Greed is good: Algorithmic results for sparse approximation," IEEE Trans. Inform. Theory, vol. 50, no. 10, pp. 2231-2242, Oct. 2004.
51. M. Elad, "Optimized projections for compressed-sensing," IEEE Trans. Signal Process., vol. 55, no. 12, pp. 5695-5702, Dec. 2007.
52. G. Yu, G. Sapiro, and S. Mallat, "Solving inverse problems with piecewise linear estimators: From Gaussian mixture models to structured sparsity," IEEE Trans. Image Process., vol. 21, no. 5, pp. 2481-2499, May 2011.
53. R. Neelamani, H. Choi, and R. Baraniuk, "Forward: Fourier-wavelet regularized deconvolution for ill-conditioned systems," IEEE Trans. Signal Process., vol. 52, no. 2, pp. 418-433, Feb. 2004.