Insights into analysis operator learning: From patch-based sparse models to higher order MRFs

IEEE Transactions on Image Processing

Volumn 23, Issue 3, 2014, Pages 1060-1072

  Chen, Yunjin ^a,b   Ranftl, Rene ^a   Pock, Thomas ^a  

^a GRAZ UNIVERSITY OF TECHNOLOGY   (Austria)

^b NATIONAL UNIVERSITY OF DEFENSE TECHNOLOGY   (China)

Author keywords

Analysis operator learning; bi level optimization; image restoration; loss specific training; MRFs

Indexed keywords

ANALYSIS OPERATOR LEARNING; ANALYSIS OPERATORS; BI-LEVEL OPTIMIZATION; IMAGE DENOISING ALGORITHM; IMAGE RESTORATION PROBLEM; MRFS; NUMERICAL EXPERIMENTS; TRAINING PROCEDURES;

IMAGE RECONSTRUCTION; OPTIMIZATION;

PERSONNEL TRAINING;

ALGORITHM; ARTICLE; AUTOMATED PATTERN RECOGNITION; COMPUTER ASSISTED DIAGNOSIS; COMPUTER SIMULATION; IMAGE ENHANCEMENT; METHODOLOGY; REPRODUCIBILITY; SENSITIVITY AND SPECIFICITY; STATISTICAL ANALYSIS; STATISTICAL MODEL;

ALGORITHMS; COMPUTER SIMULATION; DATA INTERPRETATION, STATISTICAL; IMAGE ENHANCEMENT; IMAGE INTERPRETATION, COMPUTER-ASSISTED; MODELS, STATISTICAL; PATTERN RECOGNITION, AUTOMATED; REPRODUCIBILITY OF RESULTS; SENSITIVITY AND SPECIFICITY;

EID: 84893847895     PISSN: 10577149     EISSN: None     Source Type: Journal    
DOI: 10.1109/TIP.2014.2299065     Document Type: Article

References (41)

1. M. Aharon, M. Elad, and A. Bruckstein, "K-SVD: An algorithm for designing over-complete dictionaries for sparse representation," IEEE Trans. Signal Process., vol. 54, no. 11, pp. 4311-4322, Nov. 2006. (Pubitemid 44637761)
2. P. Arbelaez, M. Maire, C. Fowlkes, and J. Malik, "Contour detection and hierarchical image segmentation," IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 5, pp. 898-916, May 2011.
3. A. Barbu, "Training an active random field for real-time image denoising," IEEE Trans. Image Process., vol. 18, no. 11, pp. 2451-2462, Nov. 2009.
4. E. J. Candes, M. B. Wakin, and S. P. Boyd, "Enhancing sparsity by reweighted l1 minimization," J. Fourier Anal. Appl., vol. 14, no. 5, pp. 877-905, 2008.
5. A. Chambolle and T. Pock, "A first-order primal-dual algorithm for convex problems with applications to imaging," J. Math. Imag. Vis., vol. 40, no. 1, pp. 120-145, 2011.
6. Y. J. Chen, T. Pock, and H. Bischof, "Learning-1-based analysis and synthesis sparsity priors using bi-level optimization," in Proc. Workshop Anal. Oper. Learn. Dictionary Learn., NIPS, 2012, pp. 1-5.
7. Y. J. Chen, T. Pock, R. Ranftl, and H. Bischof, "Revisiting loss-specific training of filter-based MRFs for image restoration," in Proc. 35th German Conf., GCPR, Sep. 2013, pp. 271-281.
8. B. Colson, P. Marcotte, and G. Savard, "An overview of bilevel optimization," Ann. Oper. Res., vol. 153, no. 1, pp. 235-256, 2007.
9. K. Dabov, A. Foi, V. Katkovnik, and K. O. Egiazarian, "Image denoising by sparse 3D transform-domain collaborative filtering," IEEE Trans. Image Process., vol. 16, no. 8, pp. 2080-2095, Aug. 2007. (Pubitemid 47225460)
10. C. B. Do, C.-S. Foo, and A. Y. Ng, "Efficient multiple hyperparameter learning for log-linear models," in Proc. 21st Adv. Conf. NIPS, 2007, pp. 377-384.
11. J. Domke, "Generic methods for optimization-based modeling," in Proc. 15th Int. Conf. Artif. Intell. Statist., 2012, pp. 318-326.
12. D. L. Donoho, M. Elad, and V. N. Temlyakov, "Stable recovery of sparse overcomplete representations in the presence of noise," IEEE Trans. Inf. Theory, vol. 52, no. 1, pp. 6-18, Jan. 2006. (Pubitemid 43263116)
13. M. Elad and M. Aharon, "Image denoising via sparse and redundant representations over learned dictionaries," IEEE Trans. Image Process., vol. 15, no. 12, pp. 3736-3745, Dec. 2006. (Pubitemid 44811686)
14. M. Elad, P. Milanfar, and R. Rubinstein, "Analysis versus synthesis in signal priors," Inverse Problems, vol. 23, no. 3, pp. 947-968, 2007. (Pubitemid 46838835)
15. M.-J. Fadili, J.-L. Starck, and F. Murtagh, "Inpainting and zooming using sparse representations," Comput. J., vol. 52, no. 1, pp. 64-79, 2009.
16. Q. Gao and S. Roth, "How well do filter-based MRFs model natural images?" in Proc. 34th DAGM 36th OAGM Symp., Aug. 2012, pp. 62-72.
17. S. Hawe, M. Kleinsteuber, and K. Diepold, "Analysis operator learning and its application to image reconstruction," IEEE Trans. Image Process., vol. 22, no. 6, pp. 2138-2150, Mar. 2013.
18. G. E. Hinton, "Products of experts," in Proc. 9th ICANN, 1999, pp. 1-6.
19. J. Huang and D. Mumford, "Statistics of natural images and models," in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Fort Collins, CO, USA, 1999, pp. 541-547.
20. J. Jancsary, S. Nowozin, and C. Rother, "Loss-specific training of nonparametric image restoration models: A new state of the art," in Proc. 12th ECCV, 2012, pp. 112-125.
21. K. Kunisch and T. Pock, "A bilevel optimization approach for parameter learning in variational models," SIAM J. Imag. Sci., vol. 6, no. 2, PP. 938-983, 2013.
22. H. Lee, A. Battle, R. Raina, and A. Y. Ng, "Efficient sparse coding algorithms," in Proc. NIPS, 2006, pp. 801-808.
23. D. C. Liu and J. Nocedal, "On the limited memory BFGS method for large scale optimization," Math. Program., vol. 45, no. 1, pp. 503-528, 1989. (Pubitemid 20660315)
24. J. Mairal, F. Bach, J. Ponce, and G. Sapiro, "Online dictionary learning for sparse coding," in Proc. 26th Annu. ICML, 2009, pp. 689-696.
25. J. Mairal, F. Bach, J. Ponce, G. Sapiro, and A. Zisserman, "Non-local sparse models for image restoration," in Proc. IEEE ICCV, Oct. 2009, pp. 2272-2279.
26. J. Mairal, M. Elad, and G. Sapiro, "Sparse representation for color image restoration," IEEE Trans. Image Process., vol. 17, no. 1, pp. 53-69, Jan. 2008.
27. B. Ophir, M. Elad, N. Bertin, and M. D. Plumbley, "Sequential minimal eigenvalues-An approach to analysis dictionary learning,"in Proc. 19th Eur. Signal Process.Conf., 2011, pp. 1465-1469.
28. G. Peyré and J. Fadili, "Learning analysis sparsity priors," in Proc. Sampta'11, 2011.
29. T. Pock and A. Chambolle, "Diagonal preconditioning for first order primal-dual algorithms in convex optimization ," in Proc. IEEE ICCV, Nov. 2011, pp. 1762-1769.
30. S. Roth and M. J. Black, "Fields of experts," Int. J. Comput. Vis., vol. 82, no. 2, pp. 205-229, 2009.
31. R. Rubinstein, T. Faktor, and M. Elad, "K-SVD dictionary-learning for the analysis sparse model," in Proc. IEEE ICASSP, Mar. 2012, pp. 5405-5408.
32. R. Rubinstein, T. Peleg, and M. Elad, "Analysis K-SVD: A dictionarylearning algorithm for the analysis sparse model," IEEE Trans. Signal Process., vol. 61, no. 3, pp. 661-677, Feb. 2013.
33. K. G. G. Samuel and M. Tappen, "Learning optimized MAP estimates in continuously-valued MRF models," in Proc. IEEE CVPR, Jun. 2009, pp. 477-484.
34. U. Schmidt, Q. Gao, and S. Roth, "A generative perspective on MRFs in low-level vision," in Proc. IEEE CVPR, Jun. 2010, pp. 1751-1758.
35. M. F. Tappen, "Utilizing variational optimization to learn Markov random fields," in Proc. IEEE CVPR, Jun. 2007, pp. 1-8.
36. M. F. Tappen, K. G. G. Samuel, C. V. Dean, and D. M. Lyle, "The logistic random field-A convenient graphical model for learning parameters for MRF-based labeling," in Proc. IEEE CVPR, Jun. 2008, pp. 1-8.
37. Y. W. Teh, M. Welling, S. Osindero, and G. E. Hinton, "Energy-based models for sparse overcomplete representations," J. Mach. Learn. Res., vol. 4, pp. 1235-1260, Dec. 2003.
38. M.Welling, G. E. Hinton, and S. Osindero, "Learning sparse topographic representations with products of student-t distributions," in Proc. 15th NIPS, Dec. 2002, pp. 1359-1366.
39. M. Yaghoobi, S. Nam, R. Gribonval, and M. E. Davies, "Analysis operator learning for over-complete co-sparse representations," in Proc. Eur. Signal Process. Conf., 2011.
40. M. Yaghoobi, S. Nam, R. Gribonval, and M. E. Davies, "Noise aware analysis operator learning for approximately cosparse signals," in Proc. IEEE ICASSP, Mar. 2012, pp. 5409-5412.
41. D. Zoran and Y. Weiss, "From learning models of natural image patches to whole image restoration," in Proc. IEEE ICCV, Nov. 2011, pp. 479-486.