Onsager-corrected deep learning for sparse linear inverse problems

2016 IEEE Global Conference on Signal and Information Processing, GlobalSIP 2016 - Proceedings

Volumn , Issue , 2017, Pages 227-231

  Borgerding, Mark ^a   Schniter, Philip ^a  

^a OHIO STATE UNIVERSITY   (United States)

Author keywords

Approximate message passing; Compressive sensing; Deep learning; Sparse coding

Indexed keywords

COMPRESSED SENSING; DEEP LEARNING; MESSAGE PASSING; NETWORK ARCHITECTURE; NEURAL NETWORKS; SIGNAL RECONSTRUCTION;

COMPRESSIVE SENSING; INFERENCE PROBLEM; LINEAR INVERSE PROBLEMS; LINEAR MEASUREMENTS; NOVEL NEURAL NETWORK; NUMERICAL EXPERIMENTS; PREDICTION ERRORS; SPARSE CODING;

INVERSE PROBLEMS;

EID: 85019234925     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/GlobalSIP.2016.7905837     Document Type: Conference Paper

References (25)

1. Y. C. Eldar and G. Kutyniok, Compressed Sensing: Theory and Applications. New York: Cambridge Univ. Press, 2012.
2. B. A. Olshausen and D. J. Field, "Sparse coding with an overcomplete basis set: A strategy employed by v1," Vision Research, vol. 37, pp. 3311-3325, 1997.
3. K. Gregor and Y. LeCun, "Learning fast approximations of sparse coding," in Proc. Int. Conf. Mach. Learning, pp. 399-406, 2010.
4. P. Sprechmann, P. Bronstein, and G. Sapiro, "Learning efficient structured-sparse models," in Proc. Int. Conf. Mach. Learning, pp. 615-622, 2012.
5. U. Kamilov and H. Mansour, "Learning optimal nonlinearities for iterative thresholding algorithms," IEEE Signal Process. Lett., vol. 23, pp. 747-751, May 2016.
6. Z. Wang, Q. Ling, and T. S. Huang, "Learning deep encoders," in Proc. AAAI Conf. Artificial Intell., pp. 2194-2200, 2016.
7. D. L. Donoho, A. Maleki, and A. Montanari, "Message passing algorithms for compressed sensing," Proc. Nat. Acad. Sci., vol. 106, pp. 18914-18919, Nov. 2009.
8. R. TibshIrani, "Regression shrinkage and selection via the lasso," J. Roy. Statist. Soc. B, vol. 58, no. 1, pp. 267-288, 1996.
9. S. S. Chen, D. L. Donoho, and M. A. Saunders, "Atomic decomposition by basis pursuit," SIAM J. Scientific Comput., vol. 20, no. 1, pp. 33-61, 1998.
10. E. Candès, J. Romberg, and T. Tao, "Stable signal recovery from incomplete and inaccurate measurements," Communications on Pure and Applied Mathematics, vol. 59, no. 8, pp. 1207-1223, 2006.
11. A. Chambolle, R. A. DeVore, N. Lee, and B. J. Lucier, "Nonlinear wavelet image processing: Variational problems, compression, and noise removal through wavelet shrinkage," IEEE Trans. Image Process., vol. 7, pp. 319-335, Mar. 1998.
12. I. Daubechies, M. Defrise, and C. D. Mol, "An iterative thresholding algorithm for linear inverse problems with a sparsity constraint,," Commun. Pure & Appl. Math., vol. 57, pp. 1413-1457, Nov. 2004.
13. A. Beck and M. Teboulle, "A fast iterative shrinkagethresholding algorithm for linear inverse problems," SIAM J. Imag. Sci., vol. 2, no. 1, pp. 183-202, 2009.
14. A. Montanari, "Graphical models concepts in compressed sensing," in Compressed Sensing: Theory and Applications (Y. C. Eldar and G. Kutyniok, eds.), Cambridge Univ. Press, 2012.
15. M. Bayati and A. Montanari, "The dynamics of message passing on dense graphs, with applications to compressed sensing," IEEE Trans. Inform. Theory, vol. 57, pp. 764-785, Feb. 2011.
16. D. L. Donoho, A. Maleki, and A. Montanari, "Message passing algorithms for compressed sensing: I. Motivation and construction," in Proc. Inform. Theory Workshop, (Cairo, Egypt), pp. 1-5, Jan. 2010.
17. I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning. MIT Press, 2016.
18. J. R. Hershey, J. Le Roux, and F. Weninger, "Deep unfolding: Model-based inspiration of novel deep architectures," Tech. Rep. TR2014-117, Mitsubishi Electric Research Labs, 2014. (See also arXiv:1409.2574).
19. U. Schmidt and S. Roth, "Shrinkage fields for effective image restoration," in Proc. IEEE Conf. Comp. Vision Pattern Recog., pp. 2774-2781, 2014.
20. C. Dong, C. C. Loy, K. He, and X. Tang, "Image superresolution using deep convolutional networks," IEEE Trans. Pattern Anal. Mach. Intell., vol. 38, pp. 295-307, Feb. 2016.
21. A. Mousavi, A. Patel, and R. Baraniuk, "A deep learning approach to structured signal recovery," in Proc. Allerton Conf. Commun. Control Comput., pp. 1336-1343, 2015.
22. K. Kulkarni, S. Lohi, P. Turaga, R. Kerviche, and A. Ashok, "ReconNet: Non-iterative reconstruction of images from compressively sensed random measurements," in Proc. IEEE Conf. Comp. Vision Pattern Recog., 2016. (see also arXiv:1601.06892).
23. M. Iliadis, L. Spinoulas, and A. K. Katsaggelos, "Deep fully-connected networks for video compressive sensing," in arXiv:1603:04930, 2016.
24. M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, S. Ghemawat, I. Goodfellow, A. Harp, G. Irving, M. Isard, Y. Jia, R. Jozefowicz, L. Kaiser, M. Kudlur, J. Levenberg, D. Mané, R. Monga, S. Moore, D. Murray, C. Olah, M. Schuster, J. Shlens, B. Steiner, I. Sutskever, K. Talwar, P. Tucker, V. Vanhoucke, V. Vasudevan, F. Viégas, O. Vinyals, P. Warden, M. Wattenberg, M. Wicke, Y. Yu, and X. Zheng, "Tensor-Flow: Large-scale machine learning on heterogeneous systems," 2015. Software available from tensorflow.org.
25. D. P. Kingma and J. Ba, "Adam: A method for stochastic optimization," in Proc. Internat. Conf. on Learning Repres., 2015. (see also arXiv:1412.6980).