Learned D-AMP: Principled! Neural network based compressive image recovery

Advances in Neural Information Processing Systems

Volumn 2017-December, Issue , 2017, Pages 1773-1784

  Metzler, Christopher A ^a   Mousavi, Ali ^a   Baraniuk, Richard G ^a  

^a RICE UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

COMPUTER ARCHITECTURE; IMAGE COMPRESSION; ITERATIVE METHODS; MESSAGE PASSING; NETWORK ARCHITECTURE; NEURAL NETWORKS; SIGNAL RECONSTRUCTION;

FAST AND ACCURATE ALGORITHMS; FAST IMPLEMENTATION; MASSIVELY PARALLELS; MEASUREMENT MATRIX; NOVEL NEURAL NETWORK; ORDERS OF MAGNITUDE; SPARSE SIGNAL RECOVERIES; STATE OF THE ART;

COMPUTER SYSTEM RECOVERY;

EID: 85047007715     PISSN: 10495258     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (49)

1. E. J. Candes, J. Romberg, and T. Tao, "Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information," IEEE Trans. Inform. Theory, vol. 52, no. 2, pp. 489-509, Feb. 2006.
2. R. G. Baraniuk, "Compressive sensing [lecture notes]," IEEE Signal Processing Mag., vol. 24, no. 4, pp. 118-121, 2007.
3. D. Needell and J. A. Tropp, "CoSaMP: Iterative signal recovery from incomplete and inaccurate samples," Appl. Comput. Harmon. Anal., vol. 26, no. 3, pp. 301-321, 2009.
4. I. Daubechies, M. Defrise, and C. D. Mol, "An iterative thresholding algorithm for linear inverse problems with a sparsity constraint," Comm. on Pure and Applied Math., vol. 75, pp. 1412-1457, 2004.
5. D. L. Donoho, A. Maleki, and A. Montanari, "Message passing algorithms for compressed sensing," Proc. Natl. Acad. Sci., vol. 106, no. 45, pp. 18914-18919, 2009.
6. S. Rangan, P. Schniter, and A. Fletcher, "Vector approximate message passing," arXiv preprint arXiv:1610.03082, 2016.
7. C. Li, W. Yin, and Y. Zhang, "User's guide for TVAL3: TV minimization by augmented Lagrangian and alternating direction algorithms," Rice CAAM Department report, vol. 20, pp. 46-47, 2009.
8. R. G. Baraniuk, V. Cevher, M. F. Duarte, and C. Hegde, "Model-based compressive sensing," IEEE Trans. Inform. Theory, vol. 56, no. 4, pp. 1982-2001, Apr. 2010.
9. W. Dong, G. Shi, X. Li, Y. Ma, and F. Huang, "Compressive sensing via nonlocal low-rank regularization," IEEE Trans. Image Processing, vol. 23, no. 8, pp. 3618-3632, 2014.
10. C. A. Metzler, A. Maleki, and R. G. Baraniuk, "From denoising to compressed sensing," IEEE Trans. Inform. Theory, vol. 62, no. 9, pp. 5117-5144, 2016.
11. P. Schniter, S. Rangan, and A. Fletcher, "Denoising based vector approximate message passing," arXiv preprint arXiv:1611.01376, 2016.
12. S. Beygi, S. Jalali, A. Maleki, and U. Mitra, "An efficient algorithm for compression-based compressed sensing," arXiv preprint arXiv:1704.01992, 2017.
13. A. Mousavi, A. B. Patel, and R. G. Baraniuk, "A deep learning approach to structured signal recovery," Proc. Allerton Conf. Communication, Control, and Computing, pp. 1336-1343, 2015.
14. K. Kulkarni, S. Lohit, P. Turaga, R. Kerviche, and A. Ashok, "Reconnet: Non-iterative reconstruction of images from compressively sensed measurements," Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 449-458, 2016.
15. A. Mousavi and R. G. Baraniuk, "Learning to invert: Signal recovery via deep convolutional networks," Proc. IEEE Int. Conf. Acoust., Speech, and Signal Processing (ICASSP), pp. 2272-2276, 2017.
16. 2-net: Deep residual reconstruction network for image compressive sensing," arXiv preprint arXiv:1702.05743, 2017.
17. P. Vincent, H. Larochelle, I. Lajoie, Y. Bengio, and P.-A. Manzagol, "Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion," J. Machine Learning Research, vol. 11, pp. 3371-3408, 2010.
18. E. W. Tramel, A. Drémeau, and F. Krzakala, "Approximate message passing with restricted Boltzmann machine priors," Journal of Statistical Mechanics: Theory and Experiment, vol. 2016, no. 7, p. 073401, 2016.
19. E. W. Tramel, A. Manoel, F. Caltagirone, M. Gabrié, and F. Krzakala, "Inferring sparsity: Compressed sensing using generalized restricted Boltzmann machines," Proc. IEEE Information Theory Workshop (ITW), pp. 265-269, 2016.
20. E. W. Tramel, M. Gabrié, A. Manoel, F. Caltagirone, and F. Krzakala, "A deterministic and generalized framework for unsupervised learning with restricted Boltzmann machines," arXiv preprint arXiv:1702.03260, 2017.
21. A. Dave, A. K. Vadathya, and K. Mitra, "Compressive image recovery using recurrent generative model," arXiv preprint arXiv:1612.04229, 2016.
22. L. Theis and M. Bethge, "Generative image modeling using spatial LSTMs," Proc. Adv. in Neural Processing Systems (NIPS), pp. 1927-1935, 2015.
23. J. Rick Chang, C.-L. Li, B. Poczos, B. Vijaya Kumar, and A. C. Sankaranarayanan, "One network to solve them all-Solving linear inverse problems using deep projection models," Proc. IEEE Int. Conf. Comp. Vision, and Pattern Recognition, pp. 5888-5897, 2017.
24. K. Gregor and Y. LeCun, "Learning fast approximations of sparse coding," Proc. Int. Conf. Machine Learning, pp. 399-406, 2010.
25. U. S. Kamilov and H. Mansour, "Learning optimal nonlinearities for iterative thresholding algorithms," IEEE Signal Process. Lett., vol. 23, no. 5, pp. 747-751, 2016.
26. M. Borgerding and P. Schniter, "Onsager-corrected deep networks for sparse linear inverse problems," arXiv preprint arXiv:1612.01183, 2016.
27. Y. Yang, J. Sun, H. Li, and Z. Xu, "Deep ADMM-net for compressive sensing MRI," Proc. Adv. in Neural Processing Systems (NIPS), vol. 29, pp. 10-18, 2016.
28. J. R. Hershey, J. L. Roux, and F. Weninger, "Deep unfolding: Model-based inspiration of novel deep architectures," arXiv preprint arXiv:1409.2574, 2014.
29. T. B. Yakar, P. Sprechmann, R. Litman, A. M. Bronstein, and G. Sapiro, "Bilevel sparse models for polyphonic music transcription." ISMIR, pp. 65-70, 2013.
30. K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, "Image denoising by sparse 3-d transform-domain collaborative filtering," IEEE Trans. Image Processing, vol. 16, no. 8, pp. 2080-2095, Aug. 2007.
31. S. V. Venkatakrishnan, C. A. Bouman, and B. Wohlberg, "Plug-and-play priors for model based reconstruction," Proc. Global Conf. on Signal and Inform. Processing (GlobalSIP), pp. 945-948, 2013.
32. G. Alain and Y. Bengio, "What regularized auto-encoders learn from the data-generating distribution," J. Machine Learning Research, vol. 15, no. 1, pp. 3563-3593, 2014.
33. C. K. Sønderby, J. Caballero, L. Theis, W. Shi, and F. Huszár, "Amortised map inference for image super-resolution," Proc. Int. Conf. on Learning Representations (ICLR), 2017.
34. D. J. Thouless, P. W. Anderson, and R. G. Palmer, "Solution of 'Solvable model of a spin glass'," Philos. Mag., vol. 35, no. 3, pp. 593-601, 1977.
35. M. Mézard and A. Montanari, Information, Physics, Computation: Probabilistic Approaches. Cambridge University Press, 2008.
36. A. Maleki, "Approximate message passing algorithm for compressed sensing," Stanford University PhD Thesis, Nov. 2010.
37. S. Ramani, T. Blu, and M. Unser, "Monte-Carlo sure: A black-box optimization of regularization parameters for general denoising algorithms," IEEE Trans. Image Processing, pp. 1540-1554, 2008.
38. H. C. Burger, C. J. Schuler, and S. Harmeling, "Image denoising: Can plain neural networks compete with BM3D?" Proc. IEEE Int. Conf. Comp. Vision, and Pattern Recognition, pp. 2392-2399, 2012.
39. K. Zhang, W. Zuo, Y. Chen, D. Meng, and L. Zhang, "Beyond a Gaussian denoiser: Residual learning of deep CNN for image denoising," IEEE Trans. Image Processing, 2017.
40. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks," Proc. Adv. in Neural Processing Systems (NIPS), pp. 1097-1105, 2012.
41. S. Ioffe and C. Szegedy, "Batch normalization: Accelerating deep network training by reducing internal covariate shift," arXiv preprint arXiv:1502.03167, 2015.
42. K. He, X. Zhang, S. Ren, and J. Sun, "Deep residual learning for image recognition," Proc. IEEE Int. Conf. Comp. Vision, and Pattern Recognition, pp. 770-778, 2016.
43. F. J. Śmieja, "Neural network constructive algorithms: Trading generalization for learning efficiency?" Circuits, Systems, and Signal Processing, vol. 12, no. 2, pp. 331-374, 1993.
44. M. Bayati and A. Montanari, "The dynamics of message passing on dense graphs, with applications to compressed sensing," IEEE Trans. Inform. Theory, vol. 57, no. 2, pp. 764-785, 2011.
45. I. Gulrajani, F. Ahmed, M. Arjovsky, V. Dumoulin, and A. Courville, "Improved training of Wasserstein GANs," arXiv preprint arXiv:1704.00028, 2017.
46. D. Martin, C. Fowlkes, D. Tal, and J. Malik, "A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics," Proc. Int. Conf. Computer Vision, vol. 2, pp. 416-423, July 2001.
47. A. Vedaldi and K. Lenc, "Matconvnet - Convolutional neural networks for MATLAB," Proc. ACM Int. Conf. on Multimedia, 2015.
48. D. Kingma and J. Ba, "Adam: A method for stochastic optimization," arXiv preprint arXiv:1412.6980, 2014.
49. E. J. Candes, X. Li, and M. Soltanolkotabi, "Phase retrieval from coded diffraction patterns," Appl. Comput. Harmon. Anal., vol. 39, no. 2, pp. 277-299, 2015.