Lensless computational imaging through deep learning

Optica

Volumn 4, Issue 9, 2017, Pages 1117-1125

  Sinha, Ayan ^a   Lee, Justin ^a   Li, Shuai ^a   Barbastathis, George ^a,b  

^a Massachusetts Institute of Technology   (United States)

^b SINGAPORE MIT ALLIANCE FOR RESEARCH AND TECHNOLOGY   (Singapore)

Author keywords

Computational imaging; Inverse problems; Neural networks; Pattern recognition; Phase retrieval

Indexed keywords

DIFFERENTIAL EQUATIONS; INVERSE PROBLEMS; NEURAL NETWORKS; PATTERN RECOGNITION;

COMPUTATIONAL IMAGING; DECISION TASK; END TO END; LENS-LESS IMAGING; PHASE OBJECT; PHASE RETRIEVAL;

DEEP NEURAL NETWORKS;

EID: 85029856412     PISSN: 23342536     EISSN: None     Source Type: Journal    
DOI: 10.1364/OPTICA.4.001117     Document Type: Article

References (47)

1. M. L. Minsky and S. Papert, Perceptions: An Introduction to Computational Geomry (MIT, 1969).
2. Y. LeCun, “Generalization and network design strategies,” in Connectionism in Perspective (Elsevier, 1989), pp. 143–155.
3. A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in Advances in Neural Information Processing Systems (2012), pp. 1097–1105.
4. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich, “Going deeper with convolu-tions,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2015), pp. 1–9.
5. K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2016), pp. 770–778.
6. K. Fukushima, “Cognitron: a self-organizing multilayered neural net-work,” Biol. Cybernet. 20, 121–136 (1975).
7. I. Goodfellow, Y. Bengio, and A. Courville, Deep Learning (MIT, 2016).
8. D. E. Rumelhart, G. E. Hinton, and R. J. Williams, “Learning representations by back-propagating errors,” Nature 5323, 533–536 (1986).
9. Y. A. LeCun, L. Bottou, G. B. Orr, and K.-R. Müller, “Efficient backprop,” in Neural Networks: Tricks of the Trade (Springer, 2012), pp. 9–48.
10. V. Mnih, K. Kavukcuoglu, D. Silver, A. Rusu, J. Veness, M. Bellemare, A. Graves, M. Riedmiller, A. Fidjeland, G. Ostrovski, S. Petersen, C. Beattie, A. Sadik, I. Antonoglou, H. King, D. Kumaran, D. Wierstra, S. Legg, and D. Hassabis, “Human-level control through deep reinforcement learning,” Nature 518, 529–533 (2015).
11. D. Silver, A. Huang, C. Maddison, A. Guez, L. Sifre, G. van den Driessche, J. Schrittwieser, I. Antonoglou, V. Panneershelvam, M. Lanctot, S. Dieleman, D. Grewe, J. Nham, N. Kalchbrenner, I. Sutskever, T. Lillicrap, M. Leach, K. Kavukcuoglu, T. Graepel, and D. Hassabis, “Mastering the game of go with deep neural networks and tree search,” Nature 529, 484–489 (2016).
12. A. Dosovitskiy, J. T. Springenberg, M. Tatarchenko, and T. Brox, “Learning to generate chairs, tables and cars with convolutional networks,” IEEE Trans. Pattern Anal. Mach. Intell. 39, 692–705 (2017).
13. Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” Nature 521, 436–444 (2015).
14. Z. Cheng, Q. Yang, and B. Sheng, “Deep colorization,” in Proceedings of the IEEE International Conference on Computer Vision (2015), pp. 415–423.
15. C. Dong, C. C. Loy, K. He, and X. Tang, “Learning a deep convolutional network for image super-resolution,” in European Conference on Computer Vision (Springer, 2014), pp. 184–199.
16. L. Xu, J. S. Ren, C. Liu, and J. Jia, “Deep convolutional neural network for image deconvolution,” in Advances in Neural Information Processing Systems (2014), pp. 1790–1798.
17. J. Sun, W. Cao, Z. Xu, and J. Ponce, “Learning a convolutional neural network for non-uniform motion blur removal,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2015), pp. 769–777.
18. J. Xie, L. Xu, and E. Chen, “Image denoising and inpainting with deep neural networks,” in Advances in Neural Information Processing Systems (2012), pp. 341–349.
19. R. Gerchberg and W. Saxton, “A practical algorithm for the determination of the phase from image and diffraction plane pictures,” Optik 35, 237–246 (1972).
20. J. R. Fienup, “Reconstruction of an object from the modulus of its Fourier transform,” Opt. Lett. 3, 27–29 (1978).
21. J. W. Goodman and R. Lawrence, “Digital image formation from electronically detected holograms,” Appl. Phys. Lett. 11, 77–79 (1967).
22. M. R. Teague, “Deterministic phase retrieval: a Green’s function solu-tion,” J. Opt. Soc. Am. 73, 1434–1441 (1983).
23. N. Streibl, “Phase imaging by the transport equation of intensity,” Opt. Commun. 49, 6–10 (1984).
24. M. Bertero and P. Boccacci, Introduction to Inverse Problems in Imaging (CRC Press, 1998).
25. A. N. Tikhonov, “On the stability of inverse problems,” Dokl. Akad. Nauk SSSR 39, 195–198 (1943).
26. N. Wiener, Extrapolation, Interpolation, and Smoothing of Stationary Time Series (MIT, 1949), Vol. 7.
27. E. J. Candès and M. B. Wakin, “An introduction to compressive sam-pling,” IEEE Signal Process. Mag. 25, 21–30 (2008).
28. D. J. Brady, Optical Imaging and Spectroscopy (Wiley, 2009).
29. D. L. Donoho and M. Elad, “Optimally sparse representation in general (nonorthogonal) dictionaries via l1 minimization,” Proc. Natl. Acad. Sci. USA 100, 2197–2202 (2003).
30. H. Ji and K. Wang, “Robust image deblurring with an inaccurate blur kernel,” IEEE Trans. Image Process. 21, 1624–1634 (2012).
31. J. J. Hopfield, “Neural networks and physical systems with emergent collective computational abilities,” Proc. Natl. Acad. Sci. USA 79, 2554–2558 (1982).
32. J.-S. Jang, S.-W. Jung, S.-Y. Lee, and S.-Y. Shin, “Optical implementation of the Hopfield model for two-dimensional associative memory,” Opt. Lett. 13, 248–250 (1988).
33. R. Horisaki, R. Takagi, and J. Tanida, “Learning-based imaging through scattering media,” Opt. Express 24, 13738–13743 (2016).
34. S. Popoff, G. Lerosey, R. Carminati, M. Fink, A. Boccara, and S. Gigan, “Measuring the transmission matrix in optics: an approach to the study and control of light propagation in disordered media,” Phys. Rev. Lett. 104, 100601 (2010).
35. S. Popoff, G. Lerosey, M. Fink, A. C. Boccara, and S. Gigan, “Image transmission through an opaque material,” arXiv:1005.0532 (2010).
36. U. S. Kamilov, I. N. Papadopoulos, M. H. Shoreh, A. Goy, C. Vonesch, M. Unser, and D. Psaltis, “Learning approach to optical tomography,” Optica 2, 517–522 (2015).
37. Y. Jo, S. Park, J. Jung, J. Yoon, H. Joo, M.-H. Kim, S.-J. Jo, M. C. Choi, S. Y. Lee, and Y. Park, “Holographic deep learning for rapid optical screening of anthrax spores,” BioRxiv (2017), 109108.
38. K. H. Jin, M. T. McCann, E. Froustey, and M. Unser, “Deep convolutional neural network for inverse problems in imaging,” IEEE Trans. Image Process. 26, 4509–4522 (2017).
39. Y. Rivenson, Y. Zhang, H. Gunaydin, D. Teng, and A. Ozcan, “Phase recovery and holographic image reconstruction using deep learning in neural networks,” arXiv:1705.04286 (2017).
40. Y. Rivenson, Z. Gorocs, H. Gunaydin, Y. Zhang, H. Wang, and A. Ozcan, “Deep learning microscopy,” arXiv:1705.04709 (2017).
41. A. Sinha, J. Lee, S. Li, and G. Barbastathis, “Lensless computational imaging through deep learning,” arXiv:1702.08516 (2017).
42. O. Russakovsky, J. Deng, H. Su, J. Krause, S. Satheesh, S. Ma, Z. Huang, A. Karpathy, A. Khosla, and M. Bernstein, “Imagenet large scale visual recognition challenge,” Int. J. Comput. Vis. 115, 211–252 (2015).
43. G. B. Huang, M. Ramesh, T. Berg, and E. Learned-Miller, “Labeled faces in the wild: a database for studying face recognition in unconstrained environments,” Technical Report (University of Massachusetts, 2007).
44. F. Yu and V. Koltun, “Multi-scale context aggregation by dilated con-volutions,” in International Conference on Learning Representations (2016).
45. A. Krizhevsky and G. Hinton, “Learning multiple layers of features from tiny images,” Technical Report (University of Toronto, 2009).
46. “AT&T database of faces,” Technical Report (AT&T Laboratories Cambridge).
47. M. D. Zeiler and R. Fergus, “Visualizing and understanding convolutional networks,” in European Conference on Computer Vision (Springer, 2014), pp. 818–833.