Density modeling of images using a generalized normalization transformation

4th International Conference on Learning Representations, ICLR 2016 - Conference Track Proceedings

Volumn , Issue , 2016, Pages

  Ballé, Johannes ^a   Laparra, Valero ^a   Simoncelli, Eero P ^a  

^a NEW YORK UNIVERSITY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

LINEAR TRANSFORMATIONS; MATHEMATICAL TRANSFORMATIONS; METADATA; NETWORK ARCHITECTURE;

DENSITY MODELING; GAUSSIANIZATION; LINEAR TRANSFORM; MUTUAL INFORMATIONS; NON-LINEAR TRANSFORMATIONS; OPTIMIZED TRANSFORMATION; PRIOR PROBABILITY; UNSUPERVISED METHOD;

ADDITIVE NOISE;

EID: 85083951049     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (43)

1. Ballé, Johannes and Simoncelli, Eero P. Learning sparse filterbank transforms with convolutional ICA. In 2014 IEEE International Conference on Image Processing (ICIP), 2014. doi: 10.1109/ICIP.2014.7025815.
2. Barlow, Horace B. Possible principles underlying the transformations of sensory messages. In Sensory Communication, pp. 217–234. M.I.T. Press, 1961. ISBN 978-0-262-51842-0.
3. Bell, Anthony J. and Sejnowski, Terrence J. The independent components of natural scenes are edge filters. Vision Research, 37(23), 1997. doi: 10.1016/S0042-6989(97)00121-1.
4. Carandini, Matteo and Heeger, David J. Normalization as a canonical neural computation. Nature Reviews Neuroscience, 13, January 2012. doi: 10.1038/nrn3136.
5. Cardoso, Jean-François. Dependence, correlation and Gaussianity in independent component analysis. Journal of Machine Learning Research, 4:1177–1203, 2003. ISSN 1533-7928.
6. Chen, Scott Saobing and Gopinath, Ramesh A. Gaussianization. In Advances in Neural Information Processing Systems 13, pp. 423–429, 2000.
7. Culpepper, B. J., Sohl-Dickstein, J., and Olshausen, Bruno. Building a better probabilistic model of images by factorization. In 2011 IEEE International Conference on Computer Vision (ICCV), 2011. doi: 10.1109/ICCV.2011.6126473.
8. Dinh, Laurent, Krueger, David, and Bengio, Yoshua. NICE: Non-linear independent components estimation. arXiv e-prints, 2014. Published as a conference paper at the 3rd International Conference for Learning Representations, San Diego, 2015.
9. Figueiredo, M. A. T. and Nowak, R. D. Wavelet-based image estimation: an empirical bayes approach using Jeffrey’s noninformative prior. IEEE Transactions on Image Processing, 10(9), September 2001. doi: 10.1109/83.941856.
10. Friedman, Jerome H., Stuetzle, Werner, and Schroeder, Anne. Projection pursuit density estimation. Journal of the American Statistical Association, 79(387), 1984. doi: 10.1080/01621459.1984. 10478086.
11. Guerrero-Colón, J. A., Simoncelli, Eero P., and Portilla, Javier. Image denoising using mixtures of Gaussian scale mixtures. In 15th IEEE International Conference on Image Processing, 2008, 2008. doi: 10.1109/ICIP.2008.4711817.
12. Heeger, David J. Normalization of cell responses in cat striate cortex. Visual Neuroscience, 9(2), 1992. doi: 10.1017/S0952523800009640.
13. Hyvärinen, Aapo. Estimation of non-normalized statistical models by score matching. Journal of Machine Learning Research, 6:695–709, 2005. ISSN 1533-7928.
14. Hyvärinen, Aapo and Hoyer, Patrik. Emergence of phase- and shift-invariant features by decomposition of natural images into independent feature subspaces. Neural Computation, 12(7), 2000. doi: 10.1162/089976600300015312.
15. Hyvärinen, Aapo, Hoyer, Patrik, and Inki, Mika. Topographic independent component analysis. Neural Computation, 13(7), 2001. doi: 10.1162/089976601750264992.
16. Jarrett, Kevin, Kavukcuoglu, Koray, Ranzato, Marc’Aurelio, and LeCun, Yann. What is the best multi-stage architecture for object recognition? In 2009 IEEE 12th International Conference on Computer Vision, 2009. doi: 10.1109/ICCV.2009.5459469.
17. Jolliffe, I. T. Principal Component Analysis. Springer, 2 edition, 2002. ISBN 978-0-387-95442-4.
18. Kingma, Diederik P. and Ba, Jimmy Lei. Adam: A method for stochastic optimization. arXiv e-prints, 2014. Published as a conference paper at the 3rd International Conference for Learning Representations, San Diego, 2015.
19. Köster, Urs and Hyvärinen, Aapo. A two-layer model of natural stimuli estimated with score matching. Neural Computation, 22(9), 2010. doi: 10.1162/NECO_a_00010.
20. Laparra, Valero, Camps-Valls, Gustavo, and Malo, Jesús. Iterative Gaussianization: From ICA to random rotations. IEEE Transactions on Neural Networks, 22(4), April 2011. doi: 10.1109/TNN. 2011.2106511.
21. LeCun, Yann, Matan, O., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hubbard, W., Jackel, L. D., and Baird, H. S. Handwritten zip code recognition with multilayer networks. In Proceedings, 10th International Conference on Pattern Recognition, volume 2, 1990. doi: 10. 1109/ICPR.1990.119325.
22. Lyu, Siwei. Divisive normalization: Justification and effectiveness as efficient coding transform. In Advances in Neural Information Processing Systems 23, pp. 1522–1530, 2010.
23. Lyu, Siwei and Simoncelli, Eero P. Nonlinear image representation using divisive normalization. In 2008 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), June 2008. doi: 10.1109/CVPR.2008.4587821.
24. Lyu, Siwei and Simoncelli, Eero P. Modeling multiscale subbands of photographic images with fields of Gaussian scale mixtures. IEEE Transactions on Pattern Analysis and Machine Intelligence, 31(4), April 2009a. doi: 10.1109/TPAMI.2008.107.
25. Lyu, Siwei and Simoncelli, Eero P. Nonlinear extraction of independent components of natural images using radial Gaussianization. Neural Computation, 21(6), 2009b. doi: 10.1162/neco. 2009.04-08-773.
26. Malo, Jesús and Laparra, Valero. Psychophysically tuned divisive normalization approximately factorizes the PDF of natural images. Neural Computation, 22(12), 2010. doi: 10.1162/NECO_ a_00046.
27. Malo, Jesús, Epifanio, I., Navarro, R., and Simoncelli, Eero P. Non-linear image representation for efficient perceptual coding. IEEE Transactions on Image Processing, 15(1), January 2006. doi: 10.1109/TIP.2005.860325.
28. Miyasawa, K. An empirical bayes estimator of the mean of a normal population. Bulletin de l’Institut international de Statistique, 38:181–188, 1961.
29. Portilla, Javier, Strela, Vasily, Wainwright, Martin J., and Simoncelli, Eero P. Image denoising using scale mixtures of Gaussians in the wavelet domain. IEEE Transactions on Image Processing, 12 (11), November 2003. doi: 10.1109/TIP.2003.818640.
30. Raphan, Martin and Simoncelli, Eero P. Least squares estimation without priors or supervision. Neural Computation, 23(2), February 2011. doi: 10.1162/NECO_a_00076.
31. Rieke, F., Bodnar, D. A., and Bialek, W. Naturalistic stimuli increase the rate and efficiency of information transmission by primary auditory afferents. Proceedings of the Royal Society of London B: Biological Sciences, 262(1365), 1995. doi: 10.1098/rspb.1995.0204.
32. Ruderman, Daniel L. The statistics of natural images. Network: Computation in Neural Systems, 5, 1994. doi: 10.1088/0954-898X_5_4_006.
33. Schwartz, Odelia and Simoncelli, Eero P. Natural signal statistics and sensory gain control. Nature Neuroscience, 4(8), August 2001. doi: 10.1038/90526.
34. Simoncelli, Eero P. and Freeman, William T. The steerable pyramid: A flexible architecture for multi-scale derivative computation. In 1995 IEEE International Conference on Image Processing (ICIP), volume 3, 1995. doi: 10.1109/ICIP.1995.537667.
35. p-nested symmetric distributions. Journal of Machine Learning Research, 11:3409–3451, 2010. ISSN 1533-7928.
36. Sinz, Fabian and Bethge, Matthias. What is the limit of redundancy reduction with divisive normalization? Neural Computation, 25(11), 2013. doi: 10.1162/NECO_a_00505.
37. Theis, Lucas and Bethge, Matthias. Generative image modeling using spatial LSTMs. In Advances in Neural Information Processing Systems 28, pp. 1918–1926, 2015.
38. Theis, Lucas, Hosseini, Reshad, and Bethge, Matthias. Mixtures of conditional Gaussian scale mixtures applied to multiscale image representations. PLoS one, 7(7), 2012. doi: 10.1371/journal.pone.0039857.
39. Theis, Lucas, van den Oord, Aäron, and Bethge, Matthias. A note on the evaluation of generative models. arXiv e-prints, 2015. Under review as a conference paper at the 4th International Conference for Learning Representations, San Juan, 2016.
40. van Hateren, J. H. and van der Schaaf, A. Independent component filters of natural images compared with simple cells in primary visual cortex. Proceedings of the Royal Society of London B: Biological Sciences, 265(1394), 1998. doi: 10.1098/rspb.1998.0303.
41. Wainwright, Martin J. and Simoncelli, Eero P. Scale mixtures of Gaussians and the statistics of natural images. In Advances in Neural Information Processing Systems 12, pp. 855–861, 2000.
42. Wang, Zhou, Bovik, Alan Conrad, Sheikh, H. R., and Simoncelli, Eero P. Image quality assessment: From error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4), April 2004. doi: 10.1109/TIP.2003.819861.
43. Zoran, Daniel and Weiss, Yair. Natural images, Gaussian mixtures and dead leaves. In Advances in Neural Information Processing Systems 25, pp. 1736–1744, 2012.