Unsupervised learning of nonlinear dependencies in natural images

International Journal of Imaging Systems and Technology

Volumn 15, Issue 1, 2005, Pages 34-47

  Park, Hyun Jin ^a   Lee, Te Won ^a  

^a UNIVERSITY OF CALIFORNIA SAN DIEGO   (United States)

Author keywords

Denoising; Dependency; Expectation maximization (EM) algorithm; Image segmentation; Independent component analysis (ICA); Nonlinear

Indexed keywords

DENOISING; DEPENDANCY; EXPECTATION MAXIMIZATION (EM) ALGORITHM; NONLINEAR;

ALGORITHMS; CORRELATION METHODS; DATA REDUCTION; IMAGE SEGMENTATION; INDEPENDENT COMPONENT ANALYSIS; LAPLACE TRANSFORMS; MATHEMATICAL MODELS; PRINCIPAL COMPONENT ANALYSIS;

IMAGE PROCESSING;

EID: 25444492274     PISSN: 08999457     EISSN: None     Source Type: Journal    
DOI: 10.1002/ima.20036     Document Type: Article

References (36)

1. H. Barlow, Sensory communication, MIT Press, Cambridge, MA, 1961, pp. 217-234.
2. A.J. Bell and T.J. Sejnowski, An information-maximization approach to blind separation and blind deconvolution, Neural Comput 7 (1995), 1129-1159.
3. A.J. Bell and T.J. Sejnowski, The independent components of natural scenes are edge filters, Vision Res 37 (1997), 3327-3338.
4. J.F. Cardoso, Infomax and maximum likelihood for blind source separation, IEEE Signal Process Lett 4 (1997), 109-111.
5. A.P. Dempster, N.M. Laird, and D.B. Rubin, Maximum likelihood from incomplete data via EM algorithm, J Royal Statist Soc 39 (1977), 1-38.
6. D.L. Donoho and I.M. Johnstone, Threshold selection for wavelet shrinkage of noisy data, In Proc 16th Annu Int Conf IEEE, Vol. 1, 1994, pp. A24-A25.
7. D.J. Field, Relations between the statistics of natural images and the response properties of cortical cells, J Opt Soc Am 4 (1987), 2379-2394.
8. D.J. Field, What is the goal of sensory coding? Neural Comput 6 (1994), 559-601.
9. G.E. Hinton and T.J. Sejnowski (Editors), Unsupervised learning: Foundations of neural computation, MIT Press, Cambridge, MA, 1999.
10. T. Hofmann, J. Puzicha, and J.M. Buhmann, Deterministic annealing for unsupervised texture segmentation, In Proc Int Workshop on Energy Minimization Methods in Comput Vis and Pattern Recognit (EMMCVPR'97), Venice, 1997, pp. 213-228.
11. A. Hyvarinen,Beyond independent component analysis, Proc 9th ICANN 99, Vol. 2, 1999, pp 809-814.
12. A. Hyvarinen and P.O. Hoyer (1997), Fast ICA matlab code, available at http://www.cis.hut.fi/projects/compneuro/extensions.html.
13. A. Hyvarinen and P.O. Hoyer, Emergence of phase and shift invariant features by decomposition of natural images into independent feature subspaces, Neural Comput 12 (2000), 1705-1720.
14. A, Hyvarinen and P.O. Hoyer, Topographic independent component analysis as a model of VI receptive fields, Neurocomputing 38-40 (2001), 1307-1315.
15. A. Hyvaerinen and E. Oja, A fast fixed-point algorithm for independent component analysis, Neural Comput 9 (1997), 1483-1492.
16. A. Hyvarinen, P.O. Hoyer, and E. Oja, Image denoising by sparse code shrinkage, In S. Haykin and B. Kosko (Editors), Intelligent signal processing, IEEE Press, 2001.
17. A. Hyvärinen, J. Karhunen, and E. Oja, Independent component analysis, John Wiley and Sons, 2002.
18. A.K. Jain, Fundamentals of digital image processing, Prentice-Hall, Englewood Cliffs, NJ, 1989.
19. T.W. Lee, Independent component analysis: Theory and applications, Kluwer Academic Publishers, Boston, 1998.
20. T.-W. Lee and M.S. Lewicki, The generalized Gaussian mixture model using ICA, InProc Int Workshop ICA, Helsinki, Finland, 19-22 June, 2000, pp. 239-244.
21. T.-W. Lee and M.S. Lewicki, Unsupervised image classification, segmentation, and enhancement using ICA mixture models, IEEE Trans Image Process 11 (2002), 270-279.
22. T.-W. Lee, M.S. Lewicki, and T.J. Sejnowski, ICA mixture models for unsupervised classification of non-Gaussian classes and automatic context switching in blind signal separation, IEEE Trans Pattern Anal Mach Intell 22 (2000), 1-12.
23. M.S. Lewicki and Y. Karklin, Learning higher-order structures in natural images, Network: Comput Neural Syst 14 (2003), 483-499.
24. M.S. Lewicki and B.A. Olshausen, Probabilistic framework for the adaptation and comparison of image codes, J Opt Soc Am 16 (1999), 1587-1601.
25. T. Mitchell, Machine learning, Carnegie Mellon University, McGraw-Hill, 1997.
26. B.A. Olshausen and D.J. Field, Emergence of simple-cell receptive-field properties by learning a sparse code for natural images, Nature 381 (1996), 607-609.
27. B.A. Olshausen and D.J. Field, Natural image statistics and efficient coding, Network 7 (1996), 333-339.
28. J. Puzicha and J.M. Buhmann,Multiscale annealing for real-time unsupervised texture segmentation, Technical Report IAI-TR-97-4, Institut fur Informatik III, University of Bonn, April 1997.
29. O. Schwartz and E.P. Simoncelli, Natural signal statistics and sensory gain control, Nat Neurosci 4 (2001), 819-825.
30. E.P. Simoncelli, Bayesian denoising of visual images in the wavelet domain, bayesian inference, In P. Muller and B. Vidakovic (Editors), Wavelet based models, Springer-Verlag, New York, 1999, pp. 291-308, Chapter 18.
31. E.P. Simoncelli and E.H. Anderson, Noise removal via Bayesian wavelet coring, In Proc 3rd IEEE Int Conf Image Process, Vol. 1,1996, pp. 379-382.
32. H. Valpola, M Harva, and J. Karhunen, Hierarchical models of variance sources, Signal Processing 84 (2004), 267-282.
33. J.H. van Hateren and A. van der Schaaf, Independent component filters of natural images compared with simple cells in primary visual cortex, Proc R Soc Lond B 265 (1998), 359-336.
34. J. Wainwright, O. Schwartz, and E.P. Simoncelli, Natural image statistics and divisive normalization: Modeling nonlinearity and adaptation in cortical neurons, In R. Rao, B. Olshausen, and M. Lewicki (Editors), Probabilistic models of the brain: Perception and neural function, MIT Press, 2002.
35. Z. Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli, Image quality assessment: From error visibility to structural similarity, IEEE Trans Image Process 13 (2004).
36. E.W. Weisstein, (1999) Laplace distribution. From MathWorld - A Wolfram Web Resource. Available at http://mathworld.wolfram.com/LaplaceDistribution. html.