A hierarchical Bayesian model for learning nonlinear statistical regularities in nonstationary natural signals

Neural Computation

Volumn 17, Issue 2, 2005, Pages 397-423

  Karklin, Yan ^a   Lewicki, Michael S ^a  

^a Carnegie Mellon University   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; BAYES THEOREM; LEARNING; NONLINEAR SYSTEM;

BAYES THEOREM; LEARNING; NONLINEAR DYNAMICS;

EID: 13244255412     PISSN: 08997667     EISSN: None     Source Type: Journal    
DOI: 10.1162/0899766053011474     Document Type: Article

References (33)

1. Bell, A. J., & Sejnowski, T. J. (1995). An information maximization approach to blind separation and blind deconvolution. Neural Computation, 7(6), 1129-1159.
2. Bell, A. J., & Sejnowski, T. J. (1997). The "independent components" of natural scenes are edge filters. Vision Res., 37(23), 3327-3338.
3. Bollerslev, T., Engle, R. F., & Nelson, D. B. (1994). ARCH models. In R. F. Engle & D. L. McFadden (Eds.), Handbook of Econometrics. Amsterdam: Elsevier.
4. Buccigrossi, R. W., & Simoncelli, E. P. (1999). Image compression via joint statistical characterization in the wavelet domain. IEEE Transactions on Image Processing, 8(12), 1688-1701.
5. Cardoso, J.-F. (1997). Infomax and maximum likelihood for blind source separation. IEEE Signal Processing Letters, 4, 109-111.
6. Choi, S., Cichocki, A., & Belouchrani, A. (2002). Second order nonstationary source separation. Journal of VLSI Signal Processing, 32(1-2), 93-104.
7. Cichocki, A., & Amari, S.-I. (2002). Adaptive blind signal and image processing: Learning algorithms and applications. New York, Wiley.
8. Everson, R., & Roberts, S. (1999). Non-stationary independent component analysis. In Proceedings of the 8th International Conference on Artificial Neural Networks (pp. 503-508). Berlin: Springer-Verlag.
9. Foldiak, P. (1991). Learning invariance from transformation sequences. Neural Computation, 3(2), 194-200.
10. Hoyer, P. O., & Hyvärinen, A. (2002). A multi-layer sparse coding network learns contour coding from natural images. Vision Res., 42, 1593-1605.
11. Hurri, J., & Hyvärinen, A. (2003). Simple-cell-like receptive fields maximize temporal coherence in natural video. Neural Computation, 15(3), 663-691.
12. Hyvärinen, A., & Hoyer, P. (2000). Emergence of phase- and shift-invariant features by decomposition of natural images into independent feature subspaces. Neural Computation, 12, 1705-1720.
13. Hyvärinen, A., Hoyer, P. O., & Inki, M. (2001). Topographic independent component analysis. Neural Comput., 13, 1527-1558.
14. Hyvärinen, A., Hurri, J., & Väyrynen, J. (2003). Bubbles: A unifying framework for low-level statistical properties of natural image sequences. Journal of the Optical Society of America A, 20(7), 1237-1252.
15. Hyvärinen, A., Karhunen, J., & Oja, E. (2001). Independent component analysis. New York: Wiley.
16. Karklin, Y., & Lewicki, M. (2003). Learning higher-order structures in natural images. Network: Computation in Neural Systems, 14, 483-499.
17. Kayser, C., Einhäuser, W., Dümmer, O., König, P., & Körding, K. (2001). Extracting slow subspaces from natural videos leads to complex cells. Artificial Neural Networks, 2130, 1075-1080.
18. Krüger, N. (1998). Collinearity and parallelism are statistically significant second order relations of complex cell responses. Neural Processing Letters, 8, 117-129.
19. LeCun, Y., Bottou, L., Orr, G., & Müller, K. (1998). Efficient backprop. In G. Orr & K. Müller (Eds.), Neural networks: tricks of the trade. New York: Springer.
20. Lee, T.-W., Koehler, B., & Orglmeister, R. (1997). Blind source separation of nonlinear mixing models. In Proceedings of IEEE International Workshop on Neural Networks for Signal Processing.
21. Lee, T.-W., & Lewicki, M. S. (2002). Unsupervised classification, segmentation and de-noising of images using ICA mixture models. IEEE Trans. Image Proc., 11 (3), 270-279.
22. Lee, T.-W., Lewicki, M. S., & Sejnowski, T. J. (2000). ICA mixture models for unsupervised classification of non-Gaussian sources and automatic context switching in blind signal separation. IEEE Trans. on Pattern Analysis and Machine Intelligence, 22(10), 1078-1089.
23. Olshausen, B. A., & Field, D. J. (1996). Emergence of simple-cell receptive-field properties by learning a sparse code for natural images. Nature, 381, 607-609.
24. O'Neill, J. C. (1999). Discrete TFDs time-frequency analysis software. Available online at: http://tfd.sourceforge.net/.
25. Pearlmutter, B. A., & Parra, L. C. (1996). A context-sensitive generalization of ICA. In International Conference on Neural Information Processing (pp. 151-157). New York: Springer.
26. Pham, D.-T., & Cardoso, J.-F. (2001). Blind separation of instantaneous mixtures of non stationary sources. IEEE Trans. Signal Processing, 49(9), 1837-1848.
27. Romberg, J., Choi, H., & Baraniuk, R. (2001). Bayesian tree-structured image modeling using wavelet domain Hidden Markov models. IEEE Transactions on Image Processing, 10(7), 1056-1068.
28. Schwartz, O., & Simoncelli, E. P. (2001). Natural signal statistics and sensory gain control. Nat. Neurosci., 4, 819-825.
29. Simoncelli, E. P. (1997). Statistical models for images: Compression, restoration and synthesis. In Proc. 31st Asilomar Conference on Signals, Systems and Computers. Pacific Grove, CA.
30. van Hateren, J. H., & van der Schaaf, A. (1998). Independent component filters of natural images compared with simple cells in primary visual cortex. Proc. Royal Soc. Lond. B, 265, 359-366.
31. Wainwright, M. J., Simoncelli, E. P., & Willsky, A. S. (2001). Random cascades on wavelet trees and their use in analyzing and modeling natural images. Applied Computational and Harmonic Analysis, 11, 89-123.
32. Welling, M., Hinton, G. E., & Osindero, S. (2003). Learning sparse topographic representations with products of student-t distributions. In S. Decker, S. Thrun, & K. Obermayer (Eds.), Advances in neural information processing systems, 15, Cambridge, MA: MIT Press.
33. Wiskott, L., & Sejnowski, T. J. (2002). Slow feature analysis: Unsupervised learning of invariances. Neural Computation, 14(4), 715-770.