Feature extraction through LOCOCODE

Neural Computation

Volumn 11, Issue 3, 1999, Pages 679-714

  Hochreiter, Sepp ^a   Schmidhuber, Jürgen ^b  

^a TECHNICAL UNIVERSITY OF MUNICH   (Germany)

^b DALLE MOLLE INSTITUTE FOR ARTIFICIAL INTELLIGENCE IDSIA   (Switzerland)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHM; ARTIFICIAL NEURAL NETWORK; AUTOMATED PATTERN RECOGNITION; LEARNING; PHYSIOLOGY; REVIEW; SENSORY NERVE CELL;

ALGORITHMS; LEARNING; NEURAL NETWORKS (COMPUTER); NEURONS, AFFERENT; PATTERN RECOGNITION, AUTOMATED;

EID: 0033114102     PISSN: 08997667     EISSN: None     Source Type: Journal    
DOI: 10.1162/089976699300016629     Document Type: Article

References (64)

1. Amari, S., Cichocki, A., & Yang, H. (1996). A new learning algorithm for blind signal separation. In D. S. Touretzky, M. C. Mozer, and M. E.Hasselmo (Eds.), Advances in neural information processing systems, 8 (pp. 757-763). Cambridge, MA: MIT Press.
2. Baldi, P., & Hornik, K. (1989). Neural networks and principal component analysis: Learning from examples without local minima. Neural Networks, 2, 53-58.
3. Barlow, H. B. (1983). Understanding natural vision. Berlin: Springer-Verlag.
4. Barlow, H. B., Kaushal, T. P., & Mitchison, G. J. (1989). Finding minimum entropy codes. Neural Computation, 1(3), 412-423.
5. Barrow, H. G. (1987). Learning receptive fields. In Proceedings of the IEEE 1st Annual Conference on Neural Networks (Vol. 4, pp. 115-121). New York: IEEE.
6. Baumgartner, M. (1996). Bilddatenvorverarbeitung mit neuronalen Netzen. Diploma thesis, Institut für Informatik, Technische Universität München.
7. Becker, S. (1991). Unsupervised learning procedures for neural networks. International Journal of Neural Systems, 2(1 & 2), 17-33.
8. Bell, A. J., & Sejnowski, T. J. (1995). An information-maximization approach to blind separation and blind deconvolution. Neural Computation, 7(6), 1129-1159.
9. Cardoso, J.-F., & Souloumiac, A. (1993). Blind beamforming for non Gaussian signals. IEE Proceedings-F, 140(6), 362-370.
10. Comon, P. (1994). Independent component analysis - A new concept? Signal Processing, 36(3), 287-314.
11. Dayan, P., & Zemel, R. (1995). Competition and multiple cause models. Neural Computation, 7, 565-579.
12. Deco, G., & Parra, L. (1994). Nonlinear features extraction by unsupervised redundancy reduction with a stochastic neural network (Tech. Rep. ZFE ST SN 41). Munich: Siemens AG.
13. DeMers, D., & Cottrell, G. (1993). Non-linear dimensionality reduction. In S. J. Hanson, J. D. Cowan, & C. L. Giles (Eds.), Advances in neural information processing systems, 5 (pp. 580-587). San Mateo, CA: Morgan Kaufmann.
14. Field, D. J. (1987). Relations between the statistics of natural images and the response properties of cortical cells. Journal of the Optical Society of America, 4, 2379-2394.
15. Field, D. J. (1994). What is the goal of sensory coding? Neural Computation, 6, 559-601.
16. Flake, G. W. (1998). Square unit augmented, radially extended, multilayer perceptrons. In G. B. Orr & K.-R. Müller (Eds.), Tricks of the trade. Berlin: Springer-Verlag. Lecture Notes in Computer Science.
17. Földiák, P. (1990). Forming sparse representations by local anti-hebbian learning. Biological Cybernetics, 64, 165-170.
18. Földiák, P., & Young, M. P. (1995). Sparse coding in the primate cortex. In M. A. Arbib (Ed.), The handbook of brain theory and neural networks (pp. 895-898). Cambridge, MA: MIT Press.
19. Ghahramani, Z. (1995). Factorial learning and the EM algorithm. In G. Tesauro, D. Touretzky, & T. Leen (Eds.), Advances in neural information processing systems, 7 (pp. 617-624). Cambridge, MA: MIT Press.
20. Hassibi, B., & Stork, D. G. (1993). Second order derivatives for network pruning: Optimal brain surgeon. In S. J. Hanson, J. D. Cowan, & C. L. Giles (Eds.), Advances in neural information processing systems, 5 (pp. 164-171). San Mateo, CA: Morgan Kaufmann.
21. Hastie, T. J., & Tibshirani, R. J. (1996). Discriminant adaptive nearest neighbor classification. IEEE Transactions on Pattern Analysis and Machine Intelligence, 18(6), 607-616.
22. Hastie, T. J., Tibshirani, R. J., & Buja, A. (1993). Flexible discriminant analysis by optimal scoring (Tech. Rep.). Murray Hill, NJ: AT&T Bell Laboratories.
23. Herrmann, M. (1997). On the merits of topography in neural maps. In T. Kohonen (Ed.), Proceedings of the Workshop on Self-Organizing Maps (pp. 112-117). Helsinki: Helsinki University of Technology.
24. Hinton, G. E., Dayan, P., Frey, B. J., & Neal, R. M. (1995). The wake-sleep algorithm for unsupervised neural networks. Science, 268, 1158-1161.
25. Hinton, G. E., & Ghahramani, Z. (1997). Generative models for discovering sparse distributed representations. Philosophical Transactions of the Royal Society B, 352, 1177-1190.
26. Hinton, G. E., & Zemel, R. S. (1994). Autoencoders, minimum description length and Helmholtz free energy. In J. D. Cowan, G. Tesauro, & J. Alspector (Eds.), Advances in neural information processing systems, 6 (pp. 3-10). San Mateo, CA: Morgan Kaufmann.
27. Hochreiter, S., & Schmidhuber, J. (1995). Simplifying nets by discovering flat minima. In G. Tesauro, D. S. Touretzky, & T. K. Leen (Eds.), Advances in neural information processing systems, 7, pp. 529-536. Cambridge, MA: MIT Press.
28. Hochreiter, S., & Schmidhuber, J. (1997a). Flat minima. Neural Computation, 9(1), 1-42.
29. Hochreiter, S., & Schmidhuber, J. (1997b). Low-complexity coding and decoding. In K. M. Wong, I. King, & D. Yeung (Eds.), Theoretical aspects of neural computation (TANC 97), Hong Kong (pp. 297-306). Berlin: Springer-Verlag.
30. Hochreiter, S., & Schmidhuber, J. (1997c). Unsupervised coding with Lococode. In W. Gerstner, A. Germond, M. Hasler, & J.-D. Nicoud (Eds.), Proceedings of the International Conference on Artificial Neural Networks, Lausanne, Switzerland (pp. 655-660). Berlin: Springer-Verlag.
31. Hochreiter, S., & Schmidhuber, J. (1998). Lococode versus PCA and ICA. In Proceedings of the International Conference on Artificial Neural Networks.
32. Jutten, C., & Herault, J. (1991). Blind separation of sources, part I: An adaptive algorithm based on neuromimetic architecture. Signal Processing, 24(1), 1-10.
33. Kohonen, T. (1988). Self-organization and associative memory. (2nd ed.) Berlin: Springer-Verlag.
34. Kramer, M. (1991). Nonlinear principal component analysis using autoassociative neural networks. AIChE Journal, 37, 233-243.
35. Li, Z. (1995). A theory of the visual motion coding in the primary visual cortex. Neural Computation, 8(4), 705-730.
36. Linsker, R. (1988). Self-organization in a perceptual network. IEEE Computer, 21, 105-117.
37. Molgedey, L., & Schuster, H. G. (1994). Separation of independent signals using time-delayed correlations. Phys. Reviews Letters, 72(23), 3634-3637.
38. Mozer, M. C. (1991). Discovering discrete distributed representations with iterative competitive learning. In R. P. Lippmann, J. E. Moody, & D. S. Touretzky (Eds.), Advances in neural information processing systems, 3 (pp. 627-634). San Mateo, CA: Morgan Kaufmann.
39. Nadal, J.-P., & Parga, N. (1997). Redundancy reduction and independent component analysis: Conditions on cumulants and adaptive approaches. Neural Computation, 9(7), 1421-1456.
40. Oja, E. (1989). Neural networks, principal components, and subspaces. International Journal of Neural Systems, 1(1), 61-68.
41. Oja, E. (1991). Data compression, feature extraction, and autoassociation in feed-forward neural networks. In T. Kohonen, K. Mäkisara, O. Simula, & J. Kangas (Eds.), Artificial neural networks, 1 (pp. 737-745). Amsterdam: Elsevier Science.
42. Olshausen, B. A., & Field, D. J. (1996). Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature, 381(6583), 607-609.
43. Pajunen, P. (1998). Blind source separation using algorithmic information theory. In C. Fyfe (Ed.), Proceedings of Independence and Artificial Neural Networks (I&ANN) (pp. 26-31). Tenerife, Spain: ICSC Academic Press.
44. Palm, G. (1992). On the information storage capacity of local learning rules. Neural Computation, 4(2), 703-711.
45. Redlich, A. N. (1993). Redundancy reduction as a strategy for unsupervised learning. Neural Computation, 5, 289-304.
46. Rissanen, J. (1978). Modeling by shortest data description. Automatica, 14, 465-471.
47. Robinson, A. J. (1989). Dynamic error propagation networks. Unpublished doctoral dissertation, Trinity Hall and Cambridge University.
48. Rumelhart, D. E., & Zipser, D. (1986). Feature discovery by competitive learning. In Parallel distributed processing (pp. 151-193). Cambridge, MA: MIT Press.
49. Saund, E. (1994). Unsupervised learning of mixtures of multiple causes in binary data. In J. D. Cowan, G. Tesauro, & J. Alspector (Eds.), Advances in neural information processing systems, 6 (pp. 27-34). San Mateo, CA: Morgan Kaufmann.
50. Saund, E. (1995). A multiple cause mixture model for unsupervised learning. Neural Computation, 7(1), 51-71.
51. Schmidhuber, J. (1992). Learning factorial codes by predictability minimization. Neural Computation, 4(6), 863-879.
52. Schmidhuber, J. (1997a). Discovering neural nets with low Kolmogorov complexity and high generalization capability. Neural Networks, 10(5), 857-873.
53. Schmidhuber, J. (1997b). Low-complexity art. Leonardo, Journal of the International Society for the Arts, Sciences, and Technology, 30(2), 97-103.
54. Schmidhuber, J., Eldracher, M., & Foltin, B. (1996). Semilinear predictability minimization produces well-known feature detectors. Neural Computation, 8(4), 773-786.
55. Schmidhuber, J., & Prelinger, D. (1993). Discovering predictable classifications. Neural Computation, 5(4), 625-635.
56. Schraudolph, N. N. (1998). On centering neural network weight updates. In G. B. Orr & K.-R. Müller (Eds.), Tricks of the trade. Berlin: Springer-Verlag.
57. Schraudolph, N. N., & Sejnowski, T. J. (1993). Unsupervised discrimination of clustered data via optimization of binary information gain. In S. J. Hanson, J. D. Cowan, & C. L. Giles (Eds.), Advances in neural information processing systems, 5 (pp. 499-506). San Mateo, CA: Morgan Kaufmann.
58. Solomonoff, R. (1964). A formal theory of inductive inference. Part I. Information and Control, 7, 1-22.
59. Tenenbaum, J. B., & Freeman, W. T. (1997). Separating style and content. In M. C. Mozer, M. I. Jordan, & T. Petsche (Eds.), Advances in neural information processing systems, 9 (pp. 662-668). Cambridge, MA: MIT Press.
60. Turney, P. D. (1993). Exploiting context when learning to classify. In Proceedings of the European Conference on Machine Learning (pp. 402-407). Available from: ftp://ai.iit.nrc.ca/pub/ksl-papers/NRC-35058.ps.Z.
61. Wallace, C. S., & Boulton, D. M. (1968). An information theoretic measure for classification. Computer Journal, 11(2), 185-194.
62. Watanabe, S. (1985). Pattern recognition: Human and mechanical. New York: Wiley.
63. Zemel, R. S. (1993). A minimum description length framework for unsupervised learning. Unpublished doctoral dissertation, University of Toronto.
64. Zemel, R. S., & Hinton, G. E. (1994). Developing population codes by minimizing description length. In J. D. Cowan, G. Tesauro, & J. Alspector (Eds.), Advances in neural information processing systems, 6 (pp. 11-18). San Mateo, CA: Morgan Kaufmann.