Information-theoretic competitive learning with inverse Euclidean distance output units

Neural Processing Letters

Volumn 18, Issue 3, 2003, Pages 163-204

  Kamimura, Ryotaro ^a  

^a TOKAI UNIVERSITY   (Japan)

Author keywords

Competitive learning; Dead neurons; Inverse Euclidean distance; Mutual information maximization; Winner take all

Indexed keywords

ALGORITHMS; CHARACTER RECOGNITION; COMPUTATIONAL COMPLEXITY; DATA REDUCTION; INFORMATION THEORY; NEURAL NETWORKS; ROADS AND STREETS;

DEAD NEURONS; INFORMATION-THEORETIC COMPETITIVE LEARNING; INVERSE EUCLIDEAN DISTANCE OUTPUT UNITS; MUTUAL INFORMATION MAXIMIZATION; WINNER-TAKE-ALL;

LEARNING SYSTEMS;

EID: 0842343631     PISSN: 13704621     EISSN: None     Source Type: Journal    
DOI: 10.1023/B:NEPL.0000011136.78760.22     Document Type: Article

References (24)

1. Linsker, R.: Self-organization in a perceptual network, Computer 21 (1988), 105-117.
2. Linsker, R.: How to generate ordered maps by maximizing the mutual information between input and output, Neural Computation 1 (1989), 402-411.
3. Atick, J. J. and Redlich, A. N.: Toward a theory of early visual processing, Neural Computation 2 (1990), 308-320.
4. Becker, S.: Mutual information maximization: models of cortical self-organization, Network: Computation in Neural Systems 7 (1996), 7-31.
5. Becker, S. and Hinton, G. E.: Learning mixture models of spatial coherence, Neural Computation 5 (1993), 267-277.
6. Kamimura, R. and Nakanishi, S.: Hidden information maximization for feature detection and rule discovery, Network 6 (1995), 577-622.
7. Kamimura, R. and Nakanishi, S.: Improving generalization performance by information minimization, IEICE Transactions on Information and Systems E78-D(2) (1995), 163-173.
8. Deco, G., Finnof, W. and Zimmermann, H. G.: Unsupervised mutual information criterion for elimination of overtraining in supervised multiplayer networks, Neural Computation 7 (1995), 86-107.
9. Akiyama, Y. and Furuya, T.: An extension of the back-propagation learning which performs entropy maximization as well as error minimization, Tech. Rep. NC91-6, IEICE Technical Report, 1991.
10. Kamimura, R., Kamimura, T. and Shultz, T. R.: Information theoretic competitive learning and linguistic rule acquisition, Transactions of the Japanese Society for Artificial Intelligence 16(2) (2001), 287-298.
11. Kamimura, R., Kamimura, T. and Uchida, O.: Flexible feature discovery and structural information, Connection Science 13(4) (2001), 323-347.
12. Kamimura, R., Kamimura, T. and Takeuchi, H.: Greedy information acquisition algorithm: A new information theoretic approach to dynamic information acquisition in neural networks, Connection Science 14(2) (2002), 137-162.
13. Kamimura, R.: Progressive feature extraction by greedy network-growing algorithm, to appear in Complex Systems, 2003.
14. Rumelhart, D. E. and Zipser, D.: Feature discovery by competitive learning, In: D. E. Rumelhart and G. E. H. et al. (eds), Parallel Distributed Processing, Cambridge: MIT Press, vol. 1, pp. 151-193, 1986.
15. Grossberg, S.: Competitive learning from interactive activation to adaptive resonance, Cognitive Science 11 (1987), 23-63.
16. DeSieno, D.: Adding a conscience to competitive learning, In: Proceedings of IEEE International Conference on Neural Networks, pp. 117-124, San Diego, IEEE, 1988.
17. Ahalt, S. C., Krishnamurthy, A. K., Chen, P. and Melton, D. E.: Competitive learning algorithms for vector quantization, Neural Networks 3 (1990), 277-290.
18. Xu, L.: Rival penalized competitive learning for clustering analysis, RBF net, and curve detection, IEEE Transaction on Neural Networks 4(4) (1993), 636-649.
19. Luk, A. and Lien, S.: Properties of the generalized lotto-type competitive learning, In: Proceedings of International conference on neural information processing, pp. 1180-1185, San Mateo: CA, Morgan Kaufmann Publishers, 2000.
20. Hulle, M. M. V.: The formation of topographic maps that maximize the average mutual information of the output responses to noiseless input signals, Neural Computation 9(3) (1997), 595-606.
21. Gatlin, L. L.: Information Theory and Living Systems, Columbia University Press, 1972.
22. Hecht-Nielsen, R.: Neurocomputing, Reading, MA: Addison-Wesley, 1990.
23. McClelland, C. A. and Hoggard, G. D.: Conflict patterns in the interactions among nations, In: R. Hayes (ed.), International Politics and Foreign Policy, p. 714, New York, Free Press, 1969.
24. Kamimura, R., Kamimura, T. and Shultz, T. R.: Self-organization by information control, In: Proceedings of IASTED International Conference on Applied Informatics, pp. 188-192, Anaheim, CA, ACTA Press, 2001.