Improving information-theoretic competitive learning by accentuated information maximization

International Journal of General Systems

Volumn 34, Issue 3, 2005, Pages 219-233

  Kamimura, Ryotaro ^a  

^a TOKAI UNIVERSITY   (Japan)

Author keywords

Accentuated information maximization; Competitive learning; Mutual information maximization; Rth power; Winner take all

Indexed keywords

ACCENTUATED INFORMATION MAXIMIZATION; COMPETITIVE LEARNING; MUTUAL INFORMATION MAXIMIZATION; RTH POWER; WINNER-TAKE-ALL;

COMPUTATION THEORY; DATA REDUCTION; INDUSTRIAL ECONOMICS; LEARNING SYSTEMS; PROBLEM SOLVING;

INFORMATION THEORY;

EID: 27344446063     PISSN: 03081079     EISSN: 15635104     Source Type: Journal    
DOI: 10.1080/03081070500065759     Document Type: Review

References (28)

1. S.C. Ahalt, A.K. Krishnamurthy, P. Chen and D.E. Melton, "Competitive learning algorithms for vector quantization", Neural Netw., 3, 1990, pp. 277-290.
2. J.J. Atick and A.N. Redlich, "Toward a theory of early visual processing", Neural Comput., 2, 1990, pp. 308-320.
3. H.B. Barlow, "Unsupervised learning", Neural Comput., 1, 1989, pp. 295-311.
4. H.B. Barlow, T.P. Kaushal and G.J. Mitchison, "Finding minimum entropy code", Neural Comput., 1, 1989, pp. 412-423.
5. S. Becker, "Mutual information maximization: Models of cortical self-organization", Netw. Comput. Neural Syst., 7, 1996, pp. 7-31.
6. S. Becker and G.E. Hinton, "Learning mixture models of spatial coherence", Neural Comput., 5, 1993, pp. 267-277.
7. G. Deco, W. Finnof and H.G. Zimmermann, "Unsupervised mutual information criterion for elimination of overtraining in supervised multiplayer networks", Neural Comput., 7, 1995, pp. 86-107.
8. D. DeSieno, "Adding a conscience to competitive learning", Proceedings of IEEE International Conference on Neural Networks, San Diego: IEEE, 1988, pp. 117-124.
9. K. Fukushima, "Cognitron: A self-organizing multi-layered neural network", Biol. Cybern., 20, 1975, pp. 121-136.
10. L.L. Gatlin, Information Theory and Living Systems, Columbia University Press, 1972.
11. S. Grossberg, "Competitive learning: From interactive activation to adaptive resonance", Cogn. Sci., 11, 1987, pp. 23-63.
12. M.M.V. Hulle, "The formation of topographic maps that maximize the average mutual information of the output responses to noiseless input signals", Neural Comput., 9(3), 1997, pp. 595-606.
13. R. Kamimura, "Minimizing α-information for generalization and interpretation", Algorithmica, 22, 1998, pp. 173-197.
14. R. Kamimura, "Information-theoretic competitive learning with inverse euclidean distance", Neural Process. Lett., 2003a (To appear).
15. R. Kamimura, "Progressive feature extraction by greedy network-growing algorithm", Complex Syst., 14(2), pp. 127-153, 2003b.
16. R. Kamimura and S. Nakanishi, "Hidden information maximization for feature detection and rule discovery", Network, 6, 1995a, pp. 577-622.
17. R. Kamimura and S. Nakanishi, "Improving generalization performance by information minimization", IEICE Trans. Inf. Syst., E78-D(2), 1995b, pp. 163-173.
18. R. Kamimura, T. Kamimura and T.R. Shultz, "Information theoretic competitive learning and linguistic rule acquistion", Trans. Jpn. Soc. Artif. Intell., 16(2), 2001a, pp. 287-298.
19. R. Kamimura, T. Kamimura and O. Uchida, "Flexible feature discovery and structural information", Connect. Sci., 13(4), 2001b, pp. 323-347.
20. R. Kamimura, T. Kamimura and H. Takeuchi, "Greedy information acquisition algorithm: A new information theoretic approach to dynamic information acquisition in neural networks", Connect. Sci., 14(2), 2002, pp. 137-162.
21. R. Linsker, "Self-organization in a perceptual network", Computer, 21, 1988, pp. 105-117.
22. R. Linsker, "How to generate ordered maps by maximizing the mutual information between input and output", Neural Comput., 1, 1989, pp. 402-411.
23. R. Linsker, "Local synaptic rules suffice to maximize mutual information in a linear network", Neural Comput., 4, 1992, pp. 691-702.
24. A. Luk and S. Lien, "Properties of the generalized lotto-type competitive learning", Proceedings of International conference on neural information processing, CA, San Mateo: Morgan Kaufmann Publishers, 2000, pp. 1180-1185.
25. V.A. Marchman, "Constraints on plasticity in a connectionist model of the English past tense", J. Cogn. Neurosci., 5(2), 1993, pp. 215-234.
26. D.E. Rumelhart and J.L. McClelland, "On learning the past tenses of English verbs", in Parallel Distributed Processing, D.E. Rumelhart, G.E. Hinton and R.J. Williams, Eds., Cambrige: MIT Press, 1986, Vol. 2, pp. 216-271.
27. D.E. Rumelhart and D. Zipser, "Feature discovery by competitive learning", in Parallel Distributed Processing, D.E. Rumelhart, G.E. Hinton and R.N. Williams, Eds., Cambridge: MIT Press, 1986, Vol. 1, pp. 151-193.
28. L. Xu, "Rival penalized competitive learning for clustering analysis, RBF net, and curve detection", IEEE Trans. Neural Netw., 4(4), 1993, pp. 636-649.