Teacher-directed learning: Information-theoretic competitive learning in supervised multi-layered networks

Connection Science

Volumn 15, Issue 2-3, 2003, Pages 117-140

  Kamimura, Ryotaro ^a   Yoshida, Fumihiko ^a  

^a TOKAI UNIVERSITY   (Japan)

Author keywords

Competitive learning; Mutual information maximization; Supervised multi layered networks; Teacher directed learning

Indexed keywords

COMPETITIVE LEARNING;

COMPETITION; INFORMATION ANALYSIS; PERSONNEL TRAINING; SYNTACTICS; TEACHING;

LEARNING SYSTEMS;

EID: 0242551550     PISSN: 09540091     EISSN: None     Source Type: Journal    
DOI: 10.1080/09540090310001611136     Document Type: Article

References (26)

1. Ahalt, S. C., Krishnamurthy, A. K., Chen, P., and Melton, D. E., 1990, Competitive learning algorithms for vector quantization. Neural Networks, 3: 277-290.
2. Atick, J. J., and Redlich, A. N., 1990, Toward a theory of early visual processing. Neural Computation, 2: 308-320.
3. Barlow, H. B., 1989, Unsupervised learning. Neural Computation, 1: 295-311.
4. Barlow, H. B., Kaushal, T. P., and Mitchison, G. J., 1989, Finding minimum entropy code. Neural Computation, 1: 412-423.
5. Becker, S., 1996, Mutual information maximization: models of cortical self-organization. Network: Computation in Neural Systems, 7: 7-31.
6. Becker, S., and Hinton, G. E., 1993, Learning mixture models of spatial coherence. Neural Computation, 5: 267-277.
7. DeSieno, D., 1988, Adding a conscience to competitive learning. In Proceedings of IEEE International Conference on Neural Networks (San Diego: IEEE), pp. 117-124.
8. Fukushima, K., 1975, Cognitron: a self-organizing multi-layered neural network. Biological Cybernetics, 20: 121-136.
9. Gatlin, L. L., 1972, Information Theory and Living Systems (Columbia: Columbia University Press).
10. Grossberg, S., 1987, Competitive learning: from interactive activation to adaptive resonance. Cognitive Science, 11: 23-63.
11. Holsti, O. R., 1969, Content Analysis for Social Sciences and Humanities (Reading MA: Addison-Wesley).
12. Hulle, M. M. V., 1997a, The formation of topographic maps that maximize the average mutual information of the output responses to noiseless input signals. Neural Computation, 9 (3): 595-606.
13. Hulle, M. M. V., 1997b, Topology-preserving map formation achieved with a purely local unsupervised competitive learning rule. Neural Networks, 10 (3): 431-446.
14. Kamimura, R., 2003a, Information theoretic competitive learning in self-adaptive multilayered networks. Connection Science, 15 (1): 3-26.
15. Kamimura, R., 2003b, Progressive feature extraction by greedy network-growing algorithm. Complex Systems, 14 (12): 127-153.
16. Kamimura, R., Kamimura, T., and Shultz, T. R., 2001, Information theoretic competitive learning and linguistic rule acquistion. Transactions of the Japanese Society for Artificial Intelligence, 16 (2): 287-298.
17. Kamimura, R., Kamimura, T., and Takeuchi, H., 2002, Greedy information acquisition algorithm: a new information theoretic approach to dynamic information acquisition in neural networks. Connection Science, 14 (2): 137-162.
18. Kamimura, R., and Nakanishi, S., 1995, Hidden information maximization for feature detection and rule discovery. Network, 6: 577-622.
19. Kohonen, T., 1995, Self-organizing Maps (Berlin: Springer).
20. Linsker, R., 1989, How to generate ordered maps by maximizing the mutual information between input and output. Neural Computation, 1: 402-411.
21. Luk, A., and Lien, S., 2000, Properties of the generalized lotto-type competitive learning. In Proceedings of International Conference on Neural Information Processing, Taejun, Korea, pp. 1180-1185.
22. McClelland, C. A., and Hoggard, G. D., 1969, Conflict patterns in the interactions among nations. In R. Hayes (ed.) International Politics and Foreign Policy (New York: Free Press), p. 714.
23. Nagao, M., 1996, Shizengengo Shori (Natural Language Processing) (Tokyo: Iwanami-shoten).
24. Rumelhart, D. E., Hinton, G., and Williams, R., 1986, Learning internal representations by error progagation. In D. E. Rumelhart, J. L. McClelland and the PDP Research Group (eds) Parallel Distributed Processing, Vol. 1 (Cambridge MA: MIT Press), pp. 318-362.
25. Rumelhart, D. E., and Zipser, D., 1986, Feature discovery by competitive learning. In D. E. Rumelhart, et al. (eds) Parallel Distributed Processing, Vol. 1 (Cambridge MA: MIT Press), pp. 151-193.
26. Xu, L., 1993, Rival penalized competitive learning for clustering analysis, RBF net, and curve detection. IEEE Transaction on Neural Networks, 4 (4): 636-649.