Enhancing and relaxing competitive units for feature discovery

Neural Processing Letters

Volumn 30, Issue 1, 2009, Pages 37-57

  Kamimura, Ryotaro ^a  

^a TOKAI UNIVERSITY   (Japan)

Author keywords

Competitive learning; Feature detection; Mutual information; SOM

Indexed keywords

COMPETITIVE LEARNING; CONNECTION WEIGHTS; FEATURE DETECTION; FEATURE DISCOVERY; INFORMATION-THEORETIC METHODS; INPUT PATTERNS; MUTUAL INFORMATION; MUTUAL INFORMATION MAXIMIZATION; MUTUAL INFORMATIONS; OECD COUNTRIES; SOM;

INFORMATION THEORY; STRENGTH OF MATERIALS;

FEATURE EXTRACTION;

EID: 68149181024     PISSN: 13704621     EISSN: 1573773X     Source Type: Journal    
DOI: 10.1007/s11063-009-9109-1     Document Type: Article

References (35)

1. SC Ahalt AK Krishnamurthy P Chen DE Melton 1990 Competitive learning algorithms for vector quantization Neural Netw 3 277 290
2. AJ Bell TJ Sejnowski 1995 An information-maximization approach to blind separation and blind deconvolution Neural Comput 7 6 1129 1159
3. G Deco W Finnof HG Zimmermann 1995 Unsupervised mutual information criterion for elimination of overtraining in supervised multiplayer networks Neural Comput 7 86 107
4. DeSieno D (1988) Adding a conscience to competitive learning. In: Proceedings of IEEE International Conference on Neural Networks, (San Diego), pp 117-124, IEEE
5. D Erdogmus J Principe 2004 Lower and upper bounds for misclassification probability based on renyi's information J VLST Signal Process Syst 37 2/3 305 317
6. Gatlin LL (1972) Information theory and living systems. Columbia University Press, New York
7. E Gokcay J Principe 2002 Information theoretic clustering IEEE Trans Pattern Anal Mach 24 2 158 171
8. RP Gorman TJ Sejnowski 1988 Analysis of hidden units in a layered network trained to classify sonar targets Neural Netw 1 75 89
9. MMV Hulle 1997 The formation of topographic maps that maximize the average mutual information of the output responses to noiseless input signals Neural Comput 9 3 595 606
10. R Kamimura 2003 Information-theoretic competitive learning with inverse euclidean distance Neural Processing Letters 18 163 184
11. R Kamimura 2003 Progressive feature extraction by greedy network-growing algorithm Complex Systems 14 2 127 153
12. R Kamimura 2003 Information theoretic competitive learning in self-adaptive multi-layered networks Connection Science 13 4 323 347
13. R Kamimura 2006 Unifying cost and information in information-theoretic competitive learning Neural Netw 18 711 718
14. R Kamimura 2006 Cooperative information maximization with gauissian activation functions for self-organizing maps IEEE Trans Neural Netw 17 4 909 919
15. R Kamimura 2006 Improving information-theoretic competitive learning by accentuated information maximization Int J Gen Syst 34 3 219 233 (Pubitemid 41523696)
16. R Kamimura T Kamimura O Uchida 2001 Flexible feature discovery and structural information Connection Science 13 4 323 347
17. R Kamimura T Kamimura H Takeuchi 2002 Greedy information acquisition algorithm: a new information theoretic approach to dynamic information acquisition in neural networks Connection Science 14 2 137 162
18. ED Karnin 1990 A simple procedure for pruning back-propagation trained networks IEEE Trans Neural Netw 1 2 239 242
19. Kaski S, Nikkilä J, Kohonen T (1998) Methods for interpreting a self-organized map in data analysis. In: Verleysen M, (eds) Proceedings of ESANN'98, 6th European Symposium on Artificial Neural Networks Bruges, D-Facto, Brussels, Belgium, pp 185-190, 22-24 April
20. Kumagai E, Funao N (2007) Data mining by R (in Japanese). 9-Ten Publishing Company
21. Le Cun Y, Denker JS, Solla SA (1990) Optimal brain damage. In: Advanced in neural information processing, pp 598-605
22. DET Lehn-Schioler Anant Hegde JC Principe 2004 Vector-quantization using information theoretic concepts Natural Comput 4 1 39 51
23. R Linsker 1988 Self-organization in a perceptual network Computer 21 105 117
24. R Linsker 1989 How to generate ordered maps by maximizing the mutual information between input and output Neural Comput 1 402 411
25. Luk A, Lien S (2000) Properties of the generalized lotto-type competitive learning. In: Proceedings of International conference on neural information processing, Morgan Kaufmann Publishers, San Mateo: CA, pp 1180-1185
26. A Micheli A Sperduti A. Starita 2001 Analysis of the internal representations developed by neural networks for structures applied to quantitative structure-activity relationship studies of benzodiazepines J Chem Inf Comput Sci 41 202 218
27. MC Mozer P Smolensky 1989 Using relevance to reduce network size automatically Connection Science 1 1 3 16
28. R Reed 1993 Pruning algorithms-a survey IEEE Trans Neural Netw 4 5 740 747
29. Rumelhart DE, Hinton GE, Williams RJ et al (1986) Learning internal representations by error progagation. In: Rumelhart DE, McClelland JL (eds) Parallel distributed processing vol.1. MIT Press, Cambridge, pp 318-362
30. Slonim N, Tishby N (1999) Agglomerative information bottleneck. In: Advanced in neural information processing, pp 617-623
31. GG Towell JW Shavlik 1993 Extracting rules from knowledge-based neural networks Mach Learn 13 71 101
32. K Torkkola 2003 Feature extraction by non-parametric mutual information maximization J Mach Learn Res 3 1415 1438
33. J Vesanto E Alhoniemi 2000 Clustering of the self-organizing map IEEE-NN 11 586
34. J Vesanto 1999 SOM-based data visualization methods Intelligent-Data- Analysis 3 111 126
35. L Xu 1993 Rival penalized competitive learning for clustering analysis, RBF net, and curve detection IEEE Trans Neural Netw 4 4 636 649