Selective information enhancement learning for creating interpretable representations in competitive learning

Neural Networks

Volumn 24, Issue 4, 2011, Pages 387-405

  Kamimura, Ryotaro ^a  

^a TOKAI UNIVERSITY   (Japan)

Author keywords

Attention; Competitive learning; Enhancement; Free energy; Mutual information; Selective information enhancement learning; SOM; Variable selection

Indexed keywords

ATTENTION; COMPETITIVE LEARNING; ENHANCEMENT; MUTUAL INFORMATIONS; SELECTIVE INFORMATION ENHANCEMENT LEARNING; SOM; VARIABLE SELECTION;

INFORMATION THEORY; OPTIMIZATION;

FREE ENERGY;

ARTICLE; ARTIFICIAL NEURAL NETWORK; INFORMATION PROCESSING; LEARNING THEORY; MATHEMATICAL COMPUTING; MATHEMATICAL VARIABLE; PRIORITY JOURNAL; PROBABILITY; SELECTIVE INFORMATION ENHANCEMENT LEARNING; STATISTICAL ANALYSIS;

ALGORITHMS; ANIMALS; ATTENTION; AUTOMATIC DATA PROCESSING; COMPETITIVE BEHAVIOR; DATA INTERPRETATION, STATISTICAL; ENTROPY; HUMANS; LEARNING; NEURAL NETWORKS (COMPUTER);

EID: 79951913762     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neunet.2010.12.009     Document Type: Article

References (48)

1. Anderson J.R. Cognitive psychology and its implication 1980, Worth Publishers, New York.
2. Battiti R. Using mutual information for selecting features in supervised neural net learning. IEEE Transactions on Neural Networks 1994, 5(4):537-550.
3. Bishop C.M. Neural networks for pattern recognition 1995, Oxford University Press.
4. Feraud R., Clerot F. A methodology to explain neural network classification. Neural Networks 2002, 15:237-246.
5. Fisher, J. W., & Principe, J. C. (1998). A methodology for information theoretic feature extraction. In IEEE world congress on computational intelligence (pp. 1712-1716).
6. Gurney K.N. Information processing in dendrites: II information theoretic complexity. Neural Networks 2001, 14:1005-1022.
7. Guyon I., Elisseeff A. An introduction to variable and feature selection. Journal of Machine Learning Research 2003, 3:1157-1182.
8. Howes P., Crook N. Using input parameter influences to support the decisions of feedforward neural networks. Neurocomputing 1999, 24:1999.
9. Ishikawa M. Structural learning with forgetting. Neural Networks 1996, 9(3):509-521.
10. Ishikawa M. Rule extraction by successive regularization. Neural Networks 2000, 13:1171-1183.
11. Kamimura R. Information-theoretic competitive learning with inverse Euclidean distance output units. Neural Processing Letters 2003, 18:163-184.
12. Kamimura R. Teacher-directed learning: information-theoretic competitive learning in supervised multi-layered networks. Connection Science 2003, 15:117-140.
13. Kamimura R. Improving information-theoretic competitive learning by accentuated information maximization. International Journal of General Systems 2006, 34(3):219-233.
14. Kamimura, R. (2007). Information loss to extract distinctive features in competitive learning. In Proceedings of IEEE conference on systems, man, and cybernetics (pp. 1217-1222).
15. Kamimura, R. (2008a). Conditional information and information loss for flexible feature extraction. In Proceedings of the international joint conference on neural networks, IJCNN2008 (pp. 2047-2083).
16. Kamimura, R. (2008b). Feature detection and information loss in competitive learning. In Proceedings of SCIS and ISIS (pp. 1144-1148).
17. Kamimura R. Feature discovery by enhancement and relaxation of competitive units. LNCS 2008, Vol. 5326:148-155. Springer.
18. Kamimura, R. (2008d). Free energy-based competitive learning for mutual information maximization. In Proceedings of IEEE conference on systems, man, and cybernetics (pp. 223-227).
19. Kamimura, R. (2008e). Free energy-based competitive learning for self-organizing maps. In Proceedings of artificial intelligence and applications (pp. 414-419).
20. Kamimura R. Enhancing and relaxing competitive units for feature discovery. Neural Processing Letters 2009, 30(1):37-57.
21. Kamimura R., Kamimura T., Shultz T.R. Information theoretic competitive learning and linguistic rule acquisition. Transactions of the Japanese Society for Artificial Intelligence 2001, 16(2):287-298.
22. Kamimura R., Kamimura T., Uchida O. Flexible feature discovery and structural information control. Connection Science 2001, 13(4):323-347.
23. Kaski, S., Nikkila, J., & Kohonen, T. (1998). Methods for interpreting a self-organized map in data analysis. In Proceedings of european symposium on artificial neural networks. Bruges, Belgium.
24. Kilmer W. Global inhibition for selecting modes of attention. Neural Networks 1996, 9(4):567-573.
25. Kohavi R., John G. Wrappers for feature subset selection. Artificial Intelligence 1997, 97(1-2):273-324.
26. Kohonen T. Self-organization and associative memory 1988, Springer-Verlag, New York.
27. Kohonen T. Self-organizing maps 1995, Springer-Verlag.
28. Lanyon L.J., Denham S.L. A biased competition computational model of spatial and object-based attention mediating active visual search. Neurocomputing 2004, 58-60:655-662.
29. Lanyon L.J., Denham S.L. A model of active visual search with object-based attention. Neural Networks 2004, 17:873-897.
30. Mao I., Jain A.K. Artificial neural networks for feature extraction and multivariate data projection. IEEE Transactions on Neural Networks 1995, 6(2):296-317.
31. Micheli A., Sperduti A., Starita A. Analysis of the internal representations developed by neural networks for structures applied to quantitative structure-activity relationship studies of benzodiazepines. Journal of Chemical Information and Computer Sciences 2001, 41:202-218.
32. Newman J., Baars B.J., Cho S.B. A neural global workspace model for conscious attention. Neural Networks 1997, 10(7):1195-1206.
33. Nord L.I., Jacobsson S.P. A novel method for examination of the variable contribution to computational neural network models. Chemometrics and Intelligent Laboratory Systems 1998, 44:153-160.
34. Perkins S., Lacker K., Theiler J. Grafting: fast, incremental feature selection by gradient descent in function space. Journal of Machine Learning Research 2003, 3:1333-1356.
35. Rakotomamonjy A. Variable selection using SVM-based criteria. Journal of Machine Learning Research 2003, 3:1357-1370.
36. Romesburg H.C. Cluster analysis for researchers 1984, Krieger Publishing Company, Florida.
37. Rumelhart D.E., Hinton G.E., Williams R. Learning internal representations by error progagation. Parallel distributed processing 1986, Vol. 1:318-362. MIT Press, Cambridge. D.E. Rumelhart, G.E.H. (Eds.).
38. Rumelhart D.E., Zipser D. Feature discovery by competitive learning. Cognitive Science 1985, 9:75-112.
39. Setiono R., Leow W.K., Zurada J.M. Extracting of rules from artificial neural networks for nonlinear regression. IEEE Transactions on Neural Networks 2002, 13(3):564-577.
40. Sindhwani V., Raskshit S., Deodhare D., Erdogmus D., Principe J.C. Feature selection in mlps and svms based on maximum output information. IEEE Transactions on Neural Networks 2004, 15(4):937-948.
41. Taylor J.G., Gragopanagos N.F. The interaction of attention and emotion. Neural Networks 2005, 18:353-369.
42. Torkkola, K. (2001). Nonlinear feature transform using maximum mutual information. In Proceedings of international joint conference on neural networks (pp. 2756-2761).
43. Torkkola K. Feature extraction by non-parametric mutual information maximization. Journal of Machine Learning Research 2003, 3:1415-1438.
44. Towell G.G., Shavlik J.W. Extracting refined rules from knowledge-based neural networks. Machine Learning 1993, 13:71-101.
45. Ultsch, A. (2003). U*-matrix: a tool to visualize clusters in high dimensional data. Technical report 36. Department of Computer Science, University of Marburg.
46. Ultsch A., Siemon H.P. Kohonen self-organization feature maps for exploratory data analysis. Proceedings of international neural network conference 1990, 305-308. Kulwer Academic Publisher, Dordrecht.
47. Vesanto J. SOM-based data visualization methods. Intelligent Data Analysis 1999, 3:111-126.
48. Vesanto, J., Himberg, J., Alhoniemi, E., & Parhankangas, J. (2000). SOM toolbox for matlab 5. Technical report A57. Helsinki University of Technology.