Supposed maximum information for comprehensible representations in SOM

Neurocomputing

Volumn 74, Issue 7, 2011, Pages 1116-1134

  Kamimura, Ryotaro ^a  

^a IT Education Center   (Japan)

Author keywords

Competitive earning; Comprehensibility; Mutual information; SOM

Indexed keywords

ANIMAL DATA; CLASS BOUNDARY; COMPETITIVE EARNING; COMPETITIVE LEARNING; COMPETITIVE NETWORK; COMPREHENSIBILITY; INFORMATION-THEORETIC METHODS; INTERPRETABLE REPRESENTATION; KOHONEN; LEARNING METHODS; MACHINE-LEARNING DATABASE; MUTUAL INFORMATIONS; SOM;

CONFORMAL MAPPING; INFORMATION THEORY; LEARNING SYSTEMS;

SELF ORGANIZING MAPS;

ARTICLE; COMPREHENSION; CONCEPTUAL FRAMEWORK; DATA BASE; IMAGE PROCESSING; INFORMATION PROCESSING; LEARNING ALGORITHM; MACHINE LEARNING; MATHEMATICAL COMPUTING; MATHEMATICAL MODEL; METHODOLOGY; NONHUMAN; PRIORITY JOURNAL; SELF ORGANIZING MAP; SENSITIVITY ANALYSIS; SINGLE PHOTON EMISSION COMPUTER TOMOGRAPHY; TOPOGRAPHY;

EID: 79751526256     PISSN: 09252312     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neucom.2010.12.002     Document Type: Article

References (69)

1. Torkkola K. Feature extraction by non-parametric mutual information maximization. Journal of Machine Learning Research 2003, 3:1415-1438.
2. Linsker R. Self-organization in a perceptual network. Computer 1988, 21:105-117.
3. Linsker R. How to generate ordered maps by maximizing the mutual information between input and output. Neural Computation 1989, 1:402-411.
4. Barlow H.B. Unsupervised learning. Neural Computation 1989, 1:295-311.
5. Barlow H.B., Kaushal T.P., Mitchison G.J. Finding minimum entropy code. Neural Computation 1989, 1:412-423.
6. Becker S. Mutual information maximization: models of cortical self-organization. Network: Computation in Neural Systems 1996, 7:7-31.
7. Becker S., Hinton G.E. Learning mixture models of spatial coherence. Neural Computation 1993, 5:267-277.
8. Deco G., Finnof W., Zimmermann H.G. Unsupervised mutual information criterion for elimination of overtraining in supervised multiplayer networks. Neural Computation 1995, 7:86-107.
9. Comon P. Independent component analysis: a new concept. Signal Processing 1994, 36:287-314.
10. Bell A., Sejnowski T.J. An information-maximization approach to blind separation and blind deconvolution. Neural Computation 1995, 7(6):1129-1159.
11. Linsker R. Local synaptic rules suffice to maximize mutual information in a linear network. Neural Computation 1992, 4:691-702.
12. Linsker R. Improved local learning rule for information maximization and related applications. Neural Networks 2005, 18:261-265.
13. Battiti R. Using mutual information for selecting features in supervised neural net learning. IEEE Transactions on Neural Networks 1994, 5(4):537-550.
14. Principe J.C., Xu D., Zhao Q., III J.W.F. Learning from examples with information theoretic criteria. The Journal of VLSI Signal Processing 2000, 26(1-2):61-77.
15. Lehn-Schioler D.E.T., Hegde A., Principe J.C. Vector-quantization using information theoretic concepts. Natural Computation 2004, 4(1):39-51.
16. Morejon R.A., Principe J.C. Advanced search algorithms for information-theoretic learning with kernel-based estimators. IEEE Transactions on Neural Networks 2004, 15(4).
17. Torkkola K. Feature extraction by non-parametric mutual information maximization. Journal of Machine Learning Research 2003, 3:1415-1438.
18. Kamimura R. Mutual information maximization by free energy-based competitive learning for self-organizing maps. Proceedings of the International Conference on Systems, Man, and Cybernetics (SMC2008) 2008, 1819-1825.
19. Kamimura R. Interpreting and improving multi-layered networks by free energy-based competitive learning. Proceedings of the International Conference on Systems, Man, and Cybernetics (SMC2008) 2008, 1812-1818.
20. Rumelhart D.E., Zipser D. Feature discovery by competitive learning. Cognitive Science 1985, 9:75-112.
21. Kohonen T. Self-Organizing Maps 1995, Springer-Verlag.
22. Guyon I., Elisseeff A. An introduction to variable and feature selection. Journal of Machine Learning Research 2003, 3:1157-1182.
23. Sung A.H. Ranking importance of input parameters of neural networks. Expert Systems with Applications 1998, 15:405-411.
24. Steppe J.M., Bauer J.K.W. Feature saliency measures. Computers and Mathematics with Applications 1997, 33(8):109-126.
25. Belue L.M., Bauer J.K.W. Determining input features for multiplayer perceptrons. Neurocomputing 1995, 7:111-121.
26. Petersen M., Talmoon J.L., Hasman A., Ambergen A.W. Assessing the importance of features for multi-layer perceptrons. Neural Networks 1998, 11:623-635.
27. Kamimura R. Information loss to extract distinctive features in competitive learning. Proceedings of IEEE Conference on Systems, Man, and Cybernetics 2007, 1217-1222.
28. Kamimura R. Feature detection and information loss in competitive learning. Proceedings of the International Conference on Soft Computing and Intelligent Systems and the International Symposium on Advanced Intelligent Systems (SCIS and ISIS2008) 2008, 1144-1148.
29. Kamimura R. Conditional information and information loss for flexible feature extraction. Proceedings of the International Joint Conference on Neural Networks (IJCNN2008) 2008, 2047-2083.
30. Kamimura R. Feature discovery by enhancement and relaxation of competitive units. Intelligent Data Engineering and Automated Learning-IDEAL2008, Lecture Notes in Computer Science 2008, vol. 5326:148-155.
31. Kamimura R. Enhancing and relaxing competitive units for feature discovery. Neural Processing Letters 2009, 30(1):37-57.
32. Kamimura R., Kamimura T., Shultz T.R. Information theoretic competitive learning and linguistic rule acquisition. Transactions of the Japanese Society for Artificial Intelligence 2001, 16(2):287-298.
33. Vesanto J. SOM-based data visualization methods. Intelligent Data Analysis 1999, 3:111-126.
34. Kamimura R. Information-theoretic competitive learning with inverse Euclidean distance output units. Neural Processing Letters 2003, 18:163-184.
35. Xu R., Wunsch D.C. Survey of clustering algorithms. IEEE Transactions on Neural Networks 2005, 16:645-678.
36. Kiviluoto K. Topology preservation in self-organizing maps. Proceedings of the IEEE International Conference on Neural Networks 1996, 294-299.
37. Oja M., Serber G.O., Blomberg J., Kaski S. Self-organizing map-based discovery and visualization of human endogenous retroviral sequence groups. International Journal of Neural Systems 2005, 15(3):163-179.
38. Kaski S., Nikkila J., Oja M., Venna J., Toronen P., Castren E. Trustworthiness and metrics in visualizing similarity of gene expression. BMC Bioinformatics 2003, 4(48).
39. Venna J., Kaski S. Neighborhood preservation in nonlinear projection methods: an experimental study. Lecture Notes in Computer Science 2001, vol. 2130:485-491.
40. Polzlbauer G. Survey and comparison of quality measures for self-organizing maps. Proceedings of the Fifth Workshop on Data Analysis (WDA04) 2004, 67-82.
41. Nikkila J., Toronen P., Kaski S., Venna J., Castren E., Wong G. Analysis and visualization of gene expression data using self-organizing maps. Neural Networks 2002, 15:953-966.
42. Vathy-Fogarassy A., Werner-Stark A., Gal B., Abonyi J. Visualization of topological representing networks. Lecture Notes in Computer Science (IDEAL2007) 2007, vol. 4881:557-566.
43. J. Himberg, From insights to innovation: data mining, visualization, and user interfaces, Dissertation, Helsinki University of Technology, 2007.
44. J. Venna, Dimensionality reduction for visual exploration of similarity structures, Dissertation, Helsinki University of Technology, 2007.
45. Lee J.A., Verleysen M. Quality assessment of nonlinear dimensionality reduction based on K-ary neighborhoods. JMLR: Workshop and Conference Proceedings 2008, vol. 4:21-35.
46. A. Ultsch, U*-matrix: a tool to visualize clusters in high dimensional data, Technical Report 36, Department of Computer Science, University of Marburg, 2003.
47. Bauer H.U., Der R., Herrmann M. Controlling the magnification factor of self-organizing maps. Neural Computation 1996, 8(4):757-771.
48. Merenyi E., Jain A. Forbidden magnification? II. Proceedings of 12th European Symposium on Artificial Neural Networks 2004, 57-62.
49. Merenyi E., Jain A., Villmann T. Explicit magnification control of self-organizing maps for forbidden data. IEEE Transactions on Neural Networks 2007, 18(3):786-797.
50. Villmann T., Claussen J.C. Magnification control in self-organizing maps and neural gas. Neural Computation 2006, 18:446-469.
51. DeSieno D. Adding a conscience to competitive learning. Proceedings of IEEE International Conference on Neural Networks 1988, 117-124. IEEE, San Diego.
52. Xu L. Rival penalized competitive learning for clustering analysis, RBF net, and curve detection. IEEE Transaction on Neural Networks 1993, 4(4):636-649.
53. Van Hulle M. Topographic map formation by maximizing unconditional entropy: a plausible strategy for 'on-line' unsupervised competitive learning and nonparametric density estimation. IEEE Transactions on Neural Networks 1996, 7(5):1299-1305.
54. Hulle M.M.V. The formation of topographic maps that maximize the average mutual information of the output responses to noiseless input signals. Neural Computation 1997, 9(3):595-606.
55. Van Hulle M.M. Faithful representations with topographic maps. Neural Networks 1999, 12(6):803-823.
56. Zhang Y.J., Liu Z.Q. Self-splitting competitive learning: a new on-line clustering paradigm. IEEE Transactions on Neural Networks 2002, 13(2):369-380.
57. Xiong H., Swamy M.N.S. Branching competitive learning network: a novel self-creating model. IEEE Transactions on Neural Networks 2004, 15(2):417-429.
58. Xiong H., Swamy M.N.S., Ahmad M.O. Competitive splitting for codebook initialization. IEEE Signal Processing Letters 2004, 11:474-477.
59. Kaski S., Nikkila J., Kohonen T. Methods for interpreting a self-organized map in data analysis. Proceedings of European Symposium on Artificial Neural Networks, Bruges, Belgium 1998.
60. Mao I., Jain A.K. Artificial neural networks for feature extraction and multivariate data projection. IEEE Transactions on Neural Networks 1995, 6(2):296-317.
61. Ultsch A., Siemon H.P. Kohonen self-organization feature maps for exploratory data analysis. Proceedings of International Neural Network Conference 1990, 305-308. Kulwer Academic Publisher, Dordrecht.
62. Kohonen T. Self-Organization and Associative Memory 1988, Springer-Verlag, New York.
63. Kamimura R. Teacher-directed learning: information-theoretic competitive learning in supervised multi-layered networks. Connection Science 2003, 15:117-140.
64. Kamimura R. Information theoretic competitive learning in self-adaptive multi-layered networks. Connection Science 2003, 13(4):323-347.
65. Kamimura R., Kamimura T., Takeuchi H. Greedy information acquisition algorithm: a new information theoretic approach to dynamic information acquisition in neural networks. Connection Science 2002, 14(2):137-162.
66. Kamimura R. Progressive feature extraction by greedy network-growing algorithm. Complex Systems 2003, 14(2):127-153.
67. Kamimura R. Cooperative information maximization with Gaussian activation functions for self-organizing maps. IEEE Transactions on Neural Networks 2006, 17(4):909-919.
68. R. Kamimura, Forced information for information-theoretic competitive learning, in: Machine Learning, IN-TECH, 2009, pp. 125-144.
69. Kamimura R. Forced information maximization to accelerate information-theoretic competitive learning. Proceedings of the International Joint Conference on Neural Networks 2007, 1779-1784.