Two-phase construction of multilayer perceptrons using information theory

IEEE Transactions on Neural Networks

Volumn 20, Issue 4, 2009, Pages 715-721

  Xing, Hong Jie ^a   Hu, Bao Gang ^b  

^a HEBEI UNIVERSITY   (China)

^b INSTITUTE OF AUTOMATION   (China)

Author keywords

Hidden units pruning; Input units selection; Multilayer perceptron (MLP); Mutual information (MI) criterion

Indexed keywords

FORWARD STRATEGIES; HIDDEN UNITS PRUNING; INPUT UNITS SELECTION; INPUT VECTORS; MULTI-LAYER PERCEPTRONS; MULTILAYER PERCEPTRON (MLP); MUTUAL INFORMATION (MI) CRITERION; RELEVANCE MEASURES; SUPPORT VECTOR REGRESSIONS;

CYBERNETICS; ELECTRIC LOADS; FEATURE EXTRACTION; INFORMATION THEORY; MULTILAYERS; PATTERN RECOGNITION SYSTEMS; SENSITIVITY ANALYSIS; SUPPORT VECTOR MACHINES; VECTORS;

MULTILAYER NEURAL NETWORKS;

EID: 65149084176     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2008.2005604     Document Type: Article

References (13)

1. A. P. Engelbrecht, "New pruning heuristics based on variance analysis of sensitivity information," IEEE Trans. Neural Netw., vol. 12, no. 6, pp. 1386-1399, Nov. 2001.
2. X. Zeng and D. S. Yeung, "Hidden neuron pruning of multilayer perceptrons using a quantified sensitivity measure," Neurocomputing vol. 69, pp. 825-837, 2006.
3. I.-S. Oh, J.-S. Lee, and B.-R. Moon, "Hybrid genetic algorithms for feature selection," IEEE Trans. Pattern Anal. Mach. Intell., vol. 26, no. 11, pp. 1424-1437, Nov. 2004.
4. G. Van Dijck and M. M. Van Hulle, "Speeding up feature subset selection through mutual information relevance filtering," in Lecture Notes in Artificial Intelligence. Berlin, Germany: Springer-Verlag, 2007, vol. 4702, pp. 277-287.
5. T. M. Cover and J. A. Thomas, Elements of Information Theory. New York: Wiley, 1991.
6. N. Kwak and C.-H. Choi, "Input feature selection by mutual information based on Parzen window," IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 12, pp. 1667-1671, Dec. 2002.
7. A. Kraskov, H. Stögbauer, and P. Grassberger, "Estimating mutual information," Phys. Rev. E, Stat. Plasmas Fluids Relat. Interdiscip. Top., vol. 69, 2004, 0661138.
8. H. Stögbauer, A. Kraskov, S. A. Astakhov, and P. Grassberger, "Least dependent component analysis based on mutual information," Phys. Rev. E, Stat. Plasmas Fluids Relat. Interdiscip. Top., vol. 70, no. 6, 2004, 066123.
9. C. Blake and C. Merz, UCI Repository of Machine Learning Datasets, 1998 Online. Available: http://www.ics.uci.edu/~mlearn/MLRepository.html
10. B. D. Ripley, Pattern Recognition and Neural Networks. Cambridge, U.K.: Cambridge Univ. Press, 1996.
11. J. H. Friedman, "Multivariate adaptive regression splines," Ann. Statist., vol. 19, no. 1, pp. 1-141, 1991.
12. L. Breiman and J. Friedman, "Estimating optimal transformations in multiple regression and correlation," J. Amer. Statist. Assoc., vol. 80, pp. 580-619, 1985.
13. M. A. Mazurowski and P. M. Szecówka, "Limitations of sensitivity analysis for neural networks in cases with dependent inputs," in Proc. 4th IEEE Int. Conf. Comput. Cybern., Tallinn, Estonia, 2006, pp. 299-303.