Feature selection using a piecewise linear network

IEEE Transactions on Neural Networks

Volumn 17, Issue 5, 2006, Pages 1101-1115

  Li, Jiang ^a,b,c   Manry, Michael T ^b   Narasimha, Pramod L ^a,b   Yu, Changhua ^b,d  

^a IEEE   (United States)

^b The University of Texas at Arlington   (United States)

^c NATIONAL INSTITUTES OF HEALTH   (United States)

^d FastVDO Inc   (United States)

Author keywords

Feature selection; Floating search; Orthonormal least squares (OLS); Piecewise linear network (PLN); Regression

Indexed keywords

FLOATING SEARCH; ORTHONORMAL LEAST SQUARES (OLS); PIECEWISE LINEAR NETWORK (PLN); REGRESSION PROBLEMS;

LEAST SQUARES APPROXIMATIONS; LINEAR NETWORKS; PIECEWISE LINEAR TECHNIQUES; REGRESSION ANALYSIS; SET THEORY;

FEATURE EXTRACTION;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; AUTOMATED PATTERN RECOGNITION; CLUSTER ANALYSIS; COMPUTER ANALYSIS; COMPUTER SIMULATION; EVALUATION; METHODOLOGY; STATISTICAL MODEL;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; CLUSTER ANALYSIS; COMPUTER SIMULATION; COMPUTING METHODOLOGIES; LINEAR MODELS; NEURAL NETWORKS (COMPUTER); PATTERN RECOGNITION, AUTOMATED;

EID: 33750124786     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2006.877531     Document Type: Article

References (57)

1. A. D. Back and T. P. Trappenberg, "Selecting inputs for modelling using normalized higher order statistics and independent component analysis," IEEE Trans. Neural Netw., vol. 12, no. 3, pp. 612-617, May 2001.
2. R. Kohavi and G. John, "Wrappers for feature subset selection," Artif. Intell., vol. 97, no. 1-2, pp. 273-324, 1997.
3. I. Guyon and A. Elisseeff, "An introduction to variable feature selection," J. Mach. Learn. Res., vol. 3, pp. 1157-1182, 2003.
4. T. W. S. Chow and D. Huang, "Estimating optimal feature subsets using efficient estimation of high-dimensional mutual information," IEEE Trans. Neural Netw., vol. 16, no. 1, pp. 213-224, Jan. 2005.
5. V. Sindhwani, S. Rakshit, D. Deodhare, D. Erdogmus, J. Principe, and P. Niyogi, "Feature selection in MLPs and SVMs based on maximum output information," IEEE Trans. Neural Netw., vol. 15, no. 4, pp. 937-948, Jul. 2004.
6. N. Kambhatla and T. K. Leen, "Dimension reduction by local principal component analysis," Neural Comput., vol. 9, no. 7, pp. 1493-1516, 1997.
7. J. T. Kwok and I. W. Tsang, "The pre-image problem in kernel methods," IEEE Trans. Neural Netw., vol. 15, no. 6, pp. 1517-1525, Nov. 2004.
8. M. D. Plumbley and E. Oja, "A nonnegative PCA algorithm for independent component analysis," IEEE Trans. Neural Netw., vol. 15, no. 1, pp. 66-76, Jan. 2004.
9. K. Z. Mao, "Fast orthogonal forward selection algorithm for feature subset selection," IEEE Trans. Neural Netw., vol. 13, no. 5, pp. 1218-1224, Sep. 2002.
10. K. Z. Mao, "Orthogonal forward selection and backward elimination algorithms for feature subset selection," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 34, no. 1, pp. 629-634, Feb. 2004.
11. R. Caruana and V. De Sa, "Benefitting from the variables that variable selection discards," J. Mach. Learn. Res., vol. 3, pp. 1245-1264, 2003.
12. Ponnapalli, "A formal selection and pruning algorithm for feedforward artificial network optimization," IEEE Trans. Neural Netw., vol. 10, no. 4, pp. 964-968, Jul. 1999.
13. L. K. Kansen and C. E. Rasmussen, "Pruning from adaptive regularization," Neural Comput., vol. 6, no. 6, pp. 1223-1232, 1994.
14. R. Reed, "Pruning algorithms - A survey," IEEE Trans. Neural Netw., vol. 4, no. 5, pp. 740-747, Sep. 1993.
15. S. E. Fahlman and C. Iebiére, "The cascade correlation learning architecture," in Advances in Neural Information Processing Systems2, 1993, pp. 524-532, San Mateo, CA: Morgan Kaufmann.
16. T. Y. Kwok and D. Y. Yeung, "Constructive algorithms for structure learning in feedforward neural networks for regression problems," IEEE Trans. Neural Netw., vol. 8, no. 3, pp. 630-645, May 1997.
17. I. Rivals and L. Personnaz, "Neural-network construction and selection in nonlinear modeling," IEEE Trans. Neural Netw., vol. 14, no. 4, pp. 804-819, Jul. 2003.
18. G. B. Huang, P. Saratchandran, and N. Sundararajan, "A generalized growing and pruning RBF (GGAP-RBF) neural network for function approximation," IEEE Trans. Neural Netw., vol. 16, no. 1, pp. 57-67, Jan. 2005.
19. F. J. Maldonado and M. T. Manry, "Optimal pruning of feed-forward neural networks based upon the Schmidt procedure," in 36th Asilomar Conf. Signals, Systems Computers, 2002, pp. 1024-1028.
20. T. Cibas, F. F.Soulié, P. Gallinanri, and S. Raudys, "Variable selection with neural networks," Neurocomput., vol. 12, pp. 223-248, 1996.
21. K. L. Priddy, S. K. Rogers, D. W. Ruch, G. L. Tarr, and M. Kabrisky, "Bayesian selection of important features for feedforward neural networks," Neurocomput., vol. 5, no. 2, 3, pp. 91-103, 1993.
22. P. Leray and P. Gallinari, "Feature selection with neural networks," Behaviormetrika, vol. 26, Jan. 1999.
23. I. V. Tetko, A. E. P. Villa, and D. J. Livingstone, "Neural network studies. 2. variable selection," J. Chem. Inf. Comput. Sci., vol. 36, no. 4, pp. 794-803, 1996.
24. P. Pudil, J. Novovičová, and J. Kittler, "Floating search methods in feature selection," Pattern Recognit. Lett., vol. 15, pp. 1119-1125, 1994.
25. P. M. Narendra and K. Fukunaga, "A branch and bound algorithm for feature subset selection," IEEE Trans. Comput., vol. C-26, no. 9, pp. 917-922, Sep. 1977.
26. S. D. Stearns, "On selecting features for pattern classifiers," 3rd Int. Conf. Pattern Recognition, pp. 71-75, 1976.
27. I. Rivals and L. Personnaz, "Mlps (mono-layer polynomials and multi-layer perceptrons) for nonlinear modeling," J. Mach. Learn. Res., vol. 3, pp. 1383-1398, 2003.
28. H. Stoppiglia, G. Dreyfus, R. Dubois, and Y. Oussar, "Ranking a random feature for variable and feature selection," J. Mach. Learn. Res., vol. 3, pp. 1399-1414, 2003.
29. S. Chen, X. Hong, and C. J. Harris, "Sparse kernel regression modelling using combined locally regularised orthogonal least squares and D-Optimality experimental design," IEEE Trans. Autom. Control, vol. 48, no. 6, pp. 1029-1026, Jun. 2003.
30. S. Chen, X. Hong, C. J. Harris, and P. M. Sharkey, "Sparse modelling using orthogonal forward regression with PRESS statistic and regularization," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 34, no. 2, pp. 898-911, Apr. 2004.
31. X. Hong and C. J. Harris, "Variable selection algorithm for the construction of MIMO operating point dependent neurofuzzy networks," IEEE Trans. Fuzzy Syst., vol. 9, no. 1, pp. 88-101, Feb. 2001.
32. S. Chen, S. A. Billings, and W. Luo, "Orthogonal least squares methods and their application to non-linear system identification," Int. J. Control, vol. 50, no. 5, pp. 1873-1896, 1989.
33. S. Chen, C. F. N. Cowan, and P. M. Grant, "Orthogonal least squares learning algorithm for radial basis function networks," IEEE Trans. Neural Netw., vol. 2, no. 2, pp. 302-309, Mar. 1991.
34. S. Chen, "Locally regularised orthogonal least squares algorithm for the construction of sparse kernel regression models," in Proc. 6th Int. Conf. Signal Processing, 2002, vol. 2, pp. 1229-1232.
35. S. A. Billings and W. S. F. Voon, "Piecewise linear identification of nonlinear systems," Int. J. Control, vol. 46, pp. 215-235, 1987.
36. L. Breiman, J. H. Friedman, R. A. Olshen, and C. J. Stone, Classification and Regression Tress. Belmont, CA: Wadsworth, 1984.
37. J. H. Friedman, "Multivariate adaptive regression splines," Ann. Statistics, vol. 19, no. 1, pp. 1-141, 1991.
38. J. S. Albus, "A new approach to manipulator control: The cerebellar model articulation controller (CMAC)," J. Dynam. Syst., Meas. Control - Trans. ASME, vol. 97, no. 3, pp. 220-227, 1975.
39. J. S. Albus, "Data storage in the cerebellar model articulation controller (CMAC)," J. Dynam. Syst., Meas. Control - Trans. ASME, vol. 97, no. 3, pp. 228-233, 1975.
40. K. K. Kim, "A Local Approach for Sizing the Multilayer Perceptron," Ph.D. dissertation, Dept. Elect. Eng., Univ. Texas, Arlington, 1996.
41. S. Subbarayan, K. Kim, M. T. Manry, V. Devarajan, and H. Chen, "Modular neural network architecture using piecewise linear mapping," 30th Asilomar Conf. Signals, Systems Computers, vol. 2, pp. 1171-1175, 1996.
42. D. J. C. MacKay, "Bayesian interpolation," Neural Comput., vol. 4, no. 3, pp. 415-447, 1992.
43. F. L. Chung and T. Lee, "Network-growth approach to design of feed-forward neural networks," IEE Proc. Control Theory Applications, vol. 142, no. 5, pp. 486-492, 1995.
44. Y. Hirose, K. Yamashita, and S. Hijiya, "Back-propagation algorithm that varies the number of hidden units," Neural Netw., vol. 4, pp. 61-66, 1991.
45. D. DeMers and G. Cottrell, "Nonlinear dimensionality reduction," in Advances in Neural Information Processing Systems. San Mateo, CA: Morgan Kaufmann, 1993, pp. 580-587.
46. N. Murata, S. Yoshizawa, and S. Amari, "Network information criterion-determining the number of hidden units for an artificial neural network model," IEEE Trans. Neural Netw., vol. 5, no. 6, pp. 865-872, Nov. 1994.
47. B. Schölkopf and A. J. Smola, Learning with Kernels. Cambridge, MA: MIT Press, 2002.
48. T. Kohonen, Self-Organization and Associative Memory, 3 ed. Heidelberg, Germany: Springer-Verlag, 1989.
49. T. Furey, N. C. Duffy, N. Bednarski, D. M. Schummer, and D. Haussler, "Support vector machine classification an validation of cancer tissue samples using microarray expression data," Bioinformatics, vol. 16, pp. 906-914, 2000.
50. A. R. McIntosh, F. L. Bookstein, J. V. Haxby, and C. L. Grady, "Spatial pattern analysis of functional brain images using partial least squares," Neuroimage, vol. 3, pp. 143-157, 1996.
51. H. Wold, "chapter Estimation of Principal Components and Related Models by Iterative Least Squares," in Multivariate Analysis. New York: Academic Press, 1966, pp. 391-420.
52. S. Haykin, "A Comprehensive Foundation," in Neural Networks, 2 ed. Englewood Cliffs, N.J.: Prentice-Hall, 1999.
53. M. Leshno, V. Lin, and S. Schocken, "Multilayer feedforward networks with a nonpolynomial activation function can approximate any function," Neural Netw., vol. 6, no. 6, pp. 861-867, 1993.
54. H. Chandrasekaran, K. K. Kim, and M. T. Manry, "Sizing of the multilayer perceptron via modular networks," in Proc. Neural Networks for Signal Processing IX (NNSP'99), Madison, WI, Aug. 1999, pp. 215-224.
55. A. Gopalakrishnan, M. S. Chen, X. Jiang, and M. T. Manry, "Constructive proof of efficient pattern storage in the multilayer perceptron," in 27th Asilomar Conf. Signals, System, Systems, Computers, 1993, vol. 1, pp. 386-390.
56. J. Bi, K. P. Bennett, M. Embrechts, C. M. Breneman, and M. Song, "Dimensionality reduction via sparse support vector machines," J. Mach. Learn. Res., vol. 3, pp. 1229-1243, Mar. 2003.
57. M. S. Dawson, A. K. Fung, and M. T. Manry, "Surface parameter retrieval using fast learning neural networks," Remote Sens. Rev., vol. 7, no. 1, pp. 1-18, 1993.