MCES: A novel Monte Carlo evaluative selection approach for objective feature selections

IEEE Transactions on Neural Networks

Volumn 18, Issue 2, 2007, Pages 431-448

  Quah, Kian Hong ^a   Quek, Chai ^a  

^a NANYANG TECHNOLOGICAL UNIVERSITY   (Singapore)

Author keywords

Accuracy measure; Adaptive neurofuzzy inference systems (ANFISs); Audio signal classification; Automobile miles per gallon (MPG) prediction; Financial data modeling; Induction algorithm; Low computational cost; Monte Carlo evaluative feature selection; Monte Carlo evaluative selection (MCES); Multilayer perceptron (MLP); Regressive series; Reinforcement learning

Indexed keywords

ALGORITHMS; ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; DATA MINING; MONTE CARLO METHODS; NONLINEAR SYSTEMS; REGRESSION ANALYSIS; REINFORCEMENT LEARNING;

ACCURACY MEASURE; ADAPTIVE NEUROFUZZY INFERENCE SYSTEMS; AUDIO SIGNAL CLASSIFICATION; FINANCIAL DATA MODELING; INDUCTION ALGORITHM;

FUZZY NEURAL NETWORKS;

ALGORITHM; ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; AUTOMATED PATTERN RECOGNITION; COMPUTER SIMULATION; INFORMATION RETRIEVAL; METHODOLOGY; MONTE CARLO METHOD; STATISTICAL MODEL;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; INFORMATION STORAGE AND RETRIEVAL; MODELS, STATISTICAL; MONTE CARLO METHOD; NEURAL NETWORKS (COMPUTER); PATTERN RECOGNITION, AUTOMATED;

EID: 34047095507     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2006.887555     Document Type: Article

References (73)

1. H. Almuallim and T. G. Dietterich, "Learning with many irrelevant features, in Proc. 9th Nat. Conf. Artif. Intell., 1991, pp. 547-552.
2. H. Almuallim and T. G. Dietterich, "Learning Boolean concepts in the presence of many irrelevant features," Artif. Intell., vol. 69, no. 1-2, pp. 276-306, 1994.
3. K. K. Ang, C. Quek, and A. Wahab, "MCMAC-CVT: A novel associative memory based CVT transmission control system," Neural Netw., vol. 15, no. 3, pp. 219-236, 2002.
4. K. K. Ang, C. Quek, and M. Pasquier, "POPFNN-CRI(S): A psuedo outer product based fuzzy neural network using the compositional rule inference and a singleton fuzzifier," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 33, no. 6, pp. 838-849, Dec. 2003.
5. K. K. Ang and C. Quek, "RSPOP: Rough set-based pseudo outer-product fuzzy rule identification algorithm," Neural Comput., vol. 17, no. 1, pp. 205-243, 2005.
6. K. K. Ang and C. Quek, "Stock trading using RSPOP: A novel rough set-based neuro-fuzzy approach," IEEE Trans. Neural Netw., vol. 17, no. 5, pp. 1301-1315, Sep. 2006.
7. A. L. Blum and P. Langley, "Selection of relevant features and examples in machine learning," Artif. Intell., vol. 97, pp. 245-271, 1997.
8. L. Breiman, J. H. Friedmanm, R. A. Olsehn, and C. J. Stone, Classification and Regression Trees. Belmont, CA: Wadsworth International Group, 1984.
9. Centre for Computational Intelligence, Nanyang Technological University Singapore [Online]. Available: http://www.c2i.ntu.edu.sg:8000/
10. E. Cantu-Paz and C. Kamath, "Evolving neural networks to identify bent-double galaxies in the FIRST survey," Neural Netw., vol. 16, pp. 507-517, 2003.
11. C. Cardie, "Using decision trees to improve case-based learning," in Proc. l0th Int. Conf. Mach. Learn., 1993, pp. 25-32.
12. T. W. S. Chow and D. Huang, "Estimating optimal feature subsets using efficient estimation of high-dimensional mutual information," IEEE Trans. Neural Netw., vol. 16, no. 1, pp. 213-224, Jan. 2005.
13. R. Duda and P. Hart, Pattern Classification and Scene Analysis. New York: Wiley, 1973.
14. X. Fu and L. Wang, "Rule extraction from an RBF classifier based on class-dependent features," CEC 2002, vol. 2, pp. 1916-1921, 2002.
15. X. Fu and L. Wang, "A GA-based novel RBF classifier with class-dependent features," in Proc. Congr. Evol. Comput. (CEC), 2002, vol. 2, pp. 1890-1894.
16. X. Fu and L. Wang, "Data dimensionality reduction with application to simplifying RBF network structure and improving classification performance," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 33, no. 3, pp. 399-409, Jun. 2003.
17. C. Furlanello, M. Serafini, S. Merler, and G. Jurman, "An accelerated procedure for recursive feature ranking on microarray data," Neural Netw., vol. 16, pp. 641-648, 2003.
18. R. P. Gorman and T. J. Sejnowski, "Analysis of hidden units in a layered network trained to classify sonar targets," Neural Netw., vol. 1, pp. 75-89, 1988.
19. M. A. Hall, "Correlation-based feature selection for discrete and numeric class machine learning," in Proc. l7th Int. Conf. Mach. Learn., 2000, pp. 359-366.
20. M. Hall and G. Holmes, "Benchmarking attribute selection techniques for discrete class data mining," IEEE Trans. Knowl. Data Eng., vol. 15, no. 6, pp. 1437-1447, Nov./Dec. 2003.
21. C. N. Hsu, H. J. Huang, and S. Dietrich, "The ANNIGMA-wrapper approach to fast feature selection for neural nets," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 32, no. 2, pp. 207-212, Apr. 2002.
22. J.-S. R. Jang, "ANFIS: Adaptive-network-based fuzzy inference systems," IEEE Trans. Syst., Man, Cybern., vol. 23, no. 3, pp. 665-685, May/Jun. 1993.
23. J.-S. R. Jang, "Input selection for ANFIS learning," in Proc. IEEE Int. Conf. Fuzzy Syst., New Orleans, LA, 1996, pp. 1493-1499.
24. R. Jensen and Q. Shen, "Fuzzy-rough attribute reduction with application to web categorization," Fuzzy Sets Syst., vol. 141, no. 3, pp. 469-485, 2004.
25. R. Jensen and Q. Shen, "Fuzzy-rough data reduction with ant colony optimization," Fuzzy Sets Syst., vol. 149, no. 1, pp. 5-20, 2005.
26. G. H. John, R. Kohavi, and K. Pfleger, "Irrelevant features and the subset selection problem," in Proc. 11th Int. Conf. Mach. Learn., 1994, pp. 121-129.
27. K. Kira and L. A. Rendell, "The feature selection problem: Traditional methods and a new algorithm," in Proc. 10th Nat. Conf. Artif. Intell., 1992, pp. 129-134.
28. K. Kira and L. A. Rendell, "A practical approach to feature selection," in Proc. 9th Int. Conf. Mach. Learn., 1992, pp. 249-256.
29. R. Kohavi and G. H. John, "Wrappers for feature subset selection," Artif. Intell., vol. 97, no. 1-2, pp. 273-324, 1997.
30. I. Kononenko, "Estimating attributes: Analysis and extensions of RELIEF" in Proc. Eur. Conf. Mach. Learn., F. Bergadano and L. D. Raedt, Eds., 1994, pp. 171-182.
31. N. Kwak and C. H. Choi, "Input feature selection for classification problems," IEEE Trans. Neural Netw., vol. 13, no. 1, pp. 143-159, Jan. 2002.
32. P. Laar, S. Gielan, and T. Heskes, "Input selection with partial retraining," in Proc. Int. Conf. Artif. Neural Netw. (ICANN), 1997, pp. 469-474.
33. P. Laar, S. Gielan, and T. Heskes, "Partial retraining: A new approach to input relevance determination," Int. J. Neural Syst., vol. 9, pp. 75-85, 1999.
34. P. Langley, "Selection of relevant features in machine learning," in Proc. AAAI Fall Symp. Relevance, 1994, pp. 140-144.
35. H. M. Lee, C. M. Chen, J. M. Chen, and Y. L. Jou, "An efficient fuzzy classifier with feature selection based on fuzzy entropy," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 31, no. 3, pp. 426-432, Jun. 2001.
36. F. Liu, Q. Quek, and G. S. Ng, "A novel generic Hebbian ordering based fuzzy rule-base reduction approach to Mamdani neuro-fuzzy system," Neural Comput., 2006, (2006), in press.
37. H. Liu and H. Motoda, Feature Selection for Knowledge Discovery and Data Mining. Norwell, MA: Kluwer, 1998.
38. K. Z. Mao, "Fast orthogonal forward selection algorithm for feature subset selection," IEEE Trans. Neural Netw., vol. 13, no. 5, pp. 1218-1224, Sep. 2002.
39. J. Moody and J. Utans, "Principled architecture selection for neural networks: Application to corporate bond rating prediction," in Advances in Neural Information Processing Systems 4, J. E. Moody, S. J. Hanson, and R. P. Lippmann, Eds. San Mateo, CA: Morgan Kaufmann, 1992, pp. 683-690.
40. J. Moody and J. Utans, "Architecture selection strategies for neural networks: Application to corporate bond rating prediction," in Neural Networks in the Capital Markets, A.-P. Refenes, Ed. New York: Wiley, 1995, pp. 277-300.
41. M. Nhut, D. Shi, and C. Quek, "FCMAC-BYY: Fuzzy CMAC using Bayesian Ying-Yang learning," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 36, no. 5, pp. 1180-1190, Oct. 2006.
42. M. Pasquier, C. Quek, and M. Toh, "Fuzzylot: A novel self-organizing fuzzy-neural rule-based pilot system for automated vehicles," Neural Netw., vol. 14, no. 8, pp. 1099-1112, 2001.
43. M. E. Petersen, J. L. Talmon, A. Hasman, and A. W. Ambergen, "Assessing the importance of features for multi-layer perceptrons," Neural Netw., vol. 11, pp. 623-635, 1998.
44. K. H. Quah and C. Quek, "FITSK: Online local learning with generic fuzzy input TSK fuzzy framework for nonlinear system estimation," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 36, no. 1, pp. 166-178, Feb. 2006.
45. K. H. Quah and C. Quek, "Maximum reward reinforcement learning: A non-cumulative reward criterion," Expert Syst. Appl., vol. 31, no. 2, pp. 351-359, 2006.
46. J. R. Quinlan, Shavlik and Dietterich, Eds., "Induction of decision trees," Mach. Learning., vol. 1, pp. 81-106, 1986.
47. J. R. Quinlan, C4.5: Programs for Machine Learning. San Mateo, CA: Morgan Kaufmann, 1993.
48. C. Quek and R. W. Zhou, "POPFNN-AARS: A pseudo outer-product based fuzzy neural network," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 29, no. 6, pp. 859-870, Dec. 1999.
49. C. Quek and W. L. Tung, "A novel approach to fuzzy membership functions using the falcon-MART architecture," Pattern Recognit. Lett., vol. 22, no. 9, pp. 941-958, 2001.
50. C. Quek, K. B. Tan, and V. J. Sagar, "Pseudo-outer product based fuzzy neural network fingerprint verification system," Neural Netw., vol. 14, no. 3, pp. 305-323, 2001.
51. C. Quek and R. W. Zhou, "Antiforgery: A novel pseudo-outer product based fuzzy neural network driven signature verification system," Pattern Recognit. Lett., vol. 23, no. 14, pp. 1795-1816, 2002.
52. C. Quek and R. W. Zhou, "Structure and learning algorithms of a nonsingleton input fuzzy neural network based on the approximate analogical reasoning schema," Fuzzy Sets Syst., vol. 157, no. 13, pp. 1814-1831, 2006.
53. C. Quek, M. Pasquier, and B. Lim, "POP-TRAFFIC: A novel fuzzy neural approach to road traffic analysis and prediction," IEEE Trans. Intell. Transp. Syst., vol. 7, no. 2, pp. 133-146, Jun. 2006.
54. M. Robnik-Sikonja and I. Kononenko, "An adaptation of relief for attribute estimation in regression," in Proc. l4th Int. Conf. Mach. Learn., 1997, pp. 296-304.
55. R. Setiono and H. Liu, "Neural-network feature selector," IEEE Trans. Neural Netw., vol. 8, no. 3, pp. 654-659, May 1997.
56. R. Setiono, W. K. Leow, and J. M. Zurada, "Extraction of rules from artificial neural networks for nonlinear regression," IEEE Trans. Neural Netw., vol. 13, no. 3, pp. 564-577, May 2002.
57. Q. Shen and A. Chouchoulas, "Combining rough sets and data-driven fuzzy learning for generation of classification rules," Pattern Recognit., vol. 32, no. 12, pp. 2073-2076, 1999.
58. Q. Shen and A. Chouchoulas, "A modular approach to generating fuzzy rules with reduced attributes for the monitoring of complex systems," Eng. Appl. Artif. Intell., vol. 13, no. 3, pp. 263-278, 2000.
59. Q. Shen and A. Chouchoulas, "Rough set-based dimensionality reduction for supervised and unsupervised learning," Int. J. Appl. Math. Comput. Sci., vol. 11, no. 3, pp. 583-601, 2001.
60. Q. Shen and A. Chouchoulas, "FuREAP: A fuzzy-rough estimator of algae populations," Artif. Intell. Eng., vol. 15, no. 1, pp. 13-24, 2001.
61. Q. Shen and A. Chouchoulas, "A rough-fuzzy approach for generating classification rules," Pattern Recognit., vol. 35, no. 11, pp. 2425-2438, 2002.
62. Q. Shen and R. Jensen, "Selecting informative features with fuzzy-rough sets and its application for complex systems monitoring," Pattern Recognit., vol. 37, no. 7, pp. 1351-1363, 2004.
63. R. S. Sutton and A. G. Barto, Reinforcement Learning: An introduction. Cambridge, MA: MIT Press, 1998.
64. J. Sim, W. L. Tung, and C. Quek, "FCMAC-Yager: A novel Yager-inference-scheme-based fuzzy CMAC," IEEE Trans. Neural Netw., vol. 17, no. 6, pp. 1394-1410, Nov. 2006.
65. T. Z. Tan, C. Quek, and G. S. Ng, "Ovarian cancer diagnosis by hippocampus and neocortex-inspired learning memory structures," Neural Netw., vol. 18, no. 5/6, pp. 818-825, 2005.
66. S. Teddy, E. Lai, and C. Quek, "Hierarchically clustered adaptive quantization cmac and its learning convergence," IEEE Trans. Neural Netw., 2006, submitted for publication.
67. C. W. Ting, "Embedded audio detection software" School Comput. Eng., Nanyang Technol. Univ., Singapore, 2001, Honors Year Project Rep..
68. W. L. Tung and C. Quek, "Falcon: Fuzzy neural control and decision systems using FKP and PFKP clustering algorithms," IEEE Trans. Syst., Man, Cybern. B, Cybern., vol. 34, no. 1, pp. 686-694, Feb. 2004.
69. W. L. Tung, C. Quek, and P. Cheng, "GenSo-EWS: A novel neural-fuzzy based early warning system for predicting bank failures," Neural Netw., vol. 17, no. 4, pp. 567-587, 2004.
70. W. L. Tung and C. Quek, "GenSo-FDSS: A neural-fuzzy decision support system for pediatric all cancer subtype identification using gene expression data," Artif. Intell. Med., vol. 33, pp. 61-88, 2005.
71. J. Utans and J. Moody, "Selecting neural network architectures via the prediction risk: Application to corporate bond rating prediction," in Proc. IEEE 1st Int. Conf. Artif. Intell. Appl. Wall Street, 1991, pp. 35-41.
72. E. J. Yeoh, "Classification, subtype discovery, and prediction of outcome in pediatric lymphoblastic leukemia by gene expression profiling," Cancer Cell, vol. 1, no. 2, pp. 133-143, 2002.
73. R. W. Zhou, C. Quek, and G. S. Ng, "A novel single pass thinning algorithm and an effective set of performing criteria," Pattern Recognit. Lett., vol. 16, no. 12, pp. 1267-1275, 1995.