A general wrapper approach to selection of class-dependent features

IEEE Transactions on Neural Networks

Volumn 19, Issue 7, 2008, Pages 1267-1278

  Wang, Lipo ^a   Zhou, Nina ^a   Chu, Feng ^a  

^a NANYANG TECHNOLOGICAL UNIVERSITY   (Singapore)

Author keywords

Class dependent feature extraction; Class dependent feature selection; Feature importance ranking; Minimal redundancy maximal relevancy (mRMR); One against all; One against one; Support vector machine (SVM)

Indexed keywords

CLASSIFICATION (OF INFORMATION); CLASSIFIERS; ERROR ANALYSIS; LEARNING ALGORITHMS; SUPPORT VECTOR MACHINES; VECTORS;

ARTIFICIAL DATA; ATTRIBUTE IMPORTANCE; BENCHMARK DATA; CLASS SEPARABILITY; CLASSIFICATIO N PROBLEMS; CLASSIFICATION ACCURACIES; CLASSIFICATION ERROR RATE; FEATURE SELECTION METHODS; FEATURE SUBSETS; REAL WORLD; REDUNDANT FEATURES; SEARCH ALGORITHMS; SINGLE INPUT; SUPPORT VECTOR MACHINE (SVM); WRAPPER APPROACHES;

LEARNING SYSTEMS;

EID: 48949116216     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2008.2000395     Document Type: Article

References (58)

1. P. M. Baggenstoss, "Class-specific classifier: Avoiding the curse of dimensionality," IEEE Aerosp. Electron. Syst. Mag., vol. 19, no. 1, pt. 2, pp. 37-52, Jan. 2004.
2. P. M. Baggenstoss, "The PDF projection theorem and the class-specific method," IEEE Trans. Signal Process., vol. 51, no. 3, pp. 672-685, Mar. 2003.
3. P. M. Baggenstoss, "Class-specific features in classification," IEEE Trans. Signal Process., vol. 50, no. 12, pp. 3428-3432, Dec. 2002.
4. J. Bins and B. Draper, "Feature selection from huge feature sets," in Proc. Int. Conf. Comput. Vis., Vancouver, BC, Canada, 2001, pp. 159-165 [Online]. Available: http://citeseer.ist.psu. edu/bins01feature.html
5. C. L. Blake and C. J. Merz, "UCI Repository of Machine Learning Databases," Dept. Inf. Comput. Sci., Univ. California, Irvine, CA, Tech. Rep., 1998 [Online]. Available: http://www.ics.uci.edu/~mlearn/ MLRepository.html
6. A. L. Blum and P. Langley, "Selection of relevant features and examples in machine learning," Proc. Nat. Acad. Sci. USA, vol. 95, pp. 933-942, 1998.
7. L. F. Bo, L. Wang, and L. C. Jiao, "Kernel Fisher Discriminant Analysis Toolbox," Inst. Intell. Inf. Process., Xidian Univ., Xi'an, Chine, Tech. Rep., Mar. 2006.
8. L. Bottou, C. Cortes, J. Denker, H. Drucker, I. Guyon, L. Jackel, Y. LeCun, U. Muller, E. Ackinger, P. Simard, and V. Vapnik, "Comparison of classifier methods: A case study in handwriting digit recognition," in Proc. Int. Conf. Pattern Recognit., 1994, pp. 77-87.
9. C.-C. Chang and C.-J. Lin, "LIBSVM: A Library for Support Vector Machines," 2007 [Online]. Available: http://www.csie.ntu.edu.tw/ ~cjlin/libsvm
10. L. H. Chen and S. Chang, "An adoptive learning algorithm for principle component analysis," IEEE Trans. Neural Netw., vol. 6, no. 5, pp. 1255-1263, Sep. 1995.
11. C. Cortes and V. Vapnik, "Support-vector network," Mach. Learn., vol. 20, pp. 273-297, 1995.
12. T. M. Cover and J. A. Thomas, Elements of Information Theory. New York: Wiley, 1991.
13. K. Crammer and Y. Singer, "On the learnability and design of output codes for multiclass problems," Comput. Learn. Theory, pp. 35-46, 2000.
14. N. Cristianini and J. Shawe-Taylor, An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods. Cambridge, U.K.: Cambridge Univ. Press, 2000.
15. P. Devijver and J. Kittler, Pattern Recognition: A Statistical Approach. Englewood Cliffs. NJ: Prentice-Hall, 1982.
16. J. Friedman, "Another approach tp polychotomous classification," Dent. Statistics, Stanford Univ., Stanford, CA, 1996 [Online]. Available: http://www-stat.stanford.edu/reports/ friedman/poly.ps.Z
17. X. J. Fu and L. P. Wang, "Data dimensionality reduction with application to simplifying RBF network structure and improving classification performance," IEEE Trans. Syst. Man Csybern. B, Cybern., vol. 33, no. 3, pp. 399-409, Jun. 2003.
18. X. J. Fu and L. P. Wall, "A GA-based novel RBF classifier with class-dependent features," in Proc. Congr. Evol. Comput., 2002, vol. 2, pp. 1890-1894.
19. R. Gilad-Bachrach, A. Navot, and N. Tishby, "Margin based feature selection-theory and algorithms," in Proc. 21st Int. Conf. Mach. Learn., Banff, AB, Canada, 2004, pp. 43-50.
20. I. Guyon and A. Elisseeff, "An introduction to variable and feature selection," J. Mach. Learn. Res., vol. 3, pp. 1157-1182, Mar. 2003.
21. S. Halgamuge, and L. P. Wang, Eds., Classification and Clustering for Knowledge Discovery. Berlin, Germany: Springer-Verlag, 2005.
22. S. Halgamuge and L. P. Wang, Eds., Computational Intelligence for Modeling and Predictions. Berlin, Germany: Springer-Verlag, 2005.
23. M.A. Hall, "Correlation-based feature selection for machine learning," Ph.D. dissertation, Dept. Comput. Sci., Waikato Univ., Hamilton, New Zealand, 1998.
24. C.-W. Hsu, C. C. Chang, and C. J. Lin, "A practical guide to support vector classification," Dept. Comput. Sci. Inf. Eng., Nat. Taiwan Univ., Taipei, Taiwan, Tech. Rep., 2003.
25. C.-W. Hsu and C.-J. Lin, "A comparison of methods for multiclass support vector machines," IEEE Trans. Neural Netw., vol. 13, no. 2, pp. 415-425, Mar. 2002.
26. H. G. John, "Enhancements to the data mining process," Ph.D. dissertation, Comput. Sci. Dept., School Eng., Stanford Univ., Stanford, CA, 1997.
27. K. Kira and L. A. Rendell, "The feature selection problem: Traditional methods and a new algorithm," in Proc. 10th Nat. Conf. Artif. Intell., 1992, pp. 129-134.
28. S. Knerr, L. Personnaz, and G. Dreyfus, "Single-layer learning revisited: A stepwise procedure for building and training a neural network," in Neurocomputing: Algorithms, Architectures and Applications, ser. NATO ASI, F. Fogelman Soulié and J. Hérault, Eds. New York: Springer-Verlag, 1990, vol. F68.
29. D. Koller and M. Sahami, "Toward optimal feature selection," in Proc. 13th Int. Conf. Mach. Learn., 1996, pp. 284-292.
30. I. Kononenko, "Estimating attributes: Analysis and extensions of RE-LIEF," in Proc. Eur. Conf. Mach. Learn., F. Bergadano and L. de Readt, Eds., Catania, Sicily, Italy, Apr. 1994, pp. 171-182.
31. U. Krebell, Pairwise Classification and Support Vector Machines, Advances in Kernel Methods-Support Vector Learning. Cambridge, MA: MIT Press, 1999, pp. 255-268.
32. P. Langley, Elements of Machine Learning. San Mateo, CA: Morgan Kaufmann, 1996.
33. P. Langley and S. Sage, "Induction of selective Bayesian classifiers," in Proc. 10th Conf. Uncertainty Artif. Intell., 1994, pp. 399-406.
34. M. Last, A. Kandel, and O. Maimon, "Information-theoretic algorithm for feature selection," Pattern Recognit. Lett., vol. 22, pp. 799-811, 2001.
35. G. Legrand and N. Nicoloyannis, "Feature selection method using preferences aggregation," in Proc. Mach. Learn. Data Mining, P. Perner and A. Imiya, Eds., 2005, pp. 203-217.
36. C. L. Liu and H. Sako, "Class-specific feature polynomial classifier for pattern classification and its application to handwritten numerical recognition," Pattern Recognit., vol. 39, no. 4, pp. 669-681, 2006.
37. Q. S. Liu, H. Q. Lu, and S. D. Ma, "Improving kernel Fisher discriminant analysis for face recognition," IEEE Trans. Circuits Syst. Video Technol., vol. 14, no. 1, pp. 42-49, Jan. 2004.
38. W. Malina, "Two-parameter Fisher criteria," IEEE Trans. Syst. Man Cybern. B, Cybern., vol. 31, no. 4, pp. 629-636, Aug. 2001.
39. S. Mika, G. Ratsch, J. Weston, B. Scholkopf, and K. R. Mullers, "Fisher discriminant analysis with kernels," in Proc. Neural Networks Signal Process. IX, 1999, pp. 41-48.
40. K. R. Müller, S. Mika, G. Rätsch, K. Tsuda, and B. Scholkopf, "An introduction to kernel-based learning algorithms," IEEE Trans. Neural Netw., vol. 12, no. 2, pp. 181-201, Mar. 2001.
41. I. S. Oh, J. S. Lee, and C. Y. Suen, "Analysis of class separation and combination of class-dependent features for handwriting recognition," IEEE Trans. Pattern Anal. Mach. Intell., vol. 21, no. 10, pp. 1089-1094, Oct. 1999.
42. I. S. Oh, J. S. Lee, and C. Y. Suen, "Using class separation for feature analysis and combination of class-dependent features," in Proc. 14th Int. Conf. Pattern Recognit., 1998, vol. 1, pp. 453-455.
43. H. Peng, F. Long, and C. Ding, "Feature selection based on MI: Criteria of max-dependency, max-relevance, and min-redundancy," IEEE Trans. Pattern Anal. Mach. Intell., vol. 27, no. 8, pp. 1226-1238, Aug. 2005.
44. J. C. Platt, N. Cristianini, and J. Shawe-Taylor, "Large margin DAGs for multiclass classification," in Advances in Neural Information Processing Systems. Cambridge, MA: MIT Press, 2000, vol. 12, pp. 547-553.
45. B. Raman and T. R. Ioerger, ""Enhancing learning using feature and example selection," M.S. Thesis, Dept. Comput. Sci., Texas A&M Univ., College Station, TX, 2003.
46. M. L. Raymer, W. F. Punch, E. D. Goodman, L. A. Kuhn, and A. K. Jain, "Dimensionality reduction using genetic algorithms," IEEE Trans. Evol. Comput., vol. 4, no. 2, pp. 164-171, Jul. 2000.
47. A. L. Rendell and R. Sheshu, "Learning hard concepts through constructive induction: Framework and rationale," Comput. Intell. vol. 6, pp. 247-270, 1990.
48. J. Reunanen, "Overfitting in making comparisons between variable selection methods," J. Mach. Learn. Res., vol. 3, pp. 1371-1382, 2003.
49. R. Ruiz, J. S. Aguilar-Ruiz, and J. C. Riquelme, "SOAP: Efficient feature selection of numeric attributes," in Ibero-Amer. Conf. Artif. Intell. (IBERAMIA), 2002, pp. 233-242.
50. W. Siedlecki and J. Sklansky, "A note on genetic algorithms for largescale feature selection," Pattern Recognit. Lett., vol. 10, pp. 335-347, 1989.
51. The International HapMap Consortium, "Integrating ethics and science in the international HapMap project," Nature Rev. Genetics, vol. 5, pp. 467-475, 2004.
52. M. P. Tu, L. Zhen, and B. A. Russ, "Choosing SNPs using feature selection," in Proc. IEEE Comput. Syst. Bioinf. Conf., 2005, pp. 301-309.
53. V. Vapnik, Statistical Learning Theory. New York: Wiley, 1998.
54. L. P.Wang, Ed., Support Vector Machines: Theory and Applications. New York: Springer-Verlag, 2005.
55. L. P. Wang and X. J. Fu, Data Mining With Computational Intelligence Berlin, Germany: Springer-Verlag, 2005, p. 276.
56. J. Weston and C. Watkins, "Multi-class support vector machines," in Proc. Eur. Symp. Artif. Neural Netw., M. Verleysen, Ed., Brussels, Belgium, 1999, pp. 219-224.
57. T.-F. Wu, C.-J. Lin, and R. C. Weng, "Probability estimates for multiclass classification by pairwise couppling," J. Mach. Learn. Res., vol. 5, pp. 975-1005, 2004.
58. Z. X. Zhu, Y.-S. Ong, and M. Dash, "Wrapper-filter feature selection algorithmusing a memetic framework," IEEE Trans. Syst. Man Cybern. B, Cybern., vol. 37, no. 1, pp. 70-76, Feb. 2007.