A selective sampling approach to active feature selection

Artificial Intelligence

Volumn 159, Issue 1-2, 2004, Pages 49-74

  Liu, Huan ^a   Motoda, Hiroshi ^b   Yu, Lei ^a  

^a Arizona State University   (United States)

^b OSAKA UNIVERSITY   (Japan)

Author keywords

Dimensionality reduction; Feature selection and ranking; Learning; Sampling

Indexed keywords

DATA PROCESSING; LEARNING ALGORITHMS; LEARNING SYSTEMS; PERFORMANCE; PROJECT MANAGEMENT; SAMPLING;

DATA VARIANCES; DIMENSIONALITY REDUCTION; FEATURE SELECTION AND RANKING; RANDOM SAMPLING;

ARTIFICIAL INTELLIGENCE;

EID: 4644347255     PISSN: 00043702     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.artint.2004.05.009     Document Type: Article

References (58)

1. R. Agrawal, T. Imielinski, A. Swami, Database mining: a performance perspective, IEEE Trans. Knowledge Data Engrg. 5 (6) (1993) 914-925.
2. R. Agrawal, R. Srikant, Fast algorithms for mining association rules, in: Proc. Internat. Conf. Very Large Data Bases, Santiago, Chile, 1994, pp. 487-499.
3. D.W. Aha, D. Kibler, M.K. Albert, Instance-based learning algorithms, Machine Learning 6 (1991) 37-66.
4. S.D. Bay, The UCI KDD archive, 1999, http://kdd.ics.uci.edu.
5. C.L. Blake, C.J. Merz, UCI Repository of machine learning databases, 1998, http://www.ics.uci.edu/~mlearn/MLRepository.html.
6. A.L. Blum, P. Langley, Selection of relevant features and examples in machine learning, Artificial Intelligence 97 (1997) 245-271.
7. P.S. Bradley, U. Fayyad, C. Reina, Scaling clustering algorithms to large databases, in: Proceedings of the Fourth International Conference on Knowledge Discovery & Data Mining, New York, AAAI Press, Menlo Park, CA, 1998, pp. 9-15.
8. C.J.C. Burges, A tutorial on support vector machines for pattern recognition, Data Mining and Knowledge Discovery 2 (1998) 121-167.
9. W.G. Cochran, Sampling Techniques, Wiley, New York, 1977.
10. D. Cohn, L. Atlas, R. Ladner, Improving generalization with active learning, Machine Learning 15 (1994) 201-221.
11. S. Das, Filters, wrappers and a boosting-based hybrid for feature selection, in: Proceedings of the Eighteenth International Conference on Machine Learning, Williamstown, MA, 2001, pp. 74-81.
12. M. Dash, H. Liu, Feature selection for classification, Intelligent Data Analysis: An Internat. J. 1 (3) (1997) 131-156.
13. M. Dash, H. Liu, Feature selection for clustering, in: Proceedings of the Fourth Pacific Asia Conference on Knowledge Discovery and Data Mining, (PAKDD-2000), Kyoto, Japan, Springer, Berlin, 2000, pp. 110-121.
14. M. Dash, H. Liu, H. Motoda, Consistency based feature selection, in: Proceedings of the Fourth Pacific Asia Conference on Knowledge Discovery and Data Mining, (PAKDD-2000), Kyoto, Japan, Springer, Berlin, 2000, pp. 98-109.
15. M. Dash, H. Liu, J. Yao, Dimensionality reduction of unsupervised data, in: Proceedings of the Ninth IEEE International Conference on Tools with AI (ICTAI'97), Newport Beach, CA, IEEE Computer Society, 1997, pp. 532-539.
16. T.G. Dietterich, Machine learning research: four current directions, AI Magazine 18 (4) (1997) 97-136.
17. J.G. Dy, C.E. Brodley, Feature subset selection and order identification for unsupervised learning, in: Proceedings of the Seventeenth International Conference on Machine Learning, Stanford, CA, 2000, pp. 247-254.
18. J.G. Dy, C.E. Brodley, Visualization and interactive feature selection for unsupervised data, in: Proceedings of the Sixth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Boston, MA, 2000, pp. 360-364.
19. Y. Freund, H. Seung, E. Shamir, N. Tishby, Selective sampling using the query by committee algorithm, Machine Learning 28 (1997) 133-168.
20. J.H. Friedman, J.L. Bentley, R.A. Finkel, An algorithm for finding best matches in logarithmic expected time, ACM Trans. Math. Software 3 (1977) 209-226.
21. V. Gaede, O. Günther, Multidimensional access methods, ACM Comput. Surv. 30 (2) (1998) 170-231.
22. B. Gu, F. Hu, H. Liu, Sampling: knowing whole from its part, in: H. Liu, H. Motoda (Eds.), Instance Selection and Construction for Data Mining, Kluwer Academic, Boston, MA, 2001, pp. 21-38.
23. M.A. Hall, Correlation based feature selection for machine learning, PhD Thesis, University of Waikato, Dept. of Computer Science, 1999.
24. M.A. Hall, Correlation-based feature selection for discrete and numeric class machine learning, in: Proceedings of the Seventeenth International Conference on Machine Learning (ICML-00), Stanford, CA, Morgan Kaufmann, San Francisco, CA, 2000.
25. Se June Hong, Use of contextual information for feature ranking and discretization, IEEE Trans. Knowledge Data Engrg. 9 (5) (1997) 718-730.
26. T. Joachims, Text categorization with support vector machines: learning with many relevant features, in: C. Nedellec, C. Rouveirol (Eds.), Proceedings of 10th European Conference on Machine Learning, Chemnitz, Germany, Springer, Berlin, 1998, pp. 137-142.
27. Y. Kim, W. Street, F. Menczer, Feature selection for unsupervised learning via evolutionary search, in: Proceedings of the Sixth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Boston, MA, 2000, pp. 365-369.
28. K. Kira, L.A. Rendell, The feature selection problem: traditional methods and a new algorithm, in: Proceedings of the Tenth National Conference on Artificial Intelligence, San Jose, CA, AAAI Press/The MIT Press, Menlo Park, CA, 1992, pp. 129-134.
29. K. Kira, L.A. Rendell, A practical approach to feature selection, in: Sleeman, P. Edwards (Eds.), Proceedings of the Ninth International Conference on Machine Learning (ICML-92), Aberdeen, Scotland, Morgan Kaufmann, San Francisco, CA, 1992, pp. 249-256.
30. J. Kivinen, H. Mannila, The power of sampling in knowledge discovery, in: SIGMOD/PODS'94, ACM, 1994, pp. 77-85.
31. R. Kohavi, G.H. John, Wrappers for feature subset selection, Artificial Intelligence 97 (1-2) (1997) 273-324.
32. I. Kononenko, Estimating attributes: Analysis and extension of RELIEF, in: F. Bergadano, L. De Raedt (Eds.), Proceedings of the European Conference on Machine Learning, Catania, Italy, Springer, Berlin, 1994, pp. 171-182.
33. I. Kononenko, E. Simec, M. Robnik-Sikonja, Overcoming the myopia of inductive learning algorithms with RELIEFF, Appl. Intelligence 7 (1997) 39-55.
34. P. Langley, Selection of relevant features in machine learning, in: Proceedings of the AAAI Fall Symposium on Relevance, AAAI Press, 1994, pp. 140-144.
35. E. Leopold, J. Kindermann, Text categorization with support vector machines. How to represent texts in input space?, Machine Learning 46 (2002) 423-444.
36. D. Lewis, W. Gale, A sequential algorithm for training text classifiers, in: Proceedings of the Seventeenth Annual ACM-SIGR Conference on Research and Development in Information Retrieval, 1994, pp. 3-12.
37. H. Liu, H. Motoda, Feature Selection for Knowledge Discovery and Data Mining, Kluwer Academic, Boston, MA, 1998.
38. H. Liu, H. Motoda (Eds.), Instance Selection and Construction for Data Mining, Kluwer Academic, Boston, MA, 2001.
39. H. Liu, H. Motoda, L. Yu, Feature selection with selective sampling, in: Proceedings of the Nineteenth International Conference on Machine Learning, Sidney, Australia, 2002, pp. 395-402.
40. T.M. Mitchell, Machine Learning, McGraw-Hill, New York, 1997.
41. A.W. Moore, An introductory tutorial on kd-trees, Extract from PhD Thesis Tech Report No. 209, Computer Laboratory, University of Cambridge, Robotics Institute, Carnegie Mellon University, Pittsburgh, PA, 1991.
42. A.Y. Ng, On feature selection: learning with exponentially many irrelevant features as training examples, in: Proceedings of the Fifteenth International Conference on Machine Learning, Madison, WI, 1998, pp. 404-412.
43. K.S. Ng, H. Liu, Customer retention via data mining, AI Rev. 14 (6) (2000) 569-590.
44. K. Nigam, A.K. McCallum, S. Thrun, T. Mitchell, Text classification from labeled and unlabeled documents using EM, Machine Learning 39 (2000) 103-134.
45. M. Robnik-Sikonja, I. Kononenko, An adaptation of relief for attribute estimation in regression, in: Proceedings of Fourteenth International Conference on Machine Learning, Nashville, TN, 1997, pp. 296-304.
46. M. Robnik-Sikonja, I. Kononenko, Attribute dependencies, understandability and split selection in tree based models, in: Proceedings of Sixteenth International Conference on Machine Learning, Bled, Slovenia, 1999, pp. 344-353.
47. M. Robnik-Sikonja, I. Kononenko, Comprehensible interpretation of relief's estimates, in: Proceedings of Eighteenth International Conference on Machine Learning, Williamstown, MA, 2001, pp. 433-440.
48. M. Robnik-Sikonja, I. Kononenko, Theoretical and empirical analysis of Relief and ReliefF, Machine Learning 53 (2003) 23-69.
49. N. Roy, A. MCCallum, Toward optimal active learning through sampling estimation of error reduction, in: Proceedings of the Eighteenth International Conference on Machine Learning, Williamstown, MA, 2001, pp. 441-448.
50. G. Schohn, D. Cohn, Less is more: active learning with support vector machines, in: Proceedings of the Seventeenth International Conference on Machine Learning, Stanford, CA, 2000, pp. 839-846.
51. M.R. Sikonja, Speeding up Relief algorithms with k-d trees, in: Proceedings of the Electrotechnical and Computer Science Conference, ERK'98, 1998.
52. N.A. Syed, H. Liu, K.K. Sung, A study of support vectors on model independent example selection, in: S. Chaudhuri, D. Madigan (Eds.), Proceedings of ACM SIGKDD, International Conference on Knowledge Discovery and Data Mining, ACM, New York, 1999, pp. 272-276.
53. L. Talavera, Feature selection as a preprocessing step for hierarchical clustering, in: Proceedings of International Conference on Machine Learning (ICML'99), Bled, Slovenia, 1999, pp. 389-397.
54. C.A. Thompson, M.E. Califf, R.J. Mooney, Active learning for natural language parsing and information extraction, in: Proceedings of the Sixteenth International Conference on Machine Learning, Bled, Slovenia, Morgan Kaufmann, San Francisco, CA, 1999, pp. 406-414.
55. S. Tong, D. Koller, Support vector machine active learning with applications to text classification, Machine Learning Res. 2 (2001) 45-66.
56. I.H. Witten, E. Frank, Data Mining - Practical Machine Learning Tools and Techniques with JAVA Implementations, Morgan Kaufmann, San Francisco, CA, 2000.
57. E. Xing, M. Jordan, R. Karp, Feature selection for high-dimensional genomic microarray data, in: Proceedings of the Eighteenth International Conference on Machine Learning, Williamstown, MA, 2001.
58. L. Yu, H. Liu, Feature selection for high-dimensional data: a fast correlation-based filter solution, in: Proceedings of the Twentieth International Conference on Machine Learning, Washington, DC, 2003, pp. 856-863.