Formalism for relevance and its application in feature subset selection

Machine Learning

Volumn 41, Issue 2, 2000, Pages 175-195

  Bell, David A ^a   Wang, Hui ^a  

^a UNIVERSITY OF ULSTER   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHMS; DATA MINING; KNOWLEDGE ACQUISITION; SET THEORY;

FEATURE SUBSET SELECTION (FSS); KNOWLEDGE DISCOVERY;

LEARNING SYSTEMS;

EID: 0034324043     PISSN: 08856125     EISSN: None     Source Type: Journal    
DOI: 10.1023/A:1007612503587     Document Type: Article

References (39)

1. Aha, D. W. & Bankert, R. L. (1994). Feature selection for case-based classification of cloud types. In Working notes of the AAAI94 Workshop on Case-based Reasoning (pp. 106-112). AAAI Press.
2. Almuallim, H. & Dietterich, T. G. (1991). Learning with many irrelevant features. In Proc. Ninth National Conference on Artificial Intelligence (pp. 547-552). MIT Press.
3. Almuallim, H. & Dietterich, T. G. (1994). Learning boolean concepts in the presence of many irrelevant features. Artificial Intelligence, 69, 279-305.
4. Amirikian, B. & Nishimura, H. (1994). What size network is good for generalization of a specific task of interest? Neural Networks, 7(2), 321-329.
5. Blum, A. (1994). Relevant examples & relevant features: thoughts from computational learning theory. In Relevance: Proc. 1994 AAAI Fall Symposium (pp. 14-18). AAAI Press.
6. Blumer, A., Ehrenfeucht, A., Haussler, D., & Warmuth, M. K. (1987). Occam's Razor. Information Processing Letters, 24, 377-380.
7. Carnap, R. (1962). Logical foundations of probability. The University of Chicago Press.
8. Caruana, R. A. & Freitag, D. (1994). Greedy attribute selection. In Proceedings of the 11th international conference on machine learning (pp. 28-36). New Brunswick, NJ: Morgan Kaufmann.
9. Cover, T. M. & Thomas, J. A. 1991. Elements of information theory. John Wiley & Sons, Inc.
10. Davies, S. & Russell, S. 1994. NP-completeness of searches for smallest possible feature sets. In Proceedings of the 1994 AAAI Fall Symposium on Relevance (pp. 37-39). AAAI Press.
11. Fayyad, U. & Irani, K. (1990). What should be minimized in a decision tree? In AAAI-90: Proceedings of 8th National Conference on Artificial Intelligence (pp. 749-754).
12. Fayyad, U. & Irani, K. (1992). The attribute selection problem in decision tree generation. In AAAI-92: Proceedings of 10th National Conference on Artificial Intelligence (pp. 104-110).
13. Gärdenfors, P. (1978). On the logic of relevance. Synthese, 37, 351-367.
14. Gennari, J. H., Langley, P., & Fisher, D. (1989). Models of incremental concept formation. Artificial Intelligence, 40, 11-61.
15. Greiner, R. & Subramanian, D. (Eds.). 1994. In Relevance: Proc. 1994 AAAI Fall Symposium. The AAAI Press. AAAI Technical Report FS-94-02.
16. John, G. H., Kohavi, R., & Pfleger, K. (1994). Irrelevant features and the subset selection problem. In Proceedings of the 11th international conference on machine learning (pp. 121-129). New Brunswick, NJ: Morgan Kaufmann.
17. Keynes, J. M. (1921). A treatise on probability. London: Macmillan.
18. Kira, K. & Rendell, L. A. (1992). The feature selection problem: traditional methods and a new algorithm. In AAAI-92 (pp. 129-134).
19. Kohavi, R. & Sommerfield, D. (1995). Feature subset selection using the wrapper method: Overfitting and dynamic search space topology. In U. M. Fayyad & R. Uthurusamy (Eds.), Proceedings of KDD'95 (pp. 192-197).
20. Kohavi, R. (1994). Feature Subset Selection as Search with Probabilistic Estimates. In R. Greiner, & D. Subramanian (Eds.). Relevance: Proc 1994 AAAI Fall Symposium (pp. 122-126). The AAAI Press.
21. Kononenko, I., Simec, E., & Robnik-Sikonja, M. (1997). Overcoming the myopia of inductive learning algorithms with relieff. Applied Intelligence, 7, 39-55.
22. Kononenko, I. (1994) Estimating attributes: Analysis and extensions of RELIEF. In Proceedings of the 1994 European Conference on Machine Learning (pp. 171-182).
23. Lakemeyer, G. (1995). A Logical account of relevance. In Proc. of IJCAI-95 (pp. 853-859).
24. Littlestone, N. Learning quickly when irrelevant attributes abound: A new linear-threshold algorithm. (1988). Machine learning, 2, 285-318.
25. Liu, H. & Setiono, R. (1998). Feature transformation and multivariate decision tree induction. In Proceedings of The First International Conference on Discovery Science (DS'98) (pp. 279-290). Fukuoka, Japan. Springer-Verlag.
26. Liu, H. & Setiono, R. (1997). Feature selection via discretization of numeric attributes. IEEE Trans on Knowledge and Data Engineering, 9(4), 642-645.
27. Muggleton, S. (ed.). 1992. Inductive Logic Programming. London: Academic Press.
28. Pearl, J. (1988). Probabilistic reasoning in intelligent systems: Networks of plausible inference. San Francisco, California: Morgan Kaufmann Publishers, Inc.
29. Quinlan, J. & Rivest, R. (1989). Inferring decision trees using the minimum description length principle. Information and Computation, 80, 227-248.
30. Rissanen, J. (1986). Stochastic complexity and modeling. Ann. Statist., 14, 1080-1100.
31. Schlimmer, J. C. (1993). Efficiently inducing determinations: A complete and systematic search algorithm that uses optimal pruning. In ML93, pp. 284-290.
32. Schweitzer, H. (1995). Occam algorithms for computing visual motion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17(11), 1033-1042.
33. Shore, J. E. & Johnson, R. W. (1980). Axiomatic derivation of the principle of maximum entropy and the principle of minimum cross-entropy. IEEE Trans. Information Theory, 26, 26-37.
34. Skalak, D. B. (1994). Prototype and feature selection by sampling and random mutation hill-climbing algorithms. In Proceedings of the 11th International Conference on Machine Learning (pp. 293-301). New Brunswick, N.J.: Morgan Kaufmann.
35. Subramanian, D. & Genesereth, M. R. (1987). The relevance of irrelevance. In Proc. of IJCAI-87 (pp. 416-422).
36. Ullman, J. D. (1989). Principles of database and knowledgebase systems. Computer Science Press.
37. Wallace, C. & Freeman, P. (1987). Estimation and inference by compact coding. Journal of the Royal Statistical Society (B), 49, 240-265.
38. Wang, H. (1996). Towards a unified framework of relevance. Ph.D. Thesis, Faculty of Informatics, University of Ulster, N. Ireland, UK. http://www.infj.ulst.ac.uk/~cbcj23/thesis.ps.
39. Wolpert, D. H. (1990). The relationship between Occam's Razor and convergent guessing. Complex Systems, 4, 319-368.