Feature selection for Bayesian network classifiers using the MDL-FS score

International Journal of Approximate Reasoning

Volumn 51, Issue 6, 2010, Pages 695-717

  Drugan, Mǎdǎlina M ^a   Wiering, Marco A ^b  

^a UTRECHT UNIVERSITY   (Netherlands)

^b UNIVERSITY OF GRONINGEN   (Netherlands)

Author keywords

Feature subset selection; Minimum Description Length; Selective Bayesian classifiers; Tree augumented networks

Indexed keywords

BAYESIAN NETWORKS; FEATURE EXTRACTION; REDUNDANCY;

BAYESIAN NETWORK CLASSIFIERS; CLASSIFICATION ACCURACY; FEATURE SUBSET SELECTION; MINIMUM DESCRIPTION LENGTH; REDUNDANT FEATURES; SELECTIVE BAYESIAN CLASSIFIERS; TREE AUGUMENTED NETWORKS; VARIOUS FUNCTIONS;

CLASSIFICATION (OF INFORMATION);

EID: 78650818467     PISSN: 0888613X     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ijar.2010.02.001     Document Type: Article

References (57)

1. P. Langley, W. Iba, K. Thompson, An analysis of Bayesian classifiers, in: Conference on Artificial Intelligence, 1992, pp. 223-228.
2. S. Kotsiantis, Supervised machine learning: a review of classification techniques, Informatica 31 (2007) 249-268. (Pubitemid 350208045)
3. H. Liu, H. Motoda, Computational Methods of Feature Selection, Chapman & Hall/CRC, 2007.
4. I. Inza, P. Larranaga, B. Sierra, Feature subset selection by Bayesian networks: a comparison with genetic and sequential algorithms, International Journal of Approximate Reasoning 27 (2) (2001) 143-164. (Pubitemid 32610599)
5. A. Blum, P. Langley, Selection of relevant features and examples in machine learning, Artificial Intelligence 97 (1-2) (1997) 245-271. (Pubitemid 127401106)
6. I. Tsamardinos, C. Aliferis, Towards principled feature selection: relevance, filters, and wrappers, in: Workshop AI&Stat, 2003.
7. R. Nilsson, J. Pena, J. Bjorkegren, J. Tegner, Consistent feature selection for pattern recognition in polynomial time, Journal of Machine Learning Research 8 (2007) 589-612. (Pubitemid 46473524)
8. J. Pena, R. Nilsson, J. Bjorkegren, J. Tegner, Towards scalable and data efficient learning of Markov boundaries, International Journal of Approximate Reasoning 45 (2) (2007) 211-232. (Pubitemid 46880195)
9. M. Minsky, Steps toward artificial intelligence, Transactions of the Institute and Radio Engineers 46 (1961) 8-30.
10. J. Rissanen, Modeling by shortest data description, Automatica 14 (1978) 465-471.
11. N. Friedman, D. Geiger, M. Goldszmidt, Bayesian network classifiers, Machine Learning 29 (2-3) (1997) 131-163. (Pubitemid 127510036)
12. Y. Guo, R. Greiner, Discriminative model selection for belief net structures, in: Proceedings of the American Association for Artificial Intelligence (AAAI), 2005, pp. 770-776. (Pubitemid 43006702)
13. E. Hruschka, M. do Carmo Nicoletti, V. de Oliveira, G. Bressan, Bayesrule: a Markov-blanket based procedure for extracting a set of probabilistic rules from Bayesian classifiers, Hybrid Intelligent Systems 5 (2008) 83-96.
14. Y. Jing, V. Pavlovic, J. Rehg, Boosted Bayesian network classifiers, Machine Learning 73 (2) (2008) 155-184.
15. A. Antonucci, M. Zaffalon, Fast algorithms for robust classification with Bayesian nets, International Journal of Approximate Reasoning 44 (3) (2007) 200-223. (Pubitemid 46157171)
16. A. Perez, P. Larranaga, I. Inaki, Bayesian classifiers based on kernel density estimation: flexible classifiers, International Journal of Approximate Reasoning 50 (2) (2009) 341-362.
17. L.M. de Campos, A.E. Romero, Bayesian network models for hierarchical text classification from a thesaurus, International Journal of Approximate Reasoning 50 (7) (2009) 932-944.
18. M.M. Drugan, L.C. van der Gaag, A new MDL-based feature selection for Bayesian network classifiers, in: Proceedings of the European Conference on Artificial Intelligence, 2004, pp. 999-1000.
19. D. Grossman, P. Domingos, Learning Bayesian networks classifiers by maximising conditional likelihood, in: Proceedings of the International Conference on Machine Learning, 2004, pp. 361-368. (Pubitemid 40290829)
20. G. Santafè, J. Lozano, P. Larranaga, Discriminative vs. generative learning of Bayesian network classifiers, in: Proceedings of Symbolic and Quantitative Approaches to Reasoning with Uncertainty, 2007, pp. 453-464.
21. Y. Rubinstein, T. Hastie, Discriminative vs informative learning, in: Knowledge Discovery and Data Mining, 1997, pp. 49-53.
22. X. Zhu, X. Wu, Class noise vs. attribute noise: a quantitative study, Artificial Intelligence Review 22 (2004) 177-210.
23. C.E. Shannon, A mathematical theory of communication, Bell System Technical Journal 27 (1948) 379-423. 623-656.
24. P. Domingos, M. Pazzani, On the optimality of the simple Bayesian classifier under zero-one loss, Machine Leaning 29 (1997) 103-130. (Pubitemid 127510035)
25. M.H. Hansen, B. Yu, Model Selection and the Principle of Minimum Description Length, Journal of the American Statistical Association 96 (454) (2001) 746-774. (Pubitemid 33695629)
26. J. Rissanen, Strong optimality of the normalized ML models as universal codes and information in data, IEEE Transactions on Information Theory 47 (2001) 1712-1717. (Pubitemid 32644690)
27. P. Kontkanen, W. Buntine, P. Myllymaki, J. Rissanen, H. Tirri, Efficient computation of stochastic complexity, in: Workshop AI&Stat, 2003, pp. 181-188.
28. T. Silander, T. Roos, P. Myllymaki, Learning locally minimax optimal Bayesian networks, International Journal of Approximate Reasoning 51 (5) (2010) 544-557.
29. P. Langley, S. Sage, Induction of selective Bayesian classifiers, in: Proceedings of Uncertainty in Artificial Intelligence (UAI), 1994, pp. 399-406.
30. M.A. Hall, Correlation-based feature selection for discrete and numeric class machine learning, in: Proceedings of the International Conference on Machine Learning (ICML), 2000, pp. 359-366.
31. G.H. John, R. Kohavi, K. Pfleger, Irrelevant features and the subset selection problem, in: Proceedings of the International Conference on Machine Learning (ICML), 1994, pp. 121-129.
32. R. Kohavi, G.H. John, Wrappers for feature subset selection, Artificial Intelligence (1997) 273-324. (Pubitemid 127401107)
33. D. Koller, M. Sahami, Toward optimal features selection., in: Proceedings of the International Conference on Machine Learning (ICML), 1996, pp. 284-292.
34. H. Liu, J. Sun, L. Liu, H. Zhang, Feature selection with dynamic mutual information, Pattern Recognition 42 (2009) 1330-1339.
35. R. Blanco, I. Inza, M. Merino, J. Quiroga, P. Larranaga, Feature selection in Bayesian classifiers for the prognosis of survival of cirrhotic patients treated with tips, Journal of Biomedical Informatics 38 (2005) 376-388. (Pubitemid 41405808)
36. L. Yu, H. Liu, Efficient feature selection via analysis of relevance and redundancy, Machine Learning 5 (2004) 1205-1224.
37. P. Kontkanen, P. MyllymSki, T. Silander, H. Tirri, BAYDA: software for Bayesian classification and feature selection, in: Proceedings of Knowledge discovery and Data-Mining (KDD), 1998, pp. 254-258.
38. T. Jebara, T. Jaakola, Feature selection and dualities in maximum entropy discrimination, in: Proceedings of Uncertainty in Artificial Intelligence (UAI), 2000, pp. 291-300.
39. J. Bilmes, Dynamic Bayesian multinets, in: Proceedings of Uncertainty in Artificial Intelligence (UAI), 2000, pp. 38-45.
40. F. Pernkopf, J. Bilmes, Discriminative versus generative parameter and structure learning of Bayesian network classifiers, in: Proceedings of the International Conference on Machine Learning (ICML), 2005, pp. 657-664. (Pubitemid 43183390)
41. J. Burge, T. Lane, Learning class-discriminative dynamic Bayesian networks, in: Proceedings of the International Conference on Machine Learning, 2005, pp. 97-104. (Pubitemid 43183321)
42. I. Guyon, A. Elisseeff, An introduction to variable and feature selection, Machine Learning 3 (2003) 1157-1182.
43. M. Drugan, Conditional log-likelihood MDL and Evolutionary MCMCs, Ph.D. Thesis, Utrecht University, The Netherlands, 2006.
44. C. Chow, C. Liu, Approximating discrete probability distributions with dependence trees, IEEE Transaction on Information Theory (14) (1968) 462-467.
45. L. Yu, H. Liu, Feature selection for high-dimensional data: a fast correlation-based filter solution, in: Proceedings of the International Conference on Machine Learning (ICML), 2003, pp. 856-863.
46. K. Kira, L. Rendell, A practical approach to feature selection, in: Proceedings of the International Conference on Machine Learning (ICML), 1992, pp. 249-256.
47. I. Kononenko, Estimating attributes: Analysis and extensions of relief, in: Proceedings of the European Conference on Machine Learning (ECML), 1994, pp. 171-182.
48. S. Gadat, L. Younes, A stochastic algorithm for feature selection in pattern recognition, Journal of Machine Learning Research 8 (2007) 509-547. (Pubitemid 46434117)
49. Y. Sun, Iterative relief for feature weighting: algorithms, theories and applications, IEEE Transactions on Pattern Analysis and Machine Intelligence 29 (6) (2007) 1035-1051. (Pubitemid 46667414)
50. F. Fleuret, Fast binary feature selection with conditional mutual information, Machine Learning 5 (2004) 1531-1555.
51. J. Huang, N. Lv, S. Li, Y. Cai, Feature selection for classificatory analysis based on information-theoretic criteria, Acta Automatica Sinica 34 (3) (2008) 383-392. (Pubitemid 351558621)
52. J. Liang, S. Yang, A. Winstanley, Invariant optimal feature selection: a distance discriminant and feature ranking based solution, Pattern Recognition 41 (5) (2008) 1429-1439.
53. Y. Song, F. Nie, C. Zhang, S. Xiang, A unified framework for semi-supervised dimensionality reduction, Pattern Recognition 41 (9) (2008) 2789-2799.
54. Z. Zhao, H. Liu, Spectral feature selection for supervised and unsupervised learning, in: Proceedings of the International Conference on Machine Learning (ICML), 2007, pp. 1151-1157. (Pubitemid 47275183)
55. U.M. Fayyad, K.B. Irani, Multi-interval discretization of continuous-valued attributes for classification learning, in: Joint Conference on Artificial Intelligence, 1993, pp. 1022-1027.
56. P. Langley, Selection of relevant features in machine learning, in: Proceedings of the Association for the Advancement of Artificial Intelligence (AAAI), 1994, pp. 140-144.
57. D. Goldberg, Genetic Algorithms in Search, Optimization and Machine Learning, Addison Wesley, 1989.