Feature subset selection by Bayesian networks: A comparison with genetic and sequential algorithms

International Journal of Approximate Reasoning

Volumn 27, Issue 2, 2001, Pages 143-164

  Inza, Iñaki ^a   Larrañaga, Pedro ^a   Sierra, Basilio ^a  

^a UNIVERSITY OF THE BASQUE COUNTRY UPV EHU   (Spain)

Author keywords

Bayesian network; Estimation of Bayesian network algorithm; Estimation of distribution algorithm; Feature subset selection; Predictive accuracy; Soft computing

Indexed keywords

EID: 0343773003     PISSN: 0888613X     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0888-613X(01)00038-X     Document Type: Article

References (60)

1. S. Acid, L.M. de Campos, Approximations of causal networks by polytrees: an empirical study, in: B. Bouchon-Meunier, R.R. Yager, L.A. Zadeh (Eds.), Advances in Intelligent Computing, vol. 945, Lecture Notes in Computer Science, Springer, Berlin, 1995, pp. 149-158.
2. D.W. Aha, R.L. Bankert, Feature selection for case-based classification of cloud types: An empirical comparison, in: Proceedings of the AAAI'94 Workshop on Case-Based Reasoning, 1994, pp. 106-112.
3. Alpaydin, E., Combined 5×2 cv F test for comparing supervised classification learning algorithms, Neural Comput. 11, 1885-1982.
4. Bäck T. Evolutionary Algorithms is Theory and Practice. 1996;Oxford University Press, Oxford.
5. S. Baluja, Population-based incremental learning: A method for integrating genetic search based function optimization and competitive learning, Technical Report CMU-CS-94-163, Carnegie Mellon University, Pittsburgh, PA, 1994.
6. S. Baluja, S. Davies, Using optimal dependency-trees for combinatorial optimization: Learning the structure of the search space, in: Proceedings of the Fourteenth International Conference on Machine Learning, 1997, pp. 30-38.
7. Bonissone P.P., Chen Y.-T., Goebel K., Khedkar P.S. Hybrid soft computing systems: Industrial and commercial applications. Proc. IEEE. 87(9):1999;1641-1667.
8. W. Buntine, Theory refinement in Bayesian networks, in: Proceedings of the Seventh Conference on Uncertainty in Artificial Intelligence, 1991, pp. 52-60.
9. Castillo E., Gutiérrez J.M., Hadi A.S. Expert Systems and Probabilistic Network Models. 1997;Springer, Berlin, Germany.
10. B. Cestnik, Estimating probabilities: a crucial task in machine learning, in: Proceedings of the European Conference on Artificial Intelligence, 1990, 147-149.
11. M. Chickering, D. Geiger, D. Heckerman, Learning Bayesian networks is NP-hard, Technical Report MSR-TR-94-17, Microsoft Research, Redmond, WA, 1994.
12. D.M. Chickering, D. Geiger, D. Heckerman, Learning Bayesian networks: Search methods and experimental results, Preliminary Papers of the 5th International Workshop on Artificial Intelligence and Statistics, 1995, pp. 112-128.
13. Dawid A.P. Conditional independence in statistical theory. J. R. Stat. Soc. B. 41:1979;1-31.
14. J.S. De Bonet, C.L. Isbell, P. Viola, MIMIC: Finding optima by estimating probability densities, in: Advances in Neural Information Processing Systems, vol. 9, MIT Press, Cambridge, MA, 1997.
15. J. Doak, An evaluation of feature selection methods and their application to computer security, Technical Report CSE-92-18, University of California, Davis, CA, 1992.
16. R. Etxeberria, P. Larrañaga, Global optimization with Bayesian networks, in: Proceedings of the Second Symposium on Artificial Intelligence, La Habana, Cuba, 1999, pp. 332-339.
17. N. Friedman, Z. Yakhini, On the sample complexity of learning Bayesian networks, in: Proceedings of the Twelfth Conference on Uncertainty in Artificial Intelligence, Portland, OR, 1996, pp. 274-282.
18. Goldberg D.E. Genetic algorithms in search, optimization, and machine learning. 1989;Addison-Wesley, Reading, MA.
19. Grefenstatte J.J. Optimization of control parameters for genetic algorithms. IEEE Trans. Syst., Man Cybern. 1:1986;122-128.
20. G.R. Harik, F.G. Lobo, D.E. Goldberg, The compact genetic algorithm, IlliGAL Report 97006, Illinois Genetic Algorithms Laboratory, University of Illinois, Urbana-Champaign, 1997.
21. G. Harik, Linkage learning via probabilistic modelling in the ECGA, IlliGAL Report 99010, Illinois Genetic Algorithms Laboratory, University of Illinois, Urbana-Champaign, 1999.
22. Henrion, M., Propagating uncertainty in Bayesian networks by probabilistic logic sampling, in: Uncertainty in Artificial Intelligence 2, Elsevier, Amsterdam, 1988, pp. 149-163
23. I. Inza, Feature weighting for nearest neighbor algorithm by Bayesian networks based combinatorial optimization, in: Proceedings of the Student Session of Advanced Course on Artificial Intelligence, 1999, pp. 33-35.
24. Inza I., Larrañaga P., Sierra B., Etxeberria R., Lozano J.A., Peña J.M. Representing the behaviour of supervised classification learning algorithms by Bayesian networks. Pattern Recogn. Lett. 20(11-13):1999;1201-1210.
25. Inza I., Larrañaga P., Etxeberria R., Sierra B. Feature subset selection by Bayesian networks based optimization. Artif. Intell. 123(1-2):2000;157-184.
26. I. Inza, M. Merino, P. Larrañaga, J. Quiroga, B. Sierra, M. Girala, Feature subset selection by genetic algorithms and estimation of distribution algorithms. A case study in the survival of cirrhotic patients treated with TIPS, Artif. Intell. Med., in press.
27. A.K. Jain, R. Chandrasekaran, Dimensionality and sample size considerations in pattern recognition practice, in: Handbook of statistics - II, North-Holland, Amsterdam, The Netherlands, 1982, pp. 835-855.
28. Jain A., Zongker D. Feature selection: Evaluation, application, and small sample performance. IEEE Trans. Pattern Anal. 19(2):1997;153-158.
29. J. Kittler, Feature Set Search Algorithms, in: Pattern Recognition and Signal Processing, Sithoff and Noordhoff, Alphen aan den Rijn, The Netherlands, 1978, pp. 41-60.
30. Kohavi R., John G. Wrappers for feature subset selection. Artif. Intell. 97(1-2):1997;273-324.
31. Kohavi R., Sommerfield D., Dougherty J. Data mining using MLC++, a Machine Learning Library in C++. International Journal of Artificial Intelligence Tools. 6:1997;537-566.
32. Kudo M., Sklansky J. Comparison of algorithms that select features for pattern classifiers. Pattern Recogn. 33:2000;25-41.
33. P. Langley, S. Sage, Induction of selective Bayesian classifiers, in: Proceedings of the Tenth Conference on Uncertainty in Artificial Intelligence, 1994, pp. 399 -406.
34. Larrañaga P., Kuijpers C.M.H., Murga R.H., Yurramendi Y. Learning Bayesian network structures by searching for the best ordering with genetic algorithms. IEEE Trans. Syst. Man Cybern. A. 26(4):1996;487-493.
35. Larrañaga P., Poza M., Yurramendi Y., Murga R.H., Kuijpers C.M.H. Structure Learning of Bayesian networks by genetic algorithms: A performance analysis of control parameters. IEEE Trans. Pattern Anal. 18(9):1996;912-926.
36. P. Larrañaga, R. Etxeberria, J.A. Lozano, J.M. Peña, Combinatorial optimization by learning and simulation of Bayesian networks, in Proceedings of the Conference in Uncertainty in Artificial Intelligence, 2000, pp. 343-352.
37. P. Larrañaga, J.A. Lozano, Estimation of Distribution Algorithms. A New Tool for Evolutionary Computation, Kluwer Academic Publishers, Norwell, MA, 2001.
38. Lauritzen S.L. Graphical Models. 1996;Oxford University Press, Oxford, England.
39. Liu H., Motoda H. Feature Selection for Knowledge Discovery and Data Mining. 1998;Kluwer Academic Publishers, Norwell, MA.
40. Miller A.J. Subset Selection in Regression. 1990;Chapman & Hall, Washington, DC.
41. D. Mladenić, Feature subset selection in text-learning, in: Proceedings of the Tenth European Conference on Machine Learning, 1988, pp. 95-100.
42. Müehlenbein H. The equation for response to selection and its use for prediction. Evol. Comput. 5(3):1997;303-346.
43. H. Müehlenbein, G. Paaß, From recombination of genes to the estimation of distributions, in: Binary Parameters, Lecture Notes in Computer Science 1411: Parallel Problem Solving from Nature - PPSN IV, 1996, pp. 178-187.
44. Müehlenbein H., Mahnig T. FDA - A scalable evolutionary algorithm for the optimization of additively decomposed functions. Evol. Comput. 7(4):1999;353-376.
45. P. Murphy, UCI Repository of Machine Learning Databases, Department of Information and Computer Science, University of California, Irvine, CA, 1995.
46. Narendra P., Fukunaga K. A branch and bound algorithm for feature subset selection. IEEE Trans. Comput. C-26(9):1977;917-922.
47. A. Ochoa, M. Soto, R. Santana, J. Madera, N. Jorge, The factorized distribution algorithm and the junction tree: A learning perspective, in: Proceedings of the Second Symposium on Artificial Intelligence, 1999, pp. 368-377.
48. Pearl J. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Morgan Kaufmann. 1988;Morgan Kaufmann, Palo Alto, CA.
49. M. Pelikan, D.E. Goldberg, E. Cantú-Paz, BOA: The Bayesian Optimization Algorithm, IlliGAL Report 99003, Illinois Genetic Algorithms Laboratory, University of Illinois, Urbana-Champaign, 1999.
50. M. Pelikan, H. Müehlenbein, The bivariate marginal distribution algorithm, in: Advances in Soft Computing-Engineering Design and Manufacturing, Springer, London, 1999, pp. 521-535.
51. G.M. Provan, M. Singh, Learning Bayesian networks using feature selection, Preliminary Papers of the Fifth International Workshop on Artificial Intelligence and Statistics, 1995, pp. 450-456.
52. Pudil P., Novovicova J., Kittler J. Floating search methods in feature selection. Pattern Recogn. Lett. 15(1):1994;1119-1125.
53. Sangüesa R., Cortés U., Gisolfi A. A parallel algorithm for building possibilistic causal networks. Int. J. Approx. Reason. 18(3-4):1998;251-270.
54. Siedelecky W., Sklansky J. On automatic feature selection. Int. J. Pattern Recogn. 2:1998;197-220.
55. Schwarz G. Estimating the dimension of a model. Ann. Stat. 7:1978;461-464.
56. M. Soto, A. Ochoa, S. Acid, L.M. de Campos, Introducing polytree approximation of distribution algorithm, in: Proceedings of the Second Symposium on Artificial Intelligence, 1999, pp. 360-367.
57. G. Syswerda, Uniform crossover in genetic algorithms, in: Proceedings of the International Conference on Genetic Algorithms, vol. 3, 1989, pp. 2-9.
58. H. Vafaie, K. De Jong, Robust feature selection algorithms, in: Proceedings of the Fifth International Conference on Tools with Artificial Intelligence, 1993, 356-363.
59. Wong M.L., Lam W., Leung K.S. Using Evolutionary programming and minimum description length principle for data mining of Bayesian networks. IEEE Trans. Pattern Anal. 21(2):1999;174-178.
60. Xiang Y., Chu T. Parallel learning of belief networks in large and difficult domains. Data Min. Knowl. Disc. 3(3):1999;315-338.