Learning bayesian network equivalence classes with ant colony optimization

Journal of Artificial Intelligence Research

Volumn 35, Issue , 2009, Pages 391-447

  Daly, Rónán ^a   Shen, Qiang ^b  

^a UNIVERSITY OF GLASGOW   (United Kingdom)

^b ABERYSTWYTH UNIVERSITY   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ANT COLONY OPTIMIZATION; ARTIFICIAL INTELLIGENCE; EQUIVALENCE CLASSES;

ANT COLONY OPTIMIZATION (ACO); GREEDY SEARCH; LEARNING BAYESIAN NETWORKS; META HEURISTIC ALGORITHM; STATE OF THE ART; UNCERTAIN KNOWLEDGE;

BAYESIAN NETWORKS;

EID: 68349117241     PISSN: None     EISSN: 10769757     Source Type: Journal    
DOI: 10.1613/jair.2681     Document Type: Article

References (81)

1. Abramson, B., Brown, J., Edwards, W., Murphy, A., & Winkler, R. L. (1996). Hailfinder: A Bayesian system for forecasting severe weather. International Journal of Forecasting, 12(1), 57-71. doi: 10.1016/0169-2070(95) 00664-668 (Pubitemid 126160223)
2. Acid, S., & de Campos, L. M. (2003). Searching for Bayesian network structures in the space of restricted acyclic partially directed graphs. Journal of Artificial Intelligence Research, 18, 445-490. (Pubitemid 41525942)
3. Aitken, S., Jirapech-Umpai, T., & Daly, R. (2005). Inferring gene regulatory networks from classified microarray data: Initial results. BMC Bioinformatics, 6(Suppl 3), S4. doi: 10.1186/1471-2105-6-S3-S4.
4. Akaike, H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19(6), 716-723. doi: 10.1109/TAC.1974.1100705.
5. Andersson, S. A., Madigan, D., & Perlman, M. D. (1997). A characterization of Markov equivalence classes for acyclic digraphs. The Annals of Statistics, 25(2), 505-541. doi: 10.1214/aos/1031833662. (Pubitemid 127064053)
6. Andreassen, S., Hovorka, R., Benn, J., Olesen, K. G., & Carson, E. (1991). A model-based approach to insulin adjustment. In Proceedings of the Third Conference on Artificial Intelligence in Medicine (AIME 91), volume 44 of Lecture Notes in Medical Informatics, (pages 239-249).
7. Baluja, S. (1994). Population-based incremental learning: A method for integrating genetic search based function optimization and competitive learning. Technical Report CMU-CS-94-163, School of Computer Science, Carnegie Mellon University.
8. Beinlich, I., Suermondt, H., Chavez, R., & Cooper, G. (1989). The ALARM monitoring system: A case study with two probabilistic inference techniques for belief networks. In Proceedings of the Second European Conference on Artificial Intelligence in Medicine (AIME 89), volume 38 of Lecture Notes in Medical Informatics, (pages 247-256). Springer.
9. Birattari, M., Caro, G. D., & Dorigo, M. (2002). Toward the formal foundation of ant programming. In Proceedings of the Third International Workshop on Ant Algorithms, volume 2463 of LectureNotes in Computer Science, (pages 188-201). Springer-Verlag.
10. Bonabeau, E., Dorigo, M., & Theraulaz, G. (1999). Swarm Intelligence: From Natural to Artificial Systems. Studies in the Sciences of Complexity. Oxford University Press.
11. Bullnheimer, B., Hartl, R. F., & Strauss, C. (1999). An improved ant system algorithm for the vehicle routing problem. Annals of Operations Research, 89, 319-328. doi: 10.1023/A: 1018940026670.
12. Buntine, W. (1991). Theory refinement on Bayesian networks. In B. D'Ambrosio, & P. Smets (Eds.), Proceedings of the Seventh Annual Conference on Uncertainty in Artificial Intelligence (UAI '91), (pages 52-60). Morgan Kaufmann.
13. Castelo, R., & Kočka, T. (2003). On inclusion-driven learning of Bayesian networks. Journal of Machine Learning Research, 4, 527-574.
14. Cheng, J., Greiner, R.,Kelly, J., Bell, D.,& Liu, W. (2002). Learning Bayesian networks from data: An information-theory based approach. Artificial Intelligence, 137(1-2), 43-90. doi: 10.1016/S0004-3702(02)00191-201 (Pubitemid 34405215)
15. Chickering, D. M. (1995). A transformational characterization of equivalent Bayesian network structures. In P. Besnard, & S. Hanks (Eds.), Proceedings of the Eleventh Conference on Uncertainty in Artificial Intelligence (UAI-95), (pages 87-98). Morgan Kaufmann.
16. Chickering, D. M. (1996a). Learning Bayesian networks is NP-complete. In D. Fisher, & H.-J. Lenz (Eds.), Learning from Data: Artificial Intelligence and Statistics V, volume 112 of Lecture Notes in Statistics, (pages 121-130). Springer.
17. Chickering, D. M. (1996b). Learning equivalence classes of Bayesian network structures. In E. Horvitz, & F. Jensen (Eds.), Proceedings of the Twelfth Conference on Uncertainty in Artificial Intelligence (UAI-96), (pages 150-157). Morgan Kaufmann.
18. Chickering, D. M. (2002a). Learning equivalence classes of Bayesian-network structures. Journal of Machine Learning Research, 2, 445-498.
19. Chickering, D. M. (2002b). Optimal structure identification with greedy search. Journal of Machine Learning Research, 3, 507-554.
20. Chickering, D. M., Geiger, D., & Heckerman, D. (1996). Learning Bayesian networks: Search methods and experimental results. In D. Fisher, & H.-J. Lenz (Eds.), Learning from Data: Artificial Intelligence and Statistics V, volume 112 of Lecture Notes in Statistics, (pages 112-128). Springer.
21. Chickering, D. M., Heckerman, D., & Meek, C. (2004). Large-sample learning of Bayesian networks is NP-hard. Journal of Machine Learning Research, 5, 1287-1330.
22. Conover, W. J. (1999). Practical Nonparametric Statistics. John Wiley & Sons, Third edition.
23. Cooper, G. F.,& Herskovits, E. (1992). A Bayesian method for the induction of probabilistic networks from data. Machine Learning, 9(4), 309-347. doi: 10.1007/BF00994110.
24. Costa, D., & Hertz, A. (1997). Ants can colour graphs. Journal of the Operational Research Society, 48(3), 295-305. doi: 10.1057/palgrave.jors. 2600357. (Pubitemid 127608040)
25. Cotta, C., & Muruzábal, J. (2004). On the learning of Bayesian network graph structures via evolutionary programming. In P. Lucas (Ed.), Proceedings of the Second European Workshop on Probabilistic Graphical Models, (pages 65-72).
26. Cowell, R. (2001). Conditions under which conditional independence and scoring methods lead to identical selection of Bayesian network models. In J. Breese, & D. Koller (Eds.), Proceedings of the Seventeenth Conference on Uncertainty in Artificial Intelligence (UAI-01), (pages 91-97). Morgan Kaufmann.
27. Dagum, P., & Luby, M. (1993). Approximating probabilistic inference in Bayesian belief networks is NP-hard. Artificial Intelligence, 60(1), 141-154. doi: 10.1016/0004-3702(93)90036-B. (Pubitemid 23622908)
28. Daly, R., Shen, Q., & Aitken, S. (2006). Speeding up the learning of equivalence classes of Bayesian network structures. In A. P. del Pobil (Ed.), Proceedings of the Tenth IASTED International Conference on Artificial Intelligence and Soft Computing, (pages 34-39). ACTA Press.
29. Daly, R., Shen, Q., & Aitken, S. (2009). Learning Bayesian networks: Approaches and issues. The Knowledge Engineering Review. In press.
30. de Campos, L. M., & Castellano, J. G. (2007). Bayesian network learning algorithms using structural restrictions. International Journal of Approximate Reasoning, 45(2), 233-254. doi: 10.1016/j.ijar.2006.06.009. (Pubitemid 46880196)
31. de Campos, L. M., Fernández-Luna, J. M., Gámez, J. A., & Puerta, J. M. (2002). Ant colony optimization for learning Bayesian networks. International Journal of Approximate Reasoning, 31(3), 291-311. doi: 10.1016/S0888-613X(02)00091-99 (Pubitemid 35411834)
32. de Campos, L. M., Gámez, J. A., & Puerta, J. M. (2002). Learning Bayesian networks by ant colony optimisation: Searching in two different spaces. Mathware & Soft Computing, 9(2-3).
33. de Campos, L. M., & Puerta, J. M. (2001a). Stochastic local algorithms for learning belief networks: Searching in the space of the orderings. In S. Benferhat, & P. Besnard (Eds.), Proceedings of the Sixth European Conference on Symbolic and Quantitative Approaches to Reasoning with Uncertainty (ECSQARU 2001), volume 2143 of Lecture Notes in Artificial Intelligence, (pages 228-239). Springer. doi: 10.1007/3-540-44652-4-21.
34. de Campos, L. M., & Puerta, J. M. (2001b). Stochastic local and distributed search algorithms for learning belief networks. In Proceedings of the Third International Symposium on Adaptive Systems: Evolutionary Computation and Probabilistic Graphical Models, (pages 109-115). ICIMAF.
35. Deneubourg, J. L., Aron, S., Goss, S., & Pasteels, J. M. (1990). The self-organizing exploratory pattern of the argentine ant. Journal of Insect Behavior, 3(2), 159-168. doi: 10.1007/BF01417909.
36. Dor, D., & Tarsi, M. (1992). A simple algorithm to construct a consistent extension of a partially oriented graph. Technical Report R-185, Cognitive Systems Laboratory, Department of Computer Science, UCLA.
37. Dorigo, M. (1992). Ottimizzazione, apprendimento automatico, ed algoritmi basati su metafora naturale. Ph.D. thesis, Politecnico di Milano, Italy.
38. Dorigo, M., & Di Caro, G. (1999). The ant colony optimization meta-heuristic. In D. Corne, M. Dorigo, & F. Glover (Eds.), New Ideas in Optimization, (pages 11-32). McGraw-Hill.
39. Dorigo, M., & Gambardella, L. M. (1997). Ant colony system: A cooperative learning approach to the traveling salesman problem. IEEE Transactions on Evolutionary Computation, 1(1), 53-66. doi: 10.1109/4235.585892. (Pubitemid 127771018)
40. Dorigo, M., Maniezzo, V., & Colorni, A. (1996). The Ant System: Optimization by a colony of cooperating agents. IEEE Transactions on Systems, Man, and Cybernetics Part B: Cybernetics, 26(1), 29-41. doi: 10.1109/3477.484436. (Pubitemid 126780573)
41. Dorigo, M., & Stützle, T. (2004). Ant Colony Optimization. The MIT Press.
42. Friedman, N., Nachman, I., & Pêer, D. (1999). Learning Bayesian network structure from massive datasets: The "Sparse Candidate" algorithm. In H. Prade, & K. Laskey (Eds.), Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence (UAI-99), (pages 206-215). Morgan Kaufmann.
43. Gambardella, L. M., & Dorgio, M. (2000). An ant colony system hybridized with a new local search for the sequential ordering problem. INFORMS Journal on Computing, 12(3), 237-255. doi: 10.1287/ijoc.12.3.237.12636.
44. Gambardella, L. M.,& Dorigo, M. (1995). Ant-Q: A reinforcement learning approach to the travelling salesman problem. In A. Prieditis, & S. J. Russell (Eds.), Proceedings of the Twelfth International Conference on Machine Learning (ICML 1995), (pages 252-260). Morgan Kaufmann.
45. Glover, F. (1989). Tabu search-Part I. ORSA Journal on Computing, 1(3), 190-206.
46. Glover, F. (1990). Tabu search-Part II. ORSA Journal on Computing, 2(1), 4-32.
47. Heckerman, D. (1995). A tutorial on learning with Bayesian networks. Technical Report MSR-TR-95-106, Microsoft Research.
48. Heckerman, D., Geiger, D., & Chickering, D. M. (1995). Learning Bayesian networks: The combination of knowledge and statistical data. Machine Learning, 20(3), 197-243. doi: 10.1023/A: 1022623210503.
49. Heckerman, D., Mamdani, A., & Wellman, M. P. (1995). Real-world applications of Bayesian networks. Communications of the ACM, 38(3), 24-26. doi: 10.1145/203330.203334.
50. Heckerman, D., Meek, C., & Cooper, G. (1999). A Bayesian approach to causal discovery. In C. Glymour, & G. F. Cooper (Eds.), Computation, Causation, & Discovery, (pages 141-165). AAAI Press.
51. Jensen, A. L. (1995). A probabilistic model based decision support system for mildew management in winter wheat. Ph.D. thesis, Dina Foulum, Research Center Foulum, Aalborg University.
52. Kayaalp, M., & Cooper, G. F. (2002). A Bayesian network scoring metric that is based on globally uniform parameter priors. In A. Darwiche, & N. Friedman (Eds.), Proceedings of the Eighteenth Conference on Uncertainty in Artificial Intelligence (UAI-02), (pages 251-258). Morgan
53. Kirkpatrick, S., Gelatt, C. D., Jr., & Vecchi, M. P. (1983). Optimization by simulated annealing. Science, 220(4598), 671-680. doi: 10.1126/science.220.4598.671.
54. Kristensen, K., & Rasmussen, I. A. (2002). The use of a Bayesian network in the design of a decision support system for growing malting barley without use of pesticides. Computers and Electronics in Agriculture, 33(3), 197-217. doi: 10.1016/S0168-1699(02)00007-8. (Pubitemid 34475872)
55. Kullback, S., & Leibler, R. A. (1951). On information and sufficiency. The Annals of Mathematical Statistics, 22(1), 79-86. doi: 10.1214/aoms/1177729694.
56. Levine, J., & Ducatelle, F. (2004). Ant colony optimisation and local search for bin packing and cutting stock problems. Journal of the Operational Research Society, 55(7), 705-716. doi: 10.1057/palgrave.jors.2601771.
57. Maniezzo, V., & Colorni, A. (1999). The ant system applied to the quadratic assignment problem. IEEE Transactions on Knowledge and Data Engineering, 11(5), 769-778. doi: 10.1109/69.806935. (Pubitemid 30518732)
58. Microsoft Research (1995). Win95pts. A model for printer troubleshooting in Microsoft Windows 95.
59. Mitchell, M. (1996). An Introduction to Genetic Algorithms. MIT Press.
60. Moore, A., & Wong, W.-K. (2003). Optimal reinsertion: A new search operator for accelerated and more accurate Bayesian network structure learning. In T. Fawcett, & N. Mishra (Eds.), Proceedings of the Twentieth International Conference on Machine Learning, (pages 552-559). AAAI Press.
61. Mühlenbein, H. (1997). The equation for response to selection and its use for prediction. Evolutionary Computation, 5(3), 303-346. doi: 10.1162/evco.1997.5.3.303. (Pubitemid 127718107)
62. Munteanu, P., & Bendou, M. (2001). The EQ framework for learning equivalence classes of Bayesian networks. In Proceedings of the 2001 IEEE International Conference on Data Mining, (pages 417-424). IEEE Computer Society. doi: 10.1109/ICDM.2001.989547.
63. Munteanu, P., & Cau, D. (2000). Efficient score-based learning of equivalence classes of Bayesian networks. In D. A. Zighed, H. J. Komorowski, & J. M. Zytkow (Eds.), Proceedings of the Fourth European Conference on the Principles of Data Mining and Knowledge Discovery (PKDD 2000), volume 1910 of Lecture Notes in Computer Science, (pages 96-105). Springer. doi: 10.1007/3-540-45372-5-10.
64. Muruzábal, J., & Cotta, C. (2004). A primer on the evolution of equivalence classes of Bayesian network structures. In X. Yao, E. Burke, J. A. Lozano, J. Smith, J. J. Merelo-Guervós, J. A. Bullinaria, J. Rowe, P. Ti?no, A. Kabán, & H.-P. Schwefel (Eds.), Proceedings of the 8th International Conference on Parallel Problem Solving from Nature -PPSN VIII, volume 3242 of Lecture Notes in Computer Science, (pages 612-621). Springer. doi: 10.1007/b100601.
65. Nielsen, J. D., Kočka, T., & Peña, J. (2003). On local optima in learning Bayesian networks. In C. Meek, & U. Kjærulff (Eds.), Proceedings of the Ninteenth Conference on Uncertainty in Artificial Intelligence, (pages 435-444). Morgan Kaufmann.
66. Ott, S., Imoto, S., & Miyano, S. (2004). Finding optimal models for small gene networks. In Proceedings of the Ninth Pacific Symposium on Biocomputing, (pages 557-567). World Scientific.
67. Ott, S., & Miyano, S. (2003). Finding optimal gene networks using biological constraints. Genome Informatics, 14, 124-133.
68. Rasmussen, L. K. (1995). Bayesian Network for Blood Typing and Parentage Verification of Cattle. Ph.D. thesis, Dina Foulum, Research Center Foulum.
69. Schwarz, G. (1978). Estimating the dimension of a model. The Annals of Statistics, 6(2), 461-464. doi: 10.1214/aos/1176344136.
70. Shaughnessy, P., & Livingston, G. (2005). Evaluating the causal explanatory value of Bayesian network structure learning algorithms. Research Paper 2005-2013, Department of Computer Science, University of Massachusetts Lowell.
71. Shimony, S. E. (1994). Finding maps for belief networks is NP-hard. Artificial Intelligence, 68(2), 399-410. doi: 10.1016/0004-3702(94)90072-90078
72. Silander, T., Kontkanen, P., & Myllymaki, P. (2007). On sensitivity of the MAP Bayesian network structure to the equivalent sample size parameter. In Proceedings of the Twenty-Third Conference on Uncertainty in Artificial Intelligence (UAI-07).
73. Spirtes, P., & Glymour, C. (1990). An algorithm for fast recovery of sparse causal graphs. Report CMU-PHIL-15, Department of Philosophy, Carnegie Mellon University.
74. Spirtes, P., Glymour, C., & Scheines, R. (2000). Causation, Prediction, and Search. Adaptive Computation and Machine Learning. The MIT Press, 2nd edition.
75. Spirtes, P., Meek, C., & Richardson, T. (1995). Causal inference in the presence of latent variables and selection bias. In P. Besnard, & S. Hanks (Eds.), Proceedings of the Eleventh Conference on Uncertainty in Artificial Intelligence (UAI-95), (pages 499-506). Morgan Kaufmann.
76. Steck, H., & Jaakkola, T. S. (2003). On the Dirichlet prior and Bayesian regularization. In S. Becker, S. Thrun, & K. Obermayer (Eds.), Advances in Neural Information Processing Systems 15 (NIPS*2002), (pages 697-704). The MIT Press.
77. Stützle, T. (1998). An ant approach to the flow shop problem. In Proceedings of the Sixth European Congress on Intelligent Techniques and Soft Computing (EUFIT '98), volume 3, (pages 1560 -1564). Aachen, Germany: ELITE Foundation.
78. Sutton, R. S., & Barto, A. G. (1998). Reinforcement Learning: An Introduction. MIT Press.
79. Tsamardinos, I., Brown, L. E., & Aliferis, C. F. (2006). The max-min hill-climbing Bayesian network structure learning algorithm. Machine Learning, 65(1), 31-78. doi: 10.1007/s10994-006-6889-7. (Pubitemid 44451193)
80. van der Putten, P., & van Someren, M. (2004).A bias-variance analysis of a real world learning problem: The coil challenge 2000.Machine Learning, 57(1-2), 177-195. doi: 10.1023/B: MACH.0000035476.95130.99.
81. Verma, T., & Pearl, J. (1991). Equivalence and synthesis of causal models. In P. Bonissone, M. Henrion, L. Kanal, & J. Lemmer (Eds.), Uncertainty in Artificial Intelligence 6, (pages 255-268). North-Holland.