Improving bayesian network structure learning with mutual information-based node ordering in the K2 algorithm

IEEE Transactions on Knowledge and Data Engineering

Volumn 20, Issue 5, 2008, Pages 628-640

  Chen, Xue Wen ^a   Anantha, Gopalakrishna ^a   Lin, Xiaotong ^a  

^a University of Kansas   (United States)

Author keywords

Classification; Data mining; Machine learning

Indexed keywords

BAYESIAN NETWORK STRUCTURE; CLASSIFICATION; COMPLEX NETWORKS; FUNCTION-BASED APPROACH; HARD TASK; K2 ALGORITHM; LEARNING STRUCTURE; MACHINE LEARNING; MUTUAL INFORMATIONS; NETWORK DATA; NODE ORDERING; STRUCTURE-LEARNING;

CLASSIFICATION (OF INFORMATION); COMPUTATIONAL EFFICIENCY; DISTRIBUTED PARAMETER NETWORKS; EDUCATION; INFERENCE ENGINES; INTELLIGENT NETWORKS; LEARNING ALGORITHMS; ROBOT LEARNING; SPEECH RECOGNITION;

BAYESIAN NETWORKS;

EID: 70350314967     PISSN: 10414347     EISSN: None     Source Type: Journal    
DOI: 10.1109/TKDE.2007.190732     Document Type: Article

References (48)

1. J. Pearl, Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, 1988.
2. O. Pietquin and T. Dutoit, "A Probabilistic Framework for Dialog Simulation and Optimal Strategy Learning," IEEE Trans. Speech and Audio Processing, vol.14, no.2, pp. 589-599, 2005.
3. J. Agosta, "The Structure of Bayes Networks for Visual Recognition," Uncertainty in Artificial Intelligence, T.L.R. Shacter, L.N. Kanal, and J.F. Lemmer, eds., vol.4, pp. 397-405, 1990.
4. D. Heckerman, "Probabilistic Similarity Networks," Networks, vol.20, pp. 607-636, 1990.
5. D. Nikovski, "Constructing Bayesian Networks for Medical Diagnosis from Incomplete and Partially Correct Statistics," IEEE Trans. Knowledge and Data Eng., vol.12, pp. 509-516, 2000.
6. N. Friedman, M. Linial, I. Nachman, and D. Pe'er, "Using Bayesian Networks to Analyze Expression Data," J. Computational Biology, vol.7, pp. 601-620, 2000. (Pubitemid 30944025)
7. J. Yu, V. Smith, P. Wang, A. Hartemink, and E. Jarvis, "Advances to Bayesian Network Inference for Generating Causal Networks from Observational Biological Data," Bioinformatics, vol.20, pp. 3594-3603, 2004.
8. L. de Campos, "Independency Relationships and Learning Algorithms for Singly Connected Networks," J. Experimental and Theoretical Artificial Intelligence, vol.10, pp. 511-549, 1998. (Pubitemid 128475478)
9. L. de Campos and J. Huete, "A New Approach for Learning Belief Networks Using Independence Criteria," Int'l J. Approximate Reasoning, vol.24, pp. 11-37, 2000.
10. J. Pearl and T. Verma, "Equivalence and Synthesis of Causal Models," Proc. Sixth Conf. Uncertainty in Artificial Intelligence, 1990.
11. W. Lam and F. Bacchus, "Learning Bayesian Belief Networks. An Approach Based on the MDL Principle," Computational Intelligence, vol.10, pp. 269-293, 1994.
12. G. Cooper and E. Herskovits, "A Bayesian Method for the Induction of Probabilistic Networks from Data," Machine Learning, vol.9, pp. 309-347, 1992.
13. D. Heckerman, D. Geiger, and D. Chickering, "Learning Bayesian Networks: The Combination of Knowledge and Statistical Data," Machine Learning, vol.20, pp. 197-243, 1995.
14. P. Spirtes, C. Glymour, and R. Scheines, "Causation, Prediction and Search," Lecture Notes in Statistics 81, Springer, 1993.
15. N. Wermuth and S. Lauritzen, "Graphical and Recursive Models for Contingence Tables," Biometrika, vol.72, pp. 537-552, 1983.
16. L. Campos and J. Huete, "On the Use of Independence Relationships for Learning Simplified Brief Networks," Int'l J. Intelligent Systems, vol.12, pp. 495-522, 1997.
17. D. Geiger, A. Paz, and J. Pearl, "Learning Causal Trees from Dependence Information," Proc. Eighth Nat'l Conf. Artificial Intelligence (AAAI), 1990.
18. D. Geiger, A. Paz, and J. Pearl, "Learning Simple Causal Structures," Int'l J. Intelligent Systems, vol.8, pp. 231-247, 1993.
19. J. Cheng, D. Bell, and W. Liu, "An Algorithm for Bayesian Belief Network Construction from Data," Proc. Sixth Int'l Workshop Artificial Intelligence and Statistics, 1997.
20. C. Meek, "Causal Inference and Causal Explanation with Background Knowledge," Proc. 11th Conf. Uncertainty in Artificial Intelligence, 1995.
21. W. Buntine, "Theory Refinement on Bayesian Networks," Proc. Seventh Conf. Uncertainty in Artificial Intelligence, pp. 52-60, 1991.
22. D. Chickering, "Learning Equivalence Classes on Bayesian-Network Structures," J. Machine Learning Research, vol.2, pp. 445-498, 2002.
23. D. Chickering, "Optimal Structure Identification with Greedy Search," J. Machine Learning Research, vol.3, pp. 507-554, 2002.
24. W. Buntine, "A Guide to the Literature on Learning Probabilistic Networks from Data," IEEE Trans. Knowledge and Data Eng., vol.8, pp. 195-210, 1996.
25. N. Friedman and D. Koller, "Being Bayesian about Network Structure: A Bayesian Approach to Structure Discovery in Bayesian Networks," Machine Learning, vol.50, pp. 95-125, 2003.
26. D. Heckerman, "A Tutorial on Learning Bayesian Networks," Technical Report MSR-TR-95-106, Microsoft Research, 1996.
27. J. Suzuki, "A Construction of Bayesian Networks from Databases Based on an MDL Principle," Proc. Ninth Conf. Uncertainty in Artificial Intelligence, pp. 266-273, 1993.
28. R. Bouckaert, "Belief Networks Construction Using the Minimum Description Length Principle," Lecture Notes in Computer Science 747, pp. 41-48, 1993.
29. N. Friedman and M. Goldszmidt, "Learning Bayesian Networks with Local Structure," Proc. 12th Conf. Uncertainty in Artificial Intelligence, 1996.
30. C. Chow and C. Liu, "Approximating Discrete Probability Distributions with Dependence Trees," IEEE Trans. Information Theory, vol.14, pp. 462-467, 1968.
31. M. Koivisto and K. Sood, "Exact Bayesian Structure Discovery in Bayesian Networks," J. Machine Learning Research, vol.5, pp. 549-573, 2004.
32. D. Chickering, "Learning Bayesian Networks Is NP-Complete," Learning from Data: Artificial Intelligence and Statistics V, D. Fisher and H. Lenz, eds., pp. 121-130, Springer, 1996.
33. D. Chickering, D. Heckerman, and C. Meek, "Large-Sample Learning of Bayesian Networks Is NP-Hard," J. Machine Learning Research, vol.5, pp. 1287-1330, 2004.
34. S. Acid and L. de Campos, "Searching for Bayesian Network Structures in the Space of Restricted Acyclic Partially Directed Graphs," J. Artificial Intelligence Research, vol.18, pp. 445-490, 2003.
35. R. Castelo and T. Kocka, "On Inclusion-Driven Learning of Bayesian Networks," J. Machine Learning Research, vol.4, pp. 527-574, 2003.
36. P. Larranaga, C. Kuijpers, R. Murga, and Y. Yurramendi, "Learning Bayesian Network Structures by Searching for the Best Ordering with Genetic Algorithms," IEEE Trans. Systems, Man, and Cybernetics, vol.26, pp. 487-493, 1996.
37. P. Larranaga, M. Poza, Y. Yurramendi, R. Murga, and C. Kuijpers, "Structure Learning of Bayesian Networks by Genetic Algorithms: A Performance Analysis of Control Parameters," IEEE Trans. Pattern Analysis and Machine Intelligence, vol.18, pp. 912-926, 1996.
38. D. Chickering, D. Geiger, and D. Heckerman, "Learning Bayesian Networks: Search Methods and Experimental Results," Preliminary Papers Fifth Int'l Workshop Artificial Intelligence and Statistics, 1995.
39. D. Madigan and J. York, "Bayesian Graphical Models for Discrete Data," Int'l Statistical Rev., vol.63, pp. 215-232, 1995.
40. F. Jensen, An Introduction to Bayesian Networks. UCL Press, 1996.
41. J. Proakis, Digital Communications. McGraw-Hill, 2000.
42. G. Chartrand and L. Lesniak, Graphs and Digraph. Chapman and Hall/CRC, 2005.
43. S. Lauritzen and D. Spiegelhalter, "Local Computations with Probabilities on Graphical Structures and Their Application on Expert Systems," J. Royal Statistical Soc., vol.50, pp. 157-224, 1988.
44. I. Beinlich, G. Suermondt, R. Chavez, and G. Cooper, "The ALARM Monitoring System: A Case Study with Two Probabilistic Inference Techniques for Belief Networks," Proc. Second European Conf. Artificial Intelligence in Medicine, 1989.
45. B. Abramson, J. Brown, W. Edwards, A. Murphy, and R. Winkler, "Hailfinder: A Bayesian System for Forecasting Severe Weather," Int'l J. Forecasting, vol.12, pp. 57-71, 1996. (Pubitemid 126160223)
46. J. Cheng, R. Grenier, J. Kelly, D. Bell, and W. Liu, "Learning Bayesian Networks from Data: An Information-Theory Based Approach," Artificial Intelligence, vol.137, pp. 43-90, 2002.
47. http://bnt.sourceforge.net/, 2006.
48. P. Murphy and D. Aha UCI Repository of Machine Learning Database, http://www.ics.uci.edu/mlearn/MLRepository.html, 1995.