Learning Bayesian network classifiers by risk minimization

International Journal of Approximate Reasoning

Volumn 53, Issue 2, 2012, Pages 248-272

  Kelner, Roy ^a   Lerner, Boaz ^a  

^a BEN GURION UNIVERSITY OF THE NEGEV   (Israel)

Author keywords

Bayesian networks; Classification; Probabilistic graphical models; Structure learning

Indexed keywords

BAYESIAN NETWORK CLASSIFIERS; BAYESIAN NETWORKS (BNS); CLASSIFICATION ACCURACY; CLASSIFICATION TREES; CONDITIONAL LIKELIHOOD; CROSS VALIDATION; GRAPHICAL MODEL; INFERENCE PROBLEM; LEARNING BAYESIAN NETWORKS; LIKELIHOOD SCORE; LOSS FUNCTIONS; MACHINE-LEARNING; NAIVE BAYESIAN CLASSIFIER; PROBABILISTIC GRAPHICAL MODELS; REAL-WORLD DATASETS; RISK MINIMIZATION; RUNTIMES; STATE OF THE ART; STRUCTURE LEARNING; SYNTHETIC PROBLEM;

ARTS COMPUTING; CLASSIFICATION (OF INFORMATION); EXPERIMENTS; FORESTRY; GRAPHIC METHODS; NEURAL NETWORKS; SPEECH RECOGNITION; TREES (MATHEMATICS);

BAYESIAN NETWORKS;

ACOUSTICS; CLASSIFICATION; EXPERIMENTATION; GRAPHIC ARTS; GRAPHIC METHODS; NEURAL NETWORKS; PROBABILITY; PROBLEM SOLVING;

EID: 84855434565     PISSN: 0888613X     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ijar.2011.10.006     Document Type: Conference Paper

References (57)

1. J. Pearl Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference 1988 Morgan Kaufmann Publishers. Inc. San Francisco, CA, USA
2. G.F. Cooper, and E. Herskovits A Bayesian method for the induction of probabilistic networks from data Mach. Learn. 9 4 1992 309 347
3. D. Heckerman, A tutorial on learning with Bayesian networks, Tech. Rep. MSR-TR-95-06, Microsoft Research, 1995.
4. N. Friedman, D. Geiger, and M. Goldszmidt Bayesian network classifiers Mach. Learn. 29 2-3 1997 131 163 (Pubitemid 127510036)
5. P. Kontkanen, P. Myllymaki, T. Sliander, H. Tirri, On supervised selection of Bayesian networks, in: Proceedings of the 15th International Conference on Uncertainty in Artificial Intelligence (UAI-1999), Morgan Kaufmann Publishers. Inc., San Francisco, CA, USA, 1999, pp. 334-342.
6. D. Grossman, P. Domingos, Learning Bayesian network classifiers by maximizing conditional likelihood, in: Proceedings of the 21st International Conference on Machine Learning (ICML-2004), 2004, pp. 361-368. (Pubitemid 40290829)
7. Y. Guo, R. Greiner, Discriminative model selection for belief net structures, in: Proceeding of the 20th National Conference on Artificial Intelligence (AAAI-2005), 2005, pp. 770-776. (Pubitemid 43006702)
8. S. Acid, L. Campos, and J. Castellano Learning Bayesian network classifiers: Searching in a space of partially directed acyclic graphs Mach. Learn. 59 2005 213 235 (Pubitemid 40890478)
9. T. Roos, H. Wettig, P. Grünwald, P. Myllymäki, and H. Tirri On discriminative Bayesian network classifiers and logistic regression Mach. Learn. 59 3 2005 267 296 (Pubitemid 40890480)
10. F. Pernkopf, and J.A. Bilmes Efficient heuristics for discriminative structure learning of Bayesian network classifiers J. Mach. Learn. Res. 11 2010 2323 2360
11. M.M. Drugan, and M.A. Wiering Feature selection for Bayesian network classifiers using the MDL-FS score Int. J. Approx. Reas. 51 6 2010 695 717
12. D. Heckerman, D. Geiger, and D.M. Chickering Learning Bayesian networks: The combination of knowledge and statistical data Mach. Learn. 20 1995 197 243
13. A.Y. Ng, M.I. Jordan, On discriminative vs. generative classifiers: A comparison of logistic regression and naive Bayes, in: Advances in Neural Information Processing Systems 14 (NIPS-2001), MIT, 1998, vol. 2, 2001, pp. 841-848.
14. T. Jebara Machine Learning: Discriminative and Generative 2004 Kluwer Academic Publishers Norwell, MA, USA
15. C.M. Bishop, and J. Lasserre Generative or discriminative? Getting the best of both worlds J.M. Bernardo, M.J. Bayarri, J.O. Berger, A.P. Dawid, D. Heckerman, A.F.M. Smith, M. West, Bayesian Statistics vol. 8 2007 Oxford University Press 3 24
16. F. Pernkopf, and J. Bilmes Discriminative versus generative parameter and structure learning of Bayesian network classifiers Proceedings of the 22nd International Conference on Machine learning (ICML-2005) 2005 Springer New York, NY, USA 657 664 (Pubitemid 43183390)
17. R. Malka, Bayesian Network Classifiers, Master's thesis, Ben-Gurion University, 2005.
18. F. Pernkopf Bayesian network classifiers versus selective k-NN classifier Pattern Recognit. 38 1 2005 1 10
19. S.L. Lauritzen, and D.J. Spiegelhalter Local computations with probabilities on graphical structures and their application to expert systems J. Roy. Statist. Soc., Ser. B 50 1988 157 224
20. W. Lam, and F. Bacchus Learning Bayesian belief networks: An approach based on the MDL principle Comput. Intell. 10 3 1994 269 293
21. A.P. Dawid Present position and potential developments: Some personal views. Statistical theory. The prequential approach J. Roy. Statist. Soc., Ser. A 147 2 1984 278 292
22. R. Kohavi, A study of cross-validation and bootstrap for accuracy estimation and model selection, in: Proceedings of the 14th International Joint Conference on Artificial Intelligence (IJCAI-1995), 1995, pp. 1137-1143.
23. V.N. Vapnik Statistical Learning Theory 1998 John Wiley & Sons New York
24. R. Cowell Introduction to inference for Bayesian networks M.I. Jordan, Learning Graphical Models 1999 MIT Press Cambridge, Massachusetts 9 26
25. P. Langley, W. Iba, K. Thompson, An analysis of Bayesian classifiers, in: Proceeding of the 10th National Conference on Artificial Intelligence (AAAI-1992), 1992, pp. 223-228.
26. A. Frank, A. Asuncion, UCI Machine Learning Repository, University of California, Irvine, School of Information and Computer Sciences, , 2010.
27. R. Kohavi, G. John, R. Long, D. Manley, and K. Pfleger MLC++: a machine learning library in C++ Proceedings of the 6th International Conference on Tools with Artificial Intelligence (TAI-1994) 1994 IEEE Comput. Soc. Press 740 743
28. K. Murphy The Bayes net toolbox for Matlab Comput. Sci. Stat. 33 2001 331 350
29. P. Leray, O. Francois, BNT structure learning package: Documentation and experiments, Tech. Rep. FRE CNRS 2645, Laboratoire PSI, 2004.
30. C. Chang, C. Lin, LIBSVM: A library for support vector machines, , 2001.
31. R.P.W. Duin, P. Juszczak, P. Paclik, E. Pekalska, D. de Ridder, D.M.J. Tax, S. Verzakov, PRTools4.1: a Matlab toolbox for pattern recognition, Delft University of Technology, 2007.
32. J. Demšar Statistical comparisons of classifiers over multiple data sets J. Mach. Learn. Res. 7 2006 1 30 (Pubitemid 43022939)
33. R.W. Robinson Counting unlabeled acyclic digraphs C.H.C. Little, Combinatorial Mathematics V Lecture Notes in Mathematics vol. 622 1997 Springer Berlin, Heidelberg 28 43
34. L. Breiman, J. Friedman, C.J. Stone, and R.H. Olshen Classification and Regression Trees 1984 Wadsworth Belmont, CA
35. C.M. Bishop Neural Networks for Pattern Recognition 1995 Oxford University Press UK
36. C. Cortes, and V. Vapnik Support vector networks Mach. Learn. 20 1995 273 297
37. B. Lerner, and R. Malka Investigation of the K2 algorithm in learning Bayesian network classifiers Appl. Artif. Intell. 25 2011 74 96
38. R.R. Bouckaert, and E. Frank Evaluating the replicability of significance tests for comparing learning algorithms Proceedings of the 8th Pacific-Asia Conference on Advances in Knowledge Discovery and Data Mining (PAKDD-2004) 2004 Springer 3 12 (Pubitemid 38824878)
39. E.J. Keogh, M.J. Pazzani, Learning augmented Bayesian classifiers: A comparison of distribution-based and classification-based approaches, in: Proceedings of the 7th International Workshop on Artificial Intelligence and Statistics (Uncertainty-1999), Ft. Lauderdale, Florida, 1999, pp. 225-230.
40. J. Cheng, R. Greiner, Comparing Bayesian network classifiers, in: Proceedings of the 15th Conference on Uncertainty in Artificial Intelligence (UAI-1999), Morgan Kaufmann Publishers. Inc., San Francisco, CA, USA, 1999, pp. 101-108.
41. B. Sierra, and P. Larrañaga Predicting the survival in malignant skin Melanoma using Bayesian networks automatically induced by genetic algorithms - an empirical comparison between different approaches Artif. Intell. Med. 14 1998 215 230 (Pubitemid 28446555)
42. C.X. Ling, and H. Zhang Toward Bayesian classifiers with accurate probabilities Proceedings of the 6th Pacific-Asia Conference on Advances in Knowledge Discovery and Data Mining (PAKDD-2002) 2002 Springer Verlag 123 134
43. P. Langley, and S. Sage Induction of selective Bayesian classifiers Proceedings of the 10th Annual Conference on Uncertainty in Artificial Intelligence (UAI-1994) 1994 Morgan Kaufmann Publishers. Inc. San Francisco, CA, USA 399 406
44. M. Singh, G.M. Provan, Efficient learning of selective Bayesian network classifiers, in: Proceedings of the 13th International Conference on Machine Learning (ICML-1996), Morgan Kaufmann Publishers. Inc., San Francisco, CA, USA, 1996, pp. 453-461.
45. R. Kohavi, and G.H. John Wrappers for feature subset selection Artif. Intell. 97 1-2 1997 273 324 (Pubitemid 127401107)
46. C. Aliferis, I. Tsamardinos, A. Statnikov, HITON: A novel Markov blanket algorithm for optimal variable selection, in: Proceedings of the American Medical Informatics Association Annual Symposium (AMIA-2003), 2003, pp. 21-25.
47. R. Greiner, and W. Zhou Structural extension to logistic regression: discriminative parameter learning of belief net classifiers Mach. Learn. 59 3 2005 297 322 (Pubitemid 40890481)
48. M.G. Madden On the classification performance of TAN and general Bayesian networks Knowl.-Based Syst. 22 7 2009 489 495
49. J. Bilmes Dynamic Bayesian multinets Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence (UAI-2000) 2000 Morgan Kaufman Publishers. Inc. San Francisco, CA, USA 38 45
50. L.C. van der Gaag, P.R. de Waal, Multi-dimensional Bayesian network classifiers, in: Proceedings of the 3rd European Workshop on Probabilistic Graphical Models (PGM-2006), 2006, pp. 107-114.
51. H. Borchani, C. Bielza, P. Larrañaga, Learning CB-decomposable multi-dimensional Bayesian network classifiers, in: Proceedings of the 5th European Workshop on Probabilistic Graphical Models (PGM-2010), 2010, pp. 25-32.
52. D. Geiger, and D. Heckerman Knowledge representation and inference in similarity networks and Bayesian multinets Artif. Intell. 82 1-2 1996 45 74 (Pubitemid 126374148)
53. M. Meila, and M.I. Jordan Learning with mixtures of trees J. Mach. Learn. Res. 1 2000 1 48 (Pubitemid 33739061)
54. K. Huang, I. King, M.R. Lyu, Discriminative training of Bayesian Chow-Liu tree multinet classifiers, in: Proceedings of the International Joint Conference on Neural Networks (IJCNN-2003), vol. 1, 2003, pp. 484-488.
55. Y. Gurwicz, and B. Lerner Bayesian class-matched multinet classifier D.Y. Yeung, J.T. Kwok, A.L.N. Fred, F. Roli, D. de Ridder, SSPR/SPR Lecture Notes in Computer Science vol. 4109 2006 Springer 145 153 (Pubitemid 44543101)
56. A. Blum, A. Kalai, and J. Langford Beating the hold-out: bounds for K-fold and progressive cross-validation Proceedings of the 12th Annual Conference on Computational Learning Theory (COLT-1999) 1999 ACM Press New York, USA 203 208 (Pubitemid 32213957)
57. Y. Jing, V. Pavlović, and J.M. Rehg Boosted Bayesian network classifiers Mach. Learn. 73 2 2008 155 184