Supervised classification with conditional Gaussian networks: Increasing the structure complexity from naive Bayes

International Journal of Approximate Reasoning

Volumn 43, Issue 1, 2006, Pages 1-25

  Pérez, Aritz ^a   Larrañaga, Pedro ^a   Inza, Iñaki ^a  

^a UNIVERSITY OF THE BASQUE COUNTRY UPV EHU   (Spain)

Author keywords

Bayesian network; Conditional Gaussian network; Filter; k Dependence Bayesian classifiers; Naive Bayes; Semi naive Bayes; Tree augmented naive Bayes; Wrapper

Indexed keywords

ALGORITHMS; BENCHMARKING; COMPUTATIONAL COMPLEXITY; INFORMATION DISSEMINATION; INFORMATION RETRIEVAL; LINEAR CONTROL SYSTEMS;

BAYESIAN NETWORK; BENCHMARK ALGORITHM; GAUSSIAN NETWORK; LINEAR DISCRIMINANT ANALYSIS;

COMPUTER NETWORKS;

EID: 33746334456     PISSN: 0888613X     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ijar.2006.01.002     Document Type: Article

References (64)

1. Anderson F.W. An Introduction to Multivariate Statistical Analysis (1958), John Wiley and Sons
2. S.G. Bottcher, Learning Bayesian networks with mixed variables, PhD thesis, Aalborg University, 2004.
3. Castillo E., Gutierrez J.M., and Hadi A.S. Expert Systems and Probabilistic Network Models (1997), Springer-Verlag
4. J. Cheng, R. Greiner, Comparing Bayesian network classifiers, in: Proceedings of the 15th Conference on Uncertainty in Artificial Intelligence, 1999, pp. 101-107.
5. Chow C., and Liu C. Approximating discrete probability distributions with dependence trees. IEEE Transactions on Information Theory 14 (1968) 462-467
6. Cover T.M., and Thomas J.A. Elements of Information Theory (1991), John Wiley and Sons
7. Cover T.T., and Hart P.E. Nearest neighbour pattern classification. IEEE Transactions on Information Theory 13 (1967) 21-27
8. DeGroot M. Optimal Statistical Decisions (1970), McGraw-Hill, New York
9. Domingos P. A unified bias-variance decomposition and its applications. Proceedings of the 17th International Conference on Machine Learning (2000), Morgan Kaufman 231-238
10. Domingos P., and Pazzani M. On the optimality of the simple Bayesian classifier under zero-one loss. Machine Learning 29 (1997) 103-130
11. Duda R., and Hart P. Pattern Classification and Scene Analysis (1973), John Wiley and Sons
12. Dudewicz E.J., and Mishra S.N. Modern Mathematical Statistics (1988), John Wiley and Sons
13. M. Egmont-Peterson, Feature selection by Markov chain Monte Carlo sampling: a Bayesian approach, in: Proceedings of the Joint IAPR Workshops SSPR 2004 and SPR 2004, 2004, pp. 1034-1042.
14. U. Fayyad, K. Irani, Multi-interval discretization of continuous-valued attributes for classification learning, in: Proceedings of the 13th International Conference on Artificial Intelligence, 1993, pp. 1022-1027.
15. Fisher R.A. The use of multiple measurements. Annals of Eugenics 7 (1936) 179-188
16. Friedman J.H. On bias, variance, 0/1 - loss, and the curse-of-dimensionality. Data Mining and Knowledge Discovery 1 (1997) 55-77
17. Friedman N., Geiger D., and Goldszmidt M. Bayesian network classifiers. Machine Learning 29 (1997) 131-163
18. N. Friedman, M. Goldszmidt, T. Lee, Bayesian network classification with continuous attributes: getting the best of both discretization and parametric fitting, in: Proceedings of the 15th National Conference on Machine Learning, 1998.
19. D. Geiger, D. Heckerman, Learning Gaussian networks, Technical Report, Microsoft Research, Advanced Technology Division, 1994.
20. Geiger D., and Heckerman D. Beyond Bayesian networks: similarity networks and Bayesian multinets. Artificial Intelligence 82 (1996) 45-74
21. German S., Bienenstock E., and Doursat R. Neural networks and the bias-variance dilemma. Neural Computation 4 (1992) 1-58
22. Giudici P., and Green P.J. Decomposable graphical Gaussian model determination. Biometrika 86 4 (1999) 785-801
23. Goldberg D.E. Genetic Algorithms in Search, Optimization and Machine Learning (1989), Addison-Wesley
24. D. Grossman, P. Domingos, Learning Bayesian network classifiers by maximizing conditional likelihood, in: Proceeding of the 21th International Conference on Machine Learning, 2004.
25. Guyon I., and Elisseeff A. An introduction to variable and feature selection. Journal of Machine Learning Research 3 (2003) 1157-1182
26. M.A. Hall, L.A. Smith, Feature subset selection: a correlation based filter approach, in: Proceeding of the Fourth International Conference on Neural Information Processing and Intelligent Information Systems, 1997, pp. 855-858.
27. Heckerman D.E., Geiger D., and Chickering D. Learning Bayesian networks: the combination of knowledge and statistical data. Machine Learning 20 (1995) 197-243
28. James G.M. Variance and bias for general loss functions. Machine Learning 51 (2003) 115-135
29. T. Jebara, Discriminative, generative, and imitative learning, PhD thesis, Massachusetts Institute of Technology, 2001.
30. G. John, P. Langley, Estimating continuous distributions in Bayesian classifiers, in: Proceedings of the 11th Conference on Uncertainty in Artificial Intelligence, 1995, pp. 338-345.
31. Johnson R.A., and Wichern D.W. Applied Multivariate Statistical Analysis (2002), Prentice-Hall
32. Jolliffe I.T. Principal Component Analysis (1986), Springer-Verlag
33. E.J. Keogh, M. Pazzani, Learning augmented Bayesian classifiers: a comparison of distribution-based and non distribution-based approaches, in: Proceedings of the 7th International Workshop on Artificial Intelligence and Statistics, 1999, pp. 225-230.
34. R. Kohavi, Wrappers for performance enhancement and oblivious decision graphs, PhD Thesis, Computer Science department, 1995.
35. Kohavi R., and John G. Wrappers for feature subset selection. Artificial Intelligence 97 1-2 (1997) 273-324
36. R. Kohavi, D.H. Wolpert, Bias plus variance decomposition for zero-one loss functions, in: International Conference on Machine Learning, 1996.
37. I. Kononenko, Semi-naive Bayesian classifiers, in: Proceedings of the 6th European Working Session on Learning, 1991, pp. 206-219.
38. Kudo M. Comparison of algorithms that select features for pattern classifiers. Machine Learning 33 1 (2000) 25-41
39. P. Langley, W. Iba, K. Thompson, An analysis of Bayesian classifiers, in: Proceedings of the 10th National Conference on Artificial Intelligence, 1992, pp. 223-228.
40. P. Langley, S. Sage, Induction of selective Bayesian classifiers, in: Proceedings of the 10th Conference on Uncertainty in Artificial Intelligence, 1994, pp. 399-406.
41. Larrañaga P., and Lozano J.A. Estimation of Distribution Algorithms. A New Tool for Evolutionary Computation (2002), Kluwer Academic Publishers
42. Lauritzen S.L. Graphical Models (1996), Oxford University Press
43. S.L. Lauritzen, N. Wermuth, Mixed interaction models. Technical Report r 84-8, Institute for Electronic Systems, Aalborg University, 1984.
44. Lauritzen S.L., and Wermuth N. Graphical models for associations between variables, some of which are qualitative and some quantitative. Annals of Statistics 17 (1989)
45. Liu H., and Motoda H. Feature Selection for Knowledge Discovery and Data Mining (1998), Kluwer Academic Publishers
46. Minsky M. Steps toward artificial intelligence. Transactions on Institute of Radio Engineers 49 (1961) 8-30
47. P.M. Murphy, D.W. Aha, UCI repository of machine learning databases, Technical Report, University of California at Irvine, 1995. Available from: .
48. Neapolitan R. Learning Bayesian Networks (2003), Prentice-Hall
49. M. Pazzani, Searching for dependencies in Bayesian classifiers, in: Learning from Data: Artificial Intelligence and Statistics V, 1997, pp. 239-248.
50. Pearl J. Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference (1988), Morgan Kaufmann Publishers
51. Pernkopf F. Bayesian network classifier versus k-NN classifier. Pattern Recognition 38 1 (2005) 1-10
52. F. Pernkopf, J. Bilmes, Discriminative versus generative parameter and structure learning of Bayesian network classifiers, in: Proceedings of the 22nd International Conference in Machine Learning, 2005.
53. Quinlan J.R. Induction of decision trees. Machine Learning 1 (1986) 81-106
54. Quinlan J.R. C4.5: Programs for Machine Learning (1993), Morgan Kaufmann
55. R. Raina, Y. Shen, A.Y. Ng, A. McCallum, Classification with hybrid generative/discriminative models, in: Advances in Neural Information Processing Systems 16, 2003.
56. Rosenblatt F. Principles of Neurodynamics (1959), Spartan Books
57. M. Sahami, Learning limited dependence Bayesian classifiers, in: Proceedings of the 2nd International Conference on Knowledge Discovery and Data Mining, 1996, pp. 335-338.
58. G. Santaf́e, J.A. Lozano, P. Larrañaga, Discriminative learning of Bayesian network classifiers via the TM algorithm, in: Proceedings of the Eighth European Conference on Symbolic and Quantitative Approaches to Reasoning with Uncertainty, 2005, pp. 148-160.
59. van der Putten P., and van Someren M. A bias-variance analysis of a real world learning problem: the CoIL challenge 2000. Machine Learning 57 (2004) 177-195
60. H. Wang, Towards a unified framework of relevance, PhD Thesis, Faculty of Informatics, University of Ulster, 1996.
61. Wang H., Bell D., and Murtagh F. Axiomatic approach to feature subset selection based on relevance. IEEE Transactions on Pattern Analysis and Machine Intelligence 21 3 (1999) 271-277
62. Witten I.H., and Frank E. Data Mining: Practical Machine Learning Tools and Techniques (2005), Morgan Kaufmann
63. Y. Yang, G.I. Webb, Discretization for naive-Bayes learning: managing discretization bias and variance, Technical Report 2003-131, School of Computer Science and Software Engineering, Monash University, 2003.
64. Yu L., and Liu H. Efficient feature selection via analysis of relevance and redundancy. Machine Learning Research 5 (2004) 1205-1224