Structural extension to logistic regression: Discriminative parameter learning of belief net classifiers

Machine Learning

Volumn 59, Issue 3, 2005, Pages 297-322

  Greiner, Russell ^a   Su, Xiaoyuan ^b   Shen, Bin ^a   Zhou, Wei ^c  

^a UNIVERSITY OF ALBERTA   (Canada)

^b UNIVERSITY OF MIAMI   (United States)

^c UNIVERSITY OF WATERLOO   (Canada)

Author keywords

(Bayesian) belief nets; Classification; Computational sample complexity; Logistic regression; PAC learning

Indexed keywords

COMPUTATIONAL COMPLEXITY; DATA REDUCTION; ERRORS; LEARNING SYSTEMS; MAXIMUM LIKELIHOOD ESTIMATION; PARAMETER ESTIMATION; REGRESSION ANALYSIS;

BAYESIAN BELIEF NETWORK (BN); CLASSIFICATION; COMPUTATIONAL/SAMPLE COMPLEXITY; LOGISTIC REGRESSION; PAC LEARNING;

CLASSIFICATION (OF INFORMATION);

EID: 21244444642     PISSN: 08856125     EISSN: 15730565     Source Type: Journal    
DOI: 10.1007/s10994-005-0469-0     Document Type: Article

References (52)

1. Abe, N., Takeuchi, J., & Warmuth, M. (1991 ). Polynomial learnability of probablistic concepts with respect to the Kullback-Leibler divergence. In Conference on Learning Theory.
2. Binder, J., Koller, D., Russell, S., & Kanazawa, K. (1997). Adaptive probabilistic networks with hidden variables. Machine Learning, 29, 213-244.
3. Bishop, C. (1998). Neural networks for pattern recognition. Oxford.
4. Blake, C., & Merz, C. (2000). UCI repository of machine learning databases, http://www.ics.uci.edu/~mlearn/ML Repository.html.
5. Boyd, S., & Vandenberghe, L. (2004). Convex optimization. Cambridge.
6. Buntine, W. (1996). A guide to the literature on learning probabilistic networks from data. IEEE Transactions on Knowledge and Data Engineering.
7. Cerquides J., & de Mántaras, R.L. (2003). Tractable Bayesian learning of tree augmented naïve Bayes models. In International Conference on Machine Learning (pp. 75-82).
8. Cheng J., & Greiner, R. (1999). Comparing Hayesian network classifiers. In Uncertainty in Artificial Intelligence.
9. Cheng, J., Greiner, R., Kelly, J., Bell, D., & Liu, W. (2002). Learning Bayesian networks from data: An information-theory based approach. Artificial Intelligence, 137.
10. Chickering, D. M., Geiger, D., & Heckerman, D. (1994). Learning Bayesian networks is NP-hard. Technical Report MSR-TR-94-17, Microsoft Research.
11. Chou, W., Juang, B., & Lee, C. (1992). Segmental GPD training of HMM based speech recognizer. In International Conference on Acoustics, Speech and Signal Processing, vol. 1 (pp. 473-476).
12. Chow, C., & Liu, C. (1968). Approximating discrete probability distributions with dependence trees. IEEE Tans. on Information Theory (pp. 462-467).
13. Cooper, G. (1990). The computational complexity of probabilistic inference using Bayesian belief networks. Artificial intelligence, 42.
14. Cooper, G., & Herskovits, E. (1992). A Bayesian method for the induction of probabilistic networks from data. Machine Learning, 9, 309-347.
15. Cox, D. R., & Snell, E. J. (1989). Analysis of binary data. Chapman & Hall, London
16. Darwiche, A. (2000). A differential approach to inference in Bayesian networks. In Uncertainty in Artificial Intelligence.
17. Dasgupta, S. (1997). The sample complexity of Learning fixed-structure Bayesian networks. Machine Learning, 29, 165-180
18. Dash D., & Cooper, G. (2002). Exact model averaging with naïve Bayesian classifiers. In International Conference on Machine Learning (pp. 91-98).
19. Dawid, A. P. (1976). Properties of diagnostic data distributions. Biometrics, 32, 647-658.
20. Dempster, A., Laird, N., & Rubin, D. (1977). Maximum likelihood from incomplete data via the EM algorithm. (with discussion). J. Royal Statistics Society, Series B, 39.
21. Duda, R., & Hart, P. (1973). Pattern classification and scene analysis. Wiley.
22. Edwards, D., & Lauritzen, S. (2001). The TM algorithm for maximising a conditional likelihood function. Biometrika, 88, 961-972.
23. Fayyad, U., & Irani, K. (1993). Multi-interval discretization of continuous-valued attributes for classification learning. In International Joint Conferences on Artificial Intelligence.
24. Friedman, N., Geiger, D., & Goldszmidt, M. (1997). Bayesian network classifiers. Machine Learning Journal, 29, 131-163.
25. Greiner, R., Grove, A., & Schuurmans, D.(1997). Learning Bayesian nets that perform well. In Uncertainty in Artificial Intelligence.
26. Greiner, R., & Zhou, W. (2002). Structural extension to logistic regression: Discriminant parameter learning of belief net classifiers. In American Association of Artificial Intelligence.
27. Grossman, D., & Domingos, P. (2004). Learning Bayesian network classifiers by maximizing conditional likelihood. In International Conference on Machine Learning.
28. Hagan, M., Demuth, H., & Beale, M. (1996). Neural network design. Boston, MA, PWS Publishing.
29. Heckerman, D. E. (1998). A tutorial on learning with Bayesian networks. In M. I. Jordan (Ed.), Learning in graphical models.
30. Jaakkola, T., Meila, M., & Jebara, T. (2000). Maximum entropy discrimination. In Neural Information Processing Systems.
31. Jordan, M. (1995). Why the logistic function? A tutorial discussion on probabilities and neural networks.
32. Kohavi, R. (1995). A study of cross-validation and bootstrap for accuracy estimation and model selection. In International Joint Conference on Artificial Intelligence.
33. Kohavi, R., & John, G. H. (1997). Wrappers for feature subset selection. Artificial intelligence, 97:1-2.
34. Kontkanen, P., Myllymäki, P., Silander, T., & Tirri, H. (1999). On supervised selection of Bayesian networks. In Uncertainty in Artificial Intelligence (pp. 334-342).
35. Laff'erty, J., McCallum, A., & Pereira, F. (2001). Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In International Conference on Machine Learning.
36. Lauritzen, S. (1995). The EM algorithm for graphical association models with missing data. Computational Statistics and Data Analysis, 19, 191-201.
37. Little, J. A., & Rubin, D. B. (1987). Statistical analysis with missing data. New York, Wiley.
38. McCullagh, P., & Nelder, J. (1989). Generalized linear models. Chapman and Hall.
39. Minka. T. (2001). Algorithms for maximum-likelihood logistic regression. Technical report, CMU CALD, http://www.stat.cmu.edu/~minka/papers/logreg/minka- logreg.pdf.
40. Mitchell, T. M. (1997). Machine learning. McGraw-Hill.
41. Nadeau, C., & Bengio, Y. (2003). Inference for the generalization error. Machine Learning, 52, 239-281
42. Nesterov, Y., & Nemirovskii, A. (1994). Interior-point polynomial methods in convex programming, Society for Industrial and Applied Mathematics.
43. Ng, A., & Jordan, M. (2001). On discriminative versus generative classifiers: A comparison of logistic regression and naive Bayes. In Neural Information Processing Systems.
44. Pearl, J. (1988). Probabilistic reasoning in intelligent systems: Networks of plausible inference. San Mateo: Morgan Kaufmann.
45. Press, W. H., Flannery, B. P., Teukolsky, S. A., & Vetterling, W. T. (2002). Numerical recipes in C. Cambridge, http://www.nr.com/.
46. Ripley, B. (1996). Pattern recognition and neural networks, Cambridge, UK: Cambridge University Press.
47. Roos, T., Wettig, H., Grünwald, P., & Myllymäki, P., Tirri, H. (2005). On discriminative Bayesian network classifiers and logistic regression. Machine Learning, 59:3, 269-298.
48. Schlüter, R., Macherey, W., Kanthak, S., Ney, H., & Welling, L. (1997). Comparison of optimization methods for discriminative training criteria. In Proc. of European Conference on Speech Communication and Technology.
49. Shen, B., Su, X., Greiner, R., Musilek, P., & Cheng, C. (2003). Discriminative parameter learning of general Bayesian network classifiers. In International Conference on Tools with Artificial Intelligence.
50. Sundberg, R. (2002). The convergence rate of the TM algorithm of Edwards and Lauritzen. Biometrika, 89, 478-483.
51. Van Allen, T., & Greiner, R. (2000). Model selection criteria for learning belief nets: An empirical comparison. In International Conference on Machine Learning (pp. 1047-1054).
52. Zhou, W. (2002). Discriminative learning of Bayesian net parameters. Master's thesis, Dept of Computing Science, University of Alberta.