Learning factor graphs in polynomial time and sample complexity

Journal of Machine Learning Research

Volumn 7, Issue , 2006, Pages 1743-1788

  Abbeel, Pieter ^a   Koller, Daphne ^a   Ng, Andrew Y ^a  

^a Stanford University   (United States)

Author keywords

Bayesian networks; Factor graphs; Markov networks; Parameter and structure learning; Probabilistic graphical models

Indexed keywords

ALGORITHMS; COMPUTATIONAL COMPLEXITY; MARKOV PROCESSES; MAXIMUM LIKELIHOOD ESTIMATION; PARAMETER ESTIMATION; POLYNOMIAL APPROXIMATION; SAMPLING;

BAYESIAN NETWORKS; FACTOR GRAPHS; MARKOV NETWORKS; PARAMETER AND STRUCTURE LEARNING; PROBABILISTIC GRAPHICAL MODELS;

GRAPH THEORY;

EID: 33747670266     PISSN: 15337928     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (41)

1. P. Abbeel, D. Koller, and A. Y. Ng. Learning factor graphs in polynomial time & sample complexity. In Proc. UAI, 2005.
2. N. Abe, J. Takeuchi, and M. Warmuth. Polynomial learnability of probabilistic concepts with respect to the Kullback-Leibler divergence. In Proc. COLT, 1991.
3. N. Abe, J. Takeuchi, and M. Warmuth. On the computational complexity of approximating probability distributions by probabilistic automata. Machine Learning, 1992.
4. F. Bach and M. Jordan. Thin junction trees. In NIPS 14, 2002.
5. F. Barahona. On the computational complexity of Ising spin glass models. J. Phys. A, 1982.
6. J. Besag. Efficiency of pseudo-likelihood estimation for simple Gaussian fields. Biometrika, 1974a.
7. J. Besag. Spatial interaction and the statistical analysis of lattice systems. Journal of the Royal Statistical Society, Series B, 1974b.
8. J. Cheng, R. Greiner, J. Kelly, D. Bell, and W. Liu. Learning Bayesian networks from data: An information-theory based approach. Artificial Intelligence Journal, 2002.
9. D. M. Chickering. Learning Bayesian networks is NP-Complete. In D. Fisher and H.J. Lenz, editors, Learning from Data: Artificial Intelligence and Statistics V, pages 121-130. Springer-Verlag, 1996.
10. D. M. Chickering and C. Meek. Finding optimal Bayesian networks. In Proc. UAI, 2002.
11. D. M. Chickering, C. Meek, and D. Heckerman. Large-sample learning of Bayesian networks is hard. In Proc. UAI, 2003.
12. C. K. Chow and C. N. Liu. Approximating discrete probability distributions with dependence trees. IEEE Transactions on Information Theory, 1968.
13. F. Comets. On consistency of a class of estimators for exponential families of Markov random fields on the lattice. Annals of Statistics, 1992.
14. T. M. Cover and J. A. Thomas. Elements of Information Theory. Wiley, 1991.
15. R. G. Cowell, A. P. Dawid, S. L. Lauritzen, and D. J. Spiegelhalter. Probabilistic Networks and Expert Systems. Springer, 1999.
16. S. Dasgupta. The sample complexity of learning fixed structure Bayesian networks. Machine Learning, 1997.
17. S. Dasgupta. Learning polytrees. In Proc. UAI, 1999.
18. S. Della Pietra, V. J. Della Pietra, and J. D. Lafferty. Inducing features of random fields. IEEE Transactions on Pattern Analysis and Machine Intelligence, 19(4):380-393, 1997.
19. A. Deshpande, M. N. Garofalakis, and M. I. Jordan. Efficient stepwise selection in decomposable models. In Proc. UAI, pages 128-135. Morgan Kaufmann Publishers Inc., 2001. ISBN 1-55860-800-1.
20. N. Friedman and Z. Yakhini. On the sample complexity of learning Bayesian networks. In Proc. UAI, 1996.
21. S. Geman and C. Graffigne. Markov random field image models and their applications to computer vision. In Proc. of the International Congress of Mathematicians, 1986.
22. C. J. Geyer and E. A. Thompson. Constrained Monte Carlo maximum likelihood for dependent data. Journal of the Royal Statistical Society, Series B, 1992.
23. B. Gidas. Consistency of maximum likelihood and pseudo-likelihood estimators for Gibbsian distributions. In W. Fleming and P.-L. Lions, editors, Stochastic differential systems, stochastic control theory and applications. Springer, New York, 1988.
24. X. Guyon and H. R. Künsch. Asymptotic comparison of estimators in the Ising model. In Stochastic Models, Statistical Methods, and Algorithms in Image Analysis, Lecture Notes in Statistics. Springer, Berlin, 1992.
25. J. M. Hammersley and P. Clifford. Markov fields on finite graphs and lattices. Unpublished, 1971.
26. K. L. Höffgen. Learning and robust learning of product distributions. In Proc. COLT, 1993.
27. F. Huang and Y. Ogata. Generalized pseudo-likelihood estimates for Markov random fields on lattice. Annals of the Institute of Statistical Mathematics, 2002.
28. M. Jerrum and A. Sinclair. Polynomial-time approximation algorithms for the Ising model. SIAM J. Comput., 1993.
29. C. Ji and L. Seymour. A consistent model selection procedure for Markov random fields based on penalized pseudolikelihood. Annals of Applied Probability, 1996.
30. D. Karger and N. Srebro. Learning Markov networks: maximum bounded tree-width graphs. In Symposium on Discrete Algorithms, pages 392-401, 2001.
31. F. R. Kschischang, B. J. Frey, and H. A. Loeliger. Factor graphs and the sum-product algorithm. IEEE Transactions on Information Theory, 2001.
32. S. Kullback. Probability theory and statistics, Wiley, 1959.
33. S. L. Lauritzen. Graphical Models. Oxford University Press, 1996.
34. F. M. Malvestuto. Approximating discrete probability distributions with decomposable models. IEEE Transactions on Systems, Man and Cybernetics, 1991.
35. A. McCallum. Efficiently inducing features of conditional random fields. In Proc. UAI, 2003.
36. C. Meek. Finding a path is harder than finding a tree. Journal of Artificial Intelligence Research, 15:383-389, 2001.
37. M. Narasimhan and J. Bilmes. PAC-learning bounded tree-width graphical models. In Proc. UAI, 2004.
38. A. Y. Ng and M. I. Jordan. On discriminative vs. generative classifiers: A comparison of logistic regression and naive Bayes. In NIPS 14, 2002.
39. P. Spirtes, C. Glymour, and R. Scheines. Causation, Prediction, and Search (second edition). MIT Press, 2000.
40. N. Srebro. Maximum likelihood bounded tree-width Markov networks. In Proc. UAI, 2001. V. N. Vapnik. Statistical Learning Theory. John Wiley & Sons, 1998.
41. J. S. Yedidia, W. T. Freeman, and Y. Weiss. Understanding belief propagation and its generalizations. Technical report, Mitsubishi Electric Reseach Laboratories, 2001.