The hidden life of latent variables: bayesian learning with mixed graph models

Journal of Machine Learning Research

Volumn 10, Issue , 2009, Pages 1187-1238

  Silva, Ricardo ^a,b   Ghahramani, Zoubin ^b,c  

^a UNIVERSITY COLLEGE LONDON   (United Kingdom)

^b UNIVERSITY OF CAMBRIDGE   (United Kingdom)

^c Carnegie Mellon University   (United States)

Author keywords

Bayesian inference; Graphical models; Latent variable models 2009 ricardo silva and zoubin ghahramani; Markov chain monte carlo; Structural equation models

Indexed keywords

BAYESIAN; BAYESIAN INFERENCE; BAYESIAN LEARNING; CONDITIONAL INDEPENDENCES; COVARIANCE MATRICES; DIRECTED ACYCLIC GRAPHS; GAUSSIAN; GRAPHICAL MODEL; GRAPHICAL MODELS; HIDDEN VARIABLE; LATENT VARIABLE; MACHINE-LEARNING; MARGINALIZATION; MARKOV CHAIN MONTE CARLO; MIXED GRAPH; MODEL DEPENDENCIES; STRUCTURAL EQUATION MODELS;

BAYESIAN NETWORKS; COVARIANCE MATRIX; EQUIVALENCE CLASSES; GRAPHIC METHODS; INFERENCE ENGINES; MARKOV PROCESSES; MONTE CARLO METHODS; ROBOT LEARNING;

MATHEMATICAL MODELS;

EID: 67650499890     PISSN: 15324435     EISSN: 15337928     Source Type: Journal    
DOI: None     Document Type: Article

References (55)

1. A. Al-Awadhi and P. Garthwaite. An elicitation method for multivariate normal distributions. Communications in Statistics - Theory and Methods, 27:1123-1142, 1998.
2. Albert and S. Chib. Bayesian analysis of binary and polychotomous response data. Journal of the American Statistical Association, 88:669-679, 1993.
3. A. Atay-Kayis and H. Massam. A Monte Carlo method for computing the marginal likelihood in nondecomposable Gaussian graphical models. Biometrika, 92:317-335, 2005.
4. Bartholomew and M. Knott. Latent Variable Models and Factor Analysis. Arnold Publishers, 1999.
5. Beal. Variational algorithms for approximate Bayesian
6. inference. PhD. Thesis, Gatsby Computational Neuroscience Unit, University College London, 2003.
7. Beal and Z. Ghahramani. Variational Baeysian learning of directed graphical models with hidden variables. Bayesian Analysis, 1:793-832, 2006.
8. P. Bickel and E. Levina. Covariance regularization by thresholding. Annals of Statistics, 36:25772604, 2008.
9. K. Bollen. Structural Equation Models with Latent Variables. John Wiley & Sons, 1989.
10. P. Brown, N. Le, and J. Zidek. Inference for a covariance matrix. In P.R. Freeman, A.F.M. Smith (editors), Aspects of Uncertainty, a tribute to D. V Lindley, pages 77-92, 1993.
11. Z. Chen and D. Dunson. Random effects selection in linear mixed models. Biometrics, 59:762-769, 2003.
12. Chickering. Optimal structure identification with greedy search. Journal of Machine Learning Research, 3:507-554, 2002.
13. Daniels and R. Kass. Nonconjugate Bayesian estimation of covariance matrices and its use in hierarchical models. Journal of the American Statistical Association, 94:1254-1263, 1999.
14. Drton and M. Perlman. Multiple testing and error control in Gaussian graphical model selection. Statistical Science, pages 430-449, 2007.
15. Drton and T. Richardson. A new algorithm for maximum likelihood estimation in Gaussian models for marginal independence. Proceedings of the 19th Conference on Uncertainty in Artificial Intelligence, 2003.
16. Drton and T. Richardson. Iterative conditional fitting for Gaussian ancestral graph models. Proceedings of the 20th Conference on Uncertainty in Artificial Intelligence, 2004.
17. Drton and T. Richardson. Binary models for marginal independence. Department of Statistics, University of Washington, Tech. report 474, 2005.
18. Drton and T. Richardson. Binary models for marginal independence. Journal of the Royal Statistical Society, Series B, 70:287-309, 2008a.
19. Drton and T. Richardson. Graphical methods for efficient likelihood inference in Gaussian covariance models. Journal of Machine Learning Research, pages 893-914, 2008b.
20. Dunson, J. Palomo, and K. Bollen. Bayesian structural equation modeling. Statistical and Applied Mathematical Sciences Institute, Technical Report #2005-5, 2005.
21. N. Friedman and D. Koller. Being Bayesian about network structure: a Bayesian approach to structure discovery in Bayesian networks. Machine Learning Journal, 50:95-126, 2003.
22. P. Hoyer, D. Janzing, J. Mooij, J. Peters, and B. Schölkopf. Nonlinear causal discovery with additive noise models. Neural Information Processing Systems, 2008.
23. J. Huang and B. Frey. Cumulative distribution networks and the derivative-sum-product algorithm. Proceedings of 24th Conference on Uncertainty in Artificial Intelligence, 2008.
24. B. Jones, C. Carvalho, A. Dobra, C. Hans, C. Carter, and M. West. Experiments in stochastic computation for high-dimensional graphical models. Statistical Science, 20:388-400, 2005.
25. Jordan. Learning in Graphical Models. MIT Press, 1998.
26. Jordan, Z. Ghaharamani, T. Jaakkola, and L. Saul. An introduction to variational methods for graphical models. In M. Jordan (Ed.), Learning in Graphical Models, pages 105-162, 1998.
27. Kang and J. Tian. Local Markov property for models satisfying the composition axiom. Proceedings of 21st Conference on Uncertainty in Artificial Intelligence, 2005.
28. J. Kotecha and P. Djuric. Gibbs sampling approach for the generation of truncated multivariate Gaussian random variables. Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, pages 1757-1760, 1999.
29. S. Lauritzen. Graphical Models. Oxford University Press, 1996.
30. S.-Y. Lee. Structural Equation Modeling: a Bayesian Approach. Wiley, 2007.
31. MacKay. Introduction to Monte Carlo methods. Learning in Graphical Models, pages 175-204, 1998.
32. Murray, Z. Ghahramani, and D. MacKay. MCMC for doubly-intractable distributions. Proceedings of 22nd Conference on Uncertainty in Artificial Intelligence, 2006.
33. R. Neal. Bayesian Learning for Neural Networks. Springer-Verlag, 1996.
34. R. Neal. Annealed importance sampling. Statistics and Computing, 11:125-139, 2001.
35. J. Pearl. Probabilistic Reasoning in Expert Systems: Networks of Plausible Inference. Morgan Kaufmann, 1988.
36. J. Pearl. Causality: Models, Reasoning and Inference. Cambridge University Press, 2000.
37. Pitt, D. Chan, and R. Kohn. Efficient Bayesian inference for Gaussian copula regression models. Biometrika, 93:537-554, 2006.
38. T. Richardson. Markov properties for acyclic directed mixed graphs. Scandinavian Journal of Statistics, 30:145-157, 2003.
39. T. Richardson and P. Spirtes. Ancestral graph Markov models. Annals of Statistics, 30:962-1030, 2002.
40. A. Roverato. Hyper inverse Wishart distribution for non-decomposable graphs and its application to Bayesian inference for Gaussian graphical models. Scandinavian Journal of Statistics, 29: 391-411, 2002.
41. R. Scheines, R. Hoijtink, and A. Boomsma. Bayesian estimation and testing of structural equation models. Psychometrika, 64:37-52, 1999.
42. R. Silva and Z. Ghahramani. Bayesian inference for Gaussian mixed graph models. Proceedings of 22nd Conference on Uncertainty in Artificial Intelligence, 2006.
43. R. Silva and Z. Ghahramani. Factorial mixtures of Gaussians and the marginal independence model. Artificial Intelligence & Statistics (AISTATS '09), 2009.
44. R. Silva and R. Scheines. Bayesian learning of measurement and structural models. 23rd International Conference on Machine Learning, 2006.
45. R. Silva, R. Scheines, C. Glymour, and P. Spirtes. Learning the structure of linear latent variable models. Journal of Machine Learning Research, 7:191-246, 2006.
46. R. Silva, W. Chu, and Z. Ghahramani. Hidden common cause relations in relational learning. Neural Information Processing Systems (NIPS '07), 2007.
47. J. Skilling. Nested sampling for general Bayesian computation. Bayesian Analysis, 1:833-860, 2006.
48. P. Spirtes. Directed cyclic graphical representations of feedback models. Proceedings of 11th Conference on Uncertainty in Artificial Intelligence, 1995.
49. P. Spirtes, C. Glymour, and R. Scheines. Causation, Prediction and Search. Cambridge University Press, 2000.
50. R. Tarjan. Decomposition by clique separators. Discrete Mathematics, 55:221-232, 1985.
51. E. Webb and J. Forster. Bayesian model determination for multivariate ordinal and binary data. Technical report, Southampton Statistical Sciences Research Institute, 2006.
52. F. Wong, C. Carter, and R. Kohn. Efficient estimation of covariance selection models. Biometrika, 90:809-830, 2003.
53. S. Wright. Correlation and causation. Journal of Agricultural Research, pages 557-585,1921.
54. J. Yedidia, W. Freeman, and Y. Weiss. Constructing free-energy approximations and generalized belief propagation algorithms. IEEE Transactions on Information Theory, 51:2282-2312, 2005.
55. J. Zhang. Causal reasoning with ancestral graphs. Journal of Machine Learning Research, pages 1437-1474, 2008.