The art of data augmentation

Journal of Computational and Graphical Statistics

Volumn 10, Issue 1, 2001, Pages 1-50

  Dyk, David A Van ^a   Meng, Xiao Li ^b  

^a HARVARD UNIVERSITY   (United States)

^b UNIVERSITY OF CHICAGO   (United States)

Author keywords

Auxiliary variables; Conditional augmentation; Em algorithm; Gibbs sampler; Haar measure; Hierarchical models; Marginal augmentation; Markov chain Monte Carlo; Mixed effects models; Nonpositive recurrent markov chain; Posterior distributions; Probit regression; Rate of convergence

Indexed keywords

EID: 0002696801     PISSN: 10618600     EISSN: 15372715     Source Type: Journal    
DOI: 10.1198/10618600152418584     Document Type: Article

References (66)

1. Albert, J. H. (1992), “Bayesian Estimation of Normal Ogive Item Response Curves Using Gibbs Sampling,” Journalof EducationalStatistics, 17, 251-269.
2. Albert, J. H., and Chib, S. (1993), “Bayesian Analysis of Binary and Polychotomous Response Data,” Journal of the American Statistical association, 88, 669-679.
3. Amit, Y. (1991), “On Rates of Convergence of Stochastic Relaxation for Gaussian and Non-Gaussian Distribu tions,” Journal ofMultivariate Analysis, 38, 82-89.
4. Besag, J., and Green, P. J. (1993), “Spatial Statistics and Bayesian Computation,” Journalof the Royal Statistical Society, Ser. B, 55, 25-37.
5. Carlin, B. P., andLouis, T. A. (1996), BayesandEmpiricalBayesMethodsforDataAnalysis, New York: Chapman and Hall.
6. Casella, G. (1996), “Statistical Inference and Monte Carlo Algorithms” (with discussion), Test, 5, 249-344.
7. Ceperley, D. M. (1995), “Path Integrals in the Theory of Condensed Helium,” Reviews of Modern Physics, 67, 279-355.
8. Cowles, M. K., and Carlin, B. P. (1996), “Markov Chain Monte Carlo Convergence Diagnostics: A Comparative Review,” Journal of the American Statistical Association, 91, 883-904.
9. Damien, P., Wakefield, J., and Walker, S. (1999), “Gibbs Sampling for Bayesian Nonconjugate and Hierarchical Models Using Auxiliary Variables,” Journal of the RoyalStatisticalSociety, Ser. B, 61, 331-344.
10. Dempster, A. P., Laird, N. M., and Rubin, D. B. (1977), MaximumLikelihoodEstimationFromIncomplete Data via the EM Algorithm” (with discussion), Journal of the Royal Statistical Society, Ser. B, 39, 1-38.
11. Edwards, R. G., and Sokal, A. D. (1988), “Generalization of the Fortuin-Kasteleyn-Swendsen-WangRepresenta- tion and Monte Carlo Algorithm,” Physical Review Letters, 38, 2009-2012.
12. Fessler, J. A., and Hero, A. O. (1994), “Space-Alternating Generalized Expectation-Maximization Algorithm,” IEEE Transactionson SignalProcessing, 42, 2664-2677.
13. Foulley, J. L., and van Dyk, D. A. (2000), “ThePX-EM Algorithmfor Fast StableFitting of Henderson's Mixed Model,” Genetics Selection Evolution, 32, 143-163.
14. Gelfand, A. E., Sahu, S. K., andCarlin, B. P. (1995), “EfficientParameterizationforNormalLinearMixedModels,” Biometrika, 82, 479-488.
15. Gelman, A., Carlin, J., Stern, H., and Rubin, D. B. (1995), Bayesian Data Analysis, London: Chapman and Hall.
16. Gelman, A., and Rubin, D. B. (1992), “Inference From Iterative Simulation Using Multiple Sequences” (with discussion), StatisticalScience, 7, 457-472.
17. Geman, S., and Geman, D. (1984), “Stochastic Relaxation, Gibbs Distributions and the Bayesian Restoration of Images,” IEEE Transactionson Pattern Analysis andMachineIntelligence, 6, 721-741.
18. Gilks, W. R., Richardson, S., and Spiegelhalter, D. J. (eds) (1996), Markov ChainMonteCarloinPractice, London: Chapman and Hall.
19. Gray, A. J. (1993), “Discussion of the Gibbs Sampler and Other Markov Chain Monte Carlo Methods,” Journal of the RoyalStatisticalSociety, Ser. B, 55, 58-61.
20. Green, P. (1997), Discussion of “TheEM Algorithm-An Old FolkSong Sung to a Fast New Tune,” by Meng and van Dyk, Journal of the Royal Statistical Society, Ser. B, 59, 554-555.
21. Haas, M. (1994), “IgG Subclass Deposits in Glomeruli of Lupus and Nonlupus Membranous Nephropathies,” American Journal ofKidney Disease, 23, 358-364.
22. Haas, M. (1998), “Value of IgG Subclasses and Ultrastructural Markers in Predicting Latent Membranous Lupus Nephritis,” Modern Pathology, 11, 147A.
23. Higdon, D. M. (1993), “Discussion of the Gibbs Sampler and Other Markov Chain MonteCarlo Methods,” Journal of the RoyalStatisticalSociety, Ser. B, 55, 78.
24. Higdon, D. M. (1998), “Auxiliary Variable Methods for Markov Chain Monte Carlo With Applications,” Journal of the American Statistical Association, 93, 585-595.
25. Hobert, J. P. (in press), “Stability Relationships Among the Gibbs Sampler and its Subchains,” Journal ofCom- putational and Graphical Statistics.
26. Hobert, J. P., and Casella, G. (1996), “The Effect of Improper Priors on Gibbs Sampling in Hierarchical Linear Mixed Models,” Journal of the American Statistical association, 91, 1461-1473.
27. Hobert, J. P., and Casella, G. (1998), “FunctionalCompatibility, Markov Chains, and Gibbs Sampling With ImproperPosteriors,” Jour nal of Computationaland Graphical Statistics, 7, 42-60.
28. Ising, E. (1925), “Beitrag zur Theorie des Ferromagnetismus,” ZeitschriftfurPhysik, 31, 253-258.
29. Laird, N., Lange, N., and Stram, D. (1987), “Maximizing Likelihood Computations With Repeated Measures: Application of the EM Algorithm,” Journalof the American Statistical association, 82, 97-105.
30. Laird, N. M., and Ware, J. H. (1982), “RandomEffects ModelsforLongitudinalData,” Biometrics, 38, 967-974.
31. Lange, N., Carlin, B. P., and Gelfand, A. E. (1992), “Hierarchical Bayes Models for the Progression of HIV Infection Using Longitudinal CD4 T-cells Numbers,” Journal of the American Statistical Association, 87, 615-632.
32. Lindstrom, M. J., and Bates, D. M. (1988), “Newton-Raphson and EM Algorithms for Linear Mixed-Effects Models for Repeated-Measure Data,” Journal of the American Statistical association, 83, 1014-1022.
33. Liu, C. (1999), “Covariance Adjustment for Markov Chain Monte Carlo-A General Framework and the Covariance-Adjusted Data Augmentation Algorithm,” Technical Report, Bell Labs, Lucent Technologies.
34. Liu, C., and Rubin, D. B. (1994), “The ECME Algorithm: A Simple Extension of EM and ECM With Fast Monotone Convergence,” Biometrika, 81, 633-648.
35. Liu, C., Rubin, D. B., andWu, Y. N. (1998), “Parameter Expansionto Accelerate EM-ThePX-EM Algorithm,” Biometrika, 85, 755-770.
36. Liu, J. S. (1994), “Fraction of Missing Information and Convergence Rate of Data Augmentation,” in Compu tationally Intensive Statistical Methods: Proceedings of the 26th Symposium Interface. eds J. Sall and A. Lehman, Fairfax Station, VA: Interface Foundation, pp. 490-497.
37. Liu, J. S., and Sabatti, C. (2000), “Generalized Gibbs Sampler and Multigrid Monte Carlo for Bayesian Compu tation,” Biometrika, 87, 353-369.
38. Liu, J. S., Wong, W. H., and Kong, A. (1994), “Covariance Structure of the Gibbs Sampler With Applications to Comparisons of Estimators and Augmentation Schemes,” Biometrika, 81, 27-40.
39. Liu, J. S., and Wu, Y. N. (1999), “Parameter ExpansionScheme for Data Augmentation,” Journalof theAmerican Statistical association, 94, 1264-1274.
40. McCulloch, R., and Rossi, P. (1994), “An Exact Likelihood Analysis of the Multinomial Probit Model,” Journal of Econometrics, 64, 207-240.
41. Meng, X. -L. (1994), “OntheRate of Convergence of theECM Algorithm,” TheAnnalsOf statistics, 22, 326-339.
42. Meng, X. -L., and Rubin, D. B. (1994), “On the Global and Component-Wise Rates of Convergence of the EM Algorithm,” Linear Algebraand its Applications, 199, 413-425.
43. Meng, X. -L., and Schilling, S. (1996), “Fitting Full-Information Factor Models and an Empirical Investigation of Bridge Sampling,” Journal of the American Statistical association, 91, 1254-1267.
44. Meng, X. -L., and van Dyk, D. A. (1997), “The EM Algorithm-An Old Folk Song Sung to a Fast New Tune” (with discussion), Journal of the Royal StatisticalSociety, Ser. B, 59, 511-567.
45. Meng, X. -L., and van Dyk, D. A. (1998), “Fast EM Implementations for Mixed-Effects Models,” Journal of the Royal Statistical Society, Ser. B, 60, 559-578.
46. Meng, X. -L., and van Dyk, D. A. (1999), “Seeking Efficient Data Augmentation Schemes via Conditional and Marginal Augmentation,” Biometrika, 86, 301-320.
47. Mira, A., and Tierney, L. (1997), “On the Use of Auxiliary Variables in Markov Chain Monte Carlo Sampling,” Technical Report, University of Minnesota, School of Statistics.
48. Nachbin, L. (1965), The Haar Integral, Princeton: Van Nostrand.
49. Neal, R. M. (1993), ProbabilisticInference Using Markov chain monte Carlo Methods, Technical Report CRG- TR-93-1, Department of ComputerScience, University of Toronto, 140 pages.
50. Neal, R. M. (1997), “MarkovChain MonteCarlo MethodsBased on 'Slicing'the Density Function,”Techincal Report No. 9722, Department of Statistics, University of Toronto.
51. Pawitan, Y. (2000), “A Reminder of the Fallibility of the Ward Statistic: Likelihood Explanation,” The American Statistician, 54, 54-56.
52. Pierson, R. A., and Ginther, O. J. (1987), “FollicularPopulationDynamics During the Estius Cycle of the Mare,” Animal ReproductionScience, 14, 219-231.
53. Potts, R. B. (1952), “Some Generalized Order-Disorder Transformations,” Proceedings of the CambridgePhilo- sophicSociety, 48, 106-109.
54. Roberts G. O. (1996), “Markov Chain Concepts Related to Sampling Algorithms,” in Markov ChainMonte Carlo in Practice, eds. W. R. Gilks, S. Richardson, and D. J. Spiegelhalter, London:Chapman and Hall, pp. 45-58.
55. Roberts, G. O., and Rosenthal, J. S. (1997), “Convergence of Slice Sampler Markov Chains,” Technical Report, Cambridge University, Statistical Laboratory.
56. Rubin, D. B. (1983), “Iteratively Reweighted Least Squares,” in Encyclopedia of Statistical Sciences, 4, eds. S. Kotz, N. L. Johnson, and C. B. Reed, New York: Wiley, pp. 272-275.
57. Sahu, S. K., and Roberts, G. O. (1999), “On Convergence of the EM Algorithm and the Gibbs Sampler,” Statistics and Computing, 9, 55-64.
58. Speckman, P. L., Lee, J., and Sun, D. (1999), “Existenceof theMLE andProprietyofPosteriorsforaGeneral Multi nomial Choice Model,” Technical Report, Department of Statistics, The University of Missouri-Columbia.
59. Sun, D., Tsutakawa, R. K., and He, Z. (1997), “Propriety ofPosteriorsWithlmproperPriors in Hierarchical Linear Mixed Models,” Technical Report, Department of Statistics, The University of Missouri-Columbia.
60. Swendsen, R. H., andWang, J. S. (1987), “NonuniversalCritical Dynamics in MonteCarlo Simulations,” Physical Review Letters, 58, 86-88.
61. Tanner, M. A. (1996), Tools for Statistical Inference: Methods for the Exploration ofPosterior Distributions and LikelihoodFunctions, New York: Springer-Verlag.
62. Tanner, M. A., andWong, W. H. (1987), “The CalculationofPosteriorDistributionsby Data Augmentation” (with discussion), Journal of the American Statistical association, 82, 528-550.
63. Tierney, L. (1994), “Markov Chains for Exploring Posterior Distributions” (with discussion), The Annals of Statistics, 22, 1701-1762.
64. Tierney, L. (1996), “Introductionto General State-Space Markov Chain Theory,” in Markov Chain Monte Carlo in Practice, eds. W. R. Gilks, S. Richardson, and D. J. Spiegelhalter, London: Chapman and Hall, pp. 59-74.
65. van Dyk, D. A. (2000), “Fitting Mixed-Effects Models Using Efficient EM-Type Algorithms,” Journal of Com- putationaland GraphicalStatistics, 9, 78-98.
66. van Dyk, D. A., and Meng, X. -L. (1997), “On the Orderings and Groupings of ConditionalMaximizations Within ECM-Type Algorithms,” Journal ofComputationaland Graphical Statistics, 6, 202-223