Multiprocess parallel antithetic coupling for backward and forward Markov chain Monte Carlo

Annals of Statistics

Volumn 33, Issue 2, 2005, Pages 661-697

  Craiu, Radu V ^a   Meng, Xiao Li ^b,c  

^a UNIVERSITY OF TORONTO   (Canada)

^b HARVARD UNIVERSITY   (United States)

^c HARVARD UNIVERSITY   (United States)

Author keywords

Antithetic variates; Exact sampling; Extreme antithesis; Latin hypercube sampling; Negative association; Negative dependence; Parallel implementation; Perfect simulation; Quasi Monte Carlo; Stratification; Swindles

Indexed keywords

EID: 22544460444     PISSN: 00905364     EISSN: None     Source Type: Journal    
DOI: 10.1214/009053604000001075     Document Type: Article

References (68)

1. ALBERT, J. and CHIB, S. (1993). Bayesian analysis of binary and polychotomous response data. J. Amer. Statist. Assoc. 88 669-679.
2. ARJAS, E. and GASBARRA, D. (1996). Bayesian inference of survival probabilities, under stochastic ordering constraints. J. Amer. Statist. Assoc. 91 1101-1109.
3. ARVIDSEN, N. I. and JOHNSSON, T. (1982). Variance reduction through negative correlation - a simulation study. J. Statist. Comput. Simulation 15 119-127.
4. BAILEY, B. J. R. (1981). Alternatives to Hastings' approximation to the inverse of the normal cumulative distribution function. Appl. Statist. 30 275-276.
5. BILLINGSLEY, P. (1986). Probability and Measure, 2nd ed. Wiley, New York.
6. BONDESSON, L. (1983). Problem 128. Statist. Neerlandica 37 149.
7. BOREL, E. (1924). Traité du Calcul des Probabilités et de ses Applications. Gauthier-Villars, Paris.
8. CASELLA, G., LAVINE, M. and ROBERT, C. P. (2001). Explaining the perfect sampler. Amer. Statist. 55 299-305.
9. CASELLA, G., MENGERSEN, K. L., ROBERT, C. P. and TITTERINGTON, D. M. (2002). Perfect samplers for mixtures of distributions. J. R. Stat. Soc. Ser. B Stat. Methodol. 64 777-790.
10. CRAIU, R. V. (2001). Multivalent framework for approximate and exact sampling and resampling. Ph.D. dissertation, Univ. Chicago.
11. CRAIU, R. V. (2004). Antithetic acceleration of the multiple-try Metropolis. Technical Report 0407, Dept. Statistics, Univ. Toronto.
12. CRAIU, R. V. and MENG, X.-L. (2001). Chance and fractals. Chance 14(2) 47-52.
13. CRAIU, R. V. and MENG, X.-L. (2002). Multi-process parallel antithetic coupling for backward and forward MCMC. Technical Report 0205, Dept. Statistics, Univ. Toronto.
14. DAMIEN, P., WAKEFIELD, J. and WALKER, S. (1999). Gibbs sampling for Bayesian non-conjugate and hierarchical models by using auxiliary variables. J. R. Stat. Soc. Ser. B Stat. Methodol. 61331-344.
15. ESARY, J. D., PROSCHAN, F. and WALKUP, D. W. (1967). Association of random variables, with applications. Ann. Math. Statist. 38 1466-1474.
16. FILL, J. A. (1998). An interruptible algorithm for perfect sampling via Markov chains. Ann. Appl. Probab. 8131-162.
17. FILL, J. A., MACHIDA, M., MURDOCH, D. J. and ROSENTHAL, J. S. (2000). Extension of Fill's perfect rejection sampling algorithm to general chains. Random Structures Algorithms 17 229-316.
18. FISHMAN, G. S. (1972). Variance reduction in simulation studies. J. Stat. Comput. Simul. 1 173-182.
19. Foss, S. G. and TWEEDIE, R. L. (1998). Perfect simulation and backward coupling. Comm. Statist. Stochastic Models 14 187-203.
20. FRÉCHET, M. (1951). Sur les tableaux de corrélation dont les marges sont données. Ann. Univ. Lyon. Sect. A(3) 14 53-77.
21. FRIGESSI, A., GÅSEMYR, J. and RUE, H. (2000). Antithetic coupling of two Gibbs sampler chains. Ann. Statist. 281128-1149.
22. GELFAND, A. E., SMITH, A. F. M. and LEE, T.-M. (1992). Bayesian analysis of constrained parameter and truncated data problems using Gibbs sampling. J. Amer. Statist. Assoc. 87 523-532.
23. GELMAN, A. and RUBIN, D. B. (1992). Inference from iterative simulation using multiple sequences (with discussion). Statist. Sci. 7 457-511.
24. GENTLE, J. (1998). Random Number Generation and Monte Carlo Methods. Springer, New York.
25. GEROW, K. and HOLBROOK, J. (1996). Statistical sampling and fractal distributions. Math. Intelligencer 18(2) 12-22.
26. HÄGGSTRÖM, O. and NELANDER, K. (1998). Exact sampling from anti-monotone systems. Statist. Neerlandica 52 360-380.
27. HAMMERSLEY, J. M. and HANDSCOMB, D. C. (1965). Monte Carlo Methods. Methuen, London.
28. HAMMERSLEY, J. M. and MORTON, K. V. (1956). A new Monte Carlo technique: Antithetic variates. Proc. Cambridge Philos. Soc. 52 449-475.
29. HELTON, J. C. and DAVIS, F. J. (2003). Latin hypercube sampling and the propagation of uncertainty in analyses of complex systems. Reliability Engineering and System Safety 81 23-69.
30. HOBERT, J. P., ROBERT, C. P. and TITTERINGTON, D. M. (1999). On perfect simulation for some mixtures of distributions. Statist. Comput. 9 287-298.
31. HOEFFDING, W. (1940). Masstabinvariante Korrelationtheorie. Schriften des Mathematischen Instituts und des Instituts für Angewandte Mathematik der Universitat Berlin 5 181-233.
32. IMAN, R. L. (1999). Latin hypercube sampling. Encyclopedia of Statistical Sciences Update 3 408-411. Wiley, New York.
33. JOAG-DEV, K. and PROSCHAN, F. (1983). Negative association of random variables, with applications. Ann. Statist. 11 286-295.
34. JOE, H. (1997). Multivariate Models and Dependence Concepts. Chapman and Hall, London.
35. KENDALL, W. S. (1998). Perfect simulation for the area-interaction point process. Probability Towards 2000. Lecture Notes in Statist. 128 218-234. Springer, New York.
36. KENNEDY, W. J. and GENTLE, J. E. (1980). Statistical Computing. Dekker, New York.
37. KISH, L. (1965). Survey Sampling. Wiley, New York.
38. LANCASTER, H. O. (1957). Some properties of the bivariate normal distribution considered in the form of a contingency table. Biometrika 44 289-292.
39. LEHMANN, E. L. (1966). Some concepts of dependence. Ann. Math. Statist. 37 1137-1153.
40. LEW, R. A. (1981). An approximation to the cumulative normal distribution with simple coefficients. Appl. Statist. 30 299-301.
41. LOH, W.-L. (1996). On Latin hypercube sampling. Ann. Statist. 24 2058-2080.
42. MCKAY, M. D., BECKMAN, R. J. and CONOVER, W. J. (1979). A comparison of three methods for selecting values of input variables in the analysis of output from a computer code. Technometrics 21 239-245.
43. MENG, X.-L. (2000). Towards a more general Propp-Wilson algorithm: Multistage backward coupling. In Monte Carlo Methods (N. Madras, ed.) 85-93. Amer. Math. Soc., Providence, RI.
44. MEYN, S. P. and TWEEDIE, R. L. (1993). Markov Chains and Stochastic Stability. Springer, London.
45. MØLLER, J. (1999). Perfect simulation of conditionally specified models. J. R. Stat. Soc. Ser. B Stat. Methodol. 61 251-264.
46. MØLLER, J. and NICHOLLS, G. K. (1999). Perfect simulation for sample-based inference. Preprint, Dept. Mathematical Statistics, Chalmers Institute of Technology.
47. MORAN, P. A. P. (1967). Testing for correlation between non-negative variates. Biometrika 54 385-394.
48. MURDOCH, D. J. (2000). Exact sampling for Bayesian inference: Unbounded state spaces. In Monte Carlo Methods (N. Madras, ed.) 111-121. Amer. Math. Soc., Providence, RI.
49. MURDOCH, D. J. and GREEN, P. J. (1998). Exact sampling from a continuous state space. Scand. J. Statist. 25 483-502.
50. MURDOCH, D. J. and MENG, X.-L. (2001). Towards perfect sampling for Bayesian mixture priors. In Bayesian Methods, with Applications to Science, Policy and Official Statistics (E. George, ed.) 381-390. Eurostat.
51. NEAL, R. M. (2003). Slice sampling (with discussion). Ann. Statist. 31 705-767.
52. OWEN, A. B. (1992). A central limit theorem for Latin hypercube sampling. J. Roy. Statist. Soc. Ser. B 54 541-551.
53. OWEN, A. B. (1997). Monte Carlo variance of scrambled net quadrature. SIAM J. Numer. Anal. 34 1884-1910.
54. OWEN, A. B. (2002). Quasi Monte Carlo. Presented at the First Cape Cod Workshop on Monte Carlo Methods.
55. PAGE, E. S. (1965). On Monte Carlo methods in congestion problems. II. Simulation of queuing systems. Oper. Res. 13 300-305.
56. PEMANTLE, R. (2000). Towards a theory of negative dependence. J. Math. Phys. 41 1371-1390.
57. PHILIPPE, A. and ROBERT, C. (2003). Perfect simulation of positive Gaussian distributions. Statist. Comput. 13179-186.
58. PROPP, J. G. and WILSON, D. B. (1996). Exact sampling with coupled Markov chains and applications to statistical mechanics. Random Structures Algorithms 9 223-252.
59. RUBINSTEIN, R. Y. and SAMORODNITSKY, G. (1985). Variance reduction by the use of common and antithetic random variables. J. Statist. Comput. Simulation 22 161-180.
60. RÜSCHENDORF, L. and UCKELMANN, L. (2000). Variance minimization and random variables with constant sum. Technical report, Institüt für Mathematische Stochastik, Univ. Freiburg.
61. SIMON, G. (1976). Computer simulation swindles, with applications to estimates of location and dispersion. Appl. Statist. 25 266-274.
62. SNIJDERS, T. A. B. (1984). Antithetic variates for Monte Carlo estimation of probabilities. Statist. Neerlandica 38 55-73.
63. STEIN, M. (1987). Large sample properties of simulations using Latin hypercube sampling. Technometrics 29 143-151. [Corrigendum Technometrics 32 (1990) 367.]
64. THORISSON, H. (2000). Coupling, Stationarity, and Regeneration. Springer, New York.
65. VAN DYK, D. and MENG, X.-L. (2001). The art of data augmentation (with discussion). J. Comput. Graph. Statist. 10 1-111.
66. WILSON, D. B. (1998). Annotated bibliography of perfectly random sampling with Markov chains. In Microsurveys in Discrete Probability (D. Aldous and J. Propp, eds.) 209-220. Amer. Math. Soc., Providence, RI. Updated versions appear at http://dimacs.rutgers.edu/~dbwilson/exact.
67. WILSON, D. B. (2000a). How to couple from the past using a read-once source of randomness. Random Structures Algorithms 16 85-113.
68. WILSON, D. B. (2000b). Layered multishift coupling for use in perfect sampling algorithms (with a primer on CFTP). In Monte Carlo Methods (N. Madras, ed.) 143-176. Amer. Math. Soc., Providence, RI.