A Review of Modern Computational Algorithms for Bayesian Optimal Design

International Statistical Review

Volumn 84, Issue 1, 2016, Pages 128-154

  Ryan, Elizabeth G ^a,b   Drovandi, Christopher C ^a   Mcgree, James M ^a   Pettitt, Anthony N ^a  

^a QUEENSLAND UNIVERSITY OF TECHNOLOGY   (Australia)

^b KING'S COLLEGE LONDON   (United Kingdom)

Author keywords

Bayesian optimal design; Decision theory; Posterior distribution approximation; Stochastic optimisation; Utility function

Indexed keywords

EID: 84930907631     PISSN: 03067734     EISSN: 17515823     Source Type: Journal    
DOI: 10.1111/insr.12107     Document Type: Article

References (113)

1. Amzal B., Bois F., Parent E. & Robert C.P. (2006). Bayesian-optimal design via interacting particle systems. J. Amer. Statist. Assoc., 101(474), 773-785.
2. Atkinson A.C. (1996). The usefulness of optimum experimental designs. J. R. Stat. Soc. Ser. B., 58, 59-76.
3. Atkinson A.C. (2008). DT-optimum designs for model discrimination and parameter estimation. J. Statist. Plann. Inference, 138, 56-64.
4. Atkinson A.C. & Donev A.N. (1992). Optimum Experimental Designs. New York: Oxford University Press.
5. Beaumont M.A., Zhang W., Balding D.J. (2002). Approximate Bayesian computation in population genetics. Genet., 162(4), 2025-2035.
6. Bengtsson T., Bickel P. & Li B. (2008). Curse-of-dimensionality 2008. revisited: Collapse of the particle filter in very large scale systems. Probability and Statistics: Essays in Honor of David a. Freedman, Vol. 2. pp. 316-334. Beechwood, Ohio, USA: Institute of Mathematical Statistics.
7. Berger J.O. (1985). Statistical Decision Theory and Bayesian Analysis 2ndedn. New York:Springer-Verlag.
8. Bernardo J.M. (1979). Expected information as expected utility. Ann. Statist., 7(3), 686-690.
9. Bernardo J.M., Smith A.F.M. (2000). Bayesian Theory 2ndedn. John Wiley & Sons 2000, Nocedal and Wright 2006, Ross 1983 and Sisson and Fan 2011. Chichester, New York:.
10. Berry D., Müller P., Grieve A., Smith M., Parke T., Blazek R., Mitchard N. & Krams M. (2000). Case Studies in Bayesian Statistics. New York: Springer.
11. Berry D.A., Ho C.-H. (1988). One-sided sequential stopping boundaries for clinical trials: A decision-theoretic approach. Biometrics, 44, 219-227.
12. Bielza C., Müller P., Insua D.R. (1999). Decision analysis by augmented probability simulation. Manag. Sci., 45(7), 995-1007.
13. Bornkamp B., Bretz F., Dette H., Pinheiro J. (2011). Response-adaptive dose-finding under model uncertainty. Ann. Appl. Stat., 5(2B), 1611-1631.
14. Borth D.M. (1975). A total entropy criterion for the dual problem of model discrimination and parameter estimation. J. R. Stat. Soc. Ser. B. Stat. Methodol., 37, 77-87.
15. Box G.E.P., Hill W.J. (1967). Discrimination among mechanistic models. Technometrics, 9, 57-71.
16. Brockwell A.E., Kadane J.B. (2003). A gridding method for Bayesian sequential decision problems. J. Comput. Graph. Statist., 12(3), 566-584.
17. Carlin B., Kadane J., Gelfand A. (1998). Approaches for optimal sequential decision analysis in clinical trials. Biometrics, 54(3), 964-975.
18. Cavagnaro D.R., Myung J.I., Pitt M.A., Kujala J.V. (2010). Adaptive design optimization: A mutual information-based approach to model discrimination in cognitive science. Neural Comput., 22(4), 887-905.
19. Chaloner K. (1984). Optimal Bayesian experimental designs for linear models. Ann. Statist., 12, 283-300.
20. Chaloner K., Verdinelli I. (1995). Bayesian experimental design: A review. Stat. Sci., 10, 273-304.
21. Chopin N. (2002). A sequential particle filter method for static models. Biometrika, 89(3), 539-552.
22. Christen J., Müller P., Wathen K., Wolf J. (2004). Bayesian randomized clinical trials: A decision-theoretic sequential design. Canad. J. Statist., 32(4), 387-402.
23. Clyde M. (2001). Experimental design: A Bayesian perspective. IESBS, 8, 5075-5081.
24. Clyde M., Chaloner K. (1996). The equivalence of constrained and weighted designs in multiple objective design problems. J. Amer. Statist. Assoc., 91, 1236-1244.
25. Clyde M.A., Müller P. & Parmigiani G. (1996). Exploring 1996, DasGupta 1995, Kim et al. 2013, Kuck et al. 2006, McGree et al. 2012a and McGree et al. 2014. expected utility surfaces by Markov chains. Technical Report. Durham, North Carolina:, Duke University.
26. Cook A., Gibson G., Gilligan C. (2008). Optimal observation times in experimental epidemic processes. Biometrics, 64(3), 860-868.
27. D'Argenio D. (1990). Incorporating prior parameter uncertainty in the design of sampling schedules for pharmacokinetic parameter estimation experiments. Math. Biosci., 99(1), 105-118.
28. DasGupta A. (1995). Review of optimal Bayes designs. Technical Report. West Lafayette, Indiana:, Purdue University.
29. DasGupta A., Mukhopadhyay S., Studden W. (1992). Compromise designs in heteroscedastic linear models. J. Statist. Plann. Inference, 32, 363-384.
30. DeGroot M.H. (1970). Optimal Statistical Decisions. New York: McGrawHill.
31. DelMoral P., Doucet A., Jasra A. (2006). Sequential Monte Carlo samplers. J. R. Stat. Soc. Ser. B. Stat. Methodol., 68(3), 411-436.
32. Dette H. (1990). A generalization of D-and D1-optimal designs in polynomial regression. The Ann. Statist., 18, 1784-1804.
33. Diggle P., Lophaven S. (2006). Bayesian geostatistical design. Scand. J. Stat., 33(1), 53-64.
34. Drovandi C., McGree J., Pettitt A. (2014). A sequential Monte Carlo algorithm to incorporate model uncertainty in Bayesian sequential design. J. Comput. Graph. Statist., 23(1), 3-24.
35. Drovandi C.C., Pettitt A.N. (2011). Estimation of parameters for macroparasite population evolution using approximate Bayesian computation. Biometrics, 67(1), 225-233.
36. Drovandi C.C., Pettitt A.N. (2013). Bayesian experimental design for models with intractable likelihoods. Biometrics, 69(4), 937-948.
37. Drovandi C.C., Pettitt A.N., Faddy M.J. (2011). Approximate Bayesian computation using indirect inference. J. R. Stat. Soc. Ser. C. Appl. Stat., 60(3), 503-524.
38. Drovandi C.C., McGree J.M., Pettitt A.N. (2014). Sequential Monte Carlo for Bayesian sequential design. Comput. Stat. Data Anal., 57(1), 320-335.
39. El-Krunz S., Studden W. (1991). Bayesian optimal designs for linear regression models. Ann. Statist., 19, 2183-2208.
40. Fedorov V.V. (1972). Theory of Optimal Experiments. New York: Academic Press.
41. Fedorov V.V., Hackl P. (1997). Model-Oriented Design of Experiments. Berlin: Springer-Verlag.
42. Geweke J. (1989). Bayesian inference in econometric models using Monte Carlo integration. Econom., 57(6), 1317-1339.
43. Hainy M., Müller W., Wagner H. (2013). Likelihood-free simulation-based optimal design. Technical Report, Johannes Kepler University, Linz.
44. Han C., Chaloner K. (2004). Bayesian experimental design for nonlinear mixed-effects models with application to HIV dynamics. Biometrics, 60, 25-33.
45. Hill W., Hunter W., Wichern D. (1968). A joint design criterion for the dual problem of model discrimination and parameter estimation. Technometrics, 10(1), 145-160.
46. Hill W.J. (1978). A review of experimental design procedures for regression model discrimination. Technometerics, 20, 15-21.
47. Ho Y.C., Cao X. (1983). Perturbation analysis and optimization of queueing networks. J. Optim. Theory Appl., 40, 559-582.
48. Huan X., Marzouk Y.M. (2013). Simulation-based optimal Bayesian experimental design for nonlinear systems. J. Comput. Phys., 232(1), 288-317.
49. Huan X., Marzouk Y.M. (2014). Gradient-based stochastic optimization methods in Bayesian experimental design. Int. J. Uncertain. Quantif., 4(6), 479-510.
50. Kim W., Pitt M.A., Lu Z.-L., Steyvers M., Myung J.I. (2013). A hierarchical adaptive approach to optimal experimental design. Technical Report. Columbus, Ohio: Ohio State University.
51. Kinas P. (1996). Bayesian fishery stock assessment and decision making using adaptive importance sampling. Can. J. Fish. Aquat.Sci., 53, 414-423.
52. Kück H., deFreitas N., Doucet A. (2006). SMC samplers for Bayesian optimal nonlinear design. Technical Report. Vancouver: University of British Columbia.
53. Kujala J.V., Lukka T.J. (2006). Bayesian adaptive estimation: The next dimension. J. Math. Psych., 50(4), 369-389.
54. Kullback S., Leibler R.A. (1951). On information and sufficiency. Ann. Math. Stat., 22(1), 79-86.
55. Kuo L., Soyer R. & Wang F. (1999). Bayesian Statistics VI. New York: Oxford University Press.
56. Lewi J., Butera R., Paninski L. (2009). Sequential optimal design of neurophysiology experiments. Neural Comput., 21, 619-687.
57. Lewis R., Berry D.A. (1994). Group sequential clinical trials: A classical evaluation of Bayesian decision-theoretic designs. J. Amer. Statist. Assoc., 89, 1528-1534.
58. Liepe J., Filippi S., Komorowski M., Stumpf M.P.H. (2013). Maximising 2013 is not a one-page article please supply the first and last pages for this article. the information content of experiments in systems biology. PLoS Comput. Biol., 1, e1002888. Online ahead of print at DOI doi:10.1371/journal.pcbi.1002888.
59. Lindley D. (1956). On a measure of the information provided by an experiment. Ann. Math. Stat., 27, 986-1005.
60. Lindley D. (1968). The choice of variables in multiple regression. J. R. Stat. Soc. Ser. B., 30, 31-53.
61. Lindley D.V. (1972). Bayesian Statistics-A Review. Philadelphia: SIAM.
62. Liu G., Rosenberger W.F., Haines L.M. (2009). Sequential designs for ordinal phase I clinical trials. Biom. J., 51(2), 335-347.
63. Loeppky J., Moore L., Williams B. (2010). Batch sequential designs for computer experiments. J. Statist. Plann. Inference, 140, 1452-1464.
64. Long Q., Scavino M, Tempone R., Wang S. (2013). Fast estimation of expected information gains for Bayesian experimental designs based on Laplace approximations. Comput. Methods Appl. Mech. Engrg., 259, 24-39.
65. Loredo T. (2004). Bayesian adaptive exploration. Bayesian Inference and Maximum Entropy Methods in Science and Engineering: 23rd International Workshop on Bayesian Inference and Maximum Entropy Methods in Science and Engineering, pp. 330-346. Jackson Hole, Wyoming, USA.
66. Maryak J., Chin D. (2004). Global random optimization by simultaneous perturbation stochastic approximation. Johns Hopkins APL Tech. Dig., 25(2), 91-100.
67. Masoumi S., Duever T.A., Reilly P.M. (2013). Sequential Markov chain Monte Carlo (MCMC) model discrimination. The Can. J. Chem. Eng., 91(5), 862-869.
68. McGree J., Drovandi C.C., Pettitt A.N. (2012a). A sequential Monte Carlo approach to the sequential design for discriminating between rival continuous data models. Technical Report. Brisbane, Queensland University of Technology.
69. McGree J., Drovandi C.C., Thompson H., Eccleston J., Duffull S., Mengersen K., Pettitt A.N., Goggin T. (2012b). Adaptive Bayesian compound designs for dose finding studies. J. Statist. Plann. Inference, 142(6), 1480-1492.
70. McGree J., Drovandi C. & White G., Pettitt A. (2014). A sequential Monte Carlo algorithm for random effects models in Bayesian sequential design. Technical Report. Brisbane, Queensland University of Technology.
71. Müller P. (1999). Simulation-based optimal design. Bayesian Stat., 6, 459-474.
72. Müller P., Parmigiani G. (1995). Optimal design via curve fitting of Monte Carlo experiments. J. Amer. Statist. Assoc., 90(432), 1322-1330.
73. Müller P., Sansó B., DeIorio M. (2004). Optimal Bayesian design by inhomogeneous Markov chain simulation. J. Amer. Statist. Assoc., 99(467), 788-798.
74. Müller P., Berry D.A., Grieve A.P., Krams M. (2006). A Bayesian decision-theoretic dose-finding trial. Decis. Anal., 3(4), 197-207.
75. Nelder J.A., Mead R. (1965). A simplex method for function minimization. The Comput. J., 7(4), 308-313.
76. Ng S.H., Chick S.E. (2004). Design of follow-up experiments for improving model discrimination and parameter estimation. Naval Res. Logist., 51, 1129-1148.
77. Nocedal J., Wright S.J. (2006). Numerical Optimization 2ndedn. New York: Springer.
78. Ormerod J., Wand M. (2010). Explaining variational approximations. J. Amer. Statist. Assoc., 64(2), 140-153.
79. Palmer J., Müller P.(1988) Bayesian optimal design in population models for haematologic data. Stat. Med., 17, 1613-1622.
80. Paninski L. (2005). Asymptotic theory of information-theoretic experimental design. Neural Comput., 17, 1480-1507.
81. Pennanen T., Koivu M. (2006). An adaptive importance sampling technique. In Monte Carlo and Quasi-Monte Carlo Methods 2004, Eds. H. Niederreiter & D. Talay, pp. 443-455. Heidelberg: Springer Berlin.
82. Pilz J. (1991). Bayesian Estimation and Experimental Design in Linear Regression Models (2nd ed). New York: Wiley.
83. Pronzato L., Walter E. (1985). Robust experimental design via stochastic approximation. Math. Biosci., 75, 103-120.
84. Pukelsheim F., Torsney B. (1991). Optimal weights for experimental designs on linearly independent support points. The Ann. Statist., 19(3), 1614-1625.
85. RiosInsua D., Ruggeri F. (2000). Robust Bayesian Analysis. New York: Springer Verlag.
86. Robbins H., Monro S. (1951). A stochastic approximation method. Ann. Math. Stat., 22(3), 400-407.
87. Ross S. (1983). Introduction to Stochastic Dynamic Programming. Orlando, Florida: Academic Press.
88. Rossell D., Müller P. (2013). Sequential stopping for high-throughput experiments. Biostat., 14(1), 75-86.
89. Rossell D., Müller P., Rosner G.L. (2007). Screening designs for drug development. Biostat., 8(3), 595-608.
90. Roth P. (1965). Design of experiments for discrimination among rival models, Ph.D. Thesis, Princeton University, New Jersey, USA.
91. Rue H., Martino S., Chopin N. (2009). Approximate Bayesian inference for latent Gaussian models using integrated nested Laplace approximations (with discussion). J. R. Stat. Soc. Ser. B., 71(2), 319-392.
92. Ryan E.G., Drovandi C.C., Thompson M., Pettitt A.N. (2014). Towards Bayesian experimental design for nonlinear models that require a large number of sampling times. Comput. Stat. Data Anal., 70, 45-60.
93. Ryan E.G., Drovandi C.C., Pettitt A.N. (2015a). Fully Bayesian experimental design for pharmacokinetic studies. Entropy, 17, 1063-1089.
94. Ryan E.G., Drovandi C.C., Pettitt A.N. (2015b). Simulation-based fully Bayesian experimental design for mixed effects models. Comput. Stat. Data Anal. (In press).
95. Ryan K. (2003). Estimating expected information gains for experimental designs with application to the random fatigue-limit model. J. Comput. Graph. Statist., 12, 585-603.
96. Shannon C. (1948). A mathematical theory of communication. Bell Syst. Tech. J., 27, 379-423, 623-656.
97. Sisson S.A., Fan Y. (2011). Handbook of Markov Chain Monte Carlo., 313-335, Boca Raton, Florida: Chapman & Hall.
98. Solonen A., Haario H., Laine M. (2012). Simulation-based optimal design using a response variance criterion. J. Comput. Graph. Statist., 21(1), 234-252.
99. Spall J.C. (1998). An overview of the simultaneous perturbation method for efficient optimization. Johns Hopkins APL Tech. Dig., 19(4), 482-492.
100. Spiegelhalter D.J., Freedman L.S., Parmar M.K.B. (1996). Bayesian approaches to randomize trials. In Bayesian Biostatistics, D.A. Berry, D.K. Stangl, pp. 67-108. New York: Dekker.
101. Stroud J., Müller P., Rosner G. (2001). Optimal sampling times in population pharmacokinetic studies. J. R. Stat. Soc. Ser. C. Appl. Stat., 50(3), 345-359.
102. Tian Y., Wang D. (2009). Sequential Bayesian design for estimation of EDp. Tianjin: The 2nd International Conference on Biomedical Engineering and Informatics, 2009. BMEI'09, pp. 883-885.
103. Tommasi C. (2009). Optimal designs for both model discrimination and parameter estimation. J. Statist. Plann. Inference, 139, 4123-4132.
104. Tommasi C., López-Fidalgo J. (2010). Bayesian optimum designs for discriminiating between models with any distribution. Comput. Stat. Data Anal., 54, 143-150.
105. VanLaarhoven P., Aarts E. (1987). Simulated Annealing: Theory and Applications. Dordrecht: Reider.
106. Vanlier J., Tiemann C., Hilbers P., van Riel N. (2014). Optimal experimental design for model selection in biochemical networks. BMC Syst. Biol., 8, 20.
107. Verdinelli I. (1992). Bayesian Statistics 4, 467-481, Advances in Bayesian Experimental Design (with Discussion), vol.4, Oxford: Oxford University Press.
108. Wakefield J. (1994). An expected loss approach to the design of dosage regimens via sampling-based methods. J. R. Stat. Soc. Ser. D (The Statistician), 43(1), 13-29.
109. Waterhouse T.H., Eccleston J.A., Duffull S.B. (2009). Optimal design criteria for discrimination and estimation in nonlinear models. J. Biopharm. Stat., 19, 386-402.
110. Whitehead J., Brunier H. (1995). Bayesian decision procedures for dose determining experiments. Stat. Med., 14, 885-893.
111. Whitehead J., Williamson D. (1998). Bayesian decision procedures based on logisic regression models for dose-finding studies. J. Biopharm. Stat., 8, 445-467.
112. Wolfson L., Kadane J., Small M. (1996). Expected utility as a policy making tool: An environmental health example. In Bayesian Biostatistics, Eds. D. Berry, D. Stangl, Dekker pp. 261-277. New York.
113. Zidek J., Sun W., Le N. (2000). Designing and integrating composite networks for monitoring multivariate Gaussian pollution fields. Appl. Stat., 49, 63-79.