Real-parameter evolutionary monte carlo with applications to bayesian mixture models

Journal of the American Statistical Association

Volumn 96, Issue 454, 2001, Pages 653-666

  Liang, Faming ^a   Wong, Wing Hung ^b  

^a NATIONAL UNIVERSITY OF SINGAPORE   (Singapore)

^b HARVARD SCHOOL OF PUBLIC HEALTH   (United States)

Author keywords

Crossover; Evolutionary Monte Carlo; Exchange; Genetic algorithm; Markov chain Monte Carlo; Metropolis algorithm; Mixture model; Mutation; Neural network; Parallel tempering

Indexed keywords

EID: 1542573405     PISSN: 01621459     EISSN: 1537274X     Source Type: Journal    
DOI: 10.1198/016214501753168325     Document Type: Article

References (44)

1. Berg, B. A., and Neuhaus, T. (1991), “Multicanonical Algorithms for First Order Phase Transitions,” Physics Letters, Ser. B, 267, 249–253.
2. Box, G. E. P., and Jenkins, G. M. (1970), Time Series Analysis, Forecast and Control, San Francisco: Holden-Day.
3. Buntine, W. L., and Weigend, A. S. (1991), “Bayesian Back-Propagation,” Complex Systems 5, 603–643.
4. Carlin, B., and Chib, S. (1993), “Bayesian Model Choice via Markov Chain Monte Carlo,” Journal of the Royal Statistical Society, Ser. B, 57, 473–484.
5. Chen, M. H., Shao, Q. M., and Ibrahim, J. G. (2000), Monte Carlo Methods in Bayesian Computation, New York: Springer-Verlag.
6. Chib, S. (1995), “Marginal Likelihood From the Gibbs Output,” Journal of American Statistical Association, 90, 1313–1321.
7. Cybenko, G. (1989), “Approximations by Superpositions of a Sigmoidal Function,” Mathematics of Control, Signals and Systems, 2, 303–314.
8. Ding, X., Canu, S., and Denoeux, T. (1996), “Neural Network Based Models for Forecasting,” in Neural Networks and Their Applications, ed. J. G. Taylor, New York: Wiley, pp. 153–167.
9. Eshelman, L. J., and Schaffer, J. D. (1993), “Real-Coded Genetic Algorithms and Interval-Schematai,” in Foundation of Genetic Algorithms 2, ed. G. J. E. Rawlins, San Mateo, CA: Morgan Kaufmann, pp. 187–202.
10. Gelman, A., Roberts, R. O., and Gilks, W. R. (1996), “Efficient Metropolis Jumping Rules,” in Bayesian Statistics 5, eds. J. M. Bernardo, J. O. Berger, A. P. Dawid, and A. F. M. Smith, New York: Oxford University Press.
11. Geman, S., and Geman, D. (1984), “Stochastic Relaxation, Gibbs Distributions and the Bayesian Restoration of Images,” IEEE Transactions on Pattern Analysis and Machine Intelligence, 6, 721–741.
12. Geyer, C. J. (1991), “Markov Chain Monte Carlo Maximum Likelihood,” in Computing Science and Statistics: Proceedings of the 23rd Symposium on the Interface ed. E. M. Keramigas, Fairfax, VA: Interface Foundation, pp. 156–163.
13. Geyer, C. J., and Thompson, E. A. (1995), “Annealing Markov Chain Monte Carlo With Applications to Pedigree Analysis,” Journal of American Statistical Association, 90, 909–920.
14. Gilks, W. R., Roberts, G. O., and George, E. I. (1994), “Adaptive Direction Sampling,” The Statistician, 43, 179–189.
15. Goldberg, D. E. (1989), Genetic Algorithms in Search, Optimization, & Machine learning, Reading, MA: Addison-Wesley.
16. Green, P. J. (1995), “Reversible Jump Markov Chain Monte Carlo Computation and Bayesian Model Determination,” Biometrika, 82, 711–732.
17. Hastings, W. K. (1970), “Monte Carlo Sampling Methods Using Markov Chain and Their Applications,” Biometrika, 57, 97–109.
18. Hill, T., O’Connor, M., and Remus, W. (1996), “Neural Network Models for Time Series Forecasts,” Management Science, 42, 1082–1092.
19. Holland, J. H. (1975), Adaptation in Natural and Artificial Systems, Ann Arbor, MI: University of Michigan Press.
20. Hornik, K., Stinchcombe, M., and White, H. (1989), “Multilayer Feedforward Networks are Universal Approximators,” Neural Networks, 2, 359–366.
21. Hukushima, K., and Nemoto, K. (1996), “Exchange Monte Carlo Method and Application to Spin Glass Simulations,” Journal of the Physics Society of Japan, 65, 1604–1608.
22. Kang, S. (1991), “An Investigation of the Use of Feedforward Neural Networks for Forecasting,” doctoral dissertation, Kent State University.
23. Liang, F., and Wong, W. H. (1999), “Dynamic Weighting in Simulations of Spin Systems,” Physics Letters, Ser. A, 252, 257–262.
24. Liang, F., and Wong, W. H. (2000), “Evolutionary Monte Carlo Sampling: Applications to Cp Model Sampling and Change-point Problem,” Statistica Sinica, 10, 317–342.
25. Liu, J. S., Liang F., and Wong, W. H. (2000), “The Use of Multiple-Try Method and Local Optimization in Metropolis Sampling,” Journal of American Statistical Association, 94, 121–134.
26. MacKay, D. J. C. (1992), “A Practical Bayesian Framework for Backprop Networks,” Neural Computation, 4, 448–472.
27. Marinari, E., and Parisi, G. (1992), “Simulated Tempering: A New Monte Carlo Scheme,” Europhysics Letters, 19, 451–458.
28. Mehrotra, K., Mohan, C. K., and Ranka, S. (1996), Elements of Artificial Neural Networks, Cambridge, MA: MIT Press.
29. Meng, X., and Wong, W. H. (1996), “Simulating Ratios of Normalizing Constants via a Simple Identity: A Theoretical Exploration,” Statistica Sinica, 6, 831–860.
30. Metropolis, N., Rosenbluth, A. W., Rosenbluth, M. N., Teller, A. H., and Teller, E. (1953), “Equation of State Calculations by Fast Computing Machines,” Journal of Chemical Physics, 21, 1087–1091.
31. Müller, P., and Insua, D. R. (1998), “Issues in Bayesian Analysis of Neural Network Models,” Neural Computation, 10, 749–770.
32. Neal, R. M. (1993), “Bayesian Learning via Stochastic Dynamics,” in Advances in Neural Information Processing Systems 5, eds. C. L. Giles, S. J. Hansn, and J. D. Cowan, San Francisco: Morgan Kaufmann.
33. Neal, R. M. (1996), Bayesian Learning for Neural Networks, New York: Springer-Verlag.
34. Neal, R. M. (1999), “Erroneous Results in ‘Marginal Likelihood from the Gibbs Output’,” http://www.cs.utoronto.ca/raford/.
35. Ono, I., and Kobayashi, S. (1997), “A Real-Coded Genetic Algorithm for Function Optimization Using Unimodal Normal Distribution Crossover,” in Proceedings of the Seventh International Conference on Genetic Algorithms, ed. T. Bäck, San Francisco: Morgan Kaufmann, pp. 246–253.
36. Phillips, D. B., and Smith, A. F. M. (1995), “Bayesian Model Comparison via Jump Diffusions,” in Markov Chain Monte Carlo in Practice, eds. W. R. Gilks, S. Richardson, and D. J. Spiegelhalter, London: Chapman and Hall, pp. 215–239.
37. Postman, M., Huchra, J. P., and Geller, M. J. (1986), “Probes of Large-Scale Structures in the Corona Borealis Region,” The Astronomical Journal, 92, 1238–1247.
38. Ritter, C., and Tanner, M. A. (1992), “Facilitating the Gibbs Sampler: The Gibbs Stopper and the Griddy-Gibbs Sampler,” Journal of the American Statistical Association, 87, 861–868.
39. Roberts, G. O., and Gilks, W. R. (1994), “Convergence of Adaptive Direction Sampling,” Journal of Multivariate Analysis, 49, 287–298.
40. Roeder, K. (1990), “Density Estimation With Confidence Sets Exemplified by Superclusters and Voids in Galaxies,” Journal of the American Statistical Association, 85, 617–624.
41. Rumelhart, D., Hinton, G., and McClelland, J. (1986), “Learning Internal Representations by Error Propagation,” in Parallel Distributed Processing, eds. D. Rumelhart and J. McClelland, Cambridge, MA: MIT Press, pp. 45–76.
42. White, H. (1992), Artificial Neural Networks: Approximation and Learning Theory, Cambridge, MA: Blackwell.
43. Wong, W. H., and Liang, F. (1997), “Dynamic Weighting in Monte Carlo and Optimization,” Proceedings of the National Academy of Sciences, 94, 14220–14224.
44. Wright, A. H. (1991), “Genetic Algorithms for Real Parameter Optimization,” in Foundations of Genetic Algorithms 1, ed. G. J. E. Rawlins, San Mateo, CA: Morgan Kaufmann, pp. 205–218.