Self-avoiding random dynamics on integer complex systems

ACM Transactions on Modeling and Computer Simulation

Volumn 23, Issue 1, 2013, Pages

  Hamze, Firas ^a   Wang, Ziyu ^b   De Freitas, Nando ^b  

^a D WAVE SYSTEMS INC   (Canada)

^b UNIVERSITY OF BRITISH COLUMBIA   (Canada)

Author keywords

Adaptive MCMC; Algorithm configuration; Bayesian optimization; Boltzmann machines; Gibbs sampling; Ising models; Markov chain Monte Carlo; Monte Carlo; SARDONICS; Swendswen Wang

Indexed keywords

ADAPTIVE MCMC; ALGORITHM CONFIGURATIONS; BAYESIAN OPTIMIZATION; BOLTZMANN MACHINES; GIBBS SAMPLING; MARKOV CHAIN MONTE-CARLO; MONTE CARLO; SARDONICS; SWENDSWEN-WANG;

ACOUSTIC SURFACE WAVE DEVICES; DYNAMICS; ISING MODEL; MONTE CARLO METHODS; OPTIMIZATION; QUANTUM COMPUTERS;

ALGORITHMS;

EID: 84873695090     PISSN: 10493301     EISSN: 15581195     Source Type: Journal    
DOI: 10.1145/2414416.2414790     Document Type: Article

References (66)

1. Ackley, D. H., Hinton, G., and Sejnowski, T. 1985. a learning algorithm for Boltzmann machines. Cog. Sci. 9, 147-169.
2. Andrieu, C. and Robert, C. 2001. Controlled MCMC for optimal sampling. Tech. rep. 0125, Cahiers de Mathematiques du Ceremade, Universite Paris-Dauphine.
3. Barahona, F. 1982. On the computational complexity of Ising spin glass models. J. Phys. A:Math. Gen. 15, 10, 3241.
4. Berg, B. A. and Neuhaus, T. 1991. Multicanonical algorithms for first order phase transitions. Phys. Lett. B 267, 2, 249-253.
5. Bertsekas, D. P. 1982. Projected Newton methods for optimization problems with simple constraints. SIAM J. Contr. Optimiz. 20, 2, 221-246.
6. Besag, J. 1974. Spatial interaction and the statistical analysis of lattice systems. J. Roy. Statist. Soc., Ser. B 36, 192-236.
7. Bian, Z., Chudak, F., Macready, W. G., and Rose, G. 2011. the Ising model: Teaching an old problem new tricks. Tech. rep., D-Wave Systems.
8. Blei, D. M., Ng, A. Y., and Jordan, M. I. 2003. Latent Dirichlet allocation. J. Mach. Learn. Res. 3, 993-1022.
9. Botev, Z. I. and Kroese, D. P. 2012. Efficient Monte Carlo simulation via the generalized splitting method. Stat. Comput. 22, 1, 1-16.
10. Brochu, E., Cora, V. M., and de Freitas, N. 2009. a tutorial on Bayesian optimization of expensive cost functions. Tech. rep. TR-2009-023, University of British Columbia, Department of Computer Science.
11. Bull, A. D. 2011. Convergence rates of efficient global optimization algorithms. Tech. rep., arXiv:1101.3501v2.
12. Cérou, F., Moral, P., Furon, T., and Guyader, A. 2012. Sequential Monte Carlo for rare event estimation. Stat. Comput. 22, 3, 795-808.
13. Diggle, P. J., Tawn, J. A., and Moyeed, R. A. 1998. Model-based geostatistics. J. Roy. Stat. Soc. Ser. C (Applied Statistics) 47, 3, 299-350.
14. Duane, S., Kennedy, A. D., Pendleton, B. J., and Roweth, D. 1987. Hybrid Monte Carlo. Phys. Lett. B 195, 2, 216-222.
15. Earl, D. J. and Deem, M. W. 2005. Parallel tempering: Theory, applications, and new perspectives. Phys. Chem. Chem. Phys. 7, 3910-3916.
16. Finkel, D. E. 2003. DIRECT Optimization Algorithm User Guide. Center for Research in Scientific Computation, North Carolina State University.
17. Freitas, N. D., Wang, Y., Mahdaviani, M., and Lang, D. 2005. Fast Krylov methods for N-body learning. in Advances in Neural Information Processing Systems. MIT Press, 251-258.
18. Geyer, C. J. 1991. Markov Chain Monte Carlo maximum likelihood. in Computer Science and Statistics: Proceedings of the 23rd Symposium on the Interface. 156-163.
19. Girolami, M. and Calderhead, B. 2011. Riemann manifold Langevin and HamiltonianMonte Carlo methods. J. Roy. Stat. Soc. Ser. B (Statistical Methodology) 73, 2, 123-214.
20. Glover, F. 1989. Tabu search - Part I. ORSA J. Comput. 1, 190-206.
21. Gramacy, R. B., Lee, H. K. H., and MacReady, W. 2004. Parameter space exploration with gaussian process trees. in Proceedings of the International Conference on Machine Learning. Omnipress and ACM Digital Library, 353-360.
22. Gubernatis, J. and Hatano, N. 2000. the multicanonical monte carlo method. Comput. Sci. Eng. 2, 2, 95-102.
23. Haario, H., Saksman, E., and Tamminen, J. 2001. An adaptive Metropolis algorithm. Bernoulli 7, 2, 223- 242.
24. Halko, N., Martinsson, P. G., and Tropp, J. A. 2011. Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions. Science 53, 2, 217-288.
25. Hamze, F. and de Freitas, N. 2004. From fields to trees. in Uncertainty in Artificial Intelligence. 243-250.
26. Hamze, F. and de Freitas, N. 2005. Hot coupling: a particle approach to inference and normalization on pairwise undirected graphs. Adv. Neural Inf. Proc. Syst. 18, 491-498.
27. Hinton, G. and Salakhutdinov, R. 2006. Reducing the dimensionality of data with neural networks. Science 313, 5786, 504-507.
28. Hoffman, M., Brochu, E., and de Freitas, N. 2011. Portfolio allocation for Bayesian optimization. in Uncertainty in Artificial Intelligence. 327-336.
29. Hoos, H. H. and Stutzle, T. 2004. Stochastic Local Search: Foundations and Applications. Elsevier, Morgan Kaufmann.
30. Hopfield, J. J. 1984. Neurons with graded response have collective computational properties like those of two-state neurons. Proc. Nat. Acad. Sci. 81, 10, 3088-3092.
31. Hyvarinen, A., Hurri, J., and Hoyer, P. 2009. Natural Image Statistics. Springer.
32. Kindermann, R. and Snell, J. L. 1980. Markov Random Fields and Their Applications. American Mathematical Society
33. Kück, H. and de Freitas, N. 2005. Learning about individuals from group statistics. in Uncertainty in Artificial Intelligence. 332-339.
34. Lee, H., Grosse, R., Ranganath, R., and Ng, A. Y. 2009. Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. in International Conference on Machine Learning. 609-616.
35. Liu, J. S. 2001. Monte Carlo strategies in scientific computing. Springer.
36. Liu, J. S., Zhang, J. L., Palumbo, M. J., and Lawrence, C. E. 2003. Bayesian clustering with variable and transformation selections. Bayesian Stat. 7, 249-275.
37. Lizotte, D. 2008. Practical Bayesian optimization. Ph.D. thesis, University of Alberta, Edmonton, Alberta, Canada.
38. Mahendran, N., Wang, Z., Hamze, F., and de Freitas, N. 2011. Bayesian optimization for adaptive MCMC. Tech. rep., arXiv:1110.6497v1.
39. Marinari, E., Parisi, G., and Ruiz-Lorenzo, J. 1997. Numerical simulations of spin glass systems. Spin Glasses and Random Fields, 59-98.
40. Marlin, B., Swersky, K., Chen, B., and de Freitas, N. 2010. Inductive principles for restricted Boltzmann machine learning. in Artificial Intelligence and Statistics. 509-516.
41. May, B. C., Korda, N., Lee, A., and Leslie, D. S. 2012. Optimistic Bayesian sampling in contextual bandit problems. J. Mach. Learn. Res. 98888, 2069-2106.
42. Memisevic, R. and Hinton, G. 2009. Learning to represent spatial transformations with factored higherorder Boltzmann machines. Neural Computation 22, 1473-1492.
43. Mockus, J. 1982. the Bayesian approach to global optimization. in System Modeling and Optimization. Vol. 38. Springer Berlin /Heidelberg, 473-481.
44. Munoz, J. D., Novotny, M. A., and Mitchell, S. J. 2003. Rejection-free Monte Carlo algorithms for models with continuous degrees of freedom. Phys. Rev. E 67.
45. Neal, R. M. 2010. MCMC using Hamiltonian dynamics. Handbook of Markov Chain Monte Carlo 54, 113- 162.
46. Newman, M. and Barkema, G. 1999. Monte Carlo Methods in Statistical Physics. Oxford University Press.
47. Osborne, M., Garnett, R., and Roberts, S. 2010. Active data selection for sensor networks with faults and changepoints. in Proceedings of the IEEE International Conference on Advanced Information Networking and Applications.
48. Ranzato, M., Mnih, V., and Hinton, G. 2010. How to generate realistic images using gated MRF's. in Advances in Neural Information Processing Systems. 2002-2010.
49. Rasmussen, C. E. and Williams, C. K. I. 2006. Gaussian Processes for Machine Learning. MIT Press, Cambridge, Massa.
50. Robert, C. P. and Casella, G. 2004. Monte Carlo Statistical Methods 2nd Ed. Springer.
51. Roberts, G. O. and Rosenthal, J. S. 2009. Examples of adaptive MCMC. J. Computat. Graph. Stat. 18, 2, 349-367.
52. Rosenbluth,M.N. and Rosenbluth, A.W. 1955. Monte Carlo calculation of the average extension of molecular chains. J. Chem. Phys. 23, 356-359.
53. Rue, H., Martino, S., and Chopin, N. 2009. Approximate Bayesian inference for latent Gaussian models by using integrated nested Laplace approximations. Journal of the Royal Statistical Society Series B 71, 2, 319-392.
54. Salakhutdinov, R. andMurray, I. 2008. On the quantitative analysis of Deep Belief Networks. in Proceedings of the International Conference on Machine Learning. 872-879.
55. Santner, T. J., Williams, B., and Notz, W. 2003. the Design and Analysis of Computer Experiments. Springer.
56. Schonlau, M., Welch, W. J., and Jones, D. R. 1998. Global versus local search in constrained optimization of computer models. Lecture Notes-Monograph Series 34, 11-25.
57. Siepmann, J. I. and Frenkel, D. 1992. Configurational bias Monte Carlo: a new sampling scheme for flexible chains. Molec. Phys.: An Int. J. Interface Between Chem. Phys. 75, 1, 59-70.
58. Smolensky, P. 1986. Information processing in dynamical systems: Foundations of harmony theory. in Parallel Distributed Processing: Explorations in theMicrostructure of Cognition, vol. 1: Foundations, 194-281.
59. Srinivas, N., Krause, A., Kakade, S. M., and Seeger, M. 2010. Gaussian process optimization in the bandit setting: No regret and experimental design. in Proceedings of the International Conference on Machine Learning.
60. Swendsen, R. H. and Wang, J.-S. 1987. Nonuniversal critical dynamics in Monte Carlo simulations. Phys. Rev. Lett. 58, 2, 86-88.
61. Swersky, K., Chen, B.,Marlin, B., and de Freitas, N. 2010. a tutorial on stochastic approximation algorithms for training restricted Boltzmann machines and deep belief nets. in Proceedings of the Information Theory and Applications Workshop. 1-10.
62. Tham, S. S., Doucet, A., and Kotagiri, R. 2002. Sparse Bayesian learning for regression and classification using Markov chain Monte Carlo. in Proceedings of the International Conference on Machine Learning. 634-641.
63. Vihola,M. 2010. Grapham: Graphical models with adaptive random walk Metropolis algorithms. Computat. Stat. Data Anal. 54, 1, 49-54.
64. Wang, F. and Landau, D. P. 2001. Efficient, multiple-range random walk algorithm to calculate the density of states. Phys. Rev. Lett. 86, 2050-2053.
65. Welsh, D. J. A. 1993. Complexity: Knots, Colourings and Counting. Cambridge University Press.
66. Ye, K. Q. 1998. Orthogonal column Latin hypercubes and their application in computer experiments. J. ASA 93, 444, 1430-1439.