Chaotic Boltzmann machines

Scientific Reports

Volumn 3, Issue , 2013, Pages

  Suzuki, Hideyuki ^a   Imura, Jun Ichi ^b   Horio, Yoshihiko ^c   Aihara, Kazuyuki ^a  

^a INSTITUTE OF INDUSTRIAL SCIENCE   (Japan)

^b TOKYO INSTITUTE OF TECHNOLOGY   (Japan)

^c TOKYO DENKI UNIVERSITY   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; ARTIFICIAL NEURAL NETWORK; COMPUTER ANALYSIS; COMPUTER SIMULATION; FEEDBACK SYSTEM; NONLINEAR SYSTEM; STATISTICAL MODEL;

COMPUTER SIMULATION; COMPUTING METHODOLOGIES; FEEDBACK; MODELS, STATISTICAL; NEURAL NETWORKS (COMPUTER); NONLINEAR DYNAMICS;

EID: 84885580432     PISSN: None     EISSN: 20452322     Source Type: Journal    
DOI: 10.1038/srep01610     Document Type: Article

References (53)

1. Korst, J. H. M. & Aarts, E. H. L. Combinatorial optimization on a Boltzmann machine. J. Parallel Distribut. Comp. 6, 331-357 (1989).
2. Geman, S. & Geman, D. Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Trans. Pattern Anal. Mach. Intell. 6, 721-741 (1984).
3. Ackley, D. H., Hinton, G. E. & Sejnowski, T. J. A learning algorithm for Boltzmann machines. Cognitive Sci. 9, 147-169 (1985).
4. Peterson, C. & Anderson, J. R. A mean field theory learning algorithm for neural networks. Complex Syst. 1, 995-1019 (1987).
5. Hinton, G. E. Deterministic Boltzmann learning performs steepest descent in weight-space. Neural Comput. 1, 143-150 (1989).
6. Smolensky, P. Information processing in dynamical systems: foundations of harmony theory. in Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Vol. 1 (ed. Rumelhart, D. E. et al.) Chap. 6, 194-281 (MIT Press, Cambridge, 1986).
7. Salakhutdinov, R. & Hinton, G. Deep Boltzmann machines. Proceedings of the 12th International Conference on Artificial Intelligence and Statistics 448-455 (2009).
8. Hinton, G. E. Training products of experts by minimizing contrastive divergence. Neural Comput. 14, 1771-1800 (2002).
9. Skubiszewski, M. An exact hardware implementation of the Boltzmann machine. Proceedings of the Fourth IEEE Symposium on Parallel and Distributed Processing 107-110 (1992).
10. Ly, D. L. & Chow, P. High-performance reconfigurable hardware architecture for restricted Boltzmann machines. IEEE Trans. Neural Netw. 21, 1780-1792 (2010).
11. Akazawa, M. & Amemiya, Y. Boltzmann machine neuron circuit using single-electron tunneling. Appl. Phys. Lett. 70, 670-672 (1997).
12. Wu, N.-J., Shibata, N. & Amemiya, Y. Boltzmann machine neuron device using quantum-coupled single electrons. Appl. Phys. Lett. 72, 3214-3216 (1998).
13. Yamada, T., Akazawa, M., Asai, T. & Amemiya, Y. Boltzmann machine neural network devices using single-electron tunnelling. Nanotech. 12, 60-67 (2001).
14. Blank, M. & Bunimovich, L. Switched flow systems: pseudo billiard dynamics. Dyn. Syst. 19, 359-370 (2004).
15. Branicky, M. S. Analog computation with continuous ODEs. Proceedings of IEEE Workshop on Physics and Computation 265-274 (1994).
16. Siegelmann, H. T. Computation beyond the Turing limit. Science 268, 545-548 (1995).
17. Moore, C. Recursion theory on the reals and continuous-time computation. Theor. Comp. Sci. 162, 23-44 (1996).
18. Blum, L., Cucker, F., Shub, M. & Smale, S. Complexity and real computation (Springer, New York, 1997).
19. Aihara, K. Chaos engineering and its application to parallel distributed processing with chaotic neural networks. Proc. IEEE 90, 919-930 (2002).
20. Horio, Y. & Aihara, K. Analog computation through high-dimensional physical chaotic neuro-dynamics. Physica D 237, 1215-1225 (2008).
21. Rendl, F., Rinaldi, G. & Wiegele, A. Solving Max-Cut to optimality by intersecting semidefinite and polyhedral relaxations. Math. Program. 121, 307-335 (2010).
22. Hasegawa, M., Ikeguchi, T. & Aihara, K. Combination of chaotic neuro-dynamics with the 2-opt algorithm to solve traveling salesman problems. Phys. Rev. Lett. 79, 2344-2347 (1997).
23. Chen, L. & Aihara, K. Global searching ability of chaotic neural networks. IEEE Trans. Circuits Syst. I 46, 974-993 (1999).
24. Elser, V., Rankenburg, L. & Thibault, P. Searching with iterated maps. Proc. Natl. Acad. Sci. USA 104, 4655-4660 (2007).
25. Ercsey-Ravasz, M. & Toroczkai, Z. Optimization hardness as transient chaos in an analog approach to constraint satisfaction. Nature Phys. 7, 966-970 (2011).
26. Ercsey-Ravasz, M. & Toroczkai, Z. The chaos within Sudoku. Sci. Rep. 2, 725 (2012).
27. Schweighofer, N., Doya, K., Fukai, H., Chiron, J. V., Furukawa, T. & Kawato, M. Chaos may enhance information transmission in the inferior olive. Proc. Natl. Acad. Sci. USA 101, 4655-4660 (2004).
28. Takemoto, T., Kohno, T. & Aihara, K. Circuit Implementation and dynamics of a two-dimensional MOSFET neuron model. Int. J. Bif. Chaos 17, 459-508 (2007).
29. Miritello, G., Pluchino, A. & Rapisarda, A. Phase transitions and chaos in long-range models of coupled oscillators. Europhys. Lett. 85, 10007 (2009).
30. Creutz, M. Microcanonical Monte Carlo simulation. Phys. Rev. Lett. 50, 1411-1414 (1983).
31. Vichniac, G. Y. Simulating physics with cellular automata. Physica 10D, 96-116 (1984).
32. Creutz, M. Deterministic Ising dynamics. Ann. Phys. 167, 62-72 (1986).
33. Kaneko, K. Pattern dynamics in spatiotemporal chaos. Physica D 34, 1-41 (1989).
34. Sakaguchi, H. Phase transitions in coupled Bernoulli maps. Prog. Theor. Phys. 80, 7-12 (1988).
35. Miller, J. & Huse, D. A. Macroscopic equilibrium from microscopic irreversibility in a chaotic coupled-map lattice. Phys. Rev. E 48, 2528-2535 (1993).
36. Egolf, D. A. Equilibrium regained: From nonequilibrium chaos to statistical mechanics. Science 287, 101-104 (2000).
37. Just, W. Phase transitions in coupled map lattices and in associated probabilistic cellular automata. Phys. Rev. E 74, 046209 (2006).
38. Chase, C., Serrano, J. & Ramadge, P. J. Periodicity and chaos from switched flow systems: contrasting examples of discretely controlled continuous systems. IEEE Trans. Automat. Contr. 38, 70-83 (1993).
39. Jin'no, K., Nakamura, T. & Saito, T. Analysis of bifurcation phenomena in a 3-cells hysteresis neural network. IEEE Trans. Circuits Syst. I 46, 851-857 (1999).
40. Skarda, C. A. & Freeman, W. J. How brains make chaos in order to make sense of the world. Behav. Brain Sci. 10, 161-195 (1987).
41. Aihara, K., Takabe, T. & Toyoda, M. Chaotic neural networks. Phys. Lett. A 144, 333-340 (1990).
42. Elbert, T., Ray, W. J., Kowalik, Z. J., Skinner, J. E., Graf, K. E. & Birbaumer, N. Chaos and physiology: deterministic chaos in excitable cell assemblies. Physiol. Rev. 74, 1-47 (1994).
43. Tsuda, I. Toward an interpretation of dynamic neural activity in terms of chaotic dynamical systems. Behav. Brain. Sci. 24, 793-810 (2001).
44. Faure, P. & Korn, H. Is there chaos in the brain? I. Concepts of nonlinear dynamics and methods of investigation. C. R. Acad. Sci. III 324, 773-793 (2001).
45. Korn, H. & Faure, P. Is there chaos in the brain? II. Experimental evidence and related models. C. R. Biol. 326, 787-840 (2003).
46. Toyoizumi, T. & Abbott, L. F. Beyond the edge of chaos: Amplification and temporal integration by recurrent networks in the chaotic regime. Phys. Rev. E 84, 051908 (2011).
47. Adachi, M. & Aihara, K. Associative dynamics in a chaotic neural network. Neural Netw. 10, 83-98 (1997).
48. Welling, M. Herding dynamic weights to learn. Proceedings of the 26th Annual International Conference on Machine Learning 1121-1128 (ACM Press, New York, 2009).
49. Aihara, K. & Suzuki, H. Theory of hybrid dynamical systems and its applications to biological and medical systems. Phil. Trans. R. Soc. A 368, 4893-4914 (2010).
50. Suzuki, H., Imura, J. & Aihara, K. Chaotic Ising-like dynamics in traffic signals. Sci. Rep. 3, 1127 (2013).
51. Landauer, R. Irreversibility and Heat Generation in the Computing Process. IBM J. Res. Dev. 5, 183-191 (1961).
52. Bennett, C. H. Logical reversibility of computation. IBM J. Res. Dev. 17, 525-532 (1973).
53. Bennett, C. H. The thermodynamics of computation-a review. Int. J. Theor. Phys. 21, 905-940 (1982).