AHaH computing-from metastable switches to attractors to machine learning

PLoS ONE

Volumn 9, Issue 2, 2014, Pages

  Nugent, Michael Alexander ^a,b,c   Molter, Timothy Wesley ^a,b,c  

^a M Alexander Nugent Consulting   (United States)

^b KnowmTech LLC   (United States)

^c Xeiam LLC   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ALGORITHM; ANTI HEBBIAN AND HEBBIAN COMPUTING; ARTICLE; CLASSIFICATION; CLASSIFIER; CLUSTER ANALYSIS; CONTROLLED STUDY; ELECTRIC RESISTANCE; FEEDBACK SYSTEM; GENERAL MEMRISTIVE DEVICE MODEL; INFORMATION PROCESSING; LINEAR NEURON MODEL; LINEAR SYSTEM; MACHINE LEARNING; MATHEMATICAL COMPUTING; MATHEMATICAL MODEL; NERVE CELL PLASTICITY; PREDICTION; SIMULATION; STATISTICAL MODEL; STATISTICAL PARAMETERS; SYNAPSE; THERMODYNAMICS;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; CLUSTER ANALYSIS; COMPUTER SIMULATION; ELECTRIC CONDUCTIVITY; ELECTRONICS; MODELS, THEORETICAL; ROBOTICS; SIGNAL PROCESSING, COMPUTER-ASSISTED;

EID: 84895511946     PISSN: None     EISSN: 19326203     Source Type: Journal    
DOI: 10.1371/journal.pone.0085175     Document Type: Article

References (121)

1. Grime JP, Crick JC, Rincon JE (1985) The ecological significance of plasticity. In: Proc. 1985 Symposia of the Society for Experimental Biology. volume 40, 5-29.
2. Desbiez MO, Kergosien Y, Champagnat P, Thellier M (1984) Memorization and delayed expression of regulatory messages in plants. Planta 160: 392-399.
3. Aphalo PJ, Ballaré CL (1995) On the importance of information-acquiring systems in plant-plant interactions. Functional Ecology 9: 5-14.
4. Falik O, Reides P, Gersani M, Novoplansky A (2003) Self/non-self discrimination in roots. Journal of Ecology 91: 525-531. (Pubitemid 36933222)
5. Scialdone A, Mugford ST, Feike D, Skeffington A, Borrill P, et al. (2013) Arabidopsis plants perform arithmetic division to prevent starvation at night. eLife 2 doi: 10.7554/eLife.00669.
6. von Bodman SB, Bauer WD, Coplin DL (2003) Quorum sensing in plant-pathogenic bacteria. Annual Review of Phytopathology 41: 455-482. (Pubitemid 37417748)
7. Nakagaki T, Yamada H, Tóth Á (2000) Intelligence: Maze-solving by an amoeboid organism. Nature 407: 470-470.
8. Bonabeau E, Dorigo M, Theraulaz G (1999) Swarm Intelligence: from Natural to 1172 Artificial Systems, volume 4. Oxford University press New York.
9. Choudhary S, Sloan S, Fok S, Neckar A, Trautmann E, et al. (2012) Silicon neurons that compute. In: Artificial Neural Networks and Machine Learning - ICANN 2012, Springer Berlin Heidelberg, volume 7552 of Lecture Notes in Computer Science. 121-128.
10. Izhikevich EM (2003) Simple model of spiking neurons. IEEE Transactions on Neural Networks 14: 1569-1572.
11. Eliasmith C, Stewart TC, Choo X, Bekolay T, DeWolf T, et al. (2012) A large scale model of the functioning brain. Science 338: 1202-1205.
12. Boser BE, Guyon IM, Vapnik VN (1992) A training algorithm for optimal margin classifiers. In: Proc. 1992 ACM 5th Annual Workshop on Computational Learning Theory. 144-152.
13. MacQueen J (1967) Some methods for classification and analysis of multivariate observations. In: Proc. 1967 5th Berkeley Symposium on Mathematical Statistics and Probability. 281-297, p.14.
14. Breiman L (2001) Random forests. Machine Learning 45: 5-32.
15. Serrano-Gotarredona R, Oster M, Lichtsteiner P, Linares-Barranco A, Paz-Vicente R, et al. (2009) CAVIAR: A 45k neuron, 5M synapse, 12G connects/s AER hardware sensory-processing-learning-actuating system for high-speed visual object recognition and tracking. IEEE Transactions on Neural Networks 20: 1417-1438.
16. Sardar S, Tewari G, Babu KA (2011) A hardware/software co-design model for face recognition using cognimem neural network chip. In: Proc. 2011 IEEE International Conference on Image Information Processing (ICIIP). 1-6.
17. Arthur JV, Merolla PA, Akopyan F, Alvarez R, Cassidy A, et al. (2012) Building block of a programmable neuromorphic substrate: A digital neurosynaptic core. In: Proc. 2012 IEEE International Joint Conference on Neural Networks (IJCNN). 1-8.
18. Ananthanarayanan R, Esser SK, Simon HD, Modha DS (2009) The cat is out of the bag: cortical simulations with 109 neurons, 1013 synapses. In: Proc. 2009 IEEE Conference on High Performance Computing Networking, Storage and Analysis. 1-12.
19. Le QV, Ranzato M, Monga R, Devin M, Chen K, et al. (2011) Building high-level features using large scale unsupervised learning. preprint arXiv cs.LG/1112.6209.
20. Pakkenberg B, Pelvig D, Marner L, Bundgaard MJ, Gundersen HJG, et al. (2003) Aging and the human neocortex. Experimental Gerontology 38: 95-99. (Pubitemid 36110503)
21. Turing AM (1936) On computable numbers, with an application to the entscheidungsproblem. Proceedings of the London Mathematical Society 42: 230-265.
22. Turing A (1948) Intelligent machinery. Report, National Physical Laboratory.
23. Schrödinger E (1992) What is Life?: With Mind and Matter and Autobiographical Sketches. Cambridge University Press.
24. Hebb DO (2002) The Organization of Behavior: A Neuropsychological Theory. Psychology Press.
25. Rosenblatt F (1958) The perceptron: a probabilistic model for information storage and organization in the brain. Psychological Review 65: 386.
26. Barlow HB (1953) Summation and inhibition in the frog's retina. The Journal of Physiology 119: 69-88.
27. Hubel DH, Wiesel TN (1959) Receptive fields of single neurones in the cat's striate cortex. The Journal of Physiology 148: 574-591.
28. Widrow B (1987) The original adaptive neural net broom-balancer. In: Proc. 1987 IEEE International Symposium on Circuits and Systems. volume 2, 351-357.
29. Minsky M, Seymour P (1969) Perceptrons. MIT press.
30. Hopfield JJ (1982) Neural networks and physical systems with emergent collective computational abilities. Proceedings of the National Academy of Sciences of the United States of America 79: 2554-2558.
31. Rumelhart DE, Hinton GE, Williams RJ (1986) Learning representations by back-propagating errors. Nature 323: 533-536. (Pubitemid 16025374)
32. Grossberg S (1987) Competitive learning: from interactive activation to adaptive resonance. Cognitive Science 11: 23-63.
33. Chua L (1971) Memristor-the missing circuit element. IEEE Transactions on Circuit Theory 18: 507-519.
34. Chua LO, Kang SM (1976) Memristive devices and systems. Proceedings of the IEEE 64: 209-223.
35. Mead C, Conway L (1980) Introduction to VLSI Systems. Boston, MA, USA: Addison-Wesley Longman Publishing Co., Inc.
36. Mead C, Ismail M (1989) Analog VLSI Implementation of Neural Systems. Springer.
37. Bienenstock EL, Cooper LN, Munro PW (1982) Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex. The Journal of Neuroscience 2: 32-48. (Pubitemid 12146056)
38. Barlow HB (1989) Unsupervised learning. Neural Computation 1: 295-311.
39. Bell AJ, Sejnowski TJ (1997) The independent components of natural scenes are edge filters. Vision Research 37: 3327-3338. (Pubitemid 27493806)
40. Hyvärinen A, Oja E (1997) A fast fixed-point algorithm for independent component analysis. Neural Computation 9: 1483-1492. (Pubitemid 127462801)
41. Comon P (1994) Independent component analysis, a new concept? Signal Processing 36: 287-314.
42. Schölkopf, Simard P, Vapnik V, Smola AJ (1997) Improving the accuracy and speed of support vector machines. Advances in Neural Information Processing Systems 9: 375-381.
43. Nugent A, Kenyon G, Porter R (2004) Unsupervised adaptation to improve fault tolerance of neural network classifiers. In: Proc. 2004 IEEE NASA/DoD Conference on Evolvable Hardware. 146-149.
44. Nugent MA, Porter R, Kenyon GT (2008) Reliable computing with unreliable components: using separable environments to stabilize long-term information storage. Physica D: Nonlinear Phenomena 237: 1196-1206.
45. Nugent A (2008). Plasticity-induced self organizing nanotechnology for the extraction of independent components from a data stream. US Patent 7,409,375.
46. Nugent A (2008). Universal logic gate utilizing nanotechnology. US Patent 7,420,396.
47. Nugent A (2009). Methodology for the configuration and repair of unreliable switching elements. US Patent 7,599,895.
48. Yang JJ, Pickett MD, Li X, Ohlberg DA, Stewart DR, et al. (2008) Memristive switching mechanism for metal/oxide/metal nanodevices. Nature Nanotechnology 3: 429-433.
49. Snider GS (2008) Spike-timing-dependent learning in memristive nanodevices. In: Proc. 2008 IEEE International Symposium on Nanoscale Architectures (NANOARCH). 85-92.
50. Stewart DR, Ohlberg DAA, Beck PA, Chen Y, Williams RS, et al. (2004) Molecule-independent electrical switching in Pt/organic monolayer/Ti devices. Nano Letters 4: 133-136. (Pubitemid 38167878)
51. Kozicki MN, Gopalan C, Balakrishnan M, Mitkova M (2006) A low-power nonvolatile switching element based on copper-tungsten oxide solid electrolyte. IEEE Transactions on Nanotechnology 5: 535-544. (Pubitemid 44444396)
52. Szot K, Speier W, Bihlmayer G, Waser R (2006) Switching the electrical resistance of individual dislocations in single-crystalline SrTiO3. Nature Materials 5: 312-320.
53. xO-metal heterostructures. Applied Physics Letters 90: 042107.
54. 3/metal structures. Advanced Materials 19: 1711-1713.
55. Oblea AS, Timilsina A, Moore D, Campbell KA (2010) Silver chalcogenide based memristor devices. In: Proc. 2010 IEEE International Joint Conference on Neural Networks (IJCNN). 1-3.
56. Yang Y, Sheridan P, Lu W (2012) Complementary resistive switching in tantalum oxide-based resistive memory devices. Applied Physics Letters 100: 203112.
57. Valov I, Kozicki MN (2013) Cation-based resistance change memory. Journal of Physics D: Applied Physics 46: 074005.
58. Hasegawa T, Nayak A, Ohno T, Terabe K, Tsuruoka T, et al. (2011) Memristive operations demonstrated by gap-type atomic switches. Applied Physics A 102: 811-815.
59. Jackson BL, Rajendran B, Corrado GS, Breitwisch M, Burr GW, et al. (2013) Nanoscale electronic synapses using phase change devices. ACM Journal on Emerging Technologies in Computing Systems (JETC) 9: 12.
60. Choi S, Ambrogio S, Balatti S, Nardi F, Ielmini D (2012) Resistance drift model for conductivebridge (CB) RAM by filament surface relaxation. In: Proc. 2012 IEEE 4th International Memory Workshop (IMW). 1-4.
61. Pino RE, Bohl JW, McDonald N, Wysocki B, Rozwood P, et al. (2010) Compact method for modeling and simulation of memristor devices: ion conductor chalcogenide-based memristor devices. In: Proc. 2010 IEEE/ACM International Symposium on Nanoscale Architectures (NANOARCH). 1-4.
62. Menzel S, Bottger U, Waser R (2012) Simulation of multilevel switching in electrochemical metallization memory cells. Journal of Applied Physics 111: 014501.
63. Chang T, Jo SH, Kim KH, Sheridan P, Gaba S, et al. (2011) Synaptic behaviors and modeling of a metal oxide memristive device. Applied Physics A 102: 857-863.
64. Sheridan P, Kim KH, Gaba S, Chang T, Chen L, et al. (2011) Device and SPICE modeling of RRAM devices. Nanoscale 3: 3833-3840.
65. Biolek D, Biolek Z, Biolkova V (2009) SPICE modeling of memristive, memcapacitative and meminductive systems. In: Proc. 2009 IEEE European Conference on Circuit Theory and Design (ECCTD). 249-252.
66. Chang T, Jo SH, Lu W (2011) Short-term memory to long-term memory transition in a nanoscale memristor. ACS Nano 5: 7669-7676.
67. Merrikh-Bayat F, Shouraki SB, Afrakoti IEP (2010) Bottleneck of using single memristor as a synapse and its solution. preprint arXiv: cs.NE/1008.3450.
68. Jo SH, Chang T, Ebong I, Bhadviya BB, Mazumder P, et al. (2010) Nanoscale memristor device as synapse in neuromorphic systems. Nano Letters 10: 1297-1301.
69. Hasegawa T, Ohno T, Terabe K, Tsuruoka T, Nakayama T, et al. (2010) Learning abilities achieved by a single solid-state atomic switch. Advanced Materials 22: 1831-1834.
70. Li Y, Zhong Y, Xu L, Zhang J, Xu X, et al. (2013) Ultrafast synaptic events in a chalcogenide memristor. Scientific Reports 3 doi:10.1038/srep01619.
71. Merrikh-Bayat F, Shouraki SB (2011) Memristor-based circuits for performing basic arithmetic operations. Procedia Computer Science 3: 128-132.
72. Merrikh-Bayat F, Shouraki SB (2013) Memristive neuro-fuzzy system. IEEE Transactions on Cybernetics 43: 269-285.
73. Morabito FC, Andreou AG, Chicca E (2013) Neuromorphic engineering: from neural systems to brain-like engineered systems. Neural Networks 45: 1-3.
74. Klimo M, Such O (2011) Memristors can implement fuzzy logic. preprint arXiv cs.ET/1110.2074.
75. Klimo M, Such O (2012) Fuzzy computer architecture based on memristor circuits. In: Proc. 2012 4th International Conference on Future Computational Technologies and Applications. 84-87.
76. Kavehei O, Al-Sarawi S, Cho KR, Eshraghian K, Abbott D (2012) An analytical approach for memristive nanoarchitectures. IEEE Transactions on Nanotechnology 11: 374-385.
77. Rosezin R, Linn E, Nielen L, Kugeler C, Bruchhaus R, et al. (2011) Integrated complementary resistive switches for passive high-density nanocrossbar arrays. IEEE Electron Device Letters 32: 191-193.
78. Kim KH, Gaba S, Wheeler D, Cruz-Albrecht JM, Hussain T, et al. (2011) A functional hybrid memristor crossbar-array/CMOS system for data storage and neuromorphic applications. Nano Letters 12: 389-395.
79. Jo SH, Kim KH, Lu W (2009) High-density crossbar arrays based on a Si memristive system. Nano Letters 9: 870-874.
80. Xia Q, Robinett W, Cumbie MW, Banerjee N, Cardinali TJ, et al. (2009) Memristor-1327 tor-CMOS hybrid integrated circuits for reconfigurable logic. Nano Letters 9: 3640-3645.
81. Strukov DB, Stewart DR, Borghetti J, Li X, Pickett M, et al. (2010) Hybrid CMOS/memristor circuits. In: Proc. 2010 IEEE International Symposium on Circuits and Systems (ISCAS). 1967-1970.
82. Snider G (2011) Instar and outstar learning with memristive nanodevices. Nanotechnology 22: 015201.
83. Thomas A (2013) Memristor-based neural networks. Journal of Physics D: Applied Physics 46: 093001.
84. Indiveri G, Linares-Barranco B, Legenstein R, Deligeorgis G, Prodromakis T (2013) Integration of nanoscale memristor synapses in neuromorphic computing architectures. preprint arXiv cs.ET/1302.7007.
85. Turing AM (1952) The chemical basis of morphogenesis. Philosophical Transactions of the Royal Society of London Series B, Biological Sciences 237: 37-72.
86. Getling AV (1998) Rayleigh-Bénard Convection: Structures and Dynamics. World Scientific.
87. Athelogou M, Merté B, Deisz P, Hübler A, Lüscher E (1989) Extremal properties of dendritic patterns: biological applications. Helvetica Physica Acta 62: 250-253.
88. Swenson R (1989) Emergent attractors and the law of maximum entropy production: foundations to a theory of general evolution. Systems Research 6: 187-197.
89. Bejan A (1997) Constructal-theory network of conducting paths for cooling a heat generating volume. International Journal of Heat and Mass Transfer 40: 799-816.
90. Jorgensen SE, Svirezhev YM (2004) Towards a Thermodynamic Theory for Ecological Systems. Elsevier.
91. Schneider ED, Sagan D (2005) Into the Cool: Energy Flow, Thermodynamics, and Life. University of Chicago Press.
92. Lotka AJ (1922) Contribution to the energetics of evolution. Proceedings of the National Academy of Sciences of the United States of America 8: 147-151.
93. Hatsopoulos GN, Keenan JH (1981) Principles of General Thermodynamics. RE Krieger Publishing Company.
94. Shang DS, Shi L, Sun JR, Shen BG, Zhuge F, et al. (2010) Improvement of reproducible resistance switching in polycrystalline tungsten oxide films by in situ oxygen annealing. Applied Physics Letters 96: 072103.
95. Kvatinsky S, Friedman EG, Kolodny A, Weiser UC (2013) TEAM: ThrEshold Adaptive Memristor model. IEEE Transactions on Circuits and Systems I: Regular Papers 60: 211-221.
96. Strukov DB, Williams RS (2009) Exponential ionic drift: fast switching and low volatility of thin-film memristors. Applied Physics A 94: 515-519.
97. Pickett MD, Strukov DB, Borghetti JL, Yang JJ, Snider GS, et al. (2009) Switching dynamics in titanium dioxide memristive devices. Journal of Applied Physics 106: 074508.
98. Williams RS, Pickett MD, Strachan JP (2013) Physics-based memristor models. In: 1365 Proc. 2013 IEEE International Symposium on Circuits and Systems (ISCAS). 217-220.
99. Zhang JJ, Sun HJ, Li Y, Wang Q, Xu XH, et al. (2013) AgInSbTe memristor with gradual resistance tuning. Applied Physics Letters 102: 183513.
100. Mladenov VM, Kirilov SM (2012) Analysis of a serial circuit with two memristors and voltage source at sine and impulse regime. In: Proc. 2012 IEEE 13th International Workshop on Cellular Nanoscale Networks and Their Applications (CNNA). 1-6.
101. Jain AK, Murty MN, Flynn PJ (1999) Data clustering: a review. ACM Computing Surveys (CSUR) 31: 264-323.
102. Lloyd S (1982) Least squares quantization in PCM. IEEE Transactions on Information Theory 28: 129-137.
103. Kriegel HP, Kröger P, Sander J, Zimek A (2011) Density-based clustering. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery 1: 231-240.
104. Ankerst M, Breunig MM, Kriegel HP, Sander J (1999) OPTICS: ordering points to identify the clustering structure. ACM SIGMOD Record 28: 49-60. (Pubitemid 129597323)
105. Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society: Series B 39: 1-38.
106. Kotsiantis SB (2007) Supervised Machine Learning: a Review of Classification Techniques. IOS Press. 3-24 pp.
107. Yu HF, Hsieh CJ, Chang KW, Lin CJ (2012) Large linear classification when data cannot fit in memory. ACM Transactions on Knowledge Discovery from Data (TKDD) 5: 23.
108. Ndong J, Salamatian K (2011) Signal processing-based anomaly detection techniques: a comparative analysis. In: Proc. 2011 3rd International Conference on Evolving Internet. 32-39.
109. Zhang G, Patuwo BE, Hu MY (1998) Forecasting with artificial neural networks: The state of the art. International Journal of Forecasting 14: 35-62.
110. Hoeting JA, Madigan D, Raftery AE, Volinsky CT (1999) Bayesian model averaging: a tutorial. Statistical Science: 382-401.
111. Cireşan DC, Meier U, Gambardella LM, Schmidhuber J (2012) Deep big multilayer perceptrons for digit recognition. In: Neural Networks: Tricks of the Trade, Springer Berlin Heidelberg, volume 7700 of Lecture Notes in Computer Science. 581-598.
112. Chaaban I, Scheessele MR (2007) Human performance on the USPS database. Report, Indiana University South Bend.
113. Kuzum D, Jeyasingh RGD, Lee B, Wong HSP (2011) Nanoelectronic programmable synapses based on phase change materials for brain-inspired computing. Nano Letters 12: 2179-2186.
114. Bache K, Lichman M (2013). UCI machine learning repository. Available: http://archive.ics.uci.edu/ml.
115. Chen HL, Yang B, Liu J, Liu DY (2011) A support vector machine classifier with rough set-based feature selection for breast cancer diagnosis. Expert Systems with Applications 38: 9014-9022.
116. Kohavi R (1996) Scaling up the accuracy of naïve-Bayes classifiers: a decision-tree hybrid. In: Proc. 1996 2nd International Conference on Knowledge Discovery and Data Mining. 202-207.
117. Deng L, Yu D (2011) Deep convex net: A scalable architecture for speech pattern classification. In: Proc. 2011 Interspeech. 2285-2288.
118. Joachims T (1998) Text Categorization with Support Vector Machines: Learning with Many Relevant Features. Springer.
119. Bennett KP, Blue JA (1998) A support vector machine approach to decision trees. In: Proc. 1998 IEEE International Joint Conference on Neural Networks Proceedings. The 1998 IEEE World Congress on Computational Intelligence. volume 3, 2396-2401.
120. Ranzato M, Huang FJ, Boureau YL, Lecun Y (2007) Unsupervised learning of invariant feature hierarchies with applications to object recognition. In: Proc. 2007 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 1-8.
121. Quinlan JR (1996) Improved use of continuous attributes in C4.5. preprint arXiv cs.AI/9603103.