STDP and sTDP variations with memristors for spiking neuromorphic learning systems

Frontiers in Neuroscience

Volumn , Issue 7 FEB, 2013, Pages

  Serrano Gotarredona, T ^a   Masquelier, T ^b,c   Prodromakis, T ^d   Indiveri, G ^e   Linares Barranco, B ^a  

^a INSTITUTO DE MICROELECTRÓNICA DE SEVILLA   (Spain)

^b UNIVERSITAT POMPEU FABRA   (Spain)

^c Groupe Enzymologie Moléculaire et Fonctionnelle   (France)

^d IMPERIAL COLLEGE LONDON   (United Kingdom)

^e UNIVERSITY OF ZURICH   (Switzerland)

Author keywords

Artificial learning synapses; Memristor cmos; Spike timing dependent plasticity; Spiking neural networks

Indexed keywords

ARTICLE; CORTICAL SYNCHRONIZATION; DESTRUCTION; LEARNING; MEMRISTOR; NERVE CELL MEMBRANE POTENTIAL; NERVE CELL PLASTICITY; NEUROFILAMENT; PHYSICS; POSTSYNAPTIC MEMBRANE; PRESYNAPTIC NERVE; SPIKE TIMING DEPENDENT PLASTICITY; SPIKE WAVE; SYNAPSE;

EID: 84878952572     PISSN: 16624548     EISSN: 1662453X     Source Type: Journal    
DOI: 10.3389/fnins.2013.00002     Document Type: Article

References (100)

1. Argall, F. (1968). Switching phenomena in titanium oxide thin films. Solid-State Electron. 11, 535-541.
2. Avizienis, A. V., Sillin, H. O., Martin-Olmos, C., Shieh, H. H., Aono, M., Stieg, A. Z., et al. (2012). Neuromorphic atomic switch networks. PLoS ONE 7:e427772. doi: 10.1371/journal.pone.0042772
3. Barbaro, M., Burgi, P. Y., Mortara, A., Nussbaum, P., and Heitger, F. (2002). A 100×100 pixel silicon retina for gradient extraction with steering filter capabilities and temporal output coding. IEEE J. Solid State Circ. 37, 160-172.
4. Bi, G., and Poo, M. (1998). Synaptic modifications in cultured hippocampal neurons: dependence on spike timing, synaptic strength, and postsynaptic cell type. J. Neurosci. 18, 10464-10472.
5. Bi, G., and Poo, M. M. (2001). Synaptic modification by correlated activity: Hebb's postulate revisited. Ann. Rev. Neurosci. 24, 139-166.
6. Bichler, O., Suri, M., Querlioz, D., Vuillaume, D., DeSalvo, B., and Gamrat, C. (2012a). Visual pattern extraction using energy-efficient "2-PCM Synapse" neuromorphic architecture. IEEE Trans. Electron Devices 59, 2206-2214.
7. Bichler, O., Querlioz, D., Thorpe, S. J., Bourgoin, J. P., and Gamrat, C. (2012b). Extraction of temporally correlated features from dynamic vision sensors withspike-timing-dependent plasticity. Neural Netw. 32, 339-348.
8. Borghetti, J., Li, Z., Straznicky, J., Li, X., Ohlberg, D. A. A., Wu, W., et al. (2009). A hybrid nanomemristor/transistor logic circuit capable of self-programming. PNAS 106, 1699-1703.
9. Brader, J. M., Senn, W., and Fusi, S. (2007). Learning real-world stimuli in a neural network with spike-driven synaptic dynamics. Neural Comput. 19, 2881-2912.
10. Camuñas-Mesa, L., Acosta-Jimènez, A., Zamarreño-Ramos, C., Serrano-Gotarredona, T., and Linares-Barranco, B. (2011). A 32× 32 convolution processor chip for address event vision sensors with 155ns event latency and 20Meps throughput. IEEE Trans. Circ. Syst. I 58, 777-790.
11. Camuñas-Mesa, L., Zamarreño-Ramos, C., Linares-Barranco, A., Acosta-Jimenez, A. J., Serrano-Gotarredona, T., and Linares-Barranco, B. (2012). An event-driven multi-kernel convolution processor module for event-driven vision sensors. IEEE J. Solid State Circ. 47, 504-517.
12. Cassenaer, S., and Laurent, G. (2007). Hebbian STDP in mushroom bodies facilitates the synchronous flow of olfactory information in locusts. Nature 448, 709-713.
13. Chan, V., Jin, C., and van Schaik, A. (2007a). An address-event vision sensor for multiple transient object detection. IEEE Trans. Biomed. Circ. Syst. 1, 278-288.
14. Chan, V., Liu, S.-C., and van Schaik, A. (2007b). AER EAR: a matched silicon cochlea pair with address event representation interface. IEEE Trans. Circ. Syst. I 54, 48-59.
15. Chanthbouala, A., Garcia, V., Cherifi, R. O., Bouzehouane, K., Fusil, S., Moya, X., et al. (2012). A ferroelectric memristor. Nat. Mater. 11, 860-864.
16. Chen, D. G., Matolin, D., Bermak, A., and Posch, C. (2011). Pulse-modulation imaging - review and performance analysis. IEEE Trans. Biomed. Circ. Syst. 5, 64-82.
17. Chicca, E., Whatley, A. M., Lichtsteiner, P., Dante, V., Delbruck, T., Del Giudice, P., et al. (2007). A multichip pulse-based neuromorphic infrastructure and its application to a model of orientation selectivity. IEEE Trans. Circ. Syst. I 54, 981-993.
18. Choi, T. Y. W., Merolla, P., Arthur, J., Boahen, K., and Shi, B. E. (2005). Neuromorphic implementation of orientation hypercolumns. IEEE Trans. Circ. Syst. I 52, 1049-1060.
19. Chua, L. (2011). Resistance switching memories are memristors. Appl. Phys. A 102, 765-783.
20. Chua, L. O. (1971). Memristor - the missing circuit element. IEEE Trans. Circ. Theory 18, 507-519.
21. Chua, L. O., Desoer, C. A., and Kuh, E. S. (1987). Linear and Nonlinear Circuits. New York, NY: McGraw-Hill.
22. Chua, L. O., and Kang, S. M. (1976). Memristive devices and systems. Proc. IEEE 64, 209-223.
23. Costas-Santos, J., Serrano-Gotarredona, T., Serrano-Gotarredona, R., and Linares-Barranco, B. (2007). A spatial contrast retina with on-chip calibration for neuromorphic spike-based AER vision systems. IEEE Trans. Circ. Syst. I 54, 1444-1458.
24. Culurciello, E., Etienne-Cummings, R., and Boahen, K. A. (2003). A biomorphic digital image sensor. IEEE J. Solid State Circ. 38, 281-294.
25. Davies, S., Galluppi, F., Rast, A., and Furber, S. (2012). A forecast-based STDP rule suitable for neuromorphic implementation. Neural Netw. 32, 3-14.
26. Feldman, D. (2000). Timing-based LTP and LTD at vertical inputs to layer II/III pyramidal cells in rat barrel cortex. Neuron 27, 45-56.
27. Fieres, J., Schemmel, J., and Meier, K. (2008). "Realizing biological spiking network models in a configurable wafer-scale hardware system," IEEE International Joint Conference on Neural Networks (IJCNN-WCCI) (Hong Kong), 969-976.
28. Finelli, L. A., Haney, S., Bazhenov, M., stopfer, M., and Sejnowski, T. J. (2008). Synaptic learning rules and sparse coding in a model sensory system. PLoS Comput. Biol. 4:e1000062. doi: 10.1371/journal.pcbi.1000062
29. Fu, Z., Delbrück, T., Lichtsteiner, P., and Culurciello, E. (2008). An address-event fall detector for assisted living applications. IEEE Trans. Biomed. Circ. Syst. 2, 88-96.
30. Fursina, A. A., Sofin, R. G. S., Shvets, I. V., and Natelson, D. (2009). Origin of hysteresis in resistive switching in magnetite is Joule heating. Phys. Rev. B 79:245131. doi: 10.1103/PhysRevB.79.245131
31. Gerstner, W., Kempter, R., Leo van Hemmen, J., and Wagner, H. (1996). A neuronal learning rule for sub-millisecond temporal coding. Nature 383, 76-78.
32. Gerstner, W., Ritz, R., and Hemmen, J. L. (1993). Why spikes? Hebbian learning and retrieval of time-resolved excitation patterns. Biol. Cybern. 69, 503-515.
33. 2 Hf/HfOx crossbar resistive RAM with excellent performance, reliability and low-energy operation," in Electron Devices Meeting (IEDM), 2011 IEEE International (Washington, DC), 31.6.1-31.6.4.
34. Gütig, R., Aharonov, R., Rotter, S., and Sompolinsky, H. (2003). Learning input correlations through nonlinear temporally asymmetric hebbian plasticity. J. Neurosci. 23, 3697-3714.
35. Hubel, D. H., and Wiesel, T. N. (1959). Receptive fields of single neurones in the cat's striate cortex. J. Physiol. 148, 574-591.
36. Hur, J. H., Lee, M.-J., Lee, C. B., Kim, Y.-B., and Kim, C.-J. (2010). Modeling for bipolar resistive memory switching in transition-metal oxides. Phys. Rev. B 82:155321. doi: 10.1103/PhysRevB.82.155321
37. Indiveri, G. (2000). Modeling selective attention using a neuromorphic analog VLSI device. Neural Comput. 12, 2857-2880.
38. Indiveri, G., Chicca, E., and Douglas, R. (2006). A VLSI array of low-power spiking neurons and bistable synapses with spike-timing dependent plasticity. IEEE Trans. Neural Netw. 17, 211-221.
39. Indiveri, G., and Horiuchi, T. K. (2011). Frontiers in neuromorphic engineering. Front. Neurosci. 5:118. doi: 10.3389/fnins.2011.00118
40. Jacob, V., Brasier, D. J., Erchova, I., Feldman, D., and Shulz, D. E. (2007). Spike-timing-dependent synaptic depression in the in vivo barrel cortex of the rat. J. Neurosci. 27, 1271-1284.
41. Jo, S. H., Chang, T., Ebong, I., Bhadviya, B. B., Mazumder, P., and Lu, W. (2010). Nanoscale memristor device as synapse in neuromorphic systems. Nano Lett. 10, 1297-1301.
42. Jo, S. H., Kim, K.-H., and Lu, W. (2009). High-density crossbar arrays based on a Si memristive system. NANO Lett. 9, 870-874.
43. Khan, M., Lester, D., Plana, L., Rast, A., Jin, X., Painkras, E., et al. (2008). Spinnaker: mapping neural networks onto a massively-parallel chip multiprocessor. IEEE International Joint Conference on Neural Networks (IJCNN-WCCI) (Hong Kong), 2849-2856.
44. Kramer, J. (1996). Compact integrated motion sensor with three-pixel interaction. IEEE Trans. Pattern Anal. Mach. Intell. 18, 455-460.
45. Kuzum, D., Jeyasingh, R. G. D., Lee, B., and Wong, H.-S. P. (2012). Nanoelectronic programmable synapses based on phase change materials for brain-inspired computing. NANO Lett. 12, 2179-2186.
46. Kwon, D.-H., Kim, K. M., Jang, J. H., Jeon, J. M., Lee, M. H., Kim, G. H., et al. (2010). Atomic structure of conducting nanofilaments in TiO2 resistive switching memory. Nat. Nanotech. 5, 148-153.
47. 2-x bilayer structures. Nat. Mater. 10, 625-630.
48. Leñero-Bardallo, J. A., Serrano-Gotarredona, T., and Linares-Barranco, B. (2010). A five-decade dynamic range ambient-light-independent calibrated signed-spatial-contrast AER retina with 0.1ms latency and optional time-to-first-spike mode. IEEE Trans. Circ. Syst. I 57, 2632-2643.
49. Leñero-Bardallo, J. A., Serrano-Gotarredona, T., and Linares-Barranco, B. (2011). A 3.6 μs latency asynchronous frame-free event-based dynamic vision sensor. IEEE J. Solid State Circ. 46, 1443-1455.
50. Lichtsteiner, P., Posch, C., and Delbrück, T. (2008). A 128×128 120 dB 15 μs latency asynchronous temporal contrast vision sensor. IEEE J. Solid State Circ. 43, 566-576.
51. Linares-Barranco, B., and Serrano-Gotarredona, T. (2009a). "Exploiting memristance in adaptive asynchronous spiking neuromorphic nanotechnology systems," in 9th IEEE Conference on Nanotechnology (IEEE-NANO 2009) (Genoa), 601-604.
52. Linares-Barranco, B., and Serrano-Gotarredona, T. (2009b). Memristance can explain spike-time-dependent-plasticity in neural synapses. Available from Nature Precedings hdl.handle.net/10101/npre.2009.3010.1
53. Mallik, U., Clapp, M., Choi, E., Cauwenberghs, G., and Etienne-Cummings, R. (2005). "Temporal change threshold detection imager," in Solid-State Circuits Conference, 2005. Digest of Technical Papers. ISSCC. 2005 IEEE International, Vol. 1 (San Francisco, CA), 362-603.
54. Markram, H., Lübke, J., Frotscher, M., and Sakmann, B. (1997). Regulation of synaptic efficacy by coincidence of postsynaptic APS and EPSPS. Science 275, 213-215.
55. Masquelier, T. (2012). Relative spike time coding and STDP-based orientation selectivity in the early visual system in natural continuous and saccadic vision: a computational model. J. Comput. Neurosci. 32, 425-441.
56. Masquelier, T., Guyonneau, R., and Thorpe, S. J. (2008). Spike timing dependent plasticity finds the start of repeating patterns in continuous spike trains. PLoS ONE 3:e1377. doi: 10.1371/journal.pone.0001377
57. Masquelier, T., Guyonneau, R., and Thorpe, S. J. (2009). Competitive STDP-based spike pattern learning. Neural Comput. 21, 1259-1276.
58. Massari, N., Gottardi, M., and Jawed, S. (2008). "A 100 μW 64×128-pixel contrast-based asynchronous binary vision sensor for wireless sensor networks," in Solid-State Circuits Conference, 2008. ISSCC 2008. Digest of Technical Papers. IEEE International (San Francisco, CA), 588-638.
59. Mill, R., Sheik, S., Indiveri, G., and Denham, S. L. (2011). A model of stimulus-specific adaptation in neuromorphic analog VLSI. IEEE Trans. Biomed. Circ. Syst. 5, 413-419.
60. Mitra, S., Fusi, S., and Indiveri, G. (2009). Real-time classification of complex patterns using spike-based learning in neuromorphic VLSI. IEEE Trans. Biomed. Circ. Syst. 3, 32-42.
61. Mu, Y., and Poo, M. M. (2006). Spike timing-dependent LTP/LTD mediates visual experience-dependent plasticity in a developing retinotectal system. Neuron 50, 115-125.
62. Nian, Y., Strozier, J., Wu, N., Chen, X., and Ignatiev, A. (2007). Evidence for an oxygen diffusion model for the electric pulse induced resistance change effect in transition-MetalOxides. Phys. Rev. Lett. 98, 146-403.
63. Ohno, T. (2011). Short-term plasticity and long-term potentiation mimicked in single inorganic synapses. Nat. Mater. 10, 591-595.
64. Oster, M., Wang, Y., Douglas, R., and Liu, S.-C. (2008). Quantification of a spike-based winner-take-all VLSI network. IEEE Trans. Circ. Syst. I 55, 3160-3169.
65. Ozalevli, E., and Higgins, C. M. (2005). Reconfigurable biologically inspired visual motion system using modular neuromorphic VLSI chips. IEEE Trans. Circ. Syst. I 52, 79-92.
66. Pershin, Y. V., and Di Ventra, M. (2011). Memory effects in complex materials and nanoscale systems. Adv. Phys. 60, 145-227.
67. Posch, C., Matolin, D., and Wohlgenannt, R. (2011). A QVGA 143 dB dynamic range frame-free PWM image sensor with lossless pixel-level video compression and time-domain CDS. IEEE J. Solid State Circ. 46, 259-275.
68. Prodromakis, T., Peh, B. P., Papavassiliou, C., and Toumazou, C. (2011). A versatile memristor model with nonlinear dopant kinetics. IEEE Trans. Electron Devices 58, 3099-3105.
69. 2 memristors," in Nature Conference on Frontiers in Electronic Materials: Correlation Effects and Memristive Phenomena (Aachen, Germany).
70. Prodromakis, T., Toumazou, C., and Chua, L. O. (2012b). Two centuries of memristors. Nat. Mater. 11, 478-481.
71. Querlioz, D., Bichler, O., and Gamrat, C. (2011). "Simulation of a memristor-based spiking neural network immune to device variations," in Proceedings of the International Joint Conference on Neural Networks (IJCNN) (San Jose, CA), 1775-1781.
72. Rast, A., Galluppi, F., Davies, S., and Furber, S. (2010). "Implementing spike-timing-dependent plasticity on SpiNNaker neuromorphic hardware," in The 2010 International Joint Conferences on Neural Networks (IJCNN) (Barcelona).
73. Rojas, R. (1996). Neural Networks: A Systematic Introduction. Berlin: Springer.
74. Rubin, J., Lee, D. D., and Sompolinsky, H. (2001). Equilibrium properties of temporally asymmetric hebbian plasticity. Phys. Rev. Lett. 86, 364-367.
75. Ruedi, P.-F., Heim, P., Kaess, F., Grenet, E., Heitger, F., Burgi, P.-Y., et al. (2003). A 128×128 pixel 120-dB dynamic-range vision sensor chip for image contrast and orientation extraction. IEEE J. Solid State Circ. 38, 2325-2333.
76. Sarpeshkar, R., Baker, M. W., Salthouse, C. D., Sit, J.-J., Turicchia, L., and Zhak, S. M. (2005). "An analog bionic ear processor with zero-crossing detection," in Solid-State Circuits Conference, 2005. Digest of Technical Papers. ISSCC. 2005 IEEE International, Vol. 1 (San Francisco, CA), 78-79.
77. Sarpeshkar, R., Kramer, J., Indiveri, G., and Koch, C. (1996). Analog VLSI architectures for motion processing: from fundamental limits to system applications. Proc. IEEE 84, 969-987.
78. 2. IEEE Trans. Electron Devices 54, 2762-2768.
79. Serrano-Gotarredona, R., Serrano-Gotarredona, T., Acosta-Jimenez, A., and Linares-Barranco, B. (2006). A neuromorphic cortical-layer microchip for spike-based event processing vision systems. IEEE Trans. Circ. Syst. I 53, 2548-2566.
80. Serrano-Gotarredona, R., Oster, M., Lichtsteiner, P., Linares-Barranco, A., Paz-Vicente, R., Gomez-Rodriguez, F., et al. (2009). CAVIAR: a 45k neuron, 5M synapse, 12G connects/s AER hardware sensory-processing-learning-actuating system for high-speed visual object recognition and tracking. IEEE Trans. Neural Netw. 20, 1417-1438.
81. Serrano-Gotarredona, T., and Linares-Barranco, B. (2013). A 128 × 128 1.5% contrast sensitivity 0.9% FPN 3us latency 4 mW asynchronous frame-free dynamic vision sensor using transimpedance amplifiers. IEEE J. Solid-State Circuits (in press).
82. Shihong, M. W., Prodromakis, T., Salaoru, I., and Toumazou, C. (2012). Modelling of current percolation channels in emerging resistive switching elements. arXiv:2746v1 [cond-mat.mes-hall].
83. Sjöström, J., and Gerstner, W. (2010). Spike-timing dependent plasticity. Scholarpedia 5:1362. doi: 10.4249/scholarpedia.1362
84. Snider, G. S. (2008). "Spike-timing-dependent learning in memristive nanodevices," in IEEE International Symposium on Nano Architectures (Anaheim, CA), 85-92.
85. Strukov, D. B., Snider, G. S., Stewart, D. R., and Williams, R. S. (2008). The missing memristor found. Nature 453, 80-83.
86. Teixeira, T., Andreou, A. G., and Culurciello, E. (2005). "Event-based imaging with active illumination in sensor networks," in Proceedings of the IEEE International Symposium on Circuits and Systems (ISCAS) (Kobe), 644-647.
87. Terabe, K., Hasegawa, T., Nakayama, T., and Aono, M. (2005). Quantized conductance atomic switch. Nature 433, 47-50.
88. van Rossum, M. C. W., Bi, G. Q., and Turrigiano, G. G. (2000). Stable hebbian learning from spike timing-dependent plasticity. J. Neurosci. 20, 8812-8821.
89. Venier, P., Mortara, A., Arreguit, X., and Vittoz, E. A. (1997). An integrated cortical layer for orientation enhancement. IEEE J. Solid State Circ. 32, 177-186.
90. Vogelstein, R., Mallik, U., Vogelstein, J., and Cauwenberghs, G. (2007). Dynamically reconfigurable silicon array of spiking neurons with conductance-based synapses. IEEE Trans. Neural Netw. 18, 253-265.
91. Wen, B., and Boahen, K. (2006). "A 360-channel speech preprocessor that emulates the cochlear amplifier," in IEEE ISSCC Digest of Technical Papers (San Francisco, CA), 556-557.
92. Wen, B., and Boahen, K. (2009). A silicon cochlea with active coupling. IEEE Trans. Biomed. Circ. Syst. 3, 444-455.
93. Wuttig, M., and Yamada, N. (2007). Phase-change materials for rewriteable data storage. Nat. Mater. 6, 824-832.
94. Yang, J. J., Pickett, M. D., Li, X., Ohlberg, D. A. A., Stewart, D. R., and Williams, R. S. (2008). Memristive switching mechanism for metal/oxide/metal nanodevices. Nat. Nanotech. 3, 429-433.
95. Yang, Y., Gao, P., Gaba, S., Chang, T., Pan, X., and Lu, W. (2012). Observation of conducting filament growth in nanoscale resistive memories. Nat. Commun. 3:732. doi: 10.1038/ncomms1737
96. Young, J. M. (2007). Cortical reorganization consistent with spike timing-but not correlation-dependent plasticity. Nat. Neurosci. 10, 887-895.
97. Zaghloul, K. A., and Boahen, K. (2004). Optic nerve signals in a neuromorphic chip I & II. IEEE Trans. Biom. Eng. 51, 657-675.
98. Zamarreño-Ramos, C., Camuñas-Mesa, L. A., Pèrez-Carrasco, J. A., Masquelier, T., Serrano-Gotarredona, T., and Linares-Barranco, B. (2011). On spike-timing-dependent-plasticity, memristive devices, and building a self-learning visual cortex. Front. Neurosci. 5:26. doi: 10.3389/fnins.2011.00026
99. Zamarreno-Ramos, C., Linares-Barranco, A., Serrano-Gotarredona, T., and Linares-Barranco, B. (2012). Multicasting mesh AER: a scalable assembly approach for reconfigurable neuromorphic structured AER systems. application to ConvNets. IEEE Trans. Biomed. Circ. Syst. Available online at: http://ieeexplore.ieee.org/xpl/articleDetails.jsp?arnumber=6211459
100. Zhang, L., Tao, H., Holt, C., Harris, W., and Poo, M. (1998). A critical window for cooperation and competition among developing retinotectal synapses. Nature 395, 37-44.