A proposal for hybrid memristor-CMOS spiking neuromorphic learning systems

IEEE Circuits and Systems Magazine

Volumn 13, Issue 2, 2013, Pages 74-88

  Serrano Gotarredona, Teresa ^a   Prodromakis, Themistoklis ^b   Linares Barranco, Bernabé ^a  

^a INSTITUTO DE MICROELECTRÓNICA DE SEVILLA   (Spain)

^b UNIVERSITY OF SOUTHAMPTON   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

CIRCUIT ARCHITECTURES; LEARNING ARCHITECTURES; LEARNING MECHANISM; NEUROMORPHIC ENGINEERING; PRACTICAL REALIZATIONS; PRINTED CIRCUIT BOARDS (PCB); SPIKE TIMING DEPENDENT PLASTICITIES; SYSTEM ARCHITECTURES;

LEARNING SYSTEMS; MEMRISTORS; NANOTECHNOLOGY; NEURONS; PRINTED CIRCUIT BOARDS; RESISTORS;

PASSIVE FILTERS;

EID: 84878304982     PISSN: 1531636X     EISSN: None     Source Type: Journal    
DOI: 10.1109/MCAS.2013.2256271     Document Type: Article

References (81)

1. D. B. Strukov, G. S. Snider, D. R. Stewart, and R. S. Williams, "The missing memristor found," Nature, vol. 453, pp. 80-83, May 2008. (Pubitemid 351630336)
2. J. Borghetti, Z. Li, J. Straznicky, X. Li, D. A. A. Ohlberg, W. Wu, D. R. Stewart, and R. S. Williams, "A hybrid nanomemristor/transistor logic circuit capable of self-programming," Proc. Natl. Acad. Sci. U. S. A., vol. 106, no. 6, pp. 1699-1703, Feb., 2009.
3. S. H. Jo, T. Chang, I. Ebong, B. B. Bhadviya, P. Mazumder, and W. Lu, "Nanoscale memristor device as synapse in neuromorphic systems," Nano Lett., vol. 10, no. 4, pp. 1297-1301, 2010.
4. S. H. Jo, K.-H. Kim, and W. Lu, "High-density crossbar arrays based on a si memristive system," Nano Lett., vol. 9, no. 2, pp. 870-874, 2009.
5. L. O. Chua, "Memristor-the missing circuit element," IEEE Trans. Circuit Theory, vol. 18, pp. 507-519, 1971.
6. L. O. Chua and S. M. Kang, "Memristive devices and systems," Proc. IEEE, vol. 64, no. 2, pp. 209-223, Feb. 1976.
7. L. O. Chua, C. A. Desoer, and E. S. Kuh, Linear and Nonlinear Circuits. New Yo rk: McGraw-Hill, 1987.
8. W. Gerstner, R. Ritz, and J. L. Hemmen, "Why spikes? Hebbian learning and retrieval of time-resolved excitation patterns," Biol. Cybern., vol. 69, pp. 503-515, 1993.
9. J. Sjöström and W. Gerstner. (2010). Spike-timing dependent plasticity. Scholarpedia [Online]. 5(2), p. 1362. Available: http://www. scholarpedia.org/article/STDP)
10. R. P. N. Rao and T. J. Sejnowski, "Spike-time-dependent Hebbian plasticity as temporal difference learning," Neural Comput., vol. 13, no. 10, pp. 2221-2237, 2001. (Pubitemid 33676882)
11. B. Porr and F. Wörgötter, "How the shape of pre-and postsynaptic signals can influence STDP: A biophysical model," Neural Comput., vol. 16, no. 3, pp. 595-625, 2004. (Pubitemid 38157017)
12. A. Delorme, L. Perrinet, and S. J. Thorpe, "Networks of integrate-and-fire neurons using Rank Order Coding B: Spike timing dependent plasticity and emergence of orientation selectivity," Neurocomputing, vol. 38-40, pp. 539-545, 2001. (Pubitemid 35276680)
13. T. Masquelier, R. Guyonneau, and S. J. Thorpe, "Spike timing dependent plasticity finds the start of repeating patterns in continuous spike trains," PLoS ONE, vol. 3 no. 1, p. e1377, 2008. (Pubitemid 351223139)
14. T. Masquelier, R. Guyonneau, and S. J. Thorpe, "Competitive STDP-based spike pattern learning," Neural Comput., vol. 21 no. 5, pp. 1259-1276, 2009, doi: 10.1162/neco.2008.06-08-804.
15. L. A. Finelli, S. Haney, M. Bazhenov, M. Stopfer, and T. J. Sejnowski, "Synaptic learning rules and sparse coding in a model sensory system," PLoS Comput. Biol., vol. 4, no. 4, p. e1000062, 2008.
16. D. O. Hebb, The Organization of Behavior: A Neuropsychological Study. New York: Wiley, 1949.
17. G. S. Snider, "Spike-timing-dependent learning in memristive nanodevices," in Proc. IEEE Int. Symp. Nano Architectures, June 2008, pp. 85-92.
18. B. Linares-Barranco and T. Serrano-Gotarredona. (2009, March). Memristance can explain spike-time-dependent-plasticity in neural synapses. Nature Precedings [Online]. Available: http://hdl.handle. net/10101/npre.2009. 3010.1
19. C. Zamarreño-Ramos, L. A. Camuñas-Mesa, J. A. Pérez-Carrasco, T. Masquelier, T. Serrano-Gotarredona, and B. Linares-Barranco, "On spike-timing-dependent-plasticity, memristive devices, and building a self-learning visual cortex," Front Neurosci., vol. 5, no. 26, 2011, doi: 10.3389/fnins.2011.00026.
20. T. Serrano-Gotarredona, T. Prodromakis, T. Masquelier, G. Indiveri, and B. Linares-Barranco, "STDP and STDP variations with memristors for spiking neuromorphic learning systems," Front. Neurosci., vol. 7, no. 2, 2013, doi: 10.3389/fnins.2013.00002.
21. H. Markram, J. Lübke, M. Frotscher, and B. Sakmann, "Regulation of synaptic efficacy by coincidence of postsynaptic APS and EPSPS," Science, vol. 275, no. 5297, pp. 213-215, 1997. (Pubitemid 27034762)
22. G. Bi and M. M. Poo, "Synaptic modification by correlated activity: Hebb's postulate revisited," Annu. Rev. Neurosci., vol. 24, pp. 139-166, 2001. (Pubitemid 32695226)
23. S. Thorpe, et al. "Speed of processing in the human visual system," Nature, vol. 381, no. 6582, pp. 520-522, 1996. (Pubitemid 26170929)
24. J. Costas-Santos, T. Serrano-Gotarredona, R. Serrano-Gotarredona, and B. Linares-Barranco, "A spatial contrast retina with on-chip calibration for neuromorphic spike-based AER vision systems," IEEE Trans. Circuits Syst. I, Reg. Papers, vol. 54, no. 7, pp. 1444-1458, July 2007. (Pubitemid 47104011)
25. J. A. Leñero-Bardallo, T. Serrano-Gotarredona, and B. Linares-Barranco, "A five-decade dynamic-range ambient-light-independent calibrated signed-spatial-contrast AER retina with 0.1-ms latency and optional time-to-first-spike mode," IEEE Trans. Circuits Systems I, Reg. Papers, vol. 57, no. 10, pp. 2632-2643, Oct. 2010.
26. J. A. Leñero-Bardallo, T. Serrano-Gotarredona, and B. Linares-Barranco, "A 3.6 s latency asynchronous frame-free event-driven dynamic-vision-sensor," IEEE J. Solid-State Circuits, vol. 46, no. 6, pp. 1443-1455, June 2011.
27. T. Serrano-Gotarredona, and B. Linares-Barranco, "A 128 # 128 1.5% contrast sensitivity 0.9% FPN 3 μ s latency 4 mW asynchronous frame-free dynamic vision sensor using transimpedance preamplifiers," IEEE J. Solid-State Circuits, vol. 48, no. 3, pp. 827-838, Mar. 2013.
28. P. Lichtsteiner, C. Posch, and T. Delbrück, "A 128 # 128 120 dB 15 μ s-latency asynchronous temporal contrast vision sensor," IEEE J. Solid-State Circuits, vol. 43, no. 2, pp. 566-576, 2008. (Pubitemid 351190224)
29. C. Posch, D. Matolin, and R. Wohlgenannt, "A QVGA 143 dB dynamic range frame-free PWM image sensor with lossless pixel-level video compression and time-domain CDS," IEEE J. Solid-State Circuits, vol. 46, no. 1, pp. 259-275, Jan. 2011.
30. E. Culurciello, R. Etienne-Cummings, and K. Boahen, "A biomor-phic digital image sensor," IEEE J. Solid-State Circuits, vol. 38, no. 2, pp. 281-294, 2003.
31. P.-F. Ruedi, P. Heim, F. Kaess, E. Grenet, F. Heitger, P.-Y. Burgi, S. Gyger, and P. Nussbaum, "A 128 # 128 pixel 120-dB dynamic-range vision-sensor chip for image contrast and orientation extraction," IEEE J. Solid-State Circuits, vol. 38, no. 12, pp. 2325-2333, Dec. 2003.
32. K. A. Zaghloul and K. Boahen, "A silicon retina that reproduces signals in the optic nerve," J. Neural Eng., vol. 3, pp. 257-267, Dec. 2006.
33. T. Serrano-Gotarredona, R. Serrano-Gotarredona, A. Acosta-Jiménez, and B. Linares-Barranco, "A neuromorphic cortical-layer microchip for spike-based event processing vision systems," IEEE Trans. Circuits and Syst. I, Reg. Papers, vol. 53, no. 12, pp. 2548-2566, Dec. 2006. (Pubitemid 46002253)
34. L. Camuñas-Mesa, A. Acosta-Jiménez, C. Zamarreño- Ramos, T. Serrano-Gotarredona, and B. Linares-Barranco, "A 32 # 32 pixel convolution processor chip for address event vision sensors with 155 ns event latency and 20 meps throughput," IEEE Trans. Circuits and Syst. I, Reg. Papers, vol. 58, no. 4, pp. 777-790, Apr. 2011.
35. L. Camuñas-Mesa, C. Zamarreño-Ramos, A. Linares-Barranco, A. J. Acosta-Jiménez, T. Serrano-Gotarredona, and B. Linares-Barranco, "An event-driven multi-kernel convolution processor module for event-driven vision sensors," IEEE J. Solid-State Circuits, vol. 47, no. 2, pp. 504-517, Feb. 2012.
36. T. Choi, P. A. Merolla, J. V. A rthur, K. A. Boahen, and B. E. Shi, "Neuromorphic implementation of orientation hypercolumns," IEEE Trans. Circuits Syst. I, Reg. Papers, vol. 52, no. 6, pp. 1049-1060, 2005. (Pubitemid 40970277)
37. J. Schemmel, D. Brüderle, A. Grübl, M. Hock, K. Meier, and S. Millner, "A wafer-scale neuromorphic hardware system for large-scale neural modeling," in Proc. 2010 Int. Symp. Circuits and Systems (ISCAS), Paris, pp. 1947-1950.
38. S. B. Furber, S. Temple, and A. D. Brown, "High performance computing for systems of spiking neurons," in Proc. AISB'06 Workshop on GC5: Architecture Brain and Mind, vol. 2, pp. 29-36.
39. R. Serrano-Gotarredona, M. Öster, P. Lichtsteiner, A. Linares-Barranco, R. Paz-Vicente, F. Gómez-Rodríguez, L. Camuñas-Mesa, R. Berner, M. Rivas-Pérez, T. Delbrück, S.-C. Liu, R. Douglas, P. Häfliger, G. Jiménez-Moreno, A. Civit-Ballcels, T. Serrano-Gotarredona, A. J. Acosta-Jiménez, and B. Linares-Barranco, "CAVIAR: A 45k neuron, 5M synapse, 12G connects/s AER hardware sensory-processing-learning-actuating system for high-speed visual object recognition and tracking," IEEE Trans. Neural Networks, vol. 20, no. 9, pp. 1417-1438, Sept. 2009.
40. C. Zamarreño-Ramos, A. Linares-Barranco, T. Serrano-Gotarredona, and B. Linares-Barranco, "Multi-casting mesh AER: A scalable assembly approach for reconfigurable neuromorphic structured AER systems. Application to ConvNets," IEEE Trans. Biomed. Circuits Syst., to be published.
41. J. A. Pérez-Carrasco, B. Zhao, C. Serrano, B. Acha, T. Serrano-Gotarredona, S. Chen, and B. Linares-Barranco, "Mapping from frame-driven to frame-free event-driven vision systems by low-rate rate-coding. Application to feed forward ConvNets," IEEE Trans. Pattern Anal. Mach. Intell., to be published.
42. C. Shoushun, P. Akselrod, B. Zhao, J. A. Pérez-Carrasco, B. Linares-Barranco, and E. Culurciello, "Efficient feedforward categorization of objects and human postures with address-event image sensors," IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 2, pp. 302-314, Feb. 2012.
43. T. Serrano-Gotarredona, J. Park, A. Linares-Barranco, R. Benosman, and B. Linares-Barranco, "Improved contrast sensitivity DVS and its application to event-driven stereo vision," in Proc. IEEE Int. Symp. Circuits and Systems, May 2013.
44. Z. Ni, C. Pacoret, R. Benosman, S. Ieng, and S. Régnier, "A sy n-chronous event-based high speed vision for microparticle tracking," J. Microsc., vol. 245, no. 3, pp. 236-244, Ma r. 2012.
45. P. Rogister, R. Benosman, S.-H. Ieng, P. Lichtsteiner, and T. Delbrück, "Asynchronous event-based binocular stereo matching," IEEE Trans. Neural Networks Learn. Syst., vol. 23, no. 2, pp. 347-353, Feb. 2012.
46. M. Hofstatter, M. Litzenberger, D. Matolin, and C. Posch, "Hardware-accelerated address-event processing for high-speed visual object recognition," in Proc. 18th IEEE Int. Conf. Electronics, Circuits and Systems, 2011.
47. Z. Ni, A. Bolopion, J. Agnus, R. Benosman, and S. Regnier, "Asyn-chronous event-based visual shape tracking for stable haptic feedback in microrobotics," IEEE Trans. Robot., 2012.
48. R. Benosman, S. H. Ieng, P. Rogister, and C. Posch, "Asynchronous event-based Hebbian epipolar geometry," IEEE Trans. Neural Networks, vol. 22, pp. 1723-1734, 2011.
49. R. Benosman, S.-H. Ieng, C. Clercq, C. Bartolozzi, and M. Srinivasan, "Asynchronous frameless event-based optical flow," Neural Netw., vol. 27, pp. 32-37, Mar. 2012.
50. J. Conradt, M. Cook, R. Berner, P. Lichtsteiner, R. J. Douglas, and T. Delbrück, "A pencil balancing robot using a pair of AER dynamic vision sensors," in Proc. Int. Symp. Circuits and Systems, ISCAS, 2009, pp. 781-784.
51. Z. Fu, T. Delbrück, P. Lichtsteiner, and E. Culurciello, "Anaddressevent fall detector for assisted living applications," IEEE Trans. Biomed. Circuits Syst., vol. 2, pp. 88-96, 2008.
52. D. Drazen, P. Lichtsteiner, P. Häfliger, T. Delbrück, and A. Jensen, "Toward real-time particle tracking using an event-based dynamic vision sensor," Exp. Fluids, vol. 51, no. 5, pp. 1465-1469, Sept. 2011.
53. S. Schraml and A. N. Belbachir, "A spatio-temporal clustering method using real-time motion analysis on event-based 3D vision," in Proc. IEEE Computer Vision and Pattern Recognition Workshop (CVPRW), 2010, pp. 57-63.
54. O. Bichler, D. Querlioz, S. J. Thorpe, J. P. Bourgoin, and C. Gamrat, "Extraction of temporally correlated features from dynamic vision sensors with spike-timing-dependent plasticity," Neural Netw., vol. 32, pp. 339-348, 2012.
55. D. Kuzum, R. G. D. Jeyasingh, B. Lee, and H.-S. P. Wong, "Nanoelec-tronic programmable synapses based on phase change materials for brain-inspired computing," Nano Lett., vol. 12, pp. 2179-2186, 2012.
56. C. Zamarreño-Ramos, T. Serrano-Gotarredona, and B. Linares-Barranco, "An instant-startup jitter-tolerant manchester-encoding serializer/deserializar scheme for event-driven bit-serial LVDS interchip AER links," IEEE Trans. Circuits Syst. I, Reg. Papers, vol. 58, no. 11, pp. 2647-2660, Nov. 2011.
57. C. Zamarreño-Ramos, T. Serrano-Gotarredona, and B. Linares-Barranco, "A 0.35 nm sub-ns wake-up time on-off switchable LVDS driver-receiver chip I/O pad pair for rate-dependent power saving in AER bit-serial links," IEEE Trans. Biomed. Circuits Syst., vol. 6, no. 5, pp. 486-497, Oct. 2012.
58. C. Zamarreño-Ramos, R. Kulkarni, J. Silva-Martínez, T. Serrano-Gotarredona, and B. Linares-Barranco, "A 1.5 ns off/on switching-time voltage-mode LVDS driver/receiver pair for asynchronous AER bit-serial chip grid links with up to 40 times event-rate dependent power savings," IEEE Trans. Biomed. Circuits Syst., to be published.
59. Y. LeCun, B. Boser, J. S. Denker, D. Henderson, R. E. Howard, W. Hubbard, and L. D. Jackel, "Backpropagation applied to handwritten zip code recognition," Neural Comput., vol. 1, no. 4, pp. 541-551, Dec. 1989.
60. C. Farabet, et al. "Large-scale FPGA-based convolutional networks," in Machine Learning on Very Large Data Sets, R. Bekkerman, M. Bilenko, and J. Langford, Eds. Cambridge, U.K.: Cambridge Univ. Press, 2011.
61. M. C. W. van Rossum, G. Q. Bi, and G. G. Turrigiano, "Stable Hebbian learning from spike timing-dependent plasticity," J. Neurosci., vol. 20, no. 23, pp. 8812-8821, Dec. 2000.
62. R. Gütig, R. Aharonov, S. Rotter, and H. Sompolinsky, "Learning input correlations through nonlinear temporally asymmetric Hebbian plasticity," J. Neurosci., vol. 23, no. 9, pp. 3697-3714, May 2003. (Pubitemid 36966039)
63. 2 thin-film devices," App. Phys. Lett., vol. 102, no. 1, pp. 013506-013506-4, 2013.
64. T. Prodromakis, B. P. Peh, C. Papavassiliou, and C. Toumazou," A versatile memristor model with nonlinear dopant kinetics," IEEE Trans. Electron Devices, vol. 58, no. 9, pp. 3099-3105.
65. F. Argall, "Switching phenomena in titanium oxide thin films," SolidState Electron., vol. 11, pp. 535-541, 1968.
66. B. Swaroop, W. C. West, G. Martinez, M. N. Kozicki, and L. A. Akers, "Programmable current mode Hebbian learning neural network using programmable metallization cell," in Proc. 1998 IEEE Int. Symp. Circuits and Systems (ISCAS), vol. 3, pp. 33-36.
67. T. Prodromakis, C. Toumazou, and L. O. Chua, "Two centuries of memristors," Nat. Mater., vol. 11, pp. 478-481, May 2012, doi:10.1038/nmat3338.
68. Y. Nian, J. Strozier, N. Wu, X. Chen, and A. Ignatiev, "Evidence for an oxygen diffusion model for the electric pulse induced resistance change effect in transition-metal oxides," Phys. Rev. Lett., vol. 98, no. 14, pp. 146-403, Apr. 2007.
69. J. J. Yang, M. D. Pickett, X. Li, D. A. A. Ohlberg, D. R. Stewart, and R. S. Williams, "Memristive switching mechanism for metal/oxide/metal nanodevices," Nat. Nanotechnol., vol. 3, no. 7, pp. 429-433, June 2008.
70. J. Hur, M.-J. Lee, C. Lee, Y.-B. Kim, and C. J. Kim, "Modeling for bipolar resistive memory switching in transition-metal oxides," Phys. Rev. B, vol. 82, no. 15, Oct. 2010.
71. M. Wuttig and N. Yamada, "Phase-change materials for rewriteable data storage," Nat. Mater., vol. 6, no. 11, pp. 824-832, 2007. (Pubitemid 350050578)
72. M. Wu-Shihong, T. Prodromakis, I. Salaoru, and C. Toumazou, "Modelling of current percolation channels in emerging resistive switching elements," arXiv:1206.2746v, June 2012.
73. 2 resistive switching memory," Nat. Nanotechnol, vol. 5, pp. 148-153, 2010.
74. Y. Yang, P. Gao, S. Gaba, T. Chang, X. Pan, and W. Lu, "Observation of conducting filament growth in nanoscale resistive memories," Nat. Commun. vol. 3, no. 732, 2012, doi: 10.1038/ncomms1737, 2012.
75. A. Chanthbouala, V. Garcia, R. O. Cherifi, K. Bouzehouane, S. Fusil, X. Moya, S. Xavier, H. Yamada, C. Deranlot, N. D. Mathur, M. Bibes, A. Bartélémy, and J. Grollier, "A ferroelectric memristor," Nat. Mater., vol. 11, pp. 860-864, Oct. 2012.
76. B. Linares Barranco and T. Serrano Gotarredona, "Exploiting mem-ristance in adaptive asynchronous spiking neuromorphic nanotechnol-og y systems," in Proc. 9th IEEE Conf. Nanotechnology (IEEE-NANO 2009), pp. 601-604.
77. O. Bichler, M. Suri, D. Querlioz, D. Vuillaume, B. DeSalvo, and C. Gamrat, "Visual pattern extraction using energy-efficient '2-PCM synapse' neuromorphic architecture," IEEE Trans. Electron Devices, vol. 59, no. 8, pp. 2206-2214, Aug. 2012.
78. D. B. Strukov and K. K. Likharev, "CMOL FPGA: A reconfigurable architecture for hybrid digital circuits with two-terminal nanodevices," Nanotechnology, vol. 16, pp. 888-900, 2005. (Pubitemid 40666599)
79. 11 bits per square centimetre," Nature, vol. 445, pp. 414-417, Jan. 2007. (Pubitemid 46160906)
80. G.-Y. Jung, E. Johnston-Halperin, W. Wu, Z. Yu, S.-Y. Wang, W. M. Tong, Z. Li, J. E. Green, B. A. Sheriff, A. Boukai, Y. Bunimovich, J. R. Heath, and R. S. Williams, "Circuit fabrication at 17 nm half-pitch by nanoimprint lithography," Nano Lett., pp. 351-354, Feb. 2006.
81. H.-J. Jeon, et al., "New top-down approach for fabricating high-aspect-ratio complex nanostructures with 10 nm scale features," Nano Lett., vol. 10, no. 9, pp. 3604-3610, 2010.