Neuromorphic Computing Based on Emerging Memory Technologies

IEEE Journal on Emerging and Selected Topics in Circuits and Systems

Volumn 6, Issue 2, 2016, Pages 198-211

  Rajendran, Bipin ^a   Alibart, Fabien ^b  

^a NEW JERSEY INSTITUTE OF TECHNOLOGY   (United States)

^b CNRS   (France)

Author keywords

Cognitive computing; memristor; neuromorphic engineering; phase change memory (PCM); resistive random Access memory (RRAM); spin Transfer torque random Access memory (STT RAM)

Indexed keywords

ENERGY EFFICIENCY; LEARNING SYSTEMS;

BIOLOGICAL COMPUTATIONS; COGNITIVE COMPUTING; EMERGING MEMORY TECHNOLOGIES; ENERGY EFFICIENT; LARGE-SCALE LEARNING; NANOSCALE DEVICE; NEUROMORPHIC COMPUTING; THEORETICAL ESTIMATION;

COGNITIVE SYSTEMS;

EID: 84960145823     PISSN: 21563357     EISSN: None     Source Type: Journal    
DOI: 10.1109/JETCAS.2016.2533298     Document Type: Review

References (98)

1. J. von Neumann, "First draft of a report on the EDVAC, " IEEE Ann. Hist. Comput., vol. 15, pp. 27-75, Oct. 1993.
2. D. Patterson et al., "A case for intelligent RAM, " IEEE Micro, vol. 17, no. 2, pp. 34-44, Mar. 1997.
3. S. Iyer, "Algorithmic memory-An order of magnitude increase in next generation embedded memory performance, " in IBM STS Workship, Apr. 2013 [Online]. Available: http://stanford.io/1EoBA5h, Available at
4. H. Esmaeilzadeh, E. Blem, R. St. Amant, K. Sankaralingam, and D. Burger, "Dark silicon and the end of multicore scaling, " in Proc. 38th Annu. Int. Symp. Comput. Archit., 2011, pp. 365-376.
5. C. Liu, I. Ganusov, M. Burtscher, and S. Tiwari, "Bridging the processor-memory performance gap with 3D IC technology, " IEEE Design Test Comput., vol. 22, pp. 556-564, Nov. 2005.
6. B. Rajendran et al., "Low thermal budget processing for sequential 3-D IC fabrication, " IEEE Trans. Electron Devices, vol. 54, no. 4, pp. 707-714, Apr. 2007.
7. G. W. Burr et al., "Overview of candidate device technologies for storage-class memory, " IBM J. Res. Develop., vol. 52, pp. 449-464, Jul. 2008.
8. "DARPA SyNAPSE project, " [Online]. Available: http://bit.ly/1LNtDL8
9. "European human brain project, " [Online]. Available: http://bit.ly/NPvqaX
10. "FACETS-Framework application for core-edge transport simulations project, " [Online]. Available: http://bit.ly/1EoG7og
11. "SPINNAKER-A universal spiking neural network architecture, " [Online]. Available: http://bit.ly/1f9p9nh
12. "National academy of engineering-Reverse-engineer the brain, " 2009 [Online]. Available: http://bit.ly/1PmsLiX
13. M. K. Qureshi, S. Gurumurthi, and B. Rajendran, Phase Change Memory: From Devices to Systems. San Rafael, CA: Morgan Claypool, 2011.
14. L. Chua, "If its pinched its a memristor, " Semiconductor Sci. Technol., vol. 29, no. 10, p. 104001, 2014.
15. P. Hasler, C. Diorio, B. Minch, and C. Mead, "Single transistor learning synapse with long term storage, " in Proc. IEEE Int. Symp. Circuits Syst., Apr. 1995, vol. 3, pp. 1660-1663.
16. G. W. Burr et al., "Phase change memory technology, " J. Vac. Sci. Technol. B, vol. 28, pp. 223-262, 2010.
17. B. Rajendran et al., "Analytical model for RESET operation of phase change memory, " in Proc. IEEE Int. Electron Devices Meet., Dec. 2008, pp. 1-4.
18. A. Redaelli, A. Pirovano, A. Benvenuti, and A. L. Lacaita, "Threshold switching and phase transition numerical models for phase change memory simulations, " J. Appl. Phys., vol. 103, no. 11, p. 111101, 2008.
19. M. Breitwisch et al., "Novel lithography-independent pore phase change memory, " in Proc. IEEE Symp. VLSI Technol., Jun. 2007, pp. 100-101.
20. T. Nirschl et al., "Write strategies for 2 and 4-bit multi-level phasechange memory, " in Proc. IEEE Int. Electron Devices Meet., 2007, pp. 461-464.
21. B. Rajendran et al., "Demonstration of CAM and TCAM using phase change devices, " in Proc. 3rd IEEE Int. Memory Workshop, May 2011, pp. 1-4.
22. D. Kuzum, R. G. D. Jeyasingh, B. Lee, and H.-S. P. Wong, "Nanoelectronic programmable synapses based on phase change materials for brain-inspired computing, " Nano Lett., vol. 12, no. 5, pp. 2179-2186, 2012.
23. B. L. Jackson et al., "Nano-scale electronic synapses using phase change devices, " ACM J. Emerg. Technol. Comput., vol. 9, pp. 12:1-12:20, Sep. 2013.
24. M. Hosomi et al., "A novel nonvolatile memory with spin torque transfer magnetization switching: Spin-RAM, " in IEEE Int. Electron Devices Meet. Tech. Digest, Dec. 2005, pp. 459-462.
25. T. Min et al., "A study of write margin of spin torque transfer magnetic random access memory technology, " IEEE Trans. Magn., vol. 46, no. 6, pp. 2322-2327, Jun. 2010.
26. E. Kultursay, M. Kandemir, A. Sivasubramaniam, and O. Mutlu, "Evaluating STT-RAM as an energy-efficient main memory alternative, " in Proc. 2013 IEEE Int. Symp. Performance Anal. Syst. Software, Apr. 2013, pp. 256-267.
27. A. Vincent et al., "Spin-Transfer torque magnetic memory as a stochastic memristive synapse for neuromorphic systems, " IEEE Trans. Biomedical Circuits Syst., vol. 9, no. 2, pp. 166-174, Apr. 2015.
28. C. Baldassi, A. Braunstein, N. Brunel, and R. Zecchina, "Efficient supervised learning in networks with binary synapses, " in Proc. Nat. Acad. Sci., 2007, vol. 104, no. 26, pp. 11079-11084.
29. J. Satel, T. Trappenberg, and A. Fine, "Are binary synapses superior to graded weight representations in stochastic attractor networks?, " Cognitive Neurodynamics, vol. abs/1409.4842, pp. 243-250, Sep. 2009.
30. A. Singha, B. Muralidharan, and B. Rajendran, "Analog memristive time dependent learning using discrete nanoscale RRAM devices, " in Proc. 2014 Int. Joint Conf. Neural Netw., Jul. 2014, pp. 2248-2255.
31. R. Waser, R. Dittmann, G. Staikov, and K. Szot, "Redox-based resistive switching memories nanoionic mechanisms, prospects, and challenges, " Adv. Mater., vol. 21, pp. 2632-2663, Jul. 2009.
32. F. Alibart, L. Gao, B. D. Hoskins, and D. B. Strukov, "High precision tuning of state for memristive devices by adaptable variation-Tolerant algorithm, " Nanotechnology, vol. 23, no. 7, p. 075201, 2012.
33. F. Xiong, A. D. Liao, D. Estrada, and E. Pop, "Low-power switching of phase-change materials with carbon nanotube electrodes, " Science, vol. 332, no. 6029, pp. 568-570, 2011.
34. B. Govoreanu et al., "10 10 nm Hf/HfO crossbar resistive ram with excellent performance, reliability and low-energy operation, " in Proc. IEEE Int. Electron Devices Meet., Dec. 2011, pp. 31.6.1-31.6.4.
35. K.-H. Kim et al., "A functional hybrid memristor crossbar-Array/CMOS system for data storage and neuromorphic applications, " Nano Lett., vol. 12, pp. 389-395, Jan. 2012.
36. T. Ohno et al., "Short-Term plasticity and long-Term potentiation mimicked in single inorganic synapses, " Nature Mater., vol. 10, pp. 591-595, 2011.
37. A. Chanthbouala et al., "Vertical-current-induced domain-wall motion in MgO-based magnetic tunnel junctions with low current densities, " Nature Phys., vol. 7, pp. 626-630, 2011.
38. A. Chanthbouala et al., "A ferroelectric memristor, " Nature Mater., vol. 11, pp. 860-864, 2012.
39. G. Agnus et al., "Two-Terminal carbon nanotube programmable devices for adaptive architectures, " Adv. Mater., vol. 22, no. 6, pp. 702-706, 2010.
40. A. Chanthbouala et al., "Solid-state memories based on ferroelectric tunnel junctions, " Nature Nanotechnol., vol. 7, pp. 101-104, Dec. 2011.
41. L. Cario, C. Vaju, B. Corraze, V. Guiot, and E. Janod, "Electric-fieldinduced resistive switching in a family of mott insulators: Towards a new class of RRAM memories, " Adv. Mater., vol. 22, pp. 5193-5197, Oct. 2010.
42. J. Scott and L. Bozano, "Nonvolatile memory elements based on organic materials, " Adv. Mater., vol. 19, no. 11, pp. 1452-1463, 2007.
43. F. Alibart et al., "An organic nanoparticle transistor behaving as a biological spiking synapse, " Adv. Funct. Mater., vol. 20, no. 2, pp. 330-337, 2010.
44. , D. Purves, G. J. Augustine, and D. Fitzpatrick, Eds., Neuroscience. Sunderland, MA: Sinauer, 2001.
45. G. Q. Bi and M. M. Poo, "Synaptic modifications in cultured hippocampal neurons: Dependence on spike timing, synaptic strength, and postsynaptic cell type, " J. Neurosci., vol. 18, pp. 10464-10472, Dec. 1998.
46. D. Hebb, Organization of Behavior. New York: Wiley, 1949.
47. Y. LeCun, Y. Bengio, and G. Hinton, "Deep learning, " Nature, vol. 521, no. 7553, p. 444, 2015.
48. C. Szegedy et al., "Going deeper with convolutions, " in Proc. IEEE Conf. Comput. Vis. Pattern Recognit., Jun. 2015, pp. 1-9.
49. C. Zhang et al., "Optimizing FPGA-based accelerator design for deep convolutional neural networks, " in Proc. 2015 ACM/SIGDA Int. Symp. Field-Programmable Gate Arrays, New York, 2015, pp. 161-170.
50. W. Maas, "Networks of spiking neurons: The third generation of neural network models, " Trans. Soc. Comput. Simul. Int., vol. 14, pp. 1659-1671, Dec. 1997.
51. H. Markram, J. Lbke, M. Frotscher, and B. Sakmann, "Regulation of synaptic efficacy by coincidence of postsynaptic APs and EPSPs, " Science, vol. 275, no. 5297, pp. 213-215, 1997.
52. B. Rajendran et al., "Specifications of nanoscale devices and circuits for neuromorphic computational systems, " IEEE Trans. Electron Devices, vol. 60, no. 1, pp. 246-253, Jan. 2013.
53. N. Anwani and B. Rajendran, "NormAD-Normalized approximate descent based supervised learning rule for spiking neurons, " in 2015 Int. Joint Conf. Neural Netw., Jul. 2015.
54. G. Hinton et al., "Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups, " IEEE Signal Process. Mag., vol. 29, no. 6, pp. 82-97, Nov. 2012.
55. C. Prasad, K. Saboo, and B. Rajendran, "Composer classification based on temporal coding in adaptive spiking neural networks, " in 2015 Int. Joint Conf. Neural Netw., Jul. 2015.
56. A. Shen, R. Tong, and Y. Deng, "Application of classification models on credit card fraud detection, " in Proc. Int. Conf. Service Syst. Service Manage., Jun. 2007, pp. 1-4.
57. B. Huval et al., "An empirical evaluation of deep learning on highway driving, " CoRR, vol. abs/1504.01716, 2015.
58. V. Mnih et al., "Human-level control through deep reinforcement learning, " Nature, vol. 518, pp. 529-533, 2015.
59. G. Snider, "Spike-Timing-dependent learning in memristive nanodevices, " in Proc. IEEE Int. Symp. Nanoscale Archit., Jun. 2008, pp. 85-92.
60. D. Querlioz, O. Bichler, P. Dollfus, and C. Gamrat, "Immunity to device variations in a spiking neural network with memristive nanodevices, " IEEE Trans. Nanotechnol., vol. 12, no. 3, pp. 288-295, May 2013.
61. S. Kulkarni and B. Rajendran, "Scalable digital CMOS architecture for spike based supervised learning, " in Proceedings of the 16th EANN 2015, Sep. 2015, LNCS.
62. J. H. Lee and K. K. Likharev, "Defect-Tolerant nanoelectronic pattern classifiers, " Int. J. Circuit Theory Appl., vol. 35, no. 3, pp. 239-264, 2007.
63. M. Suri et al., "CBRAM devices as binary synapses for low-power stochastic neuromorphic systems: Auditory (cochlea) and visual (retina) cognitive processing applications, " in Proc. IEEE Int. Electron Devices Meet., Dec. 2012, pp. 10.3.1-10.3.4.
64. M. Suri et al., "Phase change memory as synapse for ultra-dense neuromorphic systems: Application to complex visual pattern extraction, " in Proc. IEEE Int. Electron Devices Meet., Dec. 2011, pp. 4.4.1-4.4.4.
65. G. Indiveri et al., "Neuromorphic silicon neuron circuits, " Front. Neurosci., vol. 5, 2011.
66. V. Ostwal, R. Meshram, B. Rajendran, and U. Ganguly, "An ultracompact and low power neuron based on SOI platform, " in Proc. Int. Symp. VLSI Technol., Syst. Appl., Apr. 2015, pp. 1-2.
67. J. Lin, P. Merolla, J. Arthur, and K. Boahen, "Programmable connections in neuromorphic grids, " in Proc. 49th IEEE Int. Midwest Symp. Circuits Syst., Aug. 2006, vol. 1, pp. 80-84.
68. P. A. Merolla et al., "A million spiking-neuron integrated circuit with a scalable communication network and interface, " Science, vol. 345, no. 6197, pp. 668-673, 2014.
69. A. Mortara, E. Vittoz, and P. Venier, "A communication scheme for analog VLSI perceptive systems, " IEEE J. Solid-State Circuits, vol. 30, no. 6, pp. 660-669, Jun. 1995.
70. M. Boegerhausen, P. Suter, and S.-C. Liu, "Modeling short-Term synaptic depression in silicon, " Neural Comput., vol. 15, pp. 331-348, Feb. 2003.
71. G. Indiveri, E. Chicca, and R. Douglas, "A VLSI array of low-power spiking neurons and bistable synapses with spike-Timing dependent plasticity, " IEEE Trans. Neural Netw., vol. 17, no. 1, pp. 211-221, Jan. 2006.
72. B. Rajendran et al., "Characterization of poly-silicon emitter BJTs as access devices for phase change memory, " in Proc. Int. Symp. VLSI Technol., Syst., Appl., Apr. 2009, pp. 31-32.
73. S. H. Jo, T. Kumar, S. Narayanan, W. Lu, and H. Nazarian, "3D-stackable crossbar resistive memory based on field assisted superlinear threshold (FAST) selector, " in Proc. IEEE Int. Electron Devices Meet., Dec. 2014, pp. 6.7.1-6.7.4.
74. G. Burr et al., "Experimental demonstration and tolerancing of a largescale neural network (165 000 synapses), using phase-change memory as the synaptic weight element, " in Proc. IEEE Int. Electron Devices Meet., Dec. 2014, pp. 29.5.1-29.5.4.
75. F. Alibart, E. Zamanidoost, and D. B. Strukov, "Pattern classification by memristive crossbar circuits using ex situ and in situ training, " Nature Commun., vol. 4, Jun. 2013.
76. S. H. Jo et al., "Nanoscale memristor device as synapse in neuromorphic systems, " Nano Letl., vol. 10, no. 4, pp. 1297-1301, 2010.
77. D. Modha and R. Shenoy, "Electronic learning synapse with spike-Timing dependent plasticity using unipolar memory-switching elements, " U.S. Patent 8 250 010, 2000.
78. N. Panwar et al., "Memristive synaptic plasticity in PCMO RRAM by bio-mimetic programming, " in Proc. 72nd Annu. Device Res. Conf., Jun. 2014, pp. 135-136.
79. F. Alibart et al., "A memristive nanoparticle/organic hybrid synapstor for neuro-inspired computing, " Adv. Funct. Mater., vol. 22, pp. 609-616, 2011.
80. S. Yu, Y. Wu, R. Jeyasingh, D. Kuzum, and H.-S. Wong, "An electronic synapse device based on metal oxide resistive switching memory for neuromorphic computation, " IEEE Trans. Electron Devices, vol. 58, no. 8, pp. 2729-2737, Aug. 2011.
81. T. Toyoizumi, J.-P. Pfister, K. Aihara, and W. Gerstner, "Generalized Bienenstock-Cooper-Munro rule for spiking neurons that maximizes information transmission, " in Proc. Nat. Acad. Sci. USA, 2005, vol. 102, no. 14, pp. 5239-5244.
82. M. Ziegler, C. Riggert, M. Hansen, T. Bartsch, and H. Kohlstedt, "Memristive hebbian plasticity model: Device requirements for the emulation of hebbian plasticity based on memristive devices, " IEEE Trans. Biomed. Circuits Syst., vol. 9, no. 2, pp. 197-206, Apr. 2015.
83. H. Lim, I. Kim, J.-S. Kim, C. S. Hwang, and D. S. Jeong, "Short-Term memory of TiO-based electrochemical capacitors: Empirical analysis with adoption of a sliding threshold, " Nanotechnology, vol. 24, no. 38, p. 384005, 2013.
84. J. Gjorgjieva, C. Clopath, J. Audet, and J.-P. Pfister, "A triplet spiketimingdependent plasticity model generalizes the Bienenstock-Cooper-Munro rule to higher-order spatiotemporal correlations, " in Proc. Nat. Acad. Sci., 2011, vol. 108, no. 48, pp. 19383-19388.
85. C. Mayr, P. Stärke, J. Partzsch, L. Cederstroem, R. Schöffny, Y. Shuai, N. Du, and H. Schmidt, "Waveform driven plasticity in BiFeO memristive devices: Model and implementation, " in Advances in Neural Information Processing Systems 25, F. Pereira, C. Burges, L. Bottou, and K. Weinberger, Eds. Red Hook, NY: Curran Assoc., 2012, pp. 1700-1708.
86. S. Kim, C. Du, P. Sheridan, W. Ma, S. Choi, and W. D. Lu, "Experimental demonstration of a second-order memristor and its ability to biorealistically implement synaptic plasticity, " Nano Lett., vol. 15, no. 3, pp. 2203-2211, 2015
87. S. B. Eryilmaz et al., "Brain-like associative learning using a nanoscale non-volatile phase change synaptic device array, " Front. Neurosci., vol. 8, no. 205, 2014.
88. M. Prezioso et al., "Training and operation of an integrated neuromorphic network based on metal-oxide memristors, " Nature, vol. 521, no. 7550, pp. 61-64, 2015.
89. D. Kuzum, R. Jeyasingh, S. Yu, and H.-S. Wong, "Low-energy robust neuromorphic computation using synaptic devices, " IEEE Trans. Electron Devices, vol. 59, no. 12, pp. 3489-3494, Dec. 2012.
90. S. Mandal, A. El-Amin, K. Alexander, B. Rajendran, and R. Jha, "Novel synaptic memory device for neuromorphic computing, " Nature Sci. Rep., vol. 4, no. 5333, 2014.
91. Y. Shih et al., "Mechanisms of retention loss in Ge Sb Te-based phase-change memory, " in Proc. IEEE Int. Electron Devices Meet., Dec. 2008, pp. 1-4.
92. S. La Barbera, D. Vuillaume, and F. Alibart, "Filamentary switching: Synaptic plasticity through device volatility, " ACS Nano, vol. 9, no. 1, pp. 941-949, 2015.
93. L. F. Abbott, J. A. Varela, K. Sen, and S. B. Nelson, "Synaptic depression and cortical gain control, " Science, vol. 275, no. 5297, pp. 221-224, 1997.
94. T. Chang, S.-H. Jo, and W. Lu, "Short-Term memory to long-Term memory transition in a nanoscale memristor, " ACS Nano, vol. 5, no. 9, pp. 7669-7676, 2011.
95. K. Cantley, A. Subramaniam, H. Stiegler, R. Chapman, and E. M. Vogel, "Hebbian learning in spiking neural networks with nanocrystalline silicon TFTs and memristive synapses, " IEEE Trans. Nanotechnol., vol. 10, no. 5, pp. 1066-1073, Sep. 2011.
96. J. Jeong et al., "Giant reversible, facet-dependent, structural changes in a correlated-electron insulator induced by ionic liquid gating, " in Proc. Nat. Acad. Sci., 2015, vol. 112, no. 4, pp. 1013-1018.
97. J. Shi, S. D. Ha, Y. Zhou, F. Schoofs, and S. Ramanathan, "A correlated nickelate synaptic transistor, " Nature, vol. 4, no. 2676, 2013.
98. J. Seo et al., "A 45 nm CMOS neuromorphic chip with a scalable architecture for learning in networks of spiking neurons, " in Proc. IEEE Custom Integr. Circuits Conf., 2011, pp. 1-4.