Neuromorphic architectures with electronic synapses

Proceedings - International Symposium on Quality Electronic Design, ISQED

Volumn 2016-May, Issue , 2016, Pages 118-123

  Eryilmaz, Sukru Burc ^a   Joshi, Siddharth ^b   Neftci, Emre ^c   Wan, Weier ^a   Cauwenberghs, Gert ^b   Wong, H S Philip ^a  

^a Stanford University   (United States)

^b UNIVERSITY OF CALIFORNIA   (United States)

^c UNIVERSITY OF CALIFORNIA   (United States)

Author keywords

Cognitive computing; device variability; monolithic integration; neuromorphic hardware; nonvolatile memory; phase change memory; resistive switching memory

Indexed keywords

ALGORITHMS; DATA STORAGE EQUIPMENT; DIGITAL STORAGE; ENERGY EFFICIENCY; LEARNING SYSTEMS; MONOLITHIC INTEGRATED CIRCUITS; NEURAL NETWORKS; NONVOLATILE STORAGE; PHASE CHANGE MEMORY;

COGNITIVE COMPUTING; DEVICE VARIABILITY; MONOLITHIC INTEGRATION; NEUROMORPHIC HARDWARES; NON-VOLATILE MEMORY; RESISTIVE SWITCHING MEMORY;

LEARNING ALGORITHMS;

EID: 84973869304     PISSN: 19483287     EISSN: 19483295     Source Type: Conference Proceeding    
DOI: 10.1109/ISQED.2016.7479186     Document Type: Conference Paper

References (68)

1. G. Indiveri et al., "Neuromorphic Silicon Neuron Circuits," Front. Neurosci., vol. 5, 2011.
2. C. Ramos et al., "On spike-timing-dependent-plasticity, memristive devices, and building a self-learning visual cortex," Front. Neurosci., vol. 5, 2011.
3. D. George et al., "Towards a mathematical theory of cortical micro-circuits," PLoS Comput. Biol., vol. 5, no. 10, 2009.
4. Y. LeCun et al., "Deep learning," Nature, vol. 521, no. 7553, pp. 436-444, 2015.
5. G. Hinton, S. Osindero and Y.-W. Teh, "A fast learning algorithm for deep belief nets," Neural Comp., vol. 18, no. 7, pp. 1527-1554, 2006.
6. E. Neftci et al. "Unsupervised Learning in Synaptic Sampling Machines, " arXiv preprint arXiv:1511.04484, 2015.
7. G. E. Hinton and R.R. Salakhutdinov, "Reducing the dimensionality of data with neural networks," Science, vol. 313, no. 5786, pp. 504-507, 2006.
8. Y. Bengio, "Learning deep architectures for AI," Found. Trends. Mach. Learn., vol. 2, pp. 1-127.
9. P. Merolla, T. Ursell, and J. Arthur, "The thermodynamic temperature of a rhythmic spiking network," ArXiv e-prints, September 2010.
10. J. Fiser, P. Berkes, G. Orbán, and M. Lengyel, "Statistically optimal perception and learning: from behavior to neural representations," Trends in Cognitive Sciences, vol. 14, no. 3, pp. 119-130, 2010.
11. S. Esser et al., "Backpropagation for energy-efficient neuromorphic computing", NIPS, Montreal, 2015.
12. T. Yu et al., "65k-neuron integrate-and-fire array transceiver with address-event reconfigurable synaptic routing," BioCAS, IEEE, pp.21-24, 28-30 Nov. 2012.
13. J. Park et al., "A 65k-neuron 73-mevents/s 22-pj/event asynchronous micro-pipelined integrate-and-fire array transceiver," BioCAS, IEEE, Oct. 2014.
14. P. A. Merolla et al., "A million spiking-neuron integrated circuit with a scalable communication network and interface," Science, vol. 345, no. 6197, pp. 668-673, 2014.
15. S. Sheik, M. Pfeiffer, F. Stefanini, G. Indiveri, "Spatiotemporal spike pattern classification in neuromorphic systems," Biomimetic and Biohybrid Systems, Springer Berlin Heidelberg, pp. 262-273, 2013.
16. D. Koller, and N. Friedman, "Probabilistic Graphical Models: Principles and Techniques," MIT press, 2009.
17. K. Doya, S. Ishii, A. Pouget, and R. Rao, "Bayesian Brain: Probabilistic Approaches to Neural Coding," Cambridge, MA, MIT Press, 2007.
18. P. Berkes, G. Orbán, M. Lengyel, J Fiser, "Spontaneous cortical activity reveals hallmarks of an optimal internal model of the environment," Science, vol. 331, no. 6013, pp. 83-87, 2011.
19. C. Eliasmith, and C. S. Terrence, "Nengo and the Neural Engineering Framework: From Spikes to Cognition." Cognitive Science Society, 2012.
20. L. Buesing, J. Bill, B. Nessler, and W. Maass, "Neural dynamics as sampling: A model for stochastic computation in recurrent networks of spiking neurons," PLoS Comput. Biol., vol. 7, no. 11, 2011.
21. B. U. Pedroni et al., "Neuromorphic adaptations of restricted boltzmann machines and deep belief networks," Neural Networks (IJCNN), The 2013 International Joint Conference on, IEEE, pp. 1-6, 2013.
22. P. O'Connor et al., "Real-time classification and sensor fusion with a spiking deep belief network," Front. Neurosci., vol. 7, 2013.
23. L. Wan et al., "Regularization of neural networks using dropconnect," ICML, 2013.
24. E. Neftci et al., "Event-driven contrastive divergence for spiking neuromorphic systems," Front. Neurosci., vol. 7, 2013.
25. L. Aitchison, and P. E. Latham, "Synaptic sampling: A connection between PSP variability and uncertainty explains neurophysiological observations," arXiv preprint arXiv:1505.04544, 2015.
26. D. Kappel, S. Habenschuss, R. Legenstein, and W. Maass, "Synaptic sampling: A Bayesian approach to neural network plasticity and rewiring," NIPS, 2015.
27. R. Ananthanarayanan et al., "The cat is out of the bag: cortical simulations with 109 neurons, 1013 synapses," Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis. New York, NY, USA: ACM, 2009.
28. Q. V. Le et al., "Building high-level features using large scale unsupervised learning," ICML, 2012.
29. A. Krizhevsky et al., "Imagenet classification with deep convolutional neural networks," NIPS, 2012.
30. R. Raina, A. Madhavan, and A. Y. Ng, "Large-scale deep unsupervised learning using graphics processors," ICML, 2009.
31. V. Vanhoucke et al., "Improving the speed of neural networks on CPUs," Deep Learning and Unsupervised Feature Learning NIPS Workshop, 2010.
32. S. B. Eryilmaz, et al., "Device and system level design considerations for analog-non-volatile-memory based neuromorphic architectures," Electron Devices Meeting, 2015. IEDM'15 Technical Digest. IEEE International, pp. 4.1.1-4, 2015.
33. http://users.ece.gatech.edu/mrichard/ExascaleComputing StudyReports/exascale-final-report-100208.pdf
34. C. Mead, "Neuromorphic electronic systems," Proc. IEEE, vol. 78, no. 10, pp. 1629-1636, 1990.
35. B. V. Benjamin et al., "Neurogrid: A mixed-analogdigital multichip system for large-scale neural simulations," Proc. IEEE, vol. 102, no. 5, pp. 699-716, 2014.
36. R. J. Vogelstein et al., "Dynamically reconfigurable silicon array of spiking neurons with conductance-based synapses," IEEE Trans. Neural Netw., vol. 18, no. 1, pp. 253-265, 2007.
37. G. Indiveri et al., "A VLSI array of low-power spiking neurons and bistable synapses with spike-timing dependent plasticity," IEEE Trans. Neural Netw., vol. 17, no. 1, pp. 211-221, 2006.
38. A.S. Cassidy et al., "Design of silicon brains in the nano-CMOS era: Spiking neurons, learning synapses and neural architecture optimization," Neural Networks, vol. 45, pp. 4-26, 2013.
39. J. Gehlhaar, "Neuromorphic processing: A new frontier in scaling computer architecture," ASPLOS, 2014.
40. M.M. Khan et al., "SpiNNaker: mapping neural networks onto a massively-parallel chip multiprocessor," IJCNN, 2008.
41. J. Seo et al., "A 45nm CMOS neuromorphic chip with a scalable architecture for learning in networks of spiking neurons," CICC, 2011.
42. K. Boahen, "Point-to-point connectivity between neuromorphic chips using address events," Circuits and Systems II: Analog and Digital Signal Processing, IEEE Transactions on, vol 47, no. 5, pp. 416-434, 2000.
43. S. Joshi et al., "Scalable event routing in hierarchical neural array architecture with global synaptic connectivity," Cellular Nanoscale Networks and Their Applications (CNNA), 12th International Workshop on., IEEE, 2010.
44. E. A. Carara, G. M. Fernando, "Deadlock-free multicast routing algorithm for wormhole-switched mesh networks-onchip," Symposium on VLSI, 2008, IEEE Computer Society Annual, 2008.
45. S. Yu et al., "An electronic synapse device based on metal oxide resistive switching memory for neuromorphic computation," IEEE Trans. Electron Devices, vol. 58, no. 8, pp. 2729-2737, 2011.
46. D. Kuzum et al., "Nanoelectronic programmable synapses based on phase change materials for brain-inspired computing," Nano Letters, vol. 12, no. 5, pp. 2179-2186, 2011.
47. T. Ohno et al., "Short-term plasticity and long-term potentiation mimicked in single inorganic synapses," Nature Mat., vol. 10, no. 8, pp. 591-595, 2011.
48. A. Chanthbouala et al.," A ferroelectric memristor," Nature Mat., vol. 11, no. 10, pp. 860-864, 2012.
49. D. Kuzum et al., "Synaptic electronics: materials, devices and applications," Nanotechnology, vol. 24, no. 38, pp. 382001, 2013.
50. S. Yu et al., "A Low Energy Oxide-Based Electronic Synaptic Device for Neuromorphic Visual Systems with Tolerance to Device Variation," Adv. Mat., vol. 25, no. 12, pp. 1774-1779, 2013.
51. M. M. Shulaker et al., "'Monolithic 3D integration of carbon nanotube FETs, resistive RAM, silicon FETs," IEDM, 2014.
52. Y. Liauw et al., "Nonvolatile 3D-FPGA with monolithically stacked RRAM-based configuration memory," Solid-State Circuits Conference Digest of Technical Papers (ISSCC), 2012.
53. S. B. Eryilmaz et al., "Experimental demonstration of array-level learning with phase change synaptic devices," IEDM, 2013.
54. S. Park et al., "Electronic system with memristive synapses for pattern recognition," Scientific Reports, vol. 5, 2015.
55. G. Burr et al., "Experimental demonstration and tolerancing of a large-scale neural network (165, 000 synapses), using phase-change memory as the synaptic weight element," IEDM, pp. 29-5, 2014.
56. M. Prezioso et al., "Training and operation of an integrated neuromorphic network based on metal-oxide memristors," Nature, vol. 521, no. 7550, pp. 61-64, 2015.
57. P. Y. Chen et al., "Technology-design co-optimization of resistive cross-point array for accelerating learning algorithms on chip," DATE, pp. 854-859, 2015.
58. D. Kuzum, et al. "Energy efficient programming of nanoelectronic synaptic devices for large-scale implementation of associative and temporal sequence learning," IEDM, 2011.
59. O. Bichler et al., "Visual Pattern Extraction Using Energy-Efficient "2-PCM Synapse" Neuromorphic Architecture," IEEE Trans. Electron Devices, vol. 59, no. 8, pp. 2206-2214, 2012.
60. S. Yu et al., "Stochastic learning in oxide binary synaptic device for neuromorphic computing," Front. Neurosci., vol. 7, 2013.
61. M. Suri et al.," CBRAM devices as binary synapses for low-power stochastic neuromorphic systems: auditory (cochlea) and visual (retina) cognitive processing applications," IEDM, 2012.
62. Z. Jiang, S. Yu, Y. Wu, J. H. Engel, X. Guan, H.-S. P. Wong, "Verilog-A Compact Model for Oxide-based Resistive Random Access Memory (RRAM)," International Conference on Simulation of Semiconductor Processes and Devices (SISPAD), paper 3-3, pp. 41-44, Yokohama, Japan, September 9-11, 2014. Model available at: https://nanohub.org/publications/19
63. D. Querlioz et al., Simulation of a memristor-based spiking neural network immune to device variations," IJCNN, 2011.
64. G. E. Hinton, "A practical guide to training restricted Boltzmann machines," Tech. Rep. UTML TR 2010-003, Dept. Comput. Sci., Univ. Toronto, 2010.
65. A. Chen et al., "Comprehensive assessment of RRAMbased PUF for hardware security applications," IEDM, pp. 10.7.1-4, 2015.
66. S. B. Eryilmaz et al., "Brain-like associative learning using a nanoscale non-volatile phase change synaptic device array," Front. Neurosci., vol. 8, 2014.
67. S. Yu et al., "Scaling-up resistive synaptic arrays for neuro-inspired architecture: Challenges and prospect," IEDM, pp. 17.3.1-4, 2015.
68. https://nano.stanford.edu/stanford-memory-trends