Efficient training algorithms for neural networks based on memristive crossbar circuits

Proceedings of the International Joint Conference on Neural Networks

Volumn 2015-September, Issue , 2015, Pages

  Kataeva, Irina ^a   Merrikh Bayat, Farnood ^b   Zamanidoost, Elham ^b   Strukov, Dmitri ^b  

^a DENSO CORPORATION   (Japan)

^b University of California   (United States)

Author keywords

CMOS integrated circuits; Magnetic multilayers; Magnetic resonance imaging; Metals; Nonhomogeneous media; Programming; Switches

Indexed keywords

ALGORITHMS; BACKPROPAGATION ALGORITHMS; CMOS INTEGRATED CIRCUITS; MAGNETIC LEVITATION VEHICLES; MAGNETIC MULTILAYERS; MAGNETIC RESONANCE IMAGING; MATHEMATICAL PROGRAMMING; MEMRISTORS; METALS; MULTILAYERS; NETWORKS (CIRCUITS); STOCHASTIC SYSTEMS; SWITCHES;

CROSSBAR CIRCUITS; MISCLASSIFICATION RATES; MULTI-LAYER PERCEPTRON CLASSIFIERS; NONHOMOGENEOUS MEDIA; STOCHASTIC ALGORITHMS; SWITCHING DYNAMICS; TRAINING ALGORITHMS; WEIGHT UPDATE;

MULTILAYER NEURAL NETWORKS;

EID: 84951018554     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IJCNN.2015.7280785     Document Type: Conference Paper

References (39)

1. C. Mead, Analog VLSI and Neural Systems. Boston, MA, USA: Addison-Wesley Longman Publishing Co., Inc., 1989.
2. K. K. Likharev, "Crossnets: Neuromorphic hybrid cmos/nanoelectronic networks, " Sci. Adv. Mater., vol. 3, pp. 322-331, 2011.
3. S. K. Kim, L. C. McAfee, P. L. McMahon, and K. Olukotun, "A highly scalable restricted boltzmann machine fpga implementation, " in Field Programmable Logic and Applications, 2009. FPL 2009. International Coriference on, pp. 367-372, IEEE, 2009.
4. J. Kim, M. Kim, S. Lee, J. Oh, K. Kim, and H. Yoo, "A 201. 4 GOPS 496 mw real-time multi-object recognition processor with bio-inspired neural perception engine, " 1. Solid-State Circuits, vol. 45, no. I, pp. 32-45, 2010.
5. c. Farabet, B. Martini, B. Corda, P. Akselrod, E. Culurciello, and Y. LeCun, "Neuflow: A runtime reconfigurable dataflow processor for vision, " in Proceedings of Computer Vision and Pattern Recognition Workshops (CVPRW'11), pp. 109-116, June 2011.
6. S. Chakradhar, M. Sankaradas, Y. Jakkula, and S. Cadambi, "A dynamically configurable coprocessor for convolutional neural networks, " SIGARCH Comput. Archit. News, vol. 38, pp. 247-257, June 2010.
7. B. Y. Benjamin, P. Gao, E. McQuinn, S. Choudhary, A. Chandrasekaran, J.-M. Bussat, R. Alvarez-Icaza, J. Y. Arthur, P. Merolla, and K. Boahen, "Neurogrid: A mixed-analog-digital multichip system for large-scale neural simulations, " Proceedings of the IEEE, pp. 699-716, 2014.
8. P. Merolla, J. Arthur, F. Akopyan, N. Imam, R. Manohar, and D. S. Modha, "A digital neurosynaptic core using embedded crossbar memory with 45pj per spike in 45nm, " in Custom Integrated Circuits Conference (CICC), 2011 IEEE, pp. 1-4, IEEE, 2011.
9. S. Ramakrishnan and 1. Hasler, "Vector-matrix multiply and winnertake-all as an analog classifier, " IEEE Trans. Very Large Scale Integr. Syst., vol. 22, pp. 353-361, Feb. 2014.
10. J. Lu, S. Young, I. Arel, and J. Holleman, "A ltops/w analog deep machine-learning engine with floating-gate storage in 0. 13 /-lm cmos, " in IEEE Int. Solid-State Circuits Conf (ISSCC) Dig. Tech. Papers, 2014.
11. T he International Technology Roadmap for Semiconductors (ITRS). 2013 ed. Available online at http://www. itrs. netl.
12. D. Strukov, "Nanotechnology: Smart connections, " Nature, vol. 476, 2011.
13. R. Williams, "How we found the missing memristor, " IEEE Spectr., vol. 45, pp. 28-35, Dec. 2008.
14. D. Strukov and H. Kohlstedt, "Resistive switching phenomena in thin films: Materials, devices, and applications, " MRS Bulletin, vol. 37, February 2012.
15. D. Kuzum, R. G. D. Jeyasingh, B. Lee, and H.-S. P. Wong, "Nanoelectronic programmable synapses based on phase change materials for brain-inspired computing, " Nano Letters, vol. 12, no. 5, pp. 2179-2186, 2012.
16. N. Locatelli, V. Cros, and J. Grollier, "Spin-torque building blocks, " Nature materials, vol. 13, no. 1, pp. 11-20, 2014.
17. Y. Kaneko, Y. Nishitani, and M. Ueda, "Ferroelectric artificial synapses for recognition of a multi shaded image, " 2014.
18. F. Alibart, E. Zamanidoost, and D. Strukov, "Pattern classification by memristive crossbar circuits with ex-situ and in-situ training, " Nature Communications, vol. 4, p. 2072, 2013.
19. S. H. Jo, T. Chang, I. Ebong, B. B. Bhadviya, P. Mazumder, and W. Lu, "Nanoscale mernristor device as synapse in neuromorphic systems, " Nano Letters, vol. 10, no. 4, pp. 1297-1301, 2010.
20. T. Ohno, T. Hasegawa, T. Tsuruoka, K. Terabe, 1. K. Gimzewski, and M. Aono, "Short-term plasticity and long-term potentiation mimicked in single inorganic synapses, " Nature materials, vol. 10, no. 8, pp. 591-595, 2011.
21. J. Yang, D. Strukov, and D. Stewart, "Mernristive devices for computing, " Nature Nanotechnology, vol. 8, pp. 13-24, 2013.
22. M. Prezioso, F. Merrikh-Bayat, B. Hoskins, G. Adam, K. K. Likharev, and D. B. Strukov, "Training and operation of an integrated neuromorphic network based on metal-oxide mernristors, " Nature, May 2015 (in print).
23. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition, " Proceedings of the IEEE, vol. 86, pp. 2278-2324, November 1998.
24. Y. LeCun, L. Bottou, G. Orr, and K. Muller, "Efficient backprop, " in Neural Networks: Tricks of the trade (G. Orr and M. K., eds.), Springer, 1998.
25. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks, " in Advances in neural information processing systems, pp. 1097-1105, 2012.
26. J. H. Lee and K. K. Likharev, "Defect-tolerant nanoelectronic pattern classifiers, " International Journal of Circuit Theory and Applications, vol. 35, no. 3, pp. 239-264, 2007.
27. D. Chabi, D. Querlioz, W. Zhao, and J. Klein, "Robust learning approach for neuro-inspired nanoscale crossbar architecture, " JETC, vol. 10, no. 1, p. 5, 2014.
28. c. Yakopcic and T. M. Taha, "Energy efficient perceptron pattern recognition using segmented memristor crossbar arrays, " in Neural Networks (]JCNN), T he 2013 International Joint Coriference on, pp. 1-8, IEEE, 2013.
29. D. Querlioz, O. Bichler, and C. Garnrat, "Simulation of a mernristorbased spiking neural network immune to device variations, " in Neural Networks (]JCNN), T he 2011 International Joint Conference on, pp. 1775-1781, IEEE, 2011.
30. M. Hu, H. Li, Y. Chen, Q. Wu, G. S. Rose, and R. W. Linderman, "Mernristor crossbar-based neuromorphic computing system: A case study, " IEEE Trans. Neural Netw. Learning Syst., vol. 25, no. 10, pp. 1864-1878, 2014.
31. J. Starzyk and Basawaraj, "Mernristor crossbar architecture for synchronous neural networks, " IEEE Trans. Circ. Syst.-I, vol. 61, pp. 2390-2401, 2014.
32. P. Sheridan, W. Ma, and W. Lu, "Pattern recognition with mernristor networks, " in Circuits and Systems (ISCAS), 2014 IEEE International Symposium on, pp. 1078-1081, IEEE, 2014.
33. Y. Kim, Y. Zhang, and P. Li, "A digital neuromorphic vlsi architecture with mernristor crossbar synaptic array for machine learning, " in SOC Conference (SOCC), 2012 IEEE International, pp. 328-333, IEEE, 2012.
34. F. Merrikh-Bayat, B. Hoskins, and D. B. Strukov, "Phenomenological modeling of memristive devices, " available online at https:llwww. ece. ucsb. edu/strukov Ipapers120 14/mmodel20 14. pdf.
35. F. Merrikh Bayat, B. Hoskins, and D. Strukov, "Phenomenological modeling of mernristive devices, " Applied Physics A, pp. 1-8, 2015.
36. J. Hertz, A. Krogh, and R. Palmer, "Introduction to the theory of neural networks, " Lecture Notes, vol. 1, 1991.
37. F. Alibart, L. Gao, B. Hoskins, and D. Strukov, "High precision tuning of state for mernristive devices by adaptable variation-tolerant algorithm, " Nanotechnology, vol. 23, p. 075201, 2012.
38. K. K. Likharev, "Hybrid cmos/nanoelectronic circuits: Opportunities and challenges, " Journal of Nanoelectronics and Optoelectronics, vol. 3, no. 3, pp. 203-230, 2008.
39. W. Schiffmann, M. Joost, and R. Werner, "Optimization of the backpropagation algorithm for training multilayer perceptrons, " Univ. Koblenz, Inst. Physics, Rheinau, pp. 3-4, 1994.