An approximate backpropagation learning rule for memristor based neural networks using synaptic plasticity

Neurocomputing

Volumn 237, Issue , 2017, Pages 193-199

  Negrov, D ^a   Karandashev, I ^a,b   Shakirov, V ^a,b   Matveyev, Yu ^a,c   Dunin Barkowski, W ^a,b   Zenkevich, A ^a,c  

^a MOSCOW INSTITUTE OF PHYSICS AND TECHNOLOGY   (Russian Federation)

^b SCIENTIFIC RESEARCH INSTITUTE FOR SYSTEM ANALYSIS   (Russian Federation)

^c NATIONAL RESEARCH NUCLEAR UNIVERSITY MEPHI   (Russian Federation)

Author keywords

Backpropagation algorithm; Deep learning; Hardware design; Memristor; Neural networks

Indexed keywords

APPROXIMATION ALGORITHMS; BACKPROPAGATION ALGORITHMS; MEMRISTORS; MULTILAYER NEURAL NETWORKS; NEURAL NETWORKS;

BACKPROPAGATION LEARNING; COMPUTATIONAL SIMULATION; DEEP LEARNING; DEEP NEURAL NETWORKS; HARDWARE DESIGN; MEMRISTOR; SYNAPTIC PLASTICITY; SYNAPTIC WEIGHT;

BACKPROPAGATION;

ARTICLE; BACK PROPAGATION; CONTROLLED STUDY; MEMRISTOR; NERVE CELL NETWORK; NERVE CELL PLASTICITY; NONLINEAR SYSTEM; SIGNAL TRANSDUCTION; SIMULATION;

EID: 85006762162     PISSN: 09252312     EISSN: 18728286     Source Type: Journal    
DOI: 10.1016/j.neucom.2016.10.061     Document Type: Article

References (25)

1. [1] LeCun, Y., Bengio, Y., Hinton, G., Deep learning. Nature 521 (2015), 436–444.
2. [2] S. Adhikari, H. Kim, R. Budhathoki, C. Yang, L. Chua, A circuit-based learning architecture for multilayer neural networks with memristor bridge synapses, in: IEEE Transactions on Circuits and Systems I: Regular Papers, vol. 62, 2015, pp. 215–223.
3. [3] C. Zhang, P. Li, G. Sun, Y. Guan, B. Xiao, J. Cong, Optimizing fpga-based accelerator design for deep convolutional neural networks, in: Proceedings of the 2015 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays, FPGA ‘15, ACM, New York, NY, USA, 2015, pp. 161–170. 〈 http://dx.doi.org/10.1145/2684746.2689060〉.
4. [4] Likharev, K., Crossnets: neuromorphic hybrid cmos/nanoelectronic networks. Sci. Adv. Mater., 3, 2011, 322.
5. x crossbar resistive ram with excellent performance, reliability and low-energy operation., in: IEEE International Electron Devices Meeting (IEDM), 2011, pp. 31–36.
6. [6] G. Indiveri, L.-B. B., T. Hamilton, A. van Schaik, R. Etienne-Cummings, T. Delbruck, S.-C. Liu, P. Dudek, P. Hafliger, S. Renaud, J. Schemme, G. Cauwenberghs, J. Arthur, K. Hynna, F. Folowosele, S. Saighi, S. Serrano-Gotarredona, J. Wijekoon, Y. Wang, K. Boahen, Neuromorphic silicon neuron circuits., Sci. Adv. Mater., vol. 5, 2011, pp. 1–23.
7. [7] Indiveri, G., Linares-Barranco, B., Legenstein, R., Deligeorgis, T., Prodromakis, G., Integration of nanoscale memristor synapses in neuromorphic computing architectures. Nanotechnology, 24, 2013, 384010.
8. 2/tin devices. J. Appl. Phys., 117, 2015, 044901.
9. [9] Wang, L., Li, H., Duan, S., Huang, T., Wang, H., Pavlov associative memory in a memristive neural network and its circuit implementation. Neurocomputing 171 (2016), 23–29.
10. [10] Hu, X., Feng, G., Duan, S., Liu, L., Multilayer rtd-memristor-based cellular neural networks for color image processing. Neurocomputing 162 (2015), 150–162.
11. [11] Chua, L., Everything you wish to know about memristors but are afraid to ask. Radioengineering 24 (2015), 319–368.
12. [12] C. Merkel, D. Kudithipudi, R. Ptucha, Heterogeneous cmos/memristor hardware neural networks for real-time target classification., in: Proceedings of the SPIE 9119, Machine Intelligence and Bio-inspired Computation: Theory and Applications, vol. VIII, 2014, p. 911908.
13. [13] P. Werbos, Beyond Regression New Tools for Prediction and Analysis in the Behavioral Sciences, 1975.
14. [14] C. Merkel, D. Kudithipudi, Comparison of off-chip training methods for neuromemristive systems., in: Proceedings of the 28th International Conference on VLSI Design (VLSID), 2015, pp. 99–104.
15. [15] Alibart, F., Zamanidoost, E., Strukov, D.B., Pattern classification by memristive crossbar circuits using ex situ and in situ training. Nat. Commun., 4, 2013, 2072.
16. [16] Widrow, B., Greenblatt, C., Kim, Y., Park, D., The no-prop algorithm: a new learning algorithm for multilayer neural networks. Neural Netw. 37 (2013), 182–188.
17. [17] K. Singh, C. Sahu, J. Singh, Linearly separable pattern classification using memristive crossbar circuits., in: Proceedings of the 15th International Symposium on Quality Electronic Design (ISQED), 2014, pp. 323–329.
18. [18] R. Hasan, T. Taha, Enabling back propagation training of memristor crossbar neuromorphic processors, in: Proceedings of the International Joint Conference on Neural Networks (IJCNN), 2014, pp. 21–28.
19. [19] Soudry, D., Castro, D., Gal, A., Kolodny, A., Kvatinsky, S., Memristor-based multilayer neural networks with online gradient descent training. IEEE Trans. Neural Netw. Learn. Syst. 26 (2015), 2408–2421.
20. [20] Jo, S., Chang, T., Ebong, I., Bhadviya, B., Mazumder, P., Lu, W., Nanoscale memristor device as synapse in neuromorphic systems. Nano Lett. 10 (2012), 1297–1301.
21. [21] T. Lillicrap, D. Cownden, D. Tweed, C. Akerman, Random feedback weights support learning in deep neural networks, arXiv, 2014. (1411.0247).
22. y oxide with graded al depth profile. Micro. Eng. 109 (2013), 342–345.
23. [23] Y. LeCun, L. Bottou, Y. Bengio, P. Haffner, Gradient-based learning applied to document recognition, in: Proceedings of the IEEE, vol. 86(11), 1998, pp. 2278–2324. 〈 http://dx.doi.org/10.1109/5.726791〉.
24. [24] U. Meier, D. Ciresan, L. Gambardella, J. Schmidhuber, Better digit recognition with a committee of simple neural nets, in: Proceedings of the 2011 International Conference on Document Analysis and Recognition (ICDAR), 2011, pp. 1250–1254. http://dx.doi.org/10.1109/ICDAR.2011.252〉.
25. [25] P. Nair, M.V.and Dudek, Gradient-descent-based learning in memristive crossbar arrays., in: Proceedings of the 2015 IEEE International Joint Conference on Neural Networks (IJCNN), 2015, pp. C. 1–7.