Energy scaling advantages of resistive memory crossbar based computation and its application to sparse coding

Frontiers in Neuroscience

Volumn 9, Issue JAN, 2016, Pages

  Agarwal, Sapan ^a   Quach, Tu Thach ^a   Parekh, Ojas ^a   Hsia, Alexander H ^a   DeBenedictis, Erik P ^a   James, Conrad D ^a   Marinella, Matthew J ^a   Aimone, James B ^a  

^a SANDIA NATIONAL LABORATORIES   (United States)

Author keywords

Energy; Memristor; Neuromorphic computing; Resistive memory; Sparse coding

Indexed keywords

BIOLOGY; CODING ALGORITHM; LEARNING; MEMORY; NERVOUS SYSTEM; NOISE; SPEECH DISCRIMINATION; VISION;

EID: 84958093736     PISSN: 16624548     EISSN: 1662453X     Source Type: Journal    
DOI: 10.3389/fnins.2015.00484     Document Type: Article

References (36)

1. Arora, S., Ge, R., Ma, T., and Moitra, A. (2015). Simple, efficient, and neural algorithms for sparse coding. arXiv preprint arXiv:1503.00778.
2. Burr, G. W., Shelby, R. M., Sidler, S., Di Nolfo, C., Jang, J., Boybat, I., et al. (2015). Experimental demonstration and tolerancing of a large-scale neural network (165 000 Synapses) using phase-change memory as the synaptic weight element. IEEE Trans. Electron Dev. 62, 3498-3507. doi: 10.1109/TED.2015.2439635
3. Cassidy, A. S., Alvarez-Icaza, R., Akopyan, F., Sawada, J., Arthur, J. V., Merolla, P. A., et al. (2014). "Real-time scalable cortical computing at 46 giga-synaptic OPS/watt with ~100 × Speedup in Time-to-Solution and ~100,000 × reduction in energy-to-solution," in Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (New Orleans, LA: IEEE Press), 27-38.
4. Chen, A. (2013). A comprehensive crossbar array model with solutions for line resistance and nonlinear device characteristics. IEEE Trans. Electron Dev. 60, 1318-1326. doi: 10.1109/TED.2013.2246791
5. Chen, H.-Y., Gao, B., Li, H., Liu, R., Huang, P., Chen, Z., et al. (2014). "Towards high-speed, write-disturb tolerant 3D vertical RRAM arrays," in Digest of Technical Papers, 2014 Symposium on: IEEE VLSI Technology (VLSI-Technology) (Honolulu, HI), 1-2.
6. Chen, P.-Y., Kadetotad, D., Xu, Z., Mohanty, A., Lin, B., Ye, J., et al. (2015). "Technology-design co-optimization of resistive cross-point array for accelerating learning algorithms on chip," in IEEE Design, Automation and Test in Europe (DATE) 2015 (Grenoble), 854-859.
7. Cheng, C. H., Tsai, C. Y., Chin, A., and Yeh, F. S. (2010). "High performance ultra-low energy RRAM with good retention and endurance," in Electron Devices Meeting (IEDM), 2010 IEEE International (San Francisco, CA), 19.14.11-19.14.14.
8. Chua, L. O. (1971). Memristor-The missing circuit element. IEEE Trans. Circuit Theory 18, 507-519. doi: 10.1109/TCT.1971.1083337
9. Cong, X., Dimin, N., Muralimanohar, N., Balasubramonian, R., Tao, Z., Shimeng, Y., et al. (2015). "Overcoming the challenges of crossbar resistive memory architectures," in 2015 IEEE 21st International Symposium on High Performance Computer Architecture (HPCA) (Burlingame, CA), 476-488.
10. Enz, C. C., and Vittoz, E. (1996). "CMOS low-power analog circuit design," in Designing Low Power Digital Systems, Emerging Technologies (1996) (Atlanta, GA: IEEE), 79-133.
11. Fürer, M. (2009). Faster Integer Multiplication. SIAM J. Comput. 39, 979-1005. doi: 10.1137/070711761
12. Gokhale, M., Holmes, B., and Iobst, K. (1995). Processing in memory: the Terasys massively parallel PIM array. Computer 28, 23-31. doi: 10.1109/2.375174
13. Hasan, R., and Taha, T. M. (2014). "Enabling back propagation training of memristor crossbar neuromorphic processors," in Neural Networks (IJCNN), 2014 International Joint Conference on (Beijing), 21-28.
14. International Technology Roadmap for Semiconductors (ITRS) (2013). Edition Available online at: http://www.itrs.net [Accessed].
15. Jo, S. H., Chang, T., Ebong, I., Bhadviya, B. B., Mazumder, P., and Lu, W. (2010). Nanoscale memristor device as synapse in neuromorphic systems. Nano Lett. 10, 1297-1301. doi: 10.1021/nl904092h
16. Jo, S. H., Kim, K.-H., and Lu, W. (2009). High-density crossbar arrays based on a si memristive system. Nano Lett. 9, 870-874. doi: 10.1021/nl8037689
17. Cruz-Albrecht, J. M., Derosier, T., and Srinivasa, N. (2013). A scalable neural chip with synaptic electronics using CMOS integrated memristors. Nanotechnology 24:384011. doi: 10.1088/0957-4484/24/38/384011
18. Kadetotad, D., Zihan, X., Mohanty, A., Pai-Yu, C., Binbin, L., Jieping, Y., et al. (2015). Parallel Architecture with resistive crosspoint array for dictionary learning acceleration. IEEE J. Emerg. Sel. Top. Circuits Syst. 5, 194-204. doi: 10.1109/JETCAS.2015.2426495
19. Kim, K.-H., Gaba, S., Wheeler, D., Cruz-Albrecht, J. M., Hussain, T., Srinivasa, N., et al. (2012). A functional hybrid memristor crossbar-array/cmos system for data storage and neuromorphic applications. Nano Lett. 12, 389-395. doi: 10.1021/nl203687n
20. Kim, Y., Zhang, Y., and Li, P. (2015). A reconfigurable digital neuromorphic processor with memristive synaptic crossbar for cognitive computing. ACM J. Emerg. Technol. Comput. Syst. 11, 38. doi: 10.1145/2700234
21. Laughlin, S. B., de Ruyter van Steveninck, R. R., and Anderson, J. C. (1998). The metabolic cost of neural information. Nat. Neurosci. 1, 36-41. doi: 10.1038/236
22. Lee, H., Ekanadham, C., and Ng, A. Y. (2008). "Sparse deep belief net model for visual area V2," in Advances in Neural Information Processing Systems (Vancouver, BC), 873-880.
23. Mahalanabis, D., Barnaby, H. J., Gonzalez-Velo, Y., Kozicki, M. N., Vrudhula, S., and Dandamudi, P. (2014). Incremental resistance programming of programmable metallization cells for use as electronic synapses. Solid State Electron. 100, 39-44. doi: 10.1016/j.sse.2014.07.002
24. Merkle, R. C. (1989). Energy limits to the computational power of the human brain. Foresight Update 6.
25. Merolla, P. A., Arthur, J. V., Alvarez-Icaza, R., Cassidy, A. S., Sawada, J., Akopyan, F., et al. (2014). A million spiking-neuron integrated circuit with a scalable communication network and interface. Science 345, 668-673. doi: 10.1126/science.1254642
26. Miller, D. A. B. (2009). Device requirements for optical interconnects to silicon chips. Proc. IEEE 97, 1166-1185. doi: 10.1109/JPROC.2009.2014298
27. Murmann, B., and Boser, B. E. (2007). Digitally Assisted Pipeline ADCs: Theory and Implementation. Boston, MA: Springer Science & Business Media.
28. Olshausen, B. A. (1996). Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381, 607-609. doi: 10.1038/381607a0
29. Olshausen, B. A., and Field, D. J. (1997). Sparse coding with an overcomplete basis set: A strategy employed by V1? Vision Res. 37, 3311-3325. doi: 10.1016/S0042-6989(97)00169-7
30. Schüz, A., and Palm, G. (1989). Density of neurons and synapses in the cerebral cortex of the mouse. J. Comp. Neurol. 286, 442-455. doi: 10.1002/cne.902860404
31. Sharad, M., Deliang, F., Aitken, K., and Roy, K. (2014). Energy-efficient non-boolean computing with spin neurons and resistive memory. Nanotechnol. IEEE Trans. 13, 23-34. doi: 10.1109/TNANO.2013.2286424
32. Strukov, D. B., Snider, G. S., Stewart, D. R., and Williams, R. S. (2008). The missing memristor found. Nature 453, 80-83. doi: 10.1038/nature06932
33. Tai, H. Y., Hu, Y.-S., Chen, H.-W., and Chen, H.-S. (2014). "A 0.85fJ/conversion-step 10b 200kS/s subranging SAR ADC in 40nm CMOS," IEEE International Solid-State Circuits Conference (San Francisco, CA), 196-197.
34. Theis, T. N., and Solomon, P. M. (2010). In quest of the next switch: prospects for greatly reduced power dissipation in a successor to the silicon field-effect transistor. Proc. IEEE 98, 2005-2014. doi: 10.1109/JPROC.2010.2066531
35. Ting, C., Yuchao, Y., and Wei, L. (2013). Building neuromorphic circuits with memristive devices. IEEE Circuits Syst. Mag. 13, 56-73. doi: 10.1109/MCAS.2013.2256260
36. Waser, R., and Aono, M. (2007). Nanoionics-based resistive switching memories. Nat. Mater. 6, 833-840. doi: 10.1038/nmat2023