Parallel architecture with resistive crosspoint array for dictionary learning acceleration

IEEE Journal on Emerging and Selected Topics in Circuits and Systems

Volumn 5, Issue 2, 2015, Pages 194-204

  Kadetotad, Deepak ^a   Xu, Zihan ^a   Mohanty, Abinash ^a   Chen, Pai Yu ^a   Lin, Binbin ^a   Ye, Jieping ^a   Vrudhula, Sarma ^a   Yu, Shimeng ^a   Cao, Yu ^a   Seo, Jae Sun ^a  

^a Arizona State University   (United States)

Author keywords

CMOS integration; dictionary learning; memristive device; parallel computing; resistive crosspoint array

Indexed keywords

EID: 85027956317     PISSN: 21563357     EISSN: None     Source Type: Journal    
DOI: 10.1109/JETCAS.2015.2426495     Document Type: Article

References (33)

1. G. S. Song, K. D. Miller, and L. F. Abbott, "Competitive Hebbian learning through spike-timing-dependent synaptic plasticity, " Nature Neurosci., pp. 919-926, 2000.
2. G. Bi and M. Poo, "Synaptic modifications in cultured Hippocampal neurons: Dependence on spike timing, synaptic strength, postsynaptic cell type, " J. Neurosci., vol. 18, no. 24, pp. 10464-10472, Dec. 1998.
3. B. A. Olshausen and D. J. Field, "Emergence of simple-cell receptive field properties by learning a sparse code for natural images, " Lett. Nature, vol. 381, no. 13, pp. 607-609, Jun. 1996.
4. I. Tosic and P. Frossard, "Dictionary learning, " IEEE Signal Process. Mag., vol. 28, no. 2, pp. 27-38, Mar. 2011.
5. P. O. Hoyer, "Non-negative sparse coding, " in Proc. 12th IEEE Workshop Neural Netw. Signal Process., 2002, pp. 557-565.
6. I. Daubechies, M. Defrise, and C. De Mol, "An iterative thresholding algorithm for linear inverse problems with a sparsity constraint, " Commun. Pure Appl. Math., vol. 57, no. 11, pp. 1413-1457, 2004.
7. L. Bottou and O. Bousquet, "The tradeoffs of large-scale learning, " Opt. Mach. Learn., p. 351, 2011.
8. J. K. Kim, P. Knag, T. Chen, and Z. Zhang, "A 6.67 mW sparse coding ASIC enabling on-chip learning and inference, " in Symp. VLSI Circuits Dig. Tech. Papers, 2014, pp. 1-2.
9. J. K. Kim, P. Knag, T. Chen, and Z. Zhang, "Efficient hardware architecture for sparse coding, " IEEE Trans. Signal Process., vol. 62, no. 16, pp. 4173-4186, Aug. 2014.
10. J. Zylberberg, J. T. Murphy, and M. R. DeWeese, "A sparse coding model with synaptically local plasticity and spiking neurons can account for the diverse shapes of V1 simple cell receptive fields, " PLoS Comput. Biol., vol. 7, no. 10, p. e1002250, 2011.
11. A. Afifi, A. Ayatollahi, F. Raissi, and H. Hajghassem, "Efficient hybrid CMOS-Nano circuit design for spiking neurons and memristive synapses with STDP, " IEICE Trans. Fund., vol. E93-A, no. 9, pp. 1670-1677, Sep. 2010.
12. B. Rajendran et al., "Specifications of nanoscale devices and circuits for neuromorphic computational systems, " IEEE Trans. Electron Devices, vol. 60, no. 1, pp. 246-253, Jan. 2013.
13. S. H. Jo et al., "Nanoscale memristor device as synapse in neuromor-phic systems, " Nano Lett., vol. 10, no. 4, pp. 1297-1301, 2010.
14. D. Kuzum, S. Yu, H.-S., and P. Wong, "Synaptic electronics: Materials, devices and applications, " Nanotechnology, vol. 24, no. 38, 2013.
15. S. Yu et al., "A low energy oxide-based electronic synaptic device for neuromorphic visual system with tolerance to device variation, " Adv. Mater., vol. 25, no. 12, pp. 1774-1779, 2013.
16. S. Park et al., "Neuromorphic speech systems using advanced ReRAM-based synapse, " in Proc. IEEE Int. Electron Devices Meet., 2013, pp. 625-628.
17. I.-T. Wang, Y.-C. Lin, Y.-F. Wang, C.-W. Hsu, and T.-H. Hou, "3D synaptic architecture with ultralow sub-10 fJ energy per spike for neu-romorphic computation, " in Proc. IEEE Int. Electron Devices Meet., 2014, pp. 28.5.1-28.5.4.
18. J. Liang, S. Yeh, S. S. Wong, and H.-S. P. Wong, "Effect of wordline/bitline scaling on the performance, energy consumption, reliability of cross-point memory array, " ACM J. Emerg. Technol. Comput. Syst., vol. 9, no. 1, p. 9, 2013.
19. D. Kadetotad et al., "Neurophysics-inspired parallel architecture with resistive crosspoint array for dictionary learning, " in Proc. IEEE Biomed. Circuits Syst. Conf., 2014, pp. 536-539.
20. L. F. Abbott, "Lapique's introduction of the integrate-and-fire model neuron (1907), " Brain Res. Bull., vol. 50, no. 5/6, pp. 303-304, 1999.
21. C. Mead, Analog VLSI and Neural Systems. Boston, MA: Addison Wesley, 1989.
22. H.-S. P. Wong et al., "Metal oxide RRAM, " Proc. IEEE, vol. 100, no. 6, pp. 1951-1970, 2012.
23. Z. Wang, "CMOS adjustable Schmitt triggers, " IEEE Trans. Instrum. Meas., vol. 40, no. 3, pp. 601-605, Jun. 1991.
24. V. Katyal, R. L. Geiger, and D. J. Chen, "Adjustable hysteresis CMOS Schmitt triggers, " in Proc. IEEE Int. Symp. Circuits Syst., 2008, pp. 1938-1941.
25. F. Yuan, "A high-speed differential CMOS Schmitt trigger with regenerative current feedback and adjustable hysteresis, " Analog Integrat. Circuits Signal Process., vol. 63, no. 1, pp. 121-127, 2010.
26. Z. Xu et al., "Parallel programming of resistive cross-point array for synaptic plasticity, " Procedia Comput. Sci., vol. 41, pp. 126-133, 2014.
27. J. Lee et al., "Diode-less nano-scale ZrOx/HfOx RRAM device with excellent switching uniformity and reliability for high-density cross-point memory applications, " in Proc. IEEE Int. Electron Devices Meet., 2010, pp. 19.5.1-19.5.4.
28. C. Xu, X. Dong, N. P. Jouppi, and Y. Xie, "Design implications of memristor-based RRAM cross-point structures, " in Design, Automat. Test Eur. Conf. Exhibit., 2011, pp. 1-6.
29. J. Zhou, K.-H. Kim, and W. Lu, "Crossbar RRAM arrays: Selector device requirements during read operation, " IEEE Trans. Electron Devices, vol. 61, no. 5, pp. 1369-1376, May 2014.
30. [Online]. Available: http://yann.lecun.com/exdb/mnist/
31. B. Lin et al., "Stochastic coordinate coding and its application for drosophila gene expression pattern annotation, " arXiv:1407.8147, 2014.
32. J. Mairal, F. Bach, J. Ponce, and G. Sapiro, "Online dictionary learning for sparse coding, " in Proc. 26th Annu. Int. Conf. Mach. Learn., 2009, pp. 689-696.
33. A. Coates and A. Y. Ng, "The importance of encoding versus training with sparse coding and vector quantization, " in Proc. 28th Int. Conf. Mach. Learn., 2011, pp. 921-928.