Hierarchical Temporal Memory Based on Spin-Neurons and Resistive Memory for Energy-Efficient Brain-Inspired Computing

IEEE Transactions on Neural Networks and Learning Systems

Volumn 27, Issue 9, 2016, Pages 1907-1919

  Fan, Deliang ^a   Sharad, Mrigank ^a   Sengupta, Abhronil ^a   Roy, Kaushik ^a  

^a PURDUE UNIVERSITY   (United States)

Author keywords

Hierarchical temporal memory (HTM); magnetic domain walls (DWs); memristors; neural network hardware; spin Hall effect (SHE); spin transfer torque

Indexed keywords

INTEGRATED CIRCUIT DESIGN; INTEGRATED CIRCUITS;

BRAIN-INSPIRED COMPUTING; COMPREHENSIVE DESIGNS; CROSSBAR NETWORKS; ENERGY EFFICIENT; HIERARCHICAL TEMPORAL MEMORY (HTM); PRODUCT EVALUATION; RESISTIVE MEMORY; SPATIAL AND TEMPORAL PATTERNS;

ENERGY EFFICIENCY;

ALGORITHM; ARTIFICIAL NEURAL NETWORK; HUMAN; MEMORY; NEOCORTEX;

ALGORITHMS; HUMANS; MEMORY; NEOCORTEX; NEURAL NETWORKS (COMPUTER);

EID: 84939427824     PISSN: 2162237X     EISSN: 21622388     Source Type: Journal    
DOI: 10.1109/TNNLS.2015.2462731     Document Type: Article

References (53)

1. D. C. Van Essen, C. H. Anderson, and D. J. Felleman, "Information processing in the primate visual system: An integrated systems perspective, " Science, vol. 255, no. 5043, pp. 419-423, Jan. 1992.
2. R. P. Lippmann, "An introduction to computing with neural nets, " IEEE ASSP Mag., vol. 4, no. 2, pp. 4-22, Apr. 1987.
3. P. Y. Simard, D. Steinkraus, and J. C. Platt, "Best practices for convolu-tional neural networks applied to visual document analysis, " in Proc. 7th IEEE Int. Conf. Document Anal. Recognit., Edinburgh, U.K., Aug. 2003, pp. 958-963.
4. D. George and J. Hawkins, "A hierarchical Bayesian model of invariant pattern recognition in the visual cortex, " in Proc. IEEE Int. Joint Conf. Neural Netw., Montreal, QC, Canada, Jul./Aug. 2005, pp. 1812-1817.
5. D. George and J. Hawkins, "Towards a mathematical theory of cortical micro-circuits, " PLoS Comput. Biol., vol. 5, no. 10, pp. e1000532-1-e1000532-26, Oct. 2009.
6. D. Maltoni, "Pattern recognition by hierarchical temporal memory, " Dept. Comput. Sci. Eng., Univ. Bologna, Bologna, Italy, Tech. Rep., Apr. 2011.
7. W. J. C. Melis, S. Chizuwa, and M. Kameyama, "Evaluation of the hierarchical temporal memory as soft computing platform and its VLSI architecture, " in Proc. 39th Int. Symp. Multiple-Valued Logic, Naha, Japan, May 2009, pp. 233-238.
8. J. Hawkins, S. Ahmad, and D. Dubinsky, "Hierarchical temporal memory including HTM cortical learning algorithms, " Numenta, Inc., Palo Alto, CA, USA, Tech. Rep. Version 0.2.1, Sep. 2010.
9. S. Shin, K. Kim, and S.-M. S. Kang, "Memristor-based fine resolution programmable resistance and its applications, " in Proc. IEEE Int. Conf. Commun., Circuits, Syst., Milpitas, CA, USA, Jul. 2009, pp. 948-951.
10. R. Berdan, T. Prodromakis, and C. Toumazou, "High precision analogue memristor state tuning, " Electron. Lett., vol. 48, no. 18, pp. 1105-1107, Aug. 2012.
11. x-based memristors, " ACS Nano, vol. 6, no. 3, pp. 2312-2318, Feb. 2012.
12. K. K. Likharev, "Biologically inspired computing in CMOL CrossNets, " in Proc. AAAI Fall Symp. Ser., Arlington, VA, USA, Nov. 2009, p. 90.
13. Ö. Türel, J. H. Lee, X. Ma, and K. K. Likharev, "Neuromorphic architectures for nanoelectronic circuits, " Int. J. Circuits Theory Appl., vol. 32, no. 5, pp. 277-302, Sep./Oct. 2004.
14. S. H. Jo, K.-H. Kim, and W. Lu, "High-density crossbar arrays based on a Si memristive system, " Nano Lett., vol. 9, no. 2, pp. 870-874, Jan. 2009.
15. S. H. Jo and W. Lu, "CMOS compatible nanoscale nonvolatile resistance switching memory, " Nano Lett., vol. 8, no. 2, pp. 392-397, Jan. 2008.
16. L. Gao, F. Alibart, and D. B. Strukov, "Analog-input analog-weight dot-product operation with Ag/a-Si/Pt memristive devices, " in Proc. IEEE/IFIP 20th Int. Conf. VLSI Syst.-Chip (VLSI-SoC), Santa Cruz, CA, USA, Oct. 2012, pp. 88-93.
17. B. Mouttet, "Proposal for memristors in signal processing, " in Proc. 3rd Int. ICST Conf. Nano-Net, Boston, MA, USA, Sep. 2008, pp. 11-13.
18. D. Morris, D. Bromberg, J.-G. Zhu, and L. Pileggi, "mLogic: Ultra-low voltage non-volatile logic circuits using STT-MTJ devices, " in Proc. 49th ACM/EDAC/IEEE Design Autom. Conf., San Francisco, CA, USA, Jun. 2012, pp. 486-491.
19. M. Hu, H. Li, Q. Wu, and G. S. Rose, "Hardware realization of BSB recall function using memristor crossbar arrays, " in Proc. 49th ACM/EDAC/IEEE Design Autom. Conf., San Francisco, CA, USA, Jun. 2012, pp. 498-503.
20. C. K. Lim et al., "Domain wall displacement induced by subnanosecond pulsed current, " Appl. Phys. Lett., vol. 84, no. 15, pp. 2820-2822, Apr. 2004.
21. J. Vogel et al., "Direct observation of massless domain wall dynamics in nanostripes with perpendicular magnetic anisotropy, " Phys. Rev. Lett., vol. 108, no. 24, p. 247202, Jun. 2012.
22. D.-T. Ngo, K. Ikeda, and H. Awano, "Direct observation of domain wall motion induced by low-current density in TbFeCo wires, " Appl. Phys. Exp., vol. 4, no. 9, p. 093002, Sep. 2011.
23. S. Fukami et al., "Low-current perpendicular domain wall motion cell for scalable high-speed MRAM, " in Proc. Symp. VLSI Technol., Honolulu, HI, USA, Jun. 2009, pp. 230-231.
24. P. R. Kinget, "Device mismatch and tradeoffs in the design of analog circuits, " IEEE J. Solid-State Circuits, vol. 40, no. 6, pp. 1212-1224, Jun. 2005.
25. A. Demosthenous, S. Smedley, and J. Taylor, "A CMOS analog winner-take-all network for large-scale applications, " IEEE Trans. Circuits Syst. I, Fundam. Theory Appl., vol. 45, no. 3, pp. 300-304, Mar. 1998.
26. R. DŁugosz and T. Taláska, "Low power current-mode binary-tree asynchronous min/max circuit, " Microelectron. J., vol. 41, no. 1, pp. 64-73, Jan. 2010.
27. S. R. Nassif, "Process variability at the 65 nm node and beyond, " in Proc. IEEE Custom Integr. Circuits Conf., San Jose, CA, USA, Sep. 2008, pp. 1-8.
28. M. Sharad, G. Panagopoulos, and K. Roy, "Spin neuron for ultra low power computational hardware, " in Proc. 70th Annu. Device Res. Conf. (DRC), University Park, TX, USA, Jun. 2012, pp. 221-222.
29. K.-H. Kim et al., "A functional hybrid memristor crossbar-array/CMOS system for data storage and neuromorphic applications, " Nano Lett., vol. 12, no. 1, pp. 389-395, Dec. 2012.
30. C. Augustine et al., "Numerical analysis of domain wall propagation for dense memory arrays, " in Proc. IEEE Int. Electron Devices Meeting (IEDM), Washington, DC, USA, Dec. 2011, pp. 17.6.1-17.6.4.
31. C.-M. Jung, J.-M. Choi, and K.-S. Min, "Two-step write scheme for reducing sneak-path leakage in complementary memristor array, " IEEE Trans. Nanotechnol., vol. 11, no. 3, pp. 611-618, May 2012.
32. H. Manem and G. S. Rose, "A read-monitored write circuit for 1T1M multi-level memristor memories, " in Proc. IEEE Int. Symp. Circuits Syst. (ISCAS), Rio de Janeiro, Brazil, May 2011, pp. 2938-2941.
33. R. O. Duda, P. E. Hart, and D. G. Stork, Pattern Classification. New York, NY, USA: Wiley, 2001.
34. M. Sharad, D. Fan, and K. Roy, "Ultra low power associative computing with spin neurons and resistive crossbar memory, " in Proc. 50th Annu. Design Autom. Conf., Austin, TX, USA, May 2013, pp. 1-6.
35. M. Sharad, D. Fan, and K. Roy, "Spin-neurons: A possible path to energy-efficient neuromorphic computers, " J. Appl. Phys., vol. 114, no. 23, p. 234906, Dec. 2013.
36. A. V. Khvalkovskiy et al., "Matching domain-wall configuration and spin-orbit torques for efficient domain-wall motion, " Phys. Rev. B, vol. 87, no. 2, p. 020402(R), Jan. 2013.
37. I. M. Miron et al., "Fast current-induced domain-wall motion controlled by the Rashba effect, " Nature Mater., vol. 10, no. 6, pp. 419-423, May 2011.
38. K.-W. Kim, S.-M. Seo, J. Ryu, K.-J. Lee, and H.-W. Lee, "Magnetization dynamics induced by in-plane currents in ultrathin magnetic nanostruc-tures with Rashba spin-orbit coupling, " Phys. Rev. B, vol. 85, no. 18, p. 180404(R), May 2012.
39. Y. LeCun, C. Cortes, and C. J. C. Burges. MNIST Handwritten Digit Database. [Online]. Available: http://yann.lecun.com/exdb/mnist/, accessed Apr. 29, 2015.
40. CAVE | Software: COIL-20: Columbia Object Image Library. [Online]. Available: http://www.cs.columbia.edu/CAVE/software/softlib/coil-20. php, accessed Apr. 29, 2015.
41. (2015). HP Labs: CACTI. [Online]. Available: http://www.hpl.hp.com/research/cacti/accessed Apr.29, 2015.
42. Z. Zeng et al., "Ultralow-current-density and bias-field-free spin-transfer nano-oscillator, " Sci. Rep., vol. 3, Mar. 2013, Art. ID 1426.
43. C.-C. Wang, R.-C. Kuo, and T.-H. Tsai, "A high precision low dropout regulator with nested feedback loops, " Microelectron. J., vol. 42, no. 7, pp. 966-971, Jul. 2011.
44. M. Sharad, C. Augustine, and K. Roy, "Boolean and non-Boolean computation with spin devices, " in Proc. IEEE Int. Electron Devices Meeting (IEDM), San Francisco, CA, USA, Dec. 2012, pp. 11.6.1-11.6.4.
45. J. A. Currivan, Y. Jang, M. D. Mascaro, M. A. Baldo, and C. A. Ross, "Low energy magnetic domain wall logic in short, narrow, ferromagnetic wires, " IEEE Magn. Lett., vol. 3, Apr. 2012, Art. ID 3000104.
46. K. Fukushima, "Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position, " Biol. Cybern., vol. 36, no. 4, pp. 193-202, Apr. 1980.
47. M. Riesenhuber and T. Poggio, "Hierarchical models of object recognition in cortex, " Nature Neurosci., vol. 2, no. 11, pp. 1019-1025, Nov. 1999.
48. M. Sharad, D. Fan, K. Aitken, and K. Roy, "Energy-efficient non-Boolean computing with spin neurons and resistive memory, " IEEE Trans. Nanotechnol., vol. 13, no. 1, pp. 23-34, Jan. 2014.
49. S. Fukami et al., "20-nm magnetic domain wall motion memory with ultralow-power operation, " in Proc. IEEE Int. Electron Devices Meeting (IEDM), Washington, DC, USA, Dec. 2013, pp. 3.5.1-3.5.4.
50. D. B. Strukov and K. K. Likharev, "Reconfigurable hybrid CMOS/nanodevice circuits for image processing, " IEEE Trans. Nanotechnol., vol. 6, no. 6, pp. 696-710, Nov. 2007.
51. L. Wang, "Heteroassociations of spatio-temporal sequences with the bidirectional associative memory, " IEEE Trans. Neural Netw., vol. 11, no. 6, pp. 1503-1505, Nov. 2000.
52. J. A. Starzyk and H. He, "Spatio-temporal memories for machine learning: A long-term memory organization, " IEEE Trans. Neural Netw., vol. 20, no. 5, pp. 768-780, May 2009.
53. V. A. Nguyen, J. A. Starzyk, W.-B. Goh, and D. Jachyra, "Neural network structure for spatio-temporal long-term memory, " IEEE Trans. Neural Netw. Learn. Syst., vol. 23, no. 6, pp. 971-983, Jun. 2012.