Convolutional networks for fast, energy-efficient neuromorphic computing

Proceedings of the National Academy of Sciences of the United States of America

Volumn 113, Issue 41, 2016, Pages 11441-11446

  Esser, Steven K ^a   Merolla, Paul A ^a   Arthur, John V ^a   Cassidy, Andrew S ^a   Appuswamy, Rathinakumar ^a   Andreopoulos, Alexander ^a   Berg, David J ^a   McKinstry, Jeffrey L ^a   Melano, Timothy ^a   Barch, Davis R ^a   Di Nolfo, Carmelo ^a   Datta, Pallab ^a   Amir, Arnon ^a   Taba, Brian ^a   Flickner, Myron D ^a   Modha, Dharmendra S ^a  

^a IBM ALMADEN RESEARCH CENTER   (United States)

Author keywords

Convolutional network; Neural network; Neuromorphic; Truenorth

Indexed keywords

ACCURACY; ALGORITHM; ARTICLE; BRAIN FUNCTION; HUMAN; LEARNING; LUMINANCE; MEMBRANE POTENTIAL; NERVE CELL; NEUROMORPHIC COMPUTING; PRIORITY JOURNAL; RECEPTIVE FIELD; SYNAPSE;

EID: 84991503397     PISSN: 00278424     EISSN: 10916490     Source Type: Journal    
DOI: 10.1073/pnas.1604850113     Document Type: Article

References (44)

1. Rumelhart DE, Hinton GE, Williams RJ (1986) Learning representations by backpropagating errors. Nature 323:533-536.
2. Fukushima K (1980) Neocognitron: A self organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biol Cybern 36(4):193-202.
3. LeCun Y, et al. (1989) Backpropagation applied to handwritten zip code recognition. Neural Comput 1(4):541-551.
4. LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436-444.
5. Mead C (1990) Neuromorphic electronic systems. Proc IEEE 78(10):1629-1636.
6. Merolla PA, et al. (2014) A million spiking-neuron integrated circuit with a scalable communication network and interface. Science 345(6197):668-673.
7. Lin M, Chen Q, Yan S (2014) Network in network. International Conference on Learning Representations (ICLR). arXiv:1312.4400v3.
8. Bartol TM, et al. (2015) Nanoconnectomic upper bound on the variability of synaptic plasticity. eLife 4:e10778.
9. Jin J, Dundar A, Culurciello E (2014) Flattened convolutional neural networks for feedforward acceleration. arXiv:1412.5474.
10. Stromatias E, et al. (2015) Robustness of spiking Deep Belief Networks to noise and reduced bit precision of neuro-inspired hardware platforms. Front Neurosci 9:2-2.
11. Courbariaux M, Bengio Y, David JP (2015) Binaryconnect: Training deep neural networks with binary weights during propagations. Advances in Neural Information Processing Systems 28:3105-3113.
12. Wu Z, Lin D, Tang X (2015) Adjustable bounded rectifiers: Towards deep binary representations. arXiv:1511.06201.
13. Diehl PU, et al. (2016) Truehappiness: Neuromorphic emotion recognition on truenorth. arXiv:1601.04183.
14. Han S, Pool J, Tran J, Dally W (2015) Learning both weights and connections for efficient neural network. Advances in Neural Information Processing Systems 28:1135-1143.
15. Esser SK, Appuswamy R, Merolla P, Arthur JV, Modha DS (2015) Backpropagation for energy-efficient neuromorphic computing. Advances in Neural Information Processing Systems 28:1117-1125.
16. Courbariaux M, Bengio Y (2016) Binarynet: Training deep neural networks with weights and activations constrained to+ 1 or-1. arXiv:1602.02830.
17. Bell AJ, Sejnowski TJ (1997) The "independent components" of natural scenes are edge filters. Vision Res 37(23):3327-3338.
18. Vedaldi A, Lenc K (2015) MatConvNet: Convolutional neural networks for MATLAB. Proceeding of the 23rd ACM International Conference on Multimedia (ACM, New York).
19. Amir A, et al. (2013) Cognitive computing programming paradigm: A corelet language for composing networks of neurosynaptic cores. Neural Networks (IJCNN), The 2013 International Joint Conference (IEEE, Dallas), pp 1-10.
20. Painkras E, et al. (2013) Spinnaker: A 1-W 18-core system-on-chip for massively-parallel neural network simulation. IEEE J Solid-State Circuits 48(8):1943-1953.
21. Pfeil T, et al. (2013) Six networks on a universal neuromorphic computing substrate. Front Neurosci 7:11.
22. Moradi S, Indiveri G (2014) An event-based neural network architecture with an asynchronous programmable synaptic memory. IEEE Trans Biomed Circuits Syst 8(1):98-107.
23. Park J, Ha S, Yu T, Neftci E, CauwenberghsG (2014) A65k-neuron 73-Mevents/s 22-pJ/event asynchronous micro-pipelined integrate-and-fire array transceiver. Proceedings of the Biomedical Circuits and Systems Conference (BioCAS), 2014 IEEE (IEEE, Lausanne, Switzerland), pp 675-678.
24. Ioffe S, Szegedy C (2015) Batch normalization: Accelerating deep network training by reducing internal covariate shift. arXiv:1502.03167.
25. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems 25:1097-1105.
26. Krizhevsky A (2009) Learning Multiple Layers of Features From Tiny Images (University of Toronto, Toronto).
27. Netzer Y, et al. (2011) Reading digits in natural images with unsupervised feature learning. NIPS Workshop on Deep Learning and Unsupervised Feature Learning 2011 (NIPS, Granada, Spain).
28. Stallkamp J, Schlipsing M, Salmen J, Igel C (2012) Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition. Neural Netw 32:323-332.
29. Romberg S, Pueyo LG, Lienhart R, Zwol RV (2011) Scalable logo recognition in realworld images. Proceedings of the 1st ACM International Conference on Multimedia Retrieval (ACM, New York), p 25.
30. Garofolo J, et al. (1993) TIMIT Acoustic-Phonetic Continuous Speech Corpus LDC93S1 (Linguistic Data Consortium, Philadelphia).
31. Varga A, Steeneken H J M (1993) Assessment for automatic speech recognition II: NOISEX-92: A database and an experiment to study the effect of additive noise on speech recognition systems. Speech Commun 12(3):247-251.
32. Springenberg JT, Dosovitskiy A, Brox T, Riedmiller M (2014) Striving for simplicity: The all convolutional net. arXiv:1412.6806.
33. Preissl R, et al. (2012) Compass: A scalable simulator for an architecture for cognitive computing. Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis (IEEE Computer Society Press, Los Alamitos, CA), p 54.
34. Lee CY, Xie S, Gallagher P, Zhang Z, Tu Z (2014) Deeply-supervised nets. arXiv: 1409.5185.
35. Cireşan D, Meier U, Masci J, Schmidhuber J (2012) Multi-column deep neural network for traffic sign classification. Neural Netw 32:333-338.
36. Pham TV, Tang CT, Stadtschnitzer M (2009) Using artificial neural network for robust voice activity detection under adverse conditions. Proceedings of the International Conference on Computing and Communication Technologies 1:1-8.
37. Chang HA, Glass JR (2007) Hierarchical large-margin gaussian mixture models for phonetic classification. Proceedings of the IEEE Workshop on Automatic Speech Recognition and Understanding (IEEE, Kyoto).
38. Graves A, Jaitly N, Mohamed A (2013) Hybrid speech recognition with deep bidirectional lstm. Proceedings of the IEEE Workshop on Automatic Speech Recognition and Understanding (IEEE, Olomouc, Czech Republic), pp 273-278.
39. Merolla P, et al. (2011) A digital neurosynaptic core using embedded crossbar memory with 45pJ per spike in 45nm. Proceedings of the IEEE Custom Integrated Circuits Conference (IEEE, San Jose, CA), pp 1-4.
40. Esser SK, et al. (2013) Cognitive computing systems: Algorithms and applications for networks of neurosynaptic cores. Proceedings of the Neural Networks (IJCNN), The 2013 International Joint Conference (IEEE, Dallas), pp 1-10.
41. Diehl PU, Zarrella G, Cassidy A, Pedroni BU, Neftci E (2016) Conversion of artificial recurrent neural networks to spiking neural networks for low-power neuromorphic hardware. arXiv:1601.04187.
42. Das S, et al. (2015) Gibbs sampling with low-power spiking digital neurons. Proceedings of the IEEE International Symposium on Circuits and Systems (IEEE, Lisbon, Portugal).
43. Conti F, Benini L (2015) A ultra-low-energy convolution engine for fast brain-inspired vision in multicore clusters. Proceedings of the 2015 Design, Automation & Test in Europe Conference & Exhibition (IEEE, Grenoble, France), pp 683-688.
44. Chen YH, Krishna T, Emer J, Sze V (2016) 14.5 Eyeriss: An energy-efficient reconfigurable accelerator for deep convolutional neural networks. Proceedings of the 2016 IEEE International Solid-State Circuits Conference (ISSCC) (IEEE, San Francisco), pp 262-263.