Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing

Proceedings of the International Joint Conference on Neural Networks

Volumn 2015-September, Issue , 2015, Pages

  Diehl, Peter U ^a   Neil, Daniel ^a   Binas, Jonathan ^a   Cook, Matthew ^a   Liu, Shih Chii ^a   Pfeiffer, Michael ^a  

^a UNIVERSITY OF ZURICH   (Switzerland)

Author keywords

Accuracy; Handheld computers; Neuromorphics; Neurons; Robustness

Indexed keywords

ARTIFICIAL INTELLIGENCE; COMPUTER VISION; HAND HELD COMPUTERS; LEARNING SYSTEMS; NEURONS; PATTERN RECOGNITION; ROBUSTNESS (CONTROL SYSTEMS);

ACCURACY; ANALOG NEURAL NETWORK; CONVOLUTIONAL NETWORKS; NEUROMORPHIC COMPUTING; NEUROMORPHICS; NORMALIZATION METHODS; OPTIMIZATION TECHNIQUES; SPIKING NEURAL NETWORKS;

NEURAL NETWORKS;

EID: 84951159868     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IJCNN.2015.7280696     Document Type: Conference Paper

References (35)

1. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition, " Proceedings of the IEEE, vol. 86, no. 11, pp. 2278-2324, 1998.
2. D. C. Ciresan, U. Meier, L. M. Gambardella, and J. Schmidhuber, "Deep, b ig, s imple neural nets for handwritten digit recognition, " Neural Computation, vol. 22, no. 12, pp. 3207-3220, 2010.
3. c. Farabet, C. Couprie, L. Najman, and Y. LeCun, "Learning hierarchical features for scene labeling, " IEEE Trans on Pattern Analysis and Machine Intelligence, vol. 35, no. 8, pp. 1915-1929, 2013.
4. A. Krizhevsky, I. Sutskever, and G. E. Hinton, "Imagenet classification with deep convolutional neural networks, " in Proc. of NIPS, 2012, pp. 1097-1105.
5. P. Sermanet, D. Eigen, X. Zhang, M. Mathieu, R. Fergus, and Y. LeCun, "OverFeat: Integrated recognition, localization and detection using convolutional networks, " arXiv preprint, vol. 312. 6229, 2013.
6. K. Fukushima, "Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position, " Biological Cybernetics, vol. 36, no. 4, pp. 193-202, 1980.
7. J. Schmidhuber, "Deep learning in neural networks: An overview, " Neural Networks, vol. 61, pp. 85-117, 2015.
8. P. A. Merolla and others., "A million spiking-neuron integrated circuit with a scalable communication network and interface, " Science, vol. 345, no. 6197, pp. 668-673, 2014.
9. B. V. Benjamin et al., "Neurogrid: A mixed-analog-digital multichip system for large-scale neural simulations, " Proceedings of the IEEE, vol. 102, no. 5, pp. 699-716, 2014.
10. S. Furber, F. Galluppi, S. Temple, and L. Plana, "The SpiNNaker Project, " Proceedings of the IEEE, vol. 102, no. 5, pp. 652-665, 2014.
11. D. Neil and S.-C. Liu, "Minitaur, an event-driven fpga-based spiking network accelerator, " IEEE Trans on Very Large Scale Integration (VLSl) Systems, vol. 22, no. 12, pp. 2621-2628, 2014.
12. G. Indiveri, E. Chicca, and R. Douglas, "A vlsi reconfigurable network of integrate-and-fire neurons with spike-based learning synapses, " 2004.
13. X. Jin, A. Rast, F. Galluppi, S. Davies, and S. Furber, "Implementing spike-timing-dependent plasticity on spinnaker neuromorphic hardware, " in Neural Networks ([JCNN), The 2010 International Joint Conference on. IEEE, 2010, pp. 1-8.
14. P. U. Diehl and M. Cook, "Efficient implementation of stdp rules on spinnaker neuromorphic hardware, " in 2014 International Joint Coriference on Neural Networks ([JCNN). IEEE, 2014, pp. 4288-4295.
15. P. Lichtsteiner, C. Posch, and T. Delbruck, "A 128x 128 120 db 15 f. J, s latency asynchronous temporal contrast vision sensor, " IEEE Journal of Solid-State Circuits, vol. 43, no. 2, pp. 566-576, 2008.
16. S.-C. Liu and T. Del bruck, "Neuromorphic sensory systems, " Current Opinion in Neurobiology, vol. 20, no. 3, pp. 288-295, 2010.
17. R. Serrano-Gotarredona and others., "CAVIAR: A 45k neuron, 5m synapse, 12g connects/s aer hardware sensory-processing-learningactuating system for high-speed visual object recognition and tracking, " IEEE Trans on Neural Networks, vol. 20, no. 9, pp. 14l7-1438, 2009.
18. C. Farabet, R. Paz, J. Perez-Carrasco, c. Zamarreno-Ramos, A. LinaresBarranco, Y. LeCun, E. Culurciello, T. Serrano-Gotarredona, and B. Linares-Barranco, " Comparison between frame-constrained fix-pixelvalue and frame-free spiking-dynamic-pixel convnets for visual processing, " Frontiers in Neuroscience, vol. 6, 2012.
19. P. O'Connor, D. Neil, S.-C. Liu, T. Delbruck, and M. Pfeiffer, "Realtime classification and sensor fusion with a spiking deep belief network, " Frontiers in Neuroscience, vol. 7, 2013.
20. W. Maass and H. Markram, "On the computational power of circuits of spiking neurons, " Journal of Computer and System Sciences, vol. 69, no. 4, pp. 593-616, 2004.
21. J. Perez-Carrasco and others., "Mapping from frame-driven to framefree event-driven vision systems by low-rate rate coding and coincidence processing-application to feedforward convnets, " IEEE Trans on Pattern Analysis and Machine Intelligence, vol. 35, no. 11, pp. 2706-2719, 2013.
22. Y. Cao, Y. Chen, and D. Khosla, "Spiking deep convolutional neural networks for energy-efficient object recognition, " International Journal of Computer Vision, pp. 1-13, 2014.
23. N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov, "Dropout: A simple way to prevent neural networks from overfitting, " The Journal of Machine Learning Research, vol. 15, no. 1, pp. 1929-1958, 2014.
24. L. Camunas-Mesa, C. Zamarreno-Ramos, A. Linares-Barranco, A. J. Acosta-Jimenez, T. Serrano-Gotarredona, and B. Linares-Barranco, "An event-driven multi-kernel convolution processor module for eventdriven vision sensors, " IEEE Journal of Solid-State Circuits, vol. 47, no. 2, pp. 504-517, 2012.
25. P. Merolla, J. Arthur, F. Akopyan, N. Imam, R. Manohar, and D. S. Modha, "A digital neurosynaptic core using embedded crossbar memory with 45pJ per spike in 45nm, " in 2011 IEEE Custom Integrated Circuits Coriference (CICC), 2011, pp. 1-4.
26. I. J. Goodfellow, D. Warde-Farley, M. Mirza, A. Courville, and Y. Bengio, "Maxout networks, " in ICML, 2013.
27. D. Garbin, O. Bichler, E. Vianello, Q. Rafuay, C. Gamrat, L. Perniola, G. Ghibaudo, and B. DeSalvo, " Variability-tolerant convolutional neural network for pattern recognition applications based on oxram synapses, " IEEE International Electron Devices Meeting (IEDM), pp. 1-13, 2014.
28. R. B. Palm, "Prediction as a candidate for learning deep hierarchical models of data, " 2012.
29. L. Camunas-Mesa, A. Acosta-Jimenez, C. Zamarreno-Ramos, T. Serrano-Gotarredona, and B. Linares-Barranco, "A 32 x 32 pixel convolution processor chip for address event vision sensors with 155 ns event latency and 20 Meps throughput, " IEEE Trans on Circuits and Systems 1: Regular Papers, vol. 58, no. 4, pp. 777-790, 2011.
30. D. Querlioz, O. Bichler, P. Dollfus, and C. Gamrat, " Immunity to device variations in a spiking neural network with memristive nanodevices, " IEEE Trans on Nanotechnology, vol. 12, no. 3, pp. 288-295, 2013.
31. P. U. Diehl and M. Cook, "Unsupervised learning of digit recognition using spike-timing-dependent plasticity, " Frontiers in Computational Neuroscience, vol. 9, no. 99, 2015.
32. B. Zhao, R. Ding, S. Chen, B. Linares-Barranco, and H. Tang, " Feedf orward categorization on aer motion events using cortex-like features in a spiking neural network, " IEEE Trans on Neural Networks and Learning Systems, vol. PP, no. 99, 2014.
33. J. M. Brader, W. Senn, and S. Fusi, "Learning real-world stimuli in a neural network with spike-driven synaptic dynamics, " Neural Computation, vol. 19, no. 11, pp. 2881-2912, 2007.
34. E. Neftci, S. Das, B. Pedroni, K. Kreutz-Delgado, and G. Cauwenberghs, "Event-driven contrastive divergence for spiking neuromorphic systems, " Frontiers in Neuroscience, vol. 7, no. 272, 2013.
35. S. Hussain, S.-C. Liu, and A. Basu, "Improved margin multi-class classification using dendritic neurons with morphological learning, " in 2014 IEEE International Symposium on Circuits and Systems (ISCAS), 2014, pp. 2640-2643.