Learning to be efficient: Algorithms for training low-latency, low-compute Deep Spiking Neural Networks

Proceedings of the ACM Symposium on Applied Computing

Volumn 04-08-April-2016, Issue , 2016, Pages 293-298

  Neil, Daniel ^a   Pfeiffer, Michael ^a   Liu, Shih Chii ^a  

^a Institute of Neuroinformatics Zurich   (Switzerland)

Author keywords

Neural network efficiency; Spiking deep neural networks; Spiking neural network latency

Indexed keywords

BUDGET CONTROL; FEATURE EXTRACTION; INTELLIGENT AGENTS; LOW POWER ELECTRONICS; NEURAL NETWORKS;

AVERAGE NUMBERS; CLASSIFICATION ACCURACY; CLASSIFICATION PERFORMANCE; COMPUTATIONAL EFFORT; FEATURE DETECTOR; NETWORK EFFICIENCY; OPTIMIZATION APPROACH; SPIKING NEURAL NETWORKS;

DEEP NEURAL NETWORKS;

EID: 84975859254     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/2851613.2851724     Document Type: Conference Paper

References (23)

1. C. Brandli, L. Muller, and T. Delbruck. Real-time, high-speed video decompression using a frame- and event-based DAVIS sensor. In 2014 IEEE International Symposium on Circuits and Systems (ISCAS), pages 686-689. IEEE, 2014.
2. Y. Cao, Y. Chen, and D. Khosla. Spiking deep convolutional neural networks for energy-efficient object recognition. International Journal of Computer Vision, pages 1-13, 2014.
3. P. U. Diehl, D. Neil, J. Binas, M. Cook, S.-C. Liu, and M. Pfeiffer. Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing. In International Joint Conference on Neural Networks (IJCNN), 2015.
4. C. Farabet et al. Comparison between frame-constrained fix-pixel-value and frame-free spiking-dynamic-pixel convnets for visual processing. Frontiers in Neuroscience, 6, 2012.
5. S. Furber, F. Galluppi, S. Temple, and L. Plana. The SpiNNaker Project. Proceedings of the IEEE, 102(5):652-665, 2014.
6. A. Hannun et al. Deep Speech: Scaling up end-to-end speech recognition. arXiv preprint arXiv:1412.5567, 2014.
7. G. Hinton, L. Deng, D. Yu, G. E. Dahl, A.-r. Mohamed, N. Jaitly, A. Senior, V. Vanhoucke, P. Nguyen, T. N. Sainath, et al. Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. IEEE Signal Processing Magazine, 29(6):82-97, 2012.
8. G. E. Hinton et al. Improving neural networks by preventing co-adaptation of feature detectors. arXiv preprint arXiv:1207.0580, 2012.
9. G. Indiveri, E. Chicca, and R. Douglas. A VLSI array of low-power spiking neurons and bistable synapses with spike-timing dependent plasticity. IEEE Trans on Neural Networks, 17(1):211-221, 2006.
10. Y. LeCun, Y. Bengio, and G. Hinton. Deep learning. Nature, 521(7553):436-444, 2015.
11. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-2324, 1998.
12. S.-C. Liu and T. Delbruck. Neuromorphic sensory systems. Current Opinion in Neurobiology, 20(3):288-295, 2010.
13. P. Merolla et al. A million spiking-neuron integrated circuit with a scalable communication network and interface. Science, 345(6197):668-673, 2014.
14. V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, et al. Human-level control through deep reinforcement learning. Nature, 518(7540):529-533, 2015.
15. D. Neil and S.-C. Liu. Minitaur, an event-driven FPGA-based spiking network accelerator. IEEE Trans on Very Large Scale Integration (VLSI) Systems, 22(12):2621-2628, 2014.
16. P. O'Connor, D. Neil, S.-C. Liu, T. Delbruck, and M. Pfeiffer. Real-time classification and sensor fusion with a spiking Deep Belief Network. Frontiers in Neuroscience, 7, 2013.
17. R. B. Palm. Prediction as a candidate for learning deep hierarchical models of data, 2012.
18. J. Pérez-Carrasco and others. Mapping from frame-driven to frame-free event-driven vision systems by low-rate rate coding and coincidence processing-Application to feedforward ConvNets. IEEE Trans on Pattern Analysis and Machine Intelligence, 35(11):2706-2719, 2013.
19. C. Poultney, S. Chopra, Y. L. Cun, et al. Efficient learning of sparse representations with an energy-based model. In Proc. of NIPS, pages 1137-1144, 2006.
20. K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.
21. N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov. Dropout: A simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, 15(1):1929-1958, 2014.
22. E. Stromatias, D. Neil, F. Galluppi, M. Pfeiffer, S.-C. Liu, and S. Furber. Live demonstration: handwritten digit recognition using spiking Deep Belief Networks on SpiNNaker. In 2015 IEEE International Symposium on Circuits and Systems (ISCAS), pages 1901-1901, 2015.
23. C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich. Going deeper with convolutions. arXiv preprint arXiv:1409.4842, 2014.