Comparison between frame-constrained fix-pixel-value and frame-free spiking-dynamic-pixel convNets for visual processing

Frontiers in Neuroscience

Volumn , Issue APR, 2012, Pages

  Farabet, Clément ^a,b   Paz, Rafael ^c   Pérez Carrasco, Jose ^d   Zamarreño Ramos, Carlos ^d   Linares Barranco, Alejandro ^c   LeCun, Yann ^a   Culurciello, Eugenio ^b   Serrano Gotarredona, Teresa ^d   Linares Barranco, Bernabe ^d  

^a POLYTECHNIC UNIVERSITY   (United States)

^b UNIVERSITÉ PARIS EST   (France)

^c UNIVERSITY OF SEVILLE   (Spain)

^d INSTITUTO DE MICROELECTRÓNICA DE SEVILLA   (Spain)

Author keywords

Address event representation; Convolutional neural network; Fpga; Frame free vision; Image convolutions; Spike based convolutions; Vhdl

Indexed keywords

ARTICLE; BRAIN; BRAIN CORTEX; CODING; COMPUTER PROGRAM; CONVNETS; DATA PROCESSING; DEVICES; FEEDBACK SYSTEM; IMAGE PROCESSING; IMAGING AND DISPLAY; NERVE CELL; RETINA; SCALE UP; SIMULATOR; STATISTICS; SYNAPSE; VISUAL INFORMATION;

EID: 84862236095     PISSN: 16624548     EISSN: 1662453X     Source Type: Journal    
DOI: 10.3389/fnins.2012.00032     Document Type: Article

References (52)

1. Azadmehr, M., Abrahamsen, J., and Häfliger, P. (2005). "A foveated AER imager chip," in Proceedings of the IEEE International Symposium on CircuitsandSystems. (ISCAS) (Kobe: IEEE Press), 2751-2754.
2. Azevedo, F. A., Carvalho, L. R., Grin-berg, L. T., Farfel, J. M., Ferretti, R. E., Leite, R. E., Jacob Filho, W., Lent, R., and Herculano-Houzel, S. (2009). Equal numbers of neuronal and nonneuronal cells make the human brain an isometrically scaled-up primate brain. J. Comp. Neurol. 513, 532-541.
3. Boahen, K. (1999). "Retinomorphic chips that see quadruple images," in Proceedings of the International Conference Microelectronics for Neural, Fuzzy and Bio-Inspired Systems (Microneuro) (Granada: IEEE Press), 12-20.
4. Boser, B., Säckinger, E., Bromley, J., LeCun, Y., and Jackel, L. (1991). An analog neural network processor with programmable topology. IEEE I Solid State Circuits 26, 2017-2025.
5. Camuñas-Mesa, L., Acosta-Jiménez, A., Zamarreño-Ramos, C., Serrano-Gotarredona, T., and Linares-Barranco, B. (2011). A convolution processor chip for address event vision sensors with 155ns event latency and 20Meps throughput. IEEE Trans. Circuits Syst. 58, 777-790.
6. Camuñas-Mesa, L., Zamarreño-Ramos, C., Linares-Barranco, A., Acosta-Jiménez, A., Serrano-Gotarredona, T., and Linares-Barranco, B. (2012). An event-driven convolution processor module for event-driven vision sensors. IEEE J. Solid State Circuits 47, 504-517.
7. Chan, V., Liu, S.-C., and van Schaik, A. (2007). AER EAR: a matched silicon cochlea pair with address event representation interface. IEEE Trans. Circuits Syst. PartI 54, 48-59.
8. Chen, S., and Bermak, A. (2007). Arbitrated time-to-first spike CMOS image sensor with on-chip histogram equalization. IEEE Trans. VLSI Syst. 15,346-357.
9. Chicca, E., Whatley, A. M., Lichtsteiner, P., Dante, V., Delbrück, T., Del Giu-dice, P., Douglas, R. J., and Indiveri, G. (2007). A multichip pulse-based neuromorphic infrastructure and its application to a model of orientation selectivity. IEEE Trans. Circuits Syst. PartI 54, 981-993.
10. Choi, T. Y. W., Merolla, P., Arthur, J., Boahen, K., and Shi, B. E. (2005). Neuromorphic implementation of orientation hypercolumns. IEEE Trans. Circuits Syst. Part I 52, 1049-1060.
11. Cloutier, J., Cosatto, E., Pigeon, S., Boyer, E, and Simard, P. Y. (1996). "Vip: an fpga-based processor for image processing and neural networks," in Proceedings of the Fifth International Conference on Microelectronics for Neural Networks and Fuzzy Systems MicroNeuro'96 (Lausanne: IEEE Press), 330-336.
12. Costas-Santos, J., Serrano-Gotarredona, T., Serrano-Gotarredona, R., and Linares-Barranco, B. (2007). A contrast retina with on-chip calibration for neuromorphic spike-based AER vision systems. IEEE Trans. Circuits Syst. I Reg. Papers 54, 1444-1458.
13. Culurciello, E., Etienne-Cummings, R., and Boahen, K. (2003). A biomor-phic digital image sensor. IEEE J. Solid State Circuits 38, 281-294.
14. Delbrück, T. (2005). http://jaer.wiki.sourceforge.net
15. Delbrück, T. (2008). "Frame-free dynamic digital vision," in Proceedings of International Symposium on Secure-Life Electronics, Advanced Electronics for Quality Life and Society (Tokyo: University of Tokyo), 21-26.
16. Hadsell, R., Sermanet, P., Erkan, A., Ben, J., Han, J., Flepp, B., Muller, U., and LeCun, Y. (2007). 'On-line learning for offroad robots: using spatial label propagation to learn long-range tra-versability,"in Proceedings of Robotics Science and Systems '07, MIT Press, Cambridge.
17. Hadsell, R., Sermanet, P., Scoffier, M., Erkan, A., Kavackuoglu, K., Muller, U., and LeCun, Y. (2009). Learning long-range vision for autonomous off-road driving. J. Field Robotics 26, 120-144.
18. Huang, F.-J., and LeCun, Y. (2006). "Large-scale learning with svm and convolutional nets for generic object categorization," in Proceedings of Computer Vision and Pattern Recognition Conference (CVPR'06) (New York: IEEE Press).
19. Jarrett, K., Kavukcuoglu, K., Ranzato, M., and LeCun, Y. (2009). "What is the best multi-stage architecture for object recognition?," in Proceedings of International Conference on Computer Vision (ICCV'09) (Kyoto: IEEE).
20. LeCun, Y., Boser, B., Denker, J. S., Henderson, D., Howard, R. E., Hub-bard, W., and Jackel, L. D. (1990). "Handwritten digit recognition with a back-propagation network," In NIPS'89, MIT Press, Denver.
21. LeCun, Y., Bottou, L., Bengio, Y., and Haffner, P. (1998a). Gradient-based learning applied to document recognition. Proceedings of the IEEE 86, 2278-2324.
22. LeCun, Y., Bottou, L., Orr, G., and Muller, K. (1998b). "Efficient back-prop," in Neural Networks: Tricks of the Trade, eds G. Orr, and K. Muller (Springer).
23. Leñero-Bardallo, J. A., Serrano-Gotarredona, T., and Linares-Barranco, B. (2010). A five-decade dynamic-range ambient-light-independent calibrated signed-spatial-contrast AER retina with 0.1ms latency and optional time-to-first-spike mode. IEEE Trans. Circuits Syst. I Reg. Papers 57, 2632-2643.
24. Leñero-Bardallo, J. A., Serrano-Gotarredona, T., and Linares-Barranco, B. (2011). A 3.6μs latency asynchronous frame-free event-based dynamic vision sensor. IEEE J. Solid State Circuits 46,1443-1455.
25. Lichtsteiner, P., Posch, C., and Delbrück, T. (1998). A 128Ã- 128 120db 15us latency asynchronous temporal contrast vision sensor. IEEEJ. Solid State Circuits 43, 566-576.
26. Lyu, S., and Simoncelli, E. P. (2008). "Nonlinear image representation using divisive normalization," in Computer Vision and Pattern Recognition, IEEE, Anchorage.
27. Massari, N., Gottardi, M., Jawed, S. A., and Soncini, G. (2008). A 100uw 64xl28-pixel contrast-based asynchronous binary vision sensor for wireless sensor networks. IEEE ISSCCDig. Tech. Papers 588-638.
28. Neubauer, C. (1998). Evaluation of convolution neural networks for visual recognition. IEEE Trans. Neural Netw. 9, 685-696.
29. Osadchy, R., Miller, M., and LeCun, Y. (2005). "Synergistic face detection and pose estimation with energy-based model," in Advances in Neural Information Processing Systems (NIPS 2004) (Vancouver: MIT Press).
30. Oster, M., Yingxue, W., Douglas, R., and Shih-Chii, L. (2008). Quantification of a spike-based winner-take-all vlsi network. IEEE Trans. Circuits. Syst. Part 1 55,3160-3169.
31. Pérez-Carrasco, J. A. (2011). A Simulation Tool for Building and Analyzing Complex and Hierarchically Structured AER Visual Processing Systems. Ph.D. thesis, IMSE-CNM-CSIC, Universidad de Sevilla, Sevilla.
32. Pinto, N., Cox, D. D., and DiCarlo, J. J. (2008). Why is real-world visual object recognition hard? PLoS Com-put. Biol. 4, e27. doi:10.1371/jour-nal.pcbi
33. Posch, C., Matolin, D., and Wohlge-nannt, R. (2010). "A QVGA 143dB DR asynchronous address-event PWM dynamic vision and image sensor with lossless pixel-level video compression and time-domain CDS," in ISSCC Digest of Technical Papers, San Francisco, in press.
34. Ranzato, M., Huang, F., Boureau, Y., and LeCun, Y. (2007). "Unsupervised learning of invariant feature hierarchies with applications to object recognition," in Proceedings of Computer Vision and Pattern Recognition Conference (CVPR'07) (Minneapolis: IEEE Press).
35. Ruedi, P. F., Heim, P., Gyger, S., Kaess, F., Arm, C., Caseiro, R., Nagel, J.-L., and Todeschini, S. (2009). "An soc combining a 132db qvga pixel array and a 32b dsp/mcu processor for vision applications," in IEEE ISSCC Digest of Technical Papers, San Francisco, 46-47, 47a.
36. Ruedi, P. F., Heim, P., Kaess, F., Grenet, E., Heitger, F., Burgi, P.-Y., Gyger, S., and Nussbaum, P. (2003). A 128x128, pixel 120-db dynamic-range vision-sensor chip for image contrast and orientation extraction. IEEE J. Solid State Circuits 38, 2325-2333.
37. Säckinger, E., Boser, B., Bromley, J., LeCun, Y., and Jackel, L. D. (1992). Application of the ANNA neural network chip to high-speed character recognition. IEEE Trans. Neural Netw. 3, 498-505.
38. Serrano-Gotarredona, R., Oster, M., Lichtsteiner, P., Linares-Barranco, A., Paz-Vicente, R., Gómez-Rodríguez, F., Camuñas-Mesa, L., Berner, R., Rivas-Pérez, M., Delbrück, T., Liu, S.-C., Douglas, R., Häfliger, P., Jiménez-Moreno, G., Ballcels, A. C., Serrano-Gotarredona, T., Acosta-Jiménez, A. J., and Linares-Barranco, B. (2009). CAVIAR: a 45k neuron, 5M synapse, 12G connects/s AER hardware sensory-processing-learning-actuating system for highspeed visual object recognition and tracking. IEEE Trans. Neural Netw. 20,1417-1438.
39. Serrano-Gotarredona, R., Serrano-Gotarredona, T., Acosta-Jiménez, A., and Linares-Barranco, B. (2006). A neuromorphic cortical-layer microchip for spike-based event processing vision systems. IEEE Trans. Circuits Syst. I Regul. Papers 53, 2548-2566.
40. Serrano-Gotarredona, R., Serrano-Gotarredona, T., Acosta-Jiménez, A., Serrano-Gotarredona, C, Perez-Carrasco, J. A., Linares-Barranco, B., Linares-Barranco, A., Jiménez-Moreno, G., and Civit-Ballcels, A. (2008). On real-time AER 2-D convolution hardware for neuromorphic spike-based cortical processing. IEEE Trans. Neural Netw. 19,1196-1219.
41. Serrano-Gotarredona, T., Andreou, A. G., and Linares-Barranco, B. (1999). AER image filtering architecture for vision processing systems. IEEE Trans. Circuits Syst. Part I Fundam. Theory Appl. 46, 1064-1071.
42. Serre, T. (2006). Learning a Dictionary of Shape-Components in Visual Cortex: Comparison with Neurons, Humans and Machines. Ph.D. thesis, MIT, Boston.
43. Shepherd, G. (1990). TheSynaptic Organization of the Brain, 3rd Edn. Oxford: Oxford University Press.
44. Sivilotti, M. A. (1991). "Wiring considerations in analog VLSI systems, with application to field-programmable networks," in Technical Report, California Institute of Technology, Pasadena.
45. Teixeira, T., Culurciello, E., and Andreou, A.G. (2006). "An address-event image sensor network," in IEEE International Symposium on Circuits and Systems, ISCAS '06 (Kos: IEEE), 4467-4470.
46. Thorpe, S., Fize, D., and Marlot, C. (1996). Speed of processing in the human visual system. Nature 381, 520-522.
47. Vernier, P., Mortara, A., Arreguit, X., and Vittoz, E. A. (1997). An integrated cortical layer for orientation enhancement. IEEE J. Solid State Circuits 32, 177-186.
48. Vogelstein, R. J., Mallik, U., Culurciello, E., Cauwenberghs, G., and Etienne-Cummings, R. (2007). A multi-chip neuromorphic system for spike-based visual information processing. Neural Comput. 19,2281-2300.
49. Zaghloul, K. A., and Boahen, K. (2004). Optic nerve signals in a neuromorphic chip: parts 1 and 2. IEEE Trans.Biomed. Eng. 51, 657-675.
50. Zamarreño-Ramos, C. (2011). Towards Modular and Scalable High-Speed AER Vision Systems. Ph.D. thesis, IMSE-CNM-CSIC, Universidad de Sevilla, Sevilla.
51. Zamarreño-Ramos, C., Serrano-Gotarredona, T., Linares-Barranco, B., Kulkarni, R., and Silva-Martinez, J. (2011a). "Voltage mode driver for low power transmission of high speed serial aer links," in Proceedings of IEEE International Symposium on Circuits and Systems (ISCAS 2011) (Rio de Janeiro), 2433-2436.
52. Zamarreño-Ramos, C., Serrano-Gotarredona, T., and Linares-Barranco, B. (2011b). An instant-startup jitter-tolerant manchester-encoding serializer/deserializar scheme for event-driven bit-serial lvds inter-chip aer links. IEEE Trans. Circuits Syst. Part I 58, 2647-2660.