Fast vision through frameless event-based sensing and convolutional processing: Application to texture recognition

IEEE Transactions on Neural Networks

Volumn 21, Issue 4, 2010, Pages 609-620

  Perez Carrasco, José Antonio ^a,b   Acha, Begoña ^a   Serrano, Carmen ^a   Camunas Mesa, Luis ^b   Serrano Gotarredona, Teresa ^b   Linares Barranco, Bernabé ^b  

^a UNIVERSITY OF SEVILLE   (Spain)

^b INSTITUTO DE MICROELECTRÓNICA DE SEVILLA   (Spain)

Author keywords

Address event representation (AER); AER chips; Convolutional neural networks; Event coding and processing; Real time vision hardware processing; Spike signal processing; Texture retrieval; Vision chips

Indexed keywords

ADDRESS-EVENT REPRESENTATION; ADDRESS-EVENT REPRESENTATION (AER); AER CHIPS; CONVOLUTIONAL NEURAL NETWORK; CONVOLUTIONAL NEURAL NETWORKS; REAL TIME VISION; TEXTURE RETRIEVAL; VISION CHIPS;

COMPUTER SOFTWARE; CONVOLUTION; HARDWARE; SIGNAL PROCESSING; TEXTURES; VISUAL COMMUNICATION;

NEURAL NETWORKS;

ARTICLE; ARTIFICIAL NEURAL NETWORK; HUMAN; IMAGE PROCESSING; INSTRUMENTATION; METHODOLOGY; PATTERN RECOGNITION; PHYSIOLOGY; SIGNAL PROCESSING; TIME; VISION;

FORM PERCEPTION; HUMANS; IMAGE PROCESSING, COMPUTER-ASSISTED; NEURAL NETWORKS (COMPUTER); SIGNAL PROCESSING, COMPUTER-ASSISTED; TIME FACTORS; VISION, OCULAR;

EID: 77950860651     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2009.2039943     Document Type: Article

References (74)

1. G. M. Shepherd, The Synaptic Organization of the Brain, 3rd ed. New York: Oxford Univ. Press, 1990.
2. E. T. Rolls and G. Deco, Computational Neuroscience of Vision. New York: Oxford Univ. Press, 2002.
3. T. Serre, L.Wolf, S. Bileschi, M. Riesenhuber, and T. Poggio, "Robust object recognition with cortex-like mechanisms," IEEE Trans. Pattern Anal. Mach. Intell., vol.29, no.3, pp. 411-426, Mar. 2007.
4. R. DeValois, D. Albrecht, and L. Thorell, "Spatial frequency selectivity of cells in macaque visual cortex," Vis. Res., vol.22, pp. 545-559, 1982.
5. R. DeValois, E. Yund, and N. Hepler, "The orientation and direction selectivity of cells in macaque visual cortex," Vis. Res., vol.22, pp. 531-544, 1982.
6. P. H. Schiller, B. L. Finlay, and S. F. Volman, "Quantitative studies of single-cell properties in monkey striate cortex. Spatial frequency," J. Neurophysiol., vol.39, no.6, pp. 1334-1351, 1976.
7. S. Grossberg, E. Mingolla, and J.Williamson, "Synthetic aperture radar processing by a multiple scale neural system for boundary and surface representation," Neural Netw., vol. 8, no. 7/8, pp. 1005-1028, 1995.
8. K. Fukushima and N. Wake, "Handwritten alphanumeric character recognition by the neocognitron," IEEE Trans. Neural Netw., vol.2, no.3, pp. 355-365, May 1991.
9. Y. LeCun and Y. Bengio, "Convolutional networks for images, speech, and time series," in The Handbook of Brain Science and Neural Networks, M. Arbib, Ed. Cambridge, MA: MIT Press, 1995, pp. 255-258.
10. Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-based learning applied to document recognition," Proc. IEEE, vol.86, no.11, pp. 2278-2324, Nov. 1998.
11. C. Neubauer, "Evaluation of convolution neural networks for visual recognition," IEEE Trans. Neural Netw., vol.9, no.4, pp. 685-696, Jul. 1998.
12. S. Lawrence, C. L. Giles, A. Tsoi, and A. Back, "Face recognition: A convolutional neural network approach," IEEE Trans. Neural Netw., vol.8, no.1, pp. 98-113, Jan. 1997.
13. M. Sivilotti, "Wiring considerations in analog VLSI systems with application to field-programmable networks," Ph.D. dissertation, Comput. Sci. Div., California Inst. Technol., Pasadena, CA, 1991.
14. J. Lazzaro, J. Wawrzynek, M. Mahowald, M. Silvilotti, and D. Gillespie, "Silicon auditory processors as computer peripherals," IEEE Trans. Neural Netw., vol.4, no.3, pp. 523-528, May 1993.
15. E. Culurciello, R. Etienne-Cummings, and K. A. Boahen, "A biomorphic digital image sensor," IEEE J. Solid-State Circuits, vol.38, no.2, pp. 281-294, Feb. 2003.
16. P. F. Rüedi, P. Heim, F. Kaess, E. Grenet, F. Heitger, P.-Y. Burgi, S. Gyger, and P. Nussbaum, "A 128 × 128, pixel 120-dB dynamic-range vision-sensor chip for image contrast and orientation extraction," IEEE J. Solid-State Circuits, vol.38, no.12, pp. 2325-2333, Dec. 2003.
17. C. Shoushun and A. Bermak, "A low power CMOS imager based on time-to-first-spike encoding and fair AER," in Proc. IEEE Int. Symp. Circuits Syst., 2005, pp. 5306-5309.
18. M. Azadmehr, J. Abrahamsen, and P. Häfliger, "A foveated AER imager chip," in Proc. IEEE Int. Symp. Circuits Syst., Kobe, Japan, 2005, pp. 2751-2754.
19. K. A. Zaghloul and K. boahen, "Optic nerve signals in a neuromorphic chip: Part I and II," IEEE Trans. Biomed. Eng., vol.51, no.4, pp. 657-675, Apr. 2004.
20. J. Costas-Santos, T. Serrano-Gotarredona, R. Serrano-Gotarredona, and B. Linares-Barranco, "A contrast retina with on-chip calibration for neuromorphic spike-based AER vision systems," IEEE Trans. Circuits Syst. I, Reg. Papers, vol.54, no.7, pp. 1444-1458, Jul. 2007.
21. P. Lichtsteiner, C. Posch, and T. Delbruck, "A 128 × 128 120 dB 30 mW asynchronous vision sensor that responds to relative intensity change," IEEE J. Solid-State Circuits, vol.43, no.2, pp. 566-576, Feb. 2008.
22. K. Boahen, "Retinomorphic chips that see quadruple images," in Proc. Int. Conf. Microelectron. Neural Fuzzy Bio-Inspired Syst., Granada, Spain, 1999, pp. 12-20.
23. G. Cauwenberghs, N. Kumar, W. Himmelbauer, and A. G. Andreou, "An analog VLSI chip with asynchronous interface for auditory feature extraction," IEEE Trans. Circuits Syst. II, Analog Digit. Signal Process., vol.45, no.5, pp. 600-606, May 1998.
24. E. Chicca, A. M. Whatley, P. Lichtsteiner, V. Dante, T. Delbruck, P. Del Giudice, R. J. Douglas, and G. Indiveri, "A multichip pulse-based neuromorphic infrastructure and its application to a model of orientation selectivity," IEEE Trans. Circuits Syst. I, Reg. Papers, vol.54, no.5, pp. 981-993, May 2007.
25. M. Oster, Y. Wang, R. Douglas, and S.-C. Liu, "Quantification of a spike-based winner-take-all VLSI network," IEEE Trans. Circuits Syst. I, Reg. Papers, vol.55, no.10, pp. 3160-3169, Nov. 2008.
26. T. Teixeira, A. G. Andreou, and E. Culurciello, "Event-based imaging with active illumination in sensor networks," in Proc. IEEE Int. Symp. Circuits Syst., Kobe, Japan, 2005, pp. 644-647.
27. R. Serrano-Gotarredona, T. Serrano-Gotarredona, A. Acosta-Jiménez, and B. Linares-Barranco, "A neuromorphic cortical layer microchip for spike based event processing vision systems," IEEE Trans. Circuits Syst. I, Reg. Papers, vol.53, no.12, pp. 2548-2566, Dec. 2006.
28. R. Serrano-Gotarredona, T. Serrano-Gotarredona, A. Acosta-Jimenez, C. Serrano-Gotarredona, J. A. Perez-Carrasco, A. Linares-Barranco, G. Jimenez-Moreno, A. Civit-Ballcels, and B. Linares-Barranco, "On real-time AER 2D convolutions hardware for neuromorphic spike based cortical processing," IEEE Trans. Neural Netw., vol.19, no.7, pp. 1196-1219, Jul. 2008.
29. R. Serrano-Gotarredona, M. Oster, P. Lichtsteiner, A. Linares-Barranco, R. Paz-Vicente, F. Gómez-Rodríguez, L. Camuñas-Mesa, R. Berner, M. Rivas, T. Delbrück, S. C. Liu, R. Douglas, P. Häfliger, G. Jiménez-Moreno, A. Civit, T. Serrano-Gotarredona, A. Acosta- Jiménez, and B. Linares-Barranco, "CAVIAR: A 45 k-Neuron, 5 M-Synapse, 12 G-connects/sec AER hardware sensory- processing-learning-actuating system for high speed visual object recognition and tracking," IEEE Trans. Neural Netw., vol.20, no.9, pp. 1417-1438, Sep. 2009.
30. S. Vangal, J. Howard, G. Ruhl, S. Dighe, H. Wilson, J. Tschanz, D. Finan, P. Iyer, A. Singh, T. Jacob, S. Jain, S. Venkataraman, Y. Hoskote, and N. Borkar, "An 80-Tile 1.28 TFLOPS network-on-chip in 65 nm CMOS," in Proc. IEEE Int. Solid-State Circ. Conf., Feb. 2007, pp. 98-99.
31. L. Camuñas-Mesa, A. Acosta-Jiménez, T. Serrano-Gotarredona, and B. Linares-Barranco, "A fully digital event-based convolution processor chip for fast frame-less vision processing," IEEE Trans. Circuits Systems, submitted for publication.
32. F. Gomez-Rodriguez, R. Paz, A. Linares-Barranco, M. Rivas, L. Miro, S. Vicente, G. Jimenez, and A. Civit, "AER tools for communications and debugging," in Proc. IEEE Int. Symp. Circuits Syst., May 2006, DOI: 10.1109/ISCAS.2006.1693319.
33. J. A. Pérez-Carrasco, T. Serrano-Gotarredona, C. Serrano-Gotarredona, B. Acha, and B. Linares-Barranco, "On the computational power of address-event representation (AER) vision processing hardware," in Proc. XXII Int. Conf. Design Circuits Integrated Syst., Sevilla, Spain, Nov. 21-23, 2007.
34. A. Delorme, L. Perrinet, and S. J. Thorpe, "Networks of integrateand- fire neurons using rank order coding B: Spike timing dependent plasticity and emergence of orientation selectivity," Neurocomputing, vol.38-40, pp. 539-545, 2001. (Pubitemid 35276680)
35. B. S. Manjunath and W. Y. Ma, "Texture features for browsing and retrieval of image data," IEEE Trans. Pattern Anal. Mach. Intell., vol.18, no.8, pp. 837-842, Aug. 1996.
36. K. Boahen, "Point-to-point connectivity between neuromorphic chips using address events," IEEE Trans. Circuits Syst. II, Analog Digit. Signal Process., vol.47, no.5, pp. 416-434, May 2000.
37. T. Serrano-Gotarredona, A. G. Andreou, and B. Linares-Barranco, "AER image filtering architecture for vision-processing systems," IEEE Trans. Circuits Syst. I, Fundam. Theory Appl., vol.46, no.9, pp. 1064-1071, Sep. 1999.
38. D. H. Goldberg, G. Cauwenberghs, and A. G. Andreou, "Probabilistic synaptic weighting in a reconfigurable network of VLSI integrate-and-fire neurons," Neural Netw., vol.14, pp. 781-793, 2001.
39. A. Linares-Barranco, G. Jimenez-Moreno, B. Linares-Barranco, and A. Civit-Ballcels, "On algorithmic rate-coded AER generation," IEEE Trans. Neural Netw., vol.17, no.3, pp. 771-788, May 2006.
40. L. Chen, G. Lu, and D. Zhang, "Effects of different Gabor filter parameters on image retrieval by texture," in Proc. 10th Int. Multimedia Model. Conf., 2004, pp. 273-278.
41. P. Brodatz, Textures: A Photographic Album for Artists&Designers. New York: Dover, 1966.
42. T. Randen and J. H. HusZy, "Filtering for texture classification:A comparative study," IEEE Trans. Pattern Anal. Mach. Intell., vol.21, no.4, pp. 291-310, Apr. 1999.
43. R. M. Haralick, "Statistical and structural approaches to texture," Proc. IEEE, vol.67, no.5, pp. 786-804, May 1979.
44. G. V. D.Wouwer, P. Scheunders, and D. Van Dyck, "Statistical texture characterization from discrete wavelet representations," IEEE Trans. Image Process., vol.8, no.4, pp. 592-598, Apr. 1999.
45. G. R. Cross and A. K. Jain, "Markov random field texture models," IEEE Trans. Pattern Anal. Mach. Intell., vol.PAMI-5, no.1, pp. 25-39, Jan. 1983.
46. R. L. Kashyap and R. Chellappa, "Estimation and choice of neighbors in spatial-interaction models of images," IEEE Trans. Inf. Theory, vol.IT-29, no.1, pp. 60-72, Jan. 1983.
47. R. M. Haralick, K. Shanmugan, and I. Dinstein, "Texture features for image classification," IEEE Trans. Syst. Man Cybern., vol.SMC-3, no.6, pp. 610-621, Nov. 1973.
48. A. Speis and G. Healey, "Feature extraction for texture discrimination via random field models with random spatial interaction," IEEE Trans. Image Process., vol.5, no.4, pp. 635-645, Apr. 1996.
49. T. Chang and C.-C. J. Kuo, "Texture analysis and classification with trees-tructured wavelet transform," IEEE Trans. Image Process., vol.2, no.4, pp. 429-441, Apr. 1993.
50. M. Unser, "Texture classification and segmentation using wavelet frames," IEEE Trans. Image Process., vol.4, no.11, pp. 1549-1560, Nov. 1995.
51. G. M. Haley and B. S. Manjunath, "Rotation-invariant texture classification using a complete space-frequency model," IEEE Trans. Image Process., vol.8, no.2, pp. 255-269, Feb. 1999.
52. W. T. Freeman and E. H. Adelson, "The design and use of steerable filters," IEEE Trans. Pattern Anal. Mach. Intell., vol.13, no.9, pp. 891-906, Sep. 1991.
53. J. G. Rosiles and M. J. T. Smith, "Texture classification with a biorthogonal directional filter bank," in Proc. IEEE Int. Conf. Acoust. Speech Signal Process., May 2001, pp. 1549-1552.
54. J. Han 1 and K.-K. Ma, "Rotation-invariant and scale-invariant Gabor features for texture image retrieval," Image Vis. Comput., vol.25, no.9, pp. 1474-1481, Sep. 2007.
55. T. Sikora, "The MPEG-7 visual standard for content description-An overview," IEEE Trans. Circuits Syst. Video Technol., vol.11, no.6, pp. 696-702, Jun. 2001.
56. J. J. Kulikowski and P. O. Bishop, "Fourier analysis and spatial representation in the visual cortex," Experientia, vol.37, pp. 160-163, 1981.
57. D. A. Clausi and H. Deng, "Design-based texture feature fusion using Gabor filters and co-occurrence probabilities," IEEE Trans. Image Process., vol.14, no.7, pp. 925-936, Jul. 2005.
58. D. A. Clausi, "Comparison and fusion of co-occurrence, Gabor and MRF texture features for classification of SAR sea-ice imagery," Atmos. Oceans, vol.39, no.4, pp. 183-194, 2001.
59. S. Li and J. Shawe-Taylor, "Comparison and fusion of multiresolution features for texture classification," Pattern Recognit. Lett., vol.26, pp. 633-638, 2005.
60. N. Qaiser,M. Hussain, A. Hussain, and N. Qaiser, "Texture recognition by fusion of optimized moment based and Gabor energy features," Int. J. Comput. Sci. Network Security, vol.8, no.2, pp. 264-270, Feb. 2008.
61. C.-C. Chen and C.-C. Chen, "Filtering methods for texture discrimination," Pattern Recognit. Lett., vol.20, pp. 783-790, 1999.
62. R. Picard, T. Kabir, and F. Liu, "Real-time recognition with the entire Brodatz texture database," in Proc. Comput. Vis. Pattern Recognit., 1993, pp. 638-639.
63. P. P. Ohanian and R. C. Dubes, "Performance evaluation for four classes of textural features," Pattern Recognit., vol.25, no.8, pp. 819-833, 1992.
64. K.-O. Cheng, N.-F. Law, and W.-C. Siu, "A novel fast and reduced redundancy structure for multiscale directional filter banks," IEEE Trans. Image Process., vol.16, no.8, pp. 2058-2068, Aug. 2007.
65. S. Lazebnik, C. Schmid, and J. Ponce, "A sparse texture representation using local affine regions," Proc. Comput. Vis. Pattern Recognit., vol.2, pp. 319-324, 2003.
66. M. Mellor, B.-W. Hong, and M. Brady, "Locally rotation, contrast, and scale invariant descriptors for texture analysis," IEEE Trans. Pattern Anal. Mach. Intell., vol.30, no.1, pp. 52-61, Jan. 2008.
67. M. Kokare, P. K. Biswas, and B. N. Chatterji, "Rotation-invariant texture image retrieval using rotated complex wavelet filters," IEEE Trans. Syst. Man Cybern. B, Cybern., vol.36, no.6, pp. 1273-1282, Dec. 2006.
68. K.-O. Cheng, N.-F. Law, and W.-C. Siu, "Multiscale directional filter bank with applications to structured and random texture retrieval," Pattern Recognit., vol.40, no.4, pp. 1182-1194, 2007.
69. M. Pi and H. Li, "Fractal indexing with the joint statistical properties and its application in texture image retrieval," IET Image Process., vol.2, no.4, pp. 218-230, 2008.
70. M. H. Pi, C. S. Tong, S. K. Choy, and H. Zhang, "A fast and effective model for wavelet subband histograms and its application in texture image retrieval," IEEE Trans. Image Process., vol.15, no.10, pp. 3078-3088, Oct. 2006.
71. M. N. Do and M. Vetterli, "Pyramidal directional filter banks and curvelets," in Proc. IEEE Int. Conf. Image Process., Oct. 2001, vol.3, pp. 158-161.
72. M. N. Do and M. Vetterli, "The contourlet transform: An efficient directional multiresolution image representation," IEEE Trans. Image Process., vol.14, no.12, pp. 2091-2106, Dec. 2005.
73. G. M. Haley and B. S. Manjunath, "Rotation-invariant texture classification using a complete space-frequency model," IEEE Trans. Image Process., vol.8, no.2, pp. 255-269, Feb. 1999.
74. E. P. Simoncelli and W. T. Freeman, "The steerable pyramid:Aflexible architecture for multi-scale derivative computation," in Proc. Int. Conf. Image Process., Oct. 1995, pp. 444-447.