A neuromorphic cortical-layer microchip for spike-based event processing vision systems

IEEE Transactions on Circuits and Systems I: Regular Papers

Volumn 53, Issue 12, 2006, Pages 2548-2566

  Serrano Gotarredona, Rafael ^a   Serrano Gotarredona, Teresa ^a   Acosta Jiménez, Antonio ^a   Linares Barranco, Bernabé ^a  

^a UNIVERSITY OF SEVILLE   (Spain)

Author keywords

2 D convolutions; Address event representation (AER); Bio inspired systems; Digitally calibrated analog circuits; High speed signal processing; MOS transistor mismatch; Spike based processing; Subthreshold circuits; Vision; VLSI mixed circuit design

Indexed keywords

CONVOLUTIONAL CODES; IMAGE QUALITY; MOSFET DEVICES; REAL TIME SYSTEMS; VIDEO SIGNAL PROCESSING; VLSI CIRCUITS;

ADDRESS EVENT REPRESENTATION; ANALOG CIRCUITS; BIO-INSPIRED SYSTEMS; MIXED CIRCUIT DESIGN; SUBTHRESHOLD CIRCUITS;

MICROPROCESSOR CHIPS;

EID: 33845785313     PISSN: 10577122     EISSN: None     Source Type: Journal    
DOI: 10.1109/TCSI.2006.883843     Document Type: Article

References (78)

1. H. Fujii, H. Ito, K. Aihara, N. Ichinose, and M. Tsukada, "Dynamical cell assembly hypothesis-theoretical possibility of spatio-temporal coding in the cortex," Neural Netw., vol. 9, pp. 1303-1350, 1996.
2. G. A. Orban, Neural Operations in the Visual Cortex. Berlin, Germany: Springer-Verlag, 1984.
3. M. Shadlen and W. T. Newsome, "Noise, neural codes and cortical organization," Current Opinion Neurobiol., vol. 4, pp. 569-579, 1994.
4. G. M. Shepherd, The Synaptic Organization of the Brain, 3rd ed. Oxford, U.K.: Oxford Univ. Press, 1990.
5. K. Fukushima, "Visual feature extraction by a multilayered network of analog threshold elements," IEEE Trans. Syst. Sci. Cybern., vol. SC-5, no. 4, pp. 322-333, Oct. 1969.
6. K. Fukushima and S. Miyake, "Neocognitron: A new algorithm for pattern recognition tolerant of deformations and shifts in position," Pattern Recogn., vol. 15, pp. 455-469, 1982.
7. K. Fukushima, "Neocognitron: A hierarchical neural network capable of visual pattern recognition," Neural Netw., vol. 1, pp. 119-130, 1988.
8. K. Fukushima, "Analysis of the process of visual pattern recognition by the neocognitron," Neural Netw., vol. 2, pp. 413-420, 1989.
9. Y. Le Cun and Y. Bengio, "Convolutional networks for images, speech, and time series," in Handbook of Brain Theory and Neural Networks, M. A. Arbib, Ed. Cambridge, MA: MIT Press, 1995, pp. 255-258.
10. C. Neubauer, "Evaluation of convolution neural networks for visual recognition," IEEE Trans. Neural Netw., vol. 9, no. 4, pp. 685-696, Jul. 1998.
11. M. Matsugu, K. Mori, M. Ishi, and Y. Mitarai, "Convolutional spiking neural network model for robust face detection," in Proc. 9th Int. Conf. Neural Inf. Process. (ICONIP'02), 2002, vol. 2, pp. 660-664.
12. B. Fasel, "Robust face analysis using convolutional neural networks," in Proc. Int. Conf. Pattern Recogn. (ICPR'02), 2002, pp. 40-43.
13. M. Browne and S. S. Ghidary, "Convolutional neural networks for image processing: An application in robot vision," in Advances Artif. Intell.: Proc. 16th Australian Conf. on AI, Nov. 2003, pp. 641-652.
14. R. Serrano-Gotarredona, T. Serrano-Gotarredona, A. Acosta-Jiménez, and B. Linares-Barranco, "An arbitrary kernel convolution AER-transceiver chip for real-time image filtering," in Proc. IEEE Int. Conf. Circuits Syst. (ISCAS'06), 2006, pp. 3145-3148.
15. M. Sivilotti, "Wiring considerations in analog VLSI systems with application to field-programmable networks," Ph.D. dissertation, Comp. Sci. Div., California Inst. Technol., Pasadena, CA, 1991.
16. M. Mahowald, "VLSI analogs of neural visual processing: A synthesis of form and function," Ph.D. dissertation, Comp. Sci. Div., California Inst. Technol., Pasadena, CA, 1992.
17. M. Mahowald, An Analog VLSI Stereoscopic Vision System. Boston, MA: Kluwer, 1994.
18. J. Lazzaro, J. Wawrzynek, M. Mahowald, M. Silvilotti, and D. Gillespie, "Silicon auditory processors as computer peripherals," IEEE Trans. Neural Netw,, vol. 4, no. 3, pp. 523-528, May 1993.
19. J. P. Lazzaro and J. Wawrzynek, "A multi-sender asynchronous extension to the address-event protocol," in Proc. 16th Conf. Advanced Res. VLSI, W. J. Dally, J. W. Poulton, and A. T. Ishii, Eds., 1995, pp. 158-169.
20. A. Mortara and E. A. Vittoz, "A communication architecture tailored for analog VLSI artificial neural networks: Intrinsic performance and limitations," IEEE Trans. Neural Netw., vol. 5, no. 3, pp. 459-466, Jul. 1994.
21. A. Mortara, E. A. Vittoz, and P. Venier, "A communication scheme for analog VLSI perceptive systems," IEEE J. Solid-State Circuits, vol. 30, no. 6, pp. 660-669, Jun. 1995.
22. Z. Kalayjian and A. G. Andreou, "Asynchronous communication of 2-D motion information using winner-takes-all arbitration," Int. J. Analog Integr. Circuits Signal Proc., vol. 13, no. 1-2, pp. 103-109, Mar./Apr. 1997.
23. K. A. Boahen, "Communicating neuronal ensembles between neuromorphic chips," in Neuromorphic Systems Engineering: Neural Networks in Silicon, T. S. Lande, Ed. Norwell, MA: Kluwer, 1998, ch. 11.
24. K. A. Boahen, "Point-to-point connectivity between neuromorphic chips using address events," IEEE Trans. Circuits Syst. II, Analog Digit. Signal Process., vol. 47, no. 5, pp. 416-434, May 2000.
25. K. A. Boahen, "A burst-mode word-serial address-event link - Part I: Transmitter design," IEEE Trans. Circuits Syst. I, Reg. Papers, vol. 51, no. 7, pp. 1269-1280, Jul. 2004.
26. K. A. Boahen, "A burst-mode word-serial address-event link - Part II: Receiver design," IEEE Trans. Circuits Syst. I, Reg. Papers, vol. 51, no. 7, pp. 1281-1291, Jul. 2004.
27. K. A. Boahen, "A burst-mode word-serial address-event link - Part III: Analysis and test results," IEEE Trans. Circuits Syst. I, Reg. Papers, vol. 51, no. 7, pp. 1292-1300, Jul. 2004.
28. E. Culurciello, R. Etienne-Cummings, and K. A. Boahen, "A biomorphic digital image sensor," IEEE J. Solid-State Circuits, vol. 38, no. 2, pp. 281-294, Feb. 2003.
29. P. F. Ruedi et al., "A 128 × 128, pixel 120-dB dynamic-range vision-sensor chip for image contrast and orientation extraction," IEEE J. Solid-State Circuits, vol. 38, no. 12, pp. 2325-2333, Dec. 2003.
30. M. Barbaro, P. Y. Burgi, A. Mortara, P. Nussbaum, and F. Heitger, "A 100 × 100 pixel silicon retina for gradient extraction with steering filter capabilities and temporal output coding," IEEE J. Solid-State Circuits, vol. 37, no. 2, pp. 160-172, Feb. 2002.
31. C. Shoushun and A. Bermak, "A low power CMOS imager based on time-to-first-spike encoding and fair AER," in Proc. IEEE Int. Symp. Circuits Syst. (ISCAS'05), 2005, pp. 5306-5309.
32. X. G. Qi, X., and J. Harris, "A time-to-first-spike CMOS imager," in Proc. IEEE Int. Symp. Circuits Syst. (ISCAS'04), Vancouver, BC, Canada, 2004, pp. 824-827.
33. M. Azadmehr, J. Abrahamsen, and P. Häfliger, "A foveated AER imager chip," in Proc. IEEE Int. Symp. Circuits Syst. (ISCAS'05), Kobe, Japan, 2005, pp. 2751-2754.
34. J. Kramer, "An on/off transient imager with event-driven, asynchronous read-out," in Proc. IEEE Int. Symp. Circuits Syst. (ISCAS'02), Phoenix, AZ, 2002, pp. 165-168.
35. P. Lichtsteiner, T. Delbrück, and J. Kramer, "Improved on/off temporally differentiating address-event imager," in Proc. IEEE Int. Symp. Circuits Syst. (ISCAS'04), Vancouver, BC, Canada, 2004, pp. 211-214.
36. P. Lichtsteiner, C. Posch, and T. Delbruck, "A 128 × 128 120 dB 30 mW asynchronous vision sensor that responds to relative intensity change," in IEEE ISSCC Dig. Tech. Papers, San Francisco, CA, 2006, pp. 508-509.
37. M. Arias-Estrada, D. Poussart, and M. Tremblay, "Motion vision sensor architecture with asynchronous self-signaling pixels," in Proc. 7th Int. Workshop Comput. Architecture Machine Perception (CAMP'97), 1997, pp. 75-83.
38. C. M. Higgins and S. A. Shams, "A biologically inspired modular VLSI system for visual measurement of self-motion," IEEE Sensors J., vol. 2, no. 6, pp. 508-528, Dec. 2002.
39. E. Özalevli and C. M. Higgins, "Reconfigurable biologically inspired visual motion system using modular neuromorphic VLSI chips," IEEE Trans. Circuits Syst. I, Reg. Papers, vol. 52, no. 1, pp. 79-92, Jan. 2005.
40. G. Indiveri, A. M. Whatley, and J. Kramer, "A reconfigurable neuromorphic VLSI multi-chip system applied to visual motion computation," in Proc. Int. Conf. Microelectron. Neural, Fuzzy Bio-Inspired Syst. (Microneuro'99), Granada, Spain, 1999, pp. 37-44.
41. K. Boahen, "Retinomorphic chips that seez quadruple images," in Proc. Int. Conf. Microelectron. Neural, Fuzzy Bio-Inspired Syst. (Microneuro'99), Granada, Spain, 1999, pp. 12-20.
42. R. Z. Shi and T. K. Horiuchi, "A VLSI model of the bat dorsal nucleus of the lateral lemniscus for azimuthal echolocation," in Proc. IEEE Int. Symp. Circuits Syst. (ISCAS'05), Kobe, Japan, 2005, pp. 4217-4220.
43. A. van Schaik and S.-C. Liu, "AER EAR: A matched silicon cochlea pair with address event representation interface," in Proc. IEEE Int. Symp. Circuits Syst. (ISCAS'05), Kobe, Japan, 2005, pp. 4213-4216.
44. G. Cauwenberghs, N. Kumar, W. Himmelbauer, and A. G. Andreou, "An analog VLSI chip with asynchronous interface for auditory feature extraction," IEEE Trans. Circ. Syst. 11, Analog Digit. Signal Process., vol. 45, no. 5, pp. 600-606, May 1998.
45. M. Oster and S.-C. Liu, "Spiking inputs to a spiking winner-take-all circuit," in Advances in Neural Information Processing Systems, Y. Weiss, B. Schölkopf, and J. Platt, Eds. Cambridge, MA: MIT Press, 2006, vol. 18, pp. 1051-1058 Online. Available: http://books.nips.cc/ papers/files/nips18/NIPS2005_0521.pdf, (NIPS'06)
46. J. Abrahamsen, P. Häfliger, and T. S. Lande, "A time domain winner-take-all network of integrate-and-fire neurons," in Proc. IEEE Int. Symp. Circuits Syst. (ISCAS'04), Vancouver, BC, Canada, May 2004, vol. V, pp. 361-364.
47. E. Chicca, G. Indiveri, and R. J. Douglas, "An event-based VLSI network of integrate-and-fire neurons," in Proc. IEEE Int. Symp. Circuits Syst. (ISCAS'04), Vancouver, BC, Canada, 2004, vol. V, pp. 357-360.
48. T. Teixeira, A. G. Andreou, and E. Culurciello, "Event-based imaging with active illumination in sensor networks," in Proc. IEEE Int. Symp. Circuits Syst. (ISCAS'05), Kobe, Japan, 2005, pp. 644-647.
49. P. Häfliger, "Asynchronous event redirecting in bio-inspired communication," in Proc. ICECS01, 2001, pp. 87-90.
50. D. H. Goldberg, G. Cauwenberghs, and A. G. Andreou, "Probabilistic synaptic weighting in a reconfigurable network of VLSI integrate-and-fire neurons," Neural Netw., vol. 14, no. 6-7, pp. 781-793, 2001.
51. R. J. Vogelstein, U. Mallik, and G. Cauwenberghs, "Silicon spike-based synaptic array and address-event transceiver," in Proc. IEEE Int. Symp. Circuits Syst. (ISCAS'04), 2004, vol. 5, pp. 385-388.
52. R. J. Vogelstein, F. Tenore, R. Philipp, M. S. Adlerstein, D. H. Goldberg, and G. Cauwenberghs, "Spike tinting-dependent plasticity in the address domain," in Advances in Neural Information Processing Systems, S. Becker, S. Thrun, and K. Obermayer, Eds. Cambridge, MA: MIT Press, 2003, vol. 15.
53. R. J. Vogelstein, U. Mallik, G. Cauwenberghs, E. Culurciello, and R. Etienne-Cummings, "Saliency-driven image acuity modulation on a reconfigurable silicon array of spiking neurons," in Advances in Neural Information Processing Systems, L. K. Saul, Y. Weiss, and L. Bottou, Eds. Cambridge, MA: MIT Press, 2005, vol. 17, pp. 1457-1464.
54. U. Mallik, R. J. Vogelstein, E. Culurciello, R. Etienne-Cummings, and G. Cauwenberghs, "A real-time spike-domain sensory information processing system," in Proc. IEEE Int. Symp. Circuits Syst. (ISCAS'05), Kobe, Japan, 2005, pp. 1919-1922.
55. R. J. Vogelstein, U. Mallik, E. Culurciello, R. Etienne-Cummings, and G. Cauwenberghs, "Spatial acuity modulation of an address-event imager," in Proc. 11th IEEE Int. Conf. Electron., Circuits Syst. (ICECS '04), Dec. 2004, pp. 207-210.
56. S.-C. Liu, J. Kramer, G. Indiveri, T. Delbrück, and R. Douglas, "Orientation-selective aVLSI spiking neurons," Neural Netw., vol. 14, pp. 629-643, 2001.
57. P. Vernier, A. Mortara, X. Arreguit, and E. A. Vittoz, "An integrated cortical layer for orientation enhancement," IEEE J. Solid-State Circuits, vol. 32, no. 2, pp. 177-186, Feb. 1997.
58. T. Serrano-Gotarredona, A. G. Andreou, and B. Linares-Barranco, "AER image filtering architecture for vision processing systems," IEEE Trans. Circuits Syst. 11, Analog Digit. Signal Process., vol. 46, no. 9, pp. 1064-1071, Sep. 1999.
59. T. Y. W. Choi, B. E. Shi, and K. Boahen, "An ON-OFF orientation selective address event representation image transceiver chip," IEEE Trans. Circuits Syst. I, Reg. Papers, vol. 51, no. 2, pp. 342-353, Feb. 2004.
60. T. Y. W. Choi, P. A. Merolla, J. V. Arthur, K. A. Boahen, and B. E. Shi, "Neuromorphic implementation of orientation hypercolummns," IEEE Trans. Circuits. Syst. I, Reg. Papers, vol. 52, no. 6, pp. 1049-1060, Jun. 2005.
61. S. Grossberg, E. Mingolla, and J. Williamson, "Synthetic aperture radar processing by a multiple scale neural system for boundary and surface representation," Neural Netw., vol. 8, no. 7/8, pp. 1005-1028, 1995.
62. S. Grossberg, E. Mingolla, and W. D. Ross, "Visual brain and visual perception: How does the cortex do perceptual grouping?," Trends Neurosci., vol. 20, pp. 106-111, 1997.
63. E. Mingolla, W. Ross, and S. Grossberg, "A neural network for enhancing boundaries and surfaces in synthetic aperture radar images," Neural Netw., vol. 12, no. 3, pp. 499-511, 1999.
64. R. Serrano-Gotarredona, M. Oster, P. Lichtsteiner, A. Linares-Barranco, R. Paz-Vicente, F. Gómez-Rodríguez, H. Kolle Riis, T. Delbrück, S. C. Liu, S. Zahnd, A. M. Whatley, R. Douglas, P. Häfliger, G. Jimenez-Moreno, A. Civit, T. Serrano-Gotarredona, A. Acosta-Jiménez, and B. Linares-Barranco, "AER building blocks for multi-layers multi-chips neuromorphic vision systems," in Advances in Neural Information Processing Systems, Y. Weiss, B. Schölkopf, and J. Platt, Eds. Cambridge, MA: MIT Press, 2006, vol. 18, pp. 1217-1224 Online. Available: http://books.nips.cc/papers/files/nips18/NIPS2005_0268.pdf, (NIPS'06)
65. R. Serrano-Gotarredona, T. Serrano-Gotarredona, and B. Linares-Barranco, "Event generators for address event representation transmitters," in Proc. SPIE, Jun. 2005, vol. 5839, Bioengineered and Bioinspired Systems, pp. 148-159.
66. B. Linares-Barranco, T. Serrano-Gotarredona, R. Serrano-Gotarredona, and J. Costas-Santos, "A new charge-packet driven mismatch-calibrated integrate-and-fire neuron for processing positive and negative signals in AER based systems," in Proc. IEEE Int. Symp. Circuits Syst. (ISCAS'04), May 2004, vol. 5, pp. 744-747.
67. B. Linares-Barranco and T. Serrano-Gotarredona, "On the design and characterization of femtoampere current-mode circuits," IEEE J. Solid-State Circuits, vol. 38, no. 8, pp. 1353-1363, Aug. 2003.
68. B. Linares-Barranco, T. Serrano-Gotarredona, R. Serrano-Gotarredona, and C. Serrano-Gotarredona, "Current-mode techniques for sub-pico ampere circuit design," Int. J. Analog Integr. Circuits Signal Process., vol. 38, pp. 103-119, 2004.
69. T. Serrano-Gotarredona and B. Linares-Barranco, "CMOS mismatch model valid from weak to strong inversion," in Proc. Eur. Solid State Circuits Conf. (ESSCIRC'03), Sep. 2003, pp. 627-630.
70. B. Linares-Barranco, T. Serrano-Gotarredona, and R. Serrano-Gotarredona, "Compact low-power calibration mini-DACs for neural massive arrays with programmable weights," IEEE Trans. Neural Netw., vol. 14, no. 5, pp. 1207-1216, Sep. 2003.
71. A. Linares-Barranco, G. Jimenez-Moreno, B. Linares-Barranco, and A. Civit-Ballcels, "On algorithmic rate-coded AER generation," IEEE Trans. Neural Netw., vol. 17, no. 3, pp. 771-788, May 2006.
72. F. Gomez-Rodriguez and R. Paz-Vicente et al., "AER tools for communications and debugging," in Proc. IEEE Int. Symp. Circuits Syst., (ISCAS'06), May 2006, pp. 3253-3256.
73. R. Paz-Vicente and A. Linares-Barranco et al., "PCI-AER interface for neuro-inspired spiking systems," in Proc. IEEE Int. Symp. Circuits Syst. (ISCAS'06), May 2006, pp. 3161-3164.
74. S.-C. Liu and M. Öster, "Feature competition in a spike based winner-take-all VLSI network," in Proc. IEEE Int. Symp. Circuits Syst. (ISCAS'06), May 2006, pp. 3634-3637.
75. P. Häfliger, "Adaptive WTA with an analog VLSI neuromorphic learning chip," IEEE Trans. Neural Netw., submitted for publication.
76. R. Serrano-Gotarredona et al., "On real-time AER 2D convolutions hardware for neuromorphic spike based cortical processing," IEEE Trans. Neural Netw., submitted for publication.
77. K. A. Zaghloul and K. Boahen, "Optic nerve signals in a neuromorphic chip: Part I and II," IEEE Trans. Biomed. Eng., vol. 51, no. 4, pp. 657-675, Apr. 2004.
78. J. Costas Santos et al., "A contrast retina with on-chip calibration for neuromorphic spike-based AER vision systems," IEEE Trans. Circuits Syst. I, Reg. Papers, accepted for publication.