Analysis and compensation of the effects of analog VLSI arithmetic on the LMS algorithm

IEEE Transactions on Neural Networks

Volumn 22, Issue 7, 2011, Pages 1046-1060

  Carvajal, Gonzalo ^a   Figueroa, Miguel ^a   Sbarbaro, Daniel ^a   Valenzuela, Waldo ^a  

^a UNIVERSITY OF CONCEPCIÓN   (Chile)

Author keywords

Analog very large scale integration; least mean square algorithm; on chip learning; silicon neural networks

Indexed keywords

ANALOG HARDWARE; ANALOG NEURAL NETWORK; ANALOG VERY LARGE SCALE INTEGRATION; ANALOG VLSI; ARITHMETIC CIRCUIT; CALIBRATION TECHNIQUES; CHARGE LEAKAGE; COMPLEMENTARY METAL OXIDE SEMICONDUCTORS; CONVERGENCE PROPERTIES; DESIGN GUIDELINES; DEVICE MISMATCH; DIE AREA; DIGITAL COUNTERPARTS; LEAST MEAN SQUARE ALGORITHMS; LINEAR PERCEPTRONS; LMS ALGORITHMS; MATHEMATICAL ANALYSIS; MIXED SIGNAL; ON CHIPS; ON-CHIP LEARNING; PERCEPTRON; SPECIFIC EFFECTS;

ADAPTIVE SYSTEMS; ALGORITHMS; CONVERGENCE OF NUMERICAL METHODS; NEURAL NETWORKS; PATTERN RECOGNITION SYSTEMS;

ANALOG CIRCUITS;

ALGORITHM; ANALOG COMPUTER; ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK;

ALGORITHMS; ARTIFICIAL INTELLIGENCE; COMPUTERS, ANALOG; NEURAL NETWORKS (COMPUTER);

EID: 79960117899     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2011.2136358     Document Type: Article

References (39)

1. L. Gatet, H. Tap-Beteille, and F. Bony, "Comparison between analog and digital neural network implementations for range-finding applications," IEEE Trans. Neural Netw., vol. 20, no. 3, pp. 460-470, Mar. 2009.
2. R. Dlugosz, T. Talaska, W. Pedrycz, and R. Wojtyna, "Realization of the conscience mechanism in CMOS implementation of winner-takesall self-organizing neural networks," IEEE Trans. Neural Netw., vol. 21, no. 6, pp. 961-971, Jun. 2010.
3. V. Kakkar, "Comparative study on analog and digital neural networks," Int. J. Comput. Sci. Netw. Secur., vol. 9, no. 7, pp. 14-21z, Jul. 2009.
4. H. Chen, S. Saighi, L. Buhry, and S. Renaud, "Real-time simulation of biologically realistic stochastic neurons in VLSI," IEEE Trans. Neural Netw., vol. 21, no. 9, pp. 1511-1517, Sep. 2010.
5. M. Sivilotti, M. Emerling, and C. Mead, "VLSI architectures for implementation of neural networks," in Proc. AIP Conf. Neural Netw. Comput., 1987, pp. 408-413.
6. A. G. Andreou, K. A. Boahen, P. O. Pouliquen, A. Pavasović, R. E. Jenkins, and K. Strohbehn, "Current-mode subthreshold MOS circuits for analog VLSI neural systems," IEEE Trans. Neural Netw., vol. 2, no. 2, pp. 205-213, Mar. 1991.
7. P. W. Hollis, J. S. Harper, and J. J. Paulos, "The effects of precision constraints in a backpropagation learning network," Neural Comput., vol. 2, no. 3, pp. 363-373, 1990.
8. R. C. Frye, E. A. Rietman, and C. C. Wong, "Back-propagation learning and nonidealities in analog neural network hardware," IEEE Trans. Neural Netw., vol. 2, no. 1, pp. 110-117, Jan. 1991.
9. J. B. Lont and W. Guggenbiihl, "Analog CMOS implementation of a multilayer perceptron with nonlinear synapses," IEEE Trans. Neural Netw., vol. 3, no. 3, pp. 457-465, May 1992.
10. B. K. Dolenko and H. C. Card, "Tolerance to analog hardware of onchip learning in backpropagation networks," IEEE Trans. Neural Netw., vol. 6, no. 5, pp. 1045-1052, Sep. 1995.
11. A. J. Montalvo, R. S. Gyurksic, and J. J. Paulos, "Toward a generalpurpose analog VLSI neural network with on-chip learning," IEEE Trans. Neural Netw., vol. 8, no. 2, pp. 413-423, Mar. 1997. (Pubitemid 127765134)
12. G. Marshall and S. Collins, "An analogue radial basis function circuit incorporating floating-gate devices," Analog Integr. Circuits Signal Process., vol. 11, no. 1, pp. 21-34, 1996.
13. D. Hsu, M. Figueroa, and C. Diorio, "Competitive learning with floatinggate circuits," IEEE Trans. Neural Netw., vol. 13, no. 3, pp. 732-744, May 2002.
14. C. Prasanna and S. Pinjare, "Design and analog VLSI implementation of neural network architecture for signal processing," Eur. J. Sci. Res., vol. 27, no. 2, pp. 199-216, 2009.
15. B. Widrow, J. M. McCool, M. G. Larimore, and C. R. Johnson, "Stationary and nonstationary learning characteristics of the LMS adaptive filter," Proc. IEEE, vol. 64, no. 8, pp. 1151-1162, Aug. 1976.
16. M. Tucci and M. Raugi, "Adaptive FIR neural model for centroid learning in self-organizing maps," IEEE Trans. Neural Netw., vol. 21, no. 6, pp. 948-960, Jun. 2010.
17. R. Martinek and J. Zidek, "Use of adaptive filtering for noise reduction in communications systems," in Proc. Int. Conf. Appl. Electron., Sep. 2010, pp. 1-6.
18. K. Diamantaras and S. Young, Principal Component Neural Networks. New York: Wiley, 1996.
19. F. Palmieri, J. Zhu, and C. Chang, "Anti-Hebbian learning in topologically constrained linear networks: A tutorial," IEEE Trans. Neural Netw., vol. 4, no. 5, pp. 748-761, Sep. 1993.
20. C. Bishop, Neural Networks for Pattern Recognition. New York: Oxford Univ. Press, 1997.
21. G. Carvajal, M. Figueroa, and S. Bridges, "Effects of analog-VLSI hardware on the performance of the LMS algorithm," in Proc. Int. Conf. Artif. Neural Netw., 2006, pp. 963-973. (Pubitemid 44561746)
22. M. Figueroa, E. Matamala, G. Carvajal, and S. Bridges, "Adaptive signal processing in mixed-signal VLSI with anti-Hebbian learning," in Proc. IEEE Comput. Soc. Annu. Symp. VLSI, Karlsruhe, Germany, Mar. 2006, pp. 133-140.
23. C. Mead, Analog VLSI and Neural Systems. Reading, MA: Addison- Wesley, 1989.
24. M. J. Pelgrom, A. C. J. Duinmaijer, and A. P. G. Welbers, "Matching properties of MOS transistors," IEEE J. Solid-State Circuits, vol. 24, no. 5, pp. 1433-1440, Oct. 1989.
25. A. Flores and B. Widrow, "Assessment of the efficiency of the LMS algorithm based on spectral information," in Proc. 38th Asilomar Conf. Signals, Syst. Comput., vol. 1. Pacific Grove, CA, Nov. 2004, pp. 120-124. (Pubitemid 40930585)
26. B. Widrow and E. Walach, Adaptive Inverse Control. Upper Saddle River, NJ: Prentice-Hall, 1996.
27. D. B. Mundie and L. W. Massengill, "Weight decay and resolution effects in feedforward artificial neural networks," IEEE Trans. Neural Netw., vol. 2, no. 1, pp. 168-170, Jan. 1991.
28. A. F. Murray and P. J. Edwards, "Synaptic weight noise during multilayer perceptron training: Fault tolerance and training improvements," IEEE Trans. Neural Netw., vol. 4, no. 4, pp. 722-725, Jul. 1993.
29. G. Carvajal, W. Valenzuela, and M. Figueroa, "Subspace-based face recognition in analog VLSI," in Advances in Neural Information Processing Systems 20. Cambridge, MA: MIT Press, 2008, pp. 225-232.
30. G. Carvajal, M. Figueroa, and W. Valenzuela, "Image recognition in analog VLSI with on-chip learning," in Proc. 19th Int. Conf. Artif. Neural Netw., Sep. 2009, pp. 429-438.
31. Y. Hirai and K. Nishizawa, "Hardware implementation of a PCA learning network by an asynchronous PDM digital circuit," in Proc. IEEE-INNS-ENNS Int. Joint Conf. Neural Netw., vol. 2. 2000, pp. 65-70.
32. C. Diorio, P. Hasler, B. A. Minch, and C. Mead, "A complementary pair of four-terminal silicon synapses," Analog Integr. Circuits Signal Process., vol. 13, nos. 1-2, pp. 153-166, May-Jun. 1997. (Pubitemid 127507968)
33. T. Morie, "Analog VLSI implementation of self-learning neural networks," in Learning in Silicon. Norwell, MA: Kluwer, 1999, pp. 214-242.
34. Y. Xu, "Electron transport through thin film amorphous silicon - A tunneling study," Ph.D. dissertation, Dept. Appl. Phys., Stanford Univ., Stanford, CA, 1992.
35. M. Figueroa, "Adaptive signal processing and correlational learning in mixed-signal VLSI," Ph.D. dissertation, Dept. Comput. Sci. & Eng., Univ. Washington, Seattle, 2005.
36. A. Shon, D. Hsu, and C. Diorio, "Learning spike-based correlations and conditional probabilities in silicon," in Proc. Neural Inf. Process. Syst., Vancouver, BC, Canada, 2001, pp. 1-8.
37. B. V. Liempd, D. Herrera, and M. Figueroa, "An FPGA-based accelerator for analog VLSI artificial neural network emulation," in Proc. 13th Euromicro Conf. Digital Syst. Des., 2010, pp. 771-778.
38. W. Valenzuela, G. Carvajal, and M. Figueroa, "Blind source-separation in mixed-signal VLSI," Neural Netw. World, vol. 19, no. 5, pp. 641-656, 2009.
39. R. Waser and M. Aono, "Nanoionics-based resistive switching memories," Nature Mater., vol. 6, no. 11, pp. 833-840, 2007. (Pubitemid 350064191)