Real-time neural network inversion on the SRC-6e reconfigurable computer

IEEE Transactions on Neural Networks

Volumn 18, Issue 3, 2007, Pages 889-901

  Duren, Russell W ^a   Marks II, Robert J ^a   Reynolds, Paul D ^b   Trumbo, Matthew L ^c  

^a Baylor University   (United States)

^b STANFORD UNIVERSITY   (United States)

^c Pelco   (United States)

Author keywords

Field programmable gate arrays (FPGAs); Inverse problems; Neural network hardware; Particle swarm theory; Real time systems; Reconfigurable architectures; Sonar

Indexed keywords

ALGORITHMS; ESTIMATION; FIELD PROGRAMMABLE GATE ARRAYS (FPGA); INVERSE PROBLEMS; OPTIMIZATION; PERSONAL COMPUTERS; REAL TIME SYSTEMS;

PARTICLE SWARM OPTIMIZATION (PSO); PARTICLE SWARM THEORY; RECONFIGURABLE ARCHITECTURES;

FEEDFORWARD NEURAL NETWORKS;

ARTICLE; ARTIFICIAL INTELLIGENCE; ARTIFICIAL NEURAL NETWORK; AUTOMATED PATTERN RECOGNITION; COMPUTER; COMPUTER ANALYSIS; COMPUTER SIMULATION; COMPUTER SYSTEM; EQUIPMENT; EQUIPMENT DESIGN; INSTRUMENTATION; METHODOLOGY; SIGNAL PROCESSING; THEORETICAL MODEL;

ARTIFICIAL INTELLIGENCE; COMPUTER SIMULATION; COMPUTER SYSTEMS; COMPUTERS; COMPUTING METHODOLOGIES; EQUIPMENT DESIGN; EQUIPMENT FAILURE ANALYSIS; MODELS, THEORETICAL; NEURAL NETWORKS (COMPUTER); PATTERN RECOGNITION, AUTOMATED; SIGNAL PROCESSING, COMPUTER-ASSISTED;

EID: 34248681602     PISSN: 10459227     EISSN: None     Source Type: Journal    
DOI: 10.1109/TNN.2007.891679     Document Type: Article

References (39)

1. J.-B. Jung, M. A. El-Sharkawi, R. J. Marks II, R. T. Miyamoto, W. L. J. Fox, G. M. Anderson, and C. J. Eggen, "Neural network training for varying output node dimension," in Proc. Int. Joint Conf. Neural Netw., Washington, D.C., 2001, pp. 1733-1738.
2. W. L. J. Fox, R. J. Marks, M. U. Hazen, C. J. Eggen, and M. A. El-Sharkawi, "Environmentally adaptive sonar control in a tactical setting," Impact Environ. Variability Acoustic Predictions Sonar Performance pp. 595-602, 2002 [Online]. Available: http:// www.ecs.baylor.edu/faculty/marks/REPRINTS/ 2002_EnvironmentallyAdaptiveSonar.pdf
3. P. D. Reynolds, R. W. Duren, M. L. Trumbo, and R. J. Marks II, "FPGA implementation of particle swarm optimization for inversion of large neural networks," in Proc. 2005 IEEE Swarm Intell. Symp., Pasadena, CA, Jun. 8-10, 2005, pp. 389-392.
4. R. D. Reed and R. J. Marks II, Neural Smithing: Supervised Learning in Feedforward Artificial Neural Networks. Cambridge, MA: MIT Press, 1999.
5. C. A. Jensen, R. D. Reed, R. J. Marks, M. A. El-Sharkawi, J. Jung, R. T. Miyamoto, G. M. Anderson, and C. J. Eggen, "Inversion of feedforward neural networks: Algorithms and applications," Proc. IEEE, vol. 87, no. 9, pp. 1536-1549, Sep. 1999.
6. J. N. Hwang, C. H. Chan, and R. J. Marks II, "Frequency selective surface design based on iterative inversion of neural networks," in Proc. Int. Joint Conf. Neural Netw., San Diego, CA, Jun. 17-21, 1990, vol. I, pp. I39-I44.
7. B. S. Kim and A. J. Calise, "Nonlinear flight control using neural networks," J. Guid. Control Dyn., vol. 20, no. 1, pp. 26-33, 1997.
8. R. D. Reed and R. J. Marks II, "An evolutionary algorithm for function inversion and boundary marking," in Proc. IEEE Int. Conf. Evol. Comput., Nov. 26-30, 1995, pp. 794-797.
9. C. A. Jensen, M. A. El-Sharkawi, and R. J. Marks II, "Power security boundary enhancement using evolutionary-based query learning," Eng. Intell. Syst., vol. 7, no. 9, pp. 215-218, Dec. 1999.
10. I. N. Kassabalidis, M. El Sharkawi, and R. J. Marks II, "Border identification for power system security assessment using neural network inversion: An overview," in Congr. Evol. Comput./IEEE World Congr. Comput. Intell., Honolulu, HI, May 12-17, 2002, pp. 1075-1079.
11. L. Tsang, Z. Chen, S. Oh, R. J. Marks II, and A. T. C. Chang, "Inversion of snow parameters from passive microwave remote sensing measurements by a neural network trained with a multiple scattering model," IEEE Trans. Geosci. Remote Sens., vol. 30, no. 5, pp. 1015-1024, Sep. 1992.
12. B. B. Thompson, R. J. Marks, M. A. El-Sharkawi, W. J. Fox, and R. T. Miyamoto, "Inversion of neural network underwater acoustic model for estimation of bottom parameters using modified particle swarm optimizers," in Proc. Int. Joint Conf. Neural Netw., 2003, pp. 1301-1306 [Online]. Available: http://www.ecs.baylor.edu/faculty/ marks/REPRINTS/2003-07_InversionOfNeuralNetworkUnderwater.pdf
13. V. Spichak and I. Popova, "Artificial neural network inversion of magnetotelluric data in terms of three-dimensional earth macroparameters," Int. Geophys. J., vol. 142, no. 1, pp. 15-26, Jul. 2000.
14. J. N. Hwang, J. J. Choi, S. Oh, and R. J. Marks II, "Query based learning applied to partially trained multilayer perceptrons," IEEE Trans. Neural Netw., vol. 2, no. 1, pp. 131-136, Jan. 1991.
15. R. C. Eberhart, Y. Shi, and J. Kennedy, Swarm Intelligence. San Mateo, CA: Morgan Kaufmann, 2001.
16. SRC Computers, Inc., Colorado Springs, CO [Online]. Available: http:// www.srccomputers.com
17. R. Eberhart and J. Kennedy, "A new optimizer using particle swarm theory," in Proc. 6th Int. Symp. Micro Machine Human Sci., Oct. 1995, pp. 39-43.
18. R. Eberhart and J. Kennedy, "Particle swarm optimization," in Proc. IEEE Int. Conf. Neural Netw., Nov. 27-Dec. 1 1995, vol. 4, pp. 1942-1948.
19. E. Ros, E. M. Ortigosa, R. Agis, R. Carrillo, and M. Arnold, "Realtime computing platform for spiking neurons (RT-spike)," IEEE Trans. Neural Netw., vol. 17, no. 4, pp. 1050-1063, Jul. 2006.
20. Y. Maeda and M.Wakamura, "Simultaneous perturbation learning rule for recurrent neural networks and its FPGA implementation," IEEE Trans. Neural Netw., vol. 16, no. 6, pp. 1664-1672, Nov. 2005.
21. N. Mtetwa and L. S. Smith, "Precision constrained stochastic resonance in a feedforward neural network," IEEE Trans. Neural Netw., vol. 16, no. 1, pp. 250-262, Jan. 2005.
22. D. Anguita, A. Boni, and S. Ridella, "A digital architecture for support vector machines: Theory, algorithm, and FPGA implementation," IEEE Trans. Neural Netw., vol. 14, no. 5, pp. 993-1009, Sep. 2003.
23. M. Bracco, S. Ridella, and R. Zunino, "Digital implementation of hierarchical vector quantization," IEEE Trans. Neural Netw., vol. 14, no. 5, pp. 1072-1084, Sep. 2003.
24. Z. Nagy and P. Szolgay, "Configurable multilayer CNN-UM emulator on FPGA," IEEE Trans. Circuits Syst. I, Reg. Papers, vol. 50, no. 6, pp. 774-778, Jun. 2003.
25. Xilinx, Inc., "Virtex-II platform FPGAs: Complete data sheet," San Jose, CA, 2005 [Online]. Available: http://www.xilinx.com/bvdocs/ publications/ds031.pdf
26. H. Hikawa, "A digital hardware pulse-mode neuron with piecewise linear activation function," IEEE Trans. Neural Netw., vol. 14, no. 5, pp. 1028-1037, Sep. 2003.
27. M. T. Tommiska, "Efficient digital implementation of the sigmoid function for reprogrammable logic," in Inst. Electr. Eng. Proc. Comput. Digit. Tech., Nov. 2003, vol. 150, no. 6, pp. 403-411.
28. M. Martincigh and A. Abramo, "A new architecture for digital stochastic pulse-mode neurons based on the voting circuit," IEEE Trans. Neural Netw., vol. 16, no. 6, pp. 1685-1693, Nov. 2005.
29. R. Duren, D. Fouts, and D. Zulaica, "Performance comparison CORDIC implementations on the SRC-6E reconfigurable computer," presented at the 2003 MAPLD Int. Conf., Washington, D.C., Sep. 9-11, 2003 [Online]. Available: http://www.klabs.org/richcontent/MAPLDCon03/MAPLDCon03.html, unpublished
30. R. Duren, D. Fouts, and D. Zulaica, "Algorithm and programming considerations for embedded reconfigurable computers," presented at the 2003 7th Annu. Workshop High Performance Embedded Comput., Lexington, MA, Sep. 23-25, 2003 [Online]. Available: http://www.ll.mit.edu/HPEC/ pdfs/cfp03.pdf, unpublished
31. H. Diab, M. Huang, K. Gaj, T. El-Ghazawi, and N. Alexandridis, "An automated pipeline balancing in the SRC reconfigurable computer and its application to the RC5 cipher breaking," presented at the 2004 MAPLD Int. Conf., Washington, D.C., Sep. 8-10, 2004 [Online]. Available: http://www.klabs.org/mapld04/index.html, unpublished
32. R. J. Marks II, S. Oh, and L. E. Atlas, "Alternating projection neural networks," IEEE Trans. Circuits Syst., vol. 36, no. 6, pp. 846-857, Jun. 1989.
33. SRC-6 C Programming Environment ver. v1.7 Guide, SRC Computers, Inc., Colorado Springs, CO, 2004.
34. H. Hahn, D. Timmermann, B. J. Hosticka, and B. Rix, "A unified and division-free CORDIC argument reduction method with unlimited convergence domain including inverse hyperbolic functions," IEEE Trans. Comput., vol. 43, no. 11, pp. 1339-1344, Nov. 1994.
35. J. Zhu and P. Sutton, "FPGA implementation of neural networks - A survey of a decade of progress," in Proc. 13th Int. Conf. Field-Programmable Logic Appl., 2003, pp. 1062-1066 [Online]. Available: http://eprint.uq.edu.au/archive/00000827/
36. M. Clerc and J. Kennedy, "The particle swarm - Explosion, stability and convergence in a multidimensional complex space," IEEE Trans. Evol. Comput., vol. 6, no. 1, pp. 58-73, Feb. 2002.
37. J. F. Wakerly, Digital Design Principles and Practices, 3rd ed. Englewood Cliffs, NJ: Prentice-Hall, 2000, pp. 730-733.
38. W. H. Press, S. A. Teukolsky, W. T. Vetterling, and B. P. Flannery, Numerical Recipes in C. Cambridge, U.K.: Cambridge Univ. Press, 1992, pp. 274-300.
39. P. D. Reynolds, "Algorithm implementation in FPGAs demonstrated through neural network inversion on the SRC-6e," M.S. Thesis, Dept. Eng., Baylor University, Waco, TX, May 2005.