Selecting accurate, robust, and minimal feedforward neural networks

IEEE Transactions on Circuits and Systems I: Fundamental Theory and Applications

Volumn 49, Issue 12, 2002, Pages 1799-1810

  Alippi, Cesare ^a  

^a POLITECNICO DI MILANO   (Italy)

Author keywords

Acute perturbations; Constrained model selection; Feedforward neural networks; Robustness; Sensitivity; System level design

Indexed keywords

CONSTRAINT THEORY; ELECTRIC POWER SUPPLIES TO APPARATUS; ERROR ANALYSIS; GRADIENT METHODS; MATHEMATICAL MODELS; MATRIX ALGEBRA; PERTURBATION TECHNIQUES; SENSITIVITY ANALYSIS; VECTORS;

CONSTRAINED MODEL SELECTION; FINAL PREDICTION ERROR; GENERAL PREDICTION ERROR; GENERALIZATION ERROR ESTIMATE; NETWORK INFORMATION CRITERION;

FEEDFORWARD NEURAL NETWORKS;

EID: 0036959081     PISSN: 10577122     EISSN: None     Source Type: Journal    
DOI: 10.1109/TCSI.2002.805710     Document Type: Article

References (38)

1. Special Issue on Neural Networks, IEEE Trans. Circuits Syst., vol. 36, pp. 643-772, May 1989.
2. F. Girosi, M. Jones, and T. Poggio, "Regularization theory and neural networks architectures," Neural Comput., vol. 7, no. 2, pp. 219-269, 1995.
3. C. Alippi, "Randomized algorithms: A system-level, poly-time analysis of robust computation," IEEE Trans. Comput., vol. 51, pp. 740-749, July 2002.
4. P. J. Edwards and A. F. Murray, "Toward optimally distributed computation," Neural Comput., vol. 10, pp. 987-1005, 1998.
5. Y. Tohma and Y. Koyanagi, "Fault-tolerant design of neural networks for solving optimization problems," IEEE Trans. Comput., vol. 45, pp. 1450-1455, Dec. 1996.
6. G. De Micheli and M. Sami, Hardware/Software Co-Design, G. De Micheli and M. Sami, Eds. Norwell, MA: Kluwer, 1996, vol. 310.
7. D. Sciuto, "Guest editor's introduction: Design tools for embedded systems," IEEE Des. Test Comput., vol. 17, pp. 11-13, 2000.
8. Proceedings of the IEEE Annual Workshop on VLSI: System Level Design, vol. 1, 1998, pp. 1-144.
9. R.R. Vemuri and R.E. Harr, "Configurable computing: Technology and applications," IEEE Computer, vol. 33, Apr. 2000.
10. C. Alippi and L. Briozzo, "Accuracy vs. precision in digital VLSI architectures for signal processing," IEEE Trans. Comput., vol. 47, pp. 472-477, Apr. 1998.
11. J. Holt and J. Hwang, "Finite precision error analysis of neural network hardware implementations," IEEE Trans. Comput., vol. 42, pp. 281-290, Mar. 1993.
12. G. Dundar and K. Rose. "The effects of quantization on multilayer neural networks," IEEE Trans. Neural Networks, vol. 6, pp. 1446-1451, Nov. 1995.
13. P. J. Edwards and A. F. Murray, "Fault tolerance via weight noise in analog VLSI implementations of MLP's-A case study with EPSILON," IEEE Trans. Circuits Syst. II, vol. 45, pp. 1255-1262, Sept. 1998.
14. I. Bayraktaroglu, A. S. Ogrenci, G. Dundar, S. Balkir, and E. Alpoydin, "ANNSyS (An analog neural network synthesis system)," in Proc. IEEE ICNN'97, 1997, pp. 910-915.
15. M. J. Buckingham, Noise in Electronic Devices and Systems. Chichester, U.K.: Ellis Horwood, 1983.
16. A. Krogh and J. A. Hertz, "A simple weight decay can improve generalization," in Proc. 4th NIPS, vol. 4, 1992, pp. 950-957.
17. G. Seber and C. Wild, Nonlinear Regression. New York: Wiley, 1989.
18. A. U. Levin, T. K. Leen, and J. E. Moody, "Fast pruning using principal components," in Proc. 6th NIPS, vol. 6, Dec. 1994, pp. 35-42.
19. Y. Le Cun, J. S. Denker, and S. A. Solla, "Optimal brain damage," in Proc. 2nd NIPS, vol. 2, CA, Feb. 1990, pp. 598-605.
20. B. Hassibi and D. G. Stork, "Second-order derivative for network pruning: Optimal brain surgeon," in Proc. 5th NIPS, vol. 2, Jan. 1993, pp. 164-173.
21. I. Guyon, V. Vapnik, B. Boser, L. Bottou, and S. Solla, "Structural risk minimization for character recognition," in Proc. 4th NIPS, vol. 4, 1992, pp. 471-479.
22. C. Alippi and V. Piuri, "Topological minimization of multi-layered feed-forward neural networks by spectral decomposition," in Proc. IEEE IJCNN'92, Beijing, China, Nov. 3-6, 1992, pp. 805-810.
23. A. S. Weigend and D. E. Rumelhart, "The effective dimension of the space of hidden units," in Proc. IEEE IJCNN, Singapore, 1991, pp. 2069-2074.
24. D. Phatak and I. Koren, "Complete and partial fault tolerance of feedforward neural nets," IEEE Trans. Neural Networks, vol. 6, pp. 446-456, Mar. 1995.
25. N. Murata, S. Yoshizawa, and S. Amari, "Network information criterion-Determining the number of hidden units for an artificial neural network model," IEEE Trans. Neural Networks, vol. 5, pp. 865-872, Nov. 1994.
26. J. Larsen, "A generalization error estimate for nonlinear systems," in Proc. IEEE Signal Processing Workshop, 1992, pp. 29-38.
27. J. Larsen and L. K. Hansen, "Generalization performance of regularized neural network models," in Proc. IEEE Signal Processing Workshop, 1994, pp. 42-51.
28. C. Alippi, "FPE-based criteria to dimension feedforward neural topologies," IEEE Trans. Circuits Syst. I, vol. 46, pp. 962-973, Aug. 1999.
29. S. Piché, "The selection of weights accuracies for madalines," IEEE Trans. Neural Netw., vol. 6, pp. 432-445, Mar. 1995.
30. M. H. Hassoun, Fundamentals of Artificial Neural Networks. Cambridge, MA: MIT press, 1995.
31. J. E. Moody, "The effective number of parameters: An analysis of generalization in nonlinear learning systems," in Proc. 4th NIPS, 1992, pp. 847-854.
32. S. Amari, "Statistical and information-geometrical aspects of neural learning," in Computational Intelligence: A Dynamic Perspective. New York: IEEE Press, 1995, pp. 71-82.
33. C. M. Bishop, Neural Networks for Pattern Recognition. London, U.K.: Clarendon, 1995.
34. A. Poncet, "Asymptotic probability density of the generalization error," in Proc. IEEE NICROSP, 1996, pp. 66-74.
35. G. W. Stewart and J. Sun, Matrix Perturbation Theory, New York: Academic, 1990.
36. R. A. Horn and C. R. Johnson, Matrix Analysis. Cambridge, MA: Cambridge Univ. Press, 1985.
37. G. Strang, Linear Algebra and Its Applications. Orlando, FL: Harcourt Brace Jovanovich, 1988.
38. L. Ljung, System Identification, Theory for the User. Englewood Cliffs, NJ: Prentice-Hall, 1987.