Improving the tolerance of multilayer perceptrons by minimizing the statistical sensitivity to weight deviations

Neurocomputing

Volumn 31, Issue 1-4, 2000, Pages 87-103

  Bernier, J L ^a   Ortega, J ^a   Rojas, I ^a   Prieto, A ^a  

^a UNIVERSITY OF GRANADA   (Spain)

Author keywords

ANOVA; Backpropagation; Fault tolerance; Statistical sensitivity; Weight deviations

Indexed keywords

BACKPROPAGATION; FAULT TOLERANT COMPUTER SYSTEMS; FEEDFORWARD NEURAL NETWORKS; LEARNING ALGORITHMS; SENSITIVITY ANALYSIS; STATISTICAL METHODS;

WEIGHT DEVIATIONS;

MULTILAYER NEURAL NETWORKS;

ALGORITHM; ANALYTICAL ERROR; ARTICLE; ARTIFICIAL NEURAL NETWORK; PERFORMANCE; PRIORITY JOURNAL; STATISTICAL ANALYSIS; WEIGHT;

EID: 0034058275     PISSN: 09252312     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0925-2312(99)00150-2     Document Type: Article

References (27)

1. C. Alippi, V. Piuri, M. Sami, Sensitivity to errors in artificial neural networks: a behavioral approach, Proceedings of the IEEE International Symposium on Circuits & Systems, 1994, pp. 459-462.
2. J.L. Bernier, J. Ortega, A. Prieto, A modified backpropagation algorithm to tolerate weight errors, Lecture Notes in Computer Science, Vol. 1240, Springer, Berlin, 1997, pp. 763-771.
3. C.M. Bishop, Neural Networks for Pattern Recognition, Oxford University Press, Oxford, 1995.
4. G. Casella, R.L. Berger, Statistical Inference, Duxbury Press, 1990.
5. C. Chiu, K. Mehrotra, C.K. Mohan, S. Ranka, Modifying training algorithms for improved fault tolerance, Proceedings of the IEEE International Conference on Neural Networks, Vol. 1, 1994, pp. 333-338.
6. Choi J.Y., Choi C. Sensitivity analysis of multilayer perceptron with differentiable activation functions. IEEE Trans. Neural Networks. 3(1):1992;101-107.
7. Edwards P.J., Murray A.F. Can deterministic penalty terms model the effects of synaptic weight noise on network fault-tolerance? Int. J. Neural Systems. 6(4):1995;401-416.
8. P.J. Edwards, A.F. Murray, Weight saliency regularisation in augmented networks, European Symposium on Artificial Neural Networks, ESANN, D-Facto, 1998, pp. 261-266.
9. H. Elsimary, S. Mashali, A. Darwish, S. Shasheen, Performance evaluation of a novel fault tolerance training algorithm, Proceedings of the International Conference on Electronics, Circuits & Systems, 1994, pp. 566-570.
10. Fisher R.A. The comparison of samples with possibly unequal variances. Ann. Eugenics. 9:1936;174-180.
11. A. Grace, Optimization Toolbox, in: Matlab Users Guide (The MathWorks Inc., 1994).
12. Girosi F., Jones M., Poggio T. Regularization theory and neural networks architectures. Neural Computat. 7:1995;219-269.
13. P.E. Hart, The condensed nearest neighbor rule, IEEE Trans Inform. Theory 125 (1968) 515-516.
14. S. Judd, P.W. Munro, Nets with unreliable hidden nodes learn error-correcting codes, Proceedings of the Neural Information Processing Systems (NIPS) Conference (Morgan Kaufmann, Los, AHOS, CA, 1993), pp. 89-96.
15. A. Krogh, J.A. Hertz, A simple weight decay improve generalization, Proceedings of the Neural Information Processing Systems (NIPS) Conference (Morgan Kaufmann, Los AHOS, CA, 1992), pp. 950-957.
16. C. Lin, I. Wu, Maximizing fault tolerance in multilayer neural networks, Proceedings of the IEEE International Conference on Neural Networks, 1994, pp. 419-424.
17. R.P. Lippmann, An introduction to computing with neural nets, IEEE ASSP Mag. (1987) 4-22.
18. J. Mao, A.K. Jain, Regularization techniques in artificial neural networks, World Congr. Neural Networks 4 (1993) 75-79.
19. A.A. Minai, The robustness of feed-forward neural networks: a preliminary investigation, Ph.D. Thesis, University of Virginia, 1992.
20. Murray A.F., Edwards P.J. Synaptic weight noise during MLP training: enhanced MLP performance and fault tolerance resulting from synaptic weight noise during training. IEEE Trans. Neural Networks. 5(5):1994;792-802.
21. Neti C., Schneider M.H., Young E.D. Maximally fault tolerant neural networks. IEEE Trans. Neural Networks. 3(1):1992;14-23.
22. Pathak D.S., Koren I. Complete and partial fault tolerance of feedforward neural nets. IEEE Trans. Neural Networks. 6(2):1995;446-456.
23. Segee B.E., Carter M.J. Comparative fault tolerance of parallel distributed processing networks. IEEE Trans. Comput. 43(11):1994;1323-1329.
24. Stevenson M., Winter R., Widrow B. Sensitivity of neural networks to weight errors. IEEE Trans. Neural Networks. 1(1):1990;71-80.
25. Sudkamp T., Hammell R. Interpolation, completion, and learning fuzzy rules. IEEE Trans. Systems, Man Cybern. 24(2):1994;332-342.
26. L. Wang, Adaptive Fuzzy Systems and Control. Design and Stability Analysis, Prentice-Hall, Englewood Cliffs, NJ, 1994.
27. T. Yoshino, I. Nagayama, N. Akamatsu, The learning capability of cyclic activation BP for two-spirals problem, Proceedings of IIZUKA'96, 1996, pp. 684-687.