Enhanced MLP Performance and Fault Tolerance Resulting from Synaptic Weight Noise During Training

IEEE Transactions on Neural Networks

Volumn 5, Issue 5, 1994, Pages 792-802

  Murray, Alan F ^a   Edwards, Peter J ^a  

^a UNIVERSITY OF EDINBURGH   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

APPROXIMATION THEORY; COMPUTER SIMULATION; FAULT TOLERANT COMPUTER SYSTEMS; LEARNING SYSTEMS; NUMERICAL METHODS; OPTIMIZATION; PROBABILITY; REDUNDANCY; SIGNAL FILTERING AND PREDICTION; SPURIOUS SIGNAL NOISE; VLSI CIRCUITS;

ANALOG NOISE; MULTILAYER PERCEPTION TRAINING; SYNAPTIC WEIGHT NOISE; TRAINING;

NEURAL NETWORKS;

EID: 0028494739     PISSN: 10459227     EISSN: 19410093     Source Type: Journal    
DOI: 10.1109/72.317730     Document Type: Article

References (18)

1. P. W. Hollis, J. S. Harper, J. J. Paulos, “The effects of precision constraints in a Back propagation learning network,” Neural Computation, vol. 2, pp. 363–373, 1990.
2. T. J. Sejnowski, C. R. Rosenberg, “Parallel networks that learn to text,” Complex Systems, vol. 1, 145–168, 1987.
3. J. L. McClelland, “Resource requirements of standard and programmable nets,” in Parallel Distributed Processing: Explorations in the Microstructure of Cognition, D. E. Rumelhart and J. L. McClelland, Eds. Cambridge, MA: MIT Press, pp. 460–487, 1986.
4. M. Stevenson, R. Winter, and B. Widrow, “Sensitivity of feedforward neural networks to weight errors,” IEEE Trans. Neural Networks, vol. 1, no. 1, pp. 71–80, 1990.
5. M. D. Emmerson, R. I. Damper, A. J. G. Hey, and C. Upstill, “Fault tolerance and redundancy of neural nets for the classification of acoustic data,” Proc. IEEE Conf. Acoustics, Speech and Signal Processing, pp. 1053–1056, 1992.
6. M. D. Emmerson and R. I. Damper, “Relations between fault tolerance and internal representations for multilayer perceptrons,” in Proc. IEEE Conf. Acoustics, Speech and Signal Processing, 1992, pp. 281–284.
7. C. Neti, H. Schneider, and E. D. Young, “Maximally fault-tolerant neural networks,” IEE Trans. Neural Networks, vol. 3, no. 1, pp. 14–23, 1992.
8. C. H. Sequin and R. D. Clay, “Fault tolerance in feed forward artificial neural networks,” Neural Networks: Concepts, Applications and Implementations, vol. 4, pp. 111–141, 1991.
9. A. F. Murray, “Analog noise-enhanced learning in neural network circuits,” Electronics Letters, vol. 2, no. 17, pp. 1546—1548, 1991.
10. A. F. Murray, “multilayer perceptron learning optimised for on-chip implementation—A noise robust system,” Neural Computation, vol. 4, no. 3, 366–381, 1992.
11. J. G. Taylor, “Spontaneous behaviour in neural networks,” J. Theor. Biol., vol. 36, 513–528, 1972.
12. S. J. Hanson, “A stochastic version of the delta rule,” Physica D, vol. 42, pp. 265–272, 1990.
13. S. Kirkpatrick, C. D. Gelatt, Jr., and M. P. Vecchi, “Optimization by simulated annealing,” Science, vol. 220, no. 4598, pp. 671–680, 1983.
14. L. Holmstrom and P. Koistinen, “Using additive noise in backpropagation training” IEEE Trans. Neural Networks, vol. 3, no. 1, 1992.
15. C. Bishop, “Curvature-driven smoothing in backpropagation neural networks,” IJCNN, vol. 2, pp. 749–752, 1990.
16. K. Matsuoka, “Noise injection into inputs in Back propagation learning,” IEEE Trans. Syst. Man Cyber., vol. 22, no. 3, pp. 436–440, 1992.
17. Y. Le Cun, J. S. Denker, and S. A. Solla, “Optimal brain damage” in Neural Information Processing Systems (NIPS) Conference, San Mateo, CA: Morgan Kaufmann, 1990, pp. 598–605.
18. A. F. Murray and P. J. Edwards, “Synaptic weight noise during multilayer perceptron training: fault tolerance and training improvements,” IEEE Trans. Neural Networks, vol. 4, no. 4, pp. 722–725, 1993.