The Effects of Adding Noise during Backpropagation Training on a Generalization Performance

Neural Computation

Volumn 8, Issue 3, 1996, Pages 643-674

  An, Guozhong ^a  

^a SHELL RESEARCH AND TECHNOLOGY CENTRE   (Netherlands)

Author keywords

[No Author keywords available]

Indexed keywords

EID: 2342565172     PISSN: 08997667     EISSN: None     Source Type: Journal    
DOI: 10.1162/neco.1996.8.3.643     Document Type: Article

References (38)

1. Arfken, G. 1985. Mathematical Methods for Physicists. Academic Press, New York.
2. Bishop, C. M. 1995. Training with noise is equivalent to Tikhomov regularization. Neural Comp. 7, 108-116.
3. Bottou, L. 1991. Stochastic gradient learning in neural networks. NEURONIMES'91, EC2, Nanterre, France, 687-606.
4. Chauvin, Y. 1989. A back-propagation algorithm with optimal use of hidden units. In Advances in Neural Information Processing System I, D. Touretsky, ed., pp. 519-526. Morgan Kaufmann, San Mateo, CA.
5. Clay, R., and Sequin, C. 1992. Fault tolerance training improves generalization and robustness. Proc. Int. Joint Conf. Neural Networks, IEEE Neural Council, Baltimore, I-769-774.
6. Drucker, H., and Le Cun, Y. 1992. Improving generalisation performance using double back-propagation. IEEE Trans. Neural Networks 3, 991-997.
7. Geman, S., and Hwang, C. 1986. Diffusion for global optimization. SIAM J. Control Optim. 25, 1031-1043.
8. Gillespie, D. 1992. Markov Processes. Academic Press, London.
9. Guillerm, T., and Cotter, N. 1991. A diffusion process for global optimization in neural networks. Proc. Int. Joint Conf. Neural Networks, I-335.
10. Guyon, A., Vapnik, V., Boser, B., Bottou, L., and Solla, S. A. 1992. Structural risk minimization for character recognition. In Advances in Neural Information Processing System 4 (NIPS 91), J. Moody et al., eds., pp. 471-479. Morgan Kaufmann, San Mateo, CA.
11. Hanson, S. J. 1990. A stochastic version of the delta rule. Physica D 42, 265-272.
12. Hertz, J., Krogh, A., and Thorbergsson, G. 1989. Phase transitions in simple learning. J. Phys. A: Math. Gen. 22, 2133-2150.
13. Hinton, G. E. 1986. Learning distributed representations of concepts. Proc. Eighth Annu. Conf. Cog. Sci. Soc., Amherst, 1-12.
14. Hinton, G. E., and van Camp, D. 1993. Keeping neural networks simple by minimizing the description length of the weights. Sixth ACM Conf. Comp. Learning Theory (Santa Cruz), 5-13.
15. Holmström, L., and Koistinen, P. 1992. Using additive noise in back-propagation training. IEEE Trans. Neural Networks 3, 24-38.
16. Kendall, G. D., and Hall, T. J. 1993. Optimal network construction by minimum description length. Neural Comp. 5, 210-212.
17. Kendall, M., and Stuart, A. 1977. The Advanced Theory of Statistics, Vol. 1, 4th ed., Charles Griffin, London.
18. Kirkpatrick, S., Gelatt, C., and Vecchi, M. 1983. Optimization by simulated annealing. Science 220, 671-680.
19. Krogh, A., and Hertz, J. A. 1992. A simple weight decay can improve generalization. In Advances in Neural Information Processing System 4 (NIPS 91), J. E. Moody et al., eds., pp. 950-957. Morgan Kaufmann, San Mateo, CA.
20. Kushner, H. 1987. Asymptotic global behavior for stochastic approximation and diffusions with slowly decreasing noise effects: Global minimization via Monte Carlo. SIAM J. Appl. Math. 47, 169-185.
21. MacKay, D. J. C. 1992. Bayesian interpolation. Neural Comp. 4, 415-447.
22. Matsuoka, K. 1992. Noise injection into inputs in back-propagation learning. IEEE Trans. Sys. Man Cybern. 22, 436-440.
23. Murray, A. F., and Edwards, P. J. 1993. Synaptic weight noise during multilayer perceptron training: Fault tolerance and training improvements. IEEE Trans. Neural Networks 4, 722-725.
24. Murray, A. F., and Edwards, P. J. 1994. Enhanced mlp performance and fault tolerance resulting from synaptic weight noise during training. IEEE Trans. Neural Networks 5, 792-802.
25. NeuralWare. 1991. Reference Guide: Neural Works Professional II/PlUS and NeuralWorks Explorer. NeuralWare, Inc, Pittsburgh.
26. Nowlan, S. J., and Hinton, G. E. 1992. Simplifying neural networks by soft weight-sharing. Neural Comp. 4, 473-493.
27. Pao, Y. H. 1989. Adaptive Pattern Recognition and Neural Networks, chap. 2, pp. 25-35. Addison-Wesley, Reading, MA.
28. Poggio, T., and Girosi, F. 1990. Networks for approximation and learning. Proc. IEEE 78, 1481-1497.
29. Reed, R., Oh, S., and Marks, R. J., II. 1992. Regularization using jittered training data. Proc. Int. Joint Conf. Neural Networks, IEEE Neural Council, Baltimore, III-147.
30. Rissanen, J. 1989. Stochastic Complexity in Statistical Inquiry. World Scientific, Singapore.
31. Rögnvaldsson, T. 1994. On Langevin updating in multilayer perceptrons. Neural Comp. 6, 916-926.
32. Rumelhart, D. E., Hinton, G. E., and Williams, R. J. 1986. Learning representations by back-propagating errors. Nature (London) 323, 533-536.
33. Seung, H., Sompolinsky, H., and Tishby, N. 1992. Statistical mechanics of learning from examples. Phys. Rev. A 45, 6056-6091.
34. Sietsma, J., and Dow, R. 1988. Neural network pruning - why and how. Proc. IEEE Int. Conf. Neural Networks I, 325-333.
35. Sjöberg, J., and Ljung, L. 1992. Overtraining, regularisation, and searching for minimum in neural networks. Proc. Symp. Adaptive Systems Control Signal Process., Grenoble, France.
36. van Kampen, N. G. 1981. Stochastic Processes in Physics and Chemistry. North-Holland, Amsterdam.
37. Weigend, A., Rumelhart, D., and Huberman, B. 1991. Generalization by weight-elimination with application to forecasting. In Advances in Neural Information Processing System 3 (NIPS 90), J. M. R. P. Lippmann and D. Touretsky, eds., pp. 875-882. Morgan Kaufmann, San Mateo, CA.
38. White, H. 1989. Learning in artificial neural networks: A statistical perspective. Neural Comp. 1, 425-464.