Noise injection for training artificial neural networks: A comparison with weight decay and early stopping

Medical Physics

Volumn 36, Issue 10, 2009, Pages 4810-4818

  Zur, Richard M ^a   Jiang, Yulei ^a   Pesce, Lorenzo L ^a   Drukker, Karen ^a  

^a UNIVERSITY OF CHICAGO   (United States)

Author keywords

Artificial neural networks; BANN; Early stopping; Jitter; Noise injection; Overfitting; Regularization; Weight decay

Indexed keywords

CLASSIFICATION (OF INFORMATION); DECAY (ORGANIC); JITTER; STATISTICAL TESTS; ULTRASONICS;

BANN; EARLY STOPPING; NOISE INJECTION; OVERFITTING; REGULARIZATION; WEIGHT DECAY;

NEURAL NETWORKS;

ARTICLE; ARTIFICIAL NEURAL NETWORK; ECHOMAMMOGRAPHY; PRIORITY JOURNAL; SIMULATION;

EID: 70349658745     PISSN: 00942405     EISSN: None     Source Type: Journal    
DOI: 10.1118/1.3213517     Document Type: Article

References (33)

1. Y. Wu, K. Doi, C. E. Metz, N. Asada, and M. L. Giger, " Simulation studies of data classification by artificial neural networks: Potential applications in medical imaging and decision making.," J. Digit Imaging 0897-1889 6, 117-125 (1993).
2. Y. Jiang, " Malignant and benign clustered microcalcifications: Automated feature analysis and classification.," Radiology 0033-8419 198, 671-678 (1996).
3. C. M. Bishop, Neural Networks for Pattern Recognition (Oxford University Press, New York, 1995).
4. M. A. Kupinski, D. C. Edwards, M. L. Giger, and C. E. Metz, " Ideal observer approximation using Bayesian classification neural networks.," IEEE Trans. Med. Imaging 0278-0062 20, 886-899 (2001). 10.1109/42.952727 (Pubitemid 32992592)
5. J. Sietsma and R. J. F. Dow, " Creating artificial neural networks that generalize.," Neural Networks 0893-6080 4, 67-79 (1991). 10.1016/0893-6080(91)90033-2
6. S. Geman, E. Bienenstock, and R. Doursat, " Neural networks and the bias/variance dilemma.," Neural Comput. 0899-7667 4, 1-58 (1992). 10.1162/neco.1992.4.1.1
7. T. Hastie, R. Tibshirani, and J. H. Friedman, The Elements of Statistical Learning: Data Mining, Inference, and Prediction (Springer, New York, 2001).
8. D. J. C. MacKay, Bayesian Methods for Adaptive Models (California Institute of Technology, Pasadena, 1992).
9. R. M. Neal, Bayesian Learning for Neural Networks (Springer, New York, 1996).
10. W. S. Sarle, Neural Network FAQ. Retrieved September 9, 2008 (website: ftp://ftp.sas.com/pub/neural/FAQ.html).
11. G. Z. An, " The effects of adding noise during backpropagation training on a generalization performance.," Neural Comput. 0899-7667 8, 643-674 (1996). 10.1162/neco.1996.8.3.643
12. C. M. Bishop, " Training with noise is equivalent to Tikhonov regularization.," Neural Comput. 0899-7667 7, 108-116 (1995). 10.1162/neco.1995.7.1.108
13. Y. Grandvalet, S. Canu, and S. Boucheron, " Noise injection: theoretical prospects.," Neural Comput. 0899-7667 9, 1093-1108 (1997). 10.1162/neco.1997.9.5.1093
14. L. Holmström and P. Koistinen, " Using additive noise in back-propagation training.," IEEE Trans. Neural Netw. 1045-9227 3, 24-38 (1992). 10.1109/72.105415
15. K. Matsuoka, " Noise injection into inputs in backpropagation learning.," IEEE Trans. Syst. Man Cybern. 0018-9472 22, 436-440 (1992). 10.1109/21.155944
16. Y. Raviv and N. Intrator, in Combining Artificial Neural Nets: Ensemble and Modular Multi-net Systems, edited by, A. J. C. Sharkey, (Springer, New York, 1999), p. 298.
17. W. A. Wright, " Bayesian approach to neural-network modeling with input uncertainty.," IEEE Trans. Neural Netw. 1045-9227 10, 1261-1270 (1999). 10.1109/72.809073 (Pubitemid 30536598)
18. W. A. Wright, G. Ramage, D. Cornford, and I. Nabney, " Neural network modelling with input uncertainty: Theory and application.," J. VLSI Sig. Proc. Syst. 26, 169-188 (2000). 10.1023/A:1008111920791
19. R. M. Zur, Y. Jiang, and C. E. Metz, " Comparison of two methods of adding jitter to artificial neural network training.," Proceedings of CARS (Elsevier, Chicago, 2004), pp. 886-889.
20. M. S. Pepe, The Statistical Evaluation of Medical Tests for Classification and Prediction (Oxford University Press, New York, 2003).
21. C. E. Metz, " ROC methodology in radiologic imaging.," Invest. Radiol. 0020-9996 21, 720-733 (1986). 10.1097/00004424-198609000-00009
22. R. F. Wagner, C. E. Metz, and G. Campbell, " Assessment of medical imaging systems and computer aids: A tutorial review.," Acad. Radiol. 1076-6332 14, 723-748 (2007). 10.1016/j.acra.2007.03.001 (Pubitemid 46898203)
23. J. P. Egan, Signal Detection Theory and ROC-Analysis (Academic, New York, 1975).
24. C. E. Metz and X. Pan, " 'Proper binormal ROC curves: Theory and maximum-likelihood estimation.," J. Math. Psychol. 0022-2496 43, 1-33 (1999). 10.1006/jmps.1998.1218
25. K. Drukker, N. P. Gruszauskas, C. A. Sennett, and M. L. Giger, " Breast US computer-aided diagnosis workstation: Performance with a large clinical diagnostic population.," Radiology 0033-8419 248, 392-397 (2008). 10.1148/radiol.2482071778
26. I. Nabney, NETLAB: Algorithms for Pattern Recognitions (Springer, New York, 2002).
27. Y. Jiang, " Uncertainty in the output of artificial neural networks.," IEEE Trans. Med. Imaging 0278-0062 22, 913-921 (2003). 10.1109/TMI.2003.815061
28. J. Lampinen and A. Vehtari, " Bayesian approach for neural networks-review and case studies.," Neural Networks 0893-6080 14, 257-274 (2001). 10.1016/S0893-6080(00)00098-8 (Pubitemid 32288857)
29. B. Sahiner, H. P. Chan, and L. Hadjiiski, " Classifier performance estimation under the constraint of a finite sample size: resampling schemes applied to neural network classifiers.," Proceedings of IJCNN (IEEE, Orlando, 2007), pp. 1762-1766.
30. W. A. Yousef, R. F. Wagner, and M. H. Loew, " Comparison of non-parametric methods for assessing classifier performance in terms of ROC parameters.," Proceedings of the 33rd AIPR Workshop (IEEE, Washington, 2004), pp. 190-195.
31. W. S. Sarle, " Stopped training and other remedies for overfitting.," Proceedings of the 27th Symposium on the Interface: Computer Science and Statistics (Interface Foundation of North America, Pittsburgh, 1995), pp. 352-360.
32. H. P. Chan, B. Sahiner, R. F. Wagner, and N. Petrick, " Classifier design for computer-aided diagnosis: Effects of finite sample size on the mean performance of classical and neural network classifiers.," Med. Phys. 0094-2405 26, 2654-2668 (1999). 10.1118/1.598805 (Pubitemid 30007711)
33. J. Shiraishi, L. Pesce, C. E. Metz, and K. Doi, " On experimental design and data analysis in receiver operating characteristic (ROC) studies: Lessons learned from papers published in Radiology from 1997 to 2006.," Radiology 0033-8419 (in press).