Convergence of a gradient algorithm with penalty for training two-layer neural networks

Proceedings - 2009 2nd IEEE International Conference on Computer Science and Information Technology, ICCSIT 2009

Volumn , Issue , 2009, Pages 76-79

  Shao, Hongmei ^a   Liu, Lijun ^b   Zheng, Gaofeng ^c  

^a CHINA UNIVERSITY OF PETROLEUM   (China)

^b DALIAN NATIONALITIES UNIVERSITY   (China)

^c JANA Solutions Inc   (Japan)

Author keywords

[No Author keywords available]

Indexed keywords

BOUNDEDNESS; CONVERGENCE THEOREM; ERROR FUNCTION; GRADIENT ALGORITHM; GRADIENT LEARNING ALGORITHM; LINEARLY SEPARABLE; NUMERICAL EXPERIMENTS; PENALTY TERM; SIMULATION RESULT; TWO LAYERS;

COMPUTER SCIENCE; FEEDFORWARD NEURAL NETWORKS; INFORMATION TECHNOLOGY; LEARNING ALGORITHMS; THEOREM PROVING;

CONVERGENCE OF NUMERICAL METHODS;

EID: 70449130025     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICCSIT.2009.5234616     Document Type: Conference Paper

References (14)

1. B. Z. Xu, B. L. Zhang and G. Wei, Theory and Application of Neural Networks. Guangzhou: South China University of Technology Press, 1994.
2. S. Haykin, Neural Networks: A Comprehensive Foundation, 2nd Edition. Beijing: Tsinghua University Press and Prentice Hall, 2001.
3. W. Wu and Y. S. Xu, Deterministic convergence of an online gradient method for neural networks. Journal of Computational and Applied Mathematics, vol. 144(1-2), pp. 335-347, 2002.
4. N. M. Zhang, W. Wu and G. F. Zheng, Convergence of gradient method with momentum for two-Layer feedforward neural networks. IEEE Trans. Neural Networks, vol. 17(2), pp. 522-525, 2006.
5. H. M. Shao, W. Wu, F. Li and G. F. Zheng, Convergence of batch gradient algorithm for feedforward neural network training. Journal of Information and Computational Science, vol. 4(1), pp. 251-255, 2007.
6. R. Reed, Pruning algorithms-a survey. IEEE Trans. Neural Networks, vol. 4(5), pp. 740-747, 1993.
7. C. G. Looney, Advances in feedforward neural networks: demystifying knowledge acquiring black boxes. IEEE Trans. Knowledge and Data Engineering, vol. 8, pp. 465-493, 1992.
8. R. Setiono, A penalty-function approach for pruning feedforward neural networks. Neural Computation, vol. 9, pp. 185-204, 1997.
9. A. S. Weigend, D. E. Rumelhart and B. A. Huberman, Generalization by weight-elimination applied to currency exchange rate prediction. Int. Conf. on Neural Networks, Singapore, pp. 2374-2379, 1991.
10. G. E. Hinton, Connectionist learning procedures. Artificial Intelligence, vol. 40, pp. 185-234, 1989.
11. M. Ishikawa, Structual learning with forgetting. Neural Networks, vol. 9(3), pp. 509-521, 1996.
12. S. F. McLoone and G. W. Irwin, Improving neural network training solutions using regularisation. Neurocomputing, vol. 37, pp. 71-90, 2001.
13. Y. X. Yuan and W. Y. Sun, Optimization Theory and Methods. Beijing: Science Press, 2001.
14. J. M. Ortega and W. C. Rheinboldt, Iterative solution of nonlinear equations in several variables. New York: Academic Press, 1970.