Improving gradient-based learning algorithms for large scale feedforward networks

Proceedings of the International Joint Conference on Neural Networks

Volumn , Issue , 2009, Pages 3212-3219

  Ventresca, M ^a   Tizhoosh, H R ^a  

^a UNIVERSITY OF WATERLOO   (Canada)

Author keywords

Backpropgation; Gradient based learning; Large scale networks; Opposite transfer functions; Opposition based computing

Indexed keywords

BACKPROPGATION; GRADIENT-BASED LEARNING; LARGE SCALE NETWORKS; OPPOSITE TRANSFER FUNCTIONS; OPPOSITION-BASED COMPUTING;

BACKPROPAGATION ALGORITHMS; COMPUTER SCIENCE; CONVERGENCE OF NUMERICAL METHODS; EDUCATION; LEARNING SYSTEMS; NEURAL NETWORKS; SCALES (WEIGHING INSTRUMENTS); TRANSFER FUNCTIONS;

LEARNING ALGORITHMS;

EID: 70449455565     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IJCNN.2009.5178798     Document Type: Conference Paper

References (22)

1. D. N. A. Asuncion. UCI machine learning repository, 2008.
2. W. Duch and N. Jankowski. New neural transfer functions. Applied Mathematics and Computer Science, 7:639-658, 1997.
3. W. Duch and N. N. Jankowski. Transfer functions: Hidden possibilities for better neural networks. In 9th European Symposium on Artificial Neural Networks, pages 81-94, 2001.
4. M. T. Hagan, H. B. Demuth, and M. H. Beale. Neural Network Design. PWS Publishing, 1996.
5. N. Lagaros and M. Papadrakakis. Learning improvement of neural networks used in structural optimization. Advances in Software Engineering, 35 (1) :9-25, 2004.
6. S. Rahnamayn, H. R. Tizhoosh, and S. Salama. A Novel Population Initialization Method for Accelerating Evolutionary Algorithms. (to appear) Computers and Mathematics with Applications, 2006.
7. S. Rahnamayn, H. R. Tizhoosh, and S. Salama. Opposition-based Differential Evolution Algorithms. In IEEE Congress on Evolutionary Computation, pages 7363-7370, 2006.
8. S. Rahnamayn, H. R. Tizhoosh, and S. Salama. Opposition-based Differential Evolution Algorithms for Optimization of Noisy Problems. In IEEE Congress on Evolutionary Computation, pages 6756-6763, 2006.
9. Haykin. S. Neural Networks: A Comprehensive Foundation. Prentice Hall, 2 edition, 1999.
10. S. Saarinen, R. Bramly, and G. Cybenko. Ill-conditioning in neural network training problems. SIAM Journal on Scientific Computing, 14 (3) :693-714, 1993.
11. U. Seiffert. Training of large-scale feedforward networks. In IEEE International Joint Conference on Neural Networks, pages 10780- 10785, 2006.
12. M. Shokri, H. R. Tizhoosh, and M. Kamel. Opposition-based Q (lambda) Algorithm. In IEEE International Joint Conference on Neural Networks, pages 646-653, 2006.
13. T. G. Tan and J. Teo. Oppositional Concepts in Computational Intelligence, chapter Evolving Opposition-Based Pareto Solutions: Multiobjective Optimization using Competitive Coevolution, pages 161-201. Springer-Verlag, 2008.
14. A. Thome and M. Tenorio. Dynamic adaptation of the error surface for the acceleration of the training of neural networks. In IEEE World Congress on Comutational Intelligence, pages 447-452, 1992.
15. H. R. Tizhoosh. Reinforcement Learning Based on Actions and Opposite Actions. In International Conference on Artificial Intelligence and Machine Learning, 2005.
16. H. R. Tizhoosh. Opposition-based Reinforcement Learning. Journal of Advanced Computational Intelligence and Intelligent Informatics, 10 (4) :578-585, 2006.
17. H. R. Tizhoosh and M. Ventresca, editors. Oppositional Concepts in Computational Intelligence. Springer-Verlag, 2008.
18. P. van der Smagt and G. Hirzinger. Solving the ill-conditioning in neural network learning. In Neural Networks: Tricks of the Trade, pages 193- 206. Springer-Verlag, 1998.
19. M. Ventresca and H. R. Tizhoosh. Improving the Convergence of Backpropagation by Opposite Transfer Functions. In IEEE International Joint Conference on Neural Networks, pages 9527-9534, 2006.
20. M. Ventresca and H. R. Tizhoosh. Opposite Transfer Functions and Backpropagation Through Time. In IEEE Symposium on Foundations of Computational Intelligence, pages 570-577, 2007.
21. M. Ventresca and H. R. Tizhoosh. Simulated Annealing with Opposite Neighbors. In IEEE Symposium on Foundations of Computational Intelligence, pages 186-192, 2007.
22. M. Ventresca and H. R. Tizhoosh. Numerical condition of feedforward networks with opposite transfer functions. In IEEE International Joint Conference on Neural Networks, pages 3232-3239, 2008.