Numerical condition of feedforward networks with opposite transfer functions

Proceedings of the International Joint Conference on Neural Networks

Volumn , Issue , 2008, Pages 3233-3240

  Ventresca, Mario ^a   Tizhoosh, Hamid Reza ^a  

^a NONE

Author keywords

Feedforward; Ill conditioning; Numerical condition; Opposite transfer functions

Indexed keywords

FEEDFORWARD NEURAL NETWORKS; FUNCTIONS; IMAGE CLASSIFICATION; LEARNING ALGORITHMS; LEARNING SYSTEMS; PROBABILITY DENSITY FUNCTION; TRANSFER FUNCTIONS; VEGETATION;

ARTIFICIAL NEURAL NETWORKS; BENCHMARK PROBLEMS; EARLY STAGES; ERROR SURFACES; FEEDFORWARD; FEEDFORWARD NETWORKS; FEEDFORWARD NEURAL NETWORK ARCHITECTURES; ILL-CONDITIONING; LEARNING PROCESSES; LEARNING SPEEDS; NETWORK INITIALIZATIONS; NEW ALGORITHMS; NUMERICAL CONDITION; NUMERICAL CONDITIONS; OPPOSITE TRANSFER FUNCTIONS; TRAINING ALGORITHMS;

NEURAL NETWORKS;

EID: 56349095308     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/IJCNN.2008.4634257     Document Type: Conference Paper

References (33)

1. C. Bishop, Pattern Recognition and Machine Learning. Springer, 2007.
2. M. Riedmiller and H. Braun, "A Direct Adaptive Method for Faster Backpropagation Learning: The RPROP Algorithm," in IEEE Conference on Neural Networks, pp. 586-591, 1993.
3. S. E. Fahlman, "Faster-Learning Variations on Backpropagation: An Empirical Study." Morgan Kaufmann, 1988.
4. S. Haykin, Neural Networks: A Comprehensive Foundation (2nd Edition). Prentice Hall, 1998.
5. S. Saarinen, R. Bramley, and G. Cybenko, "Ill-Conditioning in Neural Network Training Problems," SIAM Journal on Scientific Computing, vol. 14, no. 3, pp. 693-714, 1993.
6. N. Schraudolph, "Centering Neural. Network Gradient Factors," in Neural Networks: Tricks of the Trade (G. Orr and K. R. Muller, eds.), pp. 207-226, Springer-Verlag, 1998.
7. D. Nguyen and B. Widrow, "Improving the Learning Speed of 2-Layer Neural Networks by Choosing Initial Values of the Adaptive Weights," IEEE Proceedings of the International Joint Conference on Neural Netowrks, vol. 3, pp. 21-26, 1990.
8. W. Duch and J. N., "New Neural Transfer Functions," Applied Mathematics and Computer Science, vol. 7, pp. 639-658, 1997.
9. W. Duch and N. Jankowski, "Transfer Functions: Hidden Possibilities for Better Neural Networks," in 9th European Symposium on Artificial Neural Networks, pp. 81-94, 2001.
10. W. Duch and N. Jankowski, "Optimal Transfer Function Neural Networks," in 9th European Symposium on Artificial Neural Networks, pp. 101-106, 2001.
11. A. Thome and M. Tenorio, "Dynamic Adaptation of the Error Surface for the Acceleration of the Training of Neural Networks," vol 1, pp. 447-452, 1994.
12. N. Lagaros and M. Papadrakakis, "Learning Improvement of Neural Networks used in Structural Optimization," Advances in Engineering Software, vol. 35, no. 1, pp. 9-25, 2004.
13. P. Chandra and Y. Singh, "An. Activation. Function Adapting Training Algorithm for Sigmoidal Feedforward Networks," Neurocomputing, vol. 61, 2004.
14. P. Chandra and Y. Singh, "A Case for the Self-Adaptation of Activation Functions in FFANNs," Neurocomputing, vol. 56, pp. 447-545, 2004.
15. M. Ventresca and H. R. Tizhoosh, "Improving the Convergence of Backpropagation by Opposite Transfer Functions," in IEEE International Joint Conference on Neural Networks, pp. 9527-9534, 2006.
16. M. Ventresca and H. R. Tizhoosh, "Opposite Transfer Functions and Backpropagation Through Time," in IEEE Symposium on Foundations of Computational Intelligence, pp. 570-577, 2007.
17. H. R. Tizhoosh, "Opposition-based Learning: A New Scheme for Machine Intelligence," in International Conference on Computational Intelligence for Modelling, Control and Automation, pp. 695-701, 2005.
18. M. Shokri, H. R. Tizhoosh, and M. Kamel, "Opposition-based Q(lambda) Algorithm," in IEEE international Joint Conference on Neural Networks, pp. 646-653, 2006.
19. H. R. Tizhoosh, "Opposition-based Reinforcement Learning," Journal of Advanced Computational Intelligence and intelligent Informatics, vol. 10, no. 4, pp. 578-585, 2006.
20. S. Rahnamayn, H. R. Tizhoosh, and S. Sal ama, "A Novel Population Initialization Method for Accelerating Evolutionary Algorithms," (to appear) Computers and Mathematics with Applications, 2006.
21. S. Rahnamayn, H. R. Tizhoosh, and S. Salama, "Opposition-based Differential Evolution Algorithms," in IEEE Congress on Evolutionary Computation, pp. 7363-7370, 2006.
22. S. Rahnamayn, H. R. Tizhoosh, and S. Salama, "Opposition-based Dif-ferential Evolution Algorithms for Optimization, of Noisy Problems," in IEEE Congress on Evolutionary Computation, pp. 6756-6763, 2006.
23. M. Ventresca and H. R. Tizhoosh, "Simulated Annealing with Opposite Neighbors," in IEEE Symposium on Foundations of Computational Intelligence, pp. 186-192, 2007.
24. E. I. Batakova and L. Spaanenburg, "Symmetry: Between Indecision and Equality of Choice," Biological and Artificial Computation: From Neuroscience to Technology, 1997.
25. A. Chen and R. Hecht-Nielsen, "On the Geometry of Feedforward Neural Network Weight Spaces," in Second international Conference on Artificial Neural Networks, pp. 1-4, 1991.
26. A. M. Chen, H. Lu, , and R. HechtNielsen, "On the Geometry of Feedforward Neural Networks Error Surfaces," Neural Computation, vol. 5, no. 6, pp. 910-927, 1993.
27. F. Albertini and E. Sontag, "For Neural Networks, Function Deter-mines Form," Neural Networks, vol. 6, pp. 975-990, 1993.
28. H. J. Sussmann, "Uniqueness of the Weights for Minimal Feedforward Nets with a Given Input-Output Map," Neural Networks, vol. 5, no. 4, pp. 589-593, 1992.
29. F. Jordan and G. Clement, "Using the Symmetries of Multilayered Network to Reduce the Weight Space," in IEEE Second International Joint Conference on Neural Networks, pp. 391-396, 1991.
30. S. Amari, H. Park, and T. Ozeki, "Singularities Affect Dynamics of Learning in Neuromanifolds," Neural Computation, vol. 18, pp. 1007-1065, 2006.
31. P. van der Smagt and G. Hirzinger, "Solving the Ill-Conditioning in Neural Network Learning," in Neural Networks: Tricks of the Trade (G. Orr and K. R. Muller, eds.), pp. 193-206, Springer-Verlag, 1998.
32. Y. Le Cun, J. S. Denker, and S. A. Solla, "Optimal Brain Damage," in Advances in Neural Information Processing Systems, pp. 598-605, 1990.
33. D. N. A. Asuncion, "UCI machine learning repository," 2007.