Empirical evaluation of the improved Rprop learning algorithms

Neurocomputing

Volumn 50, Issue , 2003, Pages 105-123

  Igel, Christian ^a   Hüsken, Michael ^a  

^a RUHR UNIVERSITY BOCHUM   (Germany)

Author keywords

Gradient based optimization; Resilient backpropagation (Rprop); Supervised learning

Indexed keywords

BENCHMARKING; ERROR ANALYSIS; LEARNING ALGORITHMS; LEARNING SYSTEMS;

FIRST-ORDER LEARNING METHODS;

NEURAL NETWORKS;

ALGORITHM; ANALYTICAL ERROR; ARTIFICIAL NEURAL NETWORK; CANCER CLASSIFICATION; CONTROLLED STUDY; DISEASE CLASSIFICATION; EMPIRICISM; EVALUATION; LEARNING; MATHEMATICAL COMPUTING; PRIORITY JOURNAL; REVIEW; TECHNIQUE;

EID: 0037238922     PISSN: 09252312     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0925-2312(01)00700-7     Document Type: Article

References (27)

1. H. Braun, Neuronale Netze: Optimierung durch Lernen und Evolution, Springer, Berlin, 1997.
2. S.E. Fahlman, Faster-learning variations of back-propagation: an empirical study, in: D. Touretzky, G. Hinton, T. Sejnowski (Eds.), Proceedings of the 1988 Connectionist Models Summer School, San Mateo, Morgan Kaufmann, CA, 1988, pp. 38-51.
3. N. Hansen, A. Ostermeier, Convergence properties of evolution strategies with the derandomized covariance matrix adaptation: the (μ/μ,λ)-CMA-ES, in: EUFIT'97, Fifth European Congress on Intelligent Techniques and Soft Computing, Aachen, Verlag Mainz, Wissenschaftsverlag, 1997, pp. 650-654.
4. N. Hansen, A. Ostermeier, A. Gawelczyk, On the adaptation of arbitrary normal mutation distributions in evolution strategies: the generating set adaptation, in: L.J. Eshelman (Ed.), Proceedings of the Sixth International Conference on Genetic Algorithms, Pittsburgh, 1995, Morgan Kaufmann, San Franscisco, CA, pp. 57-64.
5. C. Igel, K. Chellapilla, Fitness distributions: tools for designing efficient evolutionary computations, in: L. Spector, W.B. Langdon, U.-M. O'Reilly, P.J. Angeline (Eds.), Advances in Genetic Programming, Vol. 3, MIT Press, Cambridge, MA, 1999, pp. 191-216 (Chapter 9).
6. C. Igel, M. Hüsken, Improving the Rprop learning algorithm, in: H. Bothe, R. Rojas (Eds.), Proceedings of the Second International Symposium on Neural Computation, NC 2000, ICSC Academic Press, Canada/Switzerland, 2000, pp. 115-121.
7. R.A. Jacobs, Increased rates of convergence through learning rate adaptation, Neural Networks 1 (4) (1988) 295-307.
8. M. Joost, W. Schiffmann, Speeding up backpropagation algorithms by using cross-entropy combined with pattern normalization, Int. J. Uncertainty, Fuzziness Knowledge-based Systems 6 (2) (1998) 117-126.
9. Y. LeCun, L. Bottou, G.B. Orr, K.-R. Müller, Efficient backprop, in: G.B. Orr, K.-R. Müller (Eds.), Neural Networks: Tricks of the Trade, Lecture Notes in Computer Science, Vol. 1524, Springer, Berlin, 1998, pp. 9-50 (Chapter 1).
10. E.N. Lorenz, Deterministic nonperiodic flow, J. Atmos. Sci. 20 (2) (1963) 130-141.
11. L.M. Patnaik, K. Rajan, Target detection through image processing and resilient propagation algorithms, Neurocomputing 35 (1-4) (2000) 123-135.
12. L. Prechelt, PROBEN1-A set of benchmarks and benchmarking rules for neural network training algorithms, Technical Report 21/94, Fakultät für Informatik, Universität Karlsruhe, 1994.
13. L. Prechelt, A quantitative study of experimental evaluations of neural network learning algorithms: current research practice, Neural Networks 9 (3) (1996) 457-462.
14. W. Press, S. Teukolsky, W. Vetterling, B. Flannery, Numerical Recipes in C, 2nd Edition, Cambridge University Press, Cambridge, 1994.
15. R.D. Reed, R.J. Marks II, Neural Smithing, MIT Press, Cambridge, MA, 1999.
16. M. Riedmiller, Untersuchungen zu Konvergenz und Generalisierungsfahigkeit überwachter Lernverfahren im SNNS, in: A. Zell (Ed.), Workshop SNNS-93: Simulation Neuronaler Netze mit SNNS, Stuttgart, Fakultät für Informatik, Universität Stuttgart, 1993.
17. M. Riedmiller, Advanced supervised learning in multi-layer perceptrons-from backpropagation to adaptive learning algorithms, Comput. Standards Interfaces 16 (5) (1994) 265-278.
18. M. Riedmiller, H. Braun, A direct adaptive method for faster backpropagation learning: the RPROP algorithm, in: E.H. Ruspini (Ed.), Proceedings of the IEEE International Conference on Neural Networks, IEEE Press, New York, 1993, pp. 586-591.
19. W. Schiffmann, M. Joost, R. Werner, Comparison of optimized backpropagation algorithms, in: M. Verleysen (Ed.), Proceedings of the European Symposium on Artificial Neural Networks, ESANN '93, Brussels, 1993, D-Facto, Brussels, pp. 97-104.
20. J.R. Shewchuk, An Introduction to the Conjugate Gradient Method Without the Agonizing Pain, 11/4 Edition, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, 1994.
21. F.M. Silva, L.B. Almeida, Acceleration techniques for the backpropagation algorithm, in: L.B. Almeida, C.J. Wellekens (Eds.), Neural Networks-EURASIP Workshop 1990, Vol. 412, Lecture Notes in Computer Science, Springer, Berlin, 1990, pp. 110-119.
22. F.M. Silva, L.B. Almeida, Speeding up backpropagation, in: R. Eckmiller (Ed.), Advanced Neural Computers, North-Holland, Amsterdam, 1990, pp. 151-158.
23. P. Stagge, B. Sendhoff, An extended Elman net for modeling time series, in: W. Gerstner (Ed.), International Conference on Artificial Neural Networks, Lecture Notes in Computer Science, Springer, Berlin, 1997.
24. T. Tollenaere, Supersab: fast adaptive backpropagation with good scaling properties, Neural Networks 3 (1990) 561-573.
25. T.P. Vogl, J.K. Mangis, A.K. Rigler, W.T. Zink, D.L. Alkon, Accelerating the convergence of the backpropagation method, Biol. Cybernet. 59 (1988) 257-263.
26. F. Wilcoxon, Individual comparison by ranking methods, Biometrics Bull. 1 (1945) 80-83.
27. W.H. Wolberg, O.L. Mangasarian, Multisurface method of pattern separation for medical diagnosis applied to breast cytology, Proc. Natl. Acad. Sci. USA 87 (1990) 9188-9192.