The new ERA in supervised learning

Neural Networks

Volumn 10, Issue 2, 1997, Pages 343-352

  Gorse, Denise ^a   Shepherd, Adrian J ^a   Taylor, John G ^b  

^a UNIVERSITY COLLEGE LONDON   (United Kingdom)

^b KING'S COLLEGE LONDON   (United Kingdom)

Author keywords

global optimisation; homotopy; local minima; range expansion

Indexed keywords

APPROXIMATION THEORY; ERRORS; FUNCTIONS; NEURAL NETWORKS; OPTIMIZATION; VECTORS;

EXPANDED RANGE APPROXIMATION; HOMOTOPY; LOCAL MINIMA; RANGE EXPANSION; SUPERVISED LEARNING;

LEARNING SYSTEMS;

ARTICLE; ARTIFICIAL NEURAL NETWORK; LEARNING; PRIORITY JOURNAL;

EID: 0031105896     PISSN: 08936080     EISSN: None     Source Type: Journal    
DOI: 10.1016/S0893-6080(96)00090-1     Document Type: Article

References (25)

1. Barhen, J., Fijany, A., & Toomarian, N. (1994). Globally optimal neural learning. In Proceedings of WCNN'94. Hillsdale, NJ: Lawrence Erlbaum Associates & INNS Press, pp. III-370-III-375.
2. Battiti, R. (1992). First- and second-order methods for learning: between steepest descent and Newton's method. Neural Computation, 4, 141-166.
3. Blake, A., & Zisserman, A. (1987). Visual Reconstruction. Cambridge, MA: MIT Press.
4. Blum, E. K. (1989). Approximation of Boolean functions by sigmoidal networks: Part I: XOR and other two-variable functions. Neural Computation, 1, 532-540.
5. Brady, M. L., Raghavan, R., & Slawny, J. (1989). Back propagation fails to separate where perceptrons succeed. IEEE Transactions on Circuits and Systems, 36, 665-674.
6. Cetin, B. C., Barhen, J., & Burdick, J. W. (1993). Terminal repeller unconstrained subenergy tunnelling (TRUST) for fast global optimization. Journal of Optimization Theory and Applications, 77, 97-126.
7. Darken, C., & Moody, J. M. (1991). Towards faster stochastic gradient search. In Advances in Neural Information Systems 4. San Mateo, CA: Morgan Kauf\fman, pp. 1009-1016.
8. Finoff, W., & Zimmerman, H. G. (1992). Homotopy methods in circuit analysis. Preprint, Siemens (Munich).
9. Gori, M., & Tesi, A. (1992). On the problem of local minima in backpropagation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14, 76-86.
10. Gorse, D., Shepherd, A., & Taylor, J.G. (1993). Avoiding local minima using a range expansion algorithm. Neural Network World, 5, 503-510.
11. Gorse, D., Shepherd, A., & Taylor, J. G. (1994). A classical algorithm for avoiding local minima. In Proceedings of WCNN'94. Hillsdale, NJ: Lawrence Erlbaum Associates & INNS Press, pp. III-364-III-369.
12. Holland, J. (1975). Adaptation in Natural and Artificial Systems. Ann Arbor, MI: University of Michigan Press.
13. Ingber, L. (1989). Very fast simulated re-annealing. Mathematical Computational Modelling, 12, 967-973.
14. Kinsella, J. A. (1992). Comparison and evaluation of variants of the conjugate gradient method for efficient learning in feed-forward neural networks with backward error propagation. Network, 3, 27-35.
15. Kirkpatrick, S., Gelatt, C. D. Jr., & Vecchi, M. P. (1983). Optimization by simulated annealing. Science, 220, 671-680.
16. Levy, A. V., & Montalvo, A. (1985). The tunnelling algorithm for the global minimization of functions. SIAM Journal on Scientific and Statistical Computing, 6, 15-29.
17. Lisboa, P. J. G., & Perantonis, S. J. (1991). Complete solution of the local minima in the XOR problem. Network, 2, 119-124.
18. McInerney, J. M., Haines, K. G., Biafore, S., & Hecht-Nielsen, R. (1989). Error surfaces of multi-layer networks can have local minima (UCSD Technical Report No. CS89-157) San Diego: UCSD.
19. Rumelhart, D. E., & McClelland, J. L. (1986). Parallel Distributed Processing: Explorations in the Microstructure of Cognition: Vo1. 1. Cambridge, MA: MIT Press, chap. 8.
20. Sontag, E.D., & Sussman, H.J. (1989). Backpropagation can give rise to spurious local minima even for networks with no hidden layers. Complex Systems, 3, 91-106.
21. Sontag, E.D., & Sussman, H.J. (1991). Back propagation separates where perceptrons do. Neural Networks, 4, 243-249.
22. Szu, H., & Hartley, R. (1987). Fast simulated annealing. Physics Letters, 122, 157-162.
23. Wallace, D. J. (1987). Neural network models: a physicist's primer. In R. D. Kenway and G. S. Pawley (Eds.), Computational Physics (SUSSP 32). Edinburgh: Scottish Universities Summer Schools in Physics, pp. 168-211.
24. Werbos, P. (1974). Beyond Regression: New Tools for Prediction and Analysis in the Behavioral Sciences. Unpublished doctoral dissertation, Harvard University, Cambridge, MA.
25. Yao, Y. (1989). Dynamic tunnelling algorithms for global optimization. IEEE Transactions on Systems, Man and Cybernetics, 19, 1222-1230