A new convex objective function for the supervised learning of single-layer neural networks

Pattern Recognition

Volumn 43, Issue 5, 2010, Pages 1984-1992

  Fontenla Romero, Oscar ^a   Guijarro Berdiñas, Bertha ^a   Pérez Sánchez, Beatriz ^a   Alonso Betanzos, Amparo ^a  

^a UNIVERSITY OF A CORUÑA   (Spain)

Author keywords

Convex optimization; Global optimum; Incremental learning; Least squares; Single layer neural networks; Supervised learning method

Indexed keywords

GLOBAL OPTIMUM; INCREMENTAL LEARNING; LEAST SQUARE; SINGLE LAYER; SINGLE-LAYER NEURAL NETWORKS; SUPERVISED LEARNING METHODS;

COMPUTATIONAL EFFICIENCY; CONVEX OPTIMIZATION; EVOLUTIONARY ALGORITHMS; FEEDFORWARD NEURAL NETWORKS; FUNCTIONS; MEAN SQUARE ERROR; STRAIN ENERGY; SUPERVISED LEARNING; THREE TERM CONTROL SYSTEMS;

LEARNING ALGORITHMS;

EID: 75749106957     PISSN: 00313203     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.patcog.2009.11.024     Document Type: Article

References (26)

1. Bishop C.M. Neural Networks for Pattern Recognition (1995), Oxford University Press, New York
2. Pao Y.H. Adaptive Pattern Recognition and Neural Networks (1989), Addison-Wesley Publishing Company, Inc., Reading, MA
3. B. Widrow, M.A. Lehr, 30 years of adaptive neural networks: perceptron, madeline and backpropagation, in: Proceedings of the IEEE, vol. 78, 1990, pp. 1415-1442.
4. Brady M., Raghavan R., and Slawny J. Back propagation fails to separate where perceptrons succeed. IEEE Transactions on Circuits and Systems 36 (1989) 665-674
5. Sontag E., and Sussmann H.J. Backpropagation can give rise to spurious local minima even for networks without hidden layers. Complex Systems 3 (1989) 91-106
6. Budinich M., and Milotti E. Geometrical interpretation of the back-propagation algorithm for the perceptron. Physica A 185 (1992) 369-377
7. Auer P., Hebster M., and Warmuth M.K. Exponentially many local minima for single neurons. In: Touretzky D.S., Mozer M.C., and Hasselmo M.E. (Eds). Advances in Neural Information Processing Systems vol. 8 (1996), The MIT Press, Cambridge, MA 316-322
8. Sontag E., and Sussmann H.J. Back propagation separates where perceptrons do. Neural Networks 4 (1991) 243-249
9. Gori M., and Tesi A. On the problem of local minima in backpropagation. IEEE Transactions on Pattern Analysis and Machine Intelligence 14 1 (1992) 76-85
10. Coetzee F.M., and Stonick V.L. On a natural homotopy between linear and nonlinear single-layer networks. IEEE Transactions on Neural Networks 7 2 (1996) 307-317
11. Coetzee F.M., and Stonick V.L. On the uniqueness of weights in single layer perceptrons. IEEE Transactions on Neural Networks 7 2 (1996) 318-325
12. Kivinen J., and Warmuth M.K. Relative loss bounds for multidimensional regression problems. Machine Learning 45 (2001) 301-329
13. Scalero R.S., and Tepedelenlioglu N. A fast new algorithm for training feedforward neural networks. IEEE Transactions on Signal Processing 40 1 (1992) 202-210
14. Biegler-Konig F., and Barnmann F. A learning algorithm for multilayered neural networks based on linear least squares problems. Neural Networks 6 (1993) 127-131
15. Di Martino M., Fanelli S., and Protasi M. Exploring and comparing the best "direct methods" for the efficient training of MLP-networks. IEEE Transactions on Neural Networks 7 6 (1996) 1497-1502
16. Yam Y., and Chow T. A new method in determining the initial weights of feedforward neural networks. Neurocomputing 16 1 (1997) 23-32
17. Cho S.-Y., Chi Z., Siu W.-C., and Tsoi A.C. An improved algorithm for learning long-term dependency problems in adaptive processing of data structures. IEEE Transactions on Neural Networks 14 4 (2003) 781-793
18. Castillo E., Fontenla-Romero O., Alonso-Betanzos A., and Guijarro-Berdiñas B. A global optimum approach for one-layer neural networks. Neural Computation 14 6 (2002) 1429-1449
19. Carayannis G., Kalouptsidis N., and Manolakis D. Fast recursive algorithms for a class of linear equations. IEEE Transactions on Acoustics, Speech and Signal Processing ASSP-30 2 (1982) 227-239
20. Bojanczyk A. Complexity of solving linear systems in different models of computation. SIAM Journal on Numerical Analysis 21 3 (1984) 591-603
21. Minsky M., and Papert S. Perceptrons: An Introduction to Computational Geometry (1969), MIT Press, Cambridge, MA
22. Suykens J., Gestel T.V., Brabanter J.D., Moor B.D., and Vandewalle J. Least Squares Support Vector Machines (2002), World Scientific, Singapore
23. D. Newman, S. Hettich, C. Blake, C. Merz, UCI repository of machine learning databases 〈http://www.ics.uci.edu/∼mlearn/MLRepository.html〉, 1998.
24. O.L. Mangasarian, R. Ramakrishnan, Data Mining Institute, Computer Sciences Department, University of Wisconsin 〈http://www.cs.wisc.edu/dmi〉, 1999.
25. G. Fung, O.L. Mangasarian, Proximal support vector machine classifiers, in: F. Provost, R. Srikant (Eds.), Proceedings KDD-2001: Knowledge Discovery and Data Mining, August 26-29, 2001, San Francisco, CA, Association for Computing Machinery, New York, 2001, pp. 77-86.
26. V. Franc, V. Hlaváč, A new feature of the statistical pattern recognition toolbox, in: S. Scherer (Ed.), Computer Vision, Computer Graphics and Photogrammetry-A Common Viewpoint, Österreichische Computer Gesellschaft, 2001, pp. 143-150 〈http://cmp.felk.cvut.cz/cmp/software/stprtool/index.html〉.