Partial Lanczos extreme learning machine for single-output regression problems

Neurocomputing

Volumn 72, Issue 13-15, 2009, Pages 3066-3076

  Tang, Xiaoliang ^a   Han, Min ^a  

^a DALIAN UNIVERSITY OF TECHNOLOGY   (China)

Author keywords

Extreme learning machine; Generalized cross validation; Lanczos bidiagonalization; Regularization; Singular value decomposition

Indexed keywords

INVERSE PROBLEMS; ITERATIVE METHODS; KNOWLEDGE ACQUISITION; LEARNING SYSTEMS;

EXTREME LEARNING MACHINE; GENERALIZATION PERFORMANCE; GENERALIZED CROSS VALIDATION; ILL-CONDITIONING; LANCZOS; MOORE-PENROSE GENERALIZED INVERSE; REGRESSION PROBLEM; REGULARIZATION;

SINGULAR VALUE DECOMPOSITION;

ARTICLE; CONTROLLED STUDY; EXTREME LEARNING MACHINE; HISTOGRAM; INTERMETHOD COMPARISON; LEARNING ALGORITHM; MACHINE LEARNING; MATHEMATICAL COMPUTING; PARTIAL LANCZOS EXTREME LEARNING MACHINE; PRIORITY JOURNAL; PROCESS DEVELOPMENT; REGRESSION ANALYSIS; STIMULUS GENERALIZATION; SUPPORT VECTOR MACHINE; VALIDATION PROCESS;

EID: 78049530568     PISSN: 09252312     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.neucom.2009.03.016     Document Type: Article

References (44)

1. G.B. Huang, Q.Y. Zhu, C.K. Siew, Extreme learning machine: theory and applications, Neurocomputing 70 (1-3) (2006) 489-501.
2. G.B. Huang, Q.Y. Zhu, K.Z. Mao, C.K. Siew, P. Saratchandran, N. Sundararajan, Can threshold networks be trained directly?, IEEE Trans. Circuits Syst. II Exp. Briefs 53 (3) (2006) 187-191.
3. G.B. Huang, Q.Y. Zhu, C.K. Siew, Extreme learning machine: a new learning scheme of feedforward neural networks, in: IEEE International Joint Conference on Neural Networks, Budapest, Hungary, 2004.
4. R.X. Zhang, G.B. Huang, N. Sundararajan, P. Saratchandran, Multicategory classification using an extreme learning machine for microarray gene expression cancer diagnosis, IEEE ACM Trans. Comput. Biol. Bioinformatics 4 (3) (2007) 485-495.
5. G.B. Huang, L. Chen, C.K. Siew, Universal approximation using incremental constructive feedforward networks with random hidden nodes, IEEE Trans. Neural Networks 17 (4) (2006) 879-892.
6. F. Han, D.S. Huang, Improved extreme learning machine for function approximation by encoding a priori information, Neurocomputing 69 (16-18) (2006) 2369-2373.
7. A.H. Nizar, Z.Y. Dong, Y. Wang, Power utility nontechnical loss analysis with extreme learning machine method, IEEE Trans. Power Syst. 23 (3) (2008) 946-955.
8. C.W.T. Yeu, M.H. Lim, G.B. Huang, A. Agarwal, Y.S. Ong, A new machine learning paradigm for terrain reconstruction, IEEE Geosci. Remote Sensing Lett. 3 (3) (2006) 382-386.
9. G. Wang, Y. Zhao, D. Wang, A protein secondary structure prediction framework based on the Extreme Learning Machine, Neurocomputing 72 (1-3) (2008) 262-268.
10. G.-B. Huang, L. Chen, Enhanced random search based incremental extreme learning machine, Neurocomputing 71 (16-18) (2008) 3460-3468.
11. G.B. Huang, L. Chen, Convex incremental extreme learning machine, Neurocomputing 70 (16-18) (2007) 3056-3062.
12. G.B. Huang, M.B. Li, L. Chen, C.K. Siew, Incremental extreme learning machine with fully complex hidden nodes, Neurocomputing 71 (4-6) (2008) 576-583.
13. Q.Y. Zhu, A.K. Qin, P.N. Suganthan, G.B. Huang, Evolutionary extreme learning machine, Pattern Recognition 38 (10) (2005) 1759-1763.
14. X.K. Wei, Y.H. Li, Linear programming minimum sphere set covering for extreme learning machines, Neurocomputing 71 (4-6) (2008) 570-575.
15. H.-J. Rong, Y.-S. Ong, A.-H. Tan, Z. Zhu, A fast pruned-extreme learning machine for classification problem, Neurocomputing 72 (1-3) (2008) 359-366.
16. G.B. Huang, Q.Y. Zhu, C.K. Siew, Real-time learning capability of neural networks, IEEE Trans. Neural Networks 17 (4) (2006) 863-878.
17. N.Y. Liang, G.B. Huang, P. Saratchandran, N. Sundararajan, A fast and accurate online sequential learning algorithm for feedforward networks, IEEE Trans. Neural Networks 17 (6) (2006) 1411-1423.
18. G.H. Golub, C.F.V. Loan, Matrix Computations, third ed., Johns Hopkins University Press, Maryland, 1996.
19. P.C. Hansen, Rank-Deficient and Discrete Ill-Posed Problems: Numerical Aspects of Linear Inversion, SIAM, Philadelphia, 1998.
20. H. Engl, M. Hanke, A. Neubauer, Regularization of Inverse Problems (Mathematics and Its Applications), Springer, Chicago, 2000.
21. R.D. Fierro, G.H. Golub, P.C. Hansen, D.P. Oleary, Regularization by truncated total least squares, SIAM J. Sci. Comput. 18 (4) (1997) 1223-1241.
22. L. Hu, H. Wang, B. Zhao, W. Yang, A hybrid reconstruction algorithm for electrical impedance tomography, Meas. Sci. Technol. 18 (3) (2007) 813-818.
23. H.D. Simon, H.Y. Zha, Low-rank matrix approximation using the Lanczos bidiagonalization process with applications, SIAM J. Sci. Comput. 21 (6) (2000) 2257-2274.
24. L. Elden, Partial least-squares vs. Lanczos bidiagonalization-I: analysis of a projection method for multiple regression, Comput. Stat. Data Anal. 46 (1) (2004) 11-31.
25. J. Baglama, L. Reichel, Restarted block Lanczos bidiagonalization methods, Numer. Algorithm 43 (3) (2006) 251-272.
26. M.E. Kilmer, D.P. O'Leary, Choosing regularization parameters in iterative methods for ill-posed problems, SIAM J. Matrix Anal. Appl. 22 (4) (2001) 1204-1221.
27. L. Reichel, H. Sadok, A. Shyshkov, Greedy Tikhonov regularization for large linear ill-posed problems, Int. J. Comput. Math. 84 (8) (2007) 1151-1166.
28. D. Calvetti, L. Reichel, Tikhonov regularization of large linear problems, Bit 43 (2) (2003) 263-283.
29. D. Calvetti, L. Reichel, Tikhonov regularization with a solution constraint, SIAM J. Sci. Comput. 26 (1) (2004) 224-239.
30. A. Bjorck, E. Grimme, P. Vandooren, An implicit shift bidiagonalization algorithm for ill-posed systems, Bit 34 (4) (1994) 510-534.
31. Y. Saad, Iterative Methods for Sparse Linear Systems, second ed., SIAM, Philadelphia, 2003.
32. J. Baglama, L. Reichel, Augmented implicitly restarted Lanczos bidiagonalization methods, SIAM J. Sci. Comput. 27 (1) (2005) 19-42.
33. Å. Björck, Numerical Methods for Least Squares Problems, SIAM, Philadelphia, 1996.
34. C.C. Paige, M.A. Saunders, LSQR: an algorithm for sparse linear equations and sparse least squares, ACM Trans. Math. Software 8 (1) (1982) 43-71.
35. Y. Saad, On the rates of convergence of the Lanczos and the block-Lanczos methods, SIAM J. Numer. Anal. 17 (5) (1980) 687-706.
36. G.H. Golub, M.T. Heath, G. Wahba, Generalized cross-validation as a method for choosing a good ridge parameter, Technometrics 21 (1979) 215-223.
37. S. Sundararajan, S. Shevade, S.S. Keerthi, Fast generalized cross-validation algorithm for sparse model learning, Neural Comput. 19 (1) (2007) 283-301.
38. M.A. Lukas, Strong robust generalized cross-validation for choosing the regularization parameter, Inverse Probl. 24 (3) (2008) 034066.
39. A.J. Smola, B. Schölkopf, A tutorial on support vector regression, Stat. Comput. 14 (3)(2004)199-222.
40. A. Asuncion, D.J. Newman, UCI Machine Learning Repository, University of California, School of Information and Computer Science, Irvine, CA, 2007, < http://www.ics.uci.edu/~mlearn/MLRepositoryhtml>.
41. C.C. Chang, C.J. Lin, LIBSVM: a library for support vector machines, 2001. Software available at .
42. D.C. Montgomery, Design and Analysis of Experiments, Wiley, New York, 2004.
43. T.G. Dietterich, Approximate statistical tests for comparing supervised classification learning algorithms, Neural Comput. 10 (7) (1998) 1895-1923.
44. MathWorks, Compiler execution speed comparison [Online]. Available: < http://www.mathworks.com/products/compiler/examples/example2.shtml>, MathWorks, Inc., 2003.