Some greedy learning algorithms for sparse regression and classification with Mercer kernels

Journal of Machine Learning Research

Volumn 3, Issue 4-5, 2003, Pages 781-801

  Nair, Prasanth B ^a   Choudhury, Arindam ^a   Keane, Andy J ^a  

^a UNIVERSITY OF SOUTHAMPTON   (United Kingdom)

Author keywords

Classification; Greedy algorithms; Kernel methods; Model selection; Regression; Sparse learning machines

Indexed keywords

COMPUTATIONAL COMPLEXITY; DATA REDUCTION; LEARNING ALGORITHMS; MATHEMATICAL MODELS; REGRESSION ANALYSIS;

DATASETS; GREEDY ALGORITHMS; MERCER KERNELS; MINIMUM DESCRIPTIVE LENGTH; SPARSE NONLINEAR REGRESSION;

LEARNING SYSTEMS;

EID: 0141607828     PISSN: 15324435     EISSN: None     Source Type: Journal    
DOI: None     Document Type: Article

References (33)

1. Adler, J., Rao, B. D. & Kreutz-Delgado, K. (1996). Comparison of basis selection methods. Proceedings of the 30th Asilomar Conference on Signals, Systems and Computers, Pacific Grove, CA, 1, 252-257.
2. 2 loss: regression and classification. Research Report No. 98, Seminar für Statistik, ETH Zürich.
3. Chen, S. S., Donoho, D. L. & Saunders, M. A. (1998). Atomic decomposition by basis pursuit. SIAM Journal on Scientific Computing, 20, 33-61.
4. Chen, S., Cowan, C. F. N., & Grant, P. M. (1991). Orthogonal least squares learning for radial basis function networks. IEEE Transactions on Neural Networks, 2(2), 302-309.
5. Chen, S., (2001). Local regularization assisted orthogonal least squares regression. IEEE Transactions on Neural Networks, submitted.
6. Chow, E., & Saad, Y. (1997). Approximate inverse techniques for block-partitioned matrices. SIAM Journal of Scientific and Statistical Computing, 18(6), 1657-1675.
7. Chow, E., & Saad, Y. (1998). Approximate inverse preconditioners via sparse-sparse iterations. SIAM Journal of Scientific and Statistical Computing, 19(3), 955-1023.
8. Couvreur, C., & Bresler, Y. (1999). On the optimality of the backward greedy algorithm for the subset selection problem. SIAM Journal on Matrix Analysis and Its Applications, 21 (3), 797-808.
9. Daniel, J. W., Gragg, W. B., Kaufmann, L., & Stewart, G. W. (1976). Reorthogonalization and stable algorithms for updating the Gram Schmidt QR factorization. Mathematics of Computation, 30 (136), 772-795.
10. Evgeniou, T., Pontil M., & Poggio T. (2000). Regularization networks and support vector machines. Advances in Computational Mathematics, 13, 1-50.
11. Freund, Y. & Schapire, R. E. (1996). Experiments with a new boosting algorithm. Proceedings of the 13th International Conference on Machine Learning. Morgan Kauffman, San Francisco, 148-156.
12. Friedman, J. H. (2001). Greedy function approximation: a gradient boosting machine. Annals of Statistics, 29 (5), 1189-1232.
13. Friedman, J. H., Hastie, T. & Tibshirani, R. (2000). Additive logistic regression: a statistical view of boosting. Annals of Statistics, 28, 337-407 (with discussion).
14. Girosi, F. (1998). An equivalence between sparse approximation and support vector machines. Neural Computation, 10, 1455-1480.
15. Grote, M. J. & Huckle, T. (1997). Parallel preconditioning with sparse approximate inverses. SIAM Journal on Scientific Computing, 18(3), 838-853.
16. Hansen M. H. & Yu, B. (2001). Model selection and the principle of minimum description length. Journal of the American Statistical Association, 96(454), 746-774.
17. Judd, K. & Mees, A. (1995). On selecting models for nonlinear time-series. Physica D, 82(4), 426-444.
18. Mallat, S. & Zhang, Z. (1993). Matching pursuit in a time-frequency dictionary. IEEE Transactions on Signal Processing, 41, 3397-3415.
19. Natarajan, B. K. (1995). Sparse approximate solutions to linear systems. SIAM Journal of Computing, 25(2), 227-234.
20. Natarajan, B. K. (1999). On learning functions from noise-free and noisy examples via Occam's razor. SIAM Journal of Computing, 29(3), 712-727.
21. Reichel, L., & Gragg, W. B. (1990). Algorithm 686: FORTRAN Subroutines for Updating the QR Decomposition. ACM Transactions on Mathematical Software, 16 (4), 369-377.
22. Ripley, B. D. (1996). Pattern Recognition and Neural Networks. Cambridge University Press.
23. Schaback, R., & Wendland, H. (2000). Adaptive greedy techniques for approximate solution of large RBF systems. Numerical Algorithms, 24(3), 239-254.
24. Schölkopf, B., Sung, K., Burges, C., Girosi, F., Niyogi, P., Poggio, T. & V. Vapnik. (1997). Comparing support vector machines with Gaussian kernels to radial basis function classifiers. IEEE Transactions on Signal Processing, 45, 2758-2765.
25. Schölkopf, B., Smola, A. J., Williamson, R., & Bartlett, P. (2000). New support vector algorithms. Neural Computation, 12(5), 1207-1245.
26. Smola, A. J., & Bartlett, P. L. (2001). Sparse greedy Gaussian process regression. Advances in Neural Information Processing Systems 13, MIT Press, 619-625.
27. Smola, A. J., & Schölkopf, B. (2000). Sparse greedy matrix approximation for machine learning. Proceedings of the 17th International Conference on Machine Learning, Morgan Kaufmann, 911-918.
28. Sugiura, N. (1978). Further analysis of the data by Akaike's information criterion and the finite corrections. Communications in Statistics, A7, 13-26.
29. Tipping, M. E. (2001). Sparse Bayesian learning and the relevance vector machine. Journal of Machine Learning Research, 1, 211-244.
30. Vapnik, V. (1995). The nature of statistical learning. New York: Springer Verlag.
31. Vincent, P., & Bengio, Y. (2001). Kernel Matching Pursuit. Machine Learning, 48 (1-3), 165-187.
32. Williams, C., & Seeger, M. (2001). Using the Nyström method to speed up kernel machines. Advances in Neural Information Processing Systems 13, MIT Press, 682-688.
33. Zhang, T., (2001). Some sparse approximation bounds for regression problems. Proceedings of the 18th International Conference on Machine Learning, Morgan Kaufmann, 624-631.