Speeding up greedy forward selection for regularized least-squares

Proceedings - 9th International Conference on Machine Learning and Applications, ICMLA 2010

Volumn , Issue , 2010, Pages 325-330

  Pahikkala, Tapio ^a   Airola, Antti ^a   Salakoski, Tapio ^a  

^a UNIVERSITY OF TURKU   (Finland)

Author keywords

[No Author keywords available]

Indexed keywords

CROSS VALIDATION; DATA SETS; FEATURE SELECTION; FEATURE SETS; GREEDY FORWARD; LEAST SQUARE; LEAST SQUARES SUPPORT VECTOR MACHINES; LEAVE-ONE-OUT; MATRIX CALCULUS; NOVEL ALGORITHM; RIDGE REGRESSION; SIDE EFFECT; TIME COMPLEXITY; TRAINING ALGORITHMS; TRAINING EXAMPLE;

LEARNING ALGORITHMS; LEARNING SYSTEMS; REGRESSION ANALYSIS;

FEATURE EXTRACTION;

EID: 79952369514     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1109/ICMLA.2010.55     Document Type: Conference Paper

References (19)

1. R. Rifkin, "Everything old is new again: A fresh look at historical approaches in machine learning," Ph.D. dissertation, Massachusetts Institute of Technology, Cambridge, Massachusetts, USA, 2002.
2. J. Suykens, T. Van Gestel, J. De Brabanter, B. De Moor, and J. Vandewalle, Least Squares Support Vector Machines. World Scientific Pub. Co., Singapore, 2002.
3. A. E. Hoerl and R. W. Kennard, "Ridge regression: Biased estimation for nonorthogonal problems," Technometrics, vol. 12, pp. 55-67, 1970.
4. T. Pahikkala, E. Tsivtsivadze, A. Airola, J. Boberg, and T. Salakoski, "Learning to rank with pairwise regularized least-squares," in SIGIR 2007 Workshop on Learning to Rank for Information Retrieval, T. Joachims, H. Li, T.-Y. Liu, and C. Zhai, Eds., 2007, pp. 27-33.
5. T. Pahikkala, E. Tsivtsivadze, A. Airola, J. Boberg, and J. Järvinen, "An efficient algorithm for learning to rank from preference graphs," Machine Learning, vol. 75, no. 1, pp. 129-165, 2009.
6. I. Guyon and A. Elisseeff, "An introduction to variable and feature selection," Journal of Machine Learning Research, vol. 3, pp. 1157-1182, 2003.
7. R. Kohavi and G. H. John, "Wrappers for feature subset selection," Artificial Intelligence, vol. 97, no. 1-2, pp. 273-324, 1997.
8. G. H. John, R. Kohavi, and K. Pfleger, "Irrelevant features and the subset selection problem," in Proceedings of the Eleventh International Conference on Machine Learning, W. W. Cohen and H. Hirsch, Eds. San Fransisco, CA: Morgan Kaufmann Publishers, 1994, pp. 121-129.
9. E. K. Tang, P. N. Suganthan, and X. Yao, "Gene selection algorithms for microarray data based on least squares support vector machine," BMC Bioinformatics, vol. 7, p. 95, 2006.
10. F. Ojeda, J. A. Suykens, and B. D. Moor, "Low rank updated LS-SVM classifiers for fast variable selection," Neural Networks, vol. 21, no. 2-3, pp. 437-449, 2008, advances in Neural Networks Research: IJCNN '07.
11. R. Rifkin and R. Lippert, "Notes on regularized least squares," Massachusetts Institute of Technology, Tech. Rep. MIT-CSAIL-TR-2007-025, 2007.
12. T. Pahikkala, A. Airola, and T. Salakoski, "Feature selection for regularized least-squares: New computational short-cuts and fast algorithmic implementations," in Proceedings of the Twentieth IEEE International Workshop on Machine Learning for Signal Processing (MLSP 2010), S. Kaski, D. J. Miller, E. Oja, and A. Honkela, Eds. IEEE, 2010.
13. T. Pahikkala, A. Airola, P. Naula, and T. Salakoski, "Greedy RankRLS: a linear time algorithm for learning sparse ranking models," in SIGIR 2010 Workshop on Feature Generation and Selection for Information Retrieval, E. Gabrilovich, A. J. Smola, and N. Tishby, Eds. ACM, 2010, pp. 11-18.
14. S. R. Searle, Matrix Algebra Useful for Statistics. New York, NY: John Wiley and Sons, Inc., 1982.
15. C. Saunders, A. Gammerman, and V. Vovk, "Ridge regression learning algorithm in dual variables," in Proceedings of the Fifteenth International Conference on Machine Learning. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 1998, pp. 515-521.
16. L. Bottou and C.-J. Lin, "Support vector machine solvers," in Large-Scale Kernel Machines, ser. Neural Information Processing, L. Bottou, O. Chapelle, D. DeCoste, and J. Weston, Eds. Cambridge, MA, USA: MIT Press, 2007, pp. 1-28.
17. J. Shao, "Linear model selection by cross-validation," Journal of the American Statistical Association, vol. 88, no. 422, pp. 486-494, 1993.
18. T. Pahikkala, J. Boberg, and T. Salakoski, "Fast n-fold cross-validation for regularized least-squares," in Proceedings of the Ninth Scandinavian Conference on Artificial Intelligence (SCAI 2006), T. Honkela, T. Raiko, J. Kortela, and H. Valpola, Eds. Espoo, Finland: Otamedia, 2006, pp. 83-90.
19. S. An, W. Liu, and S. Venkatesh, "Fast cross-validation algorithms for least squares support vector machine and kernel ridge regression," Pattern Recognition, vol. 40, no. 8, pp. 2154-2162, 2007.