Fast implementation of ℓ1 regularized learning algorithms using gradient descent methods

Proceedings of the 10th SIAM International Conference on Data Mining, SDM 2010

Volumn , Issue , 2010, Pages 862-871

  Cai, Yunpeng ^a   Sun, Yijun ^b   Cheng, Yubo ^a   Li, Jian ^a   Goodison, Steve ^c  

^a University of Florida   (United States)

^b UNIVERSITY OF FLORIDA   (United States)

^c M D ANDERSON CANCER CENTER ORLANDO   (United States)

Author keywords

Feature selection; Gradient descent; Sparse solution

Indexed keywords

COMPUTATIONAL EFFICIENCY; CONVEX OPTIMIZATION; DATA MINING; FEATURE EXTRACTION; GRADIENT METHODS;

CONVEX OPTIMIZATION PROBLEMS; GRADIENT DESCENT; GRADIENT DESCENT TECHNIQUES; HIGH THROUGHPUT TECHNOLOGY; LARGE SCALE EXPERIMENTS; NON-DIFFERENTIABLE OBJECTIVES; SPARSE SOLUTIONS; STATE-OF-THE-ART ALGORITHMS;

LEARNING ALGORITHMS;

EID: 78650424208     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1137/1.9781611972801.75     Document Type: Conference Paper

References (32)

1. G. Andrew, and J.F. Gao, Scalable Training of L1-Regularized Log-Linear Models, Proc. 24th Intl. Conf. Mach. Learn., Corvallis, OR, USA, 2007, pp. 33-40.
2. S. Balakrishnan, and D. Madigan, Algorithms for sparse linear classifiers in the massive data setting, J. Mach. Learn. Res., 9 (2008), pp. 313-337.
3. M. Buyse, S. Loi, L. van't Veer, G. Viale, M. Delorenzi, A. M. Glas, M. S. d'Assignies, J. Bergh, R. Lidereau, P. Ellis, A. Harris, J. Bogaerts, P. Therasse, A. Floore, M. Amakrane, F. Piette, E. Rutgers, C. Sotiriou, F. Cardoso, and M. J. Piccart, Validation and clinical utility of a 70-gene prognostic signature for women with node-negative breast cancer. J. N. Cancer Inst., 98(17) (2006), pp. 1183-1192.
4. O. Chapelle, Training a support vector machine in the primal, Neural Comput., 19 (2007), pp. 1155-1178.
5. J. Duchi, S. Shalev-Shwartz, Y. Singer, and T. Chandra, Efficient projections onto the L1-ball for learning in high dimensions, Proc. 25th Intl. Conf. Mach. Learn., Helsinki, Finland, 2008, pp. 272-279.
6. Y. G. Evtushenkjo and V. G. Zhadan, Space-transformation technique: the state of the art, in Nonlinear Optimization and Applications, (1996), pp 101-123.
7. L. Faybusovich, Dynamical systems which solve optimization problems with linear constraints, IMA J. Math. Ctrl. and Info., 8 (1991), pp 135-149;
8. R. Fletcher, Practical methods of optimization, John Wiley, New York, 1997.
9. J. Friedman, T. Hastie, H. Hoefling and R. Tibshirani, Pathwise coordinate optimization, Ann. Appl. Stat., 1 (2007), pp. 302-332.
10. A. Genkin, D. Lewis, and D. Madigan, Large-scale bayesian logistic regression for text categorization, Technometrics, 49 (2007), pp. 291-304.
11. I. Guyon, S. R. Gunn, A. Ben-Hur and G. Dror, Result analysis of the NIPS 2003 feature selection challenge, 17th Adv. Neu. Info. Proc. Sys., Vancouver, Canada, (2005), pp. 545-552.
12. A. V. Ivshina, J. George, O. Senko, B. Mow, T. C. Putti, J. Smeds, T. Lindahl, Y. Pawitan, P. Hall, H. Nordgren, J. E. Wong, E. T. Liu, J. Bergh, V. A. Kuznetsov, and L. D. Miller, Genetic reclassification of histologic grade delineates new clinical subtypes of breast cancer, Cancer Res., 66(21) (2006), pp. 10292-10301.
13. J. Goodman, Exponential priors for maximum entropy models, Proc. 42nd Ann. Meet. Asso. Comp. Ling., Barcelona, Spain, 2004, pp. 305-312.
14. R. Horn, and C. Johnson, Matrix analysis. Cambridge University Press, Cambridge, 1985.
15. J. Kiefer, Sequential minimax search for a maximum, Proc. 4th Amer Math. Soc., (1953), pp. 502-506.
16. 1-regularized logistic regression, J. Mach. Learn. Res., 8 (2007), pp. 1519-1555.
17. J. Langford, L., Li, and T., Zhang, Sparse Online Learning via Truncated Gradient, J. Mach. Learn. Res., 10 (2009), pp. 777-801.
18. 1 regularized logistic regression, Proc. 21st AAAI Conf. Artif. Intell., Boston, MA, USA, 2006, pp. 1-9.
19. J. Lokhorst, The lasso and generalised linear models, Tech. Report, Dept. of Stat., Univ. of Adelaide, Australia, 1999.
20. A. Y. Ng, Feature selection, L1 vs. L2 regularization, and rotational invariance, Proc. 21st Intl. Conf. Mach. Learn., Banff, Alberta, Canada, 2004, pp. 78-86.
21. J. Nocedal, and S. J. Wright, Numerical optimization, Springer Verlag, New York, NY, 1999.
22. M. Park, and T. Hastie, L1 regularization-path algorithm for generalized linear models, J. R. Stat. Soc. Ser. B Stat. Meth., 69 (2007), pp. 659-677.
23. S. Perkins, and J. Theiler, Online feature selection using grafting, Proc. 20th Intl. Conf. Mach. Learn., Washington, DC, USA, 2003, pp. 592-599.
24. J. D. Rennie, and N. Srebro, Fast maximum margin matrix factorization for collaborative prediction, Proc. 22nd Intl. Conf. Mach. Learn., Bonn, Germany, 2005, pp. 713-719.
25. S. Rosset, Following curved regularized optimization solution paths, Proc. 17th Adv. Neu. Info. Proc. Sys., Whistler, Canada, 2005, pp. 1153-1160.
26. S. Rosset, J. Zhu, and T. Hastie, Boosting as a regularized path to a maximum margin classifier, J. Mach. Learn. Res., 5 (2004), pp. 941-973.
27. V. Roth, The generalized LASSO, IEEE Trans. Neu. Net., 15 (2004), pp. 16-28.
28. M. Schmidt, G. Fung, and R. Rosales, Fast optimization methods for L1 regularization: a comparative study and two new approaches, Proc. 18th Euro. Conf. Mach. Learn., Warsaw, Poland, 2007, pp. 286-297.
29. S. Shevade, and S. Keerthi, A simple and efficient algorithm for gene selection using sparse logistic regression, Bioinformatics, 19 (2003), pp. 2246-2253.
30. A. J. Stephenson, A. Smith, M. W. Kattan, J. Satagopan, V. E. Reuter, P. T. Scardino and W. L. Gerald, Integration of gene expression profiling and clinical variables to predict prostate carcinoma recurrence after radical prostatectomy, Cancer, 104 (2005), pp. 290-298.
31. R. Tibshirani, Regression shrinkage and selection via the LASSO, J. R. Stat. Soc. Ser. B, 58 (1996), pp. 267-288.
32. J. Zhu, S. Rosset, T. Hastie, and R. Tibshirani, 1-norm support vector machines, Proc. 16th Adv. Neu. Info. Proc. Sys., Whistler, Canada, 2002, pp. 49-56.