Elastic-net regularization in learning theory

Journal of Complexity

Volumn 25, Issue 2, 2009, Pages 201-230

  De Mol, Christine ^a   De Vito, Ernesto ^b,c   Rosasco, Lorenzo ^b,d  

^a UNIVERSITÉ LIBRE DE BRUXELLES   (Belgium)

^b UNIVERSITY OF GENOA   (Italy)

^c INFN SEZIONE DI GENOVA   (Italy)

^d MASSACHUSETTS INSTITUTE OF TECHNOLOGY   (United States)

Author keywords

Elastic net; Learning; Regularization; Sparsity

Indexed keywords

ITERATIVE METHODS; REGRESSION ANALYSIS; SAMPLING;

ELASTIC NET; LEARNING; MATHEMATICAL FRAMEWORKS; OPTIMIZATION PROCEDURES; REGULARIZATION; REGULARIZATION PARAMETERS; SPARSITY; STATISTICAL LEARNING THEORY;

COMPUTATION THEORY;

EID: 62549127689     PISSN: 0885064X     EISSN: 10902708     Source Type: Journal    
DOI: 10.1016/j.jco.2009.01.002     Document Type: Article

References (46)

1. Candès E., and Tao T. The Dantzig selector: Statistical estimation when p is much larger than n. Ann. Statist. 35 6 (2007) 2313-2351
2. A. Destrero, C. De Mol, F. Odone, A. Verri, A regularized approach to feature selection for face detection, in: Proceedings ACCV07, 2007, pages II: 881-890
3. Destrero A., Mosci S., De Mol C., Verri A., and Odone F. Feature selection for high-dimensional data. Comput. Manag. Sci. 6 1 (2009) 25-40
4. Destrero A., De Mol C., Odone F., and Verri A. A sparsity-enforcing method for learning face features. IEEE Trans. Image Process. 18 1 (2009) 188-201
5. C. De Mol, S. Mosci, M. Traskine, A. Verri, A regularized method for selecting nested groups of relevant genes from microarray data, J. Comp. Biol. (in press). Preprint available at: http://arxiv.org/abs/0809.1777
6. A. Barla, S. Mosci, L. Rosasco, A. Verri, A method for robust variable selection with significance assessment, in: ESANN 2008, 2008. Preprint available at: http://www.disi.unige.it/person/MosciS/PAPERS/esann.pdf
7. Tibshirani R. Regression selection and shrinkage via the lasso. J. R. Stat. Soc. Ser. B 58 (1996) 267-288
8. Chen S.S., Donoho D.L., and Saunders M.A. Atomic decomposition by basis pursuit. SIAM J. Sci. Comput. 20 1 (1998) 33-61
9. Fu W. Penalized regressions: The bridge versus the lasso. J. Comput. Graph. Statist. 7 3 (1998) 397-416
10. Koltchinskii V. Sparsity in penalized empirical risk minimization, Annales de l'Institut Henri Poincaré B. Probab. Statist. 45 1 (2009) 7-57
11. Knight K., and Fu W. Asymptotics for lasso-type estimators. Ann. Statist. 28 5 (2000) 1356-1378
12. Bunea F., Tsybakov A., and Wegkamp M. Aggregation and sparsity via l1 penalized least squares. Proc. 19th Annu. Conference on Comput. Learning Theory (2006), Springer 379-391
13. Loubes J.-M., and van de Geer S. Adaptive estimation with soft thresholding penalties. Statist. Neerlandica 56 4 (2002) 454-479
14. 1 complexity regularization. Bernoulli 12 6 (2006) 1045-1076
15. 1 constraint. Ann. Statist. 34 5 (2006) 2367-2386
16. van de Geer S.A. High-dimensional generalized linear models and the lasso. Ann. Statist. 36 2 (2008) 614-645
17. Zhao P., and Yu B. On model selection consistency of Lasso. J. Mach. Learn. Res. 7 (2006) 2541-2563
18. Efron B., Hastie T., Johnstone I., and Tibshirani R. Least angle regression. Ann. Statist. 32 (2004) 407-499
19. Daubechies I., Defrise M., and De Mol C. An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Comm. Pure Appl. Math. 57 11 (2004) 1413-1457
20. Zou H., and Hastie T. Regularization and variable selection via the elastic net. J. R. Stat. Soc. Ser. B 67 2 (2005) 301-320
21. Owen A. A robust hybrid of lasso and ridge regression. Contemporary Mathematics vol. 443 (2007), American Mathematical Society, Providence, Rhode Island 59-72
22. Yuan M., and Lin Y. Model selection and estimation in regression with grouped variables. J. R. Stat. Soc. Ser. B Series B 68 (2006) 49-67
23. Fornasier M., and Rauhut H. Recovery algorithms for vector-valued data with joint sparsity constraints. SIAM J. Numer. Anal. 46 2 (2008) 577-613
24. Wahba G. Spline models for observational data. CBMS-NSF Regional Conference Series in Applied Mathematics vol. 59 (1990), SIAM, Philadelphia, PA
25. Barron A., Cohen A., Dahmen W., and DeVore R. Adaptive approximation and learning by greedy algorithms. Ann. Statist. 36 1 (2008) 64-94
26. Bauer F., and Pereverzev S. Regularization without preliminary knowledge of smoothness and error behavior. European J. Appl. Math. 16 (2005) 303-317
27. Micchelli C.A., and Pontil M. On learning vector-valued functions. Neural Comput. 17 1 (2005) 177-204
28. L. Baldassarre, A. Barla, B. Gianesin, M. Marinelli, Vector valued regression for iron overload estimation, in: Proceedings of ICPR 2008, Tampa, FL, USA
29. Micchelli C.A., Pontil M., and Evgeniou T. Learning multiple tasks with kernel methods. J. Mach. Learn. Res. 6 (2005) 615-637
30. Argyriou A., Evgeniou T., and Pontil M. Multi-task feature learning. In: Schölkopf B., Platt J., and Hoffman T. (Eds). Advances in Neural Information Processing Systems, vol. 19 (2007), MIT Press, Cambridge, MA 41-48
31. Caponnetto A., Micchelli C.A., Pontil M., and Ying Y. Universal multi-task kernels. J. Mach. Learn. Res. 9 (2008) 1615-1646
32. Amato U., Antoniadis A., and Pensky M. Wavelet kernel penalized estimation for non-equispaced design regression. Statist. Comput. 16 1 (2006) 37-55
33. Carmeli C., De Vito E., and Toigo A. Vector valued reproducing kernel Hilbert spaces of integrable functions and Mercer theorem. Anal. Appl. (Singap.) 4 4 (2006) 377-408
34. van der Vaart A.W., and Wellner J.A. Weak Convergence and Empirical Processes. Springer Series in Statistics (1996), Springer-Verlag, New York
35. Ekeland I., and Turnbull T. Infinite-dimensional Optimization and Convexity. Chicago Lectures in Mathematics (1983), The University of Chicago Press, Chicago
36. Pinelis I. Optimum bounds for the distributions of martingales in Banach spaces. Ann. Probab. 22 4 (1994) 1679-1706
37. Pinelis I. Correction: "Optimum bounds for the distributions of martingales in Banach spaces". Ann. Probab. 22 4 (1994) 1679-1706 MR1331198 (96b:60010)
38. Pinelis I. Correction: "Optimum bounds for the distributions of martingales in Banach spaces". Ann. Probab. 27 4 (1999) 2119
39. Yurinsky V. Sums and Gaussian Vectors. Lecture Notes in Mathematics vol. 1617 (1995), Springer-Verlag, Berlin
40. Pinelis I.F., and Sakhanenko A.I. Remarks on inequalities for probabilities of large deviations. Theory Probab. Appl. 30 1 (1985) 143-148
41. Lepskii O. On a problem of adaptive estimation in Gaussian white noise. Theory Probab. Appl. 35 (1990) 454-466
42. Schock E., and Pereverzev S.V. On the adaptive selection of the parameter in regularization of ill-posed problems. SIAM J. Numer. Anal. 43 (2005) 2060-2076
43. E. De Vito, S. Pereverzev, L. Rosasco, Adaptive learning via the balancing principle, BCL paper-275/CSAIL and Technical Report-TR-2008-062, Massachusetts Institute of Technology, 2008. Preprint available at: http://dspace.mit.edu/bitstream/handle/1721.1/42896/MIT-CSAIL-TR-2008-062.pdf
44. Smale S., and Zhou D.-X. Learning theory estimates via integral operators and their approximations. Constr. Approx. 26 2 (2007) 153-172
45. Bauer F., Pereverzev S., and Rosasco L. On regularization algorithms in learning theory. J. Complexity 23 1 (2007) 52-72
46. Caponnetto A., and De Vito E. Optimal rates for regularized least-squares algorithm. Found. Comput. Math. 7 3 (2007) 331-368