Sparse regression learning by aggregation and langevin Monte-Carlo

COLT 2009 - The 22nd Conference on Learning Theory

Volumn , Issue , 2009, Pages

  Dalalyan, A S ^a   Tsybakov, A B ^b  

^a UNIVERSITÉ PARIS EST   (France)

^b SORBONNE UNIVERSITÉ   (France)

Author keywords

[No Author keywords available]

Indexed keywords

DETERMINISTIC DESIGN; FINITE-DIMENSIONAL LINEAR SPACES; LANGEVIN MONTE-CARLO; LINEAR COMBINATIONS; NUMERICAL EXPERIMENTS; ORACLE INEQUALITIES; REGRESSION FUNCTION; TEMPERATURE PARAMETERS;

ALGORITHMS; REGRESSION ANALYSIS;

AGGREGATES;

EID: 84898067794     PISSN: None     EISSN: None     Source Type: Conference Proceeding    
DOI: None     Document Type: Conference Paper

References (37)

1. F. Abramovich, V. Grinshtein, and M. Pensky. On optimality of Bayesian testimation in the normal means problem. Ann. Statist., 35: 2261-2286, 2007.
2. P. Alquier. Pac-Bayesian bounds for randomized empirical risk minimizers. Math. Methods Statist., 17(4): 1-26, 2008.
3. J.-Y. Audibert. Fast learning rates in statistical inference through aggregation, 2008. Ann. Statist., To appear.
4. P. Bickel, Y. Ritov, and A. B. Tsybakov. Simultaneous analysis of Lasso and Dantzig selector, 2008. To appear in The Annals of Statistics.
5. 1 penalized least squares. In Learning theory, Volume 4005 of Lecture Notes in Comput. Sci., pages 379-391. Springer, Berlin, 2006.
6. F. Bunea, A. B. Tsybakov, and M. Wegkamp. Aggregation for Gaussian regression. Ann. Statist., 35(4): 1674-1697, 2007.
7. F. Bunea, A.B. Tsybakov, and M.H. Wegkamp. Sparsity oracle inequalities for the Lasso. Electronic J. of Statist., 1: 169-194, 2007.
8. O. Catoni. Statistical learning theory and stochastic optimization. Lecture Notes in Mathematics. Springer-Verlag, Berlin, 2004.
9. O. Catoni. Pac-Bayesian Supervised Classification: The Thermodynamics of Statistical Learning, Volume 56. IMS Lecture Notes Monograph Series, December 03 2007.
10. N. Cesa-Bianchi, A. Conconi, and C. Gentile. On the generalization ability of on-line learning algorithms. IEEE Trans. Inform. Theory, 50(9): 2050-2057, 2004.
11. N. Cesa-Bianchi and G. Lugosi. Prediction, learning, and games. Cambridge University Press, Cambridge, 2006.
12. E. Candès and T. Tao. Near-optimal signal recovery from random projections: universal encoding strategies? IEEE Trans. Inform. Theory, 52(12): 5406-5425, 2006.
13. E. Candès and T. Tao. The Dantzig selector: statistical estimation when p is much larger than n. Ann. Statist., 35(6): 2313-2351, 2007.
14. D. Donoho, M. Elad, and V. Temlyakov. Stable recovery of sparse overcomplete representations in the presence of noise. IEEE Trans. Inform. Theory, 52(1): 6-18, 2006.
15. A. S. Dalalyan and A.B. Tsybakov. Aggregation by exponential weighting and sharp oracle inequalities. In Learning theory, Volume 4539 of Lecture Notes in Comput. Sci., pages 97-111. Springer, Berlin, 2007.
16. A. S. Dalalyan and A.B. Tsybakov. Aggregation by exponential weighting, sharp PAC-Bayesian bounds and sparsity. Mach. Learn., 72(1-2): 39-61, 2008.
17. B. Efron, T. Hastie, I. Johnstone, and R. Tibshirani. Least angle regression. Ann. Statist., 32(2): 407-499, 2004.
18. D. Haussler, J. Kivinen, and M. Warmuth. Sequential prediction of individual sequences under general loss functions. IEEE Trans. Inform. Theory, 44(5): 1906-1925, 1998.
19. A. Juditsky, Ph. Rigollet, and A.B. Tsybakov. Learning by mirror averaging. Ann. Statist., 36: 2183-2206, 2008.
20. I. Johnstone and B.W. Silverman. Empirical Bayes selection of wavelet thresholds. Ann. Statist, 33: 1700-1752, 2005.
21. J. Kent. Time-reversible diffusions. Adv. in Appl. Probab., 10(4): 819-835, 1978.
22. V. Koltchinskii. Sparse recovery in convex hulls via entropy penalization, 2008. Ann. Statist., To appear.
23. G. Leung and A. Barron. Information theory and mixing least-squares regressions. IEEE Trans. Inform. Theory, 52(8): 3396-3410, 2006.
24. N. Meinshausen and P. Bühlmann. High-dimensional graphs and variable selection with the Lasso. Ann. Statist., 34(3): 1436-1462, 2006.
25. D. McAllester. Pac-Bayesian stochastic model selection. Machine Learning, 51(1): 5-21, 2003.
26. V. Rivoirard. Non linear estimation over weak Besov spaces and minimax Bayes method. Bernoulli, 12(4): 609-632, 2006.
27. G. Roberts and O. Stramer. Langevin diffusions and Metropolis-Hastings algorithms. Methodol. Comput. Appl. Probab., 4(4): 337-357, 2002.
28. L. Rogers and D. Williams. Diffusions, Markov processes, and martingales. Vol. 2. Probability and Mathematical Statistics. John Wiley & Sons Inc., New York, 1987.
29. M. W. Seeger. Bayesian inference and optimal design for the sparse linear model. J. Mach. Learn. Res., 9: 759-813, 2008.
30. S. van de Geer. High-dimensional generalized linear models and the Lasso. Ann. Statist., 36(2): 614-645, 2008.
31. V. Vovk. Aggregating strategies. In COLT: Proceedings of the Workshop on Computational Learning Theory, Morgan Kaufmann Publishers, pages 371-386, 1990.
32. David P. Wipf and Bhaskar D. Rao. An empirical Bayesian strategy for solving the simultaneous sparse approximation problem. IEEE Trans. Signal Process., 55(7, part 2): 3704-3716, 2007.
33. Y. Yang. Aggregating regression procedures to improve performance. Bernoulli, 10(1): 25-47, 2004.
34. B. Yu. Embracing statistical challenges in the information technology age. Technometrics, 49(3): 237-248, 2007.
35. H. Zou and T. Hastie. Regularization and variable selection via the elastic net. J. R. Stat. Soc. Ser. B, 67(2): 301-320, 2005.
36. C.H. Zhang and J. Huang. The sparsity and biais of the Lasso selection in high-dimensionallinear regression. Ann. Statist., 36: 1567-1594, 2008.
37. P. Zhao and B. Yu. On model selection consistency of Lasso. J. Mach. Learn. Res., 7: 2541-2563, 2006.