Algorithms for subset selection in linear regression

Proceedings of the Annual ACM Symposium on Theory of Computing

Volumn , Issue , 2008, Pages 45-54

  Das, Abhimanyu ^a   Kempe, David ^a  

^a University of Southern California ^*  (United States)

Author keywords

Algorithms; Theory

Indexed keywords

ALGORITHMS; APPROXIMATION THEORY; COMPUTATION THEORY; COVARIANCE MATRIX; GENETIC ALGORITHMS; TREES (MATHEMATICS);

CONSTANT FACTOR APPROXIMATION; GOODNESS-OF-FIT STATISTICS; MULTIPLE CORRELATION; PAIRWISE CORRELATION; PERTURBATION ANALYSIS; PERTURBATION RESULTS; SPARSE APPROXIMATIONS; THEORY;

APPROXIMATION ALGORITHMS;

EID: 57049122980     PISSN: 07378017     EISSN: None     Source Type: Conference Proceeding    
DOI: 10.1145/1374376.1374384     Document Type: Conference Paper

References (39)

1. K. Anstreicher, M. Fampa, J. Lee, and J. Williams. An exact algorithm for maximum entropy sampling. Operations Research, 43(4):684-691, 1995.
2. K. Anstreicher, M. Fampa, J. Lee, and J, Williams. Maximum-entropy remote sampling. Discrete Applied Mathematics, 108(3):211-226, 2001.
3. S. Boyd and L. Vandenberghe. Convex Optimization. Cambridge University Press, 2004.
4. E. J. Candes, J. Romberg, and T. Tao. Stable signal recovery from incomplete and inaccurate measurements. Communications on Pure and Applied Mathematics, 59:1207-1223, 2005.
5. W. Cochran. Some effects of errors of measurement on multiple correlation. Journal of the American Statistical Association, 65(329):22-34, 1970.
6. J. Cohen and P. Cohen. Applied multiple regression/correlation analysis for the behavioral sciences. Lawrence Erlbaum Assoc Publishers, 2003.
7. G. Cornuejols, M. Fisher, and G. Nemhauser. Location of bank accounts to optimize float. Management Science, 23:789-810, 1977.
8. C. Couvreur and Y. Bressler. On the optimality of the backward greedy algorithm for the subset selection problem. SIAM Journal on Matrix Analysis and Applications, 21(3):797-808, 2000.
9. G. Davis, S. Mallat, and M. Avellaneda. Greedy adaptive approximation. Journal of Constructive Approximation, 13:57-98, 1997.
10. A. Deshpande, C. Guestrin, S. Madden, J. Hellerstein, and W. Hong. Model driven data acquisition in sensor networks. In Proc. International Conference on Very Large Data Bases, VLDB, 2004.
11. G. Diekhoff. Statistics for the Social and Behavioral Sciences. Wm. C. Brown Publishers, 2002.
12. D. Donoho. For most large underdetermined systems of linear equations, the minimal 11-norm near-solution approximates the sparsest near-solution. Communications on Pure and Applied Mathematics, 59:1207-1223, 2005.
13. D. L. Donoho, M. Elad, and V. N. Temlyakov. Stable recovery of sparse overcomplete representations in the presence of noise. IEEE Transaction on Information Theory, 52:6-f8, 2006.
14. V. F. Flack and P. C. Chang. Frequency of selecting noise variables in subset regression analysis: A simulation study. The American Statistician Journal, 41(1):84-86, 1987.
15. A. Gilbert, S. Muthukrishnan, and M. Strauss. Approximation of functions over redundant dictionaries using coherence. In Proc. ACM-SIAM Symposium on Discrete Algorithms, 2003.
16. C. Guestrin, A. Krause, and A. Singh. Near-optimal sensor placements in gaussian processes. In International Conference on Machine Learning, ICML, 2005.
17. R. A. Horn and C. R. Johnson. Matrix. Analysis. Cambridge University Press, 1999.
18. F. Hwang, S. Onn, and U. Rothblum. A polynomial time algorithm for shaped partition problems. SIAM Journal on Optimization, 10(1):70-81, 1999.
19. R, A. Johnson and D. W. Wichern. Applied Multivariate Statistical Analysis. Prentice Hall, 2002.
20. P. Liaskovitis and C. Schurgers. Leveraging redundancy in sampling-interpolation applications for sensor networks. In Proc. 3rd Intl. Conf. on Distributed Computing in Sensor Systems, 2007.
21. A. Miller. Subset Selection in Regression. Chapman and Hall, second edition, 2002.
22. S. Muthukrishnan. Data streams: Algorithms and applications. Foundations and Trends in Theoretical Computer Science, 1. 2005.
23. B. Natarajan. Sparse approximation solutions to linear systems. SIAM Journal on Computing, 24:227-234, 1995.
24. G. Nemhauser, L. Wolsey, and M. Fisher. An analysis of the approximations for maximizing submodular set functions. Mathematical Programming, f4:265-294, 1978.
25. S. Onn and L. Schulman. The vector partition problem for convex optimization functions. Mathematics of Operations Research, 26(3):583-590, 2001.
26. M. H. Pesaran and R. J. Smith. A generalized r2 criterion for regression models estimated by the instrumental variables method. Econometrica, 62(3):705-710, 1994.
27. J. Saxe. Dynamic programming algorithms for recognizing small bandwidth graphs in polynomial time. SIAM Journal on Algebraic Methods I, l(4):363-369, 1980.
28. G. W. Stewart and J.G. Sun. Matrix Perturbation Theory. Academic Press, 1990.
29. V. Temlyakov. Greedy algorithms and m-term approximation with regard to redundant dictionaries. Journal of Approximation Theory, 98:117-145, 1999.
30. A. Temlyakov. Nonlinear methods of approximation. Foundations of Computational Mathematics, 3:33-107, 2002.
31. R. Tibshirani. Regression shrinkage and selection via the lasso. Journal of Royal Statistical Society, 58:267-288, 1996.
32. J. Tropp. Greed is good: algorithmic results for sparse approximation. IEEE Trans. Information Theory, 50:2231-2242, 2004.
33. J. Tropp. Topics in Sparse Approximation. PhD thesis, University of Texas, Austin, 2004.
34. J. Tropp. Just relax: Convex programming methods for identifying sparse signals. IEEE Trans. Information Theory, 51:1030-1051, 2006.
35. J. Tropp, A. Gilbert, S. Muthukrishnan, and M. Strauss. Improved sparse approximation over quasi-incoherent dictionaries. In Proc. IEEE-ICIP, 2003.
36. W. F. Velicer. Suppressor variables and the semipartial correlation coefficient. Educational and Psychological Measurement, 38:953-958, 1978.
37. M. Wainwright. Sharp thresholds for noisy and high-dimensional recovery of sparsity using 11-constrained quadratic programming. In Proc. Allerton Conference on Communication, 2006.
38. D. A. Walker. Suppressor variable(s) importance within a regression model. Journal of College Student Development, 44:127-133, 2003.
39. H. Zou and T. Hastie. Regularization and variable selection via the elastic net. Journal of the R.oyal Statistical Society, Series B, 67(2):301-320, 2005.