Mixed integer second-order cone programming formulations for variable selection in linear regression

European Journal of Operational Research

Volumn 247, Issue 3, 2015, Pages 721-731

  Miyashiro, Ryuhei ^a   Takano, Yuichi ^b  

^a TOKYO UNIVERSITY OF AGRICULTURE AND TECHNOLOGY   (Japan)

^b SENSHU UNIVERSITY   (Japan)

Author keywords

Information criterion; Integer programming; Multiple linear regression; Second order cone programming; Variable selection

Indexed keywords

LINEAR REGRESSION; SET THEORY;

COMPUTATIONAL EXPERIMENT; GOODNESS-OF-FIT MEASURE; INFORMATION CRITERION; MULTIPLE LINEAR REGRESSION MODELS; MULTIPLE LINEAR REGRESSIONS; SECOND-ORDER CONE PROGRAMMING; STEPWISE REGRESSION METHOD; VARIABLE SELECTION;

INTEGER PROGRAMMING;

EID: 84940603932     PISSN: 03772217     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.ejor.2015.06.081     Document Type: Article

References (45)

1. Akaike H. A new look at the statistical model identification IEEE Transactions on Automatic Control 19 1974 716 723
2. Arthanari T.S., and Dodge Y. Mathematical programming in statistics 1981 John Wiley & Sons New York, NY
3. Bache K., and Lichman M. UCI machine learning repository 2013 University of California, School of Information and Computer Science Irvine, CA
4. Beale E.M.L. Two transportation problems Proceedings of the third international conference on operational research 1963 780 788
5. Beale E.M.L., and Tomlin J.A. Special facilities in a general mathematical programming system for non-convex problems using ordered sets of variables Proceedings of the fifth international conference on operational research 1970 447 454
6. Benson H.Y., and Saʇram Ü. Mixed-integer second-order cone programming: A survey Topaloglu H. Tutorials in operations research 2013 INFORMS Catonsville, MD 13 36
7. Bertsimas D., King A., and Mazumder R. Best subset selection via a modern optimization lens 2014 INFORMS Catonsville, MD
8. Bertsimas D., and Shioda R. Algorithm for cardinality-constrained quadratic optimization Computational Optimization and Applications 43 2009 1 22
9. Blum A.L., and Langley P. Selection of relevant features and examples in machine learning Artificial Intelligence 97 1997 245 271
10. Boyd S., and Vandenberghe L. Convex optimization 2004 Cambridge University Press Cambridge, UK
11. Burnham K.P., and Anderson D.R. Model selection and multimodel inference: a practical information theoretic approach 2nd ed. 2002 Springer New York, NY
12. Das A., and Kempe D. Algorithms for subset selection in linear regression Proceedings of the fortieth annual ACM symposium on theory of computing 2008 45 54
13. Das A., and Kempe D. Submodular meets spectral: Greedy algorithms for subset selection, sparse approximation and dictionary selection Proceedings of the twenty eighth international conference on machine learning 2011 1057 1064
14. Debnath R., Muramatsu M., and Takahashi H. An efficient support vector machine learning method with second-order cone programming for large-scale problems Applied Intelligence 23 2005 219 239
15. Efroymson M.A. Multiple regression analysis Ralston A. Wilf H.S. Mathematical methods for digital computers 1960 John Wiley & Sons New York, NY 191 203
16. Emet S. A model identification approach using MINLP techniques Proceedings of the nineth WSEAS international conference on applied mathematics 2006 347 350
17. Furnival G.M., and Wilson R.W. Jr. Regressions by leaps and bounds Technometrics 16 1974 499 511
18. Gatu C., and Kontoghiorghes E.J. Branch-and-bound algorithms for computing the best-subset regression models Journal of Computational and Graphical Statistics 15 2006 139 156
19. Guyon I., and Elisseeff A. An introduction to variable and feature selection Journal of Machine Learning Research 3 2003 1157 1182
20. Hannan E.J., and Quinn B.G. The determination of the order of an autoregression Journal of the Royal Statistical Society: Series B 41 1979 190 195
21. Hocking R.R. The analysis and selection of variables in linear regression Biometrics 32 1976 1 49
22. Hoerl A.E., and Kennard R.W. Ridge regression: Biased estimation for non-orthogonal problems Technometrics 20 1970 55 67
23. Hofmann M., Gatu G., and Kontoghiorghes E.J. Efficient algorithms for computing the best subset regression models for large-scale problems Computational Statistics & Data Analysis 52 2007 16 29
24. IBM ILOG (2012). IBM ILOG CPLEX 12.5.
25. Kohavi R., and John G.H. Wrappers for feature subset selection Artificial Intelligence 97 1997 273 324
26. Konno H., and Takaya Y. Multi-step methods for choosing the best set of variables in regression analysis Computational Optimization and Applications 46 2010 417 426
27. Konno H., and Yamamoto R. Choosing the best set of variables in regression analysis using integer programming Journal of Global Optimization 44 2009 272 282
28. Leng C. The residual information criterion, corrected 2013 Cornell University Library Ithaca, NY
29. Liu H., and Motoda H. Computational methods of feature selection 2007 Chapman and Hall/CRC Boca Raton, FL
30. Lobo M.S., Vandenberghe L., Boyd S., and Lebret H. Applications of second-order cone programming Linear Algebra and its Applications 284 1998 193 228
31. MathWorks (2012). MATLAB R2012b.
32. Meiri R., and Zahavi J. Using simulated annealing to optimize the feature selection problem in marketing applications European Journal of Operational Research 171 2006 842 858
33. Miller A. Subset selection in regression 2nd ed. 2002 Chapman and Hall/CRC Boca Raton, FL
34. p: a mixed integer programming approach Expert Systems with Applications 42 2015 325 331
35. Sato T., Takano Y., Miyashiro R., and Yoshise A. Feature subset selection for logistic regression via mixed integer optimization Discussion Paper Series, No. 1324 2015 Department of Policy and Planning Sciences, University of Tsukuba
36. Schwarz G. Estimating the dimension of a model Annals of Statistics 6 1978 461 464
37. Shi P., and Tsai C.-L. Regression model selection: A residual likelihood approach Journal of the Royal Statistical Society: Series B 64 2002 237 252
38. Shivaswamy P.K., Bhattacharyya C., and Smola A.J. Second order cone programming approaches for handling missing and uncertain data The Journal of Machine Learning Research 7 2006 1283 1314
39. Skrifvars H., Leyffer S., and Westerlund T. Comparison of certain MINLP algorithms when applied to a model structure determination and parameter estimation problem Computers & Chemical Engineering 22 1998 1829 1835
40. Sugiura N. Further analysts of the data by Akaike's information criterion and the finite corrections Communications in Statistics - Theory and Methods 7 1978 13 26
41. Tibshirani R. Regression shrinkage and selection via the lasso Journal of the Royal Statistical Society: Series B 58 1996 267 288
42. Trafalis T.B., and Gilbert R.C. Robust classification and regression using support vector machines European Journal of Operational Research 173 2006 893 909
43. Wherry R.J. A new formula for predicting the shrinkage of the coefficient of multiple correlation The Annals of Mathematical Statistics 2 1931 440 457
44. Whittingham M.J., Stephens P.A., Bradbury R.B., and Freckleton R.P. Why do we still use stepwise modelling in ecology and behaviour? Journal of Animal Ecology 75 2006 1182 1189
45. Yuan M., Ekici A., Lu Z., and Monteiro R. Dimension reduction and coefficient estimation in multivariate linear regression Journal of the Royal Statistical Society: Series B 69 2007 329 346