A comparison of simulated annealing algorithms for variable selection in principal component analysis and discriminant analysis

Computational Statistics and Data Analysis

Volumn 77, Issue , 2014, Pages 38-53

  Brusco, Michael J ^a  

^a FLORIDA STATE UNIVERSITY   (United States)

Author keywords

Discriminant analysis; Principal component analysis; Simulated annealing; Variable selection

Indexed keywords

ALGORITHMS; DISCRIMINANT ANALYSIS; PRINCIPAL COMPONENT ANALYSIS;

ALGORITHM DESIGN; COOLING SCHEME; DESIGN FEATURES; MAXIMUM AND MINIMUM TEMPERATURES; META HEURISTICS; MULTIVARIATE STATISTICS; SIMULATED ANNEALING ALGORITHMS; VARIABLE SELECTION;

SIMULATED ANNEALING;

EID: 84901935441     PISSN: 01679473     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.csda.2014.03.001     Document Type: Article

References (78)

1. E. Aarts, J. Korst, and W. Michiels Simulated annealing E.K. Burke, G. Kendall, Search Methodologies 2005 Springer New York 187 210
2. R.M. Balabin, and S.V. Smirnov Variable selection in near-infrared spectroscopy: benchmarking of feature selection methods on biodiesel data Anal. Chim. Acta 692 2011 63 72
3. E.M.L. Beale, M.G. Kendall, and D.W. Mann The discarding of variables in multivariate analysis Biometrika 54 1967 357 366
4. M.J. Brusco On the performance of simulated annealing for large-scale L2 unidimensional scaling J. Classification 23 2006 255 268 (Pubitemid 44704843)
5. M.J. Brusco, and L.W. Jacobs A simulated annealing approach to the cyclic staff scheduling problem Nav. Res. Logist. 40 1993 69 84
6. M.J. Brusco, L.W. Jacobs, R.J. Bongiorno, D.V. Lyons, and B. Tang Improving personnel scheduling at airline stations Oper. Res. 43 1995 741 751
7. M.J. Brusco, L.W. Jacobs, and G.M. Thompson A morphing procedure to supplement a simulated annealing heuristic for cost-and coverage-correlated set-covering problems Ann. Oper. Res. 86 1999 611 627
8. M.J. Brusco, and H.-F. Köhn Exemplar-based clustering via simulated annealing Psychometrika 74 2009 457 475
9. M.J. Brusco, R. Singh, and D. Steinley Variable neighborhood search heuristics for selecting a subset of variables in principal component analysis Psychometrika 74 2009 705 726
10. M.J. Brusco, and D. Steinley Exact and approximate algorithms for variable selection in linear discriminant analysis Comput. Statist. Data Anal. 55 2011 123 131
11. M.J. Brusco, D. Steinley, and J.D. Cradit An exact algorithm for hierarchically well-formulated subsets in second-order polynomial regression Technometrics 51 2009 306 315
12. Cadima, J., Cerdeira, J.O., Duarte Silva, P., Minhoto, M., 2012, The subselect R package, Retrieved from: http://cran.r-project.org/web/packages/ subselect/subselect.pdf.
13. J. Cadima, J.O. Cerdeira, and M. Minhoto Computational aspects of algorithms for variable selection in the context of principal components Comput. Statist. Data Anal. 47 2004 225 236
14. J. Cadima, and I.T. Jolliffe Loadings and correlations in the interpretation of principal components J. Appl. Stat. 22 1995 203 214
15. J. Cadima, and I.T. Jolliffe Variable selection and the interpretation of principal subspaces J. Agric. Biol. Environ. Stat. 6 2001 62 79
16. V. Cerny A thermodynamical approach to the traveling salesman problem J. Optim. Theory Appl. 45 1985 41 51
17. E. Ceulemans, and I. Van Mechelen CLASSI: A classification model for the study of sequential processes and individual differences therein Psychometrika 73 2008 107 124
18. E. Ceulemans, I. Van Mechelen, and I. Leenen The local minima problem in hierarchical classes analysis: an evaluation of a simulated annealing algorithm and various multistart procedures Psychometrika 72 2007 377 391
19. F. Chiyoshi, and D. Galvão A statistical analysis of simulated annealing applied to the p-median problem Ann. Oper. Res. 96 2000 61 74
20. S. Dray On the number of principal components: a test of dimensionality based on measurements of similarity between matrices Comput. Statist. Data Anal. 52 2008 2228 2237 (Pubitemid 350064392)
21. Z. Drezner, and G.A. Marcoulides Using simulated annealing for selection in multiple regression analysis Mult. Linear Regression Viewp. 25 1999 1 4
22. A.P. Duarte Silva Efficient variable screening for multivariate analysis J. Multivariate Anal. 76 2001 35 62 (Pubitemid 33623888)
23. A.P. Duarte Silva Discarding variables in principal component analysis: algorithms for all-subsets comparisons Comput. Statist. 17 2002 251 271
24. M.A. Efroymson Multiple regression analysis A. Ralston, H.S. Wilf, Mathematical Methods for Digital Computers 1960 Wiley New York 191 203
25. R.A. Fisher The use of multiple measurements in taxonomic problems Ann. Eugenics 7 1936 179 188
26. R.A. Fisher The statistical utilization of multiple measurements Ann. Eugenics 8 1938 376 386
27. D. Fouskakis, and D. Draper Comparing stochastic optimization methods for variable selection in binary outcome prediction, with application to health policy J. Amer. Statist. Assoc. 103 2008 1367 1381
28. G.M. Furnival, and R.W. Wilson Regression by leaps and bounds Technometrics 16 1974 499 512
29. S. Ganeshanandam, and W.J. Krzanowski On selecting variables and assessing their performance in linear discriminant analysis Aust. J. Stat. 31 1989 433 447
30. C. Gatu, and E.J. Kontoghiorghes Branch-and-bound algorithms for computing the best-subset regression models J. Comput. Graph. Statist. 15 2006 139 156 (Pubitemid 43514415)
31. C. Gatu, P. Yanev, and E.J. Kontoghiorghes A graph approach to generate all possible subset regression submodels Comput. Statist. Data Anal. 52 2007 799 815 (Pubitemid 47552606)
32. F. Glover, and M. Laguna Tabu search C. Reeves, Modern Heuristic Techniques for Combinatorial Problems 1993 Blackwell Oxford 70 141
33. D.E. Goldberg Genetic Algorithms in Search, Optimization, and Machine Learning 1989 Addison-Wesley New York
34. J.D.F. Habbema, and J. Hermans Selection of variables in discriminant analysis by - statistic and error rate Technometrics 19 1977 487 493
35. M. Hofmann, C. Gatu, and E.J. Kontoghiorghes Efficient algorithms for computing the best subset regression models for large-scale problems Comput. Statist. Data Anal. 52 2007 16 29 (Pubitemid 47331733)
36. K.Y. Hogarty, J.D. Kromrey, J.M. Ferron, and C.V. Hines Selection of variables in exploratory factor analysis: an empirical comparison of a stepwise and traditional approach Psychometrika 69 2004 593 611 (Pubitemid 40366645)
37. I.F. Horton, J.S. Russell, and A.W. Moore Multivariate-covariance and canonical analysis: a method for selecting the most effective discriminators in a multivariate situation Biometrics 24 1968 845 858
38. C.J. Huberty Issues in the use and interpretation of discriminant analysis Psychol. Bull. 95 1984 156 171
39. I.T. Jolliffe Principal Component Analysis 2nd edition 2002 Springer New York
40. I.T. Jolliffe Discarding variables in a principal component analysis, I: artificial data Appl. Stat. 21 1972 160 173
41. I.T. Jolliffe Discarding variables in a principal component analysis, II: real data Appl. Stat. 22 1973 21 31
42. Y. Kano, and A. Harada Stepwise variable selection in factor analysis Psychometrika 65 2000 7 22
43. S. Kirkpatrick, C.D. Gelatt, and M.P. Vecchi Optimization by simulated annealing Science 220 1983 671 680
44. W.J. Krzanowski Selection of variables to preserve multivariate data structure using principal components Appl. Stat. 36 1987 22 33
45. W.J. Krzanowski, and D.J. Hand A simple method for screening variables before clustering of microarray data Comput. Statist. Data Anal. 53 2009 2747 2753
46. J.-S. Lee, C.H. Park, and T. Ebrahimi Theory and applications of hybrid simulated annealing I. Zelinka, V. Snasel, A. Abraham, Handbook of Optimization: From Classical to Modern Approach 2012 Springer-Verlag Berlin 395 422
47. MathWorks, Inc Using MATLAB (Version 7) 2005 The MathWorks, Inc Natick, MA
48. G.P. McCabe Principal variables Technometrics 26 1984 137 144
49. G.P. McCabe Computations for variable selection in discriminant analysis Technometrics 17 1975 103 109
50. C.E. McHenry An efficient computational procedure for selecting a best subset Appl. Stat. 27 1978 291 296
51. R.J. McKay, and N.A. Campbell Variable selection techniques in discriminant analysis: I. Description British J. Math. Statist. Psych. 35 1982 1 29
52. R.J. McKay, and N.A. Campbell Variable selection techniques in discriminant analysis: II. Allocation British J. Math. Statist. Psych. 35 1982 30 41
53. G.J. McLachlan A criterion for selecting variables for the linear discriminant function Biometrics 32 1976 529 534
54. N. Metropolis, A. Rosenbluth, M. Rosenbluth, A. Teller, and E. Teller Equation of state calculations by fast computing machines J. Chem. Phys. 21 1953 1087 1092
55. N. Mladenović, and P. Hansen Variable neighborhood search Comput. Oper. Res. 24 1997 1097 1100
56. Y. Mori, M. Iizuka, T. Tarumi, and Y. Tanaka Variable selection in principal component analysis W. Härdle, Y. Mori, P. Vieu, Statistical Methods for Biostatistics and Related Fields 2007 Springer-Verlag Berlin 265 284
57. L2 unidimensional scaling J. Classification 22 2005 119 138 (Pubitemid 41060087)
58. G.D. Murray A cautionary note on selection of variables in discriminant analysis Appl. Stat. 26 1977 246 250
59. Y. Nourani, and B. Andresen A comparison of simulated annealing cooling strategies J. Phys. A: Math Gen. 31 1998 8373 8385 (Pubitemid 128381714)
60. H.H. Örkcü Subset selection in multiple linear regression models: a hybrid of genetic and simulated annealing algorithms Appl. Math. Comput. 219 2013 11018 11028
61. J. Pacheco, S. Casado, and L. Nunez A variable selection method based on tabu search for logistic regression models European J. Oper. Res. 199 2009 506 511
62. J. Pacheco, S. Casado, L. Nunez, and O. Gomez Analysis of new variable selection methods for discriminant analysis Comput. Statist. Data Anal. 51 2006 1463 1478 (Pubitemid 44709215)
63. J. Pacheco, S. Casado, and S. Porras Exact methods for variable selection in principal component analysis: guide functions and pre-selection Comput. Statist. Data Anal. 57 2013 95 111
64. P. Pendharkar, and S. Nanda A misclassification cost-minimizing evolutionary-neural classification approach Nav. Res. Logist. 53 2006 432 447 (Pubitemid 44117539)
65. J.O. Ramsay, J.M.F. ten Berge, and G.P.H. Styan Matrix correlation Psychometrika 49 1984 403 423
66. P. Robert, and Y. Escoufier A unifying tool for linear multivariate statistical methods Appl. Stat. 25 1976 257 265
67. J. Roy Step-down procedures in multivariate statistics Ann. Math. Stat. 29 1958 177 187
68. S.L. Seaman, and D.M. Young A non-parametric variable selection algorithm for allocatory discriminant analysis Educ. Psychol. Meas. 50 1990 837 841
69. S.N. Snappin, and J.D. Knoke Estimation of error rates in discriminant analysis with selection of variables Biometrics 45 1989 289 299 (Pubitemid 19109174)
70. H. Swierenga, P.J. de Groot, A.P. de Weijer, M.W.J. Derksen, and L.M.C. Buydens Improvement of PLS model transferability by robust wavelength Chemometr. Intell. Lab. Syst. 41 1998 237 248 (Pubitemid 28348349)
71. Y. Tanaka, and Y. Mori Principal component analysis based on a subset of variables: variable selection and sensitivity analysis Amer. J. Math. Management Sci. 17 1997 61 89 (Pubitemid 127737250)
72. N.T. Trendafilov, and I.T. Jolliffe DALASS: Variable selection in discriminant analysis via the lasso Comput. Statist. Data Anal. 51 2007 3718 3736 (Pubitemid 46497496)
73. V.Y. Urbakh Linear discriminant analysis: loss of discriminating power when a variable is omitted Biometrics 27 1971 531 534
74. E. Vande Gaer, E. Ceulemans, I. Van Mechelen, and P. Kuppens The CLASSI-N method for the study of sequential processes Psychometrika 77 2012 85 105
75. J. van Rosmalen, P.J.F. Groenen, J. Trejos, and W. Castillo Optimization strategies for two-mode partitioning J. Classification 26 2009 155 181
76. J.-F. Vera, W.J. Heiser, and A. Murillo Global optimization in any Minkowski Metric: a permutation-translation simulated annealing algorithm for multidimensional scaling J. Classification 24 2007 277 301
77. J.M. Weiner, and O.J. Dunn Elimination of variates in linear discrimination problems Biometrics 22 1966 268 275
78. M. Wood, I.T. Jolliffe, and G.W. Horgan Variable selection for discriminant analysis of fish sounds using matrix correlations J. Agric. Biol. Environ. Stat. 10 2005 1 16