Sparse models by iteratively reweighted feature scaling: A framework for wavelength and sample selection

Journal of Chemometrics

Volumn 27, Issue 3-4, 2013, Pages 50-62

  Andries, Erik ^a,b  

^a UNIVERSITY OF NEW MEXICO   (United States)

^b Central New Mexico Community College   (United States)

Author keywords

Least absolute shrinkage and selection operator (LASSO); Multivariate calibration; Sample selection; Sparsity; Support vector regression (SVR); Tikhonov regularization (TR); Wavelength selection

Indexed keywords

LEAST SQUARES APPROXIMATIONS; NUMERICAL METHODS; PRINCIPAL COMPONENT ANALYSIS; REGRESSION ANALYSIS;

LEAST ABSOLUTE SHRINKAGE AND SELECTION OPERATOR; LEAST ABSOLUTE SHRINKAGE AND SELECTION OPERATORS; MULTIVARIATE CALIBRATION; SAMPLES SELECTION; SPARSITY; SUPPORT VECTOR REGRESSION; SUPPORT VECTOR REGRESSIONS; TIKHONOV REGULARIZATION; WAVELENGTH SELECTION;

ITERATIVE METHODS;

EID: 84876080288     PISSN: 08869383     EISSN: 1099128X     Source Type: Journal    
DOI: 10.1002/cem.2492     Document Type: Article

References (49)

1. Anderson CR, Bro R. Variable selection in regression-a tutorial. J.Chemom. 2010; 24(11-12):728-737.
2. Balabin RM, Sergey SV. Variable selection in near-infrared spectroscopy: benchmarking of feature selection methods on biodiesel data. Anal. Chim. Acta 2011; 692: 63-72.
3. Claerbout J, Muir F. Robust modeling of erratic data. Geophysics 1973; 38: 826-844.
4. 1-norm. Geophysics 1979; 44: 39-52.
5. Levy S, Fullagar P. Reconstruction of a sparse spike train from a portion of its spectrum and application to high-resolution deconvolution. Geophysics 1981; 46: 1235-1243.
6. Santosa F, Symes WW. Linear inversion of band limited reflection seismograms. SIAM J. Sci. Stat. Comp. 1986; 7: 1250-1254.
7. Tibshirani RJ. Regression shrinkage and selection via the LASSO. J. R. Stat. Soc. Ser. B 1996; 58: 267-288.
8. Tibshirani RJ, Saunders M, Rosset S, Zhu J, Knight K. Sparsity and smoothness via the fused LASSO. J. R. Stat. Soc. Ser. B 2005; 67: 91-108.
9. Yuan M, Lin Y. Model selection and estimation in regression with grouped variables. J. R. Stat. Soc. Ser. B 2005; 68(1):49-67.
10. Zou H. The adaptive lasso and its oracle properties. J. Am. Stat. Assoc. 2006; 101(476):1418-1429.
11. Hastie T, Taylor J, Tibshirani RJ, Walther G. Forward stagewise regression and the monotone LASSO. Electron. J. Stat. 2007; 1: 1-29.
12. Fu W. Penalized regressions: the bridge versus the lasso. J. Comput. Graph. Stat. 1998; 7(3): 397-416.
13. Efron B, Johnstone I, Hastie T, Tibshirani RJ. Least angle regression. Ann. Stat. 2004; 32(2): 407-499.
14. Friedman J, Hastie T, Hofling H, Tibshirani H. Pathwise coordinate optimization. Ann. Appl. Stat. 2007; 1(2): 302-332.
15. Sparselab: seeking sparse solutions to linear systems of equations, Standford University. http://sparselab.stanford.edu/ [Accessed on February 12, 2013].
16. Liu J, Ji S, Ye J. SLEP: a sparse learning package, Arizona State University. http://www.public.asu.edu/~jye02/Software/SLEP [Accessed on February 12, 2013].
17. Schmidt M, Fung G, Rosales R. Fast optimization methods for L1 regularization: a comparative study and 2 new approaches. Proceedings of 2007 European Conference on Machine Learning: Warsaw, Poland, 2007. http://www.di.ens.fr/~mschmidt [Accessed on February 12, 2013].
18. Mørup M, Clemmensen L. Multiplicative updates for the lasso. Proceedings of 2007 IEEE International Workshop on Machine Learning for Signal Processing: Thessaloniki, Greece, 2007. http://www.mortenmorup.dk/ [Accessed on February 12, 2013].
19. Tikhonov A. Solution of incorrectly formulated problems and the regularization method. English translation of Dokl. Akad. Nauk. SSSR 1963; 151: 501-504.
20. Hoerl A. Application of ridge analysis to regression problem. Chem. Eng. Prog. 1962; 58: 54-59.
21. Kalivas JH, Siano GS, Andries E, Goicoechea HC. Tikhonov regularization approaches for calibration maintenance and transfer. Appl. Spectrosc. 2009; 63(7):800-809.
22. Kunz MR, Ottaway J, Kalivas JH, Andries E. Impact of standardization sample design on Tikhonov regularization variants for spectroscopic calibration maintenance and transfer. J. Chemometr. 2010; 24: 218-229.
23. Kunz MR, Kalivas JH, Andries E. Model updating for spectral calibration maintenance and transfer using 1-norm variants of Tikhonov regularization. Anal. Chem. 2010; 82(9): 3642-3649.
24. Kalivas JH. Application of 2-norm (L2) and 1-norm (L1) regularization variants for full wavelength and sparse spectral multivariate calibration models and maintenance. J. Chemometr. 2012; 26: 218-230.
25. Gorodnitsky I, Rao B. Sparse signal reconstruction from limited data using FOCUSS: a re-weighted minimum norm algorithm. IEEE T. Signal Proces. 1997; 45(3): 600-615.
26. Rao B, Kreutz-Delgado K. An affine scaling methodology for best basis selection. IEEE T. Signal Proces. 1999; 47(1): 187-200.
27. Hastie T, Tibshirani RJ, Friedman JH. The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer-Verlag: NewYork City, NY, 2001.
28. 1 minimization. J. Fourier Anal. Appl. 2007; 14: 877-905.
29. Chartrand R, Yin W. Iteratively reweighted algorithms for compressive sensing. 2008 Proceedings of International Conference on Acoustics, Speech, Signal Processing (ICASSP), 869-3872: Las Vegas, 2008.
30. Ottaway J, Kalivas JH, Andries E. Spectral multivariate calibration with wavelength selection using variants of Tikhonov regularization. Appl. Spectrosc. 2009; 64(12): 1388-1395.
31. Hansen PC. Rank-Deficient and Discrete Ill-Posed Problems: Numerical Aspects of Linear Inversion. SIAM Press: Philadelphia, PA, 1998.
32. Kondylis A, Whittaker J. Adaptively preconditioned Krylov spaces to identify irrelevant predictors. Chemometr. Intell. Lab. Syst. 2010; 104: 205-213.
33. Guyon I, Weston J, Barnhill S, Vapnik V. Gene selection for cancer classification using support vector machines. Mach. Learn. 2002; 46: 389-422.
34. Li H, Liang Y, Xu Q, Cao D. Key wavelengths screening using competitive adaptive reweighted sampling method for multivariate calibration. Anal. Chim. Acta 2009; 648: 77-84.
35. Suykens J, Van Gestel T, De Brabanter J, De Moor B, Vandewalle J. Least Squares Support Vector Machines. World Scientific: Singapore, 2002.
36. Welling M. Kernel support vector regression. Max Welling's Classnotes in Machine Learning. http://www.ics.uci.edu/~welling/classnotes/classnotes.html [Accessed on February 12, 2013].
37. Welling M. Kernel support vector machines. Max Welling's Classnotes in Machine Learning. http://www.ics.uci.edu/~welling/classnotes/classnotes.html [Accessed on February 12, 2013].
38. Cristianini N, Shawe-Taylor J. An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods. Cambridge University Press: Oxford, UK, 1999.
39. Lin CJ, Chang CC. LIBSVM: a library for support vector machines. ACM TIST 2011; 2(3): 27:1-27:27. Software available at http://www.csie.ntu.edu.tw/~cjlin/libsvm [Accessed on February 12, 2013].
40. Fung G, Mangasarian O. Proximal support vector machine classifiers. In In Proceedings KDD-2001: Knowledge Discovery and Data Mining, August 26-29, 2001, San Francisco, CA, Provost F, Srikant R (eds). Association for Computing Machinery: New York, 2001; 77-86.
41. Peng X. TSVR: an efficient twin support vector machine for regression. Neural Networks 2009; 23: 356-372.
42. Bertsekas B. Constrained Optimization and Lagrange Multiplier Methods. Athena Scientic: Belmonta, MA, 1982.
43. Franklin JN. Minimum principles for ill-posed problems. SIAM J. Math. Anal. 1978; 9(4): 638-650.
44. SpaSM: a Matlab toolbox for performing sparse regression. J. Stat. Softw. (submitted), http://www2.imm.dtu.dk/projects/spasm/ [Accessed on February 12, 2013].
45. Haaland DM, Thomas E. Partial least-squares methods for spectral analyses. 1. Relation to other quantitative calibration methods and the extraction of qualitative information. Anal. Chem. 1988; 1988: 1193-2002.
46. Andries E, Kalivas JH. Multivariate calibration leverages and spectral F-ratios via a filter factor representation. J. Chemometr. 2010; 24(5): 249-260.
47. Varmuza K, Filzmoser P. Introduction to Multivariate Statistical Analysis in Chemometrics. CRC Press: Boca Raton, FL, 2009.
48. NIR of corn samples for standardization benchmarking. Eigenvector Research. http://www.eigenvector.com/data/Corn/index.html [Accessed on February 12, 2013].
49. Wheat functionality as measured by diffuse reflectance of whole grain Canadian wheat. 2008 International Diffuse Reflectance Conference: Chambersburg, PA, USA, 2008. http://www.idrc-chambersburg.org/ss20082012.html [Accessed on February 12, 2013].