Overview of two-norm (L2) and one-norm (L1) Tikhonov regularization variants for full wavelength or sparse spectral multivariate calibration models or maintenance

Journal of Chemometrics

Volumn 26, Issue 6, 2012, Pages 218-230

  Kalivas, John H ^a  

^a IDAHO STATE UNIVERSITY   (United States)

Author keywords

Calibration maintenance; Calibration standardization; Calibration transfer; Multivariate calibration; Sparse modeling; Tikhonov regularization

Indexed keywords

LEAST SQUARES APPROXIMATIONS; PRINCIPAL COMPONENT ANALYSIS; REGRESSION ANALYSIS; STANDARDIZATION;

CALIBRATION MAINTENANCE; CALIBRATION MODEL; CALIBRATION STANDARDIZATION; CALIBRATION TRANSFER; CONDITION; MULTIVARIATE CALIBRATION; ONE NORMS; PARTIAL LEAST-SQUARES; SPARSE MODELS; TIKHONOV REGULARIZATION;

MAINTENANCE;

EID: 84862766223     PISSN: 08869383     EISSN: 1099128X     Source Type: Journal    
DOI: 10.1002/cem.2429     Document Type: Review

References (92)

1. Næs T, Isaksson T, Fern T, Davies T. A User Friendly Guide to Multivariate Calibration and Classification. NIR Publications: Chichester, 2002.
2. Hastie TJ, Tibshirani RJ, Friedman J. The Elements of Statistical Learning: Data Mining, Inference, and Prediction (2nd edn). Springer-Verlag: New York, 2009.
3. Kalivas JH. Calibration methodologies. In Comprehensive Chemometrics: Chemical and Biochemical Data Analysis Vol. 3, Brown SD, Tauler R, Walczak B (eds.-in-chief). Elsevier: Amsterdam, 2009; 1-32.
4. Galvão RKH, Araújo MCU. Variable selection. In Comprehensive Chemometrics: Chemical and Biochemical Data Analysis Vol. 3, Brown SD, Tauler R, Walczak B (eds.-in-chief). Elsevier: Amsterdam, 2009; 233-283.
5. Balabin RM, Smirnov SV. Variable selection in near-infrared spectroscopy: Benchmarking of feature selection methods on biodiesel data. Anal. Chim. Acta 2011; 692: 63-72.
6. Xiaobo Z, Jiewen Z, Povey MJW, Holmes M, Hanpin M. Variables selection methods in near-infrared spectroscopy. Anal. Chim. Acta 2010; 667: 14-32.
7. Mark H. A Monte Carlo study of the effect of noise on wavelength selection during computerized wavelength searches. Appl. Spectrosc. 1988; 42: 1427-1440.
8. Mark H. Principles and Practice of Spectroscopic Calibration. Wiley: New York, 1991.
9. Anderson KJ, Kalivas JH. Assessment of Pareto calibration, stability, and wavelength selection. Appl. Spectrosc. 2003; 57: 309-316.
10. Lorber A, Kowalski BR. Estimation of prediction error for multivariate calibration. J. Chemom. 1988; 2: 93-109.
11. Frans SD, Harris JM. Selection of analytical wavelengths for multicomponent spectrophotometric determinations. Anal. Chem. 1985; 57: 2680-2684.
12. Jouan-Rimbaud D, Khots MS, Massart DL, Last IR, Prebble KA. Calibration line adjustment to facilitate the use of synthetic calibration samples in near-infrared spectrometric analysis of pharmaceutical production samples. Anal. Chim. Acta 1995; 315: 257-266.
13. Brenchley JM, Hörchner U, Kalivas JH. Wavelength selection characterization for NIR spectra. Appl. Spectrosc. 1997; 51: 689-699.
14. Gosselin R, Rodrigue D, Duchesne C. A bootstrap-VIP approach for selecting wavelength intervals in spectral imaging applications. Chemom. Intell. Lab. Syst. 2009; 100: 12-21.
15. Sharuatu-Rad M, Hasani M. Selection of individual variables versus intervals of variables in PLSR. J. Chemom. 2009; 24: 45-56.
16. Cogdill RP, Anderson CA, Drennen III JK. Process analytical technology case study, Part III: Calibration monitoring and transfer. AAPS PharmSciTech 2005; 6: E284-E297.
17. Brown SD. Transfer of multivariate calibration models. In Comprehensive Chemometrics: Chemical and Biochemical Data Analysis Vol. 3, Brown SD, Tauler R, Walczak B (eds.). Elsevier: Amsterdam, 2009; 345-377.
18. Tikhonov AN. Solution of incorrectly formulated problems and the regularization method. Soviet Math. Dokl. 1963; 4: 1035-1038.
19. Hansen PC. Rank-Deficient and Discrete Ill-Posed Problems: Numerical Aspects of Linear Inversion. SIAM: Philadelphia, PA, 1998.
20. Aster RC, Borchers B, Thurber CH. Parameter Estimation and Inverse Problems. Elsevier: Amsterdam, 2005.
21. Gemperline PJ, Cash E. Advantages of soft versus hard constraints in self-modeling curve resolution problems. Alternating least squares with penalty functions. Anal. Chem. 2003; 75: 4236-4243.
22. Yeow YL, Leong YK. A general computational method for converting normal spectra into derivative spectra. Appl. Spectrosc. 2005; 59: 584-592.
23. Westerhuis JA, Derks EPPA, Hoefsloot HCJ, Smilde AK. Grey component analysis. J. Chemom. 2007; 21: 474-485.
24. Andries E, Hagstrom T, Atlas SR, William C. Regularization strategies for hyperplane classifiers: Application to cancer classification with gene expression data. J. Bioinformatics Comp. Biol. 2007; 5: 79-109.
25. Dax A. On regularized least norm problems. SIAM J. Optimization 1992; 2: 602-618.
26. Faber K, Kowalski BR. Propagation of measurement errors for the validation of predictions obtained by principal component regression and partial least squares. J. Chemom. 1997; 11: 181-238.
27. Faber K, Kowalski BR. Prediction error in least squares: Further critique on the deviation used in The Unscrambler. Chemom. Intell. Lab. Syst. 1996; 34: 283-292.
28. Forrester JB, Kalivas JH. Ridge regression optimization using a harmonious approach. J. Chemom. 2004; 18: 372-384.
29. Shih W, Bechtel KL, Feld MS. Constrained regularization: Hybrid method for multivariate calibration. Anal. Chem. 2007; 79: 234-239.
30. Lorber A, Faber K, Kowalski BR. Net analyte signal calculation in multivariate calibration. Anal. Chem. 1997; 69: 1620-1626.
31. Faber NM. Efficient computation of net analyte signal vector in inverse multivariate calibration. Anal. Chem. 1998; 70: 5108-5110.
32. Bro R, Andersen CM. Theory of net analyte signal vectors in inverse regression. J. Chemom. 2003; 17: 646-652.
33. Skibsted ETS, Boelens HFM, Westerhuis JA, Smilde AK, Broad NW, Rees DR, Witte DT. Net analyte signal baseed statistical quality control. Anal. Chem. 2005; 77: 7103-7114.
34. Stout F, Kalivas JH. Tikhonov regularization in standardized and general from for multivariate calibration with applications towards removing unwanted spectral artifacts. J. Chemom. 2006; 20: 22-33.
35. Kalivas JH. Pareto calibration with built-in wavelength selection. Anal. Chim. Acta 2004; 505: 9-14.
36. Ottaway J, Kalivas JH, Andries E. Spectral multivariate calibration with wavelength selection using variants of Tikhonov regularization. Appl. Spectrosc. 2010; 64: 1388-1395.
37. O'Leary DP, Simons JA. A bidiagonalization-regularization procedure for large scale discretizations of ill-posed problems. SIAM J. Sci. Stat. Comput. 1981; 2: 474-489.
38. Kilmer ME, O'Leary DP. Choosing regularization parameters in iterative methods for ill-posed problems. SIAM J. Matrix Anal. Appl. 2001; 22: 1204-1221.
39. Hoerl AE, Kennard R. Ridge regression: Biased estimation for nonorthogonal problems. Technometrics 1970; 12: 55-67.
40. Gorodnitsky IF, Rao BD. Sparse signal reconstruction from limited data using FOCUSS: A re-weighted minimum norm algorithm. IEEE Trans. Signal Process. 1997; 45: 600-616.
41. Li H, Liang Y, Xu Q, Cao D. Key wavelengths screening using competitive adaptive reweighted sampling method for multivariate calibration. Anal. Chim. Acta 2009; 648: 77-84.
42. Kondylis A, Whittaker J. Adaptively preconditioned Krylov spaces to identify irrelevant predictors. Chemom. Intell. Lab. Syst. 2010; 104: 205-213.
43. Andries E. personnel communication, erik.andries@gmail.com.
44. Tibshirani R. Regression shrinkage and selection via the LASSO. J. Roy. Statist. Soc. B 1996; 58: 267-288.
45. Claerbout JF, Muir F. Robust modeling with erratic data. Geophysics 1973; 38: 826-844.
46. 1 norm. Geophysics 1979; 44: 39-52.
47. Levy S, Fullager PK. Reconstruction of a sparse spike train from a portion of its spectrum and application to high-resolution deconvolution. Geophysics 1981; 46: 1235-1243.
48. Santosa F, Symes W. Linear inversion of band-limited reflection seismograms. SIAM J. Sci. Stat. Comput. 1986; 7: 1307-1330.
49. Efron B, Hastie T, Johnstone I, Tibshirani R. Least angle regression (with discussion). Annals Statist. 2004; 32: 407-499.
50. Chong I, Jun C. Performance of some variable selection methods when multicollinearity is present. Chemom. Intell. Lab. Syst. 2005; 78:103-112.
51. Stout F, Kalivas JH, Héberger K. Wavelength selection for multivariate calibration using Tikhonov regularization. Appl. Spectrosc. 2007; 61: 85-95.
52. Frank IE, Friedman JH. A statistical view of some chemometrics regression tools. Technometrics 1993; 35: 109-135.
53. Öjelund H, Madsen H, Thyregod P. Calibration with absolute shrinkage. J. Chemom. 2001; 15: 497-509.
54. Öjelund H, Madsen H, Thyregod P. Calibration with empirically weighted mean subset. Appl. Spectrosc. 2002; 56: 887-896.
55. Öjelund H, Madsen H, Brown PJ, Thyregod P. Prediction based mean subset. Technometrics 2002; 44: 369-378.
56. Fu WJ. Penalized regressions: the bridge versus the LASSO. J. Computnl. Graph. Statist. 1998; 7: 397-416.
57. van de Geer S. Least squares estimation with complexity penalties. Math. Meth. Statist. 2001; 10: 355-375.
58. Fan J, Li R. Variable selection via nonconcave penalized likelihood and its oracle properties. J. Am. Statist. Assoc. 2001; 96: 1348-1360.
59. Stout F, Kalivas JH. Evaluation of multivariate calibration using a Tikhonov regularization approach and the generalized pair-correlation method with non-linear data. Anal. Let. 2007; 40: 1227-1251.
60. Zou H. The adaptive LASSO ad its oracle properties. J. Am. Statist. Assoc. 2006; 101: 1418-1429.
61. Candes EJ, Wakin MB, Boyd SP. Enhancing sparsity by reweighted ℓ1 minimization. (accessed January 2012).
62. Lawson CL, Hanson RJ. Solving Least Square Problems. SIAM: Philadelphia, PA, 1995.
63. Kunz MR, Ottaway J, Kalivas JH, Andries E. Impact of standardization sample designs on Tikhonov regularization variants for spectroscopic calibration maintenance and transfer. J. Chemom. 2009; 24: 218-229.
64. Kunz MR, Kalivas JH, Andries E. Model updating for spectral calibration maintenance and transfer using 1-norm variants of Tikhonov regularization. Anal. Chem. 2010; 82: 3642-3649.
65. Fu GH, Xu QS, Li HD, Cao DS, Liang YZ. Elastic net grouping variable selection combined with partial lest squares regression (EN-PLSR) for the analysis of strongly multi-collinear spectroscopic data. Appl. Spectrosc. 2011; 65: 402-408.
66. Andries JPM, Heyden YV, Buydens LMC. Improved variable reduction in partial least squares modeling based on predictive-property-ranked variables and adaption of partial least squares complexity. Anal. Chim. Acta 2011; 705: 292-305.
67. Zou H, Hastie T. Regularization and variable selection via the elastic net. J. Royal Statist. Soc. B 2005; 67: 301-320.
68. Lê Cao KA, Rossouw D, Rabert-Granié C, Besse P. A sparse PLS for variable selection when integrating Omics data. Stat. Appl. Genet. Mol. Biol. 2008; 7: Article 35, 1-29.
69. Lee D, Lee W, Pawitan Y. Sparse partial least-squares regression and its applications to high-throughput data analysis. Chemom. Intell. Lab. Syst. 2011; 109: 1-8.
70. Huang X, Pan W, Park S, Han X, Miller LW, Hall J. Modeling the relationship between LVAD support time and gene expression changes in the human heart by penalized partial least squares. Bioinformatics 2004; 20: 888-894.
71. Chun H, Keles S. Sparse partial least squares regression for simultaneous dimension reduction and variable selection. J.R. Statist. Soc. B 2010; 72: 3-25.
72. Kalivas JH, Siano GS, Andries E, Goicoechea HC. Calibration maintenance and transfer using Tikhonov regularization approaches. Appl. Spectrosc. 2009; 63: 800-809.
73. Kunz MR, Ottaway J, Kalivas JH, Georgiou CA, Mousdis GA. Updating a synchronous fluorescence spectroscopic virgin olive oil adulteration calibration to a new geographical region. J. Agric. Food Chem. 2011; 59: 1051-1057.
74. Farrell JA, Higgins K, Kalivas JH. Updating a near-infrared multivariate calibration model formed with lab-prepared pharmaceutical tablet types to new tablet types in full production. J. Pharm. Biomed. Anal. 2012; 61: 114-121.
75. Westerhaus MO. Improving repeatability of NIR calibrations across instruments. In Proceedings of the Third International Near Infrared Spectroscopy Conference. Biston R, Bartiaux-Thill N (eds.). Agriculture Research Centre Publishing: Gembloux, 1991; 671-674.
76. DiFoggio R. Desensitizing models using covariance matrix transforms or counter-balanced distortion. J. Chemom. 2005; 19: 203-215.
77. DiFoggio R. Influencing models to improve their predictions of standard samples. J. Chemom. 2007; 21: 208-214.
78. Green RL, Kalivas JH. Graphical diagnostics for regression model determinations with consideration of the bias/variance trade-off. Chemom. Intell. Lab. Syst. 2002; 60: 173-188.
79. Stout F, Baines M, Kalivas JH. Impartial graphical comparison of multivariate calibration methods and the harmony/parsimony tradeoff. J. Chemom. 2006; 20: 464-475.
80. Pinto LA, Galvão RKH, Araújo MCU. Ensemble wavelet modeling for determination of wheat and gasoline properties by near and model infrared spectroscopy. Anal. Chim. Acta 2010; 682: 37-47.
81. Gowen AA, Downey G, Esquerre C, O'Donnell CP. Preventing over-fitting in PLS calibration models of near-infrared (NIR) spectroscopy data using regression coefficients. J. Chemom. 2011; 25: 375-381.
82. Höskuldsson A. The H-principle in modeling with applications to chemometrics. Chemom. Intell. Lab. Syst. 1992; 14: 139-153.
83. Myers RH. Classical and Modern Regression with Applications (2nd edn). Duxbury: Pacific Grove, CA, 1990.
84. Golub GH, Heath M, Wahba G. Generalized crossvalidation as a method for choosing a good ridge parameter. Technometrics 1979; 21: 215-223.
85. Leion H, Folestad S, Josefson M, Sparén A. Evaluation of basic algorithms for transferring quantitative multivariate calibrations between scanning grating and FT-IR spectrometers. J. Pharm. Biomed. Anal. 2005; 37: 47-55.
86. Bergman EL, Brage H, Josefson M, Svensson O, Sparén A. Transfer of NIR calibrations for pharmaceutical formulations between different instruments. J. Pharm. Biomed. Anal. 2006; 41: 89-98.
87. Kalivas JH. Learning from Procrustes analysis to improve multivariate calibration. J. Chemom. 2008; 22: 227-234.
88. Bro R, Smilde AK. Centering and scaling in component analysis. J. Chemom. 2003; 17: 16-33.
89. Lorber A, Faber K, Kowalski BR. Local centering in multivariate calibration. J. Chemom. 1996; 10: 215-220.
90. Seasholtz MB, Kowalski BR. The effect of mean centering in prediction in multivariate calibration. J. Chemom. 1992; 6: 103-111.
91. Anderson CE, Kalivas JH. Fundamentals of calibration transfer through Procrustes analysis. Appl. Spectrosc. 1999; 53: 1268-1276.
92. Swierenga H, Haanstra WG, de Weijer AP, Buydens LMC. Comparison of two different approaches toward model transferability in NIR spectroscopy. Appl. Spectrosc. 1998; 52: 7-16.