Characterizing multivariate calibration tradeoffs (bias, variance, selectivity, and sensitivity) to select model tuning parameters

Journal of Chemometrics

Volumn 28, Issue 5, 2014, Pages 347-357

  Kalivas, John H ^a   Palmer, Jon ^a  

^a IDAHO STATE UNIVERSITY   (United States)

Author keywords

Bias variance tradeoff; Model selectivity; Model sensitivity; Multivariate calibration; NAS; Net analyte signal; Ridge regression; Selectivity sensitivity tradeoff; Tikhonov regularization

Indexed keywords

LEAST SQUARES APPROXIMATIONS; PRINCIPAL COMPONENT ANALYSIS; REGRESSION ANALYSIS; TUNING;

BIAS-VARIANCE TRADEOFFS; MODEL SELECTIVITY; MODEL SENSITIVITY; MULTIVARIATE CALIBRATION; NAS; NET ANALYTE SIGNALS; RIDGE REGRESSION; SELECTIVITY/SENSITIVITY TRADEOFF; TIKHONOV REGULARIZATION; TUNING PARAMETER;

MEAN SQUARE ERROR;

EID: 84900311457     PISSN: 08869383     EISSN: 1099128X     Source Type: Journal    
DOI: 10.1002/cem.2555     Document Type: Article

References (66)

1. Næs T, Isaksson T, Fern T, Davies T. A User Friendly Guide to Multivariate Calibration and Classification. NIR Publications: Chichester, UK, 2002.
2. Hastie TJ, Tibshirani RJ, Friedman J. The Elements of Statistical Learning: Data Mining, Inference, and Prediction (2nd edn). Springer-Verlag: New York, 2009.
3. Kalivas JH. Calibration methodologies. In Comprehensive Chemometrics: Chemical and Biochemical Data Analysis Vol. 3, Brown SD, Tauler R, Walczak B (eds-in-chief). Elsevier: Amsterdam, 2009; 1-32.
4. Marini F. Neural networks. In Comprehensive Chemometrics: Chemical and Biochemical Data Analysis Vol. 3, Brown SD, Tauler R, Walczak B (eds-in-chief). Elsevier: Amsterdam, 2009; 477-505.
5. Marbach R. A new method for multivariate calibration. J. Near Infrared Spectrosc. 2005; 13: 241-254.
6. Boulet, JC, Roger JM. Improvement in direct calibration. Anal. Chim. Acta 2010; 668: 130-136.
7. Ottaway J, Farrell JA, Kalivas JH. Spectral multivariate calibration without laboratory prepared or determined reference analyte values. Anal. Chem. 2013; 85: 1509-1516.
8. Booksh K, Kowalski BR. Theory of analytical chemistry. Anal. Chem. 1994; 66: 782A-791A.
9. Carey WP, Kowalski BR. Chemical piezoelectric sensor and sensor array characterization. Anal. Chem. 1986; 58: 3077-3084.
10. Juhl LL, Kalivas JH. Evaluation of experimental designs for multicomponent determination by spectrophotometry. Anal. Chim. Acta 1988; 207: 125-135.
11. Kalivas JH. A simplex optimized inductively coupled plasma spectrometer with minimization of interferences. Appl. Spectrosc. 1987; 41: 13381342.
12. Kalivas JH, Lang PM. Mathematical Analysis of Spectral Orthogonality. Marcel Dekker: New York, 1994.
13. Faber NM. Multivariate sensitivity for the interpretation of the effect of spectral pretreatment methods on near-infrared calibration model predictions. Anal. Chem. 1999; 71: 557-565.
14. Faber NM. A closer look at the bias-variance trade-off in multivariate calibration. J. Chemom. 1999; 13: 185-192.
15. Höskuldsson A. Dimension of linear models. Chemom. Intell. Lab. Syst. 1996; 32: 37-55.
16. Ye S. On measuring and correcting the effects of data mining and model selection. J. Am. Stat. Assoc. 1998; 93: 120-133.
17. Denham MC. Choosing the number of factors in partial least squares regression: estimating and minimizing the mean squared error of prediction. J. Chemom. 2000; 14: 351-361.
18. Li B, Morris J, Martin EB. Model selection for partial least squares regression. Chemom. Intell. Lab. Syst. 2002; 64: 79-89.
19. Gómez-Carracedo MP, Andrade JM, Rutledge DN, Faber NM. Selecting the optimum number of partial least squares component for the calibration of attenuated total reflectance-mid-infrared spectra of undersigned kerosene samples. Anal. Chim. Acta 2007; 585: 253-265.
20. Goirvénec S, Pierna F, Massart DL, Rutledge DN. An evaluation of the PoLiSh smoothed regression and the Monte Carlo cross-validation for the determination of the complexity of a PLS model. Chemom. Intell. Lab. Syst. 2003; 68: 41-51.
21. Shao J. Linear Model selection by cross-validation. J. Am. Stat. Assoc. 1993; 88: 486-494.
22. Xu QS, Liang YZ. Monte Carlo cross validation. Chemom. Intell. Lab. Syst. 2001; 56: 1-11.
23. Baumann K, Albert H, von Korff M. A systematic evaluation of the benefits and hazards of variable selection in latent variable regression. Part I. Search algorithm, theory, and simulations. J. Chemom. 2002; 16: 339-350.
24. Filzmoser P. Liebmann B. Varmuza K. Repeated double cross-validation. J. Chemom. 2009; 23: 160-171.
25. Wold S. Cross-vaildatory estimation of the number of components in factor and principal component models. Technometrics 1978; 20: 397-405.
26. Lawson CL, Hanson RJ. Solving Least Square Problems. SIAM: Philadelphia, PA, 1995.
27. Hansen PC. Rank-Deficient and Discrete Ill-Posed Problems: Numerical Aspects of Linear Inversion. SIAM: Philadelphia, PA, 1998.
28. Hansen PC. Analysis of discrete ill-posed problems by means of the L-curve. SIAM Rev. 1992; 34: 561-580.
29. Kalivas JH. Basis sets for multivariate regression. Anal. Chim. Acta 2001; 428: 31-40.
30. Green RL, Kalivas JH. Graphical diagnostics for regression model determinations with consideration of the bias/variance trade-off. Chemom. Intell. Lab. Syst. 2002; 60: 173-188.
31. Forrester JB, Kalivas JH. Ridge regression optimization using a harmonious approach. J. Chemom. 2004; 18: 372-384.
32. Gowen AA, Downey G, Esquerre C, O'Donnell CP. Preventing over-fitting in PLS calibration models of near-infrared (NIR) spectroscopy data using regression coefficients. J. Chemom. 2011; 25: 375-381.
33. Tikhonov AN. Solution of incorrectly formulated problems and the regularization method. Soviet Math. Dokl. 1963; 4: 1035-1038.
34. Dax A. On regularized least norm problems. SIAM J. Optimization 1992; 2: 602-618.
35. Aster RC, Borchers B, Thurber CH. Parameter Estimation and Inverse Problems. Elsevier: Amsterdam, 2005.
36. 1) Tikhonov regularization variants for full wavelength or sparse spectral multivariate calibration models or maintenance. J. Chemom. 2012; 26: 218-230.
37. O'Leary DP, Simons JA. A bidiagonalization-regularization procedure for large scale discretizations of ill-posed problems. SIAM J. Sci. Stat. Comput. 1981; 2: 474-489.
38. Kilmer ME, O'Leary DP. Choosing regularization parameters in iterative methods for ill-posed problems. SIAM J. Matrix Anal. Appl. 2001; 22: 1204-1221.
39. Myers RH. Classical and Modern Regression with Applications (2nd edn). Duxbury: Pacific Grove, CA, 1990.
40. Golub GH, Heath M, Wahba G. Generalized crossvalidation as a method for choosing a good ridge parameter. Technometrics 1979; 21: 215-223.
41. Höskuldsson A. The H-principle in modeling with applications to chemometrics. Chemom. Intell. Lab. Syst. 1992; 14: 139-153.
42. Faber K, Kowalski BR. Propagation of measurement errors for the validation of predictions obtained by principal component regression and partial least squares. J. Chemom. 1997; 11: 181-238.
43. Faber NM, Duewer DL, Choquette SJ, Green TL, Chesler SN. Characterizing the uncertainty in near-infrared Spectroscopic prediction of mixed-oxygenate concentrations in gasoline:sample-specific prediction intervals. Anal. Chem. 1998; 70: 2972-2982.
44. Stout F, Baines M, Kalivas JH. Impartial graphical comparison of multivariate calibration methods and the harmony/parsimony tradeoff. J. Chemom. 2006; 20: 464-475.
45. Pinto LA, Galvão RKH, Araújo MCU. Ensemble wavelet modeling for determination of wheat and gasoline properties by near and model infrared spectroscopy. Anal. Chim. Acta 2010; 682: 37-47.
46. van der Voet H. Pseudo-degrees of freedom for complex predictice models; an example of partial least squares. J. Chemom. 1999; 13: 195-208.
47. Seipel HA, Kalivas JH. Effective rank for multivariate calibration methods. J. Chemom. 2004; 18: 306-311.
48. Kalivas JH, Seipel HA. Erratum to Seipel HA, Kalivas JH. Effective rank for multivariate calibration methods. J. Chemom. 2004; 18: 306-311. J. Chemom. 2005; 19: 64.
49. Andries E, Kalivas JH. Multivariate calibration leverages and spectral F-ratios via the filter factor representation. J. Chemom. 2010; 24: 249-260.
50. Rubingh CM, Martens H, van der Voet H, Smilde AK. The costs of complex model optimization. Chemom. Intell. Lab. Syst. 2013; 125: 139-146.
51. Esbensen KH, Geladi P. Principles of proper validation: use and abuse of re-sampling for validation. J. Chemom. 2010; 24: 168-187.
52. Hawkins DM, Kraker J. Deterministic fallacies an model validation. J. Chemom. 2010; 24: 188-193.
53. Consonni V, Ballabio D, Todeschini R. Evaluation of model predictive ability by external validation techniques. J. Chemom. 2010; 24: 194-201.
54. Brenchley JM, Hörchner U, Kalivas JH. Wavelength selection characterization for NIR spectra. Appl. Spectrosc. 1997; 51: 689-699.
55. Tarumi T, Wu Y, Small GW. Multivariate calibration with basis functions derived from optical filters. Anal. Chem. 2009; 81: 2199-2207.
56. Costa NR, Lourenço J, Pereira ZL. Desirability function approach: a review and performance evaluation in adverse conditions. Chemom. Intell. Lab. Syst. 2011; 107: 234-244.
57. Lorber A, Faber K, Kowalski BR. Net analyte signal calculation in multivariate calibration. Anal. Chem. 1997; 69: 1620-1626.
58. Ferré J, Faber NM. Net analyte signal calculation for multivariate calibration. Chemom. Intell. Lab. Syst. 2003; 69: 123-136.
59. Andries E, Kalivas JH. Interrelationships between generalized Tikhonov regularization, generalized net analyte signal, and generalized least squares for desensitizing a multivariate calibration to interferences. J. Chemom. 2013; 27: 126-140.
60. Brown CD, Green RL. Critical factors limiting the interpretation of regression vectors in multivariate calibration. Trends Anal. Chem. 2009; 28: 506-514.
61. Brown CD. Discordance between net analyte signal theory and practical multivariate calibration. Anal. Chem. 2004; 76: 4364-4373.
62. Mark H. Principles and Practice of Spectroscopic Calibration. Wiley: New York, 1991.
63. Faber NM. Notes on two competing definitions of multivariate sensitivity. Anal. Chim. Acta 1999; 381: 103-109.
64. Wiülfert F, Kok WTh, Smilde AK. Influence of temperature on vibration spectra and consequences for multivariate models. Anal. Chem. 1998; 70: 1761-1767.
65. http://www.models.life.ku.dk/datasets, accessed May 15, 2013.
66. Wise BM, Gallagher NB, PLS_Toolbox7.0.3, Eigenvector Research, Manson, Washington.