Kernel logistic PLS: A tool for supervised nonlinear dimensionality reduction and binary classification

Computational Statistics and Data Analysis

Volumn 51, Issue 9, 2007, Pages 4083-4100

  Tenenhaus, Arthur ^a,b   Giron, Alain ^a   Viennet, Emmanuel ^c   Béra, Michel ^b   Saporta, Gilbert ^d   Fertil, Bernard ^a,e  

^a PITIÉ SALPÊTRIÈRE HOSPITAL   (France)

^b KXEN research   (France)

^c UNIVERSITÉ PARIS 13   (France)

^d Bioinformatique et Chimie Moléculaire   (France)

^e AIX MARSEILLE UNIV   (France)

Author keywords

Classification; Dimensionality reduction; Kernel; Logistic regression; PLS regression

Indexed keywords

ALGORITHMS; BINARY CODES; COMPUTATIONAL COMPLEXITY; REGRESSION ANALYSIS; SUPPORT VECTOR MACHINES;

BINARY CLASSIFICATION; DATA SEGMENTATION; LOGISTIC REGRESSION; SUPERVISED NONLINEAR DIMENSIONALITY REDUCTION;

DATA REDUCTION;

EID: 34147155819     PISSN: 01679473     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.csda.2007.01.004     Document Type: Article

References (42)

1. Albert A., and Anderson J.A. On the existence of maximum likelihood estimates in logistic regression models. Biometrika 71 (1984) 1-10
2. Allison, D.P., 1999. Logistic regression using the SAS system: theory and application, SAS institute, ISBN: 1-58025-352-0.
3. Bach, F.R., Jordan, M.I., 2005. Predictive low-rank decomposition for kernel methods. In: Proceedings of the Twenty-second International Conference on Machine Learning, Bonn, Germany.
4. Baffi G., Martin E.B., and Morris A.J. Non-linear projection to latent structures revisited (the neural networks PLS algorithm). Comput. Chem. Eng. 23 (1999) 1293-1307
5. Barker M., and Rayens W.S. Partial least squares for discrimination. J. Chemometrics 17 (2003) 166-173
6. Bastien P., Vinzi V.E., and Tenenhaus M. PLS generalized linear regression. Comput. Statist. Data Anal. 48 (2005) 17-46
7. Bennett K.P., and Embrechts M.J. An optimization perspective on kernel partial least squares regression. Advances in Learning Theory: Methods, Models and Applications. NATO Sciences Series III: Computer and Systems Sciences vol. 190 (2003), IOS Press, Amsterdam 227-250
8. Boser, B.E., Guyon, I., Vapnik, V.N., 1992. A training algorithm for optimal margin classifiers. In: Proceedings of the Fifth Annual Workshop on Computational Learning Theory, vol. 5, Pittsburgh, pp. 144-152.
9. Burges C.J.C. A tutorial on support vector machines for pattern recognition. Data Mining and Knowledge Discovery 2 2 (1998) 121-167
10. Cherkassky V., Mulier F.P., and Vapnik V.N. Model complexity control for regression using VC generalization bounds. IEEE Transactions on Neural Networks 10 (1999) 1075-1089
11. Cortes C., and Vapnik V. Support vector network. Mach. Learning 20 (1995) 273-297
12. Evgeniou T., Pontil M., and Poggio T. Regularization networks and support vector machines. Adv. Comput. Math. 13 (1999) 1-50
13. Firth D. Bias reduction of maximum likelihood estimates. Biometrika 80 (1993) 27-38
14. Garthwaite P.H. An interpretation of partial least squares. J. Amer. Statist. Assoc. 89 425 (1994) 122-127
15. Hanley J.A. Receiver operating characteristic methodology: the state of the art. Critical Reviews in Diagnostic Imaging 29 (1989) 307-335
16. Heinze G., and Schemper M. A solution to the problem of separation in logistic regression. Statist. Medicine 21 (2002) 2409-2419
17. Höskuldsson A. PLS regression methods. J. Chemometrics 2 (1988) 211-228
18. Joachims J. Making large-scale SVM learning practical. In: Schölkopf B., Burges C., and Smola A. (Eds). Advances in Kernel Methods-Support Vector Learning (1999), MIT Press, Cambridge, MA
19. Li, J., Liu, H., 2002. Kent Ridge Biomedical Dataset Repository. 〈http://sdmc-lit.org.sg/GEDatasets〉
20. Rätsch G., Onoda T., and Muller K.R. Soft margin for adaboost. Mach. Learning 42 (2001) 287-320
21. Rosipal R., and Trejo L.J. Kernel partial least squares regression in reproducing kernel hilbert space. J. Mach. Learning Res. 2 (2001) 97-123
22. Rosipal, R., Trejo, L.J., Matthews, B., 2003. Kernel PLS-SVC for linear and nonlinear classification. In: Proceeding of the Twentieth International Conference on Machine Learning, Washington, USA.
23. Schölkopf B., and Smola A.J. Learning With Kernel-Support Vector Machines Regularization Optimization and Beyond (2002), MIT Press, Cambridge, MA
24. Schölkopf B., Smola A.J., and Müller K.R. Nonlinear component analysis as a kernel eigenvalue problem. Neural Comput. 10 (1998) 1299-1319
25. Shawe-Taylor J., and Cristianini N. Kernel Methods for Pattern Analysis (2004), Cambridge University Press, Cambridge
26. Shen L., and Tan E.C. Dimension reduction-based penalized logistic regression for cancer classification using microarray data. IEEE/ACM Trans. Comput. Biol. Bioinformatics 2 2 (2005) 166-175
27. Smola, A.J., Schölkopf, B., 2000. Sparse greedy matrix approximation for machine learning. In: Proceeding of the Seventeenth International Conference on Machine Learning, Stanford, USA.
28. Tenenhaus A. La Régression Logistique PLS validée par bootstrap (2002), Master degree of statistics, Pierre and Marie Curie University, Paris, France
29. Tenenhaus, M., 1998. La Régression PLS, éditions Technip.
30. Tenenhaus M. La régression logistique PLS. In: Droesbeke J.J., Lejeune M., and Saporta G. (Eds). Modèles statistiques pour données qualitatives (2005), Editions Technip
31. Tenenhaus, A., Giron, A., Saporta, G., Fertil, B., 2005. Kernel logistic PLS: a new tool for complex classification. In: 11th International Symposium on Applied Stochastic Models and Data Analysis, Brest, France.
32. Tikhonov A.N., and Arsenin V.Y. Solution of Ill-Posed Problem (1977), Wiley, New York
33. Tucker L.R. An inter-battery method of factor analysis. Psychometrika 23 2 (1958)
34. Vapnik V. Statistical Learning Theory (1998), Wiley, New York
35. Wahba, G., 1999. Support vector machines, reproducing kernel Hilbert spaces and randomized GACV. In: Advances in Kernel Methods-Support Vector Learning, pp. 69-88.
36. Webb, A.R., 1996. Nonlinear feature extraction with radial basis functions using a weighted multidimensional scaling stress measure. In: 13th International Conference on Pattern Recognition.
37. Williams, C.K.I., Seeger, M., 2000. Effect on the input density distribution on kernel-based classifiers. In: Proceeding of the Seventeenth International Conference on Machine Learning, Stanford, USA.
38. Wold S. Non-linear partial least squares modelling. II Spline inner function. Chemometrics Intell. Lab. Systems 14 (1992) 71-84
39. Wold S., Martens H., and Wold H. The multivariate calibration problem in chemistry solved by the PLS method. Lecture Notes in Mathematics (1983), Springer, Heidelberg 286-293
40. Wold S., Kettaneh-Wold N., and Skagerberg B. Non-linear PLS modelling. Chemometrics Intell. Lab. Systems 7 (1989) 53-65
41. Wu W., Massart D.L., and de Jong S. The kernel PCA algorithms for wide data. Part II: fast cross validation and application in classification of NIR data. Chemometrics Intell. Lab. Systems 37 (1997) 271-280
42. Zhu J., and Hastie T. Kernel logistic regression and the import vector machine. J. Comput. Graphical Statist. 14 1 (2005) 185-205