A Projection Pursuit framework for supervised dimension reduction of high dimensional small sample datasets

Neurocomputing

Volumn 149, Issue PB, 2015, Pages 767-776

  Espezua, Soledad ^a   Villanueva, Edwin ^a   Maciel, Carlos D ^a   Carvalho, André ^a  

^a UNIVERSITY OF SÃO PAULO   (Brazil)

Author keywords

Classification; Dimension reduction; Gene expression; Projection Pursuit

Indexed keywords

CLASSIFICATION (OF INFORMATION); MOLECULAR BIOLOGY;

CLASSIFIER SYSTEMS; DIMENSION REDUCTION; DIMENSION REDUCTION METHOD; EXPERIMENTAL EVALUATION; MICROARRAY DATA SETS; PROJECTION PURSUITS; SCIENTIFIC COMMUNITY; SUPERVISED DIMENSION REDUCTIONS;

GENE EXPRESSION;

ACCURACY; ARTICLE; CLASSIFICATION; CLASSIFIER; GENOMICS; INFORMATION PROCESSING; PARTIAL LEAST SQUARES REGRESSION; PROJECTION PURSUIT; PROTEOMICS;

EID: 85027943239     PISSN: 09252312     EISSN: 18728286     Source Type: Journal    
DOI: 10.1016/j.neucom.2014.07.057     Document Type: Article

References (83)

1. Johnstone I.M., Titterington D.M. Statistical challenges of high-dimensional data INTRODUCTION. Philos. Trans. R. Soc. A-Math. Phys. Eng. Sci. 2009, 367(1906):4237-4253.
2. Russell S., Meadows L., Russell R. Microarray Technology in Practice 2008, Elsevier/Academic Press, San Diego, USA. 1 ed.
3. Golub T.R., Slonim D.K., Tamayo P., Huard C., Gaasenbeek M., Mesirov J.P., Coller H., Loh M.L., Downing J.R., Caligiuri M.A., Bloomfield C.D., Lander E.S. Molecular classification of cancer. class discovery and class prediction by gene expression monitoring. Science 1999, 286:531-537.
4. Segal E., Friedman N., Kaminski N., Regev A., Koller D. From signatures to models. understanding cancer using microarrays. Nat. Genet. 2005, 37(S):S38-S45.
5. Jimenez L., Landgrebe D. Hyperspectral data analysis and supervised feature reduction via projection pursuit. IEEE Trans. Geosci. Remote Sens. 1999, 37(6):2653-2667.
6. Clarke R., Ressom H.W., Wang A., Xuan J., Liu M.C., Gehan E.A., Wang Y. The properties of high-dimensional data spaces. implications for exploring gene and protein expression data. Nat. Rev. Cancer 2008, 8(1):37-49.
7. Korn F., Pagel B., Faloutsos C. On the "dimensionality curse" and the "self-similarity blessing". IEEE Trans. Knowl. Data Eng. 2001, 13(1):96-111.
8. Liu H., Yu L. Toward integrating feature selection algorithms for classification and clustering. IEEE Trans. Knowl. Data Eng. 2005, 17(4):491-502.
9. Saeys Y., Inza I.n., Larrañaga P. A review of feature selection techniques in bioinformatics. Bioinformatics 2007, 23(19):2507-2517.
10. Feature Extraction, Foundations and Applications 2006, Springer-Verlag New York, Inc., Secaucus, NJ, USA. I. Guyon, S. Gunn, M. Nikravesh, L. Zadeh (Eds.).
11. C.J.C. Burges, Dimension reduction: a guided tour, Found. Trends Mach. Learn., Now Publishers Inc., Hanover, MA, USA, 2 (4).
12. Liu H., Motoda H. Feature Extraction, Construction and Selection. A Data Mining Perspective 1998, Kluwer Academic Publishers, Now Publishers Inc., Hanover, MA, USA.
13. Friedman J.H., Tukey J.W. A projection pursuit algorithm for exploratory data analysis. IEEE Trans. Comput. 1974, 23(9):881-890.
14. Friedman J.H. Exploratory projection pursuit. Am. Stat. Assoc. 1987, 82(397):249-266.
15. Huang W., Zhang X. Projection pursuit flood disaster classification assessment method based on multi-swarm cooperative particle swarm optimization. J. Water Resour. Prot. 2011, 3(6):415-420.
16. Wang S., Zhang X. Projection pursuit dynamic cluster model and its application to water resources carrying capacity evaluation. J. Water Resour. Prot. 2010, 2(5):2474-2482.
17. Malpica J.A., Rejas J.G., Alonso M.C. A projection pursuit algorithm for anomaly detection in hyperspectral imagery. Pattern Recognit. 2008, 41:3313-3327.
18. Cui X., Jin R., Asghar Xu., Condamine P., Svensson J.T., Wanamaker S., Stein N., Roose I., Close T.J. Detecting single-feature polymorphisms using oligonucleotide arrays and robustified projection pursuit. Bioinformatics 2005, 21(20):3852-3858.
19. Pena D., Prieto F. Cluster identification using projections. J. Am. Stat. Assoc. 2001, 96(456):1433-1445.
20. Bolton R., Krzanowski W. Projection pursuit clustering for exploratory data analysis. J. Comput. Graph. Stat. 2003, 12(1):121-142.
21. Lee E., Cook D., Klinke S., Lumley T. Projection pursuit for exploratory supervised classification. J. Comput. Graph. Stat. 2005, 14(4):831-846.
22. Demirci O., Clark V.P., Calhoun V.D. A projection pursuit algorithm to classify individuals using fMRI data: application to schizophrenia. Neuroimage 2008, 39(4):1774-1782.
23. Pires A.M., Branco J.A. Projection-pursuit approach to robust linear discriminant analysis. J. Multivar. Anal. 2010, 101(10):2464-2485.
24. Grochowski M., Duch W. Fast projection pursuit based on quality of projected clusters. Adaptive and Natural Computing Algorithms Lecture Notes in Computer Science 2011, vol. 6594:89-97. Springer, Berlin, Heidelberg.
25. Ren Y., Liu H., Yao X., Liu M. Prediction of ozone tropospheric degradation rate constants by projection pursuit regression. Anal. Chim. Acta 2007, 589(1):150-158.
26. Aladjem M. Projection pursuit mixture density estimation. IEEE Trans. Signal Process. 2005, 53(11):4376-4383.
27. Jee J.R. Projection pursuit. Wiley Interdiscip. Rev.: Comput. Stat. 2009, 1(2):208-215.
28. Rodriguez-Martinez E., Goulermas J.Y., Mu T., Ralph J.F. Automatic induction of projection pursuit indices. IEEE Trans. Neural Netw. 2010, 21(8):1281-1295.
29. Huber P.J. Projection pursuit. Ann. Stat. 1985, 13(2):435-475.
30. Jones M.C., Sibson R. What is projection pursuit?. J. R. Stat. Soc. Ser. A: General 1987, 150(1):1-37.
31. Nason G. Three-dimensional projection pursuit. J. R. Stat. Soc. Ser. C 1995, 44:411-430.
32. Guo Q., Wu W., Questier F., Massart D., Boucon C., de Jong S. Sequential projection pursuit using genetic algorithms for data mining of analytical data. Anal. Chem. 2000, 72(13):2846-2855.
33. Espezua S., Villanueva E., Maciel C.D. Towards an efficient genetic algorithm optimizer for sequential projection pursuit. Neurocomputing 2014, 123:40-48.
34. Webb-Robertson B., Jarman K., Harvey S., Posse C., Wright B. An improved optimization algorithm and Bayes factor termination criterion for sequential projection pursuit. Chemom. Intell. Lab. Syst. 2005, 77(1-2):149-160.
35. Berro A., Marie-Sainte S.L., Ruiz-Gazen A. Genetic algorithms and particle swarm optimization for exploratory projection pursuit. Ann. Math. Artif. Intell. 2010, 60(1-2, SI):153-178.
36. Kruskal J. Toward a practical method which helps uncover the structure of a set of multivariate observations by finding the linear transformation which optimizes a new "Index of Condensation". Stat. Comput. 1969, 427-440.
37. G. Nason, Design and choice of projection indices (Ph.D. thesis), University of Bath, 1992.
38. Posse C. Projection pursuit exploratory data analysis. Comput. Stat. Data Anal. 1995, 20:669-687.
39. Posse C. Tools for two-dimensional exploratory projection pursuit. J. Comput. Graph. Stat. 1995, 4(2):83-100.
40. S.L. Marie-Sainte, A. Berro, A. Ruiz-Gazen, An efficient optimization method for revealing local optima of projection pursuit indices, in: ANTS Conference, 2010, pp. 60-71.
41. Zhang K., Chan L.-W. Dimension reduction as a deflation method in ICA. IEEE Sign. Process. Lett. 2006, 13(1):45-48.
42. Perisic I., Posse C. Projection pursuit indices based on the empirical distribution function. J. Comput. Graph. Stat. 2005, 14(3):700-715.
43. Cook D., Buja A., Cabrera J. Projection pursuit indexes based on orthonormal function expansions. J. Comput. Graph. Stat. 1993, 2(3):225-250.
44. M. Grochowski, W. Duch, Projection pursuit constructive neural networks based on quality of projected clusters, in: V. Kurkova, R. Neruda, J. Koutnik (Eds.), Artificial Neural Networks-ICANN 2008, PT II, Lecture Notes In Computer Science, vol. 5164, 2008, pp. 754-762.
45. Lee E.-K., Cook D. A projection pursuit index for large p small n data. Stat. Comput. 2010, 20(3):381-392.
46. Friedman J.H., Stuetzle W. Projection pursuit regression. J. Am. Stat. Assoc. 1981, 76:817-823.
47. Hall P. On projection pursuit regression. Ann. Stat. 1989, 17(2):573-588.
48. J. Goldberger, S. Roweis, G. Hinton, R. Salakhutdinov, Neighbourhood components analysis, in: Advances in Neural Information Processing Systems, vol. 17, 2005, pp. 513-520.
49. Yang W., Wang K., Zuo W. Fast neighborhood component analysis. Neurocomputing 2012, 83:31-37.
50. H.V. Nguyen, L. Bai, Face verification using indirect neighbourhood components analysis, in: Proceedings of the 6th International Conference on Advances in Visual Computing-Volume Part II, ISVC'10, Springer-Verlag, Berlin, Heidelberg, 2010, pp. 637-646.
51. A. Khandelwal, P. Choudhury, R. Sarkar, S. Basu, M. Nasipuri, N. Das, Text line segmentation for unconstrained handwritten document images using neighborhood connected component analysis, in: Proceedings of the 3rd International Conference on Pattern Recognition and Machine Intelligence, PReMI '09, Springer-Verlag, Berlin, Heidelberg, 2009, pp. 369-374.
52. He X., Yan S., Hu Y., Niyogi P., Jiang Zhang H. Face recognition using Laplacianfaces. IEEE Trans. Pattern Anal. Mach. Intell. 2005, 27:328-340.
53. Asuncion Vicente M., Hoyer P.O., Hyvarinen A. Equivalence of some common linear feature extraction techniques for appearance-based object recognition tasks. IEEE Trans. Pattern Anal. Mach. Intell. 2007, 29(5):896-900.
54. Deng W., Liu Y., Hu J., Guo J. The small sample size problem of ICA. a comparative study and analysis. Pattern Recognit. 2012, 45(12):4438-4450.
55. Biswas S., Storey J., Akey J. Mapping gene expression quantitative trait loci by singular value decomposition and independent component analysis. BMC Bioinform. 2008, 9(1):244.
56. P. Cunningham, Dimension Reduction, Technical Report, 2007.
57. DiPietroPaolo D., Mller H.-P., Nolte G., Ern S.N. Noise reduction in magnetocardiography by singular value decomposition and independent component analysis. Med. Biol. Eng. Comput. 2006, 44(6):489-499.
58. Shen R., Ghosh D., Chinnaiyan A., Meng Z. Eigengene-based linear discriminant model for tumor classification using gene expression microarray data. Bioinformatics 2006, 22(21):2635-2642.
59. Alter O., Brown P.O., Botstein D. Generalized singular value decomposition for comparative analysis of genome-scale expression data sets of two different organisms. Proc. Natl. Acad. Sci. 2003, 100(6):3351-3356.
60. M. Wall, A. Rechtsteiner, L. Rocha, Singular value decomposition and principal component analysis, in: A Practical Approach to Microarray Data Analysis, 2003, pp. 91-109.
61. Alter O., Brown P.O., Botstein D. Singular value decomposition for genome-wide expression data processing and modeling. Proc. Natl. Acad. Sci. 2000, 97(18):10101-10106.
62. Jolliffe I. Principal Component Analysis 1986, Springer, New York, NY.
63. Nguyen T.S., Rojo J. Dimension reduction of microarray gene expression data. the accelerated failure time model. J. Bioinform. Comput. Biol. 2009, 7(6):939-954.
64. R. Rosipal, N. Krmer, Overview and recent advances in partial least squares, in: Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 3940, 2006, pp. 34-51, cited by (since 1996)132.
65. Su Y., Shan S., Chen X., Gao W. Classifiability-based discriminatory projection pursuit. IEEE Trans. Neural Netw. 2011, 22(12):2050-2061.
66. Pomeroy S.L., Tamayo P., Gaasenbeek M., Sturla L.M., Angelo M., Mclaughlin M.E., Kim J.Y.H., Goumnerova L.C., Black P.M., Lau C., Allen J.C., Zagzag D., Olson J.M., Curran T., Wetmore C., Biegel J.A., Poggio T., Mukherjee S., Rifkin R., Califano A., Stolovitzky G., Louis D.N., Mesirov J.P., Lander E.S., Golub T.R. Prediction of central nervous system embryonal tumour outcome based on gene expression. Nature 2002, 415(6870):436-442.
67. Nutt C.L., Mani D.R., Betensky R.A., Tamayo P., Cairncross J.G., Ladd C., Pohl U., Hartmann C., Mclaughlin M. Gene expression-based classification of malignant gliomas correlates better with survival than histological classification. Cancer Res. 2003, 63:1602-1607.
68. Alon U., Barkai N., Notterman D.A., Gishdagger K., Ybarradagger S., Mackdagger D., Levine A.J. Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays. Proc. Natl. Acad. Sci. USA 1999, 96(12):6745-6750.
69. Shipp M.A., Ross K.N., Tamayo P., Weng A.P., Kutok J.L., Aguiar R.C., Gaasenbeek M., Angelo M., Reich M., Pinkus G.S., Ray T.S., Koval M.A., Last K.W., Norton A., Lister T.A., Mesirov J., Neuberg D.S., Lander E.S., Aster J.C., Golub T.R. Diffuse large B-cell lymphoma outcome prediction by gene-expression profiling and supervised machine learning. Nat. Med. 2002, 8(1):68-74.
70. Armstrong S.A., Staunton J.E., Silverman L.B., Pieters R., den Boer M.L., Minden M.D., Sallan S.E., Lander E.S., Golub T.R., Korsmeyer S.J. MLL translocations specify a distinct gene expression profile that distinguishes a unique leukemia. Nat. Genet. 2002, 30:41-47.
71. Singh D., Febbo P.G., Ross K., Jackson D.G., Manola J., Ladd C., Tamayo P., Renshaw A.A., D'Amico A.V., Richie J.P., Lander E.S., Loda M., Kantoff P.W., Golub T.R., Sellers W.R. Gene expression correlates of clinical prostate cancer behavior. Cancer Cell 2002, 1(2):203-209.
72. Khan J., Wei J.S., Ringner M., Saal L.H., Ladanyi M., Westermann F., Berthold F., Schwab M., Antonescu C.R., Peterson C., Meltzer P.S. Classification and diagnostic prediction of cancers using gene expression profiling and artificial neural networks. Nat. Med. 2001, 7(6):673-679.
73. Berry M.P.R., Graham C.M., McNab F.W., Xu Z., Bloch S.A.A., Oni T., Wilkinson K.A., Banchereau R., Skinner J., Wilkinson R.J., Quinn C., Blankenship D., Dhawan R., Cush J.J., Mejias A., Ramilo O., Kon O.M., Pascual V., Banchereau J., Chaussabel D., O'Garra A. An interferon-inducible neutrophil-driven blood transcriptional signature in human tuberculosis. Nature 2010, 466(7309):973-977.
74. Dudoit S., Fridlyand J., Speed T.P. Comparison of discrimination methods for the classification of tumors using gene expression data. J. Am. Stat. Assoc. 2002, 97(457):77-87.
75. Bishop C.M. Pattern Recognition and Machine Learning (Information Science and Statistics) 2006, Springer-Verlag, New York, Inc.
76. MATLAB, version 7.10.0 (R2010a), The MathWorks Inc., 2010.
77. Roweis S.T., Saul L.K. Nonlinear dimensionality reduction by locally linear embedding. Science 2000, 290:2323-2326.
78. Wu H.-M. Kernel sliced inverse regression with applications to classification. J. Comput. Graph. Stat. 2008, 17(3):590-610.
79. Zhou N., Wang L. A modified T-test feature selection method and its application on the hapmap genotype data. Genomics Proteomics Bioinform. 2007, 5(3-4):242-249.
80. Kononenko I. Estimating attributes: analysis and extensions of RELIEF. European Conference on Machine Learning 1994, 171-182. Springer-Verlag New York, Inc., Secaucus, NJ, USA. F. Bergadano, L.D. Raedt (Eds.).
81. L. van der Maaten, E.O. Postma, H.J. van den Herik, Dimensionality Reduction: A Comparative Review, Technical Report, Tilburg University, 2009.
82. I.H. Witten, E. Frank, L. Trigg, M. Hall, G. Holmes, S.J. Cunningham, Weka: Practical Machine Learning Tools and Techniques with Java Implementations, 1999. http://www.cs.waikato.ac.nz/ml/weka/.
83. Brazdil P.B., Soares C., Da Costa J.P. Ranking learning algorithms. using IBL and meta-learning on accuracy and time results. Mach. Learn. 2003, 50(3):251-277.