Supervised Methods with Genomic Data: A Review and Cautionary View

Data Analysis and Visualization in Genomics and Proteomics

Volumn , Issue , 2005, Pages 193-214

  Díaz Uriarte, Ramón ^a  

^a Bioinformatics Unit   (Spain)

Author keywords

'molecular signatures'or'gene expression signatures'; Class comparison; Diagonal linear discriminant analysis (DLDA); Differential expression; K nearest neighbour (KNN); Machine learning; Multiple testing; ROC (receiver operating characteristic curve); Software 'magic bullets'

Indexed keywords

EID: 84889794570     PISSN: None     EISSN: None     Source Type: Book    
DOI: 10.1002/0470094419.ch12     Document Type: Chapter

References (101)

1. Al-Shahrour, F., Herrero, J., Mateos, Á., Santoyo, J., Díaz-Uriarte, R. and Dopazo, J. (2004) Using gene outology on genome-scale studies to find significant associations of biologically relevant terms to groups of genes. Neural Networks for Signal Processing XIII.
2. Alizadeh, A. A., Eisen, M. B., Davis, R. E., Ma, C., Lossos, I. S., Rosenwald, A., Boldrick, J. C., Sabet, H., Tran, T., Yu, X., Powell, J. I., Yang, L., Marti, G. E., Moore, T., Hudson, J. J., Lu, Lewis, D. B., Tibshirani, R., Sherlock, G., Chan, W. C., Greiner, I. C., Weisenburger, D. D., Armitage, J. O., Warnke, R. and Staudt, L. M., (2000) Distinct types of diffuse large b-cell lymphoma identified by gene expression profiling. Nature, 403, 503-511.
3. Ambroise, C. and McLachlan, G. J. (2002) Selection bias in gene extraction on the basis of microarray gene-expression data. Proc Natl Acad Sci USA, 99 (10), 6562-6566.
4. Baker, S. G., Kramer, B. S. and Srivastava, S. (2002) Markers for early detection of cancer: statistical guidelines for nested case-control studies. BMC Med Res Methodol, 2, 4.
5. Benjamini, Y. and Hochberg, Y. (1995) Controlling the false discovery rate: a practical and powerful approach to multiple testing. J R Stat Soc B, 57, 289-300.
6. Bickel, D. R. (2004) Degrees of differential gene expression: detecting biologically significant expression differences and estimating their magnitudes. Bioinformatics, 20, 682-688.
7. Boulesteix, A. and Tutz, G. (2004) Identification of interaction patterns and classification with applications to microarray data. SFB386 Discussion Paper, 369.
8. Boulesteix, A. L., Tutz, G. and Strimmer, K. (2003) A CART-based approach to discover emerging patterns in microarray data. Bioinformatics, 19, 2465-2472.
9. Braga-Neto, U. M. and Dougherty, E. R. (2004) Is cross-validation valid for small-sample microarray classification? Bioinformatics, 20, 374-380.
10. Breiman, L. (1996) Stacked regressions. Machine Learning, 24, 49-64.
11. Breiman, L. (2001a) Random forests. Machine Learning, 45, 5-32.
12. Breiman, L. (2001b) Statistical modeling: the two cultures (with discussion). Stat Sci, 16, 199-231.
13. Buckland, S. T., Burnham, K. P. and Augustin, N. H. (1997) Model selection: an integral part of inference. Biometrics, 53, 603-618.
14. Bureau, A., Dupuis, J., Hayward, B., Falls, K. and Van Eerdewegh, P. (2003) Mapping complex traits using Random Forests. BMC Genet, 4 Suppl 1, S64.
15. Burnham, K. P. and Anderson, D. R. (2002) Model Selection and Multimodel Inference: a Practical Information-Theoretic Approach, 2nd ed Springer, New York.
16. Cadima, J. F. C. L. and Jolliffe, I. T. (2001) Variable selection and the interpretation of principal subspaces. Journal of Agricultural, Biological, and Environmental Statistics, 6, 62-79.
17. Chang, C.-C. and Lin, C.-J. (2003) Libsvm: a Library for Support Vector Machines, technical report. Available from http://www.csie.ntu.edu.tw/~cjlin/libsvm/ [accessed 23/09/04].
18. Churchill, G. A. (2002) Fundamentals of experimental design for cDNA microarrays. Nat Genet, 32 (Suppl.), 490-495.
19. Collett, D. (2003) Modelling Binary Data, 2nd edn. Chapman and Hall, London.
20. Cui, X. and Churchill, G. A. (2003) Statistical tests for differential expression in cDNA microarray experiments. Genome Biol, 4, 210.
21. Davison, A. C. and Hinkley, D. V. (1997) Bootstrap Methods and their Application. Cambridge University Press, Cambridge.
22. Dettling, M. and B̈hlmann, P. (2004) Finding predictive gene groups from microarray data. J Multivariate Anal, 90, 106-131.
23. Díaz-Uriarte, R. (2004) Molecular Signatures from Gene Expression Data, technical report. Available from http://www.arxiv.org/abs/q-bio.QM/0401043 [accessed 23/09/04].
24. Díaz-Uriarte, R., Al-Shahrour, F. and Dopazo J. (2003) In Methods of Microarray Data Analysis III, papers from Camda '02. Kluwer, Dordrecht, The use of GO terms to understand the biological significance of microarray differential gene expression data. 233-247.
25. Draǎghici, S. (2002) Data Analysis for DNA Microarrays. Chapman and Hall, London.
26. Duda, R. O., Hart, P. E. and Stork, D. G. (2001) Pattern Classification, 2nd edn. Wiley, New York.
27. Dudoit, S., Fridlyand, J. and Speed, T. P. (2002) Comparison of discrimination methods for the classification of tumors using gene expression data. J Am Stat Assoc, 97 (457), 77-87.
28. Dudoit, S., Gentleman, R. C. and Quackenbush, J. (2003) Open source software for the analysis of microarray data. Biotechniques Suppl, 45-51.
29. Dudoit, S., Yang, Y. Callow, M. and Speed, T. (2002) Statistical methods for identifying differentially expressed genes in replicated cDNA experiments. Statistica Sinica, 12, 111-139.
30. Efron, B. and Gong, G. (1983) A leisurely look at the bootstrap, the jacknife, and cross-validation. Am Stat, 37 (1), 36-48.
31. Efron, B. and Tibshirani, R. J. (1993) An Introduction to the Bootstrap. Chapman and Hall, London.
32. Efron, B. and Tibshirani, R. J. (1997) Improvements on cross-validation: the 632{thorn} bootstrap method. J Am Stat Assoc, 92, 548-560.
33. Garrett, E. and Parmigiani, G. (2003) POE: statistical methods for qualitative analysis of gene expression. In The Analysis of Gene Expression Data: Methods and Software. Springer, New York, 362-387.
34. Garthwaite, P. H. (1994) An intepretation of partial least squares. JAm Stat Assoc, 89 (425), 122-127.
35. Ge, Y., Dudoit, S. and Speed, T. (2003) Resampling-based multiple testing for microarray data analysis (with discussion). TEST, 12, 1-77.
36. Ghosh, D. (2003) Penalized discriminant methods for the classification of tumors from gene expression data. Biometrics, 59, 992-1000.
37. Goeman, J. J., Van de Geer, S. A., de Kort, F. and van Houwelingen, H. C. (2004) A global test for groups of genes: testing association with a clinical outcome. Bioinformatics, 20, 93-99.
38. Golub, T. R., Slonim, D. K. Tamayo, P. Huard, C. Gaasenbeek, M. Mesirov, J. P. Coller, H. Loh, M. L. Downing, J. R. Caligiuri, M. A. Bloomfield, C. D. and Lander, E. S. (1999) Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. Science, 286, 531-537.
39. Gunther, E. C., Stone, D. J. Gerwien, R. W. Bento, P. and Heyes, M. P. (2003) Prediction of clinical drug efficacy by classification of drug-induced genomic expression profiles in vitro. Proc Natl Acad Sci USA, 100, 9608-9613.
40. Gusnanto, A., Pawitan, Y. and Ploner, A. (2003) Variable Selection in Gene and Protein Expression Data, technical report, Department of Medical Epidemiology and Biostatistics, Karolinska Institutet, Stockholm.
41. Guyon, I.,Wecton, J., Bainhill, S. and Vapnik, V. (2002) Gene selection for cancer classification using support vector machines. Machine Learning, 46, 389-422.
42. Harrell, J. F. E. (2001) Regression Modeling Strategies. Springer, New York.
43. Hastie, T., Tibshirani, R. and Friedman, J. (2001) The Elements of Statistical Learning. Springer, New York.
44. Hoeting, J. A., Madigan, D. Raftery, A. E. and Volinsky, C. T. (1999) Bayesian model averaging: a tutorial. Stat Sci, 14, 382-417.
45. Huang, X. and Pan, W. (2003) Linear regression and two-class classification with gene expression data. Bioinformatics, 19, 2072-2078.
46. Jolliffe, I. T. (2002) Principal Component Analysis, 2nd edn. Springer, New York.
47. Jörnsten, R. and Yu, B. (2003) Simultaneous gene clustering and subset selection for sample classification via MDL. Bioinformatics, 19, 1100-1109.
48. Kvzanowski, W. J. (1998) Principle of Multvariate Analysis. Oxford University Press, Oxford.
49. Lazzeroni, L. and Owen, A. (2002) Plaid models for gene expression data. Statistica Sinica,, 12, 61-86.
50. Lee, Y. and Lee, C.-K. (2003) Classification of multiple cancer types by multicategory support vector machines using gene expression data. Bioinformatics, 19, 1132-1139.
51. Liaw, A. and Wiener, M. (2002) Classification and regression by randomforest. Rnews, 2, 18-22.
52. Marshall, E. (2003) The upside of good behavior: make your data freely available. Science, 299, 990.
53. Mootha, V. K., Lindgren, C. M., Eriksson, K. F., Subramanian, A., Sihag, S., Lehar, J., Puigserver, P., Carlsson, E., Ridderstrale, M., Laurila, E., Houstis, N., Daly, M. J., Patterson, N., Mesirov, J. P., Golub, T. R., Tamayo, P., Spiegelman, B., Lander, E. S., Hirschhorn, J. N., Altshuler, D. and Groop, L. C. (2003) PGC-1alpha-responsive genes involved in oxidative phosphorylation are coordinately downregulated in human diabetes. Nat Genet, 34, 267-273.
54. Munagala, K., Tibshirani, R. and Brown, P. O. (2004) Cancer characterization and feature set extraction by discriminative margin clustering. BMC Bioinformatics, 5, 21.
55. Newton, M. A., Noueiry, A., Sarkar, D. and Ahlquist, P. (2004) Detecting differential gene expression with a semiparametric hierarchical mixture method. Biostatistics, 5, 155-176.
56. Nguyen, D. V. and Rocke, D. M. (2002) Multi-class cancer classification via partial least squares with gene expression profiles. Bioinformatics, 18 (9), 1216-1226.
57. Ntzani, E. E. and Ioannidis, J. P. (2003) Predictive ability of DNA microarrays for cancer outcomes and correlates: an empirical assessment. Lancet, 362, 1439-1444.
58. O'Neill, G. M., Catchpoole, D. R. and Golemis, E. A. (2003) From correlation to causality: microarrays, cancer, and cancer treatment. Biotechniques Suppl, 64-71.
59. Park, P. J., Tian, L. and Kohane, I. S. (2002) Linking gene expression data with patient survival times using partial least squares. Bioinformatics, 18 (S1), S120-S127.
60. Parmigiani, G., Garrett, E. Anbazhaghan, R. and Gabrielson, E. (2002) A statistical framework for expression-based molecular classification in cancer. J. R Stat Soc B, 64, 717-736.
61. Parmigiani, G., Garrett, E., Irizarry, R. and Sl. Z. (2003) The Analysis of gene expression data: an overview of methods and software. In The Analysis of Gene Expression Data: Methods and Software. Springer, New York, 1-45.
62. Patel, J. D., Bach, P. B. and Kris, M. G. (2004) Lung cancer in US women: a contemporary epidemic. JAMA, 291, 1763-1768.
63. Pawitan, Y., Bjöhle, J.,Wedren, S., Humphreys, K., Skoog, L., Huang, F., Amler, L., Shaw, P., Hall, P. and Bergh, J. (2004) Gene expression profiling for prognosis using Cox regression. Stat Med, 23 (11), 1767-1780.
64. Pepe, M. S. (2003) The Statistical Evaluation of Medical Tests for Classification and Prediction. Oxford University Press, Oxford.
65. Pepe, M. S., Etzioni, R., Feng, Z., Potter, J. D., Thompson, M. L., Thornquist, M., Winget, M. and Yasui, Y. (2001) Phases of biomarker development for early detection of cancer. J Natl Cancer Inst, 93, 1054-1061.
66. Pepe, M. S., Longton, G., Anderson, G. L. and Schummer, M. (2003) Selecting differentially expressed genes from microarray experiments. Biometrics, 59, 133-142.
67. Pittelkow, Y. E. and Wislon, S. R. (2003).Visualisation of gene expression data-the Ge-biplot, the chip-plot and the gene-plot. Stat Applications Genetics Mol Biol, 2, article 6.
68. Pomeroy, S., Tamayo, P., Gaasenbeek, M., Sturla, L., Angelo, M., McLaughlin, M., Kim, J., Goumnerova, L., Black, P., Lau, C., Allen, J., Zagzag, D., Olson, J., Curran, T., Wetmore, C., Biegel, J., Poggio, T., Mukherjee, S., Rifkin, R., Califano, A., Stolovitzky, G., Louis, D., Mesirov, J., Lander, E. and Golub, T. (2002) Prediction of central nervous system embryonal tumour outcome based on gene expression. Nature, 415, 436-442.
69. Potter, J. D. (2003) Epidemiology, cancer genetics and microarrays: making correct inferences, using appropriate designs. Trends Genet, 19, 690-695.
70. Ransohoff, D. F. (2004) Opinion: rules of evidence for cancer molecular-marker discovery and validation. Nat Rev Cancer, 4, 309-314.
71. Reiner, A., Yekutieli, D. and Benjamini, Y. (2003) Identifying differentially expressed genes using false discovery rate controlling procedures. Bioinformatics, 19, 368-375.
72. Ripley, B. D. (1996) Pattern Recognition and Neural Networks. Cambridge University Press, Cambridge.
73. Ripley, B. D. (2002) Statistical methods need software: a view of statistical computing, Opening lecture, RSS 2002. http://www.stats.ox.ac.uk/~ripley/RSS2002.pdf [accessed 23/09/04].
74. Romualdi, C., Campanaro, S., Campagna, D., Celegato, B., Cannata, N., Toppo, S., Valle, G. and Lanfranchi, G. (2003) Pattern recognition in gene expression profiling using DNA array: a comparative study of different statistical methods applied to cancer classification. Hum Mol Genet, 12, 8-823.
75. Rosenwald, A., Wright, G., Chan, W. C., Connors, J. M., Campo, E., Fisher, R. I., Gascoyne, R. D., Muller-Hermelink, H. K., Smeland, E. B., Giltnane, J. M., Hurt, E. M., Zhao, H., Averett, L., Yang, L.,Wilson,W. H., Jaffe, E. S., Simon, R., Klausner, R. D., Powell, J., Duffey, P. L., Longo, D. L., Greiner, T. C., Weisenburger, D. D., Sanger, W. G., Dave, B. J., Lynch, J. C., Vose, J., Armitage, J. O., Montserrat, E., Lopez-Guillermo, A., Grogan, T. M., Miller, T. P., LeBlanc, M.Ott, G., Kvaloy, S., Delabie, J., Holte, H., Krajci, P., Stokke, T., Staudt, L. M. and the Lymphoma/Leukemia Molecular Profiling Project (2002) The use of molecular profiling to predict survival after chemotherapy for diffuse large-b-cell lymphoma. N Engl J Med, 346 (25), 1937-1947.
76. Rosenwald, A., Wright, G., Leroy, K., Yu, X., Gaulard, P., Gascoyne, R. D., Chan, W. C., Zhao, T., Haioun, C., Greiner, T. C., Weisenburger, D. D., Lynch, J. C., Vose, J., Armitage, J. O., Smeland, E. B., Kvaloy, S., Holte, H., Delabie, J., Campo, E., Montserrat, E., Lopez-Guillermo, A., Ott, G., Muller-Hermelink, H. K., Connors, J. M., Braziel, R., Grogan, T. M., Fisher, R. I., Miller, T. P., LeBlanc, M., Chiorazzi, M., Zhao, H., Yang, L., Powell, J.Wilson,W. H., Jaffe, E. S., Simon, R., Klausner, R. D. and Staudt, L. M., (2003) Molecular diagnosis of primary mediastinal B cell lymphoma identifies a clinically favorable subgroup of diffuse large B cell lymphoma related to Hodgkin lymphoma. J Exp Med, 198, 851-862.
77. Sackett, D. L. and Haynes, R. B. (2002) The architecture of diagnostic research. BMJ, 324, 539-541.
78. Shaffer, A., Rosenwald, A., Hurt, E., Giltnane, J., Lam, L., Pickeral, O. and Staudt L. (2001) Signatures of the immune response. Immunity, 15, 375-385.
79. Shipp, M. A., Ross, K. N., Tamayo, P., Weng, A. P., Kutok, J. L., Aguiar, R. C. T., Gaasenbeek, M., Angelo, M., Reich, M., Pinkus, G. S., Ray, T. S., Koval, M. A., Last, K.W. Norton, A., Lister, T. A., Mesirov, J., Neuberg, D. S., Lander, E. S., Aster, J. C., and Golub, T. R. (2002) Diffuse large b-cell lymphoma outcome prediction by gene-expression profiling and supervised machine learning. Nature Med, 8 (1), 68-74.
80. Simon, R. M. and Dobbin, K. (2003) Experimental design of DNA microarray experiments. Biotechniques Suppl, 16-21.
81. Simon, R. M., Korn, E. L., McShane, L. M., Radmacher, M. D., Wright, G. W. and Zhao, Y. (2003a) Design and Analysis of DNA Microarray Investigations. Springer, New York.
82. Simon, R., Radmacher, M. D., Dobbin, K. and McShane, L. M. (2003b) Pitfalls in the use of DNA microarray data for diagnostic and prognostic classification. J Natl Cancer Inst, 95 (1), 14-18.
83. Slonim, D. K. (2002) From patterns to pathways: gene expression data analysis comes of age. Nat Genet, 32 Suppl., 502-508.
84. Smyth, G. K. (2004) Linear models and empirical Bayes methods for assessing differential expression in microarray experiments. Stat Applications Genetics Mol Biol, 3, article 3.
85. Somorjai, R. L., Dolenko, B. and Baumgartner, R. (2003) Class prediction and discovery using gene microarray and proteomics mass spectroscopy data: curses, caveats, cautions. Bioinformatics, 19, 1484-1491.
86. Somorjai, R. L., Dolenko, B., Nikulin, A., Nickerson, P., Rush, D., Shaw, A., Glogowski, M., Rendell, J. and Deslauriers, R. (2002) Distinguishing normal from rejecting renal allografts: application of a three-stage classification strategy to MR and IR spectra of urine. Vibrational Spectroscopy, 28, 97-102.
87. Speed, T. E. (2003) Statistical Analysis of Gene Expression Microarray Data. Chapman and Hall, London.
88. Stegmaier, K., Ross, K. N., Colavito, S. A., O'Malley, S., Stockwell, B. R. and Golub, T. R. (2004) Gene expression-based high-throughput screening(GE-HTS) and application to leukemia differentiation. Nat Genet, 36, 257-263.
89. Stone, M. and Brooks, R. J. (1990) Continuum regression: cross-validated sequentially constructed prediction embracing ordinary least squares, partial least squares and principal components regression (with discussion). J R Stat Soc B, 52 (2), 237-269.
90. Storey, J. (2002) A direct approach to false discovery rates. J R Stat Soc B, 64, 479-498.
91. Storey, J. D. and Tibshirani, R. (2003) Statistical significance for genomewide studies. Proc Natl Acad Sci USA, 100, 9440-9445.
92. Ting, K. M. andWitten, I. H. (1999) Issues in stacked generalization. J Artif Intell Res, 10, 271-289.
93. Tumor Analysis Best Practices Working Group (2004) Guidelines: expression profiling-best practices for data generation and interpretation in clinical trials. Nat Rev Genet, 5, 229-237.
94. Turner, H., Bailey, T. and Krzanowski, W. (2004) Improved biclustering of microarray data demonstrated through systematic performance tests. Comput Stat Data Anal, in press.
95. Van Belle, G. (2002) Statistical Rules of Thumb. Wiley, New York.
96. Wasserman, L. (2000) Bayesian model selection and model averaging. J Math Psychol, 44, 92-107.
97. Westfall, P. H. and Young S. S. (1993). Resampling-Based Multiple Testing. Examples and Methods for p-Value Adjustment. Wiley, New York.
98. Whitfield, C. W., Cziko, A. M. and Robinson, G. E. (2003) Gene expression profiles in the brain predict behavior in individual honey bees. Science, 302, 296-299.
99. Wolpert, D. H. (1992) Stacked generalization. Neural Networks, 5, 241-259.
100. Xu, R. and Li, X. (2003) A comparison of parametric versus permutation methods with applications to general and temporal microarray gene expression data. Bioinformatics, 19, 1284-1289.
101. Yang, Y. H. and Speed, T. (2002) Design issues for cDNA microarray experiments. Nat Rev Genet, 3, 579-588.