Resampling-based multiple testing for microarray data analysis

Test

Volumn 12, Issue 1, 2003, Pages 1-77

  Ge, Yongchao ^a   Dudoit, Sandrine ^a   Speed, Terence P ^a   Glonek, Gary ^b   Solomon, Patty ^b   Grant, Gregory R ^c   Kendziorski, Christina M ^d   Maindonald, John H ^e   Storey, John D ^a   Westfall, Peter H ^f  

^a UNIVERSITY OF CALIFORNIA   (United States)

^b UNIVERSITY OF ADELAIDE   (Australia)

^c UNIVERSITY OF PENNSYLVANIA   (United States)

^d UNIVERSITY OF WISCONSIN MADISON   (United States)

^e Biology and Environment   (Australia)

^f TEXAS TECH UNIVERSITY   (United States)

Author keywords

Adjusted p value; False discovery rate; Family wise error rate; Fast algorithm; Microarray; MinP; Multiple testing

Indexed keywords

EID: 0041530047     PISSN: 11330686     EISSN: None     Source Type: Journal    
DOI: 10.1007/BF02595811     Document Type: Review

References (60)

1. ALIZADEH, A. A., EISEN, M. B., DAVIS, R. E., MA, C., LOSSOS, I. S., ROSENWALD, A., BOLDRICK, J. C., SABET, H., TRAN, T., YU, X., POWELL, J. I., YANG, L., MARTI, G. E., MOORE, T., HUDSON JR, J., LU, L., LEWIS, D. B., TIBSHIRANI, R., SHERLOCK, G., CHAN, W. C., GREINER, T. C., WEISENBURGER, D. D., ARMITAGE, J. O., WARNKE, R., LEVY, R., WILSON, W., GREVER, M. R., BYRD, J. C., BOTSTEIN, D., BROWN, P. O., and STAUDT, L. M. (2000). Distinct types of diffuse large B-cell lymphoma identified by gene expression profiling. Nature, 403:503-511.
2. ALON, U., BARKAI, N., NOTTERMAN, D. A., GISH, K., YBARRA, S., MACK, D., and LEVINE, A. J. (1999). Broad patterns of gene expression revealed by clustering analysis of tumor and normal colon tissues probed by oligonucleotide arrays. Proceedings of the National Academy of Sciences, 96:6745-6750.
3. BENJAMINI, Y. and BRAUN, H. (2002). John W. Tukey's contriubtions to multiple comparisons. The Annals of Statistics, 30(6): 1576-1594.
4. BENJAMINI, Y. and HOCHBERG, Y. (1995). Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society, Series B, 57:289-300.
5. BENJAMINI, Y. and HOCHBERG, Y. (2000). The adaptive control of the false discovery rate in multiple hypotheses testing with independent statistics. Journal of Educational and Behavioral Statistics, 25(1):60-83.
6. BENJAMINI, Y. and YEKUTIELI, D. (2001). The control of the false discovery rate in multiple hypothesis testing under dependency. The Annals of Statistics, 29(4):1165-1188.
7. BERAN, R. (1988). Balanced simultaneous confidence sets. Journal of the American Statistical Association, 83(403):679-686.
8. BERRY, D. (1988). Multiple comparisons, multiple tests, and data dredging: A bayesian perspective. In J. Bernardo, M. DeGroot, D. Lindley, and A. Smith, eds., Bayesian Statistics, vol. 3. pp. 79-94. Oxford Univeristy Press.
9. BOLDRICK, J. C., ALIZADEH, A. A., DIEHN, M., DUDOIT, S., LIU, C. L., BELCHER, C. E., BOTSTEIN, D., STAUDT, L. M., BROWN, P. O., and RELMAN, D. A. (2002). Stereotyped and specific gene expression programs in human innate immune responses to bacteria. Proceedings of the National Academy of Sciences, 99(2):972-977.
10. BUCKLEY, M. J. (2000). The Spot user's guide. CSIRO Mathematical and Information Sciences. http://www.cmis.csiro.au/IAP/Spot/spotmanual.htm.
11. CALLOW, M. J., DUDOIT, S., GONG, E. L., SPEED, T. P., and RUBIN, E. M. (2000). Microarray expression profiling identifies genes with altered expression in HDL deficient mice. Genome Research, 10(12):2022-2029.
12. DERISI, J. L., IYER, V. R., and BROWN, P. O. (1997). Exploring the metabolic and genetic control of gene expression on a genomic scale. Science, 278:680-685.
13. DUDOIT, S., SHAFFER, J. P., and BOLDRICK, J. C. (2002a). Multiple hypothesis testing in microarray experiments. Submitted, available UC Berkeley, Division Biostatistics working paper series: 2002-110, http://www.bepress.com/ucbbiostat/paper110.
14. DUDOIT, S., YANG, Y. H., CALLOW, M. J., and SPEED, T. P. (2002b). Statistical methods for identifying differentially expressed genes in replicated cDNA microarray experiments. Statistica Sinica, 12(1):111-139.
15. DUNN. O. J. (1958). Estimation of the means of dependent variables. The Annals of Mathematical Statistics, 29:1095-1111.
16. EFRON, B. and TIBSHIRANI, R. (2002). Empirical Bayes methods and false discovery rates for microarrays. Genetic Epidemiology, 23:70-86.
17. EFRON, B., TIBSHIRANI, R., GOSS, V., and CHU, G. (2000). Microarrays and their use in a comparative experiment. Tech. Rep. 37B/213, Department of Statistics, Stanford University.
18. EFRON, B., TIBSHIRANI, R., STOREY, J. D., and TUSHER, V. (2001). Empirical Bayes analysis of a microarray experiment. Journal of the American Statistical Association, 96(456):1151-1160.
19. FINNER, H. and ROTERS, M. (2001). On the false discovery rate and expected type I errors. Biometrical Journal, 8:985-1005.
20. GENOVESE, C. and WASSERMAN, L. (2001). Operating characteristics and extensions of the FDR procedure. Journal of the Royal Statistical Society, Series B, 57:499-517.
21. GOLUB, T. R., SLONIM, D. K., TAMAYO, P., HUARD, C., GAASENBEEK, M., MESIROV, J. P., COLLER, H., LOH, M. L., DOWNING, J. R., CALIGIURI, M. A., BLOOMFIELD, C. D., and LANDER, E. S. (1999). Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. Science, 286:531-537.
22. HOLM, S. (1979). A simple sequentially rejective multiple test procedure. Scandinavian Journal of Statistics, 6:65-70.
23. IHAKA. R. and GENTLEMAN, R. (1996). R: A language for data analysis and graphics. Journal of Computational and Graphical Statistics, 5(3):299-314.
24. JOGDEO, K. (1977). Association and probability inequalities. Annals of Statistics, 5(3):495-504.
25. KENDZIORSKI, C., NEWTON, M., LAN, H., and GOULD, M. (2003). On parametric empirical bayes methods for comparing multiple groups using replicated gene expression profiles. In press.
26. KERR, M. K., MARTIN, M., and CHURCHILL, G. A. (2000). Analysis of variance for gene expression microarray data. Journal of Computational Biology, 7(6):819-837.
27. KORN, E. L., TROENDLE, J. F., McSHANE, L. M., and SIMON, R. (2001). Controlling the number of false discoveries: Application to high dimensional genomic data. Tech. Rep. 003, National Cancer Institute, Division of Cancer Treatment and Diagnosis. http://linus.nci.nih.gov/̃brb/TechReport.htm.
28. LEHMANN, E. L. (1986). Testing Statistical Hypotheses. Springer Verlag, New York, 2nd ed.
29. LOCKHART, D. J., DONG, H. L., BYRNE, M. C., FOLLETTIE, M. T., GALLO, M. V., CHEE, M. S., MITTMANN, M., WANG, C., KOBAYASHI, M., HORTON, H., and BROWN, E. L. (1996). Expression monitoring by hybridization to high-density oligonucleotide arrays. Nature Biotechnology, 14:1675-1680.
30. MANDUCHI, E., GRANT. G. R., McKENZIE, S. E., OVERTON, G. C., SURREY, S., and STOECKERT JR, C. J. (2000). Generation of patterns from gene expression data by assigning confidence to differentially expressed genes. Bioinformatics, 16:685-698.
31. MARCUS, R., PERITZ, E., and GABRIEL, K. R. (1976). On closed testing procedures with special reference to ordered analysis of variance. Biometrics, 63:655-660.
32. MORTON, N. E. (1955). Sequential the tests for detection of linkage. American Journal of Human Genetics, 7:277-318.
33. MÜLLER, P., PARMIGIANI, G., ROBERT, C., and ROUSSEAU, J. (2003). Optimal sample size for multiple testing: the case of gene expression microarrays. technical report, department of biostatistics. Tech. rep., The University of Texas M.D. Anderson Cancer Center.
34. PEROU, C. M., JEFFREY, S. S., VAN DE RIJN, M., REES, C. A., EISEN, M. B., ROSS, D. T., PERGAMENSCHIKOV, A., WILLIAMS, C. F., ZHU, S. X., LEE, J. C. F., LASHKARI, D., SHALON, D., BROWN, P. O., and BOTSTEIN, D. (1999). Distinctive gene expression patterns in human mammary epithelial cells and breast cancers. Proceedings of the National Academy of Sciences, 96:9212-9217.
35. PESARIN, F. (2001). Multivariate permutation tests with applications in biostatistics. John Wiley and Sons, Chichester.
36. POLLACK, J. R., PEROU, C. M., ALIZADEH, A. A., EISEN, M. B., PERGAMENSCHIKOV, A., WILLIAMS, C. F., JEFFREY, S. S., BOTSTEIN, D., and BROWN, P. O. (1999). Genome-wide analysis of DNA copy-number changes using cDNA microarrays. Nature Genetics, 23:41-46.
37. POLLARD, K. and VAN DER LAAN, M. (2002). Resampling-based methods for identification of significant subsets of genes in expression data. U. C. Berkeley Division of Biostatistics Working Paper Series. Working Paper 121, http://www.bepress.com/ucbbiostat.
38. POLLARD, K. and VAN DER LAAN, M. (2003). Parametric and nonparametric methods to identify significantly differentially expressed genes. Manuscript.
39. PURI, M. and SEN, P. (1971). Nonparametric Methods in Multivariate Analysis. Wiley, New York.
40. ROSS, D. T., SCHERF, U., EISEN, M. B., PEROU, C. M., REES, C., SPELLMAN, P., IYER, V., JEFFREY, S. S., VAN DE RIJN, M., WALTHAM, M., PERGAMENSCHIKOV, A., LEE, J. C. F., LASHKARI, D., SHALON, D., MYERS, T. G., WEINSTEIN, J. N., BOTSTEIN, D., and BROWN, P. O. (2000). Systematic variation in gene expression patterns in human cancer cell lines. Nature Genetics, 24:227-234.
41. SEEGER, P. (1968). A note on a method for the analysis of significance en masse. Technometrics, 10(3):586-593.
42. SHAFFER, J. P. (1995). Multiple hypothesis testing. Annu. Rev. Psychol., 46:561-584.
43. ŠIDÁK, Z. (1967). Rectangular confidence regions for the means of multivariate normal distributions. Journal of the American Statistical Association, 62:626-633.
44. SIMES, R. J. (1986). An improved Bonferroni procedure for multiple tests of significance. Biometrika, 73(3):751-754.
45. SORIĆ, B. (1989). Statistical "discoveries" and effect-size estimation. Journal of the American Statistical Association, 84(406):608-610.
46. STOREY, J. D. (2001). The positive false discovery rate: A Bayesian interpretation and the q-value. Annals of Statistics. In press.
47. STOREY, J. D. (2002a). A direct approach to false discovery rates. Journal of the Royal Statistical Society, Series B, 64:479-498.
48. STOREY, J. D. (2002b). False Discovery Rates: Theory and Applications to DNA Microarrays. Ph.D. thesis, Department of Statistics, Stanford University.
49. STOREY, J. D., TAYLOR, J. E., and SIEGMUND, D. (2002). Strong control, conservative point estimation, and simultaneous conservative consistency of false discovery rates: A unified approach. Journal of the Royal Statistical Society, Series B. In press.
50. STOREY, J. D. and TIBSHIRANI, R. (2001). Estimating false discovery rates under dependence, with applications to DNA microarrays. Tech. Rep. 2001-28, Department of Statistics, Stanford University.
51. TUSHER, V. G., TIBSHIRANI, R., and CHU, G. (2001). Significance analysis of microarrays applied to ionizing radiation response. Proceedings of the National Academy of Sciences, 98:5116-5121.
52. WELCH, B. L. (1938). The significance of the difference between two means when the population variances are unequal. Biometrika, 29:350-362.
53. WESTFALL, P., KRISHEN, A., and YOUNG, S. (1998). Using prior information to allocate significance levels for multiple endpoints. Statistics in Medicine, 17:2107-2119.
54. WESTFALL, P., KROPF, S., and FINOS, L. (2003). Weighted fwe-controlling methods in high-dimensional situations. Manuscript.
55. WESTFALL, P., LIN, Y., and YOUNG, S. (1989). A procedure for the analysis of multivariate binomial data with adjustments for multiplicity. In Proceedings of the 14th Annual SAS® User's Group International Conference, pp. 1385-1392.
56. WESTFALL, P. and SUPER, K. (2001). Using priors to improve multiple animal carcinogenicity tests. Journal of the American Statistical Association, 96:827-834.
57. WESTFALL, P. and WOLFINGER, R. (1997). Multiple tests with discrete distributions. The American Statistician, 51:3-8.
58. WESTFALL, P. H. and YOUNG, S. S. (1993). Resampling-based multiple testing: Examples and methods for p-value adjustment. John Wiley & Sons, New York.
59. WESTFALL, P. H., ZAYKIN, D. V., and YOUNG, S. S. (2001). Multiple tests for genetic effects in association studies. In S. Looney, ed., Methods in Molecular Biology, vol. 184: Biostatistical Methods, pp. 143-168. Humana Press, Toloway, NJ.
60. YEKUTIELI, D. and BENJAMINI, Y. (1999). Resampling-based false discovery rate controlling multiple test procedures for correlated test statistics. Journal of Statistical Planning and Inference, 82:171-196.