Reproducible and reliable microarray results through quality control: good laboratory proficiency and appropriate data analysis practices are essential

Current Opinion in Biotechnology

Volumn 19, Issue 1, 2008, Pages 10-18

  Shi, Leming ^a   Perkins, Roger G ^b   Fang, Hong ^b   Tong, Weida ^a  

^a NATIONAL CENTER FOR TOXICOLOGICAL RESEARCH   (United States)

^b ICF INTERNATIONAL   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

BIOMARKERS; DATA REDUCTION; DISEASES; GENE EXPRESSION; QUALITY CONTROL; TOXICITY;

GENE EXPRESSION PROFILING; PERSONALIZED IMPLEMENTATION;

MICROARRAYS;

DATA ANALYSIS; DEVELOPMENT; DNA MICROARRAY; FOOD AND DRUG ADMINISTRATION; MICROARRAY ANALYSIS; PREDICTION; PRIORITY JOURNAL; QUALITY CONTROL; RELIABILITY; REPRODUCIBILITY; REVIEW;

BIOTECHNOLOGY; DATA INTERPRETATION, STATISTICAL; GENE EXPRESSION PROFILING; MODELS, STATISTICAL; OLIGONUCLEOTIDE ARRAY SEQUENCE ANALYSIS; QUALITY CONTROL; REFERENCE STANDARDS; REPRODUCIBILITY OF RESULTS; RNA; UNITED STATES; UNITED STATES FOOD AND DRUG ADMINISTRATION;

EID: 39149111585     PISSN: 09581669     EISSN: None     Source Type: Journal    
DOI: 10.1016/j.copbio.2007.11.003     Document Type: Review

References (49)

1. Frueh F.W. Impact of microarray data quality on genomic data submissions to the FDA. Nat Biotechnol 24 (2006) 1105-1107. Description of the FDA's Voluntary Genomic Data Submission (VGDS) program.
2. Tong W, Fang H, Shi L, Perkins R, Goodsaid FM, Frueh FW: An integrated bioinformatics infrastructure essential for advancing pharmacogenomics and personalized medicine in the context of the FDA's Critical Path Initiative. Drug Discov Today 2007, in press.
3. Ratner M. FDA pharmacogenomics guidance sends clear message to industry. Nat Rev Drug Discov 4 (2005) 359
4. Tan P.K., Downey T.J., Spitznagel Jr. E.L., Xu P., Fu D., Dimitrov D.S., Lempicki R.A., Raaka B.M., and Cam M.C. Evaluation of gene expression measurements from commercial microarray platforms. Nucleic Acids Res 31 (2003) 5676-5684. A highly cited negative study on microarray crossplatform comparability.
5. ••], this article painted an excessively negative view on the reliability of microarrays in detecting differentially expressed genes.
6. Michiels S., Koscielny S., and Hill C. Prediction of cancer outcome with microarrays: a multiple random validation strategy. Lancet 365 (2005) 488-492. This paper is about questioning the reliability of microarrays in predicting patient outcomes.
7. Ioannidis J.P. Microarrays and molecular research: noise discovery?. Lancet 365 (2005) 454-455
8. Frantz S. An array of problems. Nat Rev Drug Discov 4 (2005) 362-363
9. Ein-Dor L., Zuk O., and Domany E. Thousands of samples are needed to generate a robust gene list for predicting outcome in cancer. Proc Natl Acad Sci U S A 103 (2006) 5923-5928. This paper is about questioning the reliability of microarrays in predicting patient outcomes.
10. Allison D.B., Cui X., Page G.P., and Sabripour M. Microarray data analysis: from disarray to consolidation and consensus. Nat Rev Genet 7 (2006) 55-65
11. Mecham B.H., Klus G.T., Strovel J., Augustus M., Byrne D., Bozso P., Wetmore D.Z., Mariani T.J., Kohane I.S., and Szallasi Z. Sequence-matched probes produce increased cross-platform consistency and more reproducible biological results in microarray-based gene expression measurements. Nucleic Acids Res 32 (2004) e74
12. Barczak A., Rodriguez M.W., Hanspers K., Koth L.L., Tai Y.C., Bolstad B.M., Speed T.P., and Erle D.J. Spotted long oligonucleotide arrays for human gene expression analysis. Genome Res 13 (2003) 1775-1785
13. Shippy R., Sendera T.J., Lockner R., Palaniappan C., Kaysser-Kranich T., Watts G., and Alsobrook J. Performance evaluation of commercial short-oligonucleotide microarrays and the impact of noise in making cross-platform correlations. BMC Genomics 5 (2004) 61
14. ••].
15. Hoffman E.P., et al. Expression profiling - best practices for data generation and interpretation in clinical trials. Nat Rev Genet 5 (2004) 229-237
16. Han T., Melvin C.D., Shi L., Branham W.S., Moland C.L., Pine P.S., Thompson K.L., and Fuscoe J.C. Improvement in the reproducibility and accuracy of DNA microarray quantification by optimizing hybridization conditions. BMC Bioinformat 7 Suppl 2 (2006) S17
17. Eisenstein M. Microarrays: quality control. Nature 442 (2006) 1067-1070
18. Shi L., Reid L.H., Jones W.D., Shippy R., Warrington J.A., Baker S.C., Collins J., Longueville Fd., Kawasaki E.S., Lee K.Y., et al. The MicroArray Quality Control (MAQC) project shows inter- and intraplatform reproducibility of gene expression measurements. Nat Biotechnol 24 (2006) 1151-1161. The main manuscript about the MicroArray Quality Control (MAQC) project, the most ambitious study so far on microarray quality control and data analysis issues.
19. Guo L., Lobenhofer E.K., Wang C., Shippy R., Harris S.C., Zhang L., Mei N., Chen T., Herman D., Goodsaid F.M., et al. Rat toxicogenomic study reveals analytical consistency across microarray platforms. Nat Biotechnol 24 (2006) 1162-1169. This paper is about validation of MAQC results with a toxicogenomic data set.
20. Canales R.D., Luo Y., Willey J.C., Austermiller B., Barbacioru C.C., Boysen C., Hunkapiller K., Jensen R.V., Knight C., Lee K.Y., et al. Evaluation of DNA microarray results with quantitative gene expression platforms. Nat Biotechnol 24 (2006) 1115-1122. The most comprehensive study comparing microarrays with other gene expression technologies.
21. Shippy R., Wolber P., Jones W.D., Jensen R.V., Fulmer-Smentek S., Johnson C.D., Pine P.S., Boysen C., Guo X., Chudin E., et al. Using RNA sample titrations to assess microarray platform performance and normalization techniques. Nat Biotechnol 24 (2006) 1123-1131
22. Tong W., Lucas A.B., Shippy R., Fan X., Fang H., Hong H., Orr M., Chu T.-M., Guo X., Collins J., et al. Evaluation of external RNA controls for the assessment of microarray performance. Nat Biotechnol 24 (2006) 1132-1139
23. Patterson T.A., Lobenhofer E.K., Fulmer-Smentek S.B., Collins J., Chu T.-M., Bao W., Fang H., Kawasaki E.S., Hager J., Tikhonova I., et al. Performance comparison of one-color and two-color platforms within the MicroArray Quality Control (MAQC) project. Nat Biotechnol 24 (2006) 1140-1150
24. Yauk C.L., Berndt M.L., Williams A., and Douglas G.R. Comprehensive comparison of six microarray technologies. Nucleic Acids Res 32 (2004) e124
25. Larkin J.E., Frank B.C., Gavras H., Sultana R., and Quackenbush J. Independence and reproducibility across microarray platforms. Nat Methods 2 (2005) 337-344
26. Irizarry R.A., Warren D., Spencer F., Kim I.F., Biswal S., Frank B.C., Gabrielson E., Garcia J.G., Geoghegan J., Germino G., et al. Multiple-laboratory comparison of microarray platforms. Nat Methods 2 (2005) 345-350
27. Bammler T., Beyer R.P., Bhattacharya S., Boorman G.A., Boyles A., Bradford B.U., Bumgarner R.E., Bushel P.R., Chaturvedi K., Choi D., et al. Standardizing global gene expression analysis between laboratories and across platforms. Nat Methods 2 (2005) 351-356
28. Petersen D., Chandramouli G.V., Geoghegan J., Hilburn J., Paarlberg J., Kim C.H., Munroe D., Gangi L., Han J., Puri R., et al. Three microarray platforms: an analysis of their concordance in profiling gene expression. BMC Genomics 6 (2005) 63
29. Kuo W.P., Liu F., Trimarchi J., Punzo C., Lombardi M., Sarang J., Whipple M.E., Maysuria M., Serikawa K., Lee S.Y., et al. A sequence-oriented comparison of gene expression measurements across different hybridization-based technologies. Nat Biotechnol 24 (2006) 832-840
30. Shi L., Tong W., Goodsaid F., Frueh F.W., Fang H., Han T., Fuscoe J.C., and Casciano D.A. QA/QC: challenges and pitfalls facing the microarray community and regulatory agencies. Expert Rev Mol Diagn 4 (2004) 761-777. This paper is the illustration of many pitfalls in microarray gene expression analysis.
31. Reid L.H. The value of a proficiency testing program to monitor performance in microarray laboratories. Pharmaceut Discov 5 (2005) 20-25
32. Baker S.C., Bauer S.R., Beyer R.P., Brenton J.D., Bromley B., Burrill J., Causton H., Conley M.P., Elespuru R., Fero M., et al. The External RNA Controls Consortium: a progress report. Nat Methods 2 (2005) 731-734
33. Tusher V.G., Tibshirani R., and Chu G. Significance analysis of microarrays applied to the ionizing radiation response. Proc Natl Acad Sci U S A 98 (2001) 5116-5121
34. Lin G., He X., Ji H., Shi L., Davis R.W., and Zhong S. Reproducibility Probability Score-incorporating measurement variability across laboratories for gene selection. Nat Biotechnol 24 (2006) 1476-1477
35. Strauss E. Arrays of hope. Cell 127 (2006) 657-659
36. Couzin J. Genomics. Microarray data reproduced, but some concerns remain. Science 313 (2006) 1559
37. Kiermer V. Microarray quality in the spotlight again. Nat Methods 3 (2006) 772
38. Sage L. Do microarrays measure up?. Anal Chem 78 (2006) 7358-7360
39. Klebanov L., Qiu X., Welle S., and Yakovlev A. Statistical methods and microarray data. Nat Biotechnol 25 (2007) 25-26
40. Shi L., Jones W.D., Jensen R.V., Wolfinger R.D., Kawasaki E.S., Herman D., Guo L., Goodsaid F.M., and Tong W. Reply to statistical methods and microarray data. Nat Biotechnol 25 (2007) 26-27. This paper is about explanations of the common misconceptions of statisticians in microarray data analysis.
41. Shi L, Jones W, Jensen RV, Harris SC, Perkins R, Goodsaid FM, Guo L, Croner LJ, Boysen C, Fang H et al.: The reproducibility of lists of differentially expressed genes in microarray studies. Nat Precedings 2007. http://precedings.nature.com/documents/306/version/2. This paper reports that the apparent lack of reproducibility of microarray results is largely because of the misuse or overuse of statistics in determining differentially expressed genes, instead of the microarray technology itself.
42. Shi L: New hot paper comment. ESI Special Topics 2007. http://www.esi-topics.com/nhp/2007/march-07-LemingShi.html.
43. Chen J.J., Wang S.J., Tsai C.A., and Lin C.J. Selection of differentially expressed genes in microarray data analysis. Pharmacogenomics J 7 (2007) 212-220
44. Chen J., Hsueh H.M., Delongchamp R., Lin C.J., and Tsai C.A. Reproducibility of microarray data: a further analysis of microarray quality control (MAQC) data. BMC Bioinformat 8 (2007) 412. This paper is an example of the misuse of statistics in microarray data analysis and the lack of understanding of the essential importance of data reproducibility in science.
45. Simon R. Development and evaluation of therapeutically relevant predictive classifiers using gene expression profiling. J Natl Cancer Inst 98 (2006) 1169-1171. This paper is about good advices on the development and validation of microarray-based predictive models.
46. Fuscoe J.C., Tong W., and Shi L. QA/QC issues to aid regulatory acceptance of microarray gene expression data. Environ Mol Mutagen 48 (2007) 349-353
47. Shi L, Goodsaid FM, Frueh FW, Tong W: Microarray technology: unresolved issues and future challenges from a regulatory perspective. In Standardization in Fluorometry: State-of-the-Art and Future Challenges, Edited by Resch-Genger U. Springer; 2008, in press.
48. Fan J., and Niu Y. Selection and validation of normalization methods for c-DNA microarrays using within-array replications. Bioinformatics 23 (2007) 2391-2398
49. Kerr K.F. Extended analysis of benchmark datasets for Agilent two-color microarrays. BMC Bioinformat 8 (2007) 371