An adaptable method using human mixed tissue ratiometric controls for benchmarking performance on gene expression microarrays in clinical laboratories

BMC Biotechnology

Volumn 11, Issue , 2011, Pages

  Pine, P Scott ^a,b   Rosenzweig, Barry A ^a   Thompson, Karol L ^a  

^a CENTER FOR DRUG EVALUATION AND RESEARCH   (United States)

^b NATIONAL INSTITUTE OF STANDARDS AND TECHNOLOGY   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

ALTERNATIVE METHODS; AREA UNDER THE CURVES; CHANGE DETECTION; CLINICAL UTILITY; COMMERCIAL SOURCES; COMPARATIVE MEASUREMENTS; DATA SETS; DIAGNOSTIC PERFORMANCE; DNA MICRO-ARRAY; DYNAMIC RANGE; EXPRESSION PROFILE; EXTERNAL CONTROL; GENE EXPRESSION DATA; GENE EXPRESSION MICROARRAY; GENOMIC DATA; HIGH DENSITY; HIGH QUALITY; HUMAN TISSUES; HYBRIDIZATION CONDITIONS; MOLECULAR BIOMARKER; PROCESS IMPROVEMENT; RATIOMETRIC; RECEIVER OPERATING CHARACTERISTICS; REFERENCE SET; STATISTICALLY SIGNIFICANT DIFFERENCE;

BENCHMARKING; BIOASSAY; BIOCHIPS; CLINICAL LABORATORIES; DIAGNOSIS; RNA; TISSUE;

GENE EXPRESSION;

AREA UNDER THE CURVE; ARTICLE; CLINICAL LABORATORY; GENE EXPRESSION; HUMAN; HUMAN TISSUE; HYBRIDIZATION; MICROARRAY ANALYSIS; QUALITY CONTROL; ANTIBODY SPECIFICITY; DIAGNOSIS, MEASUREMENT AND ANALYSIS; DNA MICROARRAY; GENE EXPRESSION PROFILING; GENE EXPRESSION REGULATION; LIVER; METABOLISM; METHODOLOGY; RECEIVER OPERATING CHARACTERISTIC; SKELETAL MUSCLE;

RNA;

AREA UNDER CURVE; BENCHMARKING; CLINICAL LABORATORY TECHNIQUES; GENE EXPRESSION PROFILING; GENE EXPRESSION REGULATION; HUMANS; LIVER; MUSCLE, SKELETAL; OLIGONUCLEOTIDE ARRAY SEQUENCE ANALYSIS; ORGAN SPECIFICITY; RNA; ROC CURVE;

EID: 79953875426     PISSN: None     EISSN: 14726750     Source Type: Journal    
DOI: 10.1186/1472-6750-11-38     Document Type: Article

References (26)

1. Shi L, Reid LH, Jones WD, Shippy R, Warrington JA, Baker SC, Collins PJ, de Longueville F, Kawasaki ES, Lee KY, Luo Y, Sun YA, Willey JC, Setterquist RA, Fischer GM, Tong W, Dragan YP, Dix DJ, Frueh FW, Goodsaid FM, Herman D, Jensen RV, Johnson CD, Lobenhofer EK, Puri RK, Scherf U, Thierry-Mieg J, Wang C, Wilson M, Wolber PK, Zhang L, Amur S, Bao W, Barbacioru CC, Lucas AB, Bertholet V, Boysen C, Bromley B, Brown D, Brunner A, Canales R, Cao XM, Cebula TA, Chen JJ, Cheng J, Chu TM, Chudin E, Corson J, Corton JC, Croner LJ, Davies C, Davison TS, Delenstarr G, Deng X, Dorris D, Eklund AC, Fan XH, Fang H, Fulmer-Smentek S, Fuscoe JC, Gallagher K, Ge W, Guo L, Guo X, Hager J, Haje PK, Han J, Han T, Harbottle HC, Harris SC, Hatchwell E, Hauser CA, Hester S, Hong H, Hurban P, Jackson SA, Ji H, Knight CR, Kuo WP, Leclerc JE, Levy S, Li QZ, Liu C, Liu Y, Lombardi MJ, Ma Y, Magnuson SR, Maqsodi B, McDaniel T, Mei N, Myklebost O, Ning B, Novoradovskaya N, Orr MS, Osborn TW, Papallo A, Patterson TA, Perkins RG, Peters EH, Peterson R, Philips KL, Pine PS, Pusztai L, Qian F, Ren H, Rosen M, Rosenzweig BA, Samaha RR, Schena M, Schroth GP, Shchegrova S, Smith DD, Staedtler F, Su Z, Sun H, Szallasi Z, Tezak Z, Thierry-Mieg D, Thompson KL, Tikhonova I, Turpaz Y, Vallanat B, Van C, Walker SJ, Wang SJ, Wang Y, Wolfinger R, Wong A, Wu J, Xiao C, Xie Q, Xu J, Yang W, Zhang L, Zhong S, Zong Y, Slikker W. The MicroArray Quality Control (MAQC) project shows inter- and intraplatform reproducibility of gene expression measurements. Nat Biotechnol 2006, 24(9):1151-1161. 10.1038/nbt1239, 16964229.
2. Shi L, Campbell G, Jones WD, Campagne F, Wen Z, Walker SJ, Su Z, Chu TM, Goodsaid FM, Pusztai L, Shaughnessy JD, Oberthuer A, Thomas RS, Paules RS, Fielden M, Barlogie B, Chen W, Du P, Fischer M, Furlanello C, Gallas BD, Ge X, Megherbi DB, Symmans WF, Wang MD, Zhang J, Bitter H, Brors B, Bushel PR, Bylesjo M, Chen M, Cheng J, Cheng J, Chou J, Davison TS, Delorenzi M, Deng Y, Devanarayan V, Dix DJ, Dopazo J, Dorff KC, Elloumi F, Fan J, Fan S, Fan X, Fang H, Gonzaludo N, Hess KR, Hong H, Huan J, Irizarry RA, Judson R, Juraeva D, Lababidi S, Lambert CG, Li L, Li Y, Li Z, Lin SM, Liu G, Lobenhofer EK, Luo J, Luo W, McCall MN, Nikolsky Y, Pennello GA, Perkins RG, Philip R, Popovici V, Price ND, Qian F, Scherer A, Shi T, Shi W, Sung J, Thierry-Mieg D, Thierry-Mieg J, Thodima V, Trygg J, Vishnuvajjala L, Wang SJ, Wu J, Wu Y, Xie Q, Yousef WA, Zhang L, Zhang X, Zhong S, Zhou Y, Zhu S, Arasappan D, Bao W, Lucas AB, Berthold F, Brennan RJ, Buness A, Catalano JG, Chang C, Chen R, Cheng Y, Cui J, Czika W, Demichelis F, Deng X, Dosymbekov D, Eils R, Feng Y, Fostel J, Fulmer-Smentek S, Fuscoe JC, Gatto L, Ge W, Goldstein DR, Guo L, Halbert DN, Han J, Harris SC, Hatzis C, Herman D, Huang J, Jensen RV, Jiang R, Johnson CD, Jurman G, Kahlert Y, Khuder SA, Kohl M, Li J, Li L, Li M, Li QZ, Li S, Li Z, Liu J, Liu Y, Liu Z, Meng L, Madera M, Martinez-Murillo F, Medina I, Meehan J, Miclaus K, Moffitt RA, Montaner D, Mukherjee P, Mulligan GJ, Neville P, Nikolskaya T, Ning B, Page GP, Parker J, Parry RM, Peng X, Peterson RL, Phan JH, Quanz B, Ren Y, Riccadonna S, Roter AH, Samuelson FW, Schumacher MM, Shambaugh JD, Shi Q, Shippy R, Si S, Smalter A, Sotiriou C, Soukup M, Staedtler F, Steiner G, Stokes TH, Sun Q, Tan PY, Tang R, Tezak Z, Thorn B, Tsyganova M, Turpaz Y, Vega SC, Visintainer R, von Frese J, Wang C, Wang E, Wang J, Wang W, Westermann F, Willey JC, Woods M, Wu S, Xiao N, Xu J, Xu L, Yang L, Zeng X, Zhang J, Zhang L, Zhang M, Zhao C, Puri RK, Scherf U, Tong W, Wolfinger RD, . MAQC Consortium The MicroArray Quality Control (MAQC)-II study of common practices for the development and validation of microarray-based predictive models. Nat Biotechnol 2010, 28(8):827-38. 10.1038/nbt.1665, 20676074, MAQC Consortium.
3. External RNA Controls Consortium Proposed methods for testing and selecting the ERCC external RNA controls. BMC Genomics 2005, 6:150. 10.1186/1471-2164-6-150, 1325234, 16266432, External RNA Controls Consortium.
4. Brazma A, Hingamp P, Quackenbush J, Sherlock G, Spellman P, Stoeckert C, Aach J, Ansorge W, Ball CA, Causton HC, Gaasterland T, Glenisson P, Holstege FCP, Kim IF, Markowitz V, Matese JC, Parkinson H, Robinson A, Sarkans U, Schulze-Kremer S, Stewart J, Taylor R, Vilo J, Vingron M. Minimum information about a microarray experiment (MIAME)--toward standards for microarray data. Nat Genet 2001, 29:365-71. 10.1038/ng1201-365, 11726920.
5. Enkemann SA. Standards affecting the consistency of gene expression arrays in clinical applications. Cancer Epidemiol Biomarkers Prev 2010, 19:1000-1003. 10.1158/1055-9965.EPI-10-0044, 2852479, 20332273.
6. Sharov V, Kwong KY, Frank B, Chen E, Hasseman J, Gaspard R, Yu Y, Yang I, Quackenbush J. The limits of log-ratios. BMC Biotechnol 2004, 4:3. 10.1186/1472-6750-4-3, 400743, 15113428.
7. Naef F, Socci ND, Magnasco M. A study of accuracy and precision in oligonucleotide arrays: extracting more signal at large concentrations. Bioinformatics 2003, 19(2):178-84. 10.1093/bioinformatics/19.2.178, 12538237.
8. Corvi R, Ahr HJ, Albertini S, Blakey DH, Clerici L, Coecke S, Douglas GR, Gribaldo L, Groten JP, Haase B, Hamernik K, Hartung T, Inoue T, Indans I, Maurici D, Orphanides G, Rembges D, Sansone SA, Snape JR, Toda E, Tong W, van Delft JH, Weis B, Schechtman LM. Meeting report: Validation of toxicogenomics-based test systems: ECVAM-ICCVAM/NICEATM considerations for regulatory use. Environ Health Perspect 2006, 114(3):420-9. 10.1289/ehp.8247, 1392237, 16507466.
9. Kerr KF. Extended analysis of benchmark datasets for Agilent two-color microarrays. BMC Bioinformatics 2007, 8:371. 10.1186/1471-2105-8-371, 2174956, 17915030.
10. Novoradovskaya N, Whitfield ML, Basehore LS, Novoradovsky A, Pesich R, Usary J, Karaca M, Wong WK, Aprelikova O, Fero M, Perou CM, Botstein D, Braman J. Universal Reference RNA as a standard for microarray experiments. BMC Genomics 2004, 5(1):20. 10.1186/1471-2164-5-20, 394318, 15113400.
11. Shippy R, Fulmer-Smentek S, Jensen RV, Jones WD, Wolber PK, Johnson CD, Pine PS, Boysen C, Guo X, Chudin E, Sun YA, Willey JC, Thierry-Mieg J, Thierry-Mieg D, Setterquist RA, Wilson M, Lucas AB, Novoradovskaya N, Papallo A, Turpaz Y, Baker SC, Warrington JA, Shi L, Herman D. Using RNA sample titrations to assess microarray platform performance and normalization techniques. Nat Biotechnol 2006, 24(9):1123-31. 10.1038/nbt1241, 16964226.
12. Wen Z, Wang C, Shi Q, Huang Y, Su Z, Hong H, Tong W, Shi L. Evaluation of gene expression data generated from expired Affymetrix GeneChip®microarrays using MAQC reference RNA samples. BMC Bioinformatics 2010, 11(Suppl 6):S10. 10.1186/1471-2105-11-S6-S10, 3026357, 20946593.
13. Klevebring D, Gry M, Lindberg J, Eidefors A, Lundeberg J. Automation of cDNA synthesis and labelling improves reproducibility. J Biomed Biotechnol 2009, 2009:396808. 10.1155/2009/396808, 2763127, 19841682.
14. Pine PS, Boedigheimer M, Rosenzweig BA, Turpaz Y, He YD, Delenstarr G, Ganter B, Jarnagin K, Jones WD, Reid LH, Thompson KL. Use of diagnostic accuracy as a metric for evaluating laboratory proficiency with microarray assays using mixed tissue RNA reference samples. Pharmacogenomics 2008, 9:1753-63. 10.2217/14622416.9.11.1753, 19018728.
15. Thompson KL, Rosenzweig BA, Pine PS, Retief J, Turpaz Y, Afshari CA, Hamadeh HK, Damore MA, Boedigheimer M, Blomme E, Ciurlionis R, Waring JF, Fuscoe JC, Paules R, Tucker CJ, Fare T, Coffey EM, He Y, Collins PJ, Jarnagin K, Fujimoto S, Ganter B, Kiser G, Kaysser-Kranich T, Sina J, Sistare FD. Use of a mixed tissue RNA design for performance assessments on multiple microarray formats. Nucleic Acids Res 2005, 33:e187. 10.1093/nar/gni186, 1322274, 16377776.
16. Son CG, Bilke S, Davis S, Greer BT, Wei JS, Whiteford CC, Chen QR, Cenacchi N, Khan J. Database of mRNA gene expression profiles of multiple human organs. Genome Res 2005, 15(3):443-50. 10.1101/gr.3124505, 551571,551571, 15741514.
17. Su AI, Cooke MP, Ching KA, Hakak Y, Walker JR, Wiltshire T, Orth AP, Vega RG, Sapinoso LM, Moqrich A, Patapoutian A, Hampton GM, Schultz PG, Hogenesch JB. Large-scale analysis of the human and mouse transcriptomes. Proc Natl Acad Sci USA 2002, 99(7):4465-70. 10.1073/pnas.012025199, 123671, 11904358.
18. Thompson KL, Pine PS, Rosenzweig BA, Turpaz Y, Retief J. Characterization of the effect of sample quality on high density oligonucleotide microarray data using progressively degraded rat liver RNA. BMC Biotechnol 2007, 7:57. 10.1186/1472-6750-7-57, 2082023, 17854504.
19. Ma C, Lyons-Weiler M, Liang W, LaFramboise W, Gilbertson JR, Becich MJ, Monzon FA. In vitro transcription amplification and labeling methods contribute to the variability of gene expression profiling with DNA microarrays. J Mol Diagn 2006, 8(2):183-92. 10.2353/jmoldx.2006.050077, 1867595, 16645204.
20. Ach RA, Floore A, Curry B, Lazar V, Glas AM, Pover R, Tsalenko A, Ripoche H, Cardoso F, d'Assignies MS, Bruhn L, Van't Veer LJ. Robust interlaboratory reproducibility of a gene expression signature measurement consistent with the needs of a new generation of diagnostic tools. BMC Genomics 2007, 8:148. 10.1186/1471-2164-8-148, 1904205, 17553173.
21. Kurn N, Chen P, Heath JD, Kopf-Sill A, Stephens KM, Wang S. Novel isothermal, linear nucleic acid amplification systems for highly multiplexed applications. Clin Chem 2005, 51(10):1973-81. 10.1373/clinchem.2005.053694, 16123149.
22. Barker CS, Griffin C, Dolganov GM, Hanspers K, Yang JY, Erle DJ. Increased DNA microarray hybridization specificity using sscDNA targets. BMC Genomics 2005, 6(1):57. 10.1186/1471-2164-6-57, 1090574, 15847692.
23. Thompson KL, Pine PS. Comparison of the diagnostic performance of human whole genome microarrays using mixed-tissue RNA reference samples. Toxicol Lett 2009, 186:58-61. 10.1016/j.toxlet.2008.08.018, 18822358.
24. Slodkowska EA, Ross JS. MammaPrint™ 70-gene signature: another milestone in personalized medical care for breast cancer patients. Expert Rev Mol Diagn 2009, 9(5):417-22. 10.1586/erm.09.32, 19580427.
25. Schroeder A, Mueller O, Stocker S, Salowsky R, Leiber M, Gassmann M, Lightfoot S, Menzel W, Granzow M, Ragg T. The RIN: an RNA integrity number for assigning integrity values to RNA measurements. BMC Mol Biol 2006, 7:3. 10.1186/1471-2199-7-3, 1413964, 16448564.
26. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982, 143:29-36.