An exploratory test for an excess of significant findings

Clinical Trials

Volumn 4, Issue 3, 2007, Pages 245-253

  Ioannidis, John P A ^a,b   Trikalinos, Thomas A ^a,b  

^a UNIVERSITY OF IOANNINA   (Greece)

^b TUFTS UNIVERSITY SCHOOL OF MEDICINE   (United States)

Author keywords

[No Author keywords available]

Indexed keywords

CHLORPROMAZINE; CLOZAPINE; HALOPERIDOL; LOXAPINE; NEUROLEPTIC AGENT; PIMOZIDE; PLACEBO; RISPERIDONE; SULPIRIDE; THIORIDAZINE;

ANALYTICAL RESEARCH; ARTICLE; CLINICAL RESEARCH; CLINICAL TRIAL; DRUG EFFECT; DRUG EFFICACY; EXPLORATORY RESEARCH; HUMAN; MEDICAL LITERATURE; OUTCOME ASSESSMENT; PRIORITY JOURNAL; PUBLICATION; SCHIZOPHRENIA; STATISTICAL SIGNIFICANCE;

ANTIPSYCHOTIC AGENTS; BIAS (EPIDEMIOLOGY); CLINICAL TRIALS AS TOPIC; DATA INTERPRETATION, STATISTICAL; HUMANS; META-ANALYSIS AS TOPIC; PROBABILITY; SCHIZOPHRENIA;

EID: 34548642946     PISSN: 17407745     EISSN: 17407753     Source Type: Journal    
DOI: 10.1177/1740774507079441     Document Type: Article

References (53)

1. Neslon N, Rosenthal R, Rosnow RL. Interpretation of significance levels by psychological researchers. Am Psychol 1986; 41: 1299-301.
2. Cohen J. The earth is round (p-less-than .05). Am Psychol 1994; 49: 997-1003.
3. Rosenthal R, Gaito J. The interpretation of levels of significance by psychological researchers. J Psychol 1963; 55: 33-8.
4. Rosenthal R, Gaito J. Further evidence for the cliff effect in the interpretation of levels of significance. Psychol Rep 1964; 15: 570.
5. Ioannidis JP. Why most published research findings are false. PLoS Med 2005; 2: e124.
6. Dickersin K, Min YI. Publication bias: the problem that won't go away. Ann N Y Acad Sci 1993; 703: 135-146
7. Dickersin K, Min YI, Meinert CL. Factors influencing publication of research results. Follow-up of applications submitted to two institutional review boards. JAMA 1992; 267: 374-378
8. Dickersin K. The existence of publication bias and risk factors for its occurrence. JAMA 1990; 263: 1385-1389
9. Chalmers I. Underreporting research is scientific misconduct. JAMA 1990; 263: 1405-08.
10. Easterbrook PJ, Berlin JA, Gopalan R, Matthews DR. Publication bias in clinical research. Lancet 1991; 337: 867-72.
11. Ioannidis JP. Effect of the statistical significance of results on the time to completion and publication of randomized efficacy trials. JAMA 1998; 279: 281-6.
12. Light RJ, Pillemer DB. Summing up. The science of reviewing research, London: Harvard University Press, 1984.
13. Begg CB, Mazumdar M. Operating characteristics of a rank correlation test for publication bias. Biometrics 1994; 50: 1088-101.
14. Egger M, Davey Smith G, Schneider M, Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ 1997; 315: 629-34.
15. Sterne JA, Egger M. Funnel plots for detecting bias in meta-analysis: guidelines on choice of axis. J Clin Epidemiol 2001; 54: 1046-55.
16. Sterne JA, Gavaghan D, Egger M. Publication and related bias in meta-analysis: power of statistical tests and prevalence in the literature. J Clin Epidemiol 2000; 53: 1119-29.
17. Hedges LV. Modeling publication selection effects in meta-analysis. Stat Sci 1992; 7: 246-55.
18. Copas JB. What works?: Selectivity models and meta-analysis. J Royal Stat Assoc 1999; 162: 95-109.
19. Hedges LV, Vevea JL. Estimating effect size under publication bias: Small sample properties and robustness of a random effects selection model. J Educ Behav Stat 1996; 21: 299-332.
20. Vevea JL, Hedges LV. A general linear model for estimating effect size in the presence of publication bias. Psychometrika 1995; 60: 419-35.
21. Iyengar S, Zhao PL. Maximum likelihood estimation for weighted distributions. Stat Probab Lett 1994; 50: 438-60.
22. Ioannidis JP. Differentiating biases from genuine heterogeneity: distinguishing artifactual from substantive effects. In Rothstein HR, Sutton AJ, Borenstein M eds. Publication bias in meta-analysis - Prevention, assessment and adjustments, Sussex: John Wiley, 2005.
23. Chan AW, Hrobjartsson A, Haahr MT, Gotzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA 2004; 291: 2457-65.
24. Chan AW, Krleza-Jeric K, Schmid I, Altman DG. Outcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research. CMAJ 2004; 171: 735-40.
25. Kyzas PA, Loizou KT, Ioannidis JP. Selective reporting biases in cancer prognostic factor studies. J Natl Cancer Inst 2005; 97: 1043-55.
26. Michels KB, Rosner BA. Data trawling: to fish or not to fish. Lancet 1996; 348: 1152-53.
27. Weiss RB, Gill GG, Hudis CA. An on-site audit of the South African trial of high-dose chemotherapy for metastatic breast cancer and associated publications. J Clin Oncol 2001; 19: 2771-77.
28. Ransam J, Buyse M, George SL, et al. Fraud in medical research: an international survey of biostatisticians. Control Clin Trials 2000; 21: 415-27.
29. Gardner W, Lidz CW, Hartwig KC. Authors' reports about research integrity in clinical trials. Contemp Clin Trials 2005 26: 244-51.
30. Al-Marzouki S, Evans S, Marshall T, Robert I. Are these data real? Statistical methods for the detection of data fabrication in clinical trials. BMJ 2005; 331: 267-70.
31. Lau J, Ioannidis JPA, Schmid CH. Quantitative synthesis in systematic reviews. Ann Intern Med 1997; 127: 820-26.
32. Engels EA, Schmid CH, Terrin N, Olkin I, Lau J. Heterogeneity and statistical significance in meta-analysis: an empirical study of 125 meta-analyses. Stat Med 2000; 19: 1707-28.
33. Terrin N, Schmid CH, Lau J, Olkin I. Adjusting for publication bias in the presence of heterogeneity. Stat Med 2003; 22: 2113-26.
34. Lau J, Ioannidis JPA, Schmid CH. Summing up evidence: one answer is not always enough. Lancet 1998; 351: 123-7.
35. Preston C, Ashby D, Smith R. Adjusting for publication bias: modelling the selection process. J Eval Clin Pract 2004; 10: 313-22.
36. Spiegelhalter DJ, Myles JP, Jones DR, Abrams KR. Bayesian methods in health technology assessment: a review. Health Technol Assess 2000; 4: 1-130.
37. Sutton AJ, Abrams KR, Jones DR. Generalized synthesis of evidence and the threat of dissemination bias: the example of electronic fetal heart rate monitoring (EFM). J Clin Epidemiol 2002; 55: 1013-24.
38. Trikalinos TA, Churchill R, Ferri M, et al. Effect sizes in cumulative meta-analyses of mental health randomized trials evolved over time. J Clin Epidemiol 2004 57: 1124-30.
39. Ioannidis JPA, Lau J. Evolution of treatment effects over time: empirical evidence from recursive cumulative meta-analyses. Proc Natl Acad Sci USA 2001; 98: 831-6.
40. Counsell C. Formulating questions and locating primary studies for inclusion in systematic reviews. Ann Intern Med 1997; 127: 380-7.
41. Wacholder S, Chanock S, Garcia-Closas M, El Ghormli L, Rothman N. Assessing the probability that a positive report is false: an approach for molecular epidemiology studies. J Natl Cancer Inst 2004; 96: 434-42.
42. Efron B, Tibshirani R. Empirical Bayes methods and false discovery rates for microarrays. Genet Epidemiol 2002; 23: 70-86.
43. Spiegelhalter DJ, Abrams KR, Myles JP. Bayesian approaches to clinical trials and health-care evaluation, Chichester: John Wiley & Sons, 2004.
44. Lilford RJ, Braunholtz D. The statistical basis of public policy. BMJ 1996; 313: 603-7.
45. Carlin BP, Louis TA. Bayes and empirical Bayes methods for data analysis, Boca Raton: Chapman and Hall/CRC, 2000.
46. Greenland S. Power, sample size and smallest detectable effect determination for multivariate studies. Stat Med 1985; 4: 117-27.
47. Lui KJ. Sample size determination for case-control studies: the influence of the joint distribution of exposure and confounder. Stat Med 1990; 9: 1485-93.
48. Lubin JH, Gail MH. On power and sample size for studying features of the relative odds of disease. Am J Epidemiol 1990; 131: 552-66.
49. Pan Z, Trikalinos TA, Kavvoura FK, Lau J, Ioannidis JP. Local literature bias in genetic epidemiology: an empirical evaluation of the Chinese literature. PLoS Med 2005; 2: e334.
50. Oxman AD, Guyatt GH. A consumer's guide to subgroup analyses. Ann Intern Med 1992; 116: 78-84.
51. Ioannidis JPA, Rosenberg PS, Goedert JJ, O'Brien TR. Meta-analysis of individual participants' data in genetic epidemiology. Am J Epidemiol 2002; 156: 204-10.
52. DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials 1986; 7: 177-88.
53. Higgins JP, Thompson SG. Quantifying heterogeneity in a meta-analysis. Stat Med 2002; 21: 1539-58.