Assessing small study effects and publication bias in orthodontic meta-analyses: A meta-epidemiological study

Clinical Oral Investigations

Volumn 18, Issue 4, 2014, Pages 1031-1044

  Papageorgiou, Spyridon N ^b   Papadopoulos, Moschos A ^a   Athanasiou, Athanasios E ^c  

^a ARISTOTLE UNIVERSITY OF THESSALONIKI   (Greece)

^b UNIVERSITY OF BONN   (Germany)

^c Mohammed Bin Rashid University of Medicine and Health Sciences   (United Arab Emirates)

Author keywords

File drawer problem; Funnel plot; Meta analysis; Meta epidemiological study; Orthodontics; Publication bias

Indexed keywords

EPIDEMIOLOGY; HUMAN; ORTHODONTICS; PUBLISHING;

EPIDEMIOLOGIC STUDIES; HUMANS; ORTHODONTICS; PUBLICATION BIAS;

EID: 84899987288     PISSN: 14326981     EISSN: 14363771     Source Type: Journal    
DOI: 10.1007/s00784-014-1196-3     Document Type: Review

References (73)

1. Pitak-Arnnop P, Sader R, Rapidis AD, Dhanuthai K, Bauer U, Herve C, Hemprich A (2010) Publication bias in oral and maxillofacial surgery journals: an observation on published controlled trials. J Craniomaxillofac Surg 38: 4-10.
2. Polychronopoulou A, Pandis N, Eliades T (2010) Assessment of publication bias in dental specialty journals. J Evid Based Dent Pract 10: 207-211.
3. Koletsi D, Karagianni A, Pandis N, Makou M, Polychronopoulou A, Eliades T (2009) Are studies reporting significant results more likely to be published? Am J Orthod Dentofac Orthop 136(632): e1-e5.
4. Scholey JM (2002) Publication bias in dental research. Thesis, University of Liverpool, UK.
5. Dwan K, Altman DG, Arnaiz JA, Bloom J, Chan AW, Cronin E, Decullier E, Easterbrook PJ, Von Elm E, Gamble C, Ghersi D, Ioannidis JP, Simes J, Williamson PR (2008) Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS One 3: e3081.
6. Sterne JA, Egger M, Smith GD (2001) Systematic reviews in health care: investigating and dealing with publication and other biases in meta-analysis. BMJ 323: 101-105.
7. Sterne JA, Gavaghan D, Egger M (2000) Publication and related bias in meta-analysis: power of statistical tests and prevalence in the literature. J Clin Epidemiol 53: 1119-1129.
8. Chan AW, Hrobjartsson A, Haahr MT, Gotzsche PC, Altman DG (2004) Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA 291: 2457-2465.
9. Hopewell S, Loudon K, Clarke MJ, Oxman AD, Dickersin K (2009) Publication bias in clinical trials due to statistical significance or direction of trial results. Cochrane Database Syst Rev 1: MR000006.
10. Egger M, Davey Smith G, Schneider M, Minder C (1997) Bias in meta-analysis detected by a simple, graphical test. BMJ 315: 629-634.
11. Williamson PR, Gamble C (2005) Identification and impact of outcome selection bias in meta-analysis. Stat Med 24: 1547-1561.
12. Borenstein M, Hedges LV, Higgins JPT, Rothstein HR (2009) Publication bias. In: Borenstein M, Hedges LV, Higgins JPT, Rothstein HR (eds) Introduction to meta-analysis. Wiley, Chichester, pp 277-294.
13. Rücker G (2010) Small-study effects and heterogeneity in meta-analysis. Thesis, Albert-Ludwigs-University, Germany.
14. Sutton AJ (2005) Evidence concerning the consequences of publication and related biases. In: Rothstein HR, Sutton AJ, Borenstein M (eds) Publication bias in meta-analysis: prevention, assessment and adjustments. Wiley, Chichester, pp 175-192.
15. Light R, Pillemer D (1984) Summing up: the science of reviewing research. Harvard University Press, Cambridge.
16. Begg CB, Mazumdar M (1994) Operating characteristics of a rank correlation test for publication bias. Biometrics 50: 1088-1101.
17. Duval S, Tweedie R (2000) Trim and fill: a simple funnel-plot-based method of testing and adjusting for publication bias in meta-analysis. Biometrics 56: 455-463.
18. Borenstein M (2005) Software for publication bias. In: Rothstein H, Sutton AJ, Borenstein M (eds) Publication bias in meta-analysis: prevention, assessment and adjustments. Wiley, Chichester, pp 391-403.
19. Peters JL, Sutton AJ, Jones DR, Abrams KR, Rushton L (2008) Contour-enhanced meta-analysis funnel plots help distinguish publication bias from other causes of asymmetry. J Clin Epidemiol 61: 991-996.
20. Palmer TM, Peters JL, Sutton AJ, Moreno SG (2008) Contour-enhanced funnel plots for meta-analysis. Stata J 8: 242-254.
21. Moreno SG, Sutton AJ, Turner EH, Abrams KR, Cooper NJ, Palmer TM, Ades AE (2009) Novel methods to deal with publication biases: secondary analysis of antidepressant trials in the FDA trial registry database and related journal publications. BMJ 339: b2981.
22. Sterne JAC, Egger M, Moher D (2011) Addressing reporting biases. In: Higgins JP, Green S (eds) Cochrane handbook for systematic reviews of interventions 4. 2. 6 [updated September 2006]. http://www. cochrane. org/resources/handbook/hbook. htm. Accessed 20 Mar 2013.
23. Papageorgiou SN, Papadopoulos MA, Athanasiou AE (2013) Reporting characteristics of meta-analyses in orthodontics: methodological assessment and statistical recommendations. Eur J Orthod. doi: 10. 1093/ejo/cjt008.
24. Jennions MD, Møller AP (2002) Publication bias in ecology and evolution: an empirical assessment using the 'trim and fill' method. Biol Rev Camb Philos Soc 77: 211-222.
25. Sutton AJ, Duval SJ, Tweedie RL, Abrams KR, Jones DR (2000) Empirical assessment of effect of publication bias on meta-analyses. BMJ 320: 1574-1577.
26. Higgins JP, Thompson SG, Deeks JJ, Altman DG (2003) Measuring inconsistency in meta-analyses. BMJ 327: 557-560.
27. 2010 Journal Citation Reports® Science Edition 2011, Thomson Reuters. http://thomsonreuters. com. Accessed 20 Mar 2013.
28. SCImago 2007 SJR-SCImago Journal & Country Rank. http://www. scimagojr. com. Accessed 20 Mar 2013.
29. Ioannidis JP, Trikalinos TA (2007) The appropriateness of asymmetry tests for publication bias in meta-analyses: a large survey. CMAJ 176: 1091-1096.
30. Cuijpers P, Smit F, Bohlmeijer E, Hollon SD, Andersson G (2010) Efficacy of cognitive-behavioural therapy and other psychological treatments for adult depression: meta-analytic study of publication bias. Br J Psychiatry 196: 173-178.
31. Al-Jewair TS, Azarpazhooh A, Suri S, Shah PS (2009) Computer-assisted learning in orthodontic education: a systematic review and meta-analysis. J Dent Educ 73: 730-739.
32. Antonarakis GS, Kiliaridis S (2008) Maxillary molar distalization with noncompliance intramaxillary appliances in class II malocclusion. A systematic review. Angle Orthod 78: 1133-1140.
33. Badovinac RL, Werler MM, Williams PL, Kelsey KT, Hayes C (2007) Folic acid-containing supplement consumption during pregnancy and risk for oral clefts: a meta-analysis. Birth Defects Res A Clin Mol Teratol 79: 8-15.
34. Derks A, Katsaros C, Frencken JE, van't Hof MA, Kuijpers-Jagtman AM (2004) Caries-inhibiting effect of preventive measures during orthodontic treatment with fixed appliances. A systematic review. Caries Res 38: 413-420.
35. Frazão P, Narvai PC, Latorre Mdo R, Castellanos RA (2004) Are severe occlusal problems more frequent in permanent than deciduous dentition? Rev Saude Publica 38: 247-254.
36. Fricton J, Look JO, Wright E, Alencar FG Jr, Chen H, Lang M, Ouyang W, Velly AM (2010) Systematic review and meta-analysis of randomized controlled trials evaluating intraoral orthopedic appliances for temporomandibular disorders. J Orofac Pain 24: 237-254.
37. Gleitsmann K, Kriz H, Thielke A, Bunker K, Ryan K Lorish K, King V (2012) Sleep apnea diagnosis and treatment in adults. Center for Evidence-based Policy, Portland.
38. Goh YI, Bollano E, Einarson TR, Koren G (2006) Prenatal multivitamin supplementation and rates of congenital anomalies: a meta-analysis. J Obstet Gynaecol Can 28: 680-689.
39. Hackshaw A, Rodeck C, Boniface S (2011) Maternal smoking in pregnancy and birth defects: a systematic review based on 173 687 malformed cases and 11. 7 million controls. Hum Reprod Update 17: 589-604.
40. Herkrath AP, Herkrath FJ, Rebelo MA, Vettore MV (2012) Parental age as a risk factor for non-syndromic oral clefts: a meta-analysis. J Dent 40: 3-14.
41. Jäger A, Braumann B, Kim C, Wahner S (2001) Skeletal and dental effects of maxillary protraction in patients with angle class III malocclusion. A meta-analysis. J Orofac Orthop 62: 275-284.
42. Johnson CY, Little J (2008) Folate intake, markers of folate status and oral clefts: is the evidence converging? Int J Epidemiol 37: 1041-1058.
43. Lim J, Lasserson TJ, Fleetham J, Wright J (2006) Oral appliances for obstructive sleep apnoea. Cochrane Database Syst Rev 1: CD004435.
44. Little J, Cardy A, Munger RG (2004) Tobacco smoking and oral clefts: a meta-analysis. Bull World Health Organ 82: 213-218.
45. Mattos CT, Vilani GN, Sant'Anna EF, Ruellas AC, Maia LC (2011) Effects of orthognathic surgery on oropharyngeal airway: a meta-analysis. Int J Oral Maxillofac Surg 40: 1347-1356.
46. Mitchell LE (1997) Transforming growth factor alpha locus and nonsyndromic cleft lip with or without cleft palate: a reappraisal. Genet Epidemiol 14: 231-240.
47. Nguyen QV, Bezemer PD, Habets L, Prahl-Andersen B (1999) A systematic review of the relationship between overjet size and traumatic dental injuries. Eur J Orthod 21: 503-515.
48. Nieuwenhuijsen MJ, Martinez D, Grellier J, Bennett J, Best N, Iszatt N, Vrijheid M, Toledano MB (2009) Chlorination disinfection by-products in drinking water and congenital anomalies: review and meta-analyses. Environ Health Perspect 117: 1486-1493.
49. Papadopoulos MA, Papageorgiou SN, Zogakis IP (2011) Clinical effectiveness of orthodontic miniscrew implants: a meta-analysis. J Dent Res 90: 969-976.
50. Papageorgiou SN, Zogakis IP, Papadopoulos MA (2012) Failure rates and associated risk factors of orthodontic miniscrew implants: a meta-analysis. Am J Orthod Dentofac Orthop 142: 577-595, e7.
51. Perillo L, Cannavale R, Ferro F, Franchi L, Masucci C, Chiodini P, Baccetti T (2011) Meta-analysis of skeletal mandibular changes during Frankel appliance treatment. Eur J Orthod 33: 84-92.
52. Polder BJ, Van't Hof MA, van der Linden FP, Kuijpers-Jagtman AM (2004) A meta-analysis of the prevalence of dental agenesis of permanent teeth. Community Dent Oral Epidemiol 32: 217-226.
53. Romitti PA, Herring AM, Dennis LK, Wong-Gibbons DL (2007) Meta-analysis: pesticides and orofacial clefts. Cleft Palate Craniofac J 44: 358-365.
54. Verkleij-Hagoort A, Bliek J, Sayed-Tabatabaei F, Ursem N, Steegers E, Steegers-Theunissen R (2007) Hyperhomocysteinemia and MTHFR polymorphisms in association with orofacial clefts and congenital heart defects: a meta-analysis. Am J Med Genet A 143A: 952-960.
55. Vieira AR, Orioli IM, Murray JC (2002) Maternal age and oral clefts: a reappraisal. Oral Surg Oral Med Oral Pathol Oral Radiol Endod 94: 530-535.
56. Weinberg SM, Maher BS, Marazita ML (2006) Parental craniofacial morphology in cleft lip with or without cleft palate as determined by cephalometry: a meta-analysis. Orthod Craniofac Res 9: 18-30.
57. Wyszynski DF, Duffy DL, Beaty TH (1997) Maternal cigarette smoking and oral clefts: a meta-analysis. Cleft Palate Craniofac J 34: 206-210.
58. Papageorgiou SN, Papadopoulos MA, Athanasiou AE (2011) Evaluation of methodology and quality characteristics of systematic reviews in orthodontics. Orthod Craniofac Res 14: 116-137.
59. Sterne JA, Egger M (2001) Funnel plots for detecting bias in meta-analysis: guidelines on choice of axis. J Clin Epidemiol 54: 1046-1055.
60. Terrin N, Schmid CH, Lau J (2005) In an empirical evaluation of the funnel plot, researchers could not visually identify publication bias. J Clin Epidemiol 58: 894-901.
61. Kjaergard LL, Villumsen J, Gluud C (2001) Reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses. Ann Intern Med 135: 982-989.
62. Shang A, Huwiler-Muntener K, Nartey L, Juni P, Dorig S, Sterne JA, Pewsner D, Egger M (2005) Are the clinical effects of homoeopathy placebo effects? Comparative study of placebo-controlled trials of homoeopathy and allopathy. Lancet 366: 726-732.
63. Souza JP, Pileggi C, Cecatti JG (2007) Assessment of funnel plot asymmetry and publication bias in reproductive health meta-analyses: an analytic survey. Reprod Health 4: 3.
64. Higgins JP, Green S (eds) Cochrane handbook for systematic reviews of interventions 4. 2. 6 [updated September 2006]. http://www. cochrane. org/resources/handbook/hbook. htm. Accessed 20 Mar 2013.
65. Sterne JA, Sutton AJ, Ioannidis JP, Terrin N, Jones DR, Lau J, Carpenter J, Rücker G, Harbord RM, Schmid CH, Tetzlaff J, Deeks JJ, Peters J, Macaskill P, Schwarzer G, Duval S, Altman DG, Moher D, Higgins JP (2011) Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials. BMJ 343: d4002.
66. Furukawa TA, Watanabe N, Omori IM, Montori VM, Guyatt GH (2007) Association between unreported outcomes and effect size estimates in Cochrane meta-analyses. JAMA 297: 468-470.
67. Tang JL, Liu JL (2000) Misleading funnel plot for detection of bias in meta-analysis. J Clin Epidemiol 53: 477-484.
68. Moreno SG, Sutton AJ, Ades AE, Cooper NJ, Abrams KR (2011) Adjusting for publication biases across similar interventions performed well when compared with gold standard data. J Clin Epidemiol 64: 1230-1241.
69. Rücker G, Schwarzer G, Carpenter JR, Binder H, Schumacher M (2011) Treatment-effect estimates adjusted for small-study effects via a limit meta-analysis. Biostatistics 12: 122-142.
70. Harbord RM, Egger M, Sterne JA (2006) A modified test for small-study effects in meta-analyses of controlled trials with binary endpoints. Stat Med 25: 3443-3457.
71. Moreno SG, Sutton AJ, Thompson JR, Ades AE, Abrams KR, Cooper NJ (2012) A generalized weighting regression-derived meta-analysis estimator robust to small-study effects and heterogeneity. Stat Med 31: 1407-1417.
72. Copas JB, Shi JQ (2001) A sensitivity analysis for publication bias in systematic reviews. Stat Methods Med Res 10: 251-265.
73. Schwarzer G, Carpenter J, Rücker G (2010) Empirical evaluation suggests Copas selection model preferable to trim-and-fill method for selection bias in meta-analysis. J Clin Epidemiol 63: 282-288.