Reviewer agreement trends from four years of electronic submissions of conference abstract

BMC Medical Research Methodology

Volumn 6, Issue , 2006, Pages

  Rowe, Brian H ^a   Strome, Trevor L ^b   Spooner, Carol ^a   Blitz, Sandra ^a   Grafstein, Eric ^c   Worster, Andrew ^d  

^a UNIVERSITY OF ALBERTA   (Canada)

^b WINNIPEG REGIONAL HEALTH AUTHORITY   (Canada)

^c ST PAUL S HOSPITAL   (Canada)

^d MCMASTER UNIVERSITY   (Canada)

Author keywords

[No Author keywords available]

Indexed keywords

ARTICLE; CONTROLLED STUDY; CORRELATION COEFFICIENT; HUMAN; INFORMATION PROCESSING; INTERRATER RELIABILITY; MEDICAL LITERATURE; MEDICAL SOCIETY; ONLINE SYSTEM; SCORING SYSTEM; CANADA; CLASSIFICATION; CONSENSUS; DECISION MAKING; DOCUMENTATION; EMERGENCY MEDICINE; METHODOLOGY; OBSERVER VARIATION; ORGANIZATION; PEER REVIEW; QUALITY CONTROL; REPRODUCIBILITY; STANDARDS; STATISTICAL ANALYSIS;

ABSTRACTING AND INDEXING AS TOPIC; CANADA; CONGRESSES AS TOPIC; CONSENSUS; DATA INTERPRETATION, STATISTICAL; EMERGENCY MEDICINE; HUMANS; JUDGMENT; OBSERVER VARIATION; ONLINE SYSTEMS; PEER REVIEW, RESEARCH; QUALITY CONTROL; REPRODUCIBILITY OF RESULTS; RESEARCH DESIGN;

EID: 33744934210     PISSN: 14712288     EISSN: 14712288     Source Type: Journal    
DOI: 10.1186/1471-2288-6-14     Document Type: Article

References (17)

1. Scherer RW, Langenberg P: Full publication of results initially presented in abstracts (Cochrane Methodology Review). In The Cochrane Library Issue 2 Chichester, UK: John Wiley & Sons, Ltd; 2004.
2. Scherer RW, Dickersin K, Landenberg P: Full publication of results initially presented in abstracts. A meta-analysis. JAMA 1994, 272:158-162.
3. Timmer A, Sutherland LR, Hilsden RJ: Development and evaluation of a quality score for abstracts. BMC Medical Research Methodology 2003, 3:2.
4. McAuley L, Pham B, Tugwell P, Moher D: Does the inclusion of grey literature influence estimates of intervention effectiveness reported in meta-analyses? Lancet 2000, 356:1228-31.
5. Cook DJ, Guyatt GH, Ryan G, Clifton J, Buckingham L, Willan A, McIlroy W, Oxman A: Should unpublished data be included in meta-analyses? JAMA 1993, 269:2749-53.
6. Kemper KJ, McCarthy PL, Cicchetti DV: Improving participation and interrater agreement in scoring ambulatory pediatric association abstracts. How well have we succeeded? Arch Pediatr Adolesc Med 1996, 150:380-383.
7. Abstract Submission CAEP Annual Scientific Assembly June 14-17, 2003 - Call for Abstracts. Can J Emerg Med 2002, 4(6):440.
8. Abstract Submission CAEP Annual Scientific Assembly April 26-29, 2004 - Call for Abstracts. Can J Emerg Med 2003, 5:363.
9. Canadian Association of Emergency Physicians [http://www.caep.ca/]
10. Rubin HR, Redelmeier DA, Wu AW, Steinberg EP: How reliable is peer review of scientific abstracts? J Gen Intern Med 1993, 8:255-258.1.
11. Streiner DL, Norman GR: Health measurement scales: a practical guide to their development and use. Oxford: Oxford University Press; 1989.
12. Shrout PE, Fleiss JL: Intraclass Correlations: Uses in Assessing Rater Reliability. Psychological Bulletin 1979, 86(2):420-428.
13. Muller R, Buttner P: A critical discussion of intraclass correlation coefficients. Statistics in Medicine 1994, 13:2465-76.
14. Montgomery AA, Graham A, Evans PH, Fahey T: Inter-rater agreement in the scoring of abstracts submitted to a primary care research conference. BMC Health Services Research 2002, 2(8):.
15. Rothwell PM, Martyn CN: Reproducibility of peer review in clinical neuroscience. Is agreement between reviewers any greater than would be expected by chance alone? Brain 2000, 123:1964-1969.
16. Cicchetti DV, Conn HO: A statistical analysis of reviewer agreement and bias in evaluating medical abstracts. Yale J Biol Med 1976, 49:373-83.
17. Jako RA, Murphy KR: Distributional ratings, judgment decomposition and their impact on interrater agreement and rating accuracy. J Applied Psychol 1990, 75:500-505.