A review and analysis of research on the test-retest reliability of professional judgment

Journal of Behavioral Decision Making

Volumn 13, Issue 3, 2000, Pages 277-294

  Ashton, Robert H ^a  

^a DUKE UNIVERSITY   (United States)

Author keywords

Internal consistency; Professional judgment; Temporal stability; Test retest reliability

Indexed keywords

EID: 23044519549     PISSN: 08943257     EISSN: None     Source Type: Journal    
DOI: 10.1002/1099-0771(200007/09)13:3<277::AID-BDM350>3.0.CO;2-B     Document Type: Article

References (33)

1. Abdel-khalik, A. R., Snowball, D. and Wragge, J. H. 'The effects of certain internal audit variables on the planning of external audit programs', The Accounting Review, 58 (1983), 215-227.
2. Ashton, R. H. 'An experimental study of internal control judgments', Journal of Accounting Research, 12 (1974a), 143-157.
3. Ashton, R. H. 'Cue utilization and expert judgments: A comparison of independent auditors with other judges', Journal of Applied Psychology, 59 (1974b), 437-444.
4. Ashton, R. H. and Ashton, A. H. 'Perspectives on judgment and decision-making research in accounting and auditing', in Ashton, R. H. and Ashton, A. H. (eds), Judgment and Decision-Making Research in Accounting and Auditing, New York: Cambridge University Press, (1995), 3-25.
5. Ashton, R. H. and Brown, P. R. 'Descriptive modeling of auditors' internal control judgments: Replication and extension', Journal of Accounting Research, 18 (1980), 269-277.
6. Bain, C. E. and Kilpatrick, B. G. 'A note on professionals' judgments of tax authority', The Journal of the American Taxation Association, 12 (1990), 78-87.
7. Balzer, W. K., Rohrbaugh, J. and Murphy, K. R. 'Reliability of actual and predicted judgments across time', Organizational Behavior and Human Performance, 32 (1983), 109-123.
8. Beenen, F. 'Psychiatric diagnosis and subjective probability', Acta Psychologica, 34 (1970), 328-337.
9. Bernardin, H. J. 'Effect of reciprocal leniency on the relation between consideration scores from the Leader Behavior Description Questionnaire and performance ratings', Psychological Reports, 60 (1987), 479-487.
10. Beyth-Marom, R. 'How probable is probable? A numerical translation of verbal probability expressions', Journal of Forecasting, 1 (1982), 257-269.
11. Blackburn, H. 'The exercise electrocardiogram: Differences in interpretation', The American Journal of Cardiology, 21 (1968), 871-880.
12. Borman, W. C. 'Format and training effects on rating accuracy and rater errors', Journal of Applied Psychology, 64 (1979), 410-421.
13. Bradshaw, G. L. and Shaw, D. 'Forecasting solar flares: Experts and artificial systems', Organizational Behavior and Human Decision Processes, 53 (1992), 135-157.
14. Brehmer, B. 'Response consistency in probabilistic inference tasks', Organizational Behavior and Human Performance, 22 (1978), 103-115.
15. Brehmer, A. and Brehmer, B. 'What have we learned about human judgment from thirty years of policy capturing?' in Brehmer, B. and Joyce, C. R. B. (eds), Human Judgment: The SJT View, Amsterdam: North-Holland/Elsevier, (1988), 74-114.
16. Brown, P. R. 'Independent auditor judgment in the evaluation of internal audit functions', Journal of Accounting Research, 21 (1983), 444-455.
17. Chandler, M. 'Self-awareness and its relation to other parameters of the clinical inference process', Journal of Consulting and Clinical Psychology, 35 (1970), 258-264.
18. Chang, O. H. and McCarty, T. M. 'Evidence on judgment involving the determination of substantial authority: Tax practitioners versus students', The Journal of the American Taxation Association, 10 (1988), 26-39.
19. Choo, F. and Eggleton, I. R. C. The effect of time pressure on auditors' internal control judgments', Unpublished Working Paper, Victoria University of Wellington (1982).
20. Clark, V. A., Ruffin, C. L., Hill, D. J. and Beaman, A. L. 'Ratings of orally presented verbal expressions of probability by a heterogeneous group', Journal of Applied Social Psychology, 22 (1992), 638-656.
21. Colbert, J. L. 'Inherent risk: An investigation of auditors' judgments', Accounting Organizations and Society, 13 (1988), 111-121.
22. Conway, J. M., Jako, R. A. and Goodman, D. F. 'A meta-analysis of interrater and internal consistency reliability of selection interviews', Journal of Applied Psychology, 80 (1995), 565-579.
23. Cooksey, R. W. Judgment Analysis, San Diego: Academic Press, 1996.
24. Cooksey, R. W., Freebody, P. and Bennett, A. J. The ecology of spelling: A lens model analysis of spelling errors and student judgments of spelling difficulty', Reading Psychology: An International Quarterly, 11 (1990), 293-322.
25. Cornelius, E. T. and Lyness, K. S. 'A comparison of holistic and decomposed judgment strategies in job analysis by job incumbents', Journal of Applied Psychology, 65 (1980), 155-163.
26. DeLeo, P. J. and Pritchard, R. D. 'An examination of some methodological problems in testing expectancyvalence models with survey techniques', Organizational Behavior and Human Performance, 12 (1974), 143-148.
27. Distefano, M. K., Pryer, M. W. and Erffmeyer, R. C. 'Application of content validity methods to the development of a job-related performance rating criterion', Personnel Psychology, 36 (1983), 621-631.
28. Ebert, R. J. and Kruse, T. E. 'Bootstrapping the security analyst', Journal of Applied Psychology, 63 (1978), 110-119.
29. Einhorn, H. J. 'Expert judgment: Some necessary conditions and an example', Journal of Applied Psychology, 59 (1974), 562-571.
30. Eisenberg, R. L., Hedgcock, M. W., Gooding, G. A. W., DeMartini, W. J., Akin, J. R. and Ovenfors, C. O. 'Compensation examination of the cervical and lumbar spines: Critical disagreement in radiographic interpretation', American Journal of Roentgenology, 134 (1980), 519-522.
31. Fairbanks, V. F. 'Is the peripheral blood film reliable for the diagnosis of iron deficiency anemia?', American Journal of Clinical Pathology, 55 (1971), 447-451.
32. Falek, A. and Moser, H. M. 'Classification of schizophrenia', Archives of General Psychiatry, 32 (1975), 59-67.
33. Gardner, D. G., Dunham, R. B., Cummings, L. L. and Pierce, J. L. 'Focus of attention at work: Construct definition and empirical validation', Journal of Occupational Psychology, 62 (1989), 61-77.