Researching the reliability of accreditation survey teams: Lessons learnt when things went awry

Health Information Management Journal

Volumn 42, Issue 1, 2013, Pages 4-10

  Greenfield, David ^a   Pawsey, Marjorie ^a   Naylor, Justine ^a   Braithwaite, Jeffrey ^a  

^a UNIVERSITY OF NEW SOUTH WALES   (Australia)

Author keywords

Accreditation; Australia; Hospitals; Quality improvement; Reliability; Surveys

Indexed keywords

ACCREDITATION; ARTICLE; AUSTRALIA; DECISION MAKING; HEALTH CARE SURVEY; HUMAN; METHODOLOGY; OBSERVER VARIATION; PEER REVIEW; REPRODUCIBILITY; STANDARD; STATISTICS; TEACHING HOSPITAL; VALIDATION STUDY; NEW SOUTH WALES; PROCEDURES; STANDARDS; STATISTICS AND NUMERICAL DATA;

ACCREDITATION; HEALTH CARE SURVEYS; HOSPITALS, TEACHING; HUMANS; JUDGMENT; NEW SOUTH WALES; OBSERVER VARIATION; PEER REVIEW; REPRODUCIBILITY OF RESULTS;

EID: 84885350316     PISSN: 18333583     EISSN: 18333575     Source Type: Journal    
DOI: 10.1177/183335831304200101     Document Type: Article

References (43)

1. Baglio, M. (2004). Assessment of interobserver reliability in nutrition studies that used direct observation of school means. Journal of the American Dietetic Association 104(9):1385-1392.
2. Bernard H. and Ryan G. (2010). Analyzing qualitative data: systematic approaches. Singapore, Sage.
3. Bernstein S., Hofer T., Meijler A. and Rigter H. (1997). Setting standards for effectiveness: a comparison of expert panels and decision analysis. International Journal for Quality in Health Care 9(4):255-263.
4. Braithwaite J., Westbrook J., Pawsey M., Greenfield D., Naylor J., Iedema R., Runciman B., Redman S., Jorm C., Robinson M., Nathan S. and Gibberd R. (2006). A prospective, multi-method, multi-disciplinary, multi-level, collaborative, socialorganisational design for researching health sector accreditation. BMC Health Services Research 6113.
5. Braithwaite J., Greenfield D., Westbrook J., Pawsey M., Westbrook M., Gibberd R., Naylor J., Nathan S., Robinson M., Runciman B., Jackson M., Travaglia J., Johnston B., Yen D., McDonald H., Low L., Redman S., Johnson B., Corbett A., Hennessy D., Clark J. and Lancaster J. (2010). Health service accreditation as a predictor of clinical and organisational performance: a blinded, random, stratified study. Quality and Safety in Health Care 19(1):14-21. DOI: 10.1136/qshc.2009.033928.
6. Bruce S.B. and Daniel W. (2001). Disparity in quantitative risk assessment: a review of input distributions. Risk Analysis 21(1):75-90.
7. Cheng P., Gilchrist A., Robinson K., and Paul L. (2009). The risk and consequences of clinical miscoding due to inadequate medical documentation: a case study of the Impact on health services funding. Health Information Management Journal 38(1):35-46.
8. Curtis K., Mitchell R., Dickson C., Black D. and Lam M. (2012). Do AR-DRGs adequately describe the trauma patient episode in New South Wales, Australia? Health Information Management Journal 40(1):7-13.
9. Dauphinee D. and Blackmore D. (2001). Assessing the assessors' assessment. Medical Education 35(4):317-318. DOI: doi:10.1046/j.1365-2923.2001.00927.x.
10. Duszak R., Blackham W.C., Kusiak G.M. and Majchrzak J. (2004). CPT coding by interventional radiologists: a multi-institutional evaluation of accuracy and its economic implications. Journal of the American College of Radiology 1(10):734-740.
11. Evans A., Leeson M. and Petrie A. (2007). Reliability of peer and self-assessment scores compared with trainers' scores following third molar surgery. Medical Education 41(9):866-872.
12. Frisino J. (2002). COA's accreditation system: checks, balances, and firewalls. Behavioral Health Accreditation and Accountability Alert 7(4):1-4.
13. Gawel R. and Godden P. (2008). Evaluation of the consistency of wine quality assessments from expert wine tasters. Australian Journal of Grape and Wine Research 14(1):1-8.
14. Gibbons C.T. and Downes E.M. (1998). Who audits the auditors? Annals of the Royal College of Surgeons of England 80(2):74-75.
15. Greenfield D. and Braithwaite J. (2008). Health sector accreditation research: a systematic review. International Journal for Quality in Health Care 20(3):172-183.
16. Greenfield D. and Braithwaite J. (2009). Developing the evidence base for accreditation of healthcare organisations: a call for transparency and innovation. Quality and Safety in Health Care 18(3):162-163.
17. Greenfield D., Braithwaite J. and Pawsey M. (2008). Health care accreditation surveyor styles typology. International Journal of Health Care Quality Assurance 21(5):435-443.
18. Greenfield D., Pawsey M., and Braithwaite J. (2011). What motivates professionals to improve quality and safety in healthcare organisations? Journal of Health Organisation and Management 23(1):8-14.
19. Greenfield D., Pawsey M., and Braithwaite, J. (2012a). The role and impact of accreditation on the healthcare revolution [O papel e o impacto da acreditação na revolução da atenção à saúde]. Acreditação 1(2):1-14.
20. th ed), Jones and Barlett Learning, New York. pp. 513-531.
21. Greenfield D., Pawsey M., Naylor J. and Braithwaite J. (2009). Are healthcare accreditation surveys reliable? International Journal of Health Care Quality Assurance 22(2):105-116.
22. Greenfield D., Hinchcliff R., Moldovan M., Mumford V., Pawsey M., Westbrook J. and Braithwaite J. (in press). A multi-method research investigation of consumer involvement in Australian health service accreditation programs: the ACCREDIT-SCI study protocol. BMC Open. DOI: doi:10.1136/bmjopen-2012-001726.
23. Greenfield D., Hinchcliff R., Westbrook M., Jones D., Low L., Johnston B., Banks M., Pawsey M., Moldovan M., Westbrook J. and Braithwaite J. (2012b). An empirical test of accreditation patient journey surveys: randomised trial. International Journal for Quality in Health Care first published online: July 11, 2012; doi: 10.1093/intqhc/mzs035.
24. Greenfield D., Moldovan M., Westbrook M., Jones D., Low L., Johnston B., Clark S., Banks M., Pawsey M., Hinchcliff R., Westbrook J. and Braithwaite J. (2012c). An empirical test of short notice surveys in two accreditation programs. International Journal for Quality in Health Care 24(1):65-71.
25. Greenhalgh T., Wood G.W., Bratan T., Stramer K. and Hinder S. (2008). Patients' attitudes to the summary care record and HealthSpace: qualitative study. BMJ 336(7656):1290-1295.
26. Hinchcliff R., Greenfield D., Moldovan M., Pawsey M., Mumford V., Westbrook J. and Braithwaite J. (2012). Evaluation of current Australian health service accreditation processes (ACCREDIT CAP): protocol for a mixed-method research project. BMC Open 2(2):1-7.
27. Hinchcliff R., Greenfield D., Westbrook J., Moldovan M., Pawsey M., Mumford V. and Braithwaite J. (in press). Narrative synthesis of health service accreditation literature. BMJ Quality and Safety.
28. Humphrey-Murto S., Smee S., Touchie C., Wood T.J. and Blackmore D.E. (2005). A comparison of physician examiners and trained assessors in a high-stakes OSCE setting. Academic Medicine 80(10):S59-S62.
29. Jayasinghie U., Herbert M. and Bond N. (2006). A new reader trial approach to peer review in funding research grants: an Australian experiment. Scientometrics 69(3):591-606.
30. Jörg F., Boeije H., Huijsman R., de Weert G. and Schrijvers A. (2002). Objectivity in needs assessment practice: admission to a residential home. Health and Social Care in the Community 10(6):445-456.
31. King M.S., Lipsky M.S. and Sharp L. (2002). Expert agreement in current procedural terminology evaluation and management coding. Archives of Internal Medicine 162(3):316-320.
32. Lewis P.M., Evans A.R., Kulkarni R. and Roberts P. (2003). Risk assessors need to be assessed. BMJ 327(7423):1108-b-1109.
33. rd ed. Oxford University Press, Melbourne.
34. McEnery J. and Blanchard P.N. (1999). Validity of multiple ratings of business student performance in a management simulation. Human Resource Development Quarterly 10(2):155-172.
35. Paulus I., Coppenolle H. and Schrevens E. (2000). Modelling the consensus and differences between assessors inspecting the colour quality of apples by 'tree-based modelling'. Journal of the science of food and agriculture 80(13):1953-1963.
36. Price E. and Robertson K. (2012). The coding masterpiece: a framework for the formal processes and pathways of health classification. Health Information Management Journal 40(1):14-20.
37. Rees C. and Sheard C. (2004). The reliability of assessment criteria for undergraduate medical students' communication skills portfolios: the Nottingham experience. Medical Education 38(2):138-144.
38. Rees C. and Shepherd M. (2005). Students' and assessors' attitudes towards students' self-assessment of their personal and professional behaviours. Medical Education 39(1):30-39.
39. Schroter S., Black N., Evans S., Carpenter J., Godlee F. and Smith R. (2004). Effects of training on quality of peer review: randomised controlled trial. BMJ 328(7441):673.
40. Smith M., Atherly A., Kane R. and Pacala J. (1997). Peer review of the quality of care: reliability and sources of variability for outcomes and process assessments. JAMA 278(19):1573-1578.
41. Sonnenfelds J. (1985). Shedding light on the Hawthorne studies. Journal of Occupational Behavior 6(2):111-130.
42. Southgate L., Cox J., David T., Hatch D., Howes A., Johnson N., Jolly B., Macdonald E., McAvoy P., McCrorie P. and Turner P. (2001). The assessment of poorly performing doctors: the development of the assessment programmes for the General Medical Council's Performance Procedures. Medical Education 35(s1):2-8.
43. Watterson D., Cleland H., Picton N., Simpson P. and Gabbe B. (2011). Professional practice and innovation: level of agreement between coding sources of percentage total body surface area burnt (%TBSA). Health Information Management Journal 40(1):21-24.