Making sense of the shadows: Priorities for creating a learning healthcare system based on routinely collected data

BMJ Quality and Safety

Volumn 24, Issue 8, 2015, Pages 505-515

  Deeny, Sarah R ^a   Steventon, Adam ^a  

^a Health Foundation   (United Kingdom)

Author keywords

[No Author keywords available]

Indexed keywords

ANALYTIC METHOD; ARTICLE; ATTITUDE TO COMPUTERS; DATA PROCESSING; DECISION MAKING; HEALTH CARE SYSTEM; HEALTH CARE UTILIZATION; HEALTH SERVICE; HOSPITAL UTILIZATION; HUMAN; INFORMATION PROCESSING; LEARNING; MEDICAL INFORMATICS; RANDOMIZED CONTROLLED TRIAL (TOPIC); TELEPHONE; TOTAL QUALITY MANAGEMENT; CASE CONTROL STUDY; HEALTH CARE DELIVERY; HEALTH CARE PLANNING; MEDICAL RECORD; PROCEDURES; STANDARDS; STATISTICAL ANALYSIS; STATISTICAL BIAS; UNITED KINGDOM;

BIAS (EPIDEMIOLOGY); CASE-CONTROL STUDIES; DATA COLLECTION; DATA INTERPRETATION, STATISTICAL; DELIVERY OF HEALTH CARE; HEALTH PRIORITIES; HEALTH SERVICES; HUMANS; MEDICAL RECORDS; UNITED KINGDOM;

EID: 84938515650     PISSN: 20445415     EISSN: None     Source Type: Journal    
DOI: 10.1136/bmjqs-2015-004278     Document Type: Article

References (112)

1. Plato. The myth of the cave: from book VII. In: Lee D. ed. The Republic. Penguin Classics, 2007: 312-60.
2. NHS England. Five year forward view. 2014. http://www.england.nhs.uk/wp-content/uploads/2014/10/5yfv-web.pdf
3. National Information Board. Personalised Health and Care 2020, Using Data and Technology to Transform Outcomes for Patients and Citizens, A Framework for Action. 2014. https://www.gov.uk/government/uploads/system/uploads/attachment-data/file/384650/NIB-Report.pdf
4. National Advisory Group on the, Safety of Patients in England. A promise to learn-a commitment to act: Improving the Safety of Patients in England. Leeds: Department of Health, 2013.
5. OECD. Strengthening Health information infrastructure for health care quality governance: good practices, new opportunities, and data privacy protection challenges. OECD Heal Policy Stud 2013; 9: 1-188.
6. Institute of Medicine. The Learning Healthcare System: Workshop Summary (IOM Roundtable on Evidence-Based Medicine). Washington, DC: The National Academies Press 2007: 37-80.
7. Pope C, Halford S, Tinati R, et al. What's the big fuss about 'big data'? J Health Serv Res Policy 2014; 19: 67-8.
8. Black N. Assessing the quality of hospitals. BMJ 2010; 340: c2066.
9. Shahian DM, Normand S-LT. What is a performance outlier? BMJ Qual Saf 2015; 24: 95-9.
10. Gitelman L. ed. 'Raw data' is an oxymoron. Cambridge, MA: MIT Press, 2013.
11. Department of Health. Payment by results guidance for 2013-14. 2013.
12. Billings J, Blunt I, Steventon A, et al. Development of a predictive model to identify inpatients at risk of re-admission within 30 days of discharge (PARR-30). BMJ Open 2012; 2: 1-10.
13. Amarasingham R, Patel PC, Toto K, et al. Allocating scarce resources in real-time to reduce heart failure readmissions: a prospective, controlled study. BMJ Qual Saf 2013; 22: 998-1005.
14. Schoen C, Osborn R, Squires D, et al. A survey of primary care doctors in ten countries shows progress in use of health information technology, less in other areas. Health Aff 2012; 31: 2805-16.
15. Livingstone SJ, Levin D, Looker HC, et al. Estimated life expectancy in a Scottish cohort with type 1 diabetes, 2008- 2010. JAMA Intern Med 2015; 313: 37-44.
16. Schmidt PE, Meredith P, Prytherch DR, et al. Impact of introducing an electronic physiological surveillance system on hospital mortality. BMJ Qual Saf 2014; 24: 10-20.
17. Noufaily A, Enki DG, Farrington P, et al. An improved algorithm for outbreak detection in multiple surveillance systems. Stat Med 2013; 32: 1206-22.
18. Ovretveit J, Keller C, Forsberg HH, et al. Continuous innovation: developing and using a clinical database with new technology for patient-centred care-the case of the swedish quality register for arthritis. Int J Qual Heal Care 2013; 25: 118-24.
19. Chaudhry SI, Mattera JA, Curtis JP, et al. Telemonitoring in patients with heart failure. N Engl J Med 2010; 363: 2301-9.
20. Henderson C, KnappM, Fernandez J-L, et al. Cost effectiveness of telehealth for patients with long term conditions (Whole Systems Demonstrator telehealth questionnaire study): nested economic evaluation in a pragmatic, cluster randomised controlled trial. BMJ 2013; 346: f1035.
21. Wicks P, Massagli M, Frost J, et al. Sharing health data for better outcomes on PatientsLikeMe. J Med Internet Res 2010; 12: e19.
22. Drake G, Csipke E, Wykes T. Assessing your mood online: acceptability and use of Moodscope. Psychol Med 2013; 43: 1455-64.
23. Martínez-Pérez B, de la Torre-Díez I, López-Coronado M, et al. Mobile apps in cardiology: review. JMIR mHealth uHealth 2013; 1: e15. http://www.pubmedcentral.nih.gov/articlerender.fcgi?artid=4114428&tool=pmcentrez&rendertype=abstract (accessed 12 Feb 2015).
24. Greaves F, Laverty AA, Ramirez Cano D, et al. Tweets about hospital quality: a mixed methods study. BMJ Qual Saf 2014; 23: 838-46.
25. Bardach NS, Asteria-Peñaloza R, Boscardin WJ, et al. The relationship between commercial website ratings and traditional hospital performance measures in the USA. BMJ Qual Saf 2013; 22: 194-202.
26. Butler D. When Google got flu wrong. Nature 2013; 494: 155-6.
27. Brown M, Pinchin J, Blum J, et al. Exploring the relationship between location and behaviour in out of hours hospital care. Commun Comput Inf Sci 2014; 435: 395-400.
28. Srigley JA, Furness CD, Baker GR, et al. Quantification of the Hawthorne effect in hand hygiene compliance monitoring using an electronic monitoring system: a retrospective cohort study. BMJ Qual Saf 2014; 23: 974-80.
29. Currell R, Urquhart C, Wainwright P, et al. Telemedicine versus face to face patient care: effects on professional practice and health care outcomes. Cochrane Database Syst Rev 2000; (2): CD002098. http://dx.doi.org/10.1002/14651858.CD002098
30. Steventon A, Bardsley M, Billings J, et al. Effect of telecare on use of health and social care services: findings from the Whole Systems Demonstrator cluster randomised trial. Age Ageing 2013; 42: 501-8.
31. Ryan M, Kinghorn P, Entwistle VA, et al. Valuing patients' experiences of healthcare processes: Towards broader applications of existing methodsTitle. Soc Sci Med 2014; 106: 194-203. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3988932/?report=readerhttp://dx.doi.org/10.1016/j.socscimed.2014.01.013
32. Craig R, Mindell J. Health Survey for England 2013: health, social care and lifestyles. Leeds: 2014. http://www.hscic.gov.uk/catalogue/PUB16076/HSE2013-Sum-bklet.pdf
33. Black N, Barker M, Payne M. Cross sectional survey of multicentre clinical databases in the United. BMJ 2004; 328: 1478.
34. Stuart EA, DuGoff E, Abrams M, et al. Estimating causal effects in observational studies using electronic health data: challenges and (some) solutions. eGEMs 2013; 1: 4.
35. Steventon A, Tunkel S, Blunt I, et al. Effect of telephone health coaching (Birmingham OwnHealth) on hospital use and associated costs: cohort study with matched controls. BMJ 2013; 347: f4585.
36. Jordan RE, Lancashire RJ, Adab P. An evaluation of Birmingham Own Health telephone care management service among patients with poorly controlled diabetes. A retrospective comparison with the General Practice Research Database. BMC Public Health 2011; 11: 707.
37. Van Dalen MT, Suijker JJ, Macneil-Vroomen J, et al. Self-report of healthcare utilization among community-dwelling older persons: a prospective cohort study. PLoS ONE 2014; 9: e93372.
38. Etheredge LM. A rapid-learning health system. Health Aff 2007; 26: w107-18.
39. Billings J, Georghiou T, Blunt I, et al. Choosing a model to predict hospital admission: an observational study of new variants of predictive models for case finding. BMJ Open 2013; 3: e003352.
40. Bardsley M, Billings J, Dixon J, et al. Predicting who will use intensive social care: case finding tools based on linked health and social care data. Age Ageing 2011; 40: 265-70.
41. Spencer R, Bell B, Avery AJ, et al. Identification of an updated set of prescribing-safety indicators for GPs. Br J Gen Pract 2014; 64: 181-90.
42. Collins A. Measuring what really matters: towards a coherent measurement system to support person-centred care. 2014. http://www.health.org.uk/public/cms/75/76/313/4772/Measuringwhatreallymatters.pdf?realName=7HWr4a.pdf
43. Austin PC, Mamdani MM, Stukel TA, et al. The use of the propensity score for estimating treatment effects: administrative versus clinical data. Stat Med 2005; 24: 1563-78.
44. Nelson E, Eftimovska E, Lind C, et al. Patient reported outcome measures in practice. BMJ 2015; 350: g7818.
45. Pickard L, King D, Knapp M. The 'visibility' of unpaid care in England. J Soc Work 2015; 41: 404-9.
46. Hippisley-Cox J, Fenty J, Heaps M. Trends in Consultation Rates in General Practice 1995 to 2006: Analysis of the QRESEARCH database. Final Report to the Information Centre and Department of Health. 2007. http://www.qresearch.org/Public-Documents/Trendsinconsultationratesingeneralpractice1995to2006.pdf
47. Bardsley M, Billings J, Chassin LJ, et al. Predicting social care costs: a feasibility study. London: Nuffield Trust, 2011.
48. Graham M, Shelton T. Geography and the future of big data, big data and the future of geography. Dialogues Hum Geogr 2013; 3: 255-61.
49. Krieger N. Epidemiology and the web of causation: has anyone seen the spider? Soc Sci Med 1994; 39: 887-903.
50. Acheson ED. the Oxford record linkage study: a review of the method with some preliminary results. Proc R Soc Med 1964; 57: 269-74.
51. ONS. LabourMarket Profile-Great Britain. Nomis Off. labour Mark. Stat. 2015. 2015. https://www.nomisweb.co.uk/reports/lmp/gor/2092957698/report.aspx (accessed 24Mar 2015).
52. Redelmeier DA, Tibshirani RJ. Association between cellular-telephone calls and motor vehicle collisions. N Engl J Med 1997; 336: 453-8.
53. Marmot M, Wilkinson RG. eds. Social determinants of health. 2nd edn. Oxford: Oxford University Press, 2009.
54. Rubin DB. On the limitations of comparative effectiveness research. Stat Med 2010; 29: 1991-5.
55. Currie CJ, Peters JR, Tynan A, et al. Survival as a function of HbA(1c) in people with type 2 diabetes: a retrospective cohort study. Lancet 2010; 375: 481-9.
56. Bilimoria KY, Chung J, Ju MH, et al. Evaluation of surveillance bias and the validity of the venous thromboembolism quality measure. JAMA 2013; 310: 1482-9.
57. Haut ER, Pronovost PJ. Surveillance bias in outcomes reporting. JAMA 2011; 305: 2462-3.
58. Wennberg JE, Staiger DO, Sharp SM, et al. Observational intensity bias associated with illness adjustment: cross sectional analysis of insurance claims. BMJ 2013; 346: f549.
59. Steyerberg EW, Harrell FE, Borsboom GJJM, et al. Internal validation of predictive models: efficiency of some procedures for logistic regression analysis. J Clin Epidemiol 2001; 54: 774-81.
60. Stukel TA, Fisher ES, Wennberg DE, et al. Analysis of observational studies in the presence of treatment selection bias: effects of invasive cardiac management on AMI survival using propensity score and instrumental variable methods. JAMA 2007; 297: 278-85.
61. Imbens GW, Lemieux T. Regression discontinuity designs: a guide to practice. J Econom 2008; 142: 615-35.
62. Rosenbaum PR, Rubin DB. The central role of the propensity score in observational studies for causal effects. Biometrika 1983; 70: 41-55.
63. Steventon A, Bardsley M, Billings J, et al. The role of matched controls in building an evidence base for hospital-avoidance schemes: a retrospective evaluation. Health Serv Res 2012; 47: 1679-98.
64. Stuart EA, Huskamp HA, Duckworth K, et al. Using propensity scores in difference-in-differences models to estimate the effects of a policy change. Heal Serv Outcomes Res Methodol 2014; 14: 166-82.
65. Wennberg DE, Sharp SM, Bevan G, et al. A population health approach to reducing observational intensity bias in health risk adjustment: cross sectional analysis of insurance claims. BMJ 2014; 348: g2392.
66. Diamond A, Sekhon J. Genetic matching for estimating causal effects. Rev Econ Stat 2012; 95: 932-45. http://sekhon.polisci.berkeley.edu/papers/GenMatch.pdf
67. Lee BK, Lessler J, Stuart EA. Improving propensity score weighting using machine learning. Stat Med 2010; 29: 337-46.
68. Carpenter JR, Kenward MG. A comparison of multiple imputation and doubly robust estimation for analyses with missing data. J R Stat Soc A 2006; 3: 571-84.
69. O'Neill PD, Roberts G. Bayesian inference for partially observed stochastic epidemics. J R Stat Soc A 1999; 162: 121-9.
70. Cooper B, Lipsitch M. The analysis of hospital infection data using hidden Markov models. Biostatistics 2004; 5: 223-37.
71. He D, Ionides EL, King AA. Plug-and-play inference for disease dynamics: measles in large and small populations as a case study. J R Soc Interface 2010; 7: 271-83.
72. Curtis LH, Brown J, Platt R. Four health data networks illustrate the potential for a shared national multipurpose big-data network. Health Aff 2014; 33: 1178-86.
73. Chitnis X, Steventon A, Glaser A, et al. Use of health and social care by people with cancer. London: Nuffield Trust, 2014.
74. Saunders MK. People and places: in Denmark, big data goes to work. Health Aff 2014; 33: 1245.
75. Carlsen K, Harling H, Pedersen J, et al. The transition between work, sickness absence and pension in a cohort of Danish colorectal cancer survivors. Eur J Epidemiol 2012; 27: S29-30.
76. Sanders C, Rogers A, Bowen R, et al. Exploring barriers to participation and adoption of telehealth and telecare within the Whole System Demonstrator trial: a qualitative study. BMC Health Serv Res 2012; 12: 220.
77. Bardsley M, Steventon A, Smith J, et al. Evaluating integrated and community based care. London: Nuffield Trust, 2013.
78. Patton MQ. Utilization-focused evaluation. 4th edn. Thousand Oaks: SAGE Publications, Inc, 2008.
79. Bardsley M, Blunt I, Davies S, et al. Is secondary preventive care improving? Observational study of 10-year trends in emergency admissions for conditions amenable to ambulatory care. BMJ Open 2013; 3: e002007.
80. Wennberg JE. Forty years of unwarranted variation-And still counting. Health Policy 2014; 114: 1-2.
81. Guo J, Konetzka RT, Magett E, et al. Quantifying long-term care preferences. Med Decis Mak 2015; 35: 106-13.
82. Brown B, Williams R, Ainsworth J, et al. Missed Opportunities Mapping: Computable Healthcare Quality Improvement.
83. Quality and Outcomes Framework (QOF). http://qof.hscic.gov.uk/ (accessed 18 Feb 2015).
84. Gillam SJ, Siriwardena AN, Steel N. Pay-for-performance in the United Kingdom: impact of the quality and outcomes framework - a systematic review. Ann Fam Med 2012; 10: 461-8.
85. Morris L, Milne B. Enabling Patients to Access Electronic Health Records. 2010. http://www.rcgp.org.uk/Clinical-andresearch/Practice-management-resources/~/media/Files/Informatics/Health-Informatics-Enabling-Patient-Access.ashx
86. Cresswell KM, Sheikh A. Health information technology in hospitals: current issues. Futur Hosp 2015; 2: 50-6.
87. Goldenberg SD, French GL. Diagnostic testing for Clostridium difficile: a comprehensive survey of laboratories in England. J Hosp Infect 2011; 79: 4-7.
88. Tinati R, Halford S, Carr L, et al. Big data: methodological challenges and approaches for sociological analysis. Sociology 2014; 48: 663-81.
89. Fuller C, Robotham J, Savage J, et al. The national one week prevalence audit of universal meticillin-resistant Staphylococcus aureus (MRSA) admission screening 2012. PLoS ONE 2013; 8: e74219.
90. Boland MR, Hripcsak G, Shen Y, et al. Defining a comprehensive verotype using electronic health records for personalized medicine. J Am Med Inform Assoc 2013; 20: e232-8.
91. Rogers EM. Diffusion of innovations. 5th edn. New York: Free Press, 2003. doi:citeulike-article-id:126680
92. Roski J, Bo-Linn GW, Andrews TA. Creating value in health care through big data: Opportunities and policy implications. Health Aff 2014; 33: 1115-22.
93. Lewis GH. 'Impactibility models': identifying the subgroup of high-risk patients most amenable to hospital-avoidance programs. Milbank Q 2010; 88: 240-55.
94. Forster M, Pertile P. Optimal decision rules for HTA under uncertainty: a wider, dynamic perspective. Health Econ 2013; 22: 1507-14.
95. Craig P, Dieppe P, Macintyre S, et al. Developing and evaluating complex interventions: new guidance. London: Medical Research Council, 2008.
96. Parry GJ, Carson-Stevens A, Luff DF, et al. Recommendations for evaluation of health care improvement initiatives. Acad Pediatr 2013; 13: S23-30.
97. De Silva MJ, Breuer E, Lee L, et al. Theory of Change: a theory-driven approach to enhance the Medical Research Councils' framework for complex interventions. Trials 2014; 15: 267.
98. Frangakis CE. The calibration of treatment effects from clinical trials to target populations. Clin Trials 2009; 6: 136-40.
99. De Stavola BL, Daniel RM, Ploubidis GB, et al. Mediation analysis with intermediate confounding: structural equation modeling viewed through the causal inference lens. Am J Epidemiol 2015; 181: 64-80.
100. Hartman E, Grieve R, Ramsahai, R, et al. From sample average treatment effect to population average treatment effect on the treated: combining experimental with observational studies to estimate population treatment effects. J R Stat Soc. Series A (Statistics in Society). 2015. [epub ahead of print]. doi: 10.1111/rssa.12094
101. Academic Health Science Networks. http://www.England.nhs.uk/ourwork/part-rel/ahsn/ (accessed 23 Feb 2015).
102. Department of Health. Innovation health and wealth: accelerating adoption and diffusion in the NHS. Proc R Soc Med 2011; 55: 1-35.
103. Fihn SD, Francis J, Clancy C, et al. Insights from advanced analytics at the veterans health administration. Health Aff 2014; 33: 1203-11.
104. James BC, Savitz LA. How intermountain trimmed health care costs through robust quality improvement efforts. Health Aff 2011; 30: 1185-91.
105. Healthcare Evaluation Data. https://www.hed.nhs.uk/ (accessed 23 Feb 2015).
106. Allcock C, Dormon F, Taunt R, et al. Constructive comfort: accelerating change in the NHS. The Health Foundation, 2015. http://www.health.org.uk/public/cms/75/76/313/5504/Constructivecomfort-acceleratingchangeintheNHS.pdf?realName=Dz5g23.pdf
107. Carter P, Laurie GT, Dixon-woods M. The social licence for research: why care. data ran into trouble. J Med Ethics 2015.
108. Clark S, Weale A. Information Governance in Health: an analysis of the social values involved in data linkage studies. London: Nuffield Trust, 2011: 2-39.
109. Cameron D, Pope S, Clemences M. Exploring the public's views on using administrative data for research purposes. 2014. http://www.esrc.ac.uk/-images/Dialogue-on-Data-report-tcm8-30270.pdf
110. Department of Business Skills & Innovation. Seizing the data opportunity: a strategy for UK data capability. London, 2013.
111. Wikilife Foundation. Data Donors. http://datadonors.org/ (accessed 25 Mar 2015).
112. Open SNP. https://opensnp.org/ (accessed 23 Mar 2015).