Methods for safety signal detection in healthcare databases: a literature review

Expert Opinion on Drug Safety

Volumn 16, Issue 6, 2017, Pages 721-732

  Arnaud, Mickael ^a,b   Bégaud, Bernard ^a,b,c   Thurin, Nicolas ^a,b,d   Moore, Nicholas ^a,b,c,d   Pariente, Antoine ^a,b,c,d   Salvo, Francesco ^a,b,c  

^a UNIVERSITÉ DE BORDEAUX   (France)

^b INSERM U657   (France)

^c BORDEAUX UNIVERSITY HOSPITAL   (France)

^d Centre of Clinical Investigation Clinical Epidemiology   (France)

Author keywords

data mining; Drug safety; pharmacoepidemiology; pharmacovigilance; signal detection

Indexed keywords

CASE CONTROL STUDY; DATA MINING; HUMAN; PHARMACOEPIDEMIOLOGY; REVIEW; SAFETY; SIGNAL DETECTION; COMPARATIVE STUDY; DRUG SURVEILLANCE PROGRAM; DRUG-RELATED SIDE EFFECTS AND ADVERSE REACTIONS; FACTUAL DATABASE; MACHINE LEARNING; PROCEDURES; STATISTICS AND NUMERICAL DATA;

ADVERSE DRUG REACTION REPORTING SYSTEMS; DATABASES, FACTUAL; DRUG-RELATED SIDE EFFECTS AND ADVERSE REACTIONS; EPIDEMIOLOGIC RESEARCH DESIGN; HUMANS; MACHINE LEARNING; PHARMACOEPIDEMIOLOGY;

EID: 85019841639     PISSN: 14740338     EISSN: 1744764X     Source Type: Journal    
DOI: 10.1080/14740338.2017.1325463     Document Type: Review

References (82)

1. Moore N., The past, present and perhaps future of pharmacovigilance:homage to Folke Sjoqvist. Eur J Clin Pharmacol. 2013;69(S1):33–41.• A well-documented review presenting the history and future of safety signal detection.
2. Hauben M, Norén GN. A decade of data mining and still counting. Drug Saf. 2010;33(7):527–534.
3. Almenoff JS, Pattishall EN, Gibbs TG, et al. Novel statistical tools for monitoring the safety of marketed drugs. Clin Pharmacol Ther. 2007;82(2):157–166.
4. Bégaud B, Moride Y, Tubert-Bitter P, et al. False-positives in spontaneous reporting:should we worry about them? Br J Clin Pharmacol. 1994;38(5):401–404.
5. Olivier P, Montastruc J-L. The nature of the scientific evidence leading to drug withdrawals for pharmacovigilance reasons in France. Pharmacoepidemiol Drug Saf. 2006;15(11):808–812.
6. Paludetto M-N, Olivier-Abbal P, Montastruc J-L. Is spontaneous reporting always the most important information supporting drug withdrawals for pharmacovigilance reasons in France? Pharmacoepidemiol Drug Saf. 2012;21(12):1289–1294.
7. Bégaud B, Martin K, Haramburu F, et al. Rates of spontaneous reporting of adverse drug reactions in France. Jama. 2002;288(13):1588.
8. Moride Y, Haramburu F, Requejo AA, et al. Under-reporting of adverse drug reactions in general practice. Br J Clin Pharmacol. 1997;43(2):177–181.
9. Pariente A, Gregoire F, Fourrier-Reglat A, et al. Impact of safety alerts on measures of disproportionality in spontaneous reporting databases the notoriety bias. Drug Saf. 2007;30(10):891–898.
10. Hartnell NR, Wilson JP. Replication of the Weber effect using postmarketing adverse event reports voluntarily submitted to the United States Food and Drug Administration. Pharmacother J Hum Pharmacol Drug Ther. 2004;24(6):743–749.
11. Johnstone J, Nerenberg K, Loeb M. Meta-analysis:proton pump inhibitor use and the risk of community-acquired pneumonia. Aliment Pharmacol Ther. 2010 Mar 4;31(11):1165–1177.
12. Lambert AA, Lam JO, Paik JJ, et al. Risk of community-acquired pneumonia with outpatient proton-pump inhibitor therapy:a systematic review and meta-analysis. Plos One. 2015;10(6):e0128004.
13. Avorn J. Evaluating drug effects in the post-vioxx world:there must be a better way. Circulation. 2006;113(18):2173–2176.
14. Topol EJ. Failing the public health—rofecoxib, Merck, and the FDA. N Engl J Med. 2004;351(17):1707–1709.
15. Fontanarosa PB. Postmarketing surveillance—lack of vigilance, lack of trust. Jama. 2004;292(21):2647.
16. Arnaud M, Salvo F, Ahmed I, et al. A method for the minimization of competition bias in signal detection from spontaneous reporting databases. Drug Saf. 2016;39(3):251–260.
17. Ahmed I, Pariente A, Tubert-Bitter P. Class-imbalanced subsampling lasso algorithm for discovering adverse drug reactions. Stat Methods Med Res. 2016. 25april 2016. published online. DOI:10.1177/0962280216643116.
18. Juhlin K, Ye X, Star K, et al. Outlier removal to uncover patterns in adverse drug reaction surveillance - a simple unmasking strategy. Pharmacoepidemiol Drug Saf. 2013;22(10):1119–1129.
19. Maignen F, Hauben M, Hung E, et al. A conceptual approach to the masking effect of measures of disproportionality. Pharmacoepidemiol Drug Saf. 2014;23(2):208–217.
20. Schneeweiss S, Avorn J. A review of uses of health care utilization databases for epidemiologic research on therapeutics. J Clin Epidemiol. 2005;58(4):323–337.
21. Norén GN, Hopstadius J, Bate A, et al. Safety surveillance of longitudinal databases:methodological considerations. Pharmacoepidemiol Drug Saf. 2011;20(7):714–717.
22. Trifirò G, Fourrier-Reglat A, Sturkenboom MC, et al. The EU-ADR project:preliminary results and perspective. Stud Health Technol Inf. 2009;148:43–49.
23. Wisniewski AFZ, Bate A, Bousquet C, et al. Good signal detection practices:evidence from IMI PROTECT. Drug Saf. 2016;39(6):469–490.
24. Platt R, Carnahan RM, Brown JS, et al. The U.S. food and drug administration’s mini-sentinel program:status and direction. Pharmacoepidemiol Drug Saf. 2012;21:1–8.• A clear presentation of the objectives and the challenges existing when attempting to set up a post-market drug safety framework using the information available on observational health-care databases.
25. U.S. Food and Drug Administration. Mini-Sentinel. 2014. Available at:http://www.mini-sentinel.org [Last accessed 13December 2016]
26. Stang PE, Ryan PB, Racoosin JA, et al. Advancing the science for active surveillance:rationale and design for the observational medical outcomes partnership. Ann Intern Med. 2010;153(9):600–606.
27. Observational medical outcomes partnership. Available at:http://omop.org [Last accessed 13 December 2016]
28. AsPEN collaborators, Andersen M, Bergman U, Choi N-K, et al. The asian pharmacoepidemiology network (AsPEN):promoting multi-national collaboration for pharmacoepidemiologic research in Asia. Pharmacoepidemiol Drug Saf 2013;22(7):700–704.
29. Asian pharmacoepidemiological network. Available at:http://aspennet.asia/index.html [Last accessed 13 December 2016]
30. Zorych I, Madigan D, Ryan P, et al. Disproportionality methods for pharmacovigilance in longitudinal observational databases. Stat Methods Med Res. 2013;22(1):39–56.• Study presenting the different ways to apply the disproportionality methods on observational health-care databases.
31. Schuemie MJ, Coloma PM, Straatman H, et al. Using electronic health care records for drug safety signal detection:a comparative evaluation of statistical methods. Med Care. 2012;50(10):890–897.
32. Ryan PB, Schuemie MJ, Gruber S, et al. Empirical performance of a new user cohort method:lessons for developing a risk identification and analysis system. Drug Saf. 2013;36(S1):59–72.
33. Madigan D, Schuemie MJ, Ryan PB. Empirical performance of the case–control method:lessons for developing a risk identification and analysis system. Drug Saf. 2013;36(S1):73–82.
34. Suchard MA, Zorych I, Simpson SE, et al. Empirical performance of the self-controlled case series design:lessons for developing a risk identification and analysis system. Drug Saf. 2013;36(S1):83–93.
35. Murphy SN, Castro V, Colecchi J, et al. Partners healthcare OMOP study report. 2011. Available at:http://omop.org. [Last accessed 13December 2016].•• Study presenting the first OMOP experiment that implemented, tested, and compared numerous methods:disproportionality analysis, traditional, pharmacoepidemiological designs, TPD, and sequential statistical testing.
36. Ryan PB, Schuemie MJ, Madigan D. Empirical performance of a self-controlled cohort method:lessons for developing a risk identification and analysis system. Drug Saf. 2013;36(S1):95–106.
37. Hallas J. Evidence of depression provoked by cardiovascular medication:a prescription sequence symmetry analysis. Epidemiology. 1996;7(5):478–484.• Study conceptualizing the SSA method for signal detection.
38. Brown JS, Kulldorff M, Chan KA, et al. Early detection of adverse drug events within population-based health networks:application of sequential testing methods. Pharmacoepidemiol Drug Saf. 2007;16(12):1275–1284.
39. Kulldorff M, Davis RL, Kolczak† M, et al. A maximized sequential probability ratio test for drug and vaccine safety surveillance. Seq Anal. 2011;30(1):58–78.
40. Li L. A conditional sequential sampling procedure for drug safety surveillance. Stat Med. 2009;28(25):3124–3138.
41. Jin H, Chen J, Kelman C, et al. Mining unexpected associations for signalling potential adverse drug reactions from administrative health databases. Adv Knowl Discov Data Min. 2006;3918:867–876. Ng W-K, Kitsuregawa M, Li J, Chang K, editors.
42. Jin H, Chen J, He H, et al. Signaling potential adverse drug reactions from administrative health databases. IEEE Trans Knowl Data Eng. 2010;22(6):839–853.
43. Norén GN, Hopstadius J, Bate A, et al. Temporal pattern discovery in longitudinal electronic patient records. Data Min Knowl Discov. 2010;20(3):361–387.
44. Ji Y, Massanari RM, Ager J, et al. A fuzzy logic-based computational recognition-primed decision model. Inf Sci. 2007;177(20):4338–4353.• Study presenting an unusual approach based on a TAR algorithm combined with fuzzy logic.
45. Reps JM, Garibaldi JM, Aickelin U, et al. Signalling paediatric side effects using an ensemble of simple study designs. Drug Saf. 2014;37(3):163–170.
46. Reps JM, Garibaldi JM, Aickelin U, et al. A supervised adverse drug reaction signalling framework imitating Bradford Hill’s causality considerations. J Biomed Inform. 2015;56:356–368.•• Study presenting an interesting supervised machine-learning approach incorporating Bradford-Hill’s criteria.
47. Kulldorff M, Dashevsky I, Avery TR, et al. Drug safety data mining with a tree-based scan statistic. Pharmacoepidemiol Drug Saf. 2013;22(5):517–523.• Study presenting an original approach for discovering ADRs based on hierarchical trees:the tree-based scan statistic method.
48. Dumouchel W. Bayesian data mining in large frequency tables, with an application to the FDA spontaneous reporting system. Am Stat. 1999 Aug;53(3):177–190.
49. Bate A, Lindquist M, Edwards IR, et al. A Bayesian neural network method for adverse drug reaction signal generation. Eur J Clin Pharmacol. 1998;54(4):315–321.
50. Moore N, Kreft-Jais C, Haramburu F, et al. Reports of hypoglycaemia associated with the use of ACE inhibitors and other drugs:a case/non-case study in the French pharmacovigilance system database. Br J Clin Pharmacol. 1997 Nov;44(5):513–518.
51. Evans SJW, Waller PC, Davis S. Use of proportional reporting ratios (PRRs) for signal generation from spontaneous adverse drug reaction reports. Pharmacoepidemiol Drug Saf. 2001;10(6):483–486.
52. Curtis JR, Cheng H, Delzell E, et al. Adaptation of bayesian data mining algorithms to longitudinal claims data:coxib safety as an example. Med Care. 2008;46(9):969–975.
53. Choi N-K, Chang Y, Choi YK, et al. Signal detection of rosuvastatin compared to other statins:data-mining study using national health insurance claims database. Pharmacoepidemiol Drug Saf. 2010;19(3):238–246.
54. Choi N-K, Chang Y, Kim J-Y, et al. Comparison and validation of data-mining indices for signal detection:using the Korean national health insurance claims database. Pharmacoepidemiol Drug Saf. 2011;20(12):1278–1286.
55. Kim J, Kim M, Ha J-H, et al. Signal detection of methylphenidate by comparing a spontaneous reporting database with a claims database. Regul Toxicol Pharmacol. 2011;61(2):154–160.
56. Schuemie MJ. Methods for drug safety signal detection in longitudinal observational databases:LGPS and LEOPARD. Pharmacoepidemiol Drug Saf. 2011;20(3):292–299.
57. Ryan PB, Stang PE, Overhage JM, et al. A comparison of the empirical performance of methods for a risk identification system. Drug Saf. 2013;36:(S1):143–58.•• Study presenting the second OMOP experiment, which improved the methodology for implementing and assessing the methods.
58. Ryan PB, Schuemie MJ. Evaluating performance of risk identification methods through a large-scale simulation of observational data. Drug Saf. 2013;36(S1):171–180.
59. Schuemie MJ, Gini R, Coloma PM, et al. Replication of the OMOP experiment in Europe:evaluating methods for risk identification in electronic health record databases. Drug Saf. 2013;36(S1):159–169.•• Study presenting the EU-ADR experiment that implemented and assessed in European data the methods based on disproportionality analysis, traditional pharmacoepidemiological designs.
60. Tibshirani R. Regression shrinkage and selection via the Lasso. J R Stat Soc Ser B Methodol. 1996;58(1):267–288.
61. Petri H, De Vet HCW, Naus J, et al. Prescription sequence analysis:a new and fast method for assessing certain adverse reactions of prescription drugs in large populations. Stat Med. 1988;7(11):1171–1175.• The first study to have described the SSA, a smart method for safety signal detection in observational health-care databases.
62. Tsiropoulos I, Andersen M, Hallas J. Adverse events with use of antiepileptic drugs:a prescription and event symmetry analysis. Pharmacoepidemiol Drug Saf. 2009;18(6):483–491.
63. Cook AJ, Tiwari RC, Wellman RD, et al. Statistical approaches to group sequential monitoring of postmarket safety surveillance data:current state of the art for use in the mini-sentinel pilot. Pharmacoepidemiol Drug Saf. 2012;21:72–81.
64. Ji Y, Ying H, Dews P, et al. A potential causal association mining algorithm for screening adverse drug reactions in postmarketing surveillance. IEEE Trans Inf Technol Biomed. 2011;15(3):428–437.
65. Ji Y, Ying H, Tran J, et al. A method for mining infrequent causal associations and its application in finding adverse drug reaction signal pairs. IEEE Trans Knowl Data Eng. 2013;25(4):721–733.
66. Kulldorff M, Fang Z, Walsh SJ. A tree-based scan statistic for database disease surveillance. Biometrics. 2003;59(2):323–331.
67. Avillach P, Dufour J-C, Diallo G, et al. Design and validation of an automated method to detect known adverse drug reactions in MEDLINE:a contribution from the EU–ADR project. J Am Med Inform Assoc. 2013;20(3):446–452.
68. Coloma PM, Avillach P, Salvo F, et al. A reference standard for evaluation of methods for drug safety signal detection using electronic healthcare record databases. Drug Saf. 2013;36(1):13–23.
69. Ryan PB, Schuemie MJ, Welebob E, et al. Defining a reference set to support methodological research in drug safety. Drug Saf. 2013;36(S1):33–47.
70. Reps JM, Garibaldi JM, Aickelin U, et al. Comparison of algorithms that detect drug side effects using electronic healthcare databases. Soft Comput. 2013;17(12):2381–2397.•• Study presenting and comparing the performance for signal detection of the methods based on TAR algorithms.
71. Wahab IA, Pratt NL, Wiese MD, et al. The validity of sequence symmetry analysis (SSA) for adverse drug reaction signal detection. Pharmacoepidemiol Drug Saf. 2013;22(5):496–502.•• Study demonstrating that the SSA performed pretty well for safety signal detection.
72. Gruber S, Chakravarty A, Heckbert SR, et al. Design and analysis choices for safety surveillance evaluations need to be tuned to the specifics of the hypothesized drug-outcome association. Pharmacoepidemiol Drug Saf. 2016;25:973–981.
73. Gagne JJ, Schneeweiss S. Comment on “Empirical assessment of methods for risk identification in healthcare data:results from the experiments of the observational medical outcomes partnership.”. Stat Med. 2013;32(6):1073–1074.
74. Hennessy S, Leonard CE. Comment on:“Desideratum for evidence-based epidemiology.”. Drug Saf. 2015;38(1):101–103.
75. Hansen RA, Gray MD, Fox BI, et al. How well do various health outcome definitions identify appropriate cases in observational studies? Drug Saf. 2013;36(S1):27–32.
76. Pratt NL, Ilomäki J, Raymond C, et al. The performance of sequence symmetry analysis as a tool for post-market surveillance of newly marketed medicines:a simulation study. BMC Med Res Methodol. 2014;14(1):1.
77. Pratt N, Andersen M, Bergman U, et al. Multi-country rapid adverse drug event assessment:the Asian pharmacoepidemiology network (AsPEN) antipsychotic and acute hyperglycaemia study. Pharmacoepidemiol Drug Saf. 2013;22:915–924.
78. Pratt N, Chan EW, Choi N-K, et al. Prescription sequence symmetry analysis:assessing risk, temporality, and consistency for adverse drug reactions across datasets in five countries. Pharmacoepidemiol Drug Saf. 2015;24(8):858–864.
79. Wahab IA, Pratt NL, Ellett LK, et al. Sequence symmetry analysis as a signal detection tool for potential heart failure adverse events in an administrative claims database. Drug Saf. 2016;39(4):347–354.• Study demonstrating that the SSA method is able to identify unknown ADRs, which are supported by biological plausibility and case reports.
80. Blagus R, Lusa L. Class prediction for high-dimensional class-imbalanced data. BMC Bioinformatics. 2010;11(1):523.
81. The Uppsala Monitoring Centre. Practical Pharmacovigilance. Signal definition. 2013. Available at:http://www.who-umc.org [Last accessed 13December 2016]
82. Whitaker HJ, Paddy Farrington C, Spiessens B, et al. Tutorial in biostatistics:the self-controlled case series method. Stat Med. 2006;25(10):1768–1797.