Data Mining in Drug Safety: Review of Published Threshold Criteria for Defining Signals of Disproportionate Reporting

Data mining is used in pharmacovigilance as an adjunct to traditional pharmacovigilance practices. There remains ongoing debate as to the impact automated signal detection would have on pharmacovigilance resources. An important component of this debate is the value of each statistical alert or signal of disproportional reporting (SDR) and the resources needed to evaluate SDRs that are clinically unimportant. Using the terminology of diagnostic testing, such SDRs are called false positives as they are statistically positive but are clinically negative. Based on the clinical testing paradigm, a more stringent threshold increases the sensitivity of the test by lowering the number of false positives; however, the trade off of increased sensitivity is a reduced specificity, i.e. potentially missing clinically relevant problems.

In developing the protocol to assess the clinical validity of an SDR, a literature search was conducted to determine what threshold(s) were commonly used for data mining adverse event databases. Of the more than 100 manuscripts identified, 41 published the results of data mining excursions with a clearly identified threshold for significance. The commonly used data mining algorithms were proportional reporting ratio (PRR), reporting odds ratio (ROR), multi-item gamma Poisson shrinker (MGPS) and Bayesian confidence propagation neural network ...................&more