Serially Combining Epidemiological Designs Does Not Improve Overall Signal Detection in Vaccine Safety Surveillance

Faaizah Arshad, Martijn J. Schuemie, Fan Bu, Evan P. Minty, Thamir M. Alshammari, Lana Y. H. Lai, Talita Duarte-Salles, Stephen Fortin, Fredrik Nyberg, Patrick B. Ryan, George Hripcsak, Daniel Prieto-Alhambra and Marc A. Suchard ()
Additional contact information
Faaizah Arshad: University of California, Los Angeles
Martijn J. Schuemie: University of California, Los Angeles
Fan Bu: University of California, Los Angeles
Evan P. Minty: University of Calgary
Thamir M. Alshammari: King Saud University
Lana Y. H. Lai: University of Manchester
Talita Duarte-Salles: Fundació Institut Universitari per a la recerca a l’Atenció Primària de Salut Jordi Gol i Gurina (IDIAPJGol)
Stephen Fortin: Observational Health Data Analytics, Janssen R&D
Fredrik Nyberg: University of Gothenburg
Patrick B. Ryan: Observational Health Data Sciences and Informatics
George Hripcsak: Observational Health Data Sciences and Informatics
Daniel Prieto-Alhambra: University of Oxford
Marc A. Suchard: University of California, Los Angeles

Drug Safety, 2023, vol. 46, issue 8, No 8, 797-807

Abstract: Abstract Introduction Vaccine safety surveillance commonly includes a serial testing approach with a sensitive method for ‘signal generation’ and specific method for ‘signal validation.’ The extent to which serial testing in real-world studies improves or hinders overall performance in terms of sensitivity and specificity remains unknown. Methods We assessed the overall performance of serial testing using three administrative claims and one electronic health record database. We compared type I and II errors before and after empirical calibration for historical comparator, self-controlled case series (SCCS), and the serial combination of those designs against six vaccine exposure groups with 93 negative control and 279 imputed positive control outcomes. Results The historical comparator design mostly had fewer type II errors than SCCS. SCCS had fewer type I errors than the historical comparator. Before empirical calibration, the serial combination increased specificity and decreased sensitivity. Type II errors mostly exceeded 50%. After empirical calibration, type I errors returned to nominal; sensitivity was lowest when the methods were combined. Conclusion While serial combination produced fewer false-positive signals compared with the most specific method, it generated more false-negative signals compared with the most sensitive method. Using a historical comparator design followed by an SCCS analysis yielded decreased sensitivity in evaluating safety signals relative to a one-stage SCCS approach. While the current use of serial testing in vaccine surveillance may provide a practical paradigm for signal identification and triage, single epidemiological designs should be explored as valuable approaches to detecting signals.

Date: 2023
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s40264-023-01324-1 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:drugsa:v:46:y:2023:i:8:d:10.1007_s40264-023-01324-1

Ordering information: This journal article can be ordered from
http://www.springer.com/adis/journal/40264

DOI: 10.1007/s40264-023-01324-1

Access Statistics for this article

Drug Safety is currently edited by Nitin Joshi

More articles in Drug Safety from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().