Don’t Mention It? Analyzing User-Generated Content Signals for Early Adverse Event Warnings

Ahmed Abbasi (), Jingjing Li (), Donald Adjeroh (), Marie Abate () and Wanhong Zheng ()
Additional contact information
Ahmed Abbasi: Information Technology Area and Center for Business Analytics, McIntire School of Commerce, University of Virginia, Charlottesville, Virginia 22904
Jingjing Li: Information Technology Area and Center for Business Analytics, McIntire School of Commerce, University of Virginia, Charlottesville, Virginia 22904
Donald Adjeroh: Lane Department of Computer Science and Electrical Engineering, West Virginia University, Morgantown, West Virginia 26506
Marie Abate: Center for Drug & Health Information, Department of Clinical Pharmacy, School of Pharmacy, West Virginia University, Morgantown, West Virginia 26506
Wanhong Zheng: School of Medicine, Robert C. Byrd Health Sciences Center, West Virginia University, Morgantown, West Virginia 26505

Information Systems Research, 2019, vol. 30, issue 3, 1007-1028

Abstract: With greater impetus on broad postmarket surveillance, the Voice of the Customer (VoC) has emerged as an important source of information for understanding consumer experiences and identifying potential issues. In organizations, risk management groups are increasingly interested in working with their information technology teams to develop robust VoC listening platforms. Two key challenges have impeded success. First, prior work has leveraged diverse sets of channels, adverse event types, and modeling methods, resulting in diverging conclusions regarding the viability and efficacy of various user-generated channels and accompanying modeling methods. Second, many existing detection methods rely on “mention models” that have low detection rates, have high false positives, and lack timeliness. Following the information systems design science approach, in this research note we propose a framework for examining key design elements for VoC listening platforms. As part of our framework, we also develop a novel heuristic-based method for detecting adverse events. We evaluate our framework and method on two large test beds each encompassing millions of tweets, forums postings, and search query logs pertaining to hundreds of adverse events related to the pharmaceutical and automotive industries. The results shed light on the interplay between user-generated channels and event types, as well as the potential for more robust event modeling methods that go beyond basic mention models. Our analysis framework reveals that user-generated content channels can facilitate timelier detection of adverse events: on average, two to three years or earlier than commonly used databases. The inclusion of negative sentiment polarity in the models can further reduce false-positive rates. Additionally, we find social media channels provide higher detection rates but lower precision than do search-based signals. The search and web forum channels are timelier than Twitter. The proposed heuristic-based method attains markedly better results than do existing methods—with earlier detection rates of 50%–80% and far fewer false positives across an array of VoC channels and event types. The heuristic method is also well suited for signal fusion across channels. Our note makes several contributions to research. The results also have important implications for various practitioner groups, including regulatory agencies and risk management teams at product manufacturing firms.

Keywords: signal detection; social media; data mining; smart health; predictive analytics; healthcare analytics; healthcare information technologies (search for similar items in EconPapers)
Date: 2019
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (5)

Downloads: (external link)
https://doi.org/10.1287/isre.2019.0847 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:orisre:v:30:y:2019:i:3:p:1007-1028

Access Statistics for this article

More articles in Information Systems Research from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().