Anomaly detection in consumer review analytics for idea generation in product innovation: Comparing machine learning and deep learning techniques

Xiling Cui, Zhongshan Zhu, Libo Liu, Qiang Zhou and Qiang Liu

Technovation, 2024, vol. 134, issue C

Abstract: With the development of big data analytics, consumers' online reviews are becoming increasingly useful for product innovation with hidden innovative ideas that can be extracted. However, these ideas may be only hidden in a small part of the massive reviews. This study aims to investigate the potential of using anomaly detection technology to identify unique reviews for more effective innovation generation. Three classical anomaly detection approaches (including both machine and deep learning) were explored, namely, isolation forest, density-based cluster analysis, and autoencoder methods. Using the consumer reviews on Dyson Vacuum cleaner from Xiaohongshu (one of the most popular social media platforms in China), we tested and compared the application of these three approaches in detecting innovation-relevant reviews. The results indicate that the two machine learning approaches, aka., density-based cluster analysis and isolation forest are too sensitive to the length of the reviews. The deep learning method, autoencoder, on the contrary, shows good stability and capability to detect the unique reviews from the whole dataset. Furthermore, the experts’ rating also confirms the effectiveness of autoencoder in identifying innovation-relevant reviews. Future studies and implications are then discussed.

Keywords: Product innovation; Machine learning; Deep learning; Anomaly detection; Online reviews (search for similar items in EconPapers)
Date: 2024
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0166497224000786
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:techno:v:134:y:2024:i:c:s0166497224000786

DOI: 10.1016/j.technovation.2024.103028

Access Statistics for this article

Technovation is currently edited by Jonathan Linton

More articles in Technovation from Elsevier
Bibliographic data for series maintained by Catherine Liu ().