Longitudinal Impact of Preference Biases on Recommender Systems’ Performance

Meizi Zhou (), Jingjing Zhang () and Gediminas Adomavicius ()
Additional contact information
Meizi Zhou: Boston University, Boston, Massachusetts 02215
Jingjing Zhang: Indiana University, Bloomington, Indiana 47405
Gediminas Adomavicius: University of Minnesota, Minneapolis, Minnesota 55455

Information Systems Research, 2024, vol. 35, issue 4, 1634-1656

Abstract: Research studies have shown that recommender systems’ predictions that are observed by users can cause biases in users’ postconsumption preference ratings. This can happen as part of the standard, normal system use, where biases are typically caused by the system’s inherent prediction errors (i.e., because of the less-than-perfect accuracy of recommendation methods). Because users’ preference ratings are typically fed back to the system as training data for future predictions, this process is likely to influence the performance of the system in the long run. We use a simulation approach to study the longitudinal impact of preference biases (and their magnitude) on the dynamics of recommender systems’ performance. Our simulation results show that preference biases significantly impair the system’s prediction performance (i.e., prediction accuracy) as well as users’ consumption outcomes (i.e., consumption relevance and diversity) over time. The impact is nonlinear to the size of the bias, that is, large bias causes disproportionately large negative effects. Also, items that are less popular and less distinctive (in terms of their content) are affected more by preference biases. Furthermore, given the impact of preference bias on the recommender systems’ performance, we explore the problem of debiasing user-submitted ratings. We empirically demonstrate that relying solely on historical rating data is unlikely to be effective in debiasing. We also propose and evaluate two debiasing approaches that take into account additional relevant information that can be collected by recommendation platforms. Our findings provide important implications for the design of recommender systems.

Keywords: recommender systems; longitudinal dynamics; preference biases; system performance; debiasing; agent-based simulation (search for similar items in EconPapers)
Date: 2024
References: Add references at CitEc
Citations:

Downloads: (external link)
http://dx.doi.org/10.1287/isre.2021.0133 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:orisre:v:35:y:2024:i:4:p:1634-1656

Access Statistics for this article

More articles in Information Systems Research from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().