BargCrEx: A System for Bargaining Based Aggregation of Crowd and Expert Opinions in Crowdsourcing

Ana Vukicevic (), Milan Vukicevic (), Sandro Radovanovic () and Boris Delibasic ()
Additional contact information
Ana Vukicevic: University of Belgrade
Milan Vukicevic: University of Belgrade
Sandro Radovanovic: University of Belgrade
Boris Delibasic: University of Belgrade

Group Decision and Negotiation, 2022, vol. 31, issue 4, No 4, 789-818

Abstract: Abstract Crowdsourcing and crowd voting systems are being increasingly used in societal, industry, and academic problems (labeling, recommendations, social choice, etc.) due to their possibility to exploit “wisdom of crowd” and obtain good quality solutions, and/or voter satisfaction, with high cost-efficiency. However, the decisions based on crowd vote aggregation do not guarantee high-quality results due to crowd voter data quality. Additionally, such decisions often do not satisfy the majority of voters due to data heterogeneity (multimodal or uniform vote distributions) and/or outliers, which cause traditional aggregation procedures (e.g., central tendency measures) to propose decisions with low voter satisfaction. In this research, we propose a system for the integration of crowd and expert knowledge in a crowdsourcing setting with limited resources. The system addresses the problem of sparse voting data by using machine learning models (matrix factorization and regression) for the estimation of crowd and expert votes/grades. The problem of vote aggregation under multimodal or uniform vote distributions is addressed by the inclusion of expert votes and aggregation of crowd and expert votes based on optimization and bargaining models (Kalai–Smorodinsky and Nash) usually used in game theory. Experimental evaluation on real world and artificial problems showed that the bargaining-based aggregation outperforms the traditional methods in terms of cumulative satisfaction of experts and crowd. Additionally, the machine learning models showed satisfactory predictive performance and enabled cost reduction in the process of vote collection.

Keywords: Crowd-voting; Expert knowledge; Matrix-factorisation; Machine learning; Bargaining models (search for similar items in EconPapers)
Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
http://link.springer.com/10.1007/s10726-022-09783-0 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:grdene:v:31:y:2022:i:4:d:10.1007_s10726-022-09783-0

Ordering information: This journal article can be ordered from
http://www.springer.com/journal/10726/PS2

DOI: 10.1007/s10726-022-09783-0

Access Statistics for this article

Group Decision and Negotiation is currently edited by Gregory E. Kersten

More articles in Group Decision and Negotiation from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().