A Conceptual Probabilistic Framework for Annotation Aggregation of Citizen Science Data

Jesus Cerquides, Mehmet Oğuz Mülâyim, Jerónimo Hernández-González, Amudha Ravi Shankar and Jose Luis Fernandez-Marquez
Additional contact information
Jesus Cerquides: Institut d’Investigació en Intel ligència Artificial (IIIA), CSIC, 08193 Cerdanyola, Spain
Mehmet Oğuz Mülâyim: Institut d’Investigació en Intel ligència Artificial (IIIA), CSIC, 08193 Cerdanyola, Spain
Jerónimo Hernández-González: Department de Matemàtiques, Universitat de Barcelona, 08007 Barcelona, Spain
Amudha Ravi Shankar: Citizen Cyberlab, CUI, University of Geneva, CH-1227 Geneva, Switzerland
Jose Luis Fernandez-Marquez: Citizen Cyberlab, CUI, University of Geneva, CH-1227 Geneva, Switzerland

Mathematics, 2021, vol. 9, issue 8, 1-15

Abstract: Over the last decade, hundreds of thousands of volunteers have contributed to science by collecting or analyzing data. This public participation in science, also known as citizen science, has contributed to significant discoveries and led to publications in major scientific journals. However, little attention has been paid to data quality issues. In this work we argue that being able to determine the accuracy of data obtained by crowdsourcing is a fundamental question and we point out that, for many real-life scenarios, mathematical tools and processes for the evaluation of data quality are missing. We propose a probabilistic methodology for the evaluation of the accuracy of labeling data obtained by crowdsourcing in citizen science. The methodology builds on an abstract probabilistic graphical model formalism, which is shown to generalize some already existing label aggregation models. We show how to make practical use of the methodology through a comparison of data obtained from different citizen science communities analyzing the earthquake that took place in Albania in 2019.

Keywords: data quality; citizen science; consensus models (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2021
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/9/8/875/pdf (application/pdf)
https://www.mdpi.com/2227-7390/9/8/875/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:9:y:2021:i:8:p:875-:d:536894

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().