Using meta-predictions to identify experts in the crowd when past performance is unknown

Marcellin Martinie, Tom Wilkening and Piers D L Howe

PLOS ONE, 2020, vol. 15, issue 4, 1-11

Abstract: A common approach to improving probabilistic forecasts is to identify and leverage the forecasts from experts in the crowd based on forecasters’ performance on prior questions with known outcomes. However, such information is often unavailable to decision-makers on many forecasting problems, and thus it can be difficult to identify and leverage expertise. In the current paper, we propose a novel algorithm for aggregating probabilistic forecasts using forecasters’ meta-predictions about what other forecasters will predict. We test the performance of an extremised version of our algorithm against current forecasting approaches in the literature and show that our algorithm significantly outperforms all other approaches on a large collection of 500 binary decision problems varying in five levels of difficulty. The success of our algorithm demonstrates the potential of using meta-predictions to leverage latent expertise in environments where forecasters’ expertise cannot otherwise be easily identified.

Date: 2020
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0232058 (text/html)
https://journals.plos.org/plosone/article/file?id= ... 32058&type=printable (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:plo:pone00:0232058

DOI: 10.1371/journal.pone.0232058

Access Statistics for this article

More articles in PLOS ONE from Public Library of Science
Bibliographic data for series maintained by plosone ().