Forecast of environment systems using expert judgements: performance comparison between the possibilistic and the classical model

Jeremy Rohmer () and Eric Chojnacki
Additional contact information
Jeremy Rohmer: BRGM
Eric Chojnacki: PSN-RES/SEMIA/LSMA, Cadarache

Environment Systems and Decisions, 2021, vol. 41, issue 1, 131-146

Abstract: Abstract Expert judgment is widely used to inform forecasts (e.g. using the 5th, 50th and 95th percentile of some variable of interest) for a large variety of applications related to environment systems. This task can rely on Cooke’s classical model (CM) within the probabilistic framework, and consists in combining expert information after a preliminary step where experts are weighted using calibration and informativeness scores estimated using some seed questions for which the answers can be obtained. In the literature, an alternative model (PM) has been proposed using a different framework to process the information supplied by experts, namely possibility theory. In the present study, we assess whether both models perform similarly when the seed questions are different from those used to determine the scores, i.e. by taking the viewpoint of forecast. Using an extensive out-of-sample validation procedure, two aspects are investigated using 33 expert datasets: (1) robustness to the set of calibration questions used to estimate the scores, i.e. whether the best and worst performing expert differs; (2) forecast performance, i.e. the degree of accuracy and informativeness of the derived forecast intervals. Regarding (1), the validation procedure shows that PM is less sensitive. Regarding (2), PM achieves more accuracy but with less informativeness when the averaging operator is used. Interestingly, the differences with CM only remain of moderate magnitude for the considered cases despite the conceptual dissimilarities of both models and their lack of agreement on the selection of the best performing expert.

Keywords: Expert calibration; Forecast accuracy; Informativeness; Robustness; Probability; Possibility (search for similar items in EconPapers)
Date: 2021
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
http://link.springer.com/10.1007/s10669-020-09794-9 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:envsyd:v:41:y:2021:i:1:d:10.1007_s10669-020-09794-9

Ordering information: This journal article can be ordered from
https://www.springer.com/journal/10669

DOI: 10.1007/s10669-020-09794-9

Access Statistics for this article

More articles in Environment Systems and Decisions from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().