EconPapers    
Economics at your fingertips  
 

Quantifying the Effects of Expert Selection and Elicitation Design on Experts’ Confidence in their Judgments about Future Energy Technologies

Gregory Nemet, Laura Diaz Anadon and Elena Verdolini

No 249349, MITP: Mitigation, Innovation and Transformation Pathways from Fondazione Eni Enrico Mattei (FEEM)

Abstract: Expert elicitations are frequently used to characterize future technology outcomes. However their usefulness is limited, in part because: estimates across studies are not easily comparable; choices in survey design and expert selection may bias results; and over-confidence is a persistent problem. We provide quantitative evidence of how these choices affect experts’ estimates of the costs of future energy technologies. We harmonize data from 19 elicitations, involving 215 experts, on the 2030 costs of 5 energy technologies: nuclear, biofuels, bioelectricity, solar, and carbon capture. We control for expert characteristics, survey design, and public R&D investment levels on which the elicited values are conditional. We find that, on average, when experts respond to elicitations in person, they ascribe lower confidence (larger uncertainty) to their estimates than when responding via mail or online. In-person interviews also produce more optimistic assessments of best-case (10th percentile) outcomes. The impacts of expert affiliation—government, private sector, or academic—and geography—US or EU—are also significant; academics and US experts have lower confidence than other types of experts. Higher R&D investment levels have no effect on the confidence of experts’ judgments. R&D reduces both the median and breakthrough (10th percentile) cost estimates, although the size of the effect varies across technologies. These results indicate the source, direction, and size of bias in energy technology elicitations. They also point to the technology specificity of some of the effects. These biases should be seriously considered, both in interpreting the results of existing elicitations and in designing new ones.

Keywords: Research; and; Development/Tech; Change/Emerging; Technologies (search for similar items in EconPapers)
Pages: 42
Date: 2016-11-04
New Economics Papers: this item is included in nep-ene and nep-ino
References: Add references at CitEc
Citations: View citations in EconPapers (4)

Downloads: (external link)
https://ageconsearch.umn.edu/record/249349/files/NDL2016-061.pdf (application/pdf)

Related works:
Journal Article: Quantifying the Effects of Expert Selection and Elicitation Design on Experts’ Confidence in Their Judgments About Future Energy Technologies (2017) Downloads
Working Paper: Quantifying the Effects of Expert Selection and Elicitation Design on Experts’ Confidence in their Judgments about Future Energy Technologies (2016) Downloads
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:ags:feemmi:249349

DOI: 10.22004/ag.econ.249349

Access Statistics for this paper

More papers in MITP: Mitigation, Innovation and Transformation Pathways from Fondazione Eni Enrico Mattei (FEEM) Contact information at EDIRC.
Bibliographic data for series maintained by AgEcon Search (aesearch@umn.edu).

 
Page updated 2025-01-16
Handle: RePEc:ags:feemmi:249349