Should an Analyst Share Calibration Information with Experts?

Saurabh Bansal (), Mostafa Sabbaghi () and Rashmi Sharma ()
Additional contact information
Saurabh Bansal: The Pennsylvania State University
Mostafa Sabbaghi: The Pennsylvania State University
Rashmi Sharma: The Pennsylvania State University

A chapter in Behavioral Decision Analysis, 2024, pp 81-103 from Springer

Abstract: Abstract Many times, expert judgment is used in a two-step procedure: (i) obtain judgments for calibration quantities for which true/empirical values are available to calibrate the expert’s judgmental errors, and then (ii) obtain judgments for focal quantities (quantities of interest but without historical data), adjust them using the calibration information, and then use the adjusted judgments to assist decision making. In such situations, should a decision analyst share the calibration information with the expert before the expert provides judgments for focal quantities? We answer this question using laboratory experiments. We specifically investigate the role of task complexity, numeracy, and self-awareness on the use of calibration information by the expert. We find that: (a) Expert judgment tends to be of a worse quality as task complexity increases, (b) Calibration feedback does not improve managerial judgments in a less complex task, but it does reduce the bias and (especially) noise in a more complex task, (c) Numeracy does not impact the use of calibration information by experts regardless of task complexity, and (d) Individuals are able to directionally discern whether they are doing well in less complex tasks, but not in more complex tasks. As such these results suggest that when faced with complex tasks experts do benefit from receiving calibration information. However, this benefit comes at the expense of rendering this information inapplicable for the decision analyst to adjust the focal judgments provided by an expert. In contrast, experts do not benefit from receiving calibration information for simple tasks, allowing decision analysts to continue to use the calibration information to adjust managerial judgments.

Keywords: Expert judgement; Calibration; Task complexity; Numeracy; Self-awareness (search for similar items in EconPapers)
Date: 2024
References: Add references at CitEc
Citations:

There are no downloads for this item, see the EconPapers FAQ for hints about obtaining it.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:isochp:978-3-031-44424-1_5

Ordering information: This item can be ordered from
http://www.springer.com/9783031444241

DOI: 10.1007/978-3-031-44424-1_5

Access Statistics for this chapter

More chapters in International Series in Operations Research & Management Science from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().