Traversal direction changes priorities in multiple criteria decision analysis

Sajid Siraj, Barbara Summers, Simon Thomas van Baal and Valerie Belton
Additional contact information
Simon Thomas van Baal: University of Leeds

No s5tgm_v1, SocArXiv from Center for Open Science

Abstract: Hierarchical structuring is routine in multi-criteria decision analysis (MCDA), and the direction in which decision-makers traverse the hierarchy is usually treated as a matter of analyst convenience. Weights can be elicited top-down, from broad objectives to their sub-criteria, or bottom-up, from the detailed criteria upwards. We test whether this choice really is inconsequential. In a within-participant experiment, 1,070 participants in three countries evaluated the same four alternatives under both approaches. We compare the approaches on the weights they produce and on how well their scores match participants' own informed preferences. Direction had a small effect on fit (d_z = -0.174): top-down scores were slightly closer, but bottom-up fitted better for 43% of participants. Direction had a large effect on the weights (d_z = +0.669): bottom-up elicitation moved weight towards the branches divided into more sub-criteria. Participants rated the two procedures as equally useful, intuitive, easy and enjoyable, yet chose the better-fitting one more often than chance. None of the psychological measures we collected predicted which approach suited a given person, and neither approach reproduced participants' informed top choice more than 40% of the time, against a 25% chance benchmark. Traversal direction therefore matters less for the decision reached than for the explanation of priorities that comes with it. Since the weights are what an MCDA study records, communicates and defends, we argue that they should be reported together with the hierarchy and elicitation level that produced them.

Date: 2026-11-05
References: Add references at CitEc
Citations:

Downloads: (external link)
https://osf.io/download/6aeb84a13df8171266ca345a/

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:osf:socarx:s5tgm_v1

DOI: 10.31235/osf.io/s5tgm_v1

Access Statistics for this paper

More papers in SocArXiv from Center for Open Science
Bibliographic data for series maintained by OSF ().