External validity of multi-criteria preference data obtained from non-random sampling: measuring cohesiveness within and between groups

Saeideh Babashahi (), Paul Hansen () and Ronald Peeters
Additional contact information
Saeideh Babashahi: University of Sussex
Paul Hansen: University of Otago

Annals of Operations Research, 2023, vol. 325, issue 2, No 8, 939-949

Abstract: Abstract An important component of multi-criteria decision analysis (MCDA) in the public sector is the elicitation and aggregation of preference data collected via surveys into the relative importance of the criteria for the decision at hand. These aggregated preference data, usually in the form of mean weights on the criteria, are intended to represent the preferences of the relevant population overall. However, random sampling is often not feasible for public-sector MCDA for logistical reasons, including the expense involved in identifying and recruiting participants. Instead, non-random sampling methods such as convenience, purposive or snowball sampling are widely used. Nonetheless, provided the preference data collected are sufficiently ‘cohesive’ in terms of the extent to which the weights of the individuals belonging to the various exogenously defined groups in the sample are similar, non-random sampling can still produce externally valid aggregate preference data. We explain a method for measuring cohesiveness using the Kemeny and Hellinger distance measures, which involve measuring the ‘distance’ of participants’ weights (and the corresponding rankings of the criteria) from each other, within and between the groups respectively. As an illustration, these distance measures are applied to data from a MCDA to rank non-communicable diseases according to their overall burden to society. We conclude that the method is useful for evaluating the external validity of preference data obtained from non-random sampling.

Keywords: Multi-criteria decision analysis (MCDA); Non-random sampling; External validity; Distance measure; Preference cohesiveness; Cluster analysis (search for similar items in EconPapers)
Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s10479-022-05026-2 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:annopr:v:325:y:2023:i:2:d:10.1007_s10479-022-05026-2

Ordering information: This journal article can be ordered from
http://www.springer.com/journal/10479

DOI: 10.1007/s10479-022-05026-2

Access Statistics for this article

Annals of Operations Research is currently edited by Endre Boros

More articles in Annals of Operations Research from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().