Judgements about loyalty: A noise audit in the context of Swedish protective security vetting

Daniel Mats Olle Malmgren and Mats Dahl

Judgment and Decision Making, 2025, vol. 20, -

Abstract: Judging an individual’s loyalty in security-sensitive roles is a high-stakes task, yet little is known about the extent and sources of variability in such judgments. This study examined how 58 participants with experience in personnel security assessment evaluated applicant profiles connected to five different countries. Each participant reviewed five protocols and judged whether the case contained information relevant to a personnel security clearance, then rated the applicant’s loyalty on a 7-point scale. Using Bayesian probit regression and an ordinal item response model with hierarchical structure, we analyzed both binary judgments and rating patterns, accounting for country of connection, applicant gender, and participant-specific variability. Results revealed substantial between-participant variability (‘noise’) in how likely judges were to flag foreign ties as relevant. Pattern noise, reflecting idiosyncratic differences in how individuals interpret the same case, was evident in loyalty ratings. Connections to Brazil and Thailand were associated with systematically lower loyalty ratings and heightened disagreement between judges, reflecting both bias and pattern noise. Contrary to policy guidance, fewer than half of foreign connections were judged as relevant, and this tendency did not vary by participant gender. The findings underscore the risk of inconsistency in high-stakes assessments and highlight the need for structured conceptual calibration in personnel vetting.

Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.cambridge.org/core/product/identifier/ ... type/journal_article link to article abstract page (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:cup:judgdm:v:20:y:2025:i::p:-_44

Access Statistics for this article

More articles in Judgment and Decision Making from Cambridge University Press Cambridge University Press, UPH, Shaftesbury Road, Cambridge CB2 8BS UK.
Bibliographic data for series maintained by Kirk Stebbing ().