Action subsampling supports policy compression in large action spaces

Shuze Liu and Samuel Joseph Gershman

PLOS Computational Biology, 2025, vol. 21, issue 9, 1-27

Abstract: Real-world decision-making often involves navigating large action spaces with state-dependent action values, taxing the limited cognitive resources at our disposal. While previous studies have explored cognitive constraints on generating action consideration sets or refining state-action mappings (policy complexity), their interplay remains underexplored. In this work, we present a resource-rational framework for policy compression that integrates both constraints, offering a unified perspective on decision-making under cognitive limitations. Through simulations, we characterize the suboptimality arising from reduced action consideration sets and reveal the complex interaction between policy complexity and action consideration set size in mitigating this suboptimality. We then use such normative insight to explain empirically observed phenomena in option generation, including the preferential sampling of generally valuable options and increased correlation in responses across contexts under cognitive load. We further validate the framework’s predictions through a contextual multi-armed bandit experiment, showing how humans flexibly adapt their action consideration sets and policy complexity to maintain near-optimality in a task-dependent manner. Our study demonstrates the importance of accounting for fine-grained resource constraints in understanding human cognition, and highlights the presence of adaptive metacognitive strategies even in simple tasks.Author summary: This study provides insight into how humans navigate the dual problem of deciding which actions to consider, and which actions to perform in particular contexts. It rationalizes previously observed tendencies of humans to sample generally valuable actions, explores the implication of changing either component of the dual problem on the reward yielded, and demonstrates the framework’s relevance through a human experiment. The study suggests that even in simple tasks, humans may still spontaneously simplify the original task to reduce their cognitive load.

Date: 2025
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1013444 (text/html)
https://journals.plos.org/ploscompbiol/article/fil ... 13444&type=printable (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:plo:pcbi00:1013444

DOI: 10.1371/journal.pcbi.1013444

Access Statistics for this article

More articles in PLOS Computational Biology from Public Library of Science
Bibliographic data for series maintained by ploscompbiol ().