Human decision making balances reward maximization and policy compression

Lucy Lai and Samuel J Gershman

PLOS Computational Biology, 2024, vol. 20, issue 4, 1-32

Abstract: Policy compression is a computational framework that describes how capacity-limited agents trade reward for simpler action policies to reduce cognitive cost. In this study, we present behavioral evidence that humans prefer simpler policies, as predicted by a capacity-limited reinforcement learning model. Across a set of tasks, we find that people exploit structure in the relationships between states, actions, and rewards to “compress” their policies. In particular, compressed policies are systematically biased towards actions with high marginal probability, thereby discarding some state information. This bias is greater when there is redundancy in the reward-maximizing action policy across states, and increases with memory load. These results could not be explained qualitatively or quantitatively by models that did not make use of policy compression under a capacity limit. We also confirmed the prediction that time pressure should further reduce policy complexity and increase action bias, based on the hypothesis that actions are selected via time-dependent decoding of a compressed code. These findings contribute to a deeper understanding of how humans adapt their decision-making strategies under cognitive resource constraints.Author summary: Decision making taxes cognitive resources. For example, when shopping for groceries on a budget, we must evaluate which brand offers the best value for the price. But time constraints or mental fatigue can often steer us towards familiar choices, such as sticking to the same brand. To understand how cognitive resource limitations affect human decision making, we conducted a study in which we manipulated the number of optimal choices and the time limit within which choices were made. Across three tasks, we found that people utilize task structure to compress the amount of information factored into their decision making. Information compression biases people towards their past choices. This bias persists even when multiple optimal choices are available, and intensifies under cognitive load and time pressure. A computational model of decision making under cognitive constraints accurately describes the experimental data. Our findings may have the potential to inform the design of choice environments that better align with human decision biases.

Date: 2024
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1012057 (text/html)
https://journals.plos.org/ploscompbiol/article/fil ... 12057&type=printable (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:plo:pcbi00:1012057

DOI: 10.1371/journal.pcbi.1012057

Access Statistics for this article

More articles in PLOS Computational Biology from Public Library of Science
Bibliographic data for series maintained by ploscompbiol ().