The successor representation in human reinforcement learning

I. Momennejad (), E. M. Russek, J. H. Cheong, M. M. Botvinick, N. D. Daw and S. J. Gershman
Additional contact information
I. Momennejad: Princeton University
E. M. Russek: New York University
J. H. Cheong: Dartmouth College
M. M. Botvinick: University College London
N. D. Daw: Princeton University
S. J. Gershman: Harvard University

Nature Human Behaviour, 2017, vol. 1, issue 9, 680-692

Abstract: Abstract Theories of reward learning in neuroscience have focused on two families of algorithms thought to capture deliberative versus habitual choice. ‘Model-based’ algorithms compute the value of candidate actions from scratch, whereas ‘model-free’ algorithms make choice more efficient but less flexible by storing pre-computed action values. We examine an intermediate algorithmic family, the successor representation, which balances flexibility and efficiency by storing partially computed action values: predictions about future events. These pre-computation strategies differ in how they update their choices following changes in a task. The successor representation’s reliance on stored predictions about future states predicts a unique signature of insensitivity to changes in the task’s sequence of events, but flexible adjustment following changes to rewards. We provide evidence for such differential sensitivity in two behavioural studies with humans. These results suggest that the successor representation is a computational substrate for semi-flexible choice in humans, introducing a subtler, more cognitive notion of habit.

Date: 2017
References: Add references at CitEc
Citations: View citations in EconPapers (13)

Downloads: (external link)
https://www.nature.com/articles/s41562-017-0180-8 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:nat:nathum:v:1:y:2017:i:9:d:10.1038_s41562-017-0180-8

Ordering information: This journal article can be ordered from
https://www.nature.com/nathumbehav/

DOI: 10.1038/s41562-017-0180-8

Access Statistics for this article

Nature Human Behaviour is currently edited by Stavroula Kousta

More articles in Nature Human Behaviour from Nature
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().