Images with harder-to-reconstruct visual representations leave stronger memory traces

Qi Lin (), Zifan Li, John Lafferty () and Ilker Yildirim ()
Additional contact information
Qi Lin: Yale University
Zifan Li: Yale University
John Lafferty: Yale University
Ilker Yildirim: Yale University

Nature Human Behaviour, 2024, vol. 8, issue 7, 1309-1320

Abstract: Abstract Much of what we remember is not because of intentional selection, but simply a by-product of perceiving. This raises a foundational question about the architecture of the mind: how does perception interface with and influence memory? Here, inspired by a classic proposal relating perceptual processing to memory durability, the level-of-processing theory, we present a sparse coding model for compressing feature embeddings of images, and show that the reconstruction residuals from this model predict how well images are encoded into memory. In an open memorability dataset of scene images, we show that reconstruction error not only explains memory accuracy, but also response latencies during retrieval, subsuming, in the latter case, all of the variance explained by powerful vision-only models. We also confirm a prediction of this account with ‘model-driven psychophysics’. This work establishes reconstruction error as an important signal interfacing perception and memory, possibly through adaptive modulation of perceptual processing.

Date: 2024
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.nature.com/articles/s41562-024-01870-3 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:nat:nathum:v:8:y:2024:i:7:d:10.1038_s41562-024-01870-3

Ordering information: This journal article can be ordered from
https://www.nature.com/nathumbehav/

DOI: 10.1038/s41562-024-01870-3

Access Statistics for this article

Nature Human Behaviour is currently edited by Stavroula Kousta

More articles in Nature Human Behaviour from Nature
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().