Predictive learning as a network mechanism for extracting low-dimensional latent space representations

Stefano Recanatesi (), Matthew Farrell, Guillaume Lajoie, Sophie Deneve, Mattia Rigotti and Eric Shea-Brown
Additional contact information
Stefano Recanatesi: University of Washington Center for Computational Neuroscience and Swartz Center for Theoretical Neuroscience
Matthew Farrell: University of Washington
Guillaume Lajoie: Université de Montréal
Sophie Deneve: Group for Neural Theory, Ecole Normal Superieur
Mattia Rigotti: IBM Research AI
Eric Shea-Brown: University of Washington Center for Computational Neuroscience and Swartz Center for Theoretical Neuroscience

Nature Communications, 2021, vol. 12, issue 1, 1-13

Abstract: Abstract Artificial neural networks have recently achieved many successes in solving sequential processing and planning tasks. Their success is often ascribed to the emergence of the task’s low-dimensional latent structure in the network activity – i.e., in the learned neural representations. Here, we investigate the hypothesis that a means for generating representations with easily accessed low-dimensional latent structure, possibly reflecting an underlying semantic organization, is through learning to predict observations about the world. Specifically, we ask whether and when network mechanisms for sensory prediction coincide with those for extracting the underlying latent variables. Using a recurrent neural network model trained to predict a sequence of observations we show that network dynamics exhibit low-dimensional but nonlinearly transformed representations of sensory inputs that map the latent structure of the sensory environment. We quantify these results using nonlinear measures of intrinsic dimensionality and linear decodability of latent variables, and provide mathematical arguments for why such useful predictive representations emerge. We focus throughout on how our results can aid the analysis and interpretation of experimental data.

Date: 2021
References: Add references at CitEc
Citations: View citations in EconPapers (3)

Downloads: (external link)
https://www.nature.com/articles/s41467-021-21696-1 Abstract (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:nat:natcom:v:12:y:2021:i:1:d:10.1038_s41467-021-21696-1

Ordering information: This journal article can be ordered from
https://www.nature.com/ncomms/

DOI: 10.1038/s41467-021-21696-1

Access Statistics for this article

Nature Communications is currently edited by Nathalie Le Bot, Enda Bergin and Fiona Gillespie

More articles in Nature Communications from Nature
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().