Alignment of brain embeddings and artificial contextual embeddings in natural language points to common geometric patterns

Ariel Goldstein (), Avigail Grinstein-Dabush, Mariano Schain, Haocheng Wang, Zhuoqiao Hong, Bobbi Aubrey, Samuel A. Nastase, Zaid Zada, Eric Ham, Amir Feder, Harshvardhan Gazula, Eliav Buchnik, Werner Doyle, Sasha Devore, Patricia Dugan, Roi Reichart, Daniel Friedman, Michael Brenner, Avinatan Hassidim, Orrin Devinsky, Adeen Flinker and Uri Hasson
Additional contact information
Ariel Goldstein: Hebrew University
Avigail Grinstein-Dabush: Google Research
Mariano Schain: Google Research
Haocheng Wang: Princeton University
Zhuoqiao Hong: Princeton University
Bobbi Aubrey: Princeton University
Samuel A. Nastase: Princeton University
Zaid Zada: Princeton University
Eric Ham: Princeton University
Amir Feder: Google Research
Harshvardhan Gazula: Princeton University
Eliav Buchnik: Google Research
Werner Doyle: New York University Grossman School of Medicine
Sasha Devore: New York University Grossman School of Medicine
Patricia Dugan: New York University Grossman School of Medicine
Roi Reichart: Israel Institute of Technology
Daniel Friedman: New York University Grossman School of Medicine
Michael Brenner: Google Research
Avinatan Hassidim: Google Research
Orrin Devinsky: New York University Grossman School of Medicine
Adeen Flinker: New York University Grossman School of Medicine
Uri Hasson: Google Research

Nature Communications, 2024, vol. 15, issue 1, 1-12

Abstract: Abstract Contextual embeddings, derived from deep language models (DLMs), provide a continuous vectorial representation of language. This embedding space differs fundamentally from the symbolic representations posited by traditional psycholinguistics. We hypothesize that language areas in the human brain, similar to DLMs, rely on a continuous embedding space to represent language. To test this hypothesis, we densely record the neural activity patterns in the inferior frontal gyrus (IFG) of three participants using dense intracranial arrays while they listened to a 30-minute podcast. From these fine-grained spatiotemporal neural recordings, we derive a continuous vectorial representation for each word (i.e., a brain embedding) in each patient. Using stringent zero-shot mapping we demonstrate that brain embeddings in the IFG and the DLM contextual embedding space have common geometric patterns. The common geometric patterns allow us to predict the brain embedding in IFG of a given left-out word based solely on its geometrical relationship to other non-overlapping words in the podcast. Furthermore, we show that contextual embeddings capture the geometry of IFG embeddings better than static word embeddings. The continuous brain embedding space exposes a vector-based neural code for natural language processing in the human brain.

Date: 2024
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.nature.com/articles/s41467-024-46631-y Abstract (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:nat:natcom:v:15:y:2024:i:1:d:10.1038_s41467-024-46631-y

Ordering information: This journal article can be ordered from
https://www.nature.com/ncomms/

DOI: 10.1038/s41467-024-46631-y

Access Statistics for this article

Nature Communications is currently edited by Nathalie Le Bot, Enda Bergin and Fiona Gillespie

More articles in Nature Communications from Nature
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().