Decoding dynamic affective responses to naturalistic videos with shared neural patterns

Hang-Yee Chan, Ale Smidts, Vincent C. Schoots, Alan G. Sanfey and Maarten A. S. Boksem
Additional contact information
Hang-Yee Chan: Erasmus University Rotterdam
Ale Smidts: Erasmus University Rotterdam
Vincent C. Schoots: Erasmus University Rotterdam
Alan G. Sanfey: Radboud University [Nijmegen]
Maarten A. S. Boksem: Erasmus University Rotterdam

Post-Print from HAL

Abstract: This study explored the feasibility of using shared neural patterns from brief affective episodes (viewing affective pictures) to decode extended, dynamic affective sequences in a naturalistic experience (watching movie-trailers). Twenty-eight participants viewed pictures from the International Affective Picture System (IAPS) and, in a separate session, watched various movie-trailers. We first located voxels at bilateral occipital cortex (LOC) responsive to affective picture categories by GLM analysis, then performed between-subject hyperalignment on the LOC voxels based on their responses during movie-trailer watching. After hyperalignment, we trained between-subject machine learning classifiers on the affective pictures, and used the classifiers to decode affective states of an out-of-sample participant both during picture viewing and during movie-trailer watching. Within participants, neural classifiers identified valence and arousal categories of pictures, and tracked self-reported valence and arousal during video watching. In aggregate, neural classifiers produced valence and arousal time series that tracked the dynamic ratings of the movie-trailers obtained from a separate sample. Our findings provide further support for the possibility of using pre-trained neural representations to decode dynamic affective responses during a naturalistic experience.

Keywords: cognitive; neuroscience (search for similar items in EconPapers)
Date: 2020-08-01
Note: View the original document on HAL open archive server: https://hal.science/hal-03188208
References: View references in EconPapers View complete reference list from CitEc
Citations:

Published in NeuroImage, 2020, 216, ⟨10.1016/j.neuroimage.2020.116618⟩

Downloads: (external link)
https://hal.science/hal-03188208/document (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:hal:journl:hal-03188208

DOI: 10.1016/j.neuroimage.2020.116618

Access Statistics for this paper

More papers in Post-Print from HAL
Bibliographic data for series maintained by CCSD ().