Video summarization using event‐related potential responses to shot boundaries in real‐time video watching

Hyun Hee Kim and Yong Ho Kim

Journal of the Association for Information Science & Technology, 2019, vol. 70, issue 2, 164-175

Abstract: Our aim was to develop an event‐related potential (ERP)‐based method to construct a video skim consisting of key shots to bridge the semantic gap between the topic inferred from a whole video and that from its summary. Mayer's cognitive model was examined, wherein the topic integration process of a user evoked by a visual stimulus can be associated with long‐latency ERP components. We determined that long‐latency ERP components are suitable for measuring a user's neuronal response through a literature review. We hypothesized that N300 is specific to the categorization of all shots regardless of topic relevance, N400 is specific for the semantic mismatching process for topic‐irrelevant shots, and P600 is specific for the context updating process for topic‐relevant shots. In our experiment, the N400 component led to more negative ERP signals in response to topic‐irrelevant shots than to topic‐relevant shots and showed a fronto‐central scalp pattern. P600 elicited more positive ERP signals for topic‐relevant shots than for topic‐irrelevant shots and showed a fronto‐central scalp pattern. We used discriminant and artificial neural network (ANN) analyses to decode video shot relevance and observed that the ANN produced particularly high success rates: 91.3% from the training set and 100% from the test set.

Date: 2019
References: Add references at CitEc
Citations:

Downloads: (external link)
https://doi.org/10.1002/asi.24103

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bla:jinfst:v:70:y:2019:i:2:p:164-175

Ordering information: This journal article can be ordered from
http://www.blackwell ... bs.asp?ref=2330-1635

Access Statistics for this article

More articles in Journal of the Association for Information Science & Technology from Association for Information Science & Technology
Bibliographic data for series maintained by Wiley Content Delivery ().