Relation Extraction from Videos Based on IoT Intelligent Collaboration Framework

Jinna Lv, Qi Shen (), Mingzheng Lv and Lei Shi ()
Additional contact information
Jinna Lv: School of Information Management, Beijing Information Science and Technology University, Beijing 100192, China
Qi Shen: Teacher’s College, Beijing Union University, Beijing 100101, China
Mingzheng Lv: School of International Education, Shangqiu Normal University, Shangqiu 476000, China
Lei Shi: State Key Laboratory of Media Convergence and Communication, Communication University of China, Beijing 100024, China

Mathematics, 2022, vol. 10, issue 18, 1-20

Abstract: In the Internet of Things (IoT) era, various devices generate massive videos containing rich human relations. However, the long-distance transmission of huge videos may cause congestion and delays, and the large gap between the visual and relation spaces brings about difficulties for relation analysis. Hence, this study explores an edge-cloud intelligence framework and two algorithms for cooperative relation extraction and analysis from videos based on an IoT system. First, we exploit a cooperative mechanism on the edges and cloud, which can schedule the relation recognition and analysis subtasks from massive video streams. Second, we propose a Multi-Granularity relation recognition Model (MGM) based on coarse and fined granularity features. This means that better mapping is established for identifying relations more accurately. Specifically, we propose an entity graph based on Graph Convolutional Networks (GCN) with an attention mechanism, which can support comprehensive relationship reasoning. Third, we develop a Community Detection based on the Ensemble Learning model (CDEL), which leverages a heterogeneous skip-gram model to perform node embedding and detect communities. Experiments on SRIV datasets and four movie videos validate that our solution outperforms several competitive baselines.

Keywords: edge computing; relation recognition; deep learning; video analysis (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2022
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/10/18/3308/pdf (application/pdf)
https://www.mdpi.com/2227-7390/10/18/3308/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:10:y:2022:i:18:p:3308-:d:912869

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().