CNN-Based Temporal Video Segmentation Using a Nonlinear Hyperbolic PDE-Based Multi-Scale Analysis

Tudor Barbu ()
Additional contact information
Tudor Barbu: Institute of Computer Science of Romanian Academy—Iasi Branch, 2 T. Codrescu Street, 700481 Iași, Romania

Mathematics, 2023, vol. 11, issue 1, 1-12

Abstract: An automatic temporal video segmentation framework is introduced in this article. The proposed cut detection technique performs a high-level feature extraction on the video frames, by applying a multi-scale image analysis approach combining nonlinear partial differential equations (PDE) to convolutional neural networks (CNN). A nonlinear second-order hyperbolic PDE model is proposed and its well-posedness is then investigated rigorously here. Its weak and unique solution is determined numerically applying a finite difference method-based numerical approximation algorithm that quickly converges to it. A scale-space representation is then created using that iterative discretization scheme. A CNN-based feature extraction is performed at each scale and the feature vectors obtained at multiple scales are concatenated into a final frame descriptor. The feature vector distance values between any two successive frames are then determined and the video transitions are identified next, by applying an automatic clustering scheme on these values. The proposed PDE model, its mathematical investigation and discretization, and the multi-scale analysis based on it represent the major contributions of this work. Some temporal segmentation experiments and method comparisons that illustrate the effectiveness of the proposed framework are finally described in this research paper.

Keywords: temporal video segmentation; cut detection; multi-scale analysis; nonlinear hyperbolic PDE model; finite difference method; numerical approximation algorithm; mathematical treatment; deep learning; convolutional neural networks; frame clustering (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
https://www.mdpi.com/2227-7390/11/1/245/pdf (application/pdf)
https://www.mdpi.com/2227-7390/11/1/245/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:11:y:2023:i:1:p:245-:d:1023542

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().