Deep variational Luenberger-type observer with dynamic objects channel-attention for stochastic video prediction

Dong Wang, Chuanbo Wen and Xiaoliang Feng

International Journal of Systems Science, 2024, vol. 55, issue 4, 728-740

Abstract: Considering the inherent stochasticity and uncertainty, predicting future video frames is exceptionally challenging. In this work, we study the problem of video prediction by combining interpretability of stochastic state space models and representation learning of deep neural networks with channel-attention. Our model which based on a variational encoder introduces a novel channel-attention block to enhance the ability to capture dynamic objects in video frames and a Luenberger-type observer which catches the dynamic evolution of the latent features. These enable the features of moving objects more conspicuous and decomposition of videos into static features and dynamics in an unsupervised manner. By deriving the stability theory of the nonlinear Luenberger-type observer, the hidden states in the feature space become insensitive with respect to the initial values, which improves the robustness of the overall model. Furthermore, the variational lower bound on the data log-likelihood can be derived to obtain the tractable posterior prediction distribution based on the variational principle. Finally, the experiments including the Rotaton Numbers dataset and the Kth dataset are provided to demonstrate the proposed model outperforms concurrent works.

Date: 2024
References: Add references at CitEc
Citations:

Downloads: (external link)
http://hdl.handle.net/10.1080/00207721.2023.2293686 (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:taf:tsysxx:v:55:y:2024:i:4:p:728-740

Ordering information: This journal article can be ordered from
http://www.tandfonline.com/pricing/journal/TSYS20

DOI: 10.1080/00207721.2023.2293686

Access Statistics for this article

International Journal of Systems Science is currently edited by Visakan Kadirkamanathan

More articles in International Journal of Systems Science from Taylor & Francis Journals
Bibliographic data for series maintained by Chris Longhurst ().