Arbitrary Timestep Video Frame Interpolation with Time-Dependent Decoding

Haokai Zhang, Dongwei Ren (), Zifei Yan and Wangmeng Zuo
Additional contact information
Haokai Zhang: Faculty of Computing, Harbin Institute of Technology, Harbin 150001, China
Dongwei Ren: Faculty of Computing, Harbin Institute of Technology, Harbin 150001, China
Zifei Yan: Faculty of Computing, Harbin Institute of Technology, Harbin 150001, China
Wangmeng Zuo: Faculty of Computing, Harbin Institute of Technology, Harbin 150001, China

Mathematics, 2024, vol. 12, issue 2, 1-13

Abstract: Given an observed low frame rate video, video frame interpolation (VFI) aims to generate a high frame rate video, which has smooth video frames with higher frames per second (FPS). Most existing VFI methods often focus on generating one frame at a specific timestep, e.g., 0.5, between every two frames, thus lacking the flexibility to increase the video’s FPS by an arbitrary scale, e.g., 3. To better address this issue, in this paper, we propose an arbitrary timestep video frame interpolation (ATVFI) network with time-dependent decoding. Generally, the proposed ATVFI is an encoder–decoder architecture, where the interpolation timestep is an extra input added to the decoder network; this enables ATVFI to interpolate frames at arbitrary timesteps between input frames and to increase the video’s FPS at any given scale. Moreover, we propose a data augmentation method, i.e., multi-width window sampling, where video frames can be split into training samples with multiple window widths, to better leverage training frames for arbitrary timestep interpolation. Extensive experiments were conducted to demonstrate the superiority of our model over existing baseline models on several testing datasets. Specifically, our model trained on the GoPro training set achieved 32.50 on the PSNR metric on the commonly used Vimeo90k testing set.

Keywords: video frame interpolation; deep learning; convolutional neural networks; time-dependent decoding (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2024
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/12/2/303/pdf (application/pdf)
https://www.mdpi.com/2227-7390/12/2/303/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:12:y:2024:i:2:p:303-:d:1320898

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().