Depth Map Super-Resolution Based on Semi-Couple Deformable Convolution Networks

Botao Liu, Kai Chen (), Sheng-Lung Peng and Ming Zhao
Additional contact information
Botao Liu: School of Computer Science, Yangtze University, Jingzhou 434023, China
Kai Chen: School of Computer Science, Yangtze University, Jingzhou 434023, China
Sheng-Lung Peng: Department of Creative Technologies and Product Design, National Taipei University of Business, Taipei 10051, Taiwan
Ming Zhao: School of Computer Science, Yangtze University, Jingzhou 434023, China

Mathematics, 2023, vol. 11, issue 21, 1-17

Abstract: Depth images obtained from lightweight, real-time depth estimation models and consumer-oriented sensors typically have low-resolution issues. Traditional interpolation methods for depth image up-sampling result in a significant information loss, especially in edges with discontinuous depth variations (depth discontinuities). To address this issue, this paper proposes a semi-coupled deformable convolution network (SCD-Net) based on the idea of guided depth map super-resolution (GDSR). The method employs a semi-coupled feature extraction scheme to learn unique and similar features between RGB images and depth images. We utilize a Coordinate Attention (CA) to suppress redundant information in RGB features. Finally, a deformable convolutional module is employed to restore the original resolution of the depth image. The model is tested on NYUv2, Middlebury, Lu, and a Real-Sense real-world dataset created using an Intel Real-sense D455 structured-light camera. The super-resolution accuracy of SCD-Net at multiple scales is much higher than that of traditional methods and superior to recent state-of-the-art (SOTA) models, which demonstrates the effectiveness and flexibility of our model on GDSR tasks. In particular, our method further solves the problem of an RGB texture being over-transferred in GDSR tasks.

Keywords: depth map super-resolution; guide image filter; deformable convolution; deep learning (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2023
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/11/21/4556/pdf (application/pdf)
https://www.mdpi.com/2227-7390/11/21/4556/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:11:y:2023:i:21:p:4556-:d:1274522

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().