Weakly Supervised Specular Highlight Removal Using Only Highlight Images

Yuanfeng Zheng, Guangwei Hu, Hao Jiang (), Hao Wang and Lihua Wu
Additional contact information
Yuanfeng Zheng: School of Electronic Information, Wuhan University, Wuhan 430072, China
Guangwei Hu: School of Electronic Information, Wuhan University, Wuhan 430072, China
Hao Jiang: School of Electronic Information, Wuhan University, Wuhan 430072, China
Hao Wang: School of Electronic Information, Wuhan University, Wuhan 430072, China
Lihua Wu: Wuhan Second Ship Design and Research Institute, Wuhan 430064, China

Mathematics, 2024, vol. 12, issue 16, 1-16

Abstract: Specular highlight removal is a challenging task in the field of image enhancement, while it can significantly improve the quality of image in highlight regions. Recently, deep learning-based methods have been widely adopted in this task, demonstrating excellent performance by training on either massive paired data, wherein both the highlighted and highlight-free versions of the same image are available, or unpaired datasets where the one-to-one correspondence is inapplicable. However, it is difficult to obtain the corresponding highlight-free version of a highlight image, as the latter has already been produced under specific lighting conditions. In this paper, we propose a method for weakly supervised specular highlight removal that only requires highlight images. This method involves generating highlight-free images from highlight images with the guidance of masks estimated using non-negative matrix factorization (NMF). These highlight-free images are then fed consecutively into a series of modules derived from a Cycle Generative Adversarial Network (Cycle-GAN)-style network, namely the highlight generation, highlight removal, and reconstruction modules in sequential order. These modules are trained jointly, resulting in a highly effective highlight removal module during the verification. On the specular highlight image quadruples (SHIQ) and the LIME datasets, our method achieves an accuracy of 0.90 and a balance error rate (BER) of 8.6 on SHIQ, and an accuracy of 0.89 and a BER of 9.1 on LIME, outperforming existing methods and demonstrating its potential for improving image quality in various applications.

Keywords: image processing; specular highlight removal; non-negative matrix factorization; weakly supervised learning; GANs (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2024
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/12/16/2578/pdf (application/pdf)
https://www.mdpi.com/2227-7390/12/16/2578/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:12:y:2024:i:16:p:2578-:d:1460793

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().