Exemplar-Based Sketch Colorization with Cross-Domain Dense Semantic Correspondence

Jinrong Cui, Haowei Zhong, Hailong Liu and Yulu Fu
Additional contact information
Jinrong Cui: College of Mathematics and Informatics, South China Agricultural University, Guangzhou 510642, China
Haowei Zhong: College of Mathematics and Informatics, South China Agricultural University, Guangzhou 510642, China
Hailong Liu: College of Mathematics and Informatics, South China Agricultural University, Guangzhou 510642, China
Yulu Fu: College of Mathematics and Informatics, South China Agricultural University, Guangzhou 510642, China

Mathematics, 2022, vol. 10, issue 12, 1-19

Abstract: This paper aims to solve the task of coloring a sketch image given a ready-colored exemplar image. Conventional exemplar-based colorization methods tend to transfer styles from reference images to grayscale images by employing image analogy techniques or establishing semantic correspondences. However, their practical capabilities are limited when semantic correspondences are elusive. This is the case with coloring for sketches (where semantic correspondences are challenging to find) since it contains only edge information of the object and usually contains much noise. To address this, we present a framework for exemplar-based sketch colorization tasks that synthesizes colored images from sketch input and reference input in a distinct domain. Generally, we jointly proposed our domain alignment network, where the dense semantic correspondence can be established, with a simple but valuable adversarial strategy, that we term the structural and colorific conditions. Furthermore, we proposed to utilize a self-attention mechanism for style transfer from exemplar to sketch. It facilitates the establishment of dense semantic correspondence, which we term the spatially corresponding semantic transfer module. We demonstrate the effectiveness of our proposed method in several sketch-related translation tasks via quantitative and qualitative evaluation.

Keywords: sketch colorization; image synthesis; reference-based colorization (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2022
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/10/12/1988/pdf (application/pdf)
https://www.mdpi.com/2227-7390/10/12/1988/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:10:y:2022:i:12:p:1988-:d:834676

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().