Unpaired Image-to-Image Translation with Diffusion Adversarial Network

Hangyao Tu, Zheng Wang () and Yanwei Zhao
Additional contact information
Hangyao Tu: School of Computer Science and Technology, Zhejiang University, Hangzhou 310027, China
Zheng Wang: School of Computer and Computational Science, Hangzhou City University, Hangzhou 310015, China
Yanwei Zhao: College of Mechanical Engineering, Zhejiang University of Technology, Hangzhou 310023, China

Mathematics, 2024, vol. 12, issue 20, 1-15

Abstract: Unpaired image translation with feature-level constraints presents significant challenges, including unstable network training and low diversity in generated tasks. This limitation is typically attributed to the following situations: 1. The generated images are overly simplistic, which fails to stimulate the network’s capacity for generating diverse and imaginative outputs. 2. The images produced are distorted, a direct consequence of unstable training conditions. To address this limitation, the unpaired image-to-image translation with diffusion adversarial network (UNDAN) is proposed. Specifically, our model consists of two modules: (1) Feature fusion module: In this module, one-dimensional SVD features are transformed into two-dimensional SVD features using the convolutional two-dimensionalization method, enhancing the diversity of the images generated by the network. (2) Network convergence module: In this module, the generator transitions from the U-net model to a superior diffusion model. This shift leverages the stability of the diffusion model to mitigate the mode collapse issues commonly associated with adversarial network training. In summary, the CycleGAN framework is utilized to achieve unpaired image translation through the application of cycle-consistent loss. Finally, the proposed network was verified from both qualitative and quantitative aspects. The experiments show that the method proposed can generate more realistic converted images.

Keywords: image translation; two-dimensional feature; diffusion model; generative adversarial network (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2024
References: View complete reference list from CitEc
Citations: View citations in EconPapers (2)

Downloads: (external link)
https://www.mdpi.com/2227-7390/12/20/3178/pdf (application/pdf)
https://www.mdpi.com/2227-7390/12/20/3178/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:12:y:2024:i:20:p:3178-:d:1496459

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().