Cross-Modal Attention Fusion Based Generative Adversarial Network for Text-to-Image Synthesis

Xiang Chen () and Xiaodong Luo ()
Additional contact information
Xiang Chen: Hunan University, College of Electrical and Information Engineering
Xiaodong Luo: Sichuan Tourism University, School of Information and Engineering

Chapter Chapter 14 in Generative Machine Learning Models in Medical Image Computing, 2025, pp 279-299 from Springer

Abstract: Abstract The synthesis of images from attribute descriptors is an emerging and intricate domain within the realm of computer vision, which has various application potentials in public security and multimedia. Existing attribute vector-to-face (V2F) synthesis methods mainly generate faces based on attribute label vectors that lack rich semantic feature information, which leads to low-quality generated face images. To surmount this limitation, we advocate attribute word-to-face (W2F) synthesis, leveraging sequences of attribute words rich in semantic content. A novel Cross-Modal Attention Fusion Generative Adversarial Network (CMAFGAN) is proposed to generate faces from facial attribute words. CMAFGAN stands out due to its incorporation of two innovative components, CMAF and WFT, which are proposed to explore the correlation between image features and the corresponding attribute word features. Experimental results on the CelebA and LFW datasets demonstrate that our CMAFGAN achieves state-of-the-art performance, effectively improving the quality of the synthesised faces. In particular, the consistency between the predicted images and input attribute words (R-precision) on the CelebA and LFW datasets achieved 61.24% and 64.46% respectively, representing a substantial improvement over prior techniques. Moreover, CMAFGAN achieves comparable or better performance than the current best methods on text-to-image synthesis (R-precision 83.41% on caltech-ucsd birds-200-2011, CUB). Additionally, we explore the application of CMAFGAN for X-ray image synthesis from textual descriptions, yielding finely detailed images that exhibit high fidelity to the ground-truth.

Date: 2025
References: Add references at CitEc
Citations:

There are no downloads for this item, see the EconPapers FAQ for hints about obtaining it.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:sprchp:978-3-031-80965-1_14

Ordering information: This item can be ordered from
http://www.springer.com/9783031809651

DOI: 10.1007/978-3-031-80965-1_14

Access Statistics for this chapter

More chapters in Springer Books from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().