 CrossFuse: Gated Cross-Modal Collaboration for Multi-Modal GenerationXiang Tony Cao European Journal of AI, Computing & Informatics, 2026, vol. 2, issue 3, 41-52 Abstract: Multi-modal generators often fuse visual and textual streams with a learned gate, but the usual intuition that the gate should trust the more informative modality is rarely tested directly because real corpora do not reveal per-sample modality reliability. We present CrossFuse, a compact reference implementation of gated cross-modal collaboration: linear encoders, contrastive alignment, scalar gating, gated fusion, and readout, trained with generation, alignment, and consistency losses. We also introduce a CPU-reproducible synthetic diagnostic where per-sample modality informativeness is known. The learned gate, trained only through the downstream generation loss, recovers that hidden informativeness with Pearson r ≈ 0.90, while an anti-tautology control collapses the correlation to ≈ 0 when the gate is denied the required second-order cue. Ablations show complementary roles: gating improves generation, alignment enables cross-modal retrieval, and consistency stabilises routing. The NumPy autodiff core runs in seconds, with an optional GPU-ready PyTorch MS-COCO captioner, and diagnostic benchmark. Keywords: cross-modal fusion; gated fusion; image captioning; contrastive alignment; interpretability; reproducibility (search for similar items in EconPapers) Downloads: (external link) Related works: Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text Persistent link: https://EconPapers.repec.org/RePEc:dba:ejacia:v:2:y:2026:i:3:p:41-52 Access Statistics for this article More articles in European Journal of AI, Computing & Informatics from Pinnacle Academic Press |
Is your work missing from RePEc? Here is how to contribute.
Questions or problems? Check the EconPapers FAQ or send mail to .
EconPapers is hosted by the
School of Business at Örebro University.