FourierNAT: a Fourier-mixing-based non-autoregressive transformer for parallel sequence generation

Andrew Kiruluta

International Journal of Complexity in Applied Science and Technology, 2026, vol. 2, issue 1, 63-75

Abstract: We present FourierNAT, a novel non-autoregressive transformer (NAT) architecture that leverages Fourier-based mixing in the decoder to generate output sequences in parallel. While traditional NAT approaches often face challenges in capturing global dependencies, our method uses a discrete Fourier transform with learned frequency-domain gating to mix token embeddings across the entire sequence dimension. This design enables efficient propagation of context without explicit autoregressive steps. Empirically, F_OURIERNAT achieves competitive results on WMT14 En-De and CNN/DailyMail benchmarks, highlighting that frequency-domain operations can mitigate coherence gaps often associated with NAT generation. Our approach underscores the potential of integrating spectral-domain operations to accelerate and improve parallel text generation.

Keywords: non-autoregressive transformer: NAT; Fourier mixing; parallel sequence generation; global spectral operations; NAT architecture. (search for similar items in EconPapers)
Date: 2026
References: Add references at CitEc
Citations:

Downloads: (external link)
http://www.inderscience.com/link.php?id=151886 (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:ids:ijcast:v:2:y:2026:i:1:p:63-75

Access Statistics for this article

More articles in International Journal of Complexity in Applied Science and Technology from Inderscience Enterprises Ltd
Bibliographic data for series maintained by Sarah Parker ().