DFNet: Decoupled Fusion Network for Dialectal Speech Recognition

Qianqiao Zhu, Lu Gao () and Ling Qin
Additional contact information
Qianqiao Zhu: School of Digtial and Intelligence Industry, Inner Mongolia University of Science and Technology, Baotou 014010, China
Lu Gao: School of Digtial and Intelligence Industry, Inner Mongolia University of Science and Technology, Baotou 014010, China
Ling Qin: School of Digtial and Intelligence Industry, Inner Mongolia University of Science and Technology, Baotou 014010, China

Mathematics, 2024, vol. 12, issue 12, 1-23

Abstract: Deep learning is often inadequate for achieving effective dialect recognition in situations where data are limited and model training is complex. Differences between Mandarin and dialects, such as the varied pronunciation variants and distinct linguistic features of dialects, often result in a significant decline in recognition performance. In addition, existing work often overlooks the similarities between Mandarin and its dialects and fails to leverage these connections to enhance recognition accuracy. To address these challenges, we propose the Decoupled Fusion Network (DFNet). This network extracts acoustic private and shared features of different languages through feature decoupling, which enhances adaptation to the uniqueness and similarity of these two speech patterns. In addition, we designed a heterogeneous information-weighted fusion module to effectively combine the decoupled Mandarin and dialect features. This strategy leverages the similarity between Mandarin and its dialects, enabling the sharing of multilingual information, and notably enhance the model’s recognition capabilities on low-resource dialect data. An evaluation of our method on the Henan and Guangdong datasets shows that the DFNet performance has improved by 2.64% and 2.68%, respectively. Additionally, a significant number of ablation comparison experiments demonstrate the effectiveness of the method.

Keywords: dialectal speech recognition; feature decoupled; information fusion (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2024
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/12/12/1886/pdf (application/pdf)
https://www.mdpi.com/2227-7390/12/12/1886/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:12:y:2024:i:12:p:1886-:d:1416637

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().