A multimodal fusion method for soldering quality online inspection

Jian Xie, Yu Guo (), Daoyuan Liu, Shaohua Huang (), Kaiwen Zheng and Yaning Tao
Additional contact information
Jian Xie: Nanjing University of Aeronautics and Astronautics
Yu Guo: Nanjing University of Aeronautics and Astronautics
Daoyuan Liu: Nanjing University of Aeronautics and Astronautics
Shaohua Huang: Nanjing University of Aeronautics and Astronautics
Kaiwen Zheng: Nanjing University of Aeronautics and Astronautics
Yaning Tao: Nanjing University of Aeronautics and Astronautics

Journal of Intelligent Manufacturing, 2025, vol. 36, issue 5, No 17, 3284 pages

Abstract: Abstract Effective and efficient online quality inspection of poor soldering in Surface Mount Technology (SMT) solder joints, whose defect characteristics are not visually discernible, remains a formidable obstacle in the electronics manufacturing industry, despite the availability of various inspection methods. To overcome this challenge, a novel multimodal fusion method for soldering quality inspection is proposed. First, the method innovatively introduces information from different modalities as input to the inspection model, with the aim to provide more comprehensive information for the decision making of the inspection model. Then, a combination of multimodal gated attention and tensor fusion is used to fuse the features extracted from each modality to form a comprehensive multimodal representation. Finally, this multimodal representation is used to conduct soldering quality inspection. The experimental results demonstrate that the proposed method improves the detection rate of poor soldering significantly, from 93.6 to 99.4%, with a precision level of nearly 100%. The detection rate for visually unapparent soldering defects increases from 49.3 to 95.4%. This meets both manufacturing and customer requirements and to some extent addressing the industry challenge of online SMT soldering quality inspection. This remarkable performance surpasses that of the six mainstream ResNet, ResNext, RegNet, ShuffleNet, EfficientNet and NoisyNet inspection models currently available.

Keywords: Multimodal fusion; Quality inspection; Multimodal gated attention; Defects detection; Surface mount technology (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s10845-024-02413-3 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:joinma:v:36:y:2025:i:5:d:10.1007_s10845-024-02413-3

Ordering information: This journal article can be ordered from
http://www.springer.com/journal/10845

DOI: 10.1007/s10845-024-02413-3

Access Statistics for this article

Journal of Intelligent Manufacturing is currently edited by Andrew Kusiak

More articles in Journal of Intelligent Manufacturing from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().