Knowledge distillation-based domain generalization enabling invariant feature distributions for damage detection of rotating machines and structures

Xiaoyou Wang, Jinyang Jiao, Xiaoqing Zhou and Yong Xia

Reliability Engineering and System Safety, 2025, vol. 257, issue PA

Abstract: The poor generalization ability of machine learning (ML) models in the absence of sufficient labeled data remains a major challenge hindering their practical application. Domain generalization (DG) allows ML models trained on a set of source domains to generalize directly to related but unseen target domains. This capability makes DG particularly suitable for migrating ML model to unseen structures for online structural damage detection. A main branch of DG methods is domain-invariant feature learning, which involves aligning extracted embedding to make it invariant across domains. However, as the domain variety increases, extracting domain-invariant features becomes more challenging. This study rethinks DG from a novel three-stage knowledge distillation perspective. The first stage learns features with domain-invariant conditional distribution based on variational Bayesian inference. Multiple auxiliary domain-specific student models are designed to establish a bridge between unconditional and conditional variational inference. In the second stage, auxiliary student models learn from each other to additionally excavate features with domain-invariant marginal distribution. In the third stage, a student leader model distills knowledge from all auxiliary student models to enhance the model's robustness for final decision-making. The developed method is applied to civil and mechanical structures for damage detection. Results demonstrate that the developed method outperforms the state-of-the-art DG methods. Besides, the designed mechanism enables the student leader model to achieve superior performance compared to the teacher model.

Keywords: Domain generalization; Bayesian inference; Domain invariance; Knowledge distillation; Damage detection (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0951832025000456
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:reensy:v:257:y:2025:i:pa:s0951832025000456

DOI: 10.1016/j.ress.2025.110842

Access Statistics for this article

Reliability Engineering and System Safety is currently edited by Carlos Guedes Soares

More articles in Reliability Engineering and System Safety from Elsevier
Bibliographic data for series maintained by Catherine Liu ().