Advancing Model Generalization in Continuous Cyclic Test-Time Adaptation with Matrix Perturbation Noise

Jinshen Jiang, Hao Yang, Lin Yang () and Yun Zhou ()
Additional contact information
Jinshen Jiang: National Key Laboratory of Information Systems Engineering, National University of Defense Technology, Changsha 410073, China
Hao Yang: National Key Laboratory on Blind Signal Processing, Chengdu 610041, China
Lin Yang: School of Information and Intelligent Engineering, University of Sanya Academician Guoliang Chen Team Innovation Center, University of Sanya, Sanya 572000, China
Yun Zhou: National Key Laboratory of Information Systems Engineering, National University of Defense Technology, Changsha 410073, China

Mathematics, 2024, vol. 12, issue 18, 1-15

Abstract: Test-time adaptation (TTA) aims to optimize source-pretrained model parameters to target domains using only unlabeled test data. However, traditional TTA methods often risk overfitting to the specific, localized test domains, leading to compromised generalization. Moreover, these methods generally presume static target domains, neglecting the dynamic and cyclic nature of real-world settings. To alleviate this limitation, this paper explores the continuous cyclic test-time adaptation (CycleTTA) setting. Our unique approach within this setting employs matrix-wise perturbation noise in batch-normalization statistics to enhance the adaptability of source-pretrained models to dynamically changing target domains, without the need for additional parameters. We demonstrated the effectiveness of our method through extensive experiments, where our approach reduced the average error by 39.8% on the CIFAR10-C dataset using the WideResNet-28-10 model, by 38.8% using the WideResNet-40-2 model, and by 33.8% using the PreActResNet-18 model. Additionally, on the CIFAR100-C dataset with the WideResNet-40-2 model, our method reduced the average error by 5.3%, showcasing significant improvements in model generalization in continuous cyclic testing scenarios.

Keywords: deep learning; model generalization; distribution shift; test-time adaptation; model robustness (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2024
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/12/18/2800/pdf (application/pdf)
https://www.mdpi.com/2227-7390/12/18/2800/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:12:y:2024:i:18:p:2800-:d:1475350

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().