Achieving Distributional Robustness with Group-Wise Flat Minima

Seowon Ji, Seunghyun Moon, Jiyoon Shin and Sangwoo Hong ()
Additional contact information
Seowon Ji: Department of Computer Science and Engineering, Konkuk University, Seoul 05029, Republic of Korea
Seunghyun Moon: Department of Electrical and Electronics Engineering, Konkuk University, Seoul 05029, Republic of Korea
Jiyoon Shin: Department of Artificial Intelligence, Korea Aerospace University, Goyang 10540, Republic of Korea
Sangwoo Hong: Department of Computer Science and Engineering, Konkuk University, Seoul 05029, Republic of Korea

Mathematics, 2025, vol. 13, issue 20, 1-18

Abstract: Improving robustness under distributional shifts remains a central challenge in machine learning. Although Sharpness-Aware Minimization (SAM) has proven effective in finding flatter minima for better generalization, it overlooks the heterogeneity in sharpness across different subpopulations, which can exacerbate performance gaps for minority or vulnerable groups. To address this challenge, we propose Group-gap Guided SAM (G 2 -SAM), a new optimization framework that promotes distributional robustness by steering flatness-seeking directions according to intergroup loss disparities. Our method estimates group-wise sharpness and adaptively refines perturbation strategies to minimize the worst-group loss while preserving model consistency. Through comprehensive experiments across various datasets, we show that G 2 -SAM achieves superior Worst-Group Accuracy and robustness, outperforming previous baselines. These findings highlight the importance of addressing group-specific geometry in the loss landscape to build more reliable and equitable neural networks.

Keywords: distribution shift; spurious correlation; robust optimization; flat minima (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2025
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/13/20/3343/pdf (application/pdf)
https://www.mdpi.com/2227-7390/13/20/3343/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:13:y:2025:i:20:p:3343-:d:1775479

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().