Supervised Feature Selection Method Using Stackable Attention Networks

Zhu Chen, Wei Jiang, Jun Tan, Zhiqiang Li and Ning Gui ()
Additional contact information
Zhu Chen: HUANENG Power International Inc., Beijing 100031, China
Wei Jiang: School of Computer Science and Engineering, Central South University, Changsha 410083, China
Jun Tan: School of Computer Science and Engineering, Central South University, Changsha 410083, China
Zhiqiang Li: School of Computer Science and Engineering, Central South University, Changsha 410083, China
Ning Gui: School of Computer Science and Engineering, Central South University, Changsha 410083, China

Mathematics, 2025, vol. 13, issue 22, 1-29

Abstract: Mainstream DNN-based feature selection methods share a similar design strategy: employing one specially designed feature selection module to learn the importance of features along with the model-training process. While these works achieve great success in feature selection, their shallow structures, which evaluate feature importance from one perspective, are easily disturbed by noisy samples, especially in datasets with high-dimensional features and complex structures. To alleviate this limitation, this paper innovatively introduces a Stackable Attention architecture for Feature Selection (SAFS), which can calculate stable and accurate feature weights through a set of Stackable Attention Blocks (SABlocks) rather than from a single module. To avoid information loss from stacking, a feature jump concatenation structure is designed. Furthermore, an inertia-based weight update method is proposed to generate a more robust feature weight distribution. Experiments on twelve real-world datasets, including multiple domains, demonstrate that SAFS produced the best results with significant performance edges compared to thirteen baselines.

Keywords: feature selection; supervised learning; attention networks (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/13/22/3703/pdf (application/pdf)
https://www.mdpi.com/2227-7390/13/22/3703/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:13:y:2025:i:22:p:3703-:d:1797427

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().