An Ensemble Learning Approach with Gradient Resampling for Class-Imbalance Problems

Hongke Zhao (), Chuang Zhao (), Xi Zhang (), Nanlin Liu (), Hengshu Zhu (), Qi Liu () and Hui Xiong ()
Additional contact information
Hongke Zhao: College of Management and Economics, Tianjin University, Tianjin 300000, China; Laboratory of Computation and Analytics of Complex Management Systems (CACMS), Tianjin University, Tianjin 300000, China
Chuang Zhao: College of Management and Economics, Tianjin University, Tianjin 300000, China; Laboratory of Computation and Analytics of Complex Management Systems (CACMS), Tianjin University, Tianjin 300000, China
Xi Zhang: College of Management and Economics, Tianjin University, Tianjin 300000, China; Laboratory of Computation and Analytics of Complex Management Systems (CACMS), Tianjin University, Tianjin 300000, China; School of Management and Economics, Beijing Institute of Technology, Beijing 10081, People’s Republic of China
Nanlin Liu: College of Management and Economics, Tianjin University, Tianjin 300000, China; Laboratory of Computation and Analytics of Complex Management Systems (CACMS), Tianjin University, Tianjin 300000, China
Hengshu Zhu: Career Science Laboratory, BOSS Zhipin, Beijing 100000, China
Qi Liu: Anhui Province Key Laboratory of Big Data Analysis and Application, University of Science and Technology of China, Hefei, Anhui Province 230000, China
Hui Xiong: Hong Kong University of Science and Technology (GuangZhou), Guangzhou, Guangdong Province 510000, China

INFORMS Journal on Computing, 2023, vol. 35, issue 4, 747-763

Abstract: Imbalanced classification is widely referred in many real-world applications and has been extensively studied. Most existing algorithms consider alleviating the imbalance by sampling or guiding ensemble learners with punishments. The combination of ensemble learning and sampling strategy at class level has achieved great progress. Actually, specific hard examples have little benefit for model learning and even degrade the performance. From the view of identifying classification difficulty of samples, one important motivation is to design algorithms to finely equip different samples with progressive learning. Unfortunately, how to perfectly configure the sampling and learning strategies under ensemble principles at the sample level remains a research gap. In this paper, we propose a new view from the sample level rather than class level in existing studies. We design an ensemble approach in pipe with sample-level gradient resampling, that is, balanced cascade with filters (BCWF) . Before that, as a preliminary exploration, we first design a hard examples mining algorithm to explore the gradient distribution of classification difficulty of samples and identify the hard examples. Specifically, BCWF uses an under-sampling strategy and a boosting manner to train T predictive classifiers and reidentify hard examples. In BCWF, moreover, we design two types of filters: the first is assembled with a hard filter (BCWF_h), whereas the second is assembled with a soft filter (BCWF_s). In each round of boosting, BCWF_h strictly removes a gradient/set of the hardest examples from both classes, whereas BCWF_s removes a larger number of harder and easy examples simultaneously for final balanced-class retention. Consequently, the well-trained T predictive classifiers can be used with two ensemble voting strategies: average probability and majority vote . To evaluate the proposed approach, we conduct intensive experiments on 10 benchmark data sets and apply our algorithms to perform default user detection on a real-world peer to peer lending data set. The experimental results fully demonstrate the effectiveness and the managerial implications of our approach when compared with 11 competitive algorithms.

Keywords: class-imbalance learning; ensemble learning; under-sampling strategy; gradient distribution (search for similar items in EconPapers)
Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://dx.doi.org/10.1287/ijoc.2023.1274 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:orijoc:v:35:y:2023:i:4:p:747-763

Access Statistics for this article

More articles in INFORMS Journal on Computing from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().