Remedies for Severe Class Imbalance

Max Kuhn and Kjell Johnson
Additional contact information
Max Kuhn: Pfizer Global Research and Development, Division of Nonclinical Statistics
Kjell Johnson: Arbor Analytics

Chapter Chapter 16 in Applied Predictive Modeling, 2013, pp 419-443 from Springer

Abstract: Abstract When modeling discrete classes, the relative frequencies of the classes can have a significant impact on the effectiveness of the model. An imbalance occurs when one or more classes have very low proportions in the training data as compared to the other classes. Imbalance can be present in any data set or application, and hence, the practitioner should be aware of the implications of modeling this type of data. To illustrate the impacts and remedies for severe class imbalance, we present a case study example (Section 16.1) and the impact of class imbalance on performances measures (Section 16.2). Sections 16.3-16.6 describe approaches for handling imbalance using the existing data such as maximizing minority class accuracy, adjusting classification cut-offs or prior probabilities, or adjusting sample weights prior to model tuning. Handling imbalance can also be done through sophisticated up- or down-sampling methods (Section 16.7) or by applying costs to the classification errors (Section 16.8). In the Computing Section (16.9) we demonstrate how to implement these remedies in R. Finally, exercises are provided at the end of the chapter to solidify the concepts.

Keywords: Support Vector Machine; Receiver Operating Characteristic Curve; Random Forest; Prior Probability; Class Probability (search for similar items in EconPapers)
Date: 2013
References: Add references at CitEc
Citations:

There are no downloads for this item, see the EconPapers FAQ for hints about obtaining it.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:sprchp:978-1-4614-6849-3_16

Ordering information: This item can be ordered from
http://www.springer.com/9781461468493

DOI: 10.1007/978-1-4614-6849-3_16

Access Statistics for this chapter

More chapters in Springer Books from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().