Adjusted Pearson Chi-Square feature screening for multi-classification with ultrahigh dimensional data

Lyu Ni, Fang Fang () and Fangjiao Wan
Additional contact information
Lyu Ni: East China Normal University
Fang Fang: East China Normal University
Fangjiao Wan: East China Normal University

Metrika: International Journal for Theoretical and Applied Statistics, 2017, vol. 80, issue 6, No 12, 805-828

Abstract: Abstract Huang et al. (J Bus Econ Stat 32:237–244, 2014) first proposed a Pearson Chi-Square based feature screening procedure tailored to multi-classification problem with ultrahigh dimensional categorical covariates, which is a common problem in practice but has seldom been discussed in the literature. However, their work establishes the sure screening property only in a limited setting. Moreover, the p value based adjustments when the number of categories involved by each covariate is different do not work well in several practical situations. In this paper, we propose an adjusted Pearson Chi-Square feature screening procedure and a modified method for tuning parameter selection. Theoretically, we establish the sure screening property of the proposed method in general settings. Empirically, the proposed method is more successful than Pearson Chi-Square feature screening in handling non-equal numbers of covariate categories in finite samples. Results of three simulation studies and one real data analysis are presented. Our work together with Huang et al. (J Bus Econ Stat 32:237–244, 2014) establishes a solid theoretical foundation and empirical evidence for the family of Pearson Chi-Square based feature screening methods.

Keywords: Continuous and categorical covariates; Diverging classes; Pearson Chi-Square statistics; Sure screening property (search for similar items in EconPapers)
Date: 2017
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
http://link.springer.com/10.1007/s00184-017-0629-9 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:metrik:v:80:y:2017:i:6:d:10.1007_s00184-017-0629-9

Ordering information: This journal article can be ordered from
http://www.springer.com/statistics/journal/184/PS2

DOI: 10.1007/s00184-017-0629-9

Access Statistics for this article

Metrika: International Journal for Theoretical and Applied Statistics is currently edited by U. Kamps and Norbert Henze

More articles in Metrika: International Journal for Theoretical and Applied Statistics from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().