Binary classification with covariate selection through ℓ0-penalised empirical risk minimisation

Le-Yu Chen and Sokbae (Simon) Lee

The Econometrics Journal, 2021, vol. 24, issue 1, 103-120

Abstract: SummaryWe consider the problem of binary classification with covariate selection. We construct a classification procedure by minimising the empirical misclassification risk with a penalty on the number of selected covariates. This optimisation problem is equivalent to obtaining an ℓ0-penalised maximum score estimator. We derive probability bounds on the estimated sparsity as well as on the excess misclassification risk. These theoretical results are nonasymptotic and established in a high-dimensional setting. In particular, we show that our method yields a sparse solution whose ℓ0-norm can be arbitrarily close to true sparsity with high probability and obtain the rates of convergence for the excess misclassification risk. We implement the proposed procedure via the method of mixed-integer linear programming. Its numerical performance is illustrated in Monte Carlo experiments and a real data application of the work-trip transportation mode choice.

Keywords: Classification; covariate selection; finite-sample property; maximum score estimation; mixed-integer optimisation; penalised estimation (search for similar items in EconPapers)
Date: 2021
References: Add references at CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
http://hdl.handle.net/10.1093/ectj/utaa017 (application/pdf)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:oup:emjrnl:v:24:y:2021:i:1:p:103-120.

Access Statistics for this article

The Econometrics Journal is currently edited by Jaap Abbring

More articles in The Econometrics Journal from Royal Economic Society Contact information at EDIRC.
Bibliographic data for series maintained by Oxford University Press ().