Simplex constrained sparse optimization via tail screening

Peng Chen, Jin Zhu, Junxian Zhu and Xueqin Wang

LSE Research Online Documents on Economics from London School of Economics and Political Science, LSE Library

Abstract: We consider the probabilistic simplex-constrained sparse recovery problem. The commonly used Lasso-type penalty for promoting sparsity is ineffective in this context since it is a constant within the simplex. Despite this challenge, fortunately, simplex constraint itself brings a self-regularization property, i.e., the empirical risk minimizer without any sparsity-promoting procedure obtains the usual Lasso-type estimation error. Moreover, we analyze the iterates of a projected gradient descent method and show its convergence to the ground truth sparse solution in the geometric rate until a satisfied statistical precision is attained. Although the estimation error is statistically optimal, the resulting solution is usually more dense than the sparse ground truth. To further sparsify the iterates, we propose a method called PERMITS via embedding a tail screening procedure, i.e., identifying negligible components and discarding them during iterations, into the projected gradient descent method. Furthermore, we combine tail screening and the special information criterion to balance the trade-off between fitness and complexity. Theoretically, the proposed PERMITS method can exactly recover the ground truth support set under mild conditions and thus obtain the oracle property. We demonstrate the statistical and computational efficiency of PERMITS with both synthetic and real data. The implementation of the proposed method can be found in https://github.com/abess-team/PERMITS.

Keywords: projected gradient descent; self-regularization; simplex constrained sparse recovery; special information criterion; tail screening (search for similar items in EconPapers)
JEL-codes: C1 (search for similar items in EconPapers)
Pages: 38 pages
Date: 2025
New Economics Papers: this item is included in nep-mac and nep-rmg
References: Add references at CitEc
Citations:

Published in Journal of Machine Learning Research, 2025, 26. ISSN: 1532-4435

Downloads: (external link)
http://eprints.lse.ac.uk/129540/ Open access version. (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:ehl:lserod:129540

Access Statistics for this paper

More papers in LSE Research Online Documents on Economics from London School of Economics and Political Science, LSE Library LSE Library Portugal Street London, WC2A 2HD, U.K.. Contact information at EDIRC.
Bibliographic data for series maintained by LSERO Manager ().