Beyond vintage rotation: bias-free sparse representation learning with oracle inference

Chengyu Cui, Yunxiao Chen, Jing Ouyang and Gongjun Xu

LSE Research Online Documents on Economics from London School of Economics and Political Science, LSE Library

Abstract: Learning low-dimensional latent representations is a central topic in statistics and machine learning, and rotation methods have long been used to obtain sparse and interpretable representations. Despite nearly a century of widespread use across many fields, rigorous guarantees for valid inference for the learned representation remain lacking. In this paper, we identify a surprisingly prevalent phenomenon that suggests a reason for this gap: for a broad class of vintage rotations, the resulting estimators exhibit a non-estimable bias. Because this bias is independent of the data, it fundamentally precludes the development of valid inferential procedures, including the construction of confidence intervals and hypothesis testing. To address this challenge, we propose a novel bias-free rotation method within a general representation learning framework based on latent variables. We establish an oracle inference property for the learned sparse representations: the estimators achieve the same asymptotic variance as in the ideal setting where the latent variables are observed. To bridge the gap between theory and computation, we develop an efficient computational framework and prove that its output estimators retain the same oracle property. Our results provide a rigorous inference procedure for the rotated estimators, yielding statistically valid and interpretable representation learning. Code for implementing the proposed method is available at the GitHub repository: https://github.com/chengyu06/Folomin.git.

Keywords: latent embedding; rotation method; folded concave loss; non-convex optimisation; local quadratic approximation (search for similar items in EconPapers)
JEL-codes: C1 (search for similar items in EconPapers)
Date: 2026-10-26
References: Add references at CitEc
Citations:

Published in Journal of the Royal Statistical Society. Series B: Statistical Methodology, 26, October, 2026. ISSN: 1369-7412

Downloads: (external link)
https://researchonline.lse.ac.uk/id/eprint/141229/ Open access version. (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:ehl:lserod:141229

Access Statistics for this paper

More papers in LSE Research Online Documents on Economics from London School of Economics and Political Science, LSE Library LSE Library Portugal Street London, WC2A 2HD, U.K.. Contact information at EDIRC.
Bibliographic data for series maintained by LSERO Manager ().