High-Dimensional Weighted K-Means with Serial Dependence

Zhonghui Zhang, Chihwa Kao and Jungbin Hwang
Additional contact information
Zhonghui Zhang: Nanjing Audit University

No 2025-09, Working papers from University of Connecticut, Department of Economics

Abstract: In this paper, we propose a new K-means approach for high-dimensional panel data with unknown group memberships. We highlight that the standard K-means algorithm using Euclidean distance can su¤er from misclassi cation in nite samples due to serial correlation and heteroskedasticity in the panel data. Our proposed weighted K-means algorithm addresses this issue by weighting the Euclidean distance using the full covari-ance structure of idiosyncratic shocks. Assuming that both the cross-sectional and time dimensions of the panel grow large, we develop an asymptotic theory for the weighted K-means algorithm that establishes the consistency of the estimated group centroids and the oracle property for group membership estimation. For practical implemen-tation, we propose a feasible weighted K-means method that employs a regularized estimation of the high-dimensional covariance matrix in the K-means objective func-tion. Monte Carlo simulation results demonstrate the e¤ectiveness of our weighted K-means algorithm in estimating grouped xed-e¤ects models for large panels, partic-ularly when strong serial dependencies exist in both group-level trends and idiosyncratic components.

Keywords: Banding; Grouped xed e¤ects; Heteroscedasticity and autocorrelation; K-means clustering; Sample covariance matrix (search for similar items in EconPapers)
JEL-codes: C13 C23 C38 C63 (search for similar items in EconPapers)
Pages: 59 pages
Date: 2025-08
References: Add references at CitEc
Citations:

Downloads: (external link)
https://media.economics.uconn.edu/working/2025-09.pdf Full text (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:uct:uconnp:2025-09

Access Statistics for this paper

More papers in Working papers from University of Connecticut, Department of Economics University of Connecticut 365 Fairfield Way, Unit 1063 Storrs, CT 06269-1063. Contact information at EDIRC.
Bibliographic data for series maintained by Mark McConnel ().