Missing Data Substitution for Enhanced Robust Filtering and Forecasting in Linear State-Space Models

Dobrislav Dobrev and Pawel J. Szerszen
Additional contact information
Dobrislav Dobrev: https://www.federalreserve.gov/econres/dobrislav-dobrev.htm
Pawel J. Szerszen: https://www.federalreserve.gov/econres/pawel-j-szerszen.htm

No 2025-001, Finance and Economics Discussion Series from Board of Governors of the Federal Reserve System (U.S.)

Abstract: Replacing faulty measurements with missing values can suppress outlier-induced distortions in state-space inference. We therefore put forward two complementary methods for enhanced outlier-robust filtering and forecasting: supervised missing data substitution (MD) upon exceeding a Huber threshold, and unsupervised missing data substitution via exogenous randomization (RMDX).Our supervised method, MD, is designed to improve performance of existing Huber-based linear filters known to lose optimality when outliers of the same sign are clustered in time rather than arriving independently. The unsupervised method, RMDX, further aims to suppress smaller outliers whose size may fall below the Huber detection threshold. To this end, RMDX averages filtered or forecasted targets based on measurement series with randomly induced subsets of missing data at an exogenously set randomization rate. This gives rise to regularization and bias-variance trade-off as a function of the missing data randomization rate, which can be set optimally using standard cross-validation techniques.We validate through Monte Carlo simulations that both methods for missing data substitution can significantly improve robust filtering, especially when combined together. As further empirical validation, we document consistently attractive performance in linear models for forecasting inflation trends prone to clustering of measurement outliers.

Keywords: Kalman filter; Outliers; Huberization; Missing data; Randomization (search for similar items in EconPapers)
JEL-codes: C15 C22 C53 E37 (search for similar items in EconPapers)
Pages: 38 p.
Date: 2025-01-03
New Economics Papers: this item is included in nep-ecm and nep-ets
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.federalreserve.gov/econres/feds/files/2025001pap.pdf (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:fip:fedgfe:2025-01

DOI: 10.17016/FEDS.2025.001

Access Statistics for this paper

More papers in Finance and Economics Discussion Series from Board of Governors of the Federal Reserve System (U.S.) Contact information at EDIRC.
Bibliographic data for series maintained by Ryan Wolfslayer ; Keisha Fournillier ().