Kalman filter with impulse noised outliers: a robust sequential algorithm to filter data with a large number of outliers

Cloez Bertrand (), Fontez Bénédicte (), González-García Eliel and Sanchez Isabelle ()
Additional contact information
Cloez Bertrand: MISTEA, Univ Montpellier, INRAE, Institut Agro, Montpellier, France
Fontez Bénédicte: MISTEA, Univ Montpellier, INRAE, Institut Agro, Montpellier, France
González-García Eliel: SELMET, Univ Montpellier, INRAE, CIRAD, Institut Agro, Montpellier, France
Sanchez Isabelle: MISTEA, Univ Montpellier, INRAE, Institut Agro, Montpellier, France

The International Journal of Biostatistics, 2024, vol. 20, issue 2, 641-660

Abstract: Impulse noised outliers are data points that differ significantly from other observations. They are generally removed from the data set through local regression or the Kalman filter algorithm. However, these methods, or their generalizations, are not well suited when the number of outliers is of the same order as the number of low-noise data (often called nominal measurement). In this article, we propose a new model for impulsed noise outliers. It is based on a hierarchical model and a simple linear Gaussian process as with the Kalman Filter. We present a fast forward-backward algorithm to filter and smooth sequential data and which also detects these outliers. We compare the robustness and efficiency of this algorithm with classical methods. Finally, we apply this method on a real data set from a Walk Over Weighing system admitting around 60 % of outliers. For this application, we further develop an (explicit) EM algorithm to calibrate some algorithm parameters.

Keywords: outlier detection; filter algorithm; Gaussian model; EM algorithm; walk over weighing (search for similar items in EconPapers)
Date: 2024
References: Add references at CitEc
Citations:

Downloads: (external link)
https://doi.org/10.1515/ijb-2023-0065 (text/html)
For access to full text, subscription to the journal or payment for the individual article is required.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bpj:ijbist:v:20:y:2024:i:2:p:641-660:n:1015

Ordering information: This journal article can be ordered from
https://www.degruyter.com/journal/key/ijb/html

DOI: 10.1515/ijb-2023-0065

Access Statistics for this article

The International Journal of Biostatistics is currently edited by Antoine Chambaz, Alan E. Hubbard and Mark J. van der Laan

More articles in The International Journal of Biostatistics from De Gruyter
Bibliographic data for series maintained by Peter Golla ().