Reconstructing Missing and Anomalous Data Collected from High-Frequency In-Situ Sensors in Fresh Waters

Claire Kermorvant, Benoit Liquet, Guy Litt, Jeremy B. Jones, Kerrie Mengersen, Erin E. Peterson, Rob Hyndman and Catherine Leigh
Additional contact information
Claire Kermorvant: Laboratoire de Mathématiques et de Leurs Applications de Pau Fédération MIRA, UMR CNRS 5142, Université de Pau et des Pays de l’Adour, 64600 Anglet, France
Benoit Liquet: Laboratoire de Mathématiques et de Leurs Applications de Pau Fédération MIRA, UMR CNRS 5142, Université de Pau et des Pays de l’Adour, 64600 Anglet, France
Guy Litt: National Ecological Observatory Network, Battelle Boulder, Boulder, CO 80301, USA
Jeremy B. Jones: Institute of Arctic Biology and Department of Biology and Wildlife, University of Alaska Fairbanks, Fairbanks, AK 99775, USA
Kerrie Mengersen: School of Mathematical Sciences, Queensland University of Technology, Brisbane, QLD 4000, Australia
Erin E. Peterson: ARC Centre of Excellence for Mathematics and Statistical Frontiers, Melbourne, VIC 3000, Australia
Catherine Leigh: ARC Centre of Excellence for Mathematics and Statistical Frontiers, Melbourne, VIC 3000, Australia

IJERPH, 2021, vol. 18, issue 23, 1-14

Abstract: In situ sensors that collect high-frequency data are used increasingly to monitor aquatic environments. These sensors are prone to technical errors, resulting in unrecorded observations and/or anomalous values that are subsequently removed and create gaps in time series data. We present a framework based on generalized additive and auto-regressive models to recover these missing data. To mimic sporadically missing (i) single observations and (ii) periods of contiguous observations, we randomly removed (i) point data and (ii) day- and week-long sequences of data from a two-year time series of nitrate concentration data collected from Arikaree River, USA, where synoptically collected water temperature, turbidity, conductance, elevation, and dissolved oxygen data were available. In 72% of cases with missing point data, predicted values were within the sensor precision interval of the original value, although predictive ability declined when sequences of missing data occurred. Precision also depended on the availability of other water quality covariates. When covariates were available, even a sudden, event-based peak in nitrate concentration was reconstructed well. By providing a promising method for accurate prediction of missing data, the utility and confidence in summary statistics and statistical trends will increase, thereby assisting the effective monitoring and management of fresh waters and other at-risk ecosystems.

Keywords: anomaly correction; generalised additive model (GAM); missing data reconstruction; remote sensing; water quality (search for similar items in EconPapers)
JEL-codes: I I1 I3 Q Q5 (search for similar items in EconPapers)
Date: 2021
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/1660-4601/18/23/12803/pdf (application/pdf)
https://www.mdpi.com/1660-4601/18/23/12803/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jijerp:v:18:y:2021:i:23:p:12803-:d:695074

Access Statistics for this article

IJERPH is currently edited by Ms. Jenna Liu

More articles in IJERPH from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().