Assessing Goodness of Fit for Verifying Probabilistic Forecasts

Tae-Ho Kang, Ashish Sharma and Lucy Marshall
Additional contact information
Tae-Ho Kang: School of Civil and Environmental Engineering, University of New South Wales, Sydney, NSW 2052, Australia
Ashish Sharma: School of Civil and Environmental Engineering, University of New South Wales, Sydney, NSW 2052, Australia
Lucy Marshall: School of Civil and Environmental Engineering, University of New South Wales, Sydney, NSW 2052, Australia

Forecasting, 2021, vol. 3, issue 4, 1-11

Abstract: The verification of probabilistic forecasts in hydro-climatology is integral to their development, use, and adoption. We propose here a means of utilizing goodness of fit measures for verifying the reliability of probabilistic forecasts. The difficulty in measuring the goodness of fit for a probabilistic prediction or forecast is that predicted probability distributions for a target variable are not stationary in time, meaning one observation alone exists to quantify goodness of fit for each prediction issued. Therefore, we suggest an additional dissociation that can dissociate target information from the other time variant part—the target to be verified in this study is the alignment of observations to the predicted probability distribution. For this dissociation, the probability integral transformation is used. To measure the goodness of fit for the predicted probability distributions, this study uses the root mean squared deviation metric. If the observations after the dissociation can be assumed to be independent, the mean square deviation metric becomes a chi-square test statistic, which enables statistically testing the hypothesis regarding whether the observations are from the same population as the predicted probability distributions. An illustration of our proposed rationale is provided using the multi-model ensemble prediction for El Niño–Southern Oscillation.

Keywords: probabilistic prediction; verification; reliability; goodness of fit (search for similar items in EconPapers)
JEL-codes: A1 B4 C0 C1 C2 C3 C4 C5 C8 M0 Q2 Q3 Q4 (search for similar items in EconPapers)
Date: 2021
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (2)

Downloads: (external link)
https://www.mdpi.com/2571-9394/3/4/47/pdf (application/pdf)
https://www.mdpi.com/2571-9394/3/4/47/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jforec:v:3:y:2021:i:4:p:47-773:d:666151

Access Statistics for this article

Forecasting is currently edited by Ms. Joss Chen

More articles in Forecasting from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().