Strong model dependence in statistical analysis: goodness of fit is not enough for model choice

John Copas () and Shinto Eguchi
Additional contact information
John Copas: University of Warwick
Shinto Eguchi: Institute of Statistical Mathematics

Annals of the Institute of Statistical Mathematics, 2020, vol. 72, issue 2, No 1, 329-352

Abstract: Abstract Most statistical methods are based on models, but most practical applications ignore the fact that the results depend on the model as well as on the data. This paper examines the size of this model dependence, and finds that there can be very considerable variation between the results of fitting different models to the same data, even if the models being considered are restricted to those which give an acceptable fit to the data. Under reasonable regularity conditions, we show that different empirically acceptable models can give rise to non-overlapping confidence intervals for the same parameter. Application papers need to recognize that the validity of conventional statistical results rests on the assumption that the underlying model is known to be correct, and that this is a much stronger requirement than merely confirming that the model gives a good fit to the data. The problem of model dependence is only partially resolved by using formal methods of model selection or model averaging.

Keywords: Goodness-of-fit; Model choice; Model uncertainty; Subset selection (search for similar items in EconPapers)
Date: 2020
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s10463-018-0691-8 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:aistmt:v:72:y:2020:i:2:d:10.1007_s10463-018-0691-8

Ordering information: This journal article can be ordered from
http://www.springer. ... cs/journal/10463/PS2

DOI: 10.1007/s10463-018-0691-8

Access Statistics for this article

Annals of the Institute of Statistical Mathematics is currently edited by Tomoyuki Higuchi

More articles in Annals of the Institute of Statistical Mathematics from Springer, The Institute of Statistical Mathematics
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().