Assessing the significance of model selection in ecology

Edward Wheatcroft

LSE Research Online Documents on Economics from London School of Economics and Political Science, LSE Library

Abstract: Model Selection is a key part of many ecological studies, with Akaike's Information Criterion (AIC) being by far the most commonly used technique for this purpose. Typically, a number of candidate models are defined a priori and ranked according to their expected out-of-sample performance. Model selection, however, only assesses the relative performance of the models and, as pointed out in a recent paper, a large proportion of ecology papers that use model selection do not assess the absolute fit of the 'best' model. In this paper, it is argued that assessing the absolute fit of the 'best' model alone does not go far enough. This is because a model that appears to perform well under model selection is also likely to appear to perform well under measures of absolute fit, even when there is no predictive value. This paper proposes a model selection permutation test that assesses the probability that the model selection statistic of the 'best' model could have occurred by chance alone, whilst taking account of dependencies between the models. It is argued that this test should always be performed as a part of formal model selection. The test is demonstrated on two real population modelling examples of ibex in northern Italy and wild reindeer in Norway. In both cases, the model selection permutation test gives a highly significant result, indicating that the performance of the 'best' model is unlikely to be through chance alone. R code is provided with which to perform the tests.

Keywords: Akaike's Information Criterion; Ibex; model selection; permutation test; reindeer (search for similar items in EconPapers)
JEL-codes: C1 (search for similar items in EconPapers)
Pages: 17 pages
Date: 2020-12-30
References: View references in EconPapers View complete reference list from CitEc
Citations:

Published in European Journal of Ecology, 30, December, 2020, 6(2), pp. 87-103. ISSN: 1339-8474

Downloads: (external link)
http://eprints.lse.ac.uk/115434/ Open access version. (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:ehl:lserod:115434

Access Statistics for this paper

More papers in LSE Research Online Documents on Economics from London School of Economics and Political Science, LSE Library LSE Library Portugal Street London, WC2A 2HD, U.K.. Contact information at EDIRC.
Bibliographic data for series maintained by LSERO Manager ().