A fast algorithm for finding the confidence set of large collections of models

Sylvain Barde

Studies in Economics from School of Economics, University of Kent

Abstract: The paper proposes a new algorithm for finding the confidence set of a collection of forecasts or prediction models. Existing numerical implementations for finding the confidence set use an elimination approach where one starts with the full collection of models and successively eliminates the worst performing until the null of equal predictive ability is no longer rejected at a given confidence level. The intuition behind the proposed implementation lies in reversing the process: one starts with a collection of two models and as models are successively added to the collection both the model rankings and p-values are updated. The first benefit of this updating approach is a reduction of one polynomial order in both the time complexity and memory cost of finding the confidence set of a collection of M models, falling respectively from O (M3) to O (M2) and from O (M2) to O (M). This theoretical prediction is confirmed by a Monte Carlo benchmarking analysis of the algorithms. The second key benefit of the updating approach is that it intuitively allows for further models to be added at a later point in time, thus enabling collaborative efforts using the model confidence set procedure.

Keywords: Model selection; model confidence set; bootstrapped statistics (search for similar items in EconPapers)
JEL-codes: C12 C18 C52 C55 (search for similar items in EconPapers)
Date: 2015-09
New Economics Papers: this item is included in nep-ecm, nep-for and nep-ore
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.kent.ac.uk/economics/repec/1519.pdf (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:ukc:ukcedp:1519

Ordering information: This working paper can be ordered from

Access Statistics for this paper

More papers in Studies in Economics from School of Economics, University of Kent School of Economics, University of Kent, Canterbury, Kent, CT2 7FS.
Bibliographic data for series maintained by Dr Anirban Mitra ().