XTOOS: Stata module for evaluating the out-of-sample prediction performance of panel-data models

Ugarte-Ruiz, Alfonso

XTOOS: Stata module for evaluating the out-of-sample prediction performance of panel-data models

Statistical Software Components from Boston College Department of Economics

Abstract: The package XTOOS includes four new commands that allow to evaluate the out-of-sample prediction performance of panel-data models in their time-series and cross-individual dimensions separately, also with separate procedures for different types of dependent variables—either continuous or dichotomous variables. The name of the commands are xtoos_t, xtoos_i, xtoos_bin_t, and xtoos_bin_i. The time-series procedures (xtoos_t and xtoos_bin_t) exclude a number of time periods defined by the user from the estimation sample for each individual in the panel. Similarly, the cross-individual procedures (xtoos_i and xtoos_bin_i) exclude a group of individuals (for example, countries) defined by the user from the estimation sample (including all their observations throughout time). Then, for the remaining subsamples (training-sample), they fit the specified models and use the resulting parameters to forecast the dependent variable (or the probability of a positive outcome) in the unused time-periods or the unused individuals (testing-sample). The unused time-period or individual sets are then recursively reduced by one period in every subsequent step in the time-series dimension, or in either a random or an ordered fashion in the cross-individual dimension. The estimation and forecasting evaluation are repeated until there are no more periods ahead or more individuals that could be evaluated. In the continuous cases, the model's forecasting performance is reported both in absolute terms (RMSE) and relative to a naive prediction, by means of a U-Theil ratio. In the dichotomous case, the prediction performance is evaluated based on the area under the receiver operator characteristic (ROC) statistic evaluated in both the training sample and the out of sample. The procedures allow to choose different estimation methods, including some dynamic methodologies, and could also be used in a time-series or a cross-section dataset only. They also allow evaluating the model's forecasting performance for one particular individual or for a defined group of individuals instead of the whole panel.

Language: Stata
Requires: Stata version 13
Keywords: panel data; prediction; out-of-sample forecast (search for similar items in EconPapers)
Date: 2019-11-08, Revised 2023-04-03
Note: This module should be installed from within Stata by typing "ssc install xtoos". The module is made available under terms of the GPL v3 (https://www.gnu.org/licenses/gpl-3.0.txt). Windows users should not attempt to download these files with a web browser.
References: Add references at CitEc
Citations:

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:boc:bocode:s458710

Ordering information: This software item can be ordered from
http://repec.org/docs/ssc.php

Access Statistics for this software item

More software in Statistical Software Components from Boston College Department of Economics Boston College, 140 Commonwealth Avenue, Chestnut Hill MA 02467 USA. Contact information at EDIRC.
Bibliographic data for series maintained by Christopher F Baum ().