Validation of simulation models: regression analysis revisited

Jack P.C. Kleijnen, B. Bettonville and Willem van Groenendaal ()

No 7, Discussion Paper from Tilburg University, Center for Economic Research

Abstract: This paper proves that it is wrong to require that regressing a model's outputs on the observed real outcomes gives a 45 degrees line through the origin (unit slope, zero intercept). Therefore this paper proposes an alternative requirement: the responses of the model and the real system should have the same means and the same variances. To test whether this requirement is satisfied, a novel statisti-cal procedure is derived. This procedure regresses the differences of simulated and real responses on their sums. The old and the new procedures are investigated in an extensive Monte Carlo experiment that simulates queueing systems. The conclusions of this experiment are that (i) the old test rejects a valid simulation model substantially more often than the new test does; (ii) the intuitive test shows 'perverse' behavior in a certain domain: the worse the simulation model, the higher its probability of acceptance; and (iii) the novel test does not reject a valid simulation model too often (its type I error probability is correct), provided the queueing response is transformed logarithmically.

Keywords: Statistical; Validation (search for similar items in EconPapers)
Date: 1996
View list of references View citations in EconPapers

Downloads: (external link)
http://arno.uvt.nl/show.cgi?fid=3282 (application/pdf)
http://arno.uvt.nl/show.cgi?fid=3283 (application/postscript)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: http://EconPapers.repec.org/RePEc:dgr:kubcen:19967

Access Statistics for this paper

More papers in Discussion Paper from Tilburg University, Center for Economic Research
Series data maintained by Corry Stuyts ().