How to check a simulation study

Ian White, Tra My Pham (), Matteo Quartagno and Tim P Morris
Additional contact information
Tra My Pham: MRC Clinical Trials Unit at UCL, London, UK
Matteo Quartagno: MRC Clinical Trials Unit at UCL, London, UK
Tim P Morris: MRC Clinical Trials Unit at UCL, London, UK

UK Stata Conference 2023 from Stata Users Group

Abstract: Simulation studies are a powerful tool in biostatistics, but they can be hard to conduct successfully. Sometimes unexpected results are obtained. We offer advice on how to check a simulation study when this occurs, and how to design and conduct the study to give results that are easier to check. Simulation studies should be designed to include some settings where answers are already known. Code should be written in stages and data generating mechanisms should be checked before simulated data are analysed. Results should be explored carefully, with scatterplots of standard error estimates against point estimates a surprisingly powerful tool. When estimation fails or there are outlying estimates, these should be identified, understood, and dealt with by changing data generating mechanisms or coding realistic hybrid analysis procedures. Finally, we give a series of ideas that have been useful to us in the past for checking unexpected results. Following our advice may help to prevent errors and to improve the quality of published simulation studies. We illustrate the ideas with a simple but realistic simulation study in Stata.

Date: 2023-09-10
New Economics Papers: this item is included in nep-cmp and nep-ger
References: Add references at CitEc
Citations:

Downloads: (external link)
http://repec.org/lsug2023/Stata_UK23_Pham.pptx

Related works:
Working Paper: How to check a simulation study (2023)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:boc:lsug23:16

Access Statistics for this paper

More papers in UK Stata Conference 2023 from Stata Users Group Contact information at EDIRC.
Bibliographic data for series maintained by Christopher F Baum ().