The Fragility of Meta-Regression Models in Observational Research

Stephan B. Bruns ()
Additional contact information
Stephan B. Bruns: University of Kassel

MAGKS Papers on Economics from Philipps-Universität Marburg, Faculty of Business Administration and Economics, Department of Economics (Volkswirtschaftliche Abteilung)

Abstract: Many meta-regression analyses that synthesize estimates from primary studies have now been published in economics. Meta-regression models attempt to infer the presence of genuine empirical effects even if the authors of primary studies select statistically significant and theory-confirming estimates for publication. Meta-regression models were originally developed for the synthesis of experimental research where randomization ensures unbiased and consistent estimation of the effect of interest. Most economics research is, however, observational and authors of primary studies can search across different regression specifications for statistically significant and theory-confirming estimates. Each regression specification may possibly suffer from biases such as omitted-variable biases that result in biased and inconsistent estimation of the effect of interest. We show that if the authors of primary studies search for statistically significant and theory-confirming estimates, meta-regression models tend to systematically make false-positive findings of genuine empirical effects. The ubiquity of such search processes for specific results may limit the applicability of meta-regression models in identifying genuine empirical effects in economics.

Keywords: Meta-regression; meta-analysis; p-hacking; publication bias; omitted-variable bias; sampling variability; sampling error; Monte Carlo simulation (search for similar items in EconPapers)
JEL-codes: C12 C15 C40 (search for similar items in EconPapers)
Pages: 17 pages
Date: 2016
New Economics Papers: this item is included in nep-pke
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Forthcoming in

Downloads: (external link)
http://www.uni-marburg.de/fb02/makro/forschung/mag ... 16/03-2016_bruns.pdf First 201603 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:mar:magkse:201603

Access Statistics for this paper

More papers in MAGKS Papers on Economics from Philipps-Universität Marburg, Faculty of Business Administration and Economics, Department of Economics (Volkswirtschaftliche Abteilung) Contact information at EDIRC.
Bibliographic data for series maintained by Bernd Hayo ().