Quantifying and addressing the prevalence and bias of study designs in the environmental and social sciences

Alec P. Christie (), David Abecasis, Mehdi Adjeroud, Juan C. Alonso, Tatsuya Amano, Alvaro Anton, Barry P. Baldigo, Rafael Barrientos, Jake E. Bicknell, Deborah A. Buhl, Just Cebrian, Ricardo S. Ceia, Luciana Cibils-Martina, Sarah Clarke, Joachim Claudet, Michael D. Craig, Dominique Davoult, Annelies Backer, Mary K. Donovan, Tyler D. Eddy, Filipe M. França, Jonathan P. A. Gardner, Bradley P. Harris, Ari Huusko, Ian L. Jones, Brendan P. Kelaher, Janne S. Kotiaho, Adrià López-Baucells, Heather L. Major, Aki Mäki-Petäys, Beatriz Martín, Carlos A. Martín, Philip A. Martin, Daniel Mateos-Molina, Robert A. McConnaughey, Michele Meroni, Christoph F. J. Meyer, Kade Mills, Monica Montefalcone, Norbertas Noreika, Carlos Palacín, Anjali Pande, C. Roland Pitcher, Carlos Ponce, Matt Rinella, Ricardo Rocha García (), María C. Ruiz-Delgado, Juan J. Schmitter-Soto, Jill A. Shaffer, Shailesh Sharma, Anna A. Sher, Doriane Stagnol, Thomas R. Stanley, Kevin D. E. Stokesbury, Aurora Torres, Oliver Tully, Teppo Vehanen, Corinne Watts, Qingyuan Zhao and William J. Sutherland
Additional contact information
Alec P. Christie: Conservation Science Group, Department of Zoology, University of Cambridge, The David Attenborough Building
David Abecasis: Centre of Marine Sciences (CCMar), Universidade do Algarve
Mehdi Adjeroud: Institut de Recherche pour le Développement (IRD), UMR 9220 ENTROPIE & Laboratoire d’Excellence CORAIL, Université de Perpignan Via Domitia
Juan C. Alonso: Museo Nacional de Ciencias Naturales, CSIC
Tatsuya Amano: School of Biological Sciences, University of Queensland
Alvaro Anton: Education Faculty of Bilbao, University of the Basque Country (UPV/EHU). Sarriena z/g E-48940 Leioa
Barry P. Baldigo: U.S. Geological Survey, New York Water Science Center
Rafael Barrientos: Universidad Complutense de Madrid, Departamento de Biodiversidad, Ecología y Evolución, Facultad de Ciencias Biológicas
Jake E. Bicknell: Durrell Institute of Conservation and Ecology (DICE), School of Anthropology and Conservation, University of Kent
Deborah A. Buhl: U.S. Geological Survey, Northern Prairie Wildlife Research Center
Just Cebrian: Northern Gulf Institute, Mississippi State University, 1021 Balch Blvd, John C. Stennis Space Center
Ricardo S. Ceia: MARE – Marine and Environmental Sciences Centre, Dept. Life Sciences, University of Coimbra
Luciana Cibils-Martina: Universidad Nacional de Río Cuarto (UNRC)
Sarah Clarke: Marine Institute, Rinville, Oranmore
Joachim Claudet: National Center for Scientific Research, PSL Université Paris, CRIOBE, USR 3278 CNRS-EPHE-UPVD, Maison des Océans
Michael D. Craig: School of Biological Sciences, University of Western Australia
Dominique Davoult: Sorbonne Université, CNRS, UMR 7144, Station Biologique
Annelies Backer: Flanders Research Institute for Agriculture, Fisheries and Food (ILVO)
Mary K. Donovan: Marine Science Institute, University of California Santa Barbara
Tyler D. Eddy: Baruch Institute for Marine & Coastal Sciences, University of South Carolina
Filipe M. França: Lancaster Environment Centre, Lancaster University
Jonathan P. A. Gardner: School of Biological Sciences, Victoria University of Wellington
Bradley P. Harris: Fisheries, Aquatic Science and Technology Laboratory, Alaska Pacific University
Ari Huusko: Natural Resources Institute Finland
Ian L. Jones: Memorial University
Brendan P. Kelaher: National Marine Science Centre and Marine Ecology Research Centre, Southern Cross University
Janne S. Kotiaho: University of Jyväskylä
Adrià López-Baucells: Centre for Ecology, Evolution and Environmental Changes – cE3c, Faculty of Sciences, University of Lisbon
Heather L. Major: University of New Brunswick
Aki Mäki-Petäys: Voimalohi Oy
Beatriz Martín: Fundación Migres CIMA Ctra
Carlos A. Martín: Universidad Complutense de Madrid, Departamento de Biodiversidad, Ecología y Evolución, Facultad de Ciencias Biológicas
Philip A. Martin: Conservation Science Group, Department of Zoology, University of Cambridge, The David Attenborough Building
Daniel Mateos-Molina: Universidad de Murcia
Robert A. McConnaughey: RACE Division, Alaska Fisheries Science Center, National Marine Fisheries Service, NOAA, 7600 Sand Point Way NE
Michele Meroni: European Commission, Joint Research Centre (JRC)
Christoph F. J. Meyer: Centre for Ecology, Evolution and Environmental Changes – cE3c, Faculty of Sciences, University of Lisbon
Kade Mills: Victorian National Park Association
Monica Montefalcone: Environment and Life Sciences (DiSTAV), University of Genoa
Norbertas Noreika: Swedish University of Agricultural Sciences
Carlos Palacín: Museo Nacional de Ciencias Naturales, CSIC
Anjali Pande: School of Biological Sciences, Victoria University of Wellington
C. Roland Pitcher: CSIRO Oceans & Atmosphere, Queensland Biosciences Precinct
Carlos Ponce: Museo Nacional de Ciencias Naturales, CSIC
Matt Rinella: Fort Keogh Livestock and Range Research Laboratory
María C. Ruiz-Delgado: Químicos y Naturales, Universidad Pablo de Olavide
Juan J. Schmitter-Soto: El Colegio de la Frontera Sur
Jill A. Shaffer: U.S. Geological Survey, Northern Prairie Wildlife Research Center
Shailesh Sharma: Division of Fish and Wildlife, New York State Department of Environmental Conservation
Anna A. Sher: University of Denver Department of Biological Sciences
Doriane Stagnol: Sorbonne Université, CNRS, UMR 7144, Station Biologique
Thomas R. Stanley: U.S. Geological Survey, Fort Collins Science Center
Kevin D. E. Stokesbury: School for Marine Science and Technology, University of Massachusetts Dartmouth
Aurora Torres: Georges Lemaître Earth and Climate Research Centre, Earth and Life Institute, Université Catholique de Louvain
Oliver Tully: Marine Institute, Rinville, Oranmore
Teppo Vehanen: Natural Resources Institute Finland
Corinne Watts: Manaaki Whenua – Landcare Research
Qingyuan Zhao: Statistical Laboratory, Department of Pure Mathematics and Mathematical Statistics, University of Cambridge
William J. Sutherland: Conservation Science Group, Department of Zoology, University of Cambridge, The David Attenborough Building

Nature Communications, 2020, vol. 11, issue 1, 1-11

Abstract: Abstract Building trust in science and evidence-based decision-making depends heavily on the credibility of studies and their findings. Researchers employ many different study designs that vary in their risk of bias to evaluate the true effect of interventions or impacts. Here, we empirically quantify, on a large scale, the prevalence of different study designs and the magnitude of bias in their estimates. Randomised designs and controlled observational designs with pre-intervention sampling were used by just 23% of intervention studies in biodiversity conservation, and 36% of intervention studies in social science. We demonstrate, through pairwise within-study comparisons across 49 environmental datasets, that these types of designs usually give less biased estimates than simpler observational designs. We propose a model-based approach to combine study estimates that may suffer from different levels of study design bias, discuss the implications for evidence synthesis, and how to facilitate the use of more credible study designs.

Date: 2020
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.nature.com/articles/s41467-020-20142-y Abstract (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:nat:natcom:v:11:y:2020:i:1:d:10.1038_s41467-020-20142-y

Ordering information: This journal article can be ordered from
https://www.nature.com/ncomms/

DOI: 10.1038/s41467-020-20142-y

Access Statistics for this article

Nature Communications is currently edited by Nathalie Le Bot, Enda Bergin and Fiona Gillespie

More articles in Nature Communications from Nature
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().