The decline effect in evaluations of the impact of the Strengthening Families Program for Youth 10-14 (SFP 10-14) on adolescent substance use

Dennis M. Gorman

Children and Youth Services Review, 2017, vol. 81, issue C, 29-39

Abstract: The decline effect refers to the situation whereby an initially positive set of research findings concerning a phenomenon fail to be replicated in subsequent studies. One explanation for the occurrence of the decline effect is that the initial positive results reported in publications from a single research team are the product of the use of flexible data analysis practices and selective reporting. The present paper reviews evaluations of the Strengthening Families Program for Youth 10-14 (SFP 10-14) conducted by the research group who developed the program and five independent replication studies. The focus is on the effects reported for adolescent substance use. It is argued that the isolated statistically significant results reported in the original studies were produced through the use of flexible data analysis practices such as changing outcome variables across publications and post hoc alterations in study design. In light of this, it is unsurprising that the five independent evaluations have failed to replicate these results. A strength of three of the independent evaluations is that they were registered in a clinical trials registry and had published their study protocols. Both of these procedures limit the opportunity to conduct flexible data analysis. It is recommended that future evaluations of the SFP 10-14 should employ these procedures as well as utilizing the registered reports publication process, making their data and methods openly available to other researchers, and including a skeptic in their research team.

Keywords: Substance use; Family-based prevention; Replication; Flexible data analysis; Selective reporting; Decline effect (search for similar items in EconPapers)
Date: 2017
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0190740917302761
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:cysrev:v:81:y:2017:i:c:p:29-39

DOI: 10.1016/j.childyouth.2017.07.009

Access Statistics for this article

Children and Youth Services Review is currently edited by Duncan Lindsey

More articles in Children and Youth Services Review from Elsevier
Bibliographic data for series maintained by Catherine Liu ().