Some Lessons From 50 Years of Multiarm Public Policy Experiments

Larry L. Orr and Daniel Gubits

Evaluation Review, 2023, vol. 47, issue 1, 11-38

Abstract: In this article, we explore the reasons why multiarm trials have been conducted and the design and analysis issues they involve. We point to three fundamental reasons for such designs: (1) Multiarm designs allow the estimation of “response surfaces†—that is, the variation in response to an intervention across a range of one or more continuous policy parameters. (2) Multiarm designs are an efficient way to test multiple policy approaches to the same social problem simultaneously, either to compare the effects of the different approaches or to estimate the effect of each separately. (3) Multiarm designs may allow for the estimation of the separate and combined effects of discrete program components. We illustrate each of these objectives with examples from the history of public policy experimentation over the past 50 years and discuss some design and analysis issues raised by each, including sample allocation, statistical power, multiple comparisons, and alignment of analysis with goals of the evaluation.

Keywords: economic evaluation; methodology (if appropriate); outcome evaluation (other than economic evaluation); design and evaluation of programs and policies (search for similar items in EconPapers)
Date: 2023
References: Add references at CitEc
Citations:

Downloads: (external link)
https://journals.sagepub.com/doi/10.1177/0193841X20977332 (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:sae:evarev:v:47:y:2023:i:1:p:11-38

DOI: 10.1177/0193841X20977332

Access Statistics for this article

More articles in Evaluation Review
Bibliographic data for series maintained by SAGE Publications ().