The Application of Randomized and Quasi-Experimental Designs in Federal Government Program Evaluation

Donna Smith (), Andy Handouyahia, Deen Taposh, Danièle Laliberté, Essolaba Aouli and Marie-Gaelle Njambe
Additional contact information
Donna Smith: Evaluation Directorate, Employment and Social Development Canada
Andy Handouyahia: Evaluation Directorate, Employment and Social Development Canada
Deen Taposh: Evaluation Directorate, Employment and Social Development Canada
Danièle Laliberté: Evaluation Directorate, Employment and Social Development Canada
Essolaba Aouli: Evaluation Directorate, Employment and Social Development Canada
Marie-Gaelle Njambe: Evaluation Directorate, Employment and Social Development Canada

Chapter Chapter 5 in Public Policy Evaluation and Analysis, 2024, pp 69-112 from Springer

Abstract: Abstract The chapter aims to share the knowledge of methodological approaches used to assess the effectiveness of labour market programs in the federal government setting both randomized controlled trials (RCTs) and quasi-experimental designs. A few cases of experimental studies are discussed to illustrate some of the key challenges of implementing RCTs in the Government context, including, low external validity, ethical considerations and the timing of the RCT roll-out. Given these challenges, the chapter highlights that quasi-experimental methods could help produce robust evaluation findings and recommendations to support decision making at the federal level. For example, the chapter outlines the use of matching techniques combined with the difference-in-differences method to control for observable and time-invariant unobservable factors in estimating the causal effects of labour market programs under the Employment and Social Development Canada (ESDC) portfolio. Other quasi-experimental designs are also presented, such as, the regression discontinuity design used to provide unbiased estimates of the causal effects for the Employment Insurance program, precisely the impact calculation of extending benefits at the margin. The chapter then demonstrates the application of causal machine learning to provide granular evidence about the program impacts on employment outcomes by gender, age group, disability status, indigenous, visible minority, and immigration status. Given the importance of complementary approaches to incremental impact analysis, the chapter also provides an overview of qualitative methods used in the ESDC context. Finally, examples are shared on how impact evaluations contributed to evidence-based decision making and recommendations are given for future policy and process considerations.

Keywords: Randomized controlled trials; Propensity score matching; Regression discontinuity; Causal machine learning; Employment insurance; Labour market programs (search for similar items in EconPapers)
Date: 2024
References: Add references at CitEc
Citations:

There are no downloads for this item, see the EconPapers FAQ for hints about obtaining it.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:conchp:978-3-031-67604-8_5

Ordering information: This item can be ordered from
http://www.springer.com/9783031676048

DOI: 10.1007/978-3-031-67604-8_5

Access Statistics for this chapter

More chapters in Contributions to Economics from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().