A semi-automated approach to policy-relevant evidence synthesis: combining natural language processing, causal mapping, and graph analytics for public policy

Rory Hooper, Nihit Goyal (), Kornelis Blok and Lisa Scholten
Additional contact information
Rory Hooper: Delft University of Technology
Nihit Goyal: Delft University of Technology
Kornelis Blok: Delft University of Technology
Lisa Scholten: Delft University of Technology

Policy Sciences, 2024, vol. 57, issue 4, No 7, 875-900

Abstract: Abstract Although causal evidence synthesis is critical for the policy sciences—whether it be analysis for policy or analysis of policy—its repeatable, systematic, and transparent execution remains challenging due to the growing volume, variety, and velocity of policy-relevant evidence generation as well as the complex web of relationships within which policies are usually situated. To address these shortcomings, we develop a novel, semi-automated approach to synthesizing causal evidence from policy-relevant documents. Specifically, we propose the use of natural language processing (NLP) for the extraction of causal evidence and subsequent homogenization of the text; causal mapping for the collation, visualization, and summarization of complex interdependencies within the policy system; and graph analytics for further investigation of the structure and dynamics of the causal map. We illustrate this approach by applying it to a collection of 28 articles on the emissions trading scheme (ETS), a policy instrument of increasing importance for climate change mitigation. In all, we find 300 variables and 284 cause-effect pairs in our input dataset (consisting of 4524 sentences), which are reduced to 70 unique variables and 119 cause-effect pairs after homogenization. We create a causal map depicting these relationships and analyze it to demonstrate the perspectives and policy-relevant insights that can be obtained. We compare these with select manually conducted, previous meta-reviews of the policy instrument, and find them to be not only broadly consistent but also complementary. We conclude that, despite remaining limitations, this approach can help synthesize causal evidence for policy analysis, policy making, and policy research.

Keywords: Evidence-informed policy; Emissions trading schemes (ETS); Causal mapping; Machine learning (ML); Natural language processing (NLP); Policy analysis (search for similar items in EconPapers)
Date: 2024
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s11077-024-09548-3 Abstract (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:kap:policy:v:57:y:2024:i:4:d:10.1007_s11077-024-09548-3

Ordering information: This journal article can be ordered from
http://www.springer. ... ce/journal/11077/PS2

DOI: 10.1007/s11077-024-09548-3

Access Statistics for this article

Policy Sciences is currently edited by Michael Howlett

More articles in Policy Sciences from Springer, Society of Policy Sciences
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().