Using NLP to create preliminary causal system maps for use in policy analysis

Peter Barbrook-Johnson and Yuan Fu

INET Oxford Working Papers from Institute for New Economic Thinking at the Oxford Martin School, University of Oxford

Abstract: The use of causal systems mapping in interdisciplinary and policy research has increased in recent years. Causal system maps typically rely on stakeholder opinion for their creation. This works well but does not make use of all available literature and can be time-consuming. For most topics, there is an abundance of text data in easily identifiable journal papers, grey literature, and policy documents. Using this data to support causal systems mapping exercises has the potential to make them more comprehensive and connected to evidence. There is also potential for the creation of maps using this data, to be done quickly, if the processes used become routine. In this paper, we develop an approach using Natural Language Processing (NLP) techniques and text data from journal papers to create preliminary causal system maps. Using the example topic of power sector decarbonisation policies and comparisons to a related participatory exercise, we consider the best techniques to use, the workflows which might speed up mapping exercises, and potential risks. The approach produces familiar factors and logical individual relationships, but causal maps with structure that mirrors attention in the literature rather than real causal patterns, and which overemphasise connections directly between policies and outcomes, rather than longer more realistic causal chains. We highlight the importance of choice of documents and sections of documents to use, and that the NLP workflow is full of subjective judgements and decisions. We argue that a clear purpose must be identified before beginning, to inform these choices; purely exploratory, which are relatively common with systems mapping exercises, are likely to be flawed.

Keywords: Natural language processing; Pretrained language model; Deep learning; Systems mapping; Causal maps; Policy analysis; Decarbonisation policy (search for similar items in EconPapers)
Pages: 43 pages
Date: 2025-04
References: Add references at CitEc
Citations:

Downloads: (external link)
https://oms-inet.files.svdcdn.com/production/files ... 25.pdf?dm=1744107982 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:amz:wpaper:2025-09

Access Statistics for this paper

More papers in INET Oxford Working Papers from Institute for New Economic Thinking at the Oxford Martin School, University of Oxford Contact information at EDIRC.
Bibliographic data for series maintained by INET Oxford admin team ().