Reflections on inductive thematic saturation as a potential metric for measuring the validity of an inductive thematic analysis with LLMs

Stefano De Paoli () and Walter S. Mathis ()
Additional contact information
Stefano De Paoli: Abertay University
Walter S. Mathis: Yale School of Medicine

Quality & Quantity: International Journal of Methodology, 2025, vol. 59, issue 1, No 29, 683-709

Abstract: Abstract This paper presents a set of reflections on saturation and the use of Large Language Models (LLMs) for performing Thematic Analysis (TA). The paper suggests that initial thematic saturation (ITS) could be used as a metric to assess part of the transactional validity of TA with LLM, focusing on the initial coding. The paper presents the initial coding of two datasets of different sizes, and it reflects on how the LLM reaches some form of analytical saturation during the coding. The procedure proposed in this work leads to the creation of two codebooks, one comprising the total cumulative initial codes and the other the total unique codes. The paper proposes a metric to synthetically measure ITS using a simple mathematical calculation employing the ratio between slopes of the unique and total codes. The paper contributes to the initial body of work exploring how to perform qualitative analysis with LLMs.

Keywords: Qualitative coding; Large language models; Saturation; Metric; Thematic Analysis (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s11135-024-01950-6 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:qualqt:v:59:y:2025:i:1:d:10.1007_s11135-024-01950-6

Ordering information: This journal article can be ordered from
http://www.springer.com/economics/journal/11135

DOI: 10.1007/s11135-024-01950-6

Access Statistics for this article

Quality & Quantity: International Journal of Methodology is currently edited by Vittorio Capecchi

More articles in Quality & Quantity: International Journal of Methodology from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().