Assessing political bias in large language models

Luca Rettenberger (), Markus Reischl and Mark Schutera
Additional contact information
Luca Rettenberger: Karlsruhe Institute of Technology
Markus Reischl: Karlsruhe Institute of Technology
Mark Schutera: Karlsruhe Institute of Technology

Journal of Computational Social Science, 2025, vol. 8, issue 2, No 16, 17 pages

Abstract: Abstract Evaluating bias in Large Language Models (LLMs) has become a pivotal issue in current Artificial Intelligence (AI) research due to their significant impact on societal dynamics. Recognizing political bias in LLMs is particularly important as they approach performative prediction, influencing societal behavior and political events, such as the upcoming European Parliament elections. From a German voter’s perspective, we evaluate the political bias of the currently most popular open-source LLMs concerning political issues within the European Union. To do so, we use the "Wahl-O-Mat," a voting advice application used in Germany. We show that larger models, such as Llama3-70B, tend to align more closely with left-leaning political parties, while smaller models often remain neutral, particularly when prompted in English. The central finding is that LLMs are similarly biased, with low variances in the alignment concerning a specific party. Our findings offer crucial insights for developers and policymakers to understand and mitigate LLM biases, emphasizing the need for rigorous bias assessment to ensure the integrity and trustworthiness of AI applications.

Keywords: Large language models; Political bias; European elections; Deep learning; Machine learning (search for similar items in EconPapers)
Date: 2025
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s42001-025-00376-w Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:jcsosc:v:8:y:2025:i:2:d:10.1007_s42001-025-00376-w

Ordering information: This journal article can be ordered from
http://www.springer. ... iences/journal/42001

DOI: 10.1007/s42001-025-00376-w

Access Statistics for this article

Journal of Computational Social Science is currently edited by Takashi Kamihigashi

More articles in Journal of Computational Social Science from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().