Measuring Online Media Ideology with Large Language Models and "Multi-Cue Classification"

Lucas Paulo da Silva
Additional contact information
Lucas Paulo da Silva: Trinity College Dublin

No zmtqp_v1, SocArXiv from Center for Open Science

Abstract: Measuring media ideology is essential for researching media bias, media effects, and various important topics in political science, communication, and other social sciences. However, given journalistic norms of objectivity and the complexity of ideology, measuring media ideology accurately is uniquely challenging. Large language models (LLMs) have become valuable tools in this endeavor. Based on media communication theories, I argue that media ideology is expressed via different cues -- the topic, argument, framing, criticism, and sources of the media content -- and that LLMs often miss these. Standard methods of LLM classification also offer little control, flexibility, and data granularity to researchers. Drawing on insights about computational and quantitative measurement methodologies, I introduce the "Multi-Cue Classification" (MQ-Class) approach. With MQ-Class, an LLM classifies the different ideological cues separately and researchers then apply pre-specified weights and thresholds to combine them into one label per text. I compare standard LLM and MQ-Class methods using two example tasks -- classifying the economic and cultural ideologies of a novel sample of online media articles. Across multiple tests, MQ-Class is more accurate and puts researchers "back in the driver's seat." I conclude by discussing how MQ-Class could be implemented for other classification tasks and data.

Date: 2026-02-20
References: Add references at CitEc
Citations:

Downloads: (external link)
https://osf.io/download/69976d7c82e6f047d565647b/

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:osf:socarx:zmtqp_v1

DOI: 10.31219/osf.io/zmtqp_v1

Access Statistics for this paper

More papers in SocArXiv from Center for Open Science
Bibliographic data for series maintained by OSF ().