The Content Moderator’s Dilemma: Removal of Toxic Content and Distortions to Online Discourse

Mahyar Habibi, Dirk Hovy and Carlo Schwarz
Additional contact information
Mahyar Habibi: Department of Economics, Bocconi University
Dirk Hovy: Department of Computing Sciences, Bocconi University

CAGE Online Working Paper Series from Competitive Advantage in the Global Economy (CAGE)

Abstract: There is an ongoing debate about how to moderate toxic speech on social media and the impact of content moderation on online discourse. This paper proposes and validates a methodology for measuring the content-moderation-induced distortions in online discourse using text embeddings from computational linguistics. Applying the method to a representative sample of 5 million US political Tweets, we find that removing toxic Tweets significantly alters the semantic composition of content. The magnitudes of the distortions are comparable to removing 4 out of 67 topics from the online discourse at random. This finding is consistent across different embedding models, toxicity metrics, and samples. Importantly, we demonstrate that these effects are not solely driven by toxic language but by the removal of topics often expressed in toxic form. We propose an alternative approach to content moderation that uses generative Large Language Models to rephrase toxic Tweets, preserving their salvageable content rather than removing them entirely. We show that this rephrasing strategy reduces toxicity while mitigating distortions in online content.

Keywords: social media; content moderation; content distortions; toxicity; embeddings JEL Classification: (search for similar items in EconPapers)
Date: 2026
References: Add references at CitEc
Citations:

Downloads: (external link)
https://warwick.ac.uk/fac/soc/economics/research/c ... tions/wp793.2026.pdf

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:cge:wacage:793

Access Statistics for this paper

More papers in CAGE Online Working Paper Series from Competitive Advantage in the Global Economy (CAGE) Contact information at EDIRC.
Bibliographic data for series maintained by Jane Snape ().