AI-Assisted Hate Speech Moderation—How Information on AI-Based Classification Affects the Human Brain-In-The-Loop

Nadine R. Gier-Reinartz (), Vita E. M. Zimmermann-Janssen () and Peter Kenning ()
Additional contact information
Nadine R. Gier-Reinartz: Heinrich Heine University
Vita E. M. Zimmermann-Janssen: Heinrich Heine University
Peter Kenning: Heinrich Heine University

A chapter in Information Systems and Neuroscience, 2024, pp 45-56 from Springer

Abstract: Abstract Every day, social media content moderators must decide within seconds hundreds of times whether or not user-generated content constitutes hate speech. Although IS research is making continual progress in automatically detecting potential hate speech content through AI-assisted processing, the final decision still resides in the human-in-the-loop. To support the content moderators, the results of AI-based classifications are regularly displayed during the decision-making process—but is this advisable? To approach an answer, the neural and behavioral effects of two opposing AI-based classifications are tested against each other. The results from a fNIRS experiment show that opposing AI-based classifications leads to different cortical activation patterns, which in turn depend on the individual’s importance of hate speech prevention. Moreover, this exploratory study indicates that AI-based classifications may also induce a “cortical relief” seemingly cause behavioral effects that at least cast doubt on the validity and desirability of the AI-assisted human decision.

Keywords: fNIRS; Hate speech detection; Content moderation; Human-in-the-Loop; Artificial intelligence; Decision support systems (search for similar items in EconPapers)
Date: 2024
References: Add references at CitEc
Citations:

There are no downloads for this item, see the EconPapers FAQ for hints about obtaining it.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:lnichp:978-3-031-58396-4_5

Ordering information: This item can be ordered from
http://www.springer.com/9783031583964

DOI: 10.1007/978-3-031-58396-4_5

Access Statistics for this chapter

More chapters in Lecture Notes in Information Systems and Organization from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().