Bots and baddies: supporting the integrity of online survey research in the face of a growing challenge

Elizabeth Hitches (), Dean Dudley (), Melissa Johnstone () and Stuart Woodcock ()
Additional contact information
Elizabeth Hitches: University of Queensland
Dean Dudley: Macquarie University
Melissa Johnstone: University of Queensland
Stuart Woodcock: University of Queensland

Quality & Quantity: International Journal of Methodology, 2025, vol. 59, issue 2, No 20, 1506 pages

Abstract: Abstract Numerous online surveys are impacted by bot and fraudulent data and the techniques employed are becoming increasingly sophisticated. However, many researchers are unaware of this growing challenge and may not be taking adequate measures to identify and remove this fake data, contaminating their findings. This paper describes the measures taken for two different surveys, one of which received a significantly higher rate of bot and fraudulent activity than the other, despite both having similar distribution pathways. One survey (n = 628 responses) contained 26 automatically identified red flags via Qualtrics™ and 10 manually identified red flags across the responses. The other survey (n = 690 responses) contained 263 automatically identified red flags via Qualtrics™ and 407 manually identified red flags across the responses. It stresses the need for both a “macro-view” or “birds-eye-view” to explore patterns and clusters across the dataset and a “micro-view” to examine the content of individual responses, looking for either single strong red flags of fraud or cumulative flags of suspicion. The time taken to do this illustrates the increased burden on researchers when one online survey receives high volumes of fake respondents. With bots and baddies increasing the chances of making a Type I or Type II error in quantitative research and infiltrating qualitative research with data which are not the authentic voices, perspectives, and lived experiences of participants, it is imperative that awareness is raised across the research community and steps are taken to safeguard the integrity of online survey research.

Keywords: Online survey; Survey research; Integrity; Fraud detection; Artificial intelligence; Scammer (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s11135-024-02001-w Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:qualqt:v:59:y:2025:i:2:d:10.1007_s11135-024-02001-w

Ordering information: This journal article can be ordered from
http://www.springer.com/economics/journal/11135

DOI: 10.1007/s11135-024-02001-w

Access Statistics for this article

Quality & Quantity: International Journal of Methodology is currently edited by Vittorio Capecchi

More articles in Quality & Quantity: International Journal of Methodology from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().