Data Leakage and Deceptive Performance: A Critical Examination of Credit Card Fraud Detection Methodologies

Khizar Hayat () and Baptiste Magnier
Additional contact information
Khizar Hayat: College of Arts and Sciences, University of Nizwa, Nizwa 616, Oman
Baptiste Magnier: Euromov Digital Health in Motion, Université de Montpellier, IMT Mines Alès, 30100 Alès, France

Mathematics, 2025, vol. 13, issue 16, 1-28

Abstract: This study critically examines the methodological rigor in credit card fraud detection research, revealing how fundamental evaluation flaws can overshadow algorithmic sophistication. Through deliberate experimentation with improper evaluation protocols, we demonstrate that even simple models can achieve deceptively impressive results when basic methodological principles are violated. Our analysis identifies four critical issues plaguing current approaches: (1) pervasive data leakage from improper preprocessing sequences, (2) intentional vagueness in methodological reporting, (3) inadequate temporal validation for transaction data, and (4) metric manipulation through recall optimization at precision’s expense. We present a case study showing how a minimal neural network architecture with data leakage outperforms many sophisticated methods reported in literature, achieving 99.9% recall despite fundamental evaluation flaws. These findings underscore that proper evaluation methodology matters more than model complexity in fraud detection research. The study serves as a cautionary example of how methodological rigor must precede architectural sophistication, with implications for improving research practices across machine learning applications. Compared to several recent studies reporting near-perfect recall (often exceeding 99%) using complex deep models, our corrected evaluation with a simple MLP baseline yields more modest but reliable metrics, exposing the overestimation common in flawed pipelines.

Keywords: credit card fraud; data leakage; preprocessing flaws; methodological rigor; model performance (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/13/16/2563/pdf (application/pdf)
https://www.mdpi.com/2227-7390/13/16/2563/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:13:y:2025:i:16:p:2563-:d:1721596

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().