The Dutch Scaler Performance Indicator: How Much Did My Model Actually Learn?

Etienne Pieter van de Bijl (), Jan Gerard Klein (), Joris Pries (), Sandjai Bhulai () and Robert Douwe van der Mei ()
Additional contact information
Etienne Pieter van de Bijl: Centrum Wiskunde & Informatica
Jan Gerard Klein: Centrum Wiskunde & Informatica
Joris Pries: Centrum Wiskunde & Informatica
Sandjai Bhulai: Vrije Universiteit Amsterdam
Robert Douwe van der Mei: Centrum Wiskunde & Informatica

Journal of Classification, 2025, vol. 42, issue 3, No 8, 639-659

Abstract: Abstract Evaluation metrics provide a means for quantifying and comparing performances of supervised learning models, but drawing meaningful conclusions from acquired scores requires a contextual framework. Our paper addresses this by introducing the Dutch scaler (DS), a novel performance indicator for binary classification models. It quantifies a model’s learning by contextualizing empirical metric scores with a baseline (Dutch draw) and a new instrument (Dutch oracle) representing the prediction quality of an “optimal” classifier. The DS performance indicator expresses the relative contribution of these components to obtain a model’s score, specifying the actual learning quality. We derived closed-form expressions to map metric scores to DS scores for common evaluation metrics and categorized them by their functional form and second derivative. The DS enhances the assessment of classifiers and facilitates a framework to compare prediction quality differences between models with varying metric scores.

Keywords: Performance indicator; Performance metrics; Binary classification; Machine learning; Supervised learning; Evaluation (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://link.springer.com/10.1007/s00357-025-09510-9 Abstract (text/html)
Access to the full text of the articles in this series is restricted.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:jclass:v:42:y:2025:i:3:d:10.1007_s00357-025-09510-9

Ordering information: This journal article can be ordered from
http://www.springer. ... hods/journal/357/PS2

DOI: 10.1007/s00357-025-09510-9

Access Statistics for this article

Journal of Classification is currently edited by Douglas Steinley

More articles in Journal of Classification from Springer, The Classification Society
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().