Beyond Accuracy: Benchmarking Machine Learning Models for Efficient and Sustainable SaaS Decision Support

Efthimia Mavridou, Eleni Vrochidou, Michail Selvesakis and George A. Papakostas ()
Additional contact information
Efthimia Mavridou: MLV Research Group, Department of Informatics, Democritus University of Thrace, 65404 Kavala, Greece
Eleni Vrochidou: MLV Research Group, Department of Informatics, Democritus University of Thrace, 65404 Kavala, Greece
Michail Selvesakis: MLV Research Group, Department of Informatics, Democritus University of Thrace, 65404 Kavala, Greece
George A. Papakostas: MLV Research Group, Department of Informatics, Democritus University of Thrace, 65404 Kavala, Greece

Future Internet, 2025, vol. 17, issue 10, 1-37

Abstract: Machine learning (ML) methods have been successfully employed to support decision-making for Software as a Service (SaaS) providers. While most of the published research primarily emphasizes prediction accuracy, other important aspects, such as cloud deployment efficiency and environmental impact, have received comparatively less attention. It is also critical to effectively use factors such as training time, prediction time and carbon footprint in production. SaaS decision support systems use the output of ML models to provide actionable recommendations, such as running reactivation campaigns for users who are likely to churn. To this end, in this paper, we present a benchmarking comparison of 17 different ML models for churn prediction in SaaS, which include cloud deployment efficiency metrics (e.g., latency, prediction time, etc.) and sustainability metrics (e.g., CO 2 emissions, consumed energy, etc.) along with predictive performance metrics (e.g., AUC, Log Loss, etc.). Two public datasets are employed, experiments are repeated on four different machines, locally and on the cloud, while a new weighted Green Efficiency Weighted Score (GEWS) is introduced, as steps towards choosing the simpler, greener and more efficient ML model. Experimental results indicated XGBoost and LightGBM as the models capable of offering a good balance on predictive performance, fast training, inference times, and limited emissions, while the importance of region selection towards minimizing the carbon footprint of the ML models was confirmed.

Keywords: machine learning; Software as a Service (SaaS); decision support systems; churn prediction; carbon footprint; CO 2 emissions; sustainable AI; green AI; benchmarking; machine learning (search for similar items in EconPapers)
JEL-codes: O3 (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/1999-5903/17/10/467/pdf (application/pdf)
https://www.mdpi.com/1999-5903/17/10/467/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jftint:v:17:y:2025:i:10:p:467-:d:1768738

Access Statistics for this article

Future Internet is currently edited by Ms. Grace You

More articles in Future Internet from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().