Comparison of tree-based models performance in prediction of marketing campaign results using Explainable Artificial Intelligence tools

Marcin Chlebus () and Zuzanna Osika
Additional contact information
Zuzanna Osika: Faculty of Economic Sciences, University of Warsaw

No 2020-15, Working Papers from Faculty of Economic Sciences, University of Warsaw

Abstract: The research uses tree-based models to predict the success of telemarketing campaign of Portuguese bank. The Portuguese bank dataset was used in the past in different researches with different models to predict the success of campaign. We propose to use boosting algorithms, which have not been used before to predict the response for the campaign and to use Explainable AI (XAI) methods to evaluate model’s performance. The paper tries to examine whether 1) complex boosting algorithms perform better and 2) XAI tools are better indicators of models’ performance than commonly used discriminatory power’s measures like AUC. Portuguese bank telemarketing dataset was used with five machine learning algorithms, namely Random Forest (RF), AdaBoost, GBM, XGBoost and CatBoost, which were then later compared based on their AUC and XAI tools analysis – Permutated Variable Importance and Partial Dependency Profile. Two best performing models based on their AUC were XGBoost and CatBoost, with XGBoost having slightly higher AUC. Then, these models were examined using PDP and VI, which resulted in discovery of XGBoost potenitial overfitting and choosing CatBoost over XGBoost. The results show that new boosting models perform better than older models and that XAI tools could be helpful with models’ comparisons.

Keywords: direct marketing; telemarketing; relationship marketing; data mining; machine learning; random forest; adaboost; gbm; catboost; xgboost; bank marketing; XAI; variable importance; partial dependency profile (search for similar items in EconPapers)
JEL-codes: C25 C44 M31 (search for similar items in EconPapers)
Pages: 35 pages
Date: 2020
New Economics Papers: this item is included in nep-big and nep-cmp
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.wne.uw.edu.pl/index.php/download_file/5657/ First version, 2020 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:war:wpaper:2020-15

Access Statistics for this paper

More papers in Working Papers from Faculty of Economic Sciences, University of Warsaw Contact information at EDIRC.
Bibliographic data for series maintained by Marcin Bąba ().