Metrics for evaluating the performance of machine learning based automated valuation models

Miriam Steurer, Robert Hill () and Norbert Pfeifer

Journal of Property Research, 2021, vol. 38, issue 2, 99-129

Abstract: Automated Valuation Models (AVMs) based on Machine Learning (ML) algorithms are widely used for predicting house prices. While there is consensus in the literature that cross-validation (CV) should be used for model selection in this context, the interdisciplinary nature of the subject has made it hard to reach consensus over which metrics to use at each stage of the CV exercise. We collect 48 metrics (from the AVM literature and elsewhere) and classify them into seven groups according to their structure. Each of these groups focuses on a particular aspect of the error distribution. Depending on the type of data and the purpose of the AVM, the needs of users may be met by some classes, but not by others. In addition, we show in an empirical application how the choice of metric can influence the choice of model, by applying each metric to evaluate five commonly used AVM models. Finally – since it is not always practicable to produce 48 different performance metrics – we provide a short list of 7 metrics that are well suited to evaluate AVMs. These metrics satisfy a symmetry condition that we find is important for AVM performance, and can provide a good overall model performance ranking.

Date: 2021
References: Add references at CitEc
Citations: View citations in EconPapers (6)

Downloads: (external link)
http://hdl.handle.net/10.1080/09599916.2020.1858937 (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:taf:jpropr:v:38:y:2021:i:2:p:99-129

Ordering information: This journal article can be ordered from
http://www.tandfonline.com/pricing/journal/RJPR20

DOI: 10.1080/09599916.2020.1858937

Access Statistics for this article

Journal of Property Research is currently edited by Bryan MacGregor

More articles in Journal of Property Research from Taylor & Francis Journals
Bibliographic data for series maintained by Chris Longhurst ().