Metrics for Evaluating the Performance of Automated Valuation Models

Miriam Steurer () and Robert Hill ()
Additional contact information
Miriam Steurer: University of Graz, Austria

No 2019-02, Graz Economics Papers from University of Graz, Department of Economics

Abstract: Automated Valuation Models (AVMs) based on Machine Learning (ML) algorithms are widely used for the prediction of house prices. While there is consensus in the literature that cross-validation (CV) should be used for model selection in this context, the question of which performance metrics to use is generally neglected. Here we collect the most commonly used metrics from the AVM literature and elsewhere, and evaluate them with respect to two symmetry conditions: symmetry with respect to prediction error rates and symmetry with respect to the treatment of actual and predicted values. While none of the commonly used metrics satisfy both conditions, we propose a number of new metrics that do. We also show how popular existing metrics can be altered so that they adhere to these conditions. To illustrate our findings we compare the performance of 5 ML-based AVMs and find, that the most popular metrics in the AVM literature can generate misleading results. A different picture emerges when the full set of metrics is considered, and especially when we focus on four key metrics with the best symmetry properties.

Keywords: Performance metric; Automated valuation model (AVM); Appraisal; Prediction error; Model selection (search for similar items in EconPapers)
JEL-codes: C45 C53 (search for similar items in EconPapers)
Date: 2019-02
New Economics Papers: this item is included in nep-big, nep-cmp and nep-ecm
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://unipub.uni-graz.at/obvugrveroeff/download/ ... riginalFilename=true

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:grz:wpaper:2019-02

Ordering information: This working paper can be ordered from
https://repecgrz.uni-graz.at/RePEc/

Access Statistics for this paper

More papers in Graz Economics Papers from University of Graz, Department of Economics University of Graz, Universitaetsstr. 15/F4, 8010 Graz, Austria. Contact information at EDIRC.
Bibliographic data for series maintained by Stefan Borsky ().