Building Less Flawed Metrics

David Manheim

MPRA Paper from University Library of Munich, Germany

Abstract: Metrics are useful for measuring systems and motivating behaviors. Unfortunately, naive application of metrics to a system can distort the system in ways that undermine the original goal. The problem was noted independently by Campbell and Goodhart, and in some forms it is not only common, but unavoidable due to the nature of metrics. There are two distinct but interrelated problems that must be overcome in building better metrics; first, specifying metrics more closely related to the true goals, and second, preventing the recipients from gaming the difference between the reward system and the true goal. This paper describes several approaches to designing metrics, beginning with design considerations and processes, then discussing specific strategies including secrecy, randomization, diversification, and post-hoc specification. Finally, it will discuss important desiderata and the trade-offs involved in each approach.

Keywords: Metrics; Goodhart's Law; Campbell's Law; Cobra Effect; Management; Control Theory (search for similar items in EconPapers)
JEL-codes: D8 D80 I26 I28 J48 Z18 (search for similar items in EconPapers)
Date: 2018-11-26
New Economics Papers: this item is included in nep-pke
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
https://mpra.ub.uni-muenchen.de/90649/1/MPRA_paper_90649.pdf original version (application/pdf)
https://mpra.ub.uni-muenchen.de/98288/1/MPRA_paper_90649.pdf revised version (application/pdf)
https://mpra.ub.uni-muenchen.de/98288/5/MPRA_paper_98288.pdf revised version (application/pdf)
https://mpra.ub.uni-muenchen.de/118443/13/MPRA_paper_118443.pdf revised version (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:pra:mprapa:90649

Access Statistics for this paper

More papers in MPRA Paper from University Library of Munich, Germany Ludwigstraße 33, D-80539 Munich, Germany. Contact information at EDIRC.
Bibliographic data for series maintained by Joachim Winter ().