A confidence paradigm for classification systems

Nathan J. Leap and Kenneth W. Bauer

Naval Research Logistics (NRL), 2011, vol. 58, issue 3, 236-254

Abstract: There is no universally accepted methodology to determine how much confidence one should place in the output of a classification system. In this article, we develop a confidence paradigm. This is a theoretical framework that attempts to unite the viewpoints of the classification system developer (or engineer) and the classification system user (or warfighter). The developer designs and tests the classification system at a macro‐level. The user fields the system in an environment often quite different than the environment used to develop the system. The user operates at a micro‐level and is interested in the indications as they are made by the system. The paradigm is based on the assumptions that the system confidence acts like or can be modelled as value, and that indication confidence can be modelled as a function of the posterior probability estimates. The viewpoints of the developer and the user are unified through the fundamental proposition that the expected value of the user's confidence should be approximately equal to the developer's confidence. This paradigm provides a direct link between traditional decision analysis techniques and traditional pattern recognition techniques. This methodology is applied to an automatic target recognition data set, and the results demonstrate the sort of behavior that would be expected from a rational confidence measure. © 2011 Wiley Periodicals, Inc. Naval Research Logistics, 2011

Date: 2011
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://doi.org/10.1002/nav.20426

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:wly:navres:v:58:y:2011:i:3:p:236-254

Access Statistics for this article

More articles in Naval Research Logistics (NRL) from John Wiley & Sons
Bibliographic data for series maintained by Wiley Content Delivery ().