Divergence versus decision P‐values: A distinction worth making in theory and keeping in practice: Or, how divergence P‐values measure evidence even when decision P‐values do not

Sander Greenland

Scandinavian Journal of Statistics, 2023, vol. 50, issue 1, 54-88

Abstract: There are two distinct definitions of “P‐value” for evaluating a proposed hypothesis or model for the process generating an observed dataset. The original definition starts with a measure of the divergence of the dataset from what was expected under the model, such as a sum of squares or a deviance statistic. A P‐value is then the ordinal location of the measure in a reference distribution computed from the model and the data, and is treated as a unit‐scaled index of compatibility between the data and the model. In the other definition, a P‐value is a random variable on the unit interval whose realizations can be compared to a cutoff α to generate a decision rule with known error rates under the model and specific alternatives. It is commonly assumed that realizations of such decision P‐values always correspond to divergence P‐values. But this need not be so: Decision P‐values can violate intuitive single‐sample coherence criteria where divergence P‐values do not. It is thus argued that divergence and decision P‐values should be carefully distinguished in teaching, and that divergence P‐values are the relevant choice when the analysis goal is to summarize evidence rather than implement a decision rule.

Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
https://doi.org/10.1111/sjos.12625

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bla:scjsta:v:50:y:2023:i:1:p:54-88

Ordering information: This journal article can be ordered from
http://www.blackwell ... bs.asp?ref=0303-6898

Access Statistics for this article

Scandinavian Journal of Statistics is currently edited by ÿrnulf Borgan and Bo Lindqvist

More articles in Scandinavian Journal of Statistics from Danish Society for Theoretical Statistics, Finnish Statistical Society, Norwegian Statistical Association, Swedish Statistical Association
Bibliographic data for series maintained by Wiley Content Delivery ().