On The Problem of Relevance in Statistical Inference

Subhadeep Mukhopadhyay and Kaijun Wang

Econometrics and Statistics, 2023, vol. 25, issue C, 93-109

Abstract: Given a large cohort of “similar” cases one can construct an efficient statistical inference procedure by learning from the experience of others (also known as “borrowing strength” from the ensemble). But, it is not obvious how to go about gathering strength when each piece of information is fuzzy—part of a massive database of heterogeneous cases. The danger is that borrowing information from irrelevant cases might heavily damage the quality of the inference! This raises some fundamental questions for big data inference: When (not) to borrow? Whom (not) to borrow? How (not) to borrow? These questions are at the heart of the “Problem of Relevance” in statistical inference – a puzzle that has remained too little addressed since its inception nearly half a century ago [Efron and Morris, J. Am. Stat. Assoc, 67, 337 (1972)]. A new model of large-scale inference is developed to tackle some of the unsettled issues that surround the relevance problem. Through examples, it is demonstrated how our new statistical perspective answers previously unanswerable questions in a realistic and feasible way.1

Keywords: Customized inference; Empirical Bayes; Global-to-local representation; Heterogeneity; LASER; Relevance paradox; Relevant null; Relevant prior; Reproducibility (search for similar items in EconPapers)
Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S2452306221001283
Full text for ScienceDirect subscribers only. Contains open access articles

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:ecosta:v:25:y:2023:i:c:p:93-109

DOI: 10.1016/j.ecosta.2021.10.013

Access Statistics for this article

Econometrics and Statistics is currently edited by E.J. Kontoghiorghes, H. Van Dijk and A.M. Colubi

More articles in Econometrics and Statistics from Elsevier
Bibliographic data for series maintained by Catherine Liu ().