Testing Exchangeability

Vladimir Vovk, Alexander Gammerman and Glenn Shafer
Additional contact information
Vladimir Vovk: University of London, Royal Holloway
Alexander Gammerman: University of London, Royal Holloway
Glenn Shafer: Rutgers University

Chapter Chapter 8 in Algorithmic Learning in a Random World, 2022, pp 227-263 from Springer

Abstract: Abstract In Chaps. 2 – 7 we assumed that all examples output by Reality are exchangeable. This is a strong assumption, but it is standard in machine learning (where the even stronger assumption of randomness is usually made). We start this chapter (in Sect. 8.1) by discussing how to test this assumption in the online mode: at each point in time we would like to have a valid measure of the amount of evidence found against the hypothesis of exchangeability. Conformal prediction is a valuable tool for designing such online testing methods, and can also be adapted for detecting different kinds of deviations from exchangeability (Sect. 8.2). Such methods of conformal testing can be applied in multistage testing, when the task is to raise an alarm soon after the assumption of exchangeability becomes violated (Sect. 8.3), and in deciding when a machine learning algorithm depending on the exchangeability assumption should be retrained (Sect. 8.4).

Keywords: Exchangeability; Statistical hypothesis testing; Anticausal classification; CUSUM; Shiryaev–Roberts procedure; Retraining (search for similar items in EconPapers)
Date: 2022
References: Add references at CitEc
Citations:

There are no downloads for this item, see the EconPapers FAQ for hints about obtaining it.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:sprchp:978-3-031-06649-8_8

Ordering information: This item can be ordered from
http://www.springer.com/9783031066498

DOI: 10.1007/978-3-031-06649-8_8

Access Statistics for this chapter

More chapters in Springer Books from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().