Bayesian A/B Inference

John Geweke

A chapter in Topics in Identification, Limited Dependent Variables, Partial Observability, Experimentation, and Flexible Modeling: Part B, 2019, vol. 40B, pp 111-140 from Emerald Group Publishing Limited

Abstract: Bayesian A/B inference (BABI) is a method that combines subjective prior information with data from A/B experiments to provide inference for lift – the difference in a measure of response in control and treatment, expressed as its ratio to the measure of response in control. The procedure is embedded in stable code that can be executed in a few seconds for an experiment, regardless of sample size, and caters to the objectives and technical background of the owners of experiments. BABI provides more powerful tests of the hypothesis of the impact of treatment on lift, and sharper conclusions about the value of lift, than do legacy conventional methods. In application to 21 large online experiments, the credible interval is 60% to 65% shorter than the conventional confidence interval in the median case, and by close to 100% in a significant proportion of cases; in rare cases, BABI credible intervals are longer than conventional confidence intervals and then by no more than about 10%.

Date: 2019
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.emerald.com/insight/content/doi/10.110 ... d&utm_campaign=repec (text/html)
https://www.emerald.com/insight/content/doi/10.110 ... 1-90532019000040B007
https://www.emerald.com/insight/content/doi/10.110 ... d&utm_campaign=repec (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eme:aecozz:s0731-90532019000040b007

DOI: 10.1108/S0731-90532019000040B007

Access Statistics for this chapter

More chapters in Advances in Econometrics from Emerald Group Publishing Limited
Bibliographic data for series maintained by Emerald Support ().