Sequential Confidence Intervals for Comparing Two Proportions with Applications in A/B Testing

Jun Hu (), Lijia Zheng and Ibtihal Alanazi
Additional contact information
Jun Hu: Department of Mathematics and Statistics, Oakland University, Rochester, MI 48309-4486, USA
Lijia Zheng: Department of Health Policy, Stanford University School of Medicine, Stanford, CA 94305, USA
Ibtihal Alanazi: Department of Mathematics and Statistics, Oakland University, Rochester, MI 48309-4486, USA

Mathematics, 2025, vol. 13, issue 1, 1-14

Abstract: This article addresses the use of fixed-width confidence intervals (FWCIs) for comparing two independent Bernoulli populations in A/B testing scenarios. Two sequential estimation procedures are proposed: one for estimating the difference in log probabilities of success and the other for log odds ratios. Both methods showcase great efficiency, as established via theoretical analysis and Monte Carlo simulations. The practical utility of these methods is demonstrated through two real-world applications: analyzing retention rates in mobile game Cookie Cats and evaluating the effectiveness of online advertising.

Keywords: A/B testing; Bernoulli proportions; fixed-width confidence interval; mobile gaming; online advertising; sequential estimation (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2025
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/13/1/161/pdf (application/pdf)
https://www.mdpi.com/2227-7390/13/1/161/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:13:y:2025:i:1:p:161-:d:1560504

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().