 On the performance of the Neyman Allocation with small pilotsYong Cai and Ahnaf Rafi Journal of Econometrics, 2024, vol. 242, issue 1 Abstract: The Neyman Allocation is used in many papers on experimental design, which typically assume that researchers have access to large pilot studies. This may be unrealistic. To understand the properties of the Neyman Allocation with small pilots, we study its behavior in an asymptotic framework that takes pilot size to be fixed even as the size of the main wave tends to infinity. Our analysis shows that the Neyman Allocation can lead to estimates of the ATE with higher asymptotic variance than with (non-adaptive) balanced randomization. In particular, this happens when the outcome variable is relatively homoskedastic with respect to treatment status or when it exhibits high kurtosis. We provide a series of empirical examples showing that such situations can arise in practice. Our results suggest that researchers with small pilots should not use the Neyman Allocation if they believe that outcomes are homoskedastic or heavy-tailed. Finally, we examine some potential methods for improving the finite sample performance of the FNA via simulations. Keywords: Experiment design; Treatment effect estimation; Alternative asymptotics (search for similar items in EconPapers) Downloads: (external link) Related works: Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text Persistent link: https://EconPapers.repec.org/RePEc:eee:econom:v:242:y:2024:i:1:s0304407624001398 DOI: 10.1016/j.jeconom.2024.105793 Access Statistics for this article Journal of Econometrics is currently edited by T. Amemiya, A. R. Gallant, J. F. Geweke, C. Hsiao and P. M. Robinson More articles in Journal of Econometrics from Elsevier |
Is your work missing from RePEc? Here is how to contribute.
Questions or problems? Check the EconPapers FAQ or send mail to .
EconPapers is hosted by the
School of Business at Örebro University.