An Improved Fst Estimator

Guanjie Chen, Ao Yuan, Daniel Shriner, Fasil Tekola-Ayele, Jie Zhou, Amy R Bentley, Yanxun Zhou, Chuntao Wang, Melanie J Newport, Adebowale Adeyemo and Charles N Rotimi

PLOS ONE, 2015, vol. 10, issue 8, 1-15

Abstract: The fixation index Fst plays a central role in ecological and evolutionary genetic studies. The estimators of Wright (F ^ s t 1), Weir and Cockerham (F ^ s t 2), and Hudson et al. (F ^ s t 3) are widely used to measure genetic differences among different populations, but all have limitations. We propose a minimum variance estimator F ^ s t m using F ^ s t 1 and F ^ s t 2. We tested F ^ s t m in simulations and applied it to 120 unrelated East African individuals from Ethiopia and 11 subpopulations in HapMap 3 with 464,642 SNPs. Our simulation study showed that F ^ s t m has smaller bias than F ^ s t 2 for small sample sizes and smaller bias than F ^ s t 1 for large sample sizes. Also, F ^ s t m has smaller variance than F ^ s t 2 for small Fst values and smaller variance than F ^ s t 1 for large Fst values. We demonstrated that approximately 30 subpopulations and 30 individuals per subpopulation are required in order to accurately estimate Fst.

Date: 2015
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0135368 (text/html)
https://journals.plos.org/plosone/article/file?id= ... 35368&type=printable (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:plo:pone00:0135368

DOI: 10.1371/journal.pone.0135368

Access Statistics for this article

More articles in PLOS ONE from Public Library of Science
Bibliographic data for series maintained by plosone ().