Exact limits of inference in coalescent models

James E. Johndrow and Julia A. Palacios

Theoretical Population Biology, 2019, vol. 125, issue C, 75-93

Abstract: Recovery of population size history from molecular sequence data is an important problem in population genetics. Inference commonly relies on a coalescent model linking the population size history to genealogies. The high computational cost of estimating parameters from these models usually compels researchers to select a subset of the available data or to rely on insufficient summary statistics for statistical inference. We consider the problem of recovering the true population size history from two possible alternatives on the basis of coalescent time data previously considered by Kim et al. (2015). We improve upon previous results by giving exact expressions for the probability of correctly distinguishing between the two hypotheses as a function of the separation between the alternative size histories, the number of individuals, loci, and the sampling times. In more complicated settings we estimate the exact probability of correct recovery by Monte Carlo simulation. Our results give considerably more pessimistic inferential limits than those previously reported. We also extended our analyses to pairwise SMC and SMC’ models of recombination. This work is relevant for optimal design when the inference goal is to test scientific hypotheses about population size trajectories in coalescent models with and without recombination.

Keywords: Bayes error rates; Coalescent; Effective population size; Sequentially Markov coalescent (search for similar items in EconPapers)
Date: 2019
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0040580918300248
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:thpobi:v:125:y:2019:i:c:p:75-93

DOI: 10.1016/j.tpb.2018.11.004

Access Statistics for this article

Theoretical Population Biology is currently edited by Jeremy Van Cleve

More articles in Theoretical Population Biology from Elsevier
Bibliographic data for series maintained by Catherine Liu ().