Fitting a reversible Markov chain by maximum likelihood: Converting an awkwardly constrained optimization problem to an unconstrained one

Iain L. MacDonald and Etienne A.D. Pienaar

Physica A: Statistical Mechanics and its Applications, 2021, vol. 561, issue C

Abstract: We consider here the problem of fitting, by maximum likelihood, a discrete-time, finite-state–space Markov chain that is required to be reversible in time. The technique we use is to impose the detailed balance and other constraints by transforming the transition probabilities to a set of unconstrained ‘working parameters’, after which any general-purpose routine for unconstrained numerical optimization can be used to maximize the likelihood. The main advantages of this procedure are its simplicity and its use of standard, well-tested optimizers; very little computational expertise is needed, and all computations can easily be carried out in R on a standard machine. We provide several examples of applications, to simulated and other data, and give an example of the computation of standard errors and confidence intervals for parameters. We discuss also the variation of the problem in which it is required that the transition probabilities satisfy detailed balance with respect to a pre-specified stationary distribution, demonstrate that this case can be handled by a modification of our methods, but suggest two altogether different routes that may be preferable. A feature of our methods is the availability of excellent starting-values for the optimizations.

Keywords: Markov chain; Reversibility; Detailed balance; Likelihood; Numerical optimization; Parametric bootstrap (search for similar items in EconPapers)
Date: 2021
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0378437120306178
Full text for ScienceDirect subscribers only. Journal offers the option of making the article available online on Science direct for a fee of $3,000

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:phsmap:v:561:y:2021:i:c:s0378437120306178

DOI: 10.1016/j.physa.2020.125182

Access Statistics for this article

Physica A: Statistical Mechanics and its Applications is currently edited by K. A. Dawson, J. O. Indekeu, H.E. Stanley and C. Tsallis

More articles in Physica A: Statistical Mechanics and its Applications from Elsevier
Bibliographic data for series maintained by Catherine Liu ().