Optimal Stopping of Seasonal Observations and Projection of a Markov Chain

Isaac M. Sonin ()
Additional contact information
Isaac M. Sonin: University of North Carolina at Charlotte, Department of Mathematics and Statistics

A chapter in Inspired by Finance, 2014, pp 535-543 from Springer

Abstract: Abstract We consider the recently solved problem of Optimal Stopping of Seasonal Observations and its more general version. Informally, there is a finite number of dice, each for a state of “underlying” finite MC. If this MC is in a state k, then k-th die is tossed. A Decision Maker (DM) observes both MC and the value of a die, and at each moment of discrete time can either continue observations or to stop and obtain a discounted reward. The goal of a DM is to maximize the total expected discounted reward. This problem belongs to an important class of stochastic optimization problems—the problem of optimal stopping of Markov chains (MCs). The solution was obtained via an algorithm which is based on the general, so called, State Elimination algorithm developed by the author earlier. An important role in the solution is played by the relationship between the fundamental matrix of a transient MC in the “large” state space and the fundamental matrix for the modified underlying transient MC. In this paper such relationship is presented in a transparent way using the general concept of a projection of a Markov model. The general relationship between two fundamental matrices is obtained and used to clarify the solution of the optimal stopping problem.

Keywords: Markov chain; Optimal stopping; Elimination algorithm; Seasonal observations; 60G42; 60J10; 82B35 (search for similar items in EconPapers)
Date: 2014
References: Add references at CitEc
Citations:

There are no downloads for this item, see the EconPapers FAQ for hints about obtaining it.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:sprchp:978-3-319-02069-3_25

Ordering information: This item can be ordered from
http://www.springer.com/9783319020693

DOI: 10.1007/978-3-319-02069-3_25

Access Statistics for this chapter

More chapters in Springer Books from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().