A comparison of three algorithms in the filtering of a Markov-modulated non-homogeneous Poisson process

Yuying Li and Rogemar Mamon

International Journal of Systems Science, 2024, vol. 55, issue 4, 741-770

Abstract: A Markov-modulated non-homogeneous Poisson process (MMNPP), whose intensity process is designed to capture both the cyclical and nonrecurring trends, is considered for modelling the total count of cyber incidents. Extending the Expectation-Maximisation (EM) algorithm for the current MMPP literature, we derive the filters and smoothers to support the MMNPP online parameter estimation. A scaling transformation is introduced to address the numerical issue for large data sizes whilst maintaining accuracy. The filter- and smoother-based EM algorithms are then benchmarked to the maximum likelihood-based EM algorithm at the theoretical level. The differences emerge in the E-step of the EM procedure. Both the filtering and smoothing schemes, in conjunction with the change-of-measure technique, avoid the computing complication caused by the hidden regimes. In contrast to the usual EM algorithm, the said two algorithms could be implemented given only the incident counts data without the specific times of jumps. Within the data compiled by the U.S. Department of Health and Human Services, the filter-based algorithm performs better than the algorithm involving smoothers. The benchmarked algorithm may do well in calibration under the presence of extreme incident counts with an extremely low frequency; however, overfitting may occur. For most practical applications involving 2 or 3 regimes, both algorithms are superior when it comes to efficiency, real-time update, and low computational cost. The benchmarked algorithm is better when there are more regimes under relatively closer intensities. Overall, the filter-based algorithm gives better estimation, especially if there is a low-frequency regime and the flexible binning of the data set is an important consideration.

Date: 2024
References: Add references at CitEc
Citations:

Downloads: (external link)
http://hdl.handle.net/10.1080/00207721.2023.2294747 (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:taf:tsysxx:v:55:y:2024:i:4:p:741-770

Ordering information: This journal article can be ordered from
http://www.tandfonline.com/pricing/journal/TSYS20

DOI: 10.1080/00207721.2023.2294747

Access Statistics for this article

International Journal of Systems Science is currently edited by Visakan Kadirkamanathan

More articles in International Journal of Systems Science from Taylor & Francis Journals
Bibliographic data for series maintained by Chris Longhurst ().