Convex Optimization of Markov Decision Processes Based on Z Transform: A Theoretical Framework for Two-Space Decomposition and Linear Programming Reconstruction

Shiqing Qiu, Haoyu Wang, Yuxin Zhang, Zong Ke () and Zichao Li
Additional contact information
Shiqing Qiu: School of Mathematical Sciences, Chengdu University of Technology, Chengdu 610059, China
Haoyu Wang: School of Mathematical Sciences, Chengdu University of Technology, Chengdu 610059, China
Yuxin Zhang: School of Business, Henan University, Zhengzhou 450001, China
Zong Ke: Department of Statistics and Data Science, Faculty of Science, National University of Singapore, 21 Lower Kent Ridge Road, Singapore 119077, Singapore
Zichao Li: Department of Management Science and Engineering, University of Waterloo, Waterloo, ON N2L 3G1, Canada

Mathematics, 2025, vol. 13, issue 11, 1-27

Abstract: This study establishes a novel mathematical framework for stochastic maintenance optimization in production systems by integrating Markov decision processes (MDPs) with convex programming theory. We develop a Z-transformation-based dual-space decomposition method to reconstruct MDPs into a solvable linear programming form, resolving the inherent instability of traditional models caused by uncertain initial conditions and non-stationary state transitions. The proposed approach introduces three mathematical innovations: (i) a spectral clustering mechanism that reduces state-space dimensionality while preserving Markovian properties, (ii) a Lagrangian dual formulation with adaptive penalty functions to handle operational constraints, and (iii) a warm start algorithm accelerating convergence in high-dimensional convex optimization. Theoretical analysis proves that the derived policy achieves stability in probabilistic transitions through martingale convergence arguments, demonstrating structural invariance to initial distributions. Experimental validations on production processes reveal that our model reduces long-term maintenance costs by 36.17% compared to Monte Carlo simulations (1500 vs. 2350 average cost) and improves computational efficiency by 14.29% over Q-learning methods. Sensitivity analyses confirm robustness across Weibull-distributed failure regimes (shape parameter β ∈ [1.2, 4.8]) and varying resource constraints.

Keywords: Markov decision process; linear programming; Z-transform; convex optimization; production system optimization (search for similar items in EconPapers)
JEL-codes: C (search for similar items in EconPapers)
Date: 2025
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.mdpi.com/2227-7390/13/11/1765/pdf (application/pdf)
https://www.mdpi.com/2227-7390/13/11/1765/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jmathe:v:13:y:2025:i:11:p:1765-:d:1664719

Access Statistics for this article

Mathematics is currently edited by Ms. Emma He

More articles in Mathematics from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().