Reference for citation: Zemskov А. V. Analytical approach to selective search for state probability functions in Markov chains. Journal of Instrument Engineering. 2023. Vol. 66, N 12. P. 1035—1049 (in Russian). DOI: 10.17586/0021-3454-2023-66-12-1035-1049.

Abstract. An analytical approach to studying countable homogeneous Markov chains is proposed. An algorithm for selective search of input-output operators in z-form is presented. Simple analytical procedures for obtaining probability functions for real and complex conjugate eigenvalues of the transition probability matrix are described. Estimates are given for the boundaries of the onset of a steady state in clock time. Additional results are presented for the steady state using an algorithm for expanding the characteristic determinants. The main calculations are illustrated by assessing the characteristics of Markov chains taking into account the influence of transient dynamics when changing the probabilities of states over the operating interval in clock time. Aspects of calculation reliability when estimating probability values using the proposed approach are also considered.

Keywords: Markov chain, input-output operator in z-form, eigenvalues, eigenvectors, state probability function

References:

1. Tikhonov V.I., Mironov M.A. Markovskiye protsessy (Markov Processes), Moscow, 1977, 485 р. (in Russ.)
2. Ventzel E.S. Issledovaniye operatsiy: zadachi, printsipy, metodologiya (Operations Research: Objectives, Principles, Methodology), Moscow, 2010, 190 р. (in Russ.)
3. Nummelin E. General Irreducible Markov Chains and Non-Negative Operators, Cambridge etc., 1984, 156 p.
4. Furman Ya.A., Yuryev A.N., Yanshin V.V. Tsifrovyye metody obrabotki i raspoznavaniya binarnykh izobrazheniy (Digital Methods of Processing and Recognition of Binary Images), Krasnoyarsk, 1992, 245 р. (in Russ.)
5. Harrison P.G. J. Appl. Prob., 1981, no. 2(18), pp. 482–490.
6. Dudin A.N., Karolik A.V. Performance Evaluat., 2001, no. 1(45), pp. 19–32.
7. Dharmaraja S., Rakesh Kumar, OPSEARCH, 2015, no. 4(52), pp. 810–826.
8. Kumar B. Krishna, Madheshwari S. Pavai, Venkatakrishanan K.S. Int. J. Inform. Management Sci., 2017, no. 1(18), pp. 63–80.
9. Miller A.B., Miller B.M., and Stepanyan K.V. Automation and Remote Control, 2020, no. 3, рр. 469–482, DOI: 10.31857/S0005231020030071.
10. Vytovtov K.A. and Barabanova E.A. Automation and Remote Control, 2021, no. 12, рр. 2112–2124, DOI: 10.31857/S0005231021120060.
11. Kuo B.C. Digital Contrоl Systems, NY, Chicago, San Francisco, Holt, Rinehart and Winston, Inc., 1980.
12. Jury E.J. Sampled-Data Control Systems, NY, Wiley, London, Chapmen and Hall, 1958.
13. Faddeev D.K., Faddeeva V.N. Vychislitel'nyye metody lineynoy algebry (Computational Methods of Linear Algebra), Moscow, 1960, 654 р. (in Russ.)
14. Babakov N.A., Voronov A.A., Voronova A.A. et al. Teoriya avtomaticheskogo upravleniya. Ch. I. Teoriya lineynykh sistem avtomaticheskogo upravleniya (Theory of Automatic Control. Part I. Theory of Linear Systems of Automatic Control), Moscow, 1986, 367 р. (in Russ.)
15. Zemskov A.V. Journal of Instrument Engineering, 1989, no. 11(32), pp. 20–22. (in Russ.)
16. Zemskov V.A., Zemskov A.V. Analiticheskaya teoriya avtomaticheskogo upravleniya i yeye prilozheniya (Analytical Theory of Automatic Control and Its Applications), Proceedings of the International Scientific Conference, Saratov, 2000, рр. 17–20. (in Russ.)
17. Golub G.H. & Uhlig F. IMA Journal of Numerical Analysis, 2009, no. 3(29), pp. 467–485.
18. Zemskov A.V. Computational Mathematical and Mathematical Physics, 1998, no. 3(38), pp. 351–361.
19. Bellman R.E. Adaptive Control Processes, Princeton University Press, Princeton, NJ, 1961, 276 p.
20. Langville A.N., Meyer C.D. Google's PageRank and beyond: the science of search engine rankings, Princeton University Press, 2006, 224 р.