Автор: Аль-Егоби Хуссейн Али Абдулла

Рубрика: Информатика

Опубликовано в Молодой учёный №6 (53) июнь 2013 г.

Статья просмотрена: 33 раза

Библиографическое описание:

Аль-Егоби Х. А. Probability Generating Functions For Markov Matrix // Молодой ученый. — 2013. — №6. — С. 187-189.

A general matrix representation is given for the multivariate transition probability generating functions of a Markov Process with a finite number of states. It is indicated how numerous derived probability distributions can be obtained by simple substitutions. FinallyanapplicationismadetothePoisson

Общее представление матрицы дается для многомерного вероятность перехода производящих функций марковского процесса с конечным числом состояний. Она обозначена как многочисленные полученных распределений вероятности может быть получено простой замены. НаконецподанозаявлениекраспределениюПуассона.


Let N(t) = (N1(t),…, Nm(t)) denote a Markov Renewal Process with a finite number m of states and with a matrix of transition probability distributions Q =Qij [2,3]. The Qij(t) are non-decreasing left-continuous functions satisfying

Qij(0)=0 for i, j =1,…,m

 for i=1,…,m                                                                                      (1)

The random variable Ni (t) is equal to the number of visits to state i during the time interval [0,t]. The stochastic process Zt is referred to as the (semi- Markov process) associated with the Markov renewal process. Zt= i when state i is being visited at time t. We assume that

P{Z0= i} = Pi0 with                                                                                   (2)

Let K = (k1..., km) denote an m-tuple of non-negative integers and T(k) as follows:

T(k) = inf { t: N(1)(t) = k1, …, Nm(t) = km }                                                                   (3)

i.e,T(k) is the random time at which the Markov renewal process enters the stat k=(k1,…..,k2).

Let Z(k) = ZT(k) and T'(k) denote the time instant at which the (semi- Markov process) leaves the state Z(k). We define the following transition probabilities for the Markov renewal process.

Cj (k, t) = P {T(k)  t and Z(k) = j }                                                                             (4)


Cj (k, t) = P {T(k)  t  T'(k) and Zt = j }                                                                    (5)

The probabilities defined in (4) and (5) satisfy the following relations.



Hj(t) =

and I(t) is the distribution degenerate at zero The m-tuple ei has all components but the i th equal to zero and the ith equal to one.We introduce the following notations for the Laplace transforms.


The formulae (6) become


We now introduce the multivariate probability generating functions.

Gj*(z,s)=Gj*(z1,…,zm,s) =                                 (9)


Kj*(z,s)=Kj*(z1,…,zm,s) =

And the column vectors

G*=(G1*(z,s), …, Gm*(z,s)), K*=(K1*(z,s), ….., Km*(z,s))

The column-vectors G* and К* now have the following matrix representation

K*(z,s) = (I — (H*))G*(z,s)

K*(z,s) =(I — (H*))(K-(z)Q*')-1 (z)p0                                                                      (10)

In which (H*) = diag (H1*(s),…., H1*(s))

(z) = diag (z1,…,zm) and \zi\ > 1 i=1,…,m

Q* ={Qij*} and p0 is the column vector (p0,…,pm0)

The first equality is equivalent to the third in (8). Moreover it follows from the other equalities in (8) that

Gj*(z,s)= pj0zj +zj

which implies the second equality in (10) if the inverse of I — (z) Q*' exists. This is easily seen to be so in view of

\zjQrj*(s)\ Qrj(∞)     (11)

The numbers Qjr(∞) form, an m x m stochastic matrix which therefore has spectral radius equal to one.

2-Discrete time finite Markov chains:

If Cj(k) denotes the probability that in k1 +.,. + km-1 transitions a discrete Markov chain reaches state j and has visited state r exactly kr times (r= 1,...,m) then

Cj*(k,s) = Cj(k)e-s(k1+……+km-1), Q*=e-sp, Gj*(z,s)=esGj(ze-s)

generating function of the Cj(k). After setting zie-s=ξ. we obtain

G(ξ)=(I — (z)p')-1 (z)p0                                                                                             (12)

3-Generating Functions

Generating functions for many related probabilities can be derived from K(z,s) by an appropriate choice of the variables zi

If we set some of the zi equal to a same variable u we find the transition probabilities of the semi- Markov process which specify only the number of visits to certain but not all states. If we set certain variables z^ in formula (10) equal to zero, we find generating functions for taboo-probabilities, i.e. transition probabilities of events in which one specifies that certain states should not be visited. Finally if we perform the substitutes

Zr=                                                                                                      (13)

for all or some of the variables z in which αi is equal to zero, plus or minus one we obtain generating functions for events defined with respect to algebraic sums of the random variables Ni(t).

3- An application Poisson

We consider a single server Poisson queue with input rate λ and service rate μ.We wish to evaluate the probabilities πijn(t) that in the time-interval [0,t] there have been n transitions in the queue, the queue length at time t is j and neither of the queue lengths zero and b have been attained, given that the initial queue length was i. 0< i, j < b.

Let us consider the b + 1 state semi- Markov process in which.


If we substitute Q into formula (10) and set p° = ei and z0=zb=0,z1…zb-1=u we obtain

Kj*(0,u, ……,u,0,s)= u                                                             (15)

For j=1,…….,b-1 set



pij*(u,s)= (1|u)k*(δ,u,…,u,0,s)=(1-Hj*(s))(I-Q* ')-1)ij

where diag (0,u,…,u,0), After inversion of the jacobi-matrdx I-Q* we find



ξ1,2=(1|2){1 [1-(4λμu2)|(s+λ+μ)]1|2}

if we set [(2)|(s+λ+μ)]=1|,we find for ji

pij*(u,s)=s λ(j-i-1)|2μ(i-j-1)|2 u-1 (

u-1 ( is a rational function of u with b-1 distinct poles at


Partial fraction expansion yields:

(1|s)pij*(u,s)=(λ|μ)j-i|2           (17)

πijn(t) =2nλ(n+i+j)|2μ(n+i-j)|2(1|n!)tne-(λ+μ)t  (18)


1.         Kazuoki Azuma, Weighted sums of certain dependent random variables. Tohoku Mathematical Journal,1967.

2.         Wassily Hoeffding. Probability inequalities for sums bounded random variables. Journal of the American Statistical Association,1963

3.         Andrei N,Kolmogorov.Grundbegriffe der W ahrscheinlichkeitsrechnung. Springer, Berlin 1933.English Translation: Foundation of the theory of Probability, Chelsea, New York,1950.

4.         Glenn Shafer and Vladimir Vovk.Probability and Finance, Wiley, New York,2001

5.         McClenahan, C.L. “Ratemaking.” In Foundations of Casualty Actuarial Science, Fourth Edition. Arlington, VA: Casualty Actuarial Society, 2000

Основные термины (генерируются автоматически): markov renewal process, semi- Markov process, transition probabilities, time finite Markov, state semi- Markov, discrete Markov chain, multivariate transition probability, transition probability distributions, numerous derived probability, following transition probabilities, stochastic process Zt, general matrix representation, server Poisson queue, random variables Ni, random variable Ni, finite number, initial queue length, following matrix representation, American Statistical Association,1963, z,s.


Социальные комментарии Cackle
Задать вопрос