Stochastic Solutions Manual

Stochastic Stochastic Processes: An Introduction Introduction Solutions Manual

Peter W Jones and Peter Smith School of Computing and Mathematics, Keele University, UK May 2009

Preface The website includes answers and solutions of all the end-of-chapter problems in the textbook Stochastic Stochastic Processes: Processes: An Introduction  Introduction . We hope that they will prove prove of help to lecturer lecturerss and students. Both the original problems as numbered in the text are also included so that the material can be used as an additional source of worked problems. There are obviously references to results and examples from the textbook, and the manual should be viewed as a supplement to the book. To help identify the sections and chapters, the full contents of  Stochastic Processes follow this preface. Every effort has been made to eliminate misprints or errors (or worse), and the authors, who were responsible for the LaTeX code, apologise in advance for any which occur. Peter Jones Peter Smith

Keele, May 2009

1

Contents of Stochastic Processes Chapter Chapter 1: Some Background Background in Probabil Probability ity

1.1 Introduction 1.2 Probability 1.3 Conditional probability and independence 1.4 Discrete random variables 1.5 Continuous random variables 1.6 Mean and variance 1.7 Some standard discrete probability distributions 1.8 Some standard continuous probability distributions 1.9 Generating functions 1.10 Conditional expectation Problems Chapter Chapter 2: Some Gambling Gambling Problems Problems

2.1 Gambler’s ruin 2.2 Probability of ruin 2.3 Some numerical simulations 2.4 Expected duration of the game 2.5 Some variations of gambler’s ruin 2.5.1 The infinitely infinitely rich rich opponent opponent 2.5.2 The generous gambler 2.5.3 Changing the stakes Problems Chapter Chapter 3: Random Random Walks

3.1 Introduction 3.2 Unrestricted random walks 3.3 Probability distribution after n steps 3.4 First returns of the symmetric random walk 3.5 Other random walks Problems Chapter Chapter 4: Markov Markov Chains

4.1 States and transitions 4.2 Transition probabilities 4.3 General two-state Markov chain 4.4 Powers of the transition matrix for the m-state chain 4.5 Gambler’s ruin as a Markov chain 4.6 Classification of states 4.7 Classification of chains Problems Chapter Chapter 5: Poisson Poisson Processes Processes

5.1 Introduction 5.2 The Poisson process 5.3 Partition theorem approach 5.4 Iterative method 5.5 The generating generating function 5.6 Variance for the Poisson process

2

5.7 Arrival times 5.8 Summary of the Poisson process Problems Chapter 6: Birth and Death Processes

6.1 Introduction 6.2 The birth process 6.3 Birth process: generating function equation 6.4 The death process 6.5 The combined birth and death process 6.6 General population processes Problems Chapter 7: Queues

7.1 Introduction 7.2 The single server queue 7.3 The stationary process 7.4 Queues with multiple servers 7.5 Queues with fixed service times 7.6 Classification of queues 7.7 A general approach to the M (λ)/G/1 queue Problems Chapter 8: Reliability and Renewal

8.1 Introduction 8.2 The reliability function 8.3 The exponential distribution and reliability 8.4 Mean time to failure 8.5 Reliability of series and parallel systems 8.6 Renewal processes 8.7 Expected number of renewals Problems Chapter 9: Branching and Other Random Processes

9.1 Introduction 9.2 Generational growth 9.3 Mean and variance 9.4 Probability of extinction 9.5 Branching processes and martingales 9.6 Stopping rules 9.7 The simple epidemic 9.8 An iterative scheme for the simple epidemic Problems Chapter 10: Computer Simulations and Projects

3

Chapters of the Solutions Manual Chapter 1: Some Background in Probability Chapter 2: Some Gambling Problems Chapter 3: Random Walks

30

Chapter 4: Markov Chains

44

Chapter 5: Poisson Processes

16

65

Chapter 6: Birth and Death Processes Chapter 7: Queues

5

71

93

Chapter 8: Reliability and Renewal

108

Chapter 9: Branching and Other Random Processes

4

116

Chapter 1

Some background in probability 1.1. The Venn diagram of three events is shown in Figure 1.5(in the text). Indicate on the diagram  the following events: (a) A B; (b) A (B C ); (c) A (B C ); (d) (A C )c ; (e) (A B) C c .

∪

∪ ∪

∩ ∪

∩

∩ ∪

S  A

S  A

B

B 

C 

C 

(a)

(b) S 

S  A

B 

B 

A

C 

C 

(c)

(d) S  B 

A

C 

(e)

Figure 1.1: The events are shaded in Figure 1.1. 1.2. In a random experiment, A, B, C  are three events. In set notation write down expressions  for the events: (a) only  A occurs; (b) all three events A, B, C  occur; (c) A and  B occur but  C  does not; (d) at least one of the events A, B, C  occurs; (e) exactly one of the events A, B, C  occurs; (f) not more than two of the events occur.

5

(a) A (e) A

∩ (B ∪ C )cc ; (b) A ∩ (B ∩ C ) = A ∩ B ∩ C ; (c) (A ∩ B) ∩ C c ; (d) A ∪ B ∪ C ; ∩ (B ∪ C ) represents an event in A but not in either B nor C : therefore the answer is (A ∩ (B ∪ C )c ) ∪ (B ∩ (A ∪ C ))c ∪ (C  ∩ (A ∪ B)c ).

1.3. For two events A and  B, P(A) = 0.4, P(B) = 0.5 and  P(A (a) P(A B); (b) P(A B c ); (c) P(Ac B c ).

∩ B) = 0.3. Calculate ∪ (a) From (1.1) P(A ∪ B) = P(A) + P(B) − P(A ∩ B), it follows that P(A ∪ B) = 0.4 + 0.5 − 0.3 = 0.6. (b) Since A = (A ∩ B c ) ∪ (A ∩ B) and A ∩ B c , and A ∩ B are mutually exclusive, then, P(A) = P[(A ∩ B c ) ∪ (A ∩ B)] = P(A ∩ B c ) + P(A ∩ B), ∪

∩

so that P(A

(c) Since Ac

∩ Bc ) = P(A) − P(A ∩ B) = 0.4 − 0.3 = 0.1.

∪ Bc = (A ∩ B)c , then P(Ac ∪ B c ) = P[(A ∩ B)c ] = 1 − P(A ∩ B) = 1 − 0.3 = 0.7.

1.4. Two distinguishable fair dice a and  b are rolled. What are the elements of the sample space?  What is the probability that the sum of the face values of the two dice is 9? What is the probability  that at least one 5 or at least one 3 appears?  The elements of the sample space are listed in Example 1.1. The event A1 , that the sum is 9, is given by A1 = (3, 6), (4, 5), (5, 4), (6, 3) .

{

}

4 Hence P = 36 = 19 . Let A2 be the event that at least one 5 or at least one 3 appears. Then by counting the elements in the sample space in Example 11, P(A2 ) = 20 = 59 . 36

1.5. Two distinguishable fair dice a and  b are rolled. What is the probability that the sum of the  faces is not more than 6?  Let the random variable X  be the sum of the faces. By counting events in the sample space in 5 Example 1.1, P(X ) = 15 = 12 . 36 1.6. A probability function   pn , (n = 0, 1, 2, . . .) has a probability generating function 

{ }

G(s) =

∞



1 2

 pn sn = 14 (1 + s)(3 + s) .

n=0

Find the probability function   pn and its mean.

{ }

Note that G(1) = 1. Using the binomial theorem G(s)

= =

1 (1 + 4

√3 4

1 2

s)(3 + s) =

∞

  

n=0

1 2

n

s 3

n

6

√

3 (1 + 4

√

s)(1 + 13 s)

∞

1 2

 −  

3 3 + 4 n=1 n

1 2

1

s 3

n

.

The probabilities can now be read off from the coefficients of the series:  p0 =

√3 4

,

pn =

√3

    1 2

3n 4

n

+3

1 2

n

−1

,

(n = 1, 2, . . .).

The expected value is given by 1 d = G (1) = (1 + s)(3 + s)

µ



=

4 ds





1 2

s=1

1 1 (3 + s) + (1 + s)(3 + s)− 4 8 1 2

1 2

5 8

=



s=1

1.7. Find the probability generating function  G(s) of the Poisson distribution (see Section 1.7) with  parameter  α given by  e−α αn  pn = , n = 0, 1, 2, . . . . n! Determine the mean and variance of   pn from the generating function.

{ }

Given pn = e−α αn /n!, the generating function is given by G(s) =

∞



n

 pn s =

n=0

∞ e−α αn sn



n=0

n!

= e−α

∞ (αs)n



n!

n=0

= eα(s−1) .

The mean and variance are given by µ

σ

2

= G (1) + G (1)

= G (1) =

d α(s−1) e ds

 

s=1

= α,

− [G(1)]2 = [α2eα(s−1) + αeα(s−1) − α2e2α(s−1)]s=1 = α,

as expected. 1.8. A panel contains n warning lights. The times to failure of the lights are the independent  random variables T 1 , T 2 , . . . , Tn  which have exponential distributions with parameters α1 , α2 , . . . , αn respectively. Let  T  be the random variable of the time to first failure, that is T  = min T 1 , T 2 , . . . , Tn  .

{

}

Show that T  has an exponential distribution with parameter  that the i-th panel light fails first is αi /( nj=1 αj ).





n j =1 αj .

Show also that the probability 

The probability that no warning light has failed by time t is P(T 

≥ t)

= P(T 1 = P(T 1 =

≥ t ∩ T 2 ≥ t ∩ · · · ∩ T n ≥ t) ≥ t)P(T 2 ≥ t) ··· P(T n ≥ t) − α t −α t ··· e−α t = e−(α +α +···+α )t. e e 1

2

n

7

1

2

n

Let T i represent the random variable that the ith component fails first. The probability that the ith component fails first is

  −  ≈     → −

P(T i = T )

=

P(T n > t)P(t < T i < t + δt)

δt n=i



P(T n > t)[e−αi t

=

e−αi (t+δt) ]

δt n=i



P(T n > t)αi δte −αi t

δt n=i



∞

n

αi exp

t

0

as δt

→ 0.

αi dt =

i=1

αi n i=1 αi

1.9. The geometric probability function with parameter  p is given by   p(x) = qx−1 p,

x= 1, 2, . . .

where q = 1  p (see Section 1.7). Find its probability generating function. Calculate the mean  and variance of the geometric distribution from its pgf .

−

The generating function is given by G(s) =

∞



∞ p p qs − x q  ps = (qs)x = x 1

q

x=1



q1

x=1

− qs

=

ps , 1 qs

−

using the formula for the sum of a geometric series. The mean is given by d  µ = G (1) =

   − −  

ps ds 1 qs

For the variance,

G (s) =

2

s=1

1

pqs + qs (1 qs)2

−



= s=1

1 .  p

d p 2 pq = . 2 ds (1 qs) (1 qs)3

is required. Hence σ

=

p

= G (1) + G (1)

−

−

− [G(1)]2 =  p2q2 +  p1 −  p12 = 1 p−2 p .

1.10. Two distinguishable fair dice a and  b are rolled. What are the probabilities that: (a) at least one 4 appears; (b) only one 4 appears; (c) the sum of the face values is 6; (d) the sum of the face values is 5 and one 3 is shown; (e) the sum of the face values is 5 or only one 3 is shown?  From the Table in Example 1.1: (a) If A1 is the event that at least one 4 appears, then P(A1 ) = 11 36 . 5 (b) If A2 is the event that only one 4 appears, then P(A2 ) = 10 = . 36 18 5 (c) If  A3 is the event that the sum of the faces is 6, then P(A3 ) = 36 . 8

2 1 (d) If A4 is the event that the face values is 5 and one 3 is shown, then P(A4 ) = 36 = 18 . 7 (e) If  A5 is the event that the sum of the faces is 5 or only one 3 is shown, then P(A5 ) = 36 .

1.11. Two distiguishable fair dice a and  b are rolled. What is the expected sum of the face values?  What is the variance of the sum of the face values?  Let N  be the random variable representing the sum x + y, where x and y are face values of the two dice. Then 6 6 6 6 1 1 E(N ) = (x + y) = 6 x+6 y = 7. 36 x=1 y =1 36 x=1 y=1

 



and

6

V(N )

6

1 (x + y)2 E(N ) = 36 x=1 y=1

 −    − 2

= E(N  )

6

− 72

6

=

1 12 x2 + 2( x)2 36 x=1 x=1

=

1 [(12 36

49

× 91) + 2 × 212] − 49 = 356 = 5.833 . . . .

1.12. Three distinguishable fair dice a, b and  c are rolled. How many possible outcomes are there  for the faces shown? When the dice are rolled, what is the probability that just two dice show the same face values and the third one is different?  The sample space contains 63 = 216 elements of the form, (in the order a,b,c), S  = (i,j,k) ,

{

}

(i = 1, . . . , 6; j = 1, . . . , 6; k = 1, . . . , 6).

Let A be the required event. Suppose that a and b have the same face values, which can occur in 6 ways, and that c has a different face value which can occurs in 5 ways. Hence the total number of ways in which a and b are the same but c is different is 6 5 = 30 ways. The faces b and c, and c and a could also be the same so that the total number of ways for the possible outcome is 3 30 = 90 ways. Therefore the required probability is

×

×

P(A) =

90 5 = . 216 12

1.13. In a sample space S , the events B and  C  are mutually exclusive, but  A and  B, and  A and  C  are not. Show that  P(A

∪ (B ∪ C )) = P(A) + P(B) + P(C ) − P(A ∩ (B ∪ C )).

From a well-shuffled pack of 52 playing cards a single card is randomly drawn. Find the probability that it is a club or an ace or the king of hearts. From (1.1) (in the book) P(A

∪ (B ∪ C )) = P(A) + P(B ∪ C ) − P(A ∩ (B ∪ C ))

(i).

Since B and C  are mutually exclusive, P(B

∪ C ) = P(B) + P(C ). 9

(ii)

From (i) and (ii), it follows that P(A

∪ (B ∪ C )) = P(A) + P(B) + P(C ) − P(A ∩ (B ∪ C )).

Let A be the event that the card is a club, B the event that it is an ace, and C  the event that it is the king of hearts. We require P(A (B C )). Since B and C  are mutually exclusive, we can use the result above. The individual probabilities are

∪ ∪

13 1 = ; 52 4

P(A) =

and since A

P(B) =

4 1 = ; 52 13

P(C ) =

1 , 52

∩ (B ∪ C ) is the ace of clubs, P(A

Finally P(A

∩ (B ∪ C )) = 521 .

. ∪ (B ∪ C )) = 14 + 131 + 521 − 521 = 17 52

1.14. Show that  f (x) =

 

0

x<0 0 x x>a

1 2a 1 (x a)/a 2a e

≤ ≤a

− −

is a possible probability density function. Find the corresponding probability function. Check the density function as follows:

 

∞

1 2a

f (x)dx =

−∞

1 2

=

 

−

The probability function is given by, for 0

 

a

1 dx + 2a

∞

e−(x−a)/a dx

a 0 1 [e (x a)/a ]a 2

− −

∞ = 1.

≤ x ≤ a,

x

F (x) =

 

 

x

f (u)du =

0

−∞

1 x du = , 2a 2a

and, for x > a, by

 

x

F (x)

=

a

f (u)du =

0

= =

1 2

 

− 1−

0

1 du + 2a

1 [ae−(u−a)/a ]xa 2a 1 −(x−a)/a e . 2

 

a

0

1 −(u−a)/a e du 2a

1.15. A biased coin is tossed. The probability of a head is p. The coin is tossed until the first head  appears. Let the random variable N  be the total number of tosses including the first head. Find  P(N  = n), and its pgf  G(s). Find the expected value of the number of tosses. The probability that the total number of throws is n (including the head) until the first head appears is (n−1) times P(N  = n) = (1

−

 p)(1



 

− p) ··· (1 − p) p = (1 − p)n−1 p, 10

(n

≥ 1)

The probability generating function is given by G(s)

∞



=

p

n 1

(1 − p) −  psn =

1

n=1

p

∞



− p n=1[(1 − p)s]

n

s(1 − p) ps = , · 1 − p [1 − s(1 − p)] 1 − s(1 − p)

=

after summing the geometric series. For the mean, we require G (s) given by, G (s) = The mean is given by

µ

[1

−

p s(1

− p)]

+

sp(1  p) = [1 s(1  p)]2 [1

= G (1) = 1/p.

− − −

−

p s(1

− p)]2 .

1.16. The n random variables X 1 , X 2 , . . . , Xm  are independent and identically distributed each with  a gamma distribution with parameters n and  α. The random variable Y  is defined by  Y  = X 1 + X 2 +

··· + X m.

Using the moment generating function, find the mean and variance of  Y . The probability density function for the gamma distribution with parameters n and α is αn n−1 −αx f (x) = x e . Γ(n) It was shown in Section 1.9 that the moment generating function for Y  is given, in general, by M Y  (s) = [M X (s)]m , where X  has a gamma distribution with the same parameters. Hence M Y  (s)

Hence E(Y ) =

  −  nm

s nm α s α nm nm(nm + 1) 2 = 1+ s+ s + α 2α2 =

nm , α

α

= 1

−

V(Y ) = E(Y 2 )

···

. − [E(Y )]2 = nm α2

1.17. A probability generating function with parameter  0 < α < 1 is given by  G(s) =

1 α(1 1 + α(1

−

− s) . − s)

Find  pn = P(N  = n) by expanding the series in powers of  s. What is the mean of the probability   function   pn ? 

{ }

Applying the binomial theorem G(s)

= = =

1 α(1 1 + α(1

−

− s) = (1 − α)[1 + (α/(1 − α))s] − s) (1 + α)[1 − (α/(1 + α))s] ∞

 −      −     −  1 α 1+α

∞ α

αs 1+ 1 α

1 α 1 + α n=0 1 + α

n

n=

αs 1+α

∞

α s + 1 + α n=0 n

11

n

α 1+α

n

sn+1 .

The summation of the two series leads to G(s)

The mean is given by

n

s +

1+α , 1 α

pn =

−



d 1 − α(1 − s) G (1) = ds 1 + α(1

− s)

α 1+α

2αn , (1 + α)n+1

n

sn

sn

(n = 1, 2, . . .).

  s=1

α 1+α

n=1 n

∞

−

∞

n

α 1+α

1 α 2 + 1 + α 1 + α n=1

=

 p0 =

∞

−

=

Hence

     

1 α 1 + α n=0



2α = [1 + a(1 s)]2

−

= 2α s=1

1.18. Find the moment generating function of the random variables X  which has the uniform  distribution  1/(b a) a x b f (x) = 0 for all other values of  x



Deduce E(X n ).

−

≤ ≤

The moment generating function of the uniform distribution is

 

b

M X (s)

=

a

= Hence E(X ) =

1 1 bs exs dx = [e b a b as

−

 ∞

1

b

bn

− a n=1

1 (b + a), 2

−

− an n!

E(X n ) =



− eas]

sn−1

bn+1 an+1 . (n + 1)(b a)

−

−

1.19. A random variable X  has the normal distribution  N (µ, σ 2 ). Find its momemt generating   function. By definition

 

∞ 1 M X (s) = E(e ) = esx exp σ 2π −∞ ∞ 1 2σ 2 xs (x = exp 2σ 2 σ 2π −∞ Xs

  

√

Apply the substitution x = µ + σ (v M X (s)

√

−

σ s):

−

− −

(x

−

2σ 2 2

µ)



2

µ)



dx

dx

then

= exp(sµ + = exp(sµ +

 

1 2 2 s ) 2σ 1 2 2 s ) 2σ

(see the Appendix for the integral).

12

∞

−∞

√12π e−

1 2

v2

dv

+ 1 = exp(sµ +

1 2 2 s ) 2σ

Expansion of the exponential function in powers of  s gives 1 M X (s) = 1 + µs + (σ2 + µ2 )s2 + 2

··· .

So, for example, E(X 2 ) = µ2 + σ 2 . 1.20. Find the probability generating functions of the following distributions, in which  0 < p < 1: (a) Bernoulli distribution: pn = pn (1  p)n , (n = 0, 1); (b) geometric distribution: pn = p(1  p)n−1 , (n = 1, 2, . . .); (c) negative binomial distribution with parameter  r expressed in the form:

− −

 pn =





r+n 1 r  p (1 r 1

−

−

− p)n,

(n = 0, 1, 2, . . .)

where r is a positive integer. In each case find also the mean and variance of the distribution using  the probability generating function. (a) For the Bernoulli distribution G(s) = p0 + p1 s = (1

− p) + ps.

The mean is given by µ

= G (1) = p,

and the variance by

− [G(1)]2 = p − p2 = p(1 − p). (b) For the geometric distribution (with q = 1 − p) σ

2

= G (1) + G (1)

∞

G(s) =



n 1 n

 pq − s = ps

n=1

∞



ps 1 qs

(qs)n =

−

n=0

summing the geometric series. The mean and variance are given by

 µ = G (1) = σ

2

= G (1) + G (1)

  −  

− [G(1)]2 =

p

1

qs

G(s)

=

∞

n=0

=

s=1

2 pq (1 qs)3

−

(c) For the negative binomial distribution (with q = 1



=



1 ,  p +

s=1

1  p

−  p12 = 1 p−2 p .

− p)



r+n 1 r n n r(r + 1)  p q s = pr 1 + r(qs) + (qs)2 + r 1 2!

pr (1 qs)r

−

−

−

The derivatives of  G(s) are given by G (s) =

rqp r , (1 qs)r+1

r(r + 1)q2 pr  G (s) = . (1

−

Hence the mean and variance are given by µ

= G (1) = 13

rq ,  p

− qs)r+2

 ···

σ

2

= G (1) + G (1)

− [G(1)]2 = r(r  p+21)q

2

+

rq  p

2 2

− r pq2

=

rq  p2

1.21. A word of five letters is transmitted by code to a receiver. The transmission signal is weak, and there is a 5% probability that any letter is in error independently of the others. What is the probability that the word is received correctly? The same word is transmitted a second time with  the same errors in the signal. If the same word is received, what is the probability now that the word is correct?  Let A1 , A2 , A3 , A4 , A5 be the events that the letters in the word are correct. Since the events are independent, the probability that the word is correctly transmitted is P(A1

∩ A2 ∩ A3 ∩ A4 ∩ A5) = 0.955 ≈ 0.774.

If a letter is sent a second time the probability that one error occurs twice is 0 .052 = 0.0025. Hence the probability that the letter is correct is 0.9975. For 5 letters the probability that the word is correct is 0.99755 0.988.

≈

1.22. A random variable N  over the positive integers has the probability distribution with  n

 pn = P(N  = n) =

− n ln(1α − α) ,

(0 < α < 1; n = 1, 2, 3, . . .).

What is its probability generating function? Find the mean of the random variable. The probability generating function is given by G(s) =

∞

αn sn log(1 αs) = n log(1 α) log(1 α) n=1

 −

−

for 0 s < 1/α. Since

− −

≤

G (s) = the mean is µ

−α

− αs) log(1 − α) , −α = G (1) = . (1 − α) log(1 − α) (1

1.23. The source of a beam of light is a perpendicular distance d from a wall of length  2a, with  the perpendicular from the source meeting the wall at its midpoint. The source emits a pulse of  light randomly in a direction  θ, the angle between the direction of the pulse and the perpendicular  is chosen uniformly in the range tan−1 (a/d) θ tan−1 (a/d). Find the probability distribution  of  x ( a x a) where the pulses hit the wall. Show that its density function is given by 

− ≤ ≤

−

f (x) =

≤ ≤

d , 2(x2 + d2 )tan−1 (a/d)

(this the density function of a  Cauchy distribution). If  a mean of this distribution? 

→ ∞, what can you say about the

Figure 1.2 shows the beam and wall. Let X be the random variable representing any displacement

14

x 

a 

wall

a 

d 

beam θ

source

Figure 1.2: Source and beam for Problem 1.23 between

−a and x. Then P(−a ≤ X  ≤ x)

= P( a d tan θ x) = P(tan−1 ( a/d) + tan−1 (x/d)) tan−1 (x/d) + tan−1 (a/d) = 2tan−1 (a/d)

− ≤

≤

−

by uniformity. The density is given by



d tan−1 (x/d) + tan−1 (a/d) dx 2tan−1 (a/d) d 2(x2 + d2 )tan−1 (a/d)

f (x) = = The mean is given by

 

a

µ

=

xd

−a

2(x2

+

dx d2 )tan 1 (a/d)



= 0,

−

since the integrand is an odd function and the limits are a. For the infinite wall the integral defining the mean becomes divergent.

±

1.24. Suppose that the random variable X  can take the integer values 0, 1, 2, . . .. Let  pj and  qj be the probabilities  pj = P(X  = j), qj = P(X > j ), ( j = 0, 1, 2, . . .). Show that, if  G(s) =

∞



j

 pj s ,

H (s) =

j =0

then  (1 s)H (s) = 1 G(s). Show also that  E(X ) = H (1).

−

−

∞



qj sj ,

j =0

Using the series for H (s), (1

− s)H (s)

= (1

∞

s)

= q0 +

∞

(qj

j =1

∞

j

qj s =

j =0

= q0

∞

∞

  − −  −  − j =0

qj −1 )sj

P(X  = j)sj

j =1

15

j

qj s

j =0

qj sj +1

= 1

− p0

∞

 −

 pj sj = 1

j =1

− G(s)

Note that generally H (s) is not a probability  generating function. The mean of the random variable X  is given by E(X ) =

∞



 jp j = G (1) = H (1),

j =1

differentiating the formula above.

16

Chapter 2

Some gambling problems 2.1. In the standard gambler’s ruin problem with total stake a and gambler’s stake k and the gambler’s probability of winning at each play is p, calculate the probability of ruin in the following  cases; (a) a = 100, k = 5, p = 0.6; (b) a = 80, k = 70, p = 0.45; (c) a = 50, k = 40, p = 0.5. Also find the expected duration in each case. For p = 12 , the probability of ruin uk and the expected duration of the game dk are given by



sk uk = 1 (a) uk 0.132, dk (b) uk 0.866, dk (c) For p = 12 ,

≈ ≈

− sa , − sa

dk =

1 1

− 2 p

− k



a(1 sk ) . (1 sa )

− −

≈ 409. ≈ 592. uk =

a

− k,

dk = k(a

a

− k).

so that uk = 0.2, dk = 400. 2.2. In a casino game based on the standard gambler’s ruin, the gambler and the dealer each start  with 20 tokens and one token is bet on at each play. The game continues until one player has no   further tokens. It is decreed that the probability that any gambler is ruined is 0.52 to protect the casino’s profit. What should the probability that the gambler wins at each play be?  The probability of ruin is

sk u= 1

− sa , − sa

where k = 20, a = 40, p is the probability that the gambler wins at each play, and s = (1 Let r = s20 . Then u = r/(1 + r), so that r = u/(1 u) and

−

s=

  u

1

−u

1/20

.

Finally  p =

1 (1 u)1/20 = 1+s (1 u)1/20 + u1/20

−

−

17

≈ 0.498999.

− p)/p.

2.3. Find general solutions of the following difference equations: (a) uk+1 4uk + 3uk−1 = 0; (b) 7uk+2 8uk+1 + uk = 0; (c) uk+1 3uk + uk−1 + uk−2 = 0. (d) puk+2 uk + (1  p)uk−1 = 0, (0 < p < 1).

− − − −

−

(a) The characteristic equation is

m2

− 4m + 3 = 0

which has the solutions m1 = 1 and m2 = 3. The general solution is uk = Amk1 + Bm k2 = A + B.3k , where A and B are any constants. (b) The characteristic equation is

7m2

− 8m + 1 = 0,

which has the solutions m1 = 1 and m2 = 17 . The general solution is uk = A + B

1 . 7k

(c) The characteristic equation is the cubic equation m3

− 3m2 + m + 1 = (m − 1)(m2 − 2m + 1) = 0, √ √ which has the solutions m1 = 1, m2 = 1 + 2, and m3 = 1 − 2. The general solution is √ √ uk = A + B(1 + 2)k + C (1 − 2)k . (d) The characteristic equation is the cubic equation  pm3

− m + (1 − p) = (m − 1)( pm2 + pm − (1 − p)) = 0, √ √ which has the solutions m1 = 1, m2 = − 12 + 12 [(4 − 3 p)/p] and m3 = − 12 − 12 [(4 − 3 p)/p]. The general solution is

uk = A + Bm k2 + Cm k3 .

2.4 Solve the following difference equations subject to the given boundary conditions: (a) uk+1 6uk + 5uk−1 = 0, u0 = 1, u4 = 0; (b) uk+1 2uk + uk−1 = 0, u0 = 1, u20 = 0; (c) dk+1 2dk + dk−1 = 2, d0 = 0, d10 = 0; (d) uk+2 3uk + 2uk−1 = 0, u0 = 1, u10 = 0, 3u9 = 2u8 .

− − − −

−

(a) The charactedristic equation is

m2

− 6m + 5 = 0,

which has the solutions m1 = 1 and m2 = 5. Therefore the general solution is given by uk = A + 5k B. The boundary conditions u0 = 1, u4 = 0 imply A + 54 B = 0,

A + B = 1,

18

which have the solutions A = 625/624 and B =

−1/624. The required solution is

625 uk = 624

−

5k . 624

(b) The characteristic equation is m2

− 2m + 1 = (m − 1)2 = 0,

which has one solution m = 1. Using the rule for repeated roots, uk = A + Bk. The boundary conditions u0 = 1 and u20 = 0 imply A = 1 and B = 1/20. The required solution is uk = (20 k)/20. (c) This is an inhomogeneous equation. The characteristic equation is

−

−

m2

− 2m + 1 = (m − 1)2 = 0,

which has the repeated solution m = 1. Hence the complementary function is A + Bk. For a particular solution, we must try uk = Ck2 . Then dk+1 if  C  =

− 2dk + dk−1 = C (k + 1)2 − 2Ck 2 + C (k − 1)2 = 2C  = −2

−1. Hence the general solution is uk = A + Bk

− k2 .

The boundary conditions d0 = d10 = 0 imply A = 0 and B = 10. Therefore the required solution is uk = k(10 k). (d) The characteristic equation is

−

m3

− 3m + 2 = (m − 1)2(m + 2) = 0, which has two solutions m1 = 1 (repeated) and m2 = −2. The general solution is given by uk = A + Bk + C (−2)k . The boundary conditions imply A + C  = 1,

A + 10B + C ( 2)10 = 0,

−

3A + 27B + 3C ( 2)9 = 2[A + 8B + C ( 2)8 ].

−

−

The solutions of these linear equations are A=

31744 , 31743

B=

3072 , 31743

C  =

1 − 31743

so that the required solution is uk =

1024(31

− 2k) − (−2)k . 31743

2.5. Show that a difference equation of the form  auk+2 + buk+1

− uk + cuk−1 = 0,

where a,b,c 0 are probabilities with  a + b + c = 1, can never have a characteristic equation with  complex roots.

≥

19

The characteristic equation can be expressed in the form am3 + bm2

− m + c = (m − 1)[am2 + (a + b)m − (1 − a − b)] = 0,

since a + b + c = 1. One solution is m1 = 1, and the others satisfy the quadratic equation am2 + (a + b)m

− (1 − a − b) = 0.

The discriminant is given by (a + b)2 + 4(1 since a

− a − b) = (a − b)2 + 4a(1 − a) ≥ 0,

≤ 1.

2.6. In the standard gambler’s ruin problem with equal probabilities p = q = 12 , find the expected  duration of the game given the usual initial stakes of  k units for the gambler and  a k units for  the opponent.

−

The expected duration dk satisfies dk+1

− 2dk + dk−1 = −2.

The complementary function is A + Bk, and for a particular solution try dk = Ck 2 . Then dk+1 if  C  =

− 2dk + dk−1 + 2 = C (k + 1)2 − 2Ck2 + C (k − 1)2 + 2 = 2C + 2 = 0

−1. Hence

dk = A + Bk

− k2 .

The boundary conditions d0 = da = 0 imply A = 0 and B = a. The required solution is therefore dk = k(a

− k).

2.7. In a gambler’s ruin problem the possibility of a draw is included. Let the probability that the gambler wins, loses or draws against an opponent be respectively, p,p, 1 2 p, (0 < p < 12 ). Find the probability that the gambler loses the game given the usual initial stakes of  k units for the gambler  and  a k units for the opponent. Show that  dk , the expected duration of the game, satisfies

−

−

 pdk+1

− 2 pdk + pdk−1 = −1.

Solve the difference equation and find the expected duration of the game. The difference equation for the probability of ruin uk is uk = puk+1 + (1

− 2 p)uk + puk−1

or

uk+1

− 2uk + uk−1 = 0.

The general solution is uk = A + Bk. The boundary conditions u0 = 1 and ua = 0 imply A = 1 and B = 1/a, so that the required probability is given by uk = (a k)/a. The expected duration dk satisfies

−

−

dk+1

− 2dk + dk+1 = −1/p.

The complementary function is A + Bk. For the particular solution try dk = Ck 2 . Then C (k + 1)2

− 2Ck 2 + C (k − 1)2 = 2C  = −1/p 20

if  C  = 1/(2 p). The boundary conditions d0 = da = 0 imply A = 0 and B = a/(2 p), so that the required solution is dk = k(a 2 p)/(2 p).

−

−

2.8. In the changing stakes game in which a game is replayed with each player having twice as many units, 2k and  2(a k) respectively, suppose that the probability of a win for the gambler at  each play is 12 . Whilst the probability of ruin is unaffected by how much is the expected duration of  the game extended compared with the original game? 

−

With initial stakes of  k and a k, the expected duration is dk = k(a k). If the initial stakes are doubled to 2k and 2a 2k, then the expected duration becomes, using the same formula,

−

−

−

d2k = 2k(2a

− 2k) = 4k(a − k) = 4dk .

2.9. A roulette wheel has 37 radial slots of which  18 are red, 18 are black and  1 is green. The gambler bets one unit on either red or black. If the ball falls into a slot of the same colour, then  the gambler wins one unit, and if the ball falls into the other colour (red or black), then the casino wins. If the ball lands in the green slot, then the bet remains for the next spin of the wheel or  more if necessary until the ball lands on a red or black. The original bet is either returned or lost  depending on whether the outcome matches the original bet or not (this is the Monte Carlo system). Show that the probability  uk of ruin for a gambler who starts with  k chips with the casino holdiing  a k chips satisfies the difference equation 

−

36uk+1

− 73uk + 37uk−1 = 0. Solve the difference equation for  uk . If the house starts with  ∈1,000,000 at the roulette wheel and  the gambler starts with  ∈10,000, what is the probability that the gambler breaks the bank if  ∈5,000  are bet at each play. In the US system the rules are less generous to the players. If the ball lands on green then the player simply loses. What is the probability now that the player wins given the same initial stakes?  (see Luenberger (1979))

There is the possibility of a draw (see Example 2.1). At each play the probability that the gambler wins is p = 18 . The stake is returned with probability 37 1 37

  18 37

+

1 372

18 37

+

1 ··· = 361 18 = , 37 74

or the gambler loses after one or more greens also with probability 1/74 by the same argument. Hence uk , the probability that the gambler loses satisfies uk =

18 1 18 uk+1 + (uk + uk−1 ) + uk+1 , 37 74 37

or 36uk+1

− 73uk + 37uk−1 = 0.

The charactersitic equation is 36m2

− 73m + 37 = (m − 1)(36m − 37) = 0,

which has the solutions m1 = 1 and m2 = 37/36. With u0 = 1 and ua = 0, the required solution is uk =

sk 1

− sa , − sa 21

s=

37 . 36

The bets are equivalent to k = 10000/5000 = 2, a = 1010000/5000 = 202. The probability that the gambler wins is 1 sk 1 s2 1 uk = = = 2.23 10−4 . a 202 1 s 1 s In the US system, uk satisfies

− −

−

uk =

− −

18 19 uk+1 + uk−1 , 37 37

or

×

18uk+1

− 37uk + 19uk−1 = 0.

in this case the ratio is s = 19/18. Hence the probability the the gambler wins is 1

2 − uk = 11−−ss202 = 2.06 × 10−6,

which is less than the previous value. 2.10. In a single trial the possible scores 1 and  2 can occur each with probability  12 . If  pn is the probability of scoring exactly  n points at some stage, show that   pn = 12 pn−1 + 12 pn−2 . Calculate p1 and  p2 , and find a formula for  pn . How does pn behave as n becomes large? How do you interpret the result?  Let An be the event that the score is n at some stage. Let B1 be the event score 1, and B2 score 2. Then P(An ) = P(An B1 )P(B1 ) + P(An B2 )P(B2 ) = P(An−1 ) 12 + P(An−2 ) 12 ,

|

|

or  pn = 12 pn−1 + 12 pn−2 . Hence 2 pn

− pn−1 − pn−2 = 0.

The characteristic equation is 2m2

− m − 1 = (m − 1)(2m + 1) = 0, which has the solutions m1 = 1 and m2 = − 12 . Hence  pn = A + B(− 12 )n . The initial conditions are p1 =

1 2

and p2 = A

1 2

+

11 22

− 12 B = 12 ,

= 34 . Hence A + 14 B = 34 ,

so that A = 23 , B = 13 . Hence  pn = As n

2 3

+ 13 (

− 12 )n,

(n = 1, 2, . . .).

→ ∞, pn → 23 .

2.11. In a single trial the possible scores 1 and  2 can occur with probabilities q and  1 q, where 0 < p < 1. Find the probability of scoring exactly  n points at some stage in an indefinite succession  of trials. Show that  1  pn , 2  p

−

→ − 22

as n

→ ∞.

Let pn be the probability. Then  pn = qpn−1 + (1

− q) pn−2,

or

pn

− qpn−1 − (1 − q) pn−2 = 0.

The characteristic equation is m2

− qm − (1 − q) = (m − 1)[m + (1 − q)] = 0, which has the solutions m1 = 1 and m2 = −(1 − q). Hence  pn = A + B(q − 1)n . The initial conditions are p1 = q, p2 = 1 − q + q2 , which imply q = A + B(q − 1), 1 − q + q 2 = A + B(q − 1)2 . The solution of these equations leads to A = 1/(2 − q) and B = (q − 1)/(q − 2), so that  pn =

1

2

n+1

− q [1 − (q − 1)

].

2.12. The probability of success in a single trial is 13 . If  un is the probability that there are no two consecutive successes in  n trials, show that  un satisfies un+1 = 23 un + 29 un−1 . What are the values of  u1 and  u2 ? Hence show that 



√

1 un = (3 + 2 3) 6

 √ 1+ 3 3

n

+ (3

√

−2

3)

 −√   1

3

n

3

.

Let An be the event that there have not  been two consecutive successes in the first n trials. Let B1 be the event of success and B2 the event of failure. Then P(An ) = P(An B1 )P(B1 ) + P(An B2 )P(B2 ).

|

|

Now P(An B2 ) = P(An−1 ): failure will not change the probability. Also

|

P(An B1 ) = P(An−1 B2 )P(B2 ) = P(An−2 )P(B2 ).

|

|

Since P(B1 ) = 13 , P(B2 ) = 23 , un = 29 un−2 + 23 un−1 where un = P(An ). The characteristic equation is which has the solutions m1

or

9un

− 6un−1 − 2un = 0,

9m2

− 6m − 2 = 0, √ √ = 1 (1 + 3) and m = 1 (1 − 3). Hence 2

3

un = A

1 (1 + 3n

√

3

3)n + B 23

1 (1 3n

−

√

3)n .

− 13 13 = 89 . Therefore A and B are defined by √ B √ A 1 = (1 + 3) + (1 − 3), 3 3 √ √ √ B √ 8 A B A = (1 + 3)2 + (1 − 3)2 = (4 + 2 3) + (4 − 2 3). 9 9 9 9 9 √ √ The solutions are A = 1 (2 3 + 3) and B = 1 (−2 3 + 3). Finally The initial conditions are u1 = 1 and u2 = 1

6

6

un =

√

1 [(2 3 + 3)(1 + 6 3n

·

√

√

3)n + ( 2 3 + 3)(1

−

−

√

3)n ].

2.13. A gambler with initial capital  k units plays against an opponent with initial capital  a k units. At each play of the game the gambler either wins one unit or loses one unit with probability  1 . Whenever the opponent loses the game, the gambler returns one unit so that the game may  2 continue. Show that the expected duration of the game is k(2a 1 k) plays.

−

− −

The expected duration dk satisfies dk+1

− 2dk + dk−1 = −2,

(k = 1, 2, . . . , a

− 1).

The boundary conditions are d0 = 0 and da = da−1 , indicating the return of one unit when the gambler loses. The general solution for the duration is

− k2 .

dk = A + Bk The boundary conditions imply

− a2 = A + B(a − 1) − (a − 1)2, so that B = 2a − 1. Hence dk = k(2a − 1 − k). A = 0,

A + Ba

2.14. In the usual gambler’s ruin problem, the probability that the gambler is eventually ruined is uk =

sk 1

− sa , − sa

s=

q ,  p

( p = 12 ).



In a new game the stakes are halved, whilst the players start with the same initial sums. How does this affect the probability of losing by the gambler? Should the gambler agree to this change of rule if  p < 12 ? By how many plays is the expected duration of the game extended?  The new probability of ruin vk (with the stakes halved) is, adapting the formula for uk , vk = u2k Given p < 12 , then s = (1

s2k s2a (sk + sa )(sk sa ) = = = uk 1 s2a (1 sa )(1 + sa )

− −

−

−



sk + sa 1 + sa

− p)/p > 1 and sk > 1. It follows that vk > uk

  1 + sa 1 + sa

= uk .

With this change the gambler is more likely to lose. From (2.9), the expected duration of the standard game is given by dk =

1 1

− 2 p

− k

24



a(1 sk ) . (1 sa )

− −



.

With the stakes halved the expected duration hk is hk = d2k =



1 1

2k

− 2 p

−



2a(1 s2k ) . (1 s2a )

− −

The expected duration is extended by hk

− dk

= =

1 1

− 2 p 1

1

− 2 p

− 

 

2a(1 s2k ) a(1 sk ) + (1 s2a ) (1 sa )

− − k − − a(1 − sk )(sa − 1 − 2sk ) k+ (1 − s2a )

.

2.15. In a gambler’s ruin game, suppose that the gambler can win £2 with probability  with probability  23 . Show that  uk =

(3k

1 3

or lose £1

− 1 − 3a)(−2)a + (−2)k . 1 − (3a + 1)(−2)a

Compute uk if  a = 9 for  k = 1, 2, . . . , 8. The probability of ruin uk satisfies uk = 13 uk+2 + 23 uk−1

or

uk+2

− 3uk + 2uk = 0.

The characteristic equation is m3

− 3m + 2 = (m − 1)2(m + 2) = 0, which has the solutions m1 = 1 (repeated) and m2 = −2. Hence uk = A + Bk + C (−2)k . The boundary conditions are u0 = 1, ua = 0, ua−1 = 23 ua−2 . The constants A, B and C  satisfy A + Ba + C ( 2)a = 0,

A + C  = 1,

−

3[A + B(a + 1) + C ( 2)a−1 ] = 2[A + B(a

−

or A + B(a + 1)

− 2) + C (−2)a−2],

− 8C (−2)a−2 = 0.

The solution of these equations is A=

−(−2)a (3a + 1) , 1 − (−2)a (3a + 1)

B=

Finally uk =

(3k

1

3( 2)a , ( 2)a (3a + 1)

−−

−

C  =

1

1 . ( 2)a (3a + 1)

−−

− 1 − 3a)(−2)a + (−2)k . 1 − (−2)a (3a + 1)

The values of the probabilities uk for a = 9 are shown in the table below. k

1

2

3

4

5

6

7

8

uk

0.893

0.786

0.678

0.575

0.462

0.362

0.241

0.161

25

2.16. Find the general solution of the difference equation  uk+2

− 3uk + 2uk−1 = 0.

A reservoir with total capacity of  a volume units of water has, during each day, either a net  inflow of two units with probability  13 or a net outflow of one unit with probability  23 . If the reservoir is full or nearly full any excess inflow is lost in an overflow. Derive a difference equation   for this model for  uk , the probability that the reservoir will eventually become empty given that it  initially contains k units. Explain why the upper boundary conditions can be written  ua = ua−1 and  ua = ua−2 . Show that the reservoir is certain to be empty at some time in the future. The characteristic equation is m3

− 3m + 2 = (m − 1)2(m + 2) = 0.

The general solution is (see Problem 2.15) uk = A + Bk + C ( 1)k .

−

The boundary conditions for the reservoir are ua = 13 ua + 23 ua−1 ,

u0 = 1,

ua−1 = 13 ua + 23 ua−2 .

The latter two conditions are equivalent to ua = ua−1 = ua−2 . Hence A + Ba + C ( 2)a = A + B(a

A + C  = 1,

−

− 1) + C (−2)a−1 = A + B(a − 2) + C (−2)a−2.

which have the solutions A = 1, B = C  = 0. The solution is uk = 1, which means that that the reservoir is certain to empty at some future date. 2.17. Consider the standard gambler’s ruin problem in which the total stake is a and gambler’s stake is k, and the gambler’s probability of winning at each play is p and losing is q = 1  p. Find  uk , the probability of the gambler losing the game, by the following alternative method. List the difference equation (2.2) as

−

u2 u3

uk

− u1 − u2

= =

− uk−1

=

s(u1 u0 ) = s(u1 1) s(u2 u1 ) = s2 (u1 1) .. . s(uk−1 uk−2 ) = sk−1 (u1

− −

− −

−

− 1),

where s = q/p = 12 and  k = 2, 3, . . . a. The boundary condition  u0 = 1 has been used in the first  equation. By adding the equations show that 



uk = u1 + (u1

−

s sk 1) . 1 s

− −

Determine u1 from the other boundary condition  ua = 0, and hence find  uk . Adapt the same method for the special case p = q = 12 . Addition of the equations gives uk

k − u1 = (u1 − 1)(s + s2 + ··· + sk−1) = (u1 − 1) s1−−ss

26

summing the geometric series. The condition ua = 0 implies a −u1 = (u1 − 1) s1−−ss .

Hence

− sa , − sa sk − sa uk = . 1−s u1 =

so that

s 1

2.18. A car park has 100 parking spaces. Cars arrive and leave randomly. Arrivals or departures of cars are equally likely, and it is assumed that simultaneous events have negligible probability. The ‘state’ of the car park changes whenever a car arrives or departs. Given that at some instant  there are k cars in the car park, let  uk be the probability that the car park first becomes full before it becomes empty. What are the boundary conditions for  u0 and  u100 ? How many car movements can be expected before this occurs?  The probability uk satisfies the difference equation uk =

1 1 uk+1 + uk−1 2 2

or

uk+1

− 2uk + uk−1.

The general solution is uk = A + Bk. The boundary conditions are u0 = 0 and u100 = 1. Hence A = 0 and B = 1/100, and uk = k/100. The expected duration of car movements until the car park becomes full is dk = k(100 k).

−

2.19. In a standard gambler’s ruin problem with the usual parameters, the probability that the gambler loses is given by  sk sa 1  p uk = , s= . a 1 s  p If  p is close to that 

1 2,

given say by  p = uk =

a

1 2

− −

−

+ ε where ε is small, show, by using binomial expansions,

||





− k 1 − 2kε − 4 (a − 2k)ε2 + O(ε3) a 3

as ε 0. (The order  O terminology is defined as follows: we say that a function  g(ε) = O(εb ) as ε 0 if  g(ε)/εb is bounded in a neighbourhood which contains ε = 0. See also the Appendix in the book.)

→

→

Let p =

1 2

+ ε. Then s = (1

− 2ε)/(1 + 2ε), and (1 − 2ε)k (1 + 2ε)−k − (1 − 2ε)a (1 + 2ε)−a uk = . 1 − (1 − 2ε)a (1 + 2ε)−a

Apply the binomial theorem to each term. The result is uk =

a





− k 1 − 2kε − 4 (a − 2k)ε2 + O(ε3) a 3

.

[Symbolic computation of the series is a useful check.] 2.20. A gambler plays a game against a casino according to the following rules. The gambler and  casino each start with 10 chips. From a deck of 53 playing cards which includes a joker, cards are 27

randomly and successively drawn with replacement. If the card is red or the joker the casino wins 1 chip from the gambler, and if the card is black the gambler wins 1 chip from the casino. The game continues until either player has no chips. What is the probability that the gambler wins? What  will be the expected duration of the game?  From (2.4) the probability uk that the gambler loses is (see (2.4)) sk uk = 1

− sa , − sa

with k = 10, a = 20, p = 26/53, and s = 27/26. Hence u10 =

(27/26)10 (27/26)20 1 (27/26)20

−

−

≈ 0.593.

Therefore the probability that the gambler wins is approximately 0.407. By (2.9) 1 a(1 sk dk = k = 98.84, 1 2 p 1 sa

−

−

for the given data.

− −



2.21. In the standard gambler’s ruin problem with total stake a and gambler’s stake k, the probability  that the gambler loses is sk sa uk = , 1 sa where s = (1 that,

− p)/p. Suppose that  uk = 12 , that is fair odds. Express k as a function of  a. Show 

Given

then 1

− −

ln[ 12 (1 + sa )] k= . ln s

sk uk = 1

− sa − sa

and

uk = 12 ,

− sa = 2(sk − sa) or sk = 12 (1 + sa). Hence k=

ln[ 12 (1 + sa )] , ln s

but generally k will not be an integer. 2.22. In a gambler’s ruin game the probability that the gambler wins at each play is αk and loses is 1 αk , (0 < αk < 1, 0 k a 1), that is, the probability varies with the current stake. The probability  uk that the gambler eventually loses satisfies

−

≤ ≤ −

uk = αk uk+1 + (1

− αk )uk−1,

uo = 1,

Suppose that  uk is a specified function such that  0 < uk < 1, (1 Express αk in terms of  uk−1 , uk and  uk+1 . Find  αk in the following cases: (a) uk = (a k)/a; (b) uk = (a2 k2 )/a2 ; (c) uk = 12 [1 + cos(kπ/a)].

− −

28

ua = 0.

≤ k ≤ a − 1), u0 = 1 and  ua = 0.

From the difference equation αk = (a) uk = (a

− k)/a. Then αk =

uk uk−1 . uk+1 uk−1

− −

(a k) (a k + 1) 1 = , (a k 1) (a k + 1) 2

− − − − − − −

which is to be anticipated from eqn (2.5). (b) uk = (a2 k 2 )/a2 . Then

−

αk =

(a2

− k2) − [a2 − (k − 1)2] = 2k − 1 . [a2 − (k + 1)2 ] − [a2 − (k − 1)2 ] 4k

(c) uk = 1/(a + k). Then αk =

[1/(a + k)] [1/(a + k 1)] a+k+1 = . [1/(a + k + 1)] [1/(a + k 1)] 2(a + k)

−

−

−

−

2.23. In a gambler’s ruin game the probability that the gambler wins at each play is αk and loses is 1 αk , (0 < αk < 1, 1 k a 1), that is, the probability varies with the current stake. The probability  uk that the gambler eventually loses satisfies

−

≤ ≤ −

uk = αk uk+1 + (1

− αk )uk−1,

uo = 1,

ua = 0.

Reformulate the difference equation as uk+1 where β k = (1

− uk = β k (uk − uk−1),

− αk )/αk . Hence show that  uk = u1 + γ k−1 (u1

− 1),

(k = 2, 3, . . . , a)

where γ k = β 1 + β 1 β 2 +

··· + β 1β 2 . . . βk  .

Using the boundary condition at  k = a, confirm that  uk =

γ a−1 γ k−1 . 1 + γ a−1

−

Check that this formula gives the usual answer if  αk = p = 12 , a constant.



The difference equation can be expressed in the equivalent form uk+1 where β k = (1

− uk = β k (uk − uk−1),

− αk )/αk . Now list the equations as follows, noting that u0 = 0,: u2 − u1 = β 1 (u1 − 1) u3 − u2 = β 1 β 2 (u1 − 1) ··· = ··· uk − uk−1 = β 1 β 2 ··· β k−1 (u1 − 1)

Adding these equations, we obtain uk

− u1 = γ k−1(u1 − 1), 29

where γ k−1 = β 1 + β 1 β 2 +

··· + β 1β 2 ··· β k−1.

The condition ua = 0 implies

−u1 = γ a−1(u1 − 1), so that u1 + Finally uk = If  αk = p = 12 , then β k = (1



γ a−1 γ k−1 . 1 + γ a−1

−

− p)/p = s, say, and 2

γ k = s + s + Hence uk = as required.

(s

γ a−1 . 1 + γ a−1

k

··· + s

s

=

− sk+1 . 1−s

− sa)/(1 − s) − (s − sk )/(1 − s) = sk − sa 1 + (s − sa )/(1 − s) 1 − sa

2.24. Suppose that a fair  n-sided die is rolled  n independent times. A match is said to occur if side i is observed on the ith trial, where i = 1, 2, . . . , n. (a) Show that the probability of at least one match is

  − −

1

1 n

1

n

.

(b) What is the limit of this probability as n ?  (c) What is the probability that just one match occurs in  n trials?  (d) What value does this probability approach as n ?  (e) What is the probability that two or more matches occur in  n trials? 

→∞

→∞

(a) The probability of  no matches is

− n

n

1

n

.

The probability of at least one match is 1 (b) As n

→ ∞,

    − − − − − → n

n

1

=1

n

1 n

1

1

1 n

n

.

n

e−1 .

Hence for large n, the probability of at least one match approaches 1 (c) There is only one match with probability

− n

1

n

30

n 1

−

.

− e−1 = (e − 1)/e.

(d) As n

→∞

 −   −  −  n

1

n 1

−

n

=

1

1 n

1

1 n

n

= e−1 .

(e) Probability of two or more matches is

 −  − −   −  n

1

n

n 1

−

n

1

n

31

n

1 = n

1

1 n

n 1

−

.

Chapter 3

Random Walks 3.1. In a simple random walk the probability that the walk advances by one step is p and retreats by one step is q = 1  p. At step n let the position of the walker be the random variable Xn . If the walk starts at  x = 0, enumerate all possible sample paths which lead to the value X4 = 2. Verify that 

−

P X4 =

{

−2} =

−



4  pq 3 . 1

If the walks which start at x = 0 and finish at x = 2, then each walk must advance one step with probability p and retreat 3 steps with probability q 3 . the possible walks are:

−

0 0 0 0

→ −1 → −2 → −3 → −2 → −1 → −2 → −1 → −2 → 1 → 0 → −1 → −2 → −1 → 0 → −1 → −2

By (3.4), P X4 =

{

−2} =



4  pq 3 . 1

3.2. A symmetric random walk which starts from the origin. Find the probability that the walker is at the origin at step 8. What is the probability also at step 8 that the walker is at the origin but that it is not the  first visit there?  By (3.4), the probability that the walker is at the origin at step 8 is given by P(X8 ) =



8 1 8! 1 = = 0 .273. 4 28 4!4! 28

The generating function of the first returns f n is given by (see Section 3.4) Q(s) =

∞



f n sn = 1

n=0

2

− (1 − s )

1 2

.

We require f 8 in the expansion of  Q(s). Thus, using the binomial theorem, the series expansion for Q(s) is 1 1 1 6 5 8 Q(s) = s2 + s4 + s + s + O(s10 ). 2 8 16 128 Therefore the probability of a first  return at step 8 is 5 /128 = 0.039. Hence the probability that the walk is at the origin but not a first return is 0 .273 0.039 = 0.234.

−

32

3.3. An asymmetric walk starts at the origin. From eqn (3.4), the probability that the walk reaches x in  n steps is given by  n vn,x = 1  p (n+x) q (n−x) , (n + x) 2





1 2

1 2

where n and  x are both even or both odd. If  n = 4, show that the mean value position  x is 4( p confirming the result in Section 3.2.

− q),

The furthest positions that the walk can reach from the origin in 4 steps are x = 4 and x = 4, and since n is even, the only other positions reachable are x = 2, 0, 2. Hence the required mean value is

−

−

µ

= = = =

−4v − − 2v − + 2v + 4v −4 40 q − 2 41  pq + 2 43  p q + 4 −4q − 8 pq + 8 p q + 4 p 4( p − q)( p + q ) = 4( p − q ). 4, 4

4, 2

4,2

4,4

    4

3

4

3

3

3

4 4  p 4

4

3

3.4. The pgf for the first return distribution  f n , (n = 1 , 2, . . .), to the origin in a symmetric random walk  is given by  F (s) = 1 (1 s2 ) ,

{ }

1 2

− −

(see Section 3.4). (a) Using the binomial theorem find a formula for  f n , the probability that the first return occurs at the n-th  step. (b) What is the variance of  f n ?  Using the binomial theorem F (s) = 1

2

− (1 − s )

1 2

   − −  ∞

=1

1 2

n=0

n

( 1)n s2n .

(a) From the series, the probability of a first return at step n is f n =

 − 

( 1) 0

(b) The variance of  f n is defined by



V = F  (1)

1 2

n+1

1 2 1 n 2

n even n odd

2

− F  (1) − [F (1)] .

We can anticipate that the variance will be infinite (as is the case with the mean) since the limit of the derivatives of  F (s) are unbounded as s 1.

→

3.5. A coin is spun  2n times and the sequence of heads and tails is recorded. What is the probability that  the number of heads equals the number of tails after  2n spins?  This problem can be viewed as a symmetric random walk starting at the origin in which a head is represented as a step to the right, say, and a tail a step to the left. We require the probability that the walk returns to the origin after 2 n steps, which is (see Section 3.3) v2n,0



1 2n = 2n n 2

=

(2n)! 22n n!n!

.

3.6. For an asymmetric walk with parameters p and  q = 1  p, the probability that the walk is at the origin  after  n steps is n  p n q n n even , 1 qn = vn,0 = n 2 0 n odd 

−

    33

1 2

1 2

 from eqn (3.4). Show that its generating function is H (s) = (1

1 2

2

− 4 pqs )−

.

If  p = q , show that the mean number of steps to the return is



4 pqs

m = H  (1) =

(1

−

3

4 pqs2 ) 2

What is its variance? 



4 pq

=

(1

s=1

− 4 pq)

.

3 2

The generating function H (s) is defined by H (s)

  ∞

=

n=0

=

(1

 −  ∞

2n n n 2n  p q s = n 2

− 4 pqs )−

1 2

22n

n

n=0

1 2

 pn q n s2n

using the binomial identity from Section 3.3 or the Appendix. The mean number of steps to the return is

 µ = H  (1) =

4 pqs (1

−

3

4 pqs2 ) 2

The second derivative of  H (s) is H  (s) =



4 pq

=

(1

s=1

4 pq + 32 p2 q2 s2 (1

2

− 4 pqs )

5 2

− 4 pq)

3 2

.

.

Hence the variance is V(W )

= =

H  (1) + H  (1)

4 pq [(1

− 4 pq)

1 2

2

− [H  (1)]

4 pqs

=

3 2

(1

− 4 pq) (1 + 4 pq − 4 p q ) − 4 pq ] . (1 − 4 pq )

+

4 pq (1

2 2

− 4 pq)

3 2

+

16 p2 q 2 (1 4 pq )3

−

3

3.7. Using the results of Problem 3.6 and eqn (3.12) relating to the generating functions of the returns and   first returns to the origin, namely  H (s) 1 = H (s)Q(s),

−

which is still valid for the asymmetric walk, show that  Q(s) = 1

1 2

2

− (1 − 4 pqs )

,

where p = q . Show that a first return to the origin is not certain unlike the situation in the symmetric walk. Find the mean number of steps to the first return.



From Problem 3.6, H (s) = (1

so that, by (3.12), Q(s) = 1

2

− 4 pqs )−

1 2

,

− H 1(s) = 1 − (1 − 4 pqs ) 2

1 2

.

It follows that Q(1)

=

∞



f n = 1

n=1

=

[( p

2

− q) ]

1 2

− (1 − 4 pq) =1

1 2

=1

− | p − q| < 1, 34

2

− [( p + q) − 4 pq]

1 2

if  p = q . Hence a first return to the origin is not certain. The mean number of steps is



µ

= Q (1) =

4 pq (1

− 4 pq)

4 pq

=

1 2

| p − q| .

3.8. A symmetric random walk starts from the origin. Show that the walk does not revisit the origin in the  first  2n steps with probability  hn = 1 f 2 f 4 f 2n ,

− − −···−

where f n is the probability that a first return occurs at the n-th step. The generating function for the sequence f n is

{ }

Q(s) = 1

2

− (1 − s )

1 2

,

(see Section 3.4). Show that  f n =

−  ( 1)

1 2

1 2 1 2

n+1

n even 

n

n odd 

0

(n = 1 , 2, 3, . . .).

,

Show that  hn satisfies the first-order difference equation  hn+1

−h

n

n+1

= ( 1)

−

  1 2

n+1

.

Verify that this equation has the general solution  hn = C  +



2n 1 , n 22n

where C  is a constant. By calculating  h1 , confirm that the probability of no return to the origin in the first  2n steps is 2n /22n . n



The probability that a first return occurs at step 2 j is f 2j : a first return cannot occur after an odd number of steps. Therefore the probability hn that a first return has not occurred in the first 2 n steps is given by the difference hn = 1 f 2 f 4 f 2n .

− − −···−

The probability f m , that the first return to the origin occurs at step m is the coefficient of  sm in the expansion Q(s) = 1

Therefore f m =

2

− (1 − s )

=

   − − ∞

n=1

1 2

n

( s2 ) n .

 −    −  − −· ·· −   − −··· − ( 1)

1 2

1 2

m+1

hn

=

1

=

1

m even

1 2m

0

The difference

1 2

f 2

m odd

f 4

1 2

1

(m = 1 , 2, . . .)

f 2n

+

1 2

+

2

1 2

n

Hence hn staisfies the difference equation hn+1

−h

n

=

 − 1 2

n+1

35

( 1)n+1 .

( 1)n

The homogeneous part of the difference equation has a constant solution C , say. For the particular solution try the choice suggested in the question, namely

   −  −  −  −   −  2n 1 . n 22n

hn =

Then hn+1

−h

n

= = = =

Hence

2n + 2 1 2n+2 n+1 2

2n 1 n 22n

2n 1 1 n 22n+1 (n + 1)

( 1)n+1 2(n + 1)

( 1)n+1

1 2

n

(using the binomial identity before (3.7)

1 2

n+1

hn = C  +

1 2n . 22n n

The initial condition is h1 = 12 , from which it follows that C  = 0. Therefore the probability that no return to the origin has occurred in the first 2 n steps is



1 2n hn = 2n . n 2 3.9. A walk can be represented as a connected graph between coordinates (n, y ) where the ordinate y is the position on the walk, and the abscissa  n represents the number of steps. A walk of  7 steps which joins (0, 1) and  (7, 2) is shown in Fig. 3.1. Suppose that a walk starts at  (0, y1 ) and finishes at  (n, y2 ), where

−

Figure 3.1: Representation of a random walk y1 > 0, y2 > 0 and  n + y2 y1 is an even number. Suppose also that the walk first visits the origin at  n = n1 . Reflect that part of the path for which  n n1 in the n-axis (see Fig. 3.1), and use a reflection  argument to show that the number of paths from  (0, y1 ) to (n, y2 ) which touch or cross the n-axis equals the number of all paths from  (0, y1 ) to (n, y2 ). This is known as the reflection principle.

−

≤

−

All paths from (0, y1 ) to (n, y2 ) must cut the n axis at least once. Let (n1 , 0) be the first such contact n1 and y with n axis. Reflect the path for n 0 in the n axis. The result is a path from (0, y1 ) to (n, y2 ) which touches or cuts the n axis at least once. All such paths must be included.

−

≤

≤

36

3.10. A walk starts at  (0, 1) and returns to (2n, 1) after 2n steps. Using the reflection principle (see Problem  3.9) show that there are (2n)! n!(n + 1)! different paths between the two points which do not ever revisit the origin. What is the probability that  the walk ends at  (2n, 1) after  2n steps without ever visiting the origin, assuming that the random walk is symmetric?  Show that the probability that the first visit to the origin after 2n+1 steps is  pn =

1 22n+1

(2n)! . n!(n + 1)!

Let M (m, d) represent the total number of different paths in the ( n, y ) plane which are of length m  joining positions denoted by y = y1 and y = y2 : here d is the absolute difference d = y2 y1 . The total number of paths from (0 , 1) to (0, 2n) is

| − |

M (2n, 0) =



2n . n

By the reflection principle (Problem 3.9) the number of paths which cross the n axis (that is, visit the origin) is 2n M (2n, 2) = . n 1

  −  − 

Hence the number of paths from (0 , 1) to (0, 2n) which do not  visit the origin is M (2n, 0)

− M (2n, 2)

= =

2n n

2n

n

(2n)!) n!(n + 1)!

=

−1

(2n)! n!n!

(2n)! − (n − 1)!( n + 1)!

The total number of paths is 2 2n . Also to visit the origin for the first time at step 2n + 1, the walk must be at y = 1 at step 2n, from where there is a probability of  12 that the walk moves to the origin. Hence the probability is (2n)! 1 (2n)! 1 1  pn = 2n . = 2n+1 n!(n + 1)! 2 n!(n + 1)! 2 2 3.11. A symmetric random walks starts at the origin. Let  f n,1 be the probability that the first visit to position  x = 1 occurs at the n-th step. Obviously, f 2n,1 = 0. The result from Problem 3.10 can be adapted  to give (2n)! 1 f 2n+1,1 = 2n+1 , (n = 0 , 1, 2, . . .). n!(n + 1)! 2 Suppose that its pgf is G1 (s) =

∞



f 2n+1,1 s2n+1 .

n=0

Show that 

G1 (s) = [1

[Hint: the identity  1 22n+1

2

− (1 − s )

 

1 (2n)! 2 = ( 1)n , n!(n + 1)! n+1

−

1 2

]/s.

(n = 0 , 1, 2, . . .)

is useful in the derivation of  G1 (s).]  Show that any walk starting at the origin is certain to visit  x > 0 at some future step, but that the mean number of steps in achieving this is infinite.

37

The result f 2n+1,1 =

1 22n+1

(2n)! n!(n + 1)!

is simply the result in the last part of Problem 3.10. For the pgf 

∞

  

G1 (s) =

∞

n=0

(2n)!s2n+1 n!(n + 1)!

1

   − −   − − −     −   

f 2n+1,1 s2n+1 =

n=0

22n+1

The identity before (3.7) (in the book) states that 2n n

Therefore, using this result (2n)! = 2n+1 n!(n + 1)! 2

1 2

= ( 1)n

22n .

n

1 2

2n 1 = ( 1)n 2n+1 n 2 (n + 1)

Hence G1 (s)

=

∞

1 2

( 1)n

=

1 2

1

1 [1 s

1 2

s

1 2

s3 +

2

2

− (1 − s )

1 2

−

s2n+1

n+1

s=0

=

n

 

1 1 2 . = ( 1)n n+1 2(n + 1)

3

s5

−···

]

using the binomial theorem. That G1 (1) = 1 implies that the random walk is certain to visit x > 0 at some future step. However,  which means that expected number to that event is infinite. G (1) =

∞

3.12. A symmetric random walk starts at the origin. Let  f n,x be the probability that the first visit to position  x occurs at the n-th step (as usual, f n,x = 0 if  n + x is an odd number). Explain why  n 1

f n,x =

−



(n

f n−k,x−1 f k,1 ,

k=1

If  Gx (s) is its pgf, deduce that  Gx (s) = G1 (s)

{

}

x

≥ x > 1).

,

where G1 (s) is given explicitly in Problem 3.11. What are the probabilities that the walk first visits x = 3 at the steps n = 3, n = 5 and  n = 7?  Consider k = 1. The first visit to x 1 has probability f n−1,x−1 in n 1 steps. Having reached there the walk must first visit x in one further step with probability f 1,1 . Hence the probability is f n−1,x−1 f 1,1 . If  k = 2, the first visit to x 1 in n 2 steps occurs with probability f n−2,x−1 : its first visit to x must occur after two steps. Hence the probability is f n−2,x−1 f 2,1 . And so on. The sum of these probabilities gives

−

−

−

−

n 1

f n,x =

−



f n−k,x−1 f k,1 ,

(n

k=1

≥ x > 1).

Multiply both sides of this equation by sn and sum over n from n = x: Gx (s) =

∞ n−1



f n−k,x−1 f k,1 sn = Gx−1 (s)G1 (s).

n=x k=1

By repeated application of this difference equation, it follows that

 −

1 Gx (s) = s

1

38

(1

2

−s )

1 2



x

.

For x = 3,

 −



3 1 G1 (s) = . 1 (1 s2 ) s Expansion of this function as a Taylor series in s gives the coefficients and probabilities:

G2 (s) =

1 2

−

1 3 1 5 1 7 s + s + s + O(s9 ). 4 8 64

3.13. Problem 3.12 looks at the probability of a first visit to position  x 1 at the n-th step in a symmetric random walk which starts at the origin. Why is the pgf for the first visit to position  x where x 1 given  by  Gx (s) = G1 (s) |x| ,

≥

{

where F 1 (s) is defined in Problem 3.11?  First visits to x > 0 and to

| |≥

}

−x at step n must be equally likely. Hence f  Gx(s) = [1 − (1 − s ) ]| | .

n,x

2

1 2

= f n,−x . Therefore

x

3.14. An asymmetric walk has parameters p and  q = 1  p = p. Let  gn,1 be the probability that the first visit  to x = 1 occurs at the n-th step. As in Problem 3.11, g2n,1 = 0. It was effectively shown in Problem 3.10  that the number of paths from the origin, which return to the origin after  2n steps is

− 

(2n)! . n!(n + 1)! Explain why  g2n+1,1 =

Suppose that its pgf is G1 (s) =

(2n)!  pn+1 q n . n!(n + 1)!

∞



g2n+1,1 s2n+1 .

n=0

Show that  G1 (s) = [1

2

− (1 − 4 pqs )

1 2

]/(2qs ).

(The identity in Problem 3.11 is required again.) What is the probability that the walk ever visits x > 0? How does this result compare with that for the symmetic random walk?  What is the pgf for the distribution of first visits of the walk to x = 1 at step 2n + 1? 

−

The probability that the first return to x = 1 at the (2n + 1)th step is g2n+1,1 The number of paths of  length 2n which never visit x = 1 is (adapt the answer in Problem 3.10), (2n)! . n!(n + 1)! The consequent probability of this occurrence is, since there are n steps to the right with probability p and to the left with probability q, (2n)!  pn q n . n!(n + 1)! the probability that the next step visits x = 1 is (2n)!  pn+1 q n , n!(n + 1)! which is the previous probability multiplied by p. Using the identity in Problem 3.11, g2n+1,1 =

 − 1 2

n+1

( 1)n (2 p)n (2q )n 2 p.

39

Its pgf  G1 (s) is given by G1 (s) =

  − ∞

n=0

1 2

n+1

( 1)n (4 pq )n 2 ps2n+1 = [1

2

− (1 − 4 pqs )

1 2

]/(2qs )

using a binomial expansion. Use an argumant that any walk which enters x > 0 must first visit x = 1 as follows. The probability that the walks first visit to x = 1 occurs at all is

∞



g2n+1,1 = G1 (1) =

n=0

1 [1 2q

2

− {( p + q) − 4 pq}

1 2

]=

1 [1 2q

− | p − q|].

A symmetry argument in which p and q are interchanged gives the pgf for the distribution of first visits to x = 1, namely H 1 (s) = [1 (1 4 pqs2 ) ]/(2 ps).

−

1 2

− −

3.15. It was shown in Section 3.3 that, in a random walk with parameters p and  q = 1 that a walk is at position  x at step n is given by  vn,x =

where

1 n 2(





n  p 1 (n + x) 2

1 2

(n+x)

q

1 2

(n x)

− ,

− p, the probability 

|x| ≤ n,

+ x) must be an integer. Verify that  vn,x satisfies the difference equation  vn+1,x = pvn,x−1 + qvn,x+1 ,

subject to the initial conditions v0,0 = 1 ,

vn,x = 0 ,

(x = 0) .



Note that this difference equation has differences on two arguments. Can you develop a direct argument which justifies the difference equation for the random walk?  Given vn,x =

then  pvn,x−1 + qv n,x+1

=

    −   n  p + x)

1 (n 2

p

1 (n 2

n +x

+q

1 2

(n+x+1)

1 2

(n+x)

1)

1 2 (n 1 2

 p

1 2

q

1 2

(n x)

− ,

(n+x 1)

− q

n  p + x + 1)

1 2

|x| ≤ n, 1 2

(n x+1)

−

(n+x+1)

n 1 ( +x 2 n

(n x+1)

−

q

1 2

(n x 1)

−−



=

p

=

p (n+x+1) q (n−x+1) n! [ 1 (n + x + 1) + 12 (n [ 12 (n + x + 1)]![ 12 (n x + 1)]! 2

=

q

1 2

1 2

−

p

1 2

(n+x+1)

q

1 2

(n x+1)

−



n+1 1 n x 2( + )

− 1)



+

n 1 ( + x + 1) 2 n



− x + 1)]

= vn+1,x .

3.16. In the usual notation, v2n,0 is the probability that, in a symmetric random walk, the walk visits the origin after  2n steps. Using the difference equation from Problem 3.15, v2n,0 satisfies v2n,0 = 12 v2n−1,−1 + 12 v2n−1,1 = v2n−1,1 .

How can the last step be justified? Let  G1 (s) =

∞



v2n−1,1 s2n−1

n=1

40

be the pgf of the distribution  v2n−1,1 . Show that 

{

}

G1 (s) = [(1

1 2

2

− s )− − 1]/s.

By expanding  G1 (s) as a power series in  s show that 

 − 2n

v2n−1,1 =

1

1

− .

22n 1

n

By a repetition of the argument show that  G2 (s) =

∞



v2n,2 s2n = [(2

n=0

1 2

2

− s)(1 − s )− − 2]/s.

Use a symmetry argument. Multiply both sides of the difference equation by s2n and sum from n = 1 to infinity. Then

∞



v2n,0 s2n = s

n=1

H (s) 2

−s )

1 2

v2n−1,1 s2n−1 .

n=1

Therefore in the notation in the problem

where H (s) = (1

∞



− 1 = sG (s), 1

. Therefore G1 (s) =

∞



v2n−1,1 s2n−1

n=1

as required. From the series for G1 (s) expanded as a binomial series, the general coefficient is v2n−1,1 =

−  −   1 2

n

(2n)! 2n 1 = 2n = 2n n 2 2 n!n!

n

( 1) =

 −

2n 1 1 . 2n n 2 −1

From the difference equation v2n+1,1 = 12 v2n,0 + 12 v2n,2 .

Multiplying by s2n+1 and summing over n 1 G1 (s) = 2

∞



2n+1

v2n,0 s

n=0

Therefore G2 (s) =

∞

1 + 2



1 [(2 3

− s)(1 − s )− − 2].

v2n,2 s2n+1 =

n=0

2

1 1 sH (s) + sG2 (s). 2 2

1 2

3.17. A random walk takes place on a circle which is marked out with  n positions. Thus, as shown in  Fig. 3.2, position  n is the same as position  O. This is known as a  cyclic random walk of  period n. A symmetric random walk starts at  O . What is the probability that the walk is at  O after  j steps in the cases: (a) j < n; (b) n  j < 2n?  Distinguish carefully the cases in which  j and  n are even and odd.

≤

(a) j < n. The walk cannot circumscribe the circle. This case is the same as the walk on a line. Let  pj be the probability that the walk is at O at step j . Then by (3.6)

    n

 pn = vn,0 =

0

1 n 2

41

1 2n

n even n odd

Figure 3.2: The cyclic random walk of period n for Problem 4.17. (b) n

≤  j < 2n. Since n can b e reached in both clockwise and counterclockwise directions,  pj

=

=

vj,0 + vj,n + vj,−n

           − 0

1 + 2j

j 1  j 2

1 ( j 2

j 1 + 1  j n 2j 2( + )

j 1 + + n) 2j

j

1 ( j 2

j 1  j 2(

−



1 n) 2j

1 n) 2j

( j,n both even) ( j odd, n even, or j even, n odd) ( j,n both odd)

3.18. An unrestricted random walk with parameters p and  q starts from the origin, and lasts for  50 paces. Estimate the probability that the walk ends at 12 or more paces from the origin in the cases: (a) p = q = 12 ; (b) p = 0 .6, q = 0 .4. Consult Section 3.2. From (3.2) Z n =

Xn

−√ n( p − q) ≈ N (0, 1), 4npq

where Xn is the random variable of the position of the random walk at step n. Since n = 50 is discrete we use the approximation 11) P( 11.5 < X 50 < 11.5). P( 11 X50

− ≤

≤

≈ −

(a) Symmetric random walk: p = q = 12 . Then

−1.626 = − √1150.5 < Z 

50

=

√X50 < √1150.5 = 1 .626. 50

Hence P( 1.626 < Z 50 < 1.626) =

−Φ(−1.626) + Φ(1.626) = 2Φ(1.626) − 1 = 0 .896. Therefore the probability that the final position is 12 or more paces from the origin is 1 − 0.896 = 0.104 −

approximately. (b) p = 0 .6, q = 0 .4. The bounds on Z 50 are given by

−3.103 = −11√.548− 10 < Z 

50

=

− 10 < 11.√5 − 10 = 0 .217. √48 48

X50

Hence P( 3.103 < Z 50 < 0.217) = Φ(0.217)

− Φ(−3.103) = 0.585. The probability that the final position is 12 or more paces from the origin is 1 − 0.585 = 0.415. −

42

3.19. In an unrestricted random walk with parameters p and  q, for what value of  p are the mean and  variance of the probability distribution of the position of the walk at stage n the same?  From Section 3.2 the mean and variance of  Xn , the random variable of the position of the walk at step n, are given by E(Xn ) = n( p q ), V(Xn ) = 4 npq,

−

where q = 1

− p. The mean and variance are equal if 2 p − 1 = 4 p(1 − p), that is, if  4 p − 2 p − 1 = 0. √ The required probability is p = (1 + 5). 2

1 4

3.20. Two walkers each perform symmetric random walks with synchronized steps both starting from the origin at the same time. What is the probability that they are both at the origin at step n?  If  A and B are the walkers, then the probability an that A is at the origin is given by

     n

an =

0

1 2n

1 2n

(n even)

.

(n odd)

The probability bn for B is given by the same formula. They can only visit the origin if  n is even, in which case the probability that they are both there is an b n =

n 1 n 2

2

1 . 22n

3.21. A random walk takes place on a two-dimensional lattice as shown in Fig. 3.3. In the example shown the walk starts at  (0, 0) and ends at  (2, 1) after  13 steps. In this walk direct diagonal steps are not 

−

Figure 3.3: A two-dimensional random walk. permitted. We are interested the probability that, in the symmetric random walk, which starts at the origin, has returned there after  2n steps. Symmetry in the two-dimensional walk means that there is a probability of  1 that, at any position, the walk goes right, left, up, or down at the next step. The total number of different  4 walks of length  2n which start at the origin is 42n . For the walk considered, the number of right steps (positive x direction) must equal the number of left steps, and the number of steps up (positive y direction) must equal those down. Also the number of right steps must range from  0 to n, and the corresponding steps up from  n to 0. Explain why the probability that the walk returns to the origin after  2n steps is  p2n

(2n)! = 2n 4

n

 r=0

43

1 [r!(n

2

− r)!]

.

Prove the two identities

       

(2n)! = [r!(n r )!]2

2n n

−

2

n r

n

2n n

,

=

r=0

2

n r

.

[Hint: compare the coefficients of  xn in  (1 + x)2n and  [(1 + x)n ]2 .] Hence show that   p2n



1 2n = 2n n 4

2

.

Calculate p2 , p4 , 1/(πp 40 ), 1/(πp 80 ). How do you would you guess that  p2n behaves for large n?  At each intersection there are 4 possible paths. Hence there are 42n different paths which start the origin. For a walk which returns to the origin there must be r, say, left and right steps, and n r up and down steps (r = 0, 1, 2, . . . , n) to ensure the return. For fixed r, the number of ways in which r left, r right, (n r ) up and (n r) down steps can be chosen from 2 n is the multinomial formula

−

−

−

(2n)! r!r!(n r )!(n

−

− r)! .

For all  r , the total number of ways is n

 r=0

(2n)! r !r !(n r)!(n

−

− r)! .

Therefore the probability that a return to the origin occurs is (2n)! = 2n 4

 p2n

n

 r=0

1 [r!(n

2

− r)!]

.

For example if  n = 2, then 4!  p4 = 4 4

2

 r=0

1 r !(2

− r)!

4! 44

=



1 1 +1+ 2 2



=

3 . 16

For the first identity (2n)! (2n)! = n!n! [r!(n r )!]2

−



n! r!(n r )!

−

    2

2n n

=

n r

2

.

For the second identity

 2n n

=

the coefficient of xn in the expansion of (1 + x)2n

=

the coefficient of xn in the expansion of [(1 + x)n ]2

= = =

     ···          · · ·        · · · −       −    n

the coefficient of x in n 0 n 0

n

=

r=0

n n

+

+

n 1

2

n r

n 1

2

+

1+

n

n

+

1

n n

n x+ 1

+

2

2

44

since

n 2 x + 2

n n

n r

+

n 0

=

n

n

r

n n x n

2

Hence



2 2n = 2n 4 n

 p2n

2

.

The computed values are p20 = 0 .01572 and p40 = 0 .00791. Then 1 = 20 .25, πp 20

1 = 40 .25, πp 40

which imply that possibly p2n

∼ 1/(nπ) as n → ∞. A random walk takes place on the positions {. . . , −2, −1, 0, 1, 2, . . .}.

The walk starts at 0. At step 3.22. n, the walker has a probability  qn of advancing one position, or a probability  1 qn of retreating one step (note that the probability depends on the step not the position of the walker). Find the expected position of  ∞ r is convergent, then  the walker at step n. Show that if  qn = 12 + rn , ( 12 < rn < 12 ), and the series j=1 j the expected position of the walk will remain finite as n .

−

−



→∞

If  Xn is the random variable representing the position of the walker at step n, then P(Xn+1 = j + 1 Xn = j ) = qn ,

P(Xn+1 = j

|

− 1|X

n

= j) = 1

−q

n.

If  W i is the modified Bernoulli random variable (Section 3.2), then n

E(Xn )

=

n

  − W i =

E

i=1 n

=

2

n

E(W i ) =

i=1

qi



[1.qi + ( 1)(1

−

i=1

− q )] i

n.

i=1

Let qn =

1 2

+ rn , (

−

1 2

< r n < 12 ). Then n

E(Xn ) = 2



n

( 12

+ ri )

−n=

i=1

Hence E(Xn ) is finite as n



ri .

i=1

→ ∞ if the series on the right is convergent.

3.23. A symmetric random walk starts at  k on the positions 0, 1, 2, . . . , a, where 0 < k < a. As,in the gambler’s ruin problem, the walk stops whenever  0 or  a is first reached. Show that the expected number of  visits to position  j where 0 < j < k is 2 j (a k)/a before the walk stops.

−

One approach is to this problem by repeated application of result (2.5) for gamnler’s ruin. A walk which starts at k first reached j before a with probability (by (2.5))  p =

(a

− j ) − (k − j ) = a − k . a − j a − j

A walk which starts at j reaches a (and stops) before reaching j (again by (2.5)) with probability 1 1 1 , = a  j a  j ) 2 2(

−

−

and reaches 0 before returning to j with probability  j

− ( j − 1) = 2 j

1 . 2 j

Hence the probability that the walk from j stops without returning to j with probability q=

1 2(a

− j )

+

1 a = . 2 j 2 j (a  j )

45

−

Given that the walk is at j , it nexts visits j with probability r =1

− q = 1 − 2 j (aa− j ) = 2 j2( ja(−a  j−) j−) a .

Therefore the probability that the walk starts from k visits j m times before stopping is hm = pr m−1 q.

The expected number of visits to j is µ=

∞



mhm = pq

m=1

∞



mrm−1 =

m=1

pq , (1 r)2

−

summing the quasi-geometric series. Substituting for p, q, and r, µ

=

2

=





2 j (a − j ) − a −k · a 1− − j 2 j (a − j ) 2 j (a − j ) 2(a − k) j a(a − k) 4 j (a − j ) · = 2 j (a − j ) a a a a

2

2

46

2

−2

Chapter 4

Markov chains 4.1. If  T  = [ pij ], (i, j = 1 , 2, 3) and 

i + j , 6 + 3i show that  T  is a row-stochastic matrix. What is the probability that a transition between states E 2 and  E 3 occurs at any step?  If the initial probability distribution in a Markov chain is  pij =

(0)

=

p



1 2

1 4



1 4

,

what are the probabilities that states E 1 , E 2 and  E 3 are occupied after one step. Explain why the probability  that the chain finishes in state E 2 is 13 irrespective of the number of steps. Since  pij =

then

3

3

   pij =

j=1

j=1

i + j , 6 + 3i

i + j 3i 6 = + = 1, 6 + 3i 6 + 3 i 6 + 3i

for all i. Also 0 < pi,j < 1. Therefore T  is a stochastic matrix. The probability that a transition from E 2 5 to E 3 occurs is p23 = 12 . The probabilities that the states E 1 , E 2 and E 3 are occupied after one step given p(0) are given by (1)

p

= p(0) T  =



1 2

1 4

 

1 4

2 9 1 4 4 15

1 3 1 3 1 3

4 9 5 12 2 5

  =

173 720

1 3

307 720



Each term in the second column of  T  is a 13 . By row-on-column matrix multiplication, each element in the second column of  T n is a 13 . Hence the second term in p(0) T n is 13 independently of  p(0) . 4.2. If  T  = (2) (2) calculate p(2) 22 , p31 and  p13 .

 

1 2 1 3 1 4

1 4 1 3 1 2

1 4 1 3 1 4

 

,

(2)

 pij are the elements of  T 2 . The matrix multiplication gives T 2 =

 

19 48 13 36 17 48

1 3 13 36 17 48

47

13 48 5 18 7 24

 

.

Therefore

13 , 36

 p(2) 22 =

p(2) 31 =

4.3. For the transition matrix 

17 , 48

p(2) 13 =

13 . 48

    1 3 1 4

T  = (2) calculate p(3) and  p(3) given that  p(0) = 12 , p2  formula for  T n and obtain  limn→∞ T n .

1 2

2 3 3 4

1 2

. Also find the eigenvalues of  T , construct a 

We require 2

T  =

   1 3 1 4

2

2 3 3 4

5 18 13 48

=



13 18 35 48

3

T  =

,

   1 3 1 4

3

2 3 3 4

59 216 157 576

=

157 216 419 576



.

(3)

Directly from T 3 , p12 = 157 216 . The element can be read off from (2)

(0)

=p

p

2

(3)

1 2

T  =

209 namely p(2) 2 = 288 . The vector

p

   

= p(0) T 3 =

5 18 13 48

1 2

1 2

1 2

13 18 35 48

 

59 216 157 576

157 216 419 576

or,

12λ2

=

79 209 , 288 288

=

943 3456



,

2513 3456



The eigenvalues of  T  are given by



1 3

−λ 1 4

The eigenvalues are λ1 = 1, λ2 =

3 4

1 . 12

2 3

−λ



= 0,

− 13λ + 1 = 0.

Corresponding eigenvectors are given by the transposes

r1 =

The matrix C  is defined as

  −    − 1

C  =

By (4.18),

1

T 

r1

,

r2 =

8 3

1 1

8 3

=

r2

T 

1

.

1

T n = CD n C −1 ,

where D is the diagonal matrix of eigenvalues. Therefore n

T  =



1 1

−

8 3

1



1

0

0

1 12n



3 11 3 11

−

It follows that lim T n =

n

→∞

1 11

    8 11 3 11

3 3

1 = 11

8 8

3+ 3

−

8 12n 3 12n

8

−

8+



8 12n 3 12n

.

.

4.4. Sketch transition diagrams for each of the following three-state Markov chains. (a) A =



1 3

0 1

1 3

0 0

1 3

1 0



;

(b) B =



1 2

1 4

0

1

1 2

1 2

48

1 4

0 0



;

(c) C  =



1 2

1 2

0 1

0

0

1 3

1 3

1 3



.

(a)

(b) 1 2

E 1

E 1

1 2

1 4

1 2 1 4

1 2

1

1 2

E 2

1

E 3

E 3

1 2

E 2

1 E 1

(c) 1 2

1 2

1

1 3

E 3

E 2

1 3

1 3

Figure 4.1: Transition diagrams for Problem 4.4 The transition diagrams are shown in Figure 4.1. 4.5. Find the eigenvalues of  T  =



a c b

b a c



c b a

(a > 0, b > 0, c > 0).

,

Show that the eigenvalues are complex if  b = c. (If  a + b + c = 1, then  T  is a  doubly- stochastic matrix.) Find the eigenvalues and eigenvectors in the following cases: (a) a = 12 , b = 14 , c = 14 ;



(b) a = 12 , b = 18 , c = 38 . The eigenvalues are given by

or (a + b + c The eigenvalues are

2

− λ)(λ

 

a

−λ

b

c b

−λ c

+ (b + c

λ1 = a + b + c,

a

c b a

− 2a)λ + a λ2,3

2

−λ

 

= 0,

+ b2 + c2

− bc − ca − ab) = 0 . √ 1 = (2a − b − c ± i 3|b − c|). 2

(a) a = 12 , b = 14 , c = 14 . The eigenvalues are λ1 = 1, λ2 = λ3 = 14 . The eigenvectors are r1 =

  1 1 1

,

r2 =

−  1 1 0

,

r3 =

−  1 0 1

.

(b) a = 14 , b = 18 , c = 38 . The eigenvectors are r1 =

  1 1 1

,

r2 =

√ 

− −  −  1 2 1 2

3 i √  2 + i 23 1

,

r3 =

√ 

−  − −  1 2 1 2

3 + i √  2 i 23 1

.

4.6. Find the eigenvalues, eigenvectors, the matrix of eigenvectors C , its inverse C −1 , a formula for  T n and  limn→∞ T n  for each of the following transition matrices;

49

(a)

  1 8 1 2

T  =

(b)



T  =

(a) The eigenvalues are λ1 = 1, λ2 =

−

r1 =

3 8.

7 8 1 2

1 2 1 4 1 4

1 8 3 8 5 8

;



3 8 3 8 1 8

;

The corresponding eigenvectors are

  1 1

r2 =

− 



C −1 =



1 0



,

7 4

1

.

The matrix C  and its inverse are given by C  =

The matrix T n is given by n



r1

  =

r2

n

= CD C −1 =

T 

=

1 11

as n . (b) The eigenvalues are λ1 =

→∞



−

1

−

7 4

−

7 4

1

1 1

−) 4 − (− )

1 4,

7

r2 =

,



r1

r2

=

CD n C −1 =

=

1 3

− −

+ 13 21−2n 1 3 1 3

− −

4n 3 4−n 3

1 11

−  2 1 1

=

r3 =

,

3 7 3 7

−2

1

1

1

1

1

1

3 7 3 7

1

 

3 n 8 3 n 8

   −−  − 

r3

Then T n

4 11 4 11

.

7 11 4 11

 → − 

− 7(− ) 7 + (− )

1

C  =

3 8



7 11 4 11

− n

0

−

4 11 4 11

4

7

4

7



λ2 = 14 , λ3 = 1, and the corresponding eigenvectors are 3 7 3 7

The matrix C  is given by

,



1

3 n 8 3 n 8

4 + 7(

−  − 

r1 =

1

−2

1

1

1

1

1

1 n ) 4

(

0 0

n 1 2n 11 3 30 + 5 ( 1) 2 11 + 35 ( 1)n 2 1 2n 30 n 1 2n 11 7 30 5 ( 1) 2

− − − −

so that lim T n =

t

→∞

−−

−−

1 3 1 3 1 3

1 1 2n 32 −n + 43 −n + 43

−

−

−−

 

0 ( 14 )n 0

11 30 11 30 11 30

3 10 3 10 3 10

 

0 0 1 3 10 3 10 3 10

  1 1 1

   

3 5 3 5 7 5

0

7 7 10 10 1 1 3 11 3 30 10 1 2n

−

1 3 1 3

−

n

2− −

n

2−1−2n

n

2−1−2n

− (−1) − (−1) + (−1)

 

  ,

.

4.7. The weather in a certain region can be characterized as being sunny(S), cloudy(C) or rainy(R) on  any particular day. The probability of any type of weather on one day depends only on the state of the

50

weather on the previous day. For example, if it is sunny one day then sun or clouds are equally likely on  the next day with no possibility of rain. Explain what other the day-to-day possibilities are if the weather  is represented by the transition matrix. S  C  R

T  =

S

C

1 2 1 2

1 2 1 4 1 2

0

R 0 1 4 1 2

Find the eigenvalues of  T  and a formula for  T n . In the long run what percentage of the days are sunny, cloudy and rainy?  The eigenvalues of  T  are given by

Let λ1 =

−

1 4,

λ2 =

1 2

 

1 2

−λ 1 2

1 4

0

1 2

0

−λ 1 2

1 4

1 2

−λ

 

=

− 18 (4λ + 1)(2λ − 1)(λ − 1) = 0.

and λ3 = 1. The coresponding eigenvectors are

  −  1

r1 =

3 2

r2 =

,

1

−  1 2

0 1

r3 =

,

  1 1 1

The matrix of eigenvectors C  is given by

  −  − 

1

C  =



r1

r2

=

r3

−

3 2

1 2

1

0

1

1

1

0 1 n (2) 0

0 0 1

1

If  D is the diagonal matrix of eigenvalues, then by (4.18)

 −

1

T n

=

CD n C −1 =

3 2

−

1

As n

→∞

 → −

1

T n

3 2

1

−

1 2

0

1

1 1 1

 

0 0 0

1 2

1

0

1

1

1

0 0 0

1 n 4)

(

0 0

 

0 0 1

4 15 2 3 2 5

−

2 5

0

−

2 5

2 15 2 3 1 5

   

4 15 2 3 2 5

−

   1 = 5

2 2 2

−

2 5

0 2 5

2 2 2

 

2 15 2 3 1 5

1 1 1



.

In the long run 40% of the days are of the days are sunny, 40% are cloudy and 20% are rainy. 4.8. The eigenvalue method of Section 4.4 for finding general powers of stochastic matrices is only guaranteed to work if the eigenvalues are distinct. Several possibilities occur if the stochastic matrix of a Markov  chain has a repeated eigenvalue. The fol lowing three examples illustrate these possibilities. (a) Let  T  =



1 4

1 4

1 2

1

0

0

1 2

1 4

1 4



be the transition matrix of a three-state Markov chain. Show that  T  has the repeated eigenvalue λ1 = λ2 = 1 and  λ3 = 1, and two distinct eigenvectors 4

−

r1 =

  1 4 1

r3 =

−

51

  1 1 1

.

In this case diagonalization of  T  is not possible. However it is possible to find a non-singular matrix  C  such that  T  = CJ C −1 , where J  is the Jordan decomposition matrix given by  J  =



1 λ1 0

λ1 0 0

C  =

and  r2 satisfies (T  Show that we can choose



T n

r2

− λ I  )r 1 3

1 4

1

0 0

=

r1

r2 =

Find a formula for  J n and confirm that, as n

 −

0 0 1

−

1 4

0



r3

2

0 0 1



,

,

= r1 .

−  10 24 0

.

1 5 1 5 1 5

8 25 8 25 8 25

→ ∞,

 →  

12 25 12 25 12 25

   

(b) A four-state Markov chain has the transition matrix  1 3 4

S  =

0 0

0 0

0

1 4

0 0

0 0

1 4

0

3 4

1

.

.

Sketch the transition diagram for the chain, and note that the chain has two absorbing states and is therefore not a regular chain. Show that the eigenvalues of  S  are 14 , 14 and  1 repeated. Show that there are four  distinct eigenvectors. Choose the diagonalizing matrix  C  as

−

  − →∞  → 

0 1 1 0

C  =

Find its inverse, and show that, as n

0 1 1 0

0 1

1 0

,

S n

1

0 0 0 0

4 5 1 5

0

   

−4 −5 −3 4

0 0 0 0

0 1 5 4 5

.

.

1

Note that since the rows are not the same this chain does not have an invariant distribution: this is caused  by the presence of two absorbing states. (c) Show that the transition matrix  1 0 12 2 1 1 1 U  = 6 3 2 1 0 56 6





has a repeated eigenvalue, but that, in this case, three independent eigenvectors can be associated with  U . Find a diagonalizing matrix  C , and find a formula for  U n using  U n = CD n C −1 , where D=



1 3

0

0 0

52

1 3

0

0 0 1



.

Confirm also that this chain has an invariant distribution. (a) The eigenvalues are given by

Hence they are λ1 =

1 4

−

 

1 4

1 4

−λ 1

0

−λ

1 2

 

1 2

1 4

1 4

−λ

=

− 161 (λ − 1)(1 + 4λ)

2

(repeated) and λ2 = 1. This leads to the two eigenvectors r1 =

  1 4 1

−

  1 1 1

r3 =

,

The Jordan decomposition matrix is given by

−

J  =

− λ I  ]r 1 3

2

= r1

1

0 0

Let r2 satisfy [T 

1 4

−

or

1 4

0

 



0 0 1

1 2

1 4 1 4 1 4

1 1 2

.

    − 

1 2

1

0

4

r2 =

1 2

.

1

The solution for the linear equations for the components of  r2 gives r2 =

The matrix C  is defined in the usual way as C  =

Its computed inverse is



r1

−

r2

r3

10

24



0

 −

1 4 1

=

T 

.

−10

1 1 1

24 0

− −  −   − −   −   −   −    − −  −  12 25 1 10 12 25

C −1 =

1 5

17 25 1 10 8 25

0 1 5



.

.

If  T  = CJ C −1 , then T n = CJ n C −1 , where

−   → −  

1 4

J n =

0 0

As n

→ ∞,

1

−

T n

0

24

1

0

12 25 12 25 12 25

1 5 1 5 1 5

1 n ) 4

(

=

0

1

−10

4

=

1 4

0

1

n

0

n(

(

0

1

0

0

0

1

0

0

0

1

0

0

1

8 25 8 25 8 25

1 n 1 ) 4 1 n 4)

0

0

1

−

12 25 1 10 12 25

0

−

1 5

0 1 5

 

.

17 25 1 10 8 25

 

.

(b) The transition diagram is shown in Figure 4.2. The eigenvectors are given by

 

1

−λ 3 4

0 0

0 λ

−

1 4

0

0

0 0

1 4

−λ 0

1

3 4

=

1 (λ 16

λ

53

1)2 (4λ

1)(4λ + 1) = 0 .

3 4

E 1

E 2

1 1 4

1 4

E 3 1

E 4

3 4

Figure 4.2: Transition diagram for Problem 4.8(b). The eigenvalues are λ1 =

−

1 , 4

r1 =

λ2 = 14 , λ3 = 1 (repeated). The eigenvectors for λ1 and λ2 are



0

−1

1



T 

0

Let the eigenvector for the repeated λ3 be r3 =

where the constants a,b,c,d satisfy 3 a 4

1 c 4

−b+



r2 =

,

a

b

c

d

1 b 4

= 0,

 

0

T 

−c+

1

1



T 

0

.

,

3 d 4

= 0.

We can express the solution in the form c=

−3a + 4b,

Hence the eigenvector is r=

d = 12 a

   −  − a b 3a + 4b 4a 5b

− 15b.

,

which contains two arbitrary constants a and b. In this case of a repeated eigenvalue, two eigenvectors can be defined using different pairs of values for r3 =

−

4

−3

0

1

The matrix C  and its inverse are

C  =

  −

0 1 1 0

0 1 1 0

−4 −3 0 1

5 4 1 0

The matrix power of  T  is given by T n

= =

CD n C −1 0 1 1 1 10 0

  → 

  −

1 4 5 1 5

0

0 1 1 0

0 0

0 0

0 0

0 0

−4 −3 0 1

0 1 5 4 5

1

 

5 4 1 0

 



T 

,

r4 =

C −1 =

,

 −   (

1 n 4)

0 0 0

54

1 10

0 ( 14 )n 0 0

   −

4

0 0 1 0

0 0 0 1

5

3 5 0 2

1

0

−5

5 5 0 0

5 0 0

   −



3 5 0 2

T 

.

−3 −5 1 8

−5 5 0 0

 

.

5 5 0 0

−3 −5 1 8

 

as n (c) The eigenvaues are given by

→∞

 

1 2

Hence the eigenvalues are λ1 = obtain the eigenvector

1 6 1 6

1 3

 − − − − −  −  −    1 2 1 2

0

−λ

1 3

λ

5 6

0

1 (λ 9

=

λ

1)(3λ

1)2 .

(repeated) and λ3 = 1. Corresponding to the eigenvalue λ1 , we can

r1 =

3b a b

3 0 1

=b

0 1 0

+a

,

where a and b are arbitrary. Two distinct eigenvectors can be obtained by putting a = 0, b = 1 and by putting a = 1, b = 0. The three eigenvectors are r1 =

−  3 0 1

r2 =

,

  0 1 0

r3 =

,

 

.

0 4 0



1 1 1

The matrix C  and its inverse become C  =

−

3 0 1

0 1 0

1 1 1



,

With D=

− − 

1 1 1

1 C −1 = 4



1 3

0 1 3

0 0

0

0 0 1

1 3 3

−

.

,

then U n

− − →   =

=

as n

→ ∞.

3 0 1

0 1 0

1 1 1

3 0 1

0 1 0

1 1 1

1 4 1 4 1 4

0 0 0

3 4 3 4 3 4

   

( 13 )n 0 0

0 1 n (3) 0

0 0 0

0 0 1

0 0 0

0 0 1

 − 1 4

 −

1 1 1

1 4

1 1 1

−

0 4 0

−

0 4 0

1 3 3

−

1 3 3

−





,

4.9. Miscellaneous problems on transition matrices. In each case find the eigenvalues of  T , a formula for  . The special cases discussed in Problem 4.8 can occur. T n and the limit of  T n as n (a)

→∞

T  =

(b) T  =

(c) T  =

  

1 2

7 32

9 32

1 2

1 4

1 4

1 3

1

1 4

0

5 12

1 4

1 4

1 2

1 4 3 4 1 4

3 16

1

0

0 1 4

55

0

0

9 16 1 4 1 2

  

;

;

;

(d)



T  =

(e)

 

T  =

(a) The eigenvalues of  T  are λ1 =

−

1 8

r1 =

1 4 5 12 1 2

1 1 2

0 0

1 4 1 3 1 4



1 2 1 4 1 4

0 0 0

0 0 1

1 2

1 2

;

 

0 1 2

0 0

.

(repeated) and λ3 = 1, with the corresponding eigenvectors

  1 4

−2

r2 =

,

1

  1 1 1

.

The Jordan decomposition matrix J  is required, where

−

1 8

1

0 0

J  =

and r2 is given by

1 8

−

0

− λ I  ]r 1 3

= r1

2

7 32 1 8 1 4

1

or

1 2

The remaining eigenvector is r2 =

,

      −  −  5 8

[T 



0 0 1

3 8

9 32

1 4

0

2

r2 =

3 8

1

.

4 3

The matrix C  and its inverse are given by

 −

1 4

C  =

−3

2

1

Then

 −

1 4

T n =

2

1

−3

1

8

1

4 3

1

1

4 3

1

(

C −1 =

,

 − 

1

8

 

1 n ) 8

1

0

(

−

0

Matrix multiplcation gives

lim T n =

n

→∞

1 n ) 8

16 27 16 27 16 27

10 27 1 6 16 27

13 54 1 24 5 27

0

1

5 27 5 27 5 27

2 9 2 9 2 9

11 18 1 8 2 9

10 27 1 6 16 27

0

0

 

− −  −   − −   −    13 54 1 24 5 27

11 18 1 8 2 9

.

(b) The eigenvalues of  T  are λ1 = 14 , λ2 = 16 , and λ3 = 1, which are all different, so that the calculations are straightforward. The eigenvectors are given by

−

  − 

−

1

r1 =

4

C  =

4

1

5 12 5 2

1

1

1

−

1

,

r3 =

1

1

The matrix C  and its inverse are 1

5 12 5 2

r2 =

,

1

 −

    −      − −  − 

1

,

C −1 =

56

1

6 5 12 7 18 35

1 5

0 1 5

1

12 7 2 7

Finally

 −

5 12 5 2

1

lim T n =

n

→∞

4

1

 

1 1

−

1

1

(c) The eigenvalues are given by

 

1 4 3 4 1 4

3 16

0

0

0

0

0

0

0

9 16 1 4 1 2

0 1 4

0

 

r1 =

− −

3 26 31 13

+

−

1

  −

15 26 i 14 i 13

D=

−

−

1 8 (1

r2 =

1 (1 8

+ i)

0 0

−

3 26 31 13

−

+ 1

0 1 (1 8 0

−1 12 7 2 7

1 5

− i), and λ

− −

1 5

0

− 321 (λ − 1)(32λ

=

,

The diagonal matrix of eigenvalues is

6 5 12 7 18 35

1

Hence the eigenvalues are λ1 = 18 (1+i), λ2 = eigenvalues. The corresponding eigenvectors are

−

  −

2

 

18 35 18 35 18 35

=

1 5 1 5 1 5

2 7 2 7 2 7

 

.

+ 8λ + 1). = 1. This stochastic matrix has complex

3

 

15 26 i 14 i 13

0 0 1

− i)

r3 =



  1 1 1

.

.

After some algebra (easily computed using software) lim T n =

n

→∞

(d) The eigenvalues are given by λ1 = r1 =

−

1 , 4

 

14 41 14 41 14 41

λ2 =

15 82 15 82 15 82

1 , 12

1

r2 =

,

1

8 3

1

The matrix C  and the diagonal matrix D are given by C  =

11 9 4 9

1

8 3

1

It follows that

1

1 1 1

    −    r3 =

1 4

21 55 21 55 21 55

lim T n =

n

→∞

3 11 3 11 3 11

0

1 12

0 0

D=

,

.

and λ3 = 1. The corresponding eigenvectors are

−     − −  −   11 9 4 9

 

39 82 39 82 39 82

19 55 19 55 19 55

0

1 1 1

.

0 0 1

.

.

(e) The eigenvalues of  T  are λ1 = 12 , λ1 = 12 , λ3 = 1 (repeated). There is a repeated eigenvalue but we can still find four eigenvectors given by

−

   −    −

r1 =

0 1 0 1

    −  − 

r2 =

,

0 1 0 1

r3 =

,

  −     −   − −   − −  3 2 0 1

,

2 1 1 0

r4 =

The matrix C  and its inverse can be compiled from these eigenvectors:

C  =

0 1 0 1

0 1 0 1

3 2 0 1

2 1 1 0

,

C −1 =

1 6 1 2 1 3

0

57

1 2 1 2

0 0

1 6 1 2 2 3

1

1 2 1 2

0 0

.

The diagonal matrix D in this case is given by

D=

Hence T n

=

CD n C −1 0 0 1 1 0 0 1 1

3 2 0 1

−2 −1

0 1 0 1

3 2 0 1

2 1 1 0

  −  →  −   =

1 2 3

=

0 1 3

0 1 0 1 0 0 0 0

0 0 1

− 

1 2

0 1 2

0 0 0

 −   −  −     (

0 0

1 n 2)

0 0 0 0

0 0 0 0

0 0 1 0

0 0

2 3

0 0 1 0

.

0 0 0 1

   −

1 6 1 2 1 3

0 0 0 1

   −

1 6 1 2 1 3

1 2 1 2

1 2 1 2

−

0 0

1 6 1 2 2 3

−

− −

0 0

0

0

0 1 3

 

0 0 0 1

0 ( 12 )n 0 0

0 0 0

1 0

0 0 1 0

1 6 1 2 2 3

− −

1

1 1 2 1 2

0 0

 

1 2 1 2

0 0

 

.

4.10. A four-state Markov chain has the transition matrix 

T  =

 

1 2

1 2

1

0

1 4 3 4

1 2

0

0 0 0

0 0

1 4

0

1 4

 

.

Find  f i , the probability that the chain returns at some step to state E i , for each state. Determine which  states are transient and which are persistent. Which states form a closed subset? Find the eigenvalues of  . T , and the limiting behaviour of  T n as n

→∞

The transition diagram for the chain is shown in Figure 4.3. For each state, the probability that a first return occurs is as follows, using the diagram: (1)

State E 1 : f 1

(n)

= 12 ,f 1 (2) = 12 , f 1

= 0, (n

≥ 3); 1

1 2

E 2

E 1

1 2

1

3 4

1 2

4

1 4

E 4

E 3 1 1 4

Figure 4.3: Transition diagram for Problem 4.10. (1)

State E 2 : f 2

(2)

= 0, f 2

(n)

= 12 , f 2

= 1 /2n−1 (n

≥ 3); 58

(1)

State E 3 : f 3

(2)

= 0, f 3

=

1 , 42

(n)

= 1 /4n , (n

f 3

≥ 3):

State E 4 : f 4(n) = f 3(n) for all n. The probability of a return at any stage is f n =

∞



f r(n) ,

r=1

for each n. Therefore

1 1 1 + 2 + 3 + = 1, 2 2 2 1 1 1 f 3 = f 4 = 2 + 3 + , = 4 4 12 summing geometric series for f 2 and f 3 . Hence E 1 and E 2 are persistent states , but E 3 and E 4 are transient. The eigenvalues pf  T  are λ1 = 12 , λ2 = 14 , λ3 = 14 , and λ4 = 1. The corresponding eigenvectors are f 1 =

1 1 + = 1, 2 2

f 2 =

···

·· ·

−

r1 =

−

−   −    −  − −  −  −− 1 3 2 3

,

1 1

   −  −  − −      0 0 1 1

r2 =

,

r3 =

Therefore D , C  and its inverse are given by

D=

− 

1 2

0 0

0

−

0

Hence

lim T n

n

→∞

1 4

0 0

=

0

0

0

0

1 4

0

0

1

,

1 3 2 3

2 3 2 3 2 3 2 3

0

1

0

0

1

1

1

1

1

1

1

1

0

1

0

0

0

0

0

1

0

0

0

1

1

0

0

0

1

1

0

0

0

1

1

−

1

0

0

1

1

=

C  =

1 3 2 3

1 3 1 3 1 3 1 3

0 0 0 0

0

         −   − −− −  −    − −− −  0 0 1 1

,

,

r4 =

1

0

1

0

2 3 2 3

1

.

1

1

1

C −1 =

0

1 1 1 1

1

2 3 2 3

1

1

1 3 1 3

1 3 1 3

0

0

1 2 1 2

1 2 1 2

0

0

0

0

1 2 1 2

1 2 1 2

0

0

 

0 0 0

4.11. A six-state Markov chain has the transition matrix 

T  =

  

1 4

0 0 0 0 0 0

1 2

0 1 4

0 0 0 0

0 0 0 0 0 0 1

0 0

0 0

1 4

1 2

0

1

1 2 1 2

1 2 1 2

0

0

1 4

1 0 0 0 0 0

  

.

Sketch its transition diagram. From the diagram which states do you think are transient and which do you think are persistent? Which states form a closed subset? Determine the invariant distribution in the subset. Intuitively E 1 , E 2 , E 3 and E 6 are transient since paths can always escape through E 3 and not return.

59

1 2

E 6

E 5 1 2

1 4

1 4

1 2

1

E 1

1

E 4

1

1

1 2

4

E 2

E 3

1 4

Figure 4.4: Transition diagram for Problem 4.11. For state E 4 , the probabilities of first returns are (1)

f 4

(n)

= 0,

f 4

1 , 2n−1

=

(n = 2 , 3, 4, . . .).

It follows that a return to E 4 occurs at some step is f 4 =

∞

∞

  (n) f 4

=

n=1

n=2

1

− = 1.

2n 1

Hence E 4 is persistent. For E 5 , (1)

f 5

=

1 , 2

(2)

f 5

=

1 , 2

(n)

f 5

= 0,

(n

≥ 3),

so that f 5 = 12 + 12 = 1. Hence E 5 is also persistent. The states E 1 , E 2 form a closed subset since no escape paths occur. The subset has the transition matrix 0 1 S  = . 1 1

  2

In the notation of Section 4.3, α = 1 and β  =

1 . 2

2

Hence the invariant distribution is

  1 3

2 3

.

4.12. Draw the transition diagram for the seven-state Markov chain with transition matrix 

T  =

  

0 0 1 2

0 0 1 2

0

1 0 0 0 0 0 0

0 1 0 0 0 0 0

0 0 1 2

0 0 0 0

0 0 0 1 0 0 0

0 0 0 0 1 0 0

0 0 0 0 0 1 2

1

  

.

(n)

(n)

Hence discuss the periodicity of the states of the chain. From the transition diagram calculate p11 and  p44 (3) (3) (3n)  for  n = 2 , 3, 4, 5, 6. (In this example you should confirm that  p11 = 12 but that  p44 = 0: however, p44 = 0  for  n = 2 , 3, . . . confirming that state E 4 is periodic with period  3.)



Consider state E 1 . Returns to E 1 can occur in the sequence E 1 E 2 E 3 E 4 which takes 3 steps, or as E 1 E 2 E 3 E 4 E 5 E 6 E 1 which takes 6 steps which is a multiple of 3. Hence returns to E 1 can only occur at steps 3, 6, 9, . . .: hence E 1 has periodicity 3. Similarly E 2 and E 3 also have periodicity 3. On the other hand for E 4 returns are possible at steps 6 , 9, 12, . . . but it still has periodicity. The same is true of states E 5 and E 6 .

60

E 1

E 2

1

E 3

1

E 4

1 2

E 5

1

E 6

1

E 7

1 2

1 2

1

1 2

Figure 4.5: Transition diagram for Problem 4.12. E 7 is an absorbing state.

4.13. The transition matrix of a 3-state Markov chain is given by 

T  =

 

0

3 4

1 2 3 4

0

1 4 1 2

1 4

0

 

.

Show that  S  = T 2 is the transition matrix of a regular chain. Find its eigenvectors and confirm that  S  has 13 10 an invariant distribution given by  14 for even steps in the chain. 37 37 37



The matrix S  is given by

S  = T 2 =

  

9 16 3 8 1 8

1 16 1 2 9 16

 

3 8 1 8 5 16

,

which is regular since all elements are non-zero. The eigenvalues of  S  are given by λ1 = E 1

3 4

1 4 3 4

1 2

1 2

E 2

E 3

1 4

Figure 4.6: Transition diagram for Problem 4.13. λ2 =

3 16

+ 14 i, λ3 = 1, with corresponding eigenvectors r1 =

− −

16 25 16 25

+

−

13 25 i 13 i 25

1

 

r2 =

,

− −

2 25 2 25

14 25 i 14 i 25

−

+ 1

The matrix C  is given by the matrix of eigenvectors, namely

C  =

Let

− − 

D=

16 25 16 25

+

−

13 25 i 13 25 i

1 3 16

− 0 0

2 25 2 25

− − − +

14 25 i 14 25 i

1

1 4i

61

3 16

0 + 14 i 0

 

1

r3 =

,

 

1 1



1 1

1 0 0 1

   

.

.

.

3 16

−

1 i, 4

Finally (computation simplifies the algebra) CDC −1 =

which gives the limiting distribution.

 

14 37 14 37 14 37

13 37 13 37 13 37

 

10 37 10 37 10 37

,

4.14. An insect is placed in the maze of cells shown in Figure 4.9. The state E j is the state in which the insect is in cell  j . A transition occurs when the insect moves from one cell to another. Assuming that exits are equally likely to be chosen where there is a choice, construct the transition matrix  T  for the Markov  chain representing the movements of the insect. Show that all states are periodic with period  2. Show that  T 2 has two subchains which are both regular. Find the invariant distributions of both subchains. Interpret  the results. If the insect the insect starts in any compartment (state), then it can only return to that compartment after an even number of steps. Hence all states are periodic with period 2. 1

E 4

E 2

1 2 1 2

E 2

E 4

1 2

E 1

E 1 1 2 1 2

1 2

E 5

E 3

1

E 5

E 3 1 2

Figure 4.7: Transition diagram for Problem 4.13, and the maze. The matrix

2

S  = T  =

  

1 2 1 2

1 4 1 2

0

0

1 4

0

0

0

0

0 0

1 4 3 4

0

0

3 4 1 4

1 2

0

0

0

1 2

0

  

has two subchains corresponding to E 1 , E 2 , E 5 and E 3 , E 4 : this follows since the zeros in columns 3 and 4, and the zeros in rows 3 and 4, remain for all powers of  S . The subchains have the transition matrices

T 1 =

 

1 2 1 2 1 2

1 4 1 2

1 4

0

0

1 2

 

T 2 =

,

  3 4 1 4

1 4 3 4

.

The eigenvalues of  T 1 are λ1 = 0, λ2 = 12 , λ3 = 1, with the corresponding eigenvectors r1 =

−  1 1 1

,

r2 =

  0 1 1

−

r3 =

,

  1 1 1

.

The matrix C 1 and its inverse, and the diagonal D1 are given by C 1 =

−

1 1 1

0 1 1

−

1 1 1



,

C 1−1 =

−

1 2

0 1 2

62

1 4 1 2 1 4

−

1 4 1 2 1 4



,

D=



0 0 0

0 1 2

0

0 0 1



Therefore T 1n

C 1 D1n C 1−1

=

−  

1

=

1 1

1

−1

1

1

1 2 1 2 1 2

→

0

1 4 1 4 1 4

 

1



1 4 1 4 1 4

0

0

0

0

( 12 )n

0

0

0

1

 − 

1 2

0 1 2

1 4 1 2 1 4

 

1 4 1 2 1 4

−

,

as n . The eigenvalues of  T 2 are λ1 = 12 , λ2 = 1, and the corresponding eigenvectors are

→∞

r1 =

−  1

1

r2 =

,

  1

.

1

The matrix C 2 and its inverse, and the diagonal matric D2 are given by C 2 =

Hence

−  −  −   −  −  −  →   1

1

1

C 2−1 =

,

1

1

T 2n = C 2 D2n C 2−1 =

1

1

(

1

1 2 1 2

1 2 1 2

1 n 2)

0

0

1

D2 =

,

1 2 1 2

1 2

0

0

1 2 1 2

.

1

1 2 1 2

1 2 1 2

,

as n . Combination of the two limiting matrices leads to

→∞

lim T n =

n

→∞

  

1 2 1 2

0 0

0 0

0 0

1 4 1 4

0 0

1 2

1 4

1 2 1 2

0

1 2 1 2

0

1 4 1 4

0 0 1 4

  

.

4.15. The transition matrix of a four-state Markov chain is given by 

T  =

 

1 1 1

−a −b −c 1

a 0 0 0

0 b 0 0

0 0 c 0

 

,

(0 < a, b,c < 1).

Draw a transition diagram, and, from the diagram, calculate f 1(n) , (n = 1 , 2, . . .), the probability that a first  return to state E 1 occurs at the n-th step. Calculate also the mean recurrence time µ1 . What type of state is E 1 ?  The first return probabilities for state E 1 are f 1(1) = 1

− a,

f 1(2) = a(1

Hence f 1 =

∞

 n=1

− b),

f 1(3) = ab(1

(n)

− a + a(1 − b) + ab(1 − c) + abc = 1 ,

f 1

=1

− c),

f 1(4) = abc,

f1(n)   = 0,

(n

≥ 5).

which implies that E 1 is persistent. Also, the mean µ1

=

∞

 n=1

Hence

µ1

(n)

nf 1

=1

− a + 2a(1 − b) + 3ab(1 − c) + 4abc = 1 + a + ab + abc.

is finite so that E 1 is non-null. It is also aperiodic so that E 1 is an ergodic state.

63

1-a 

E 1

E 2

a 

E 

b 

E 4

c 

3

1-b  1-c 

1 1

Figure 4.8: Transition diagram for Problem 4.15. 4.16. Show that the transition matrix 

T  =

 

1 1 1

−a −a −a

0 a 0 0

a 0 0 0

1

 

0 0 a 0

where 0 < a < 1, has two imaginary (conjugate) eigenvalues. If  a = 12 , confirm that T has the invariant  8 4 2 1 distribution  p = 15 . 15 15 15





The eigenvalues of  T  are given by λ1 = a, λ2 = ai, λ3 = ai, λ4 = 1, of which two are imaginary conjugates. If  a = 12 , then λ1 = 12 , λ2 = 12 i, λ3 = 12 i, λ4 = 1, with corresponding eigenvectors

− − 1 −

    1 2

−

−

1 2

r1 =

−

r2 =

,

1

 −     −   − −−    −   −− − −− − −   −  1 2 1 2

1 i 2

+ 12 i +i 1

1 2 1 2

r3 =

,

1 i 2

1 i 2

i

,

r4 =

1

    1 1 1 1

.

The matrix C  of eigenvalues and its inverse, and the diagonal matrix D are given by

C  =

−  −

1 2

1 1 2

1

−

−

1 2 1 2

1 i 2

+ 12 i

+i 1

1 i 2 1 2 1 2

1 3

1 1 2

− − i −i 1

1

,

1

+

3 10 i 3 i 10

3 10 3 10

8 15

1

1 2

0

1 i 2

0

D=

In the limit n

1 10 1 10

C −1 =

1 3



8 15

0

0

0

0

1

0

0

0

0

0

→ ∞, it follows that all the rows of  CD 4 15

n

4 15

0

1 2i

+

1 3

1 10 i 1 i 10

1 10 1 10

− −

+ 2 15

1 3 3 10 i 3 i 10

3 10 3 10

+

−

1 15

1 10 i 1 i 10

 

,

.

C −1 are given by

2 15

1 15

which is the same as the last row of  C −1 .



,

4.17. A production line consists of two manufacturing stages. At the end of each manufacturing stage each  item in the line is inspected, where there is a probability  p that it will be scrapped, q that it will be sent  back to that stage for reworking, and  (1  p q ) that it will be passed to the next stage or completed. The production line can be modelled by a Markov chain with four states: E 1 , item scrapped; E 2 , item completed; E 3 , item in first manufacturing stage; E 4 , item in second manufacturing stage. We define states E 1 and  E 2 to be absorbing states so that the transition matrix of the chain is

− −

T  =

 

1 0  p  p

1

0 1 0  p

− −q 64

0 0 q 0

1

0 0  p q

− −q

 

.

An item starts along the production line. What is the probability that it is completed in two stages?  (n) (n) Calculate f 3 and  f 4 . Assuming that  0 < p + q < 1, what kind of states are E 3 and  E 4 ? What is the probability that an item starting along the production line is ultimately completed?  The transition diagram is shown in Figure 4.9. E 1 and E 2 are absorbing states. The initial position of  the item can be represented by the vector p(0) = 0 0 1 0 . We require



q 

q 



1 1-p-q 

1-p-q 

E 3

E 2

E 4 p

p 

E 1 1

1

Figure 4.9: Transition diagram for Problem 4.17.

(2)

p

= =

(0)

p



T 2 =

2 p

− p



2

0

0

(1

1

 

0 2

q2

− p − q)

1 0  p  p

0 1 0  p

1

− −q q(1 − p − q )

0 0 q 0

1



(2)

(1)

= q,

f 3

(1)

= q,

f 4

f 4

(n)

= 0,

(n)

= 0,

2

− −q

Therefore the probability that an item is completed in two stages is p2 = (1 The first return probabilities are f 3

0 0  p q

  2

− p − q) .

(n

≥ 3), (n ≥ 3).

Hence E 3 and E 4 are transient states. The probability of an item is completed without reworking is (1 p q)2 , with one reworking is 2q(1  p q )2 , with two reworkings 3q 2 (1  p q)2 , and n reworkings (n + 1) q n (1  p q)2 . Hence the probability that an item starting is ultimately completed is

− −

− −

− −

(1

2

− p − q) [1 + 2q + 3q

2

+

− −

(1  p q)2 , ]= (1 q)2

− − −

···

after summing the geometric series. 4.18. The step-dependent transition matrix of Example 4.9 is T n =



1 2

0 1/(n + 1)

1 2

0 0

0 1 n/(n + 1)



,

(n = 1 , 2, 3, . . .).

Find the mean recurrence time for state E 3 , and confirm that  E 3 is a persistent, non-null state. The transition matrix is shown in Figure 4.10. Assuming that a walk starts at E 3 , the probabilities of  first returns to state E 3 are (using the diagram) f 3(1) =

1 , 2

f 3(2) = 0 ,

f 3(3) =

1 1+1

× 12 × 1 = 14 , . . . , f  

(n) 3

Hence

65

=

1 1+1

× 2 1− × 12 × 1 = 2 1− n 3

n 1

, ....

1 2

E 1

1/(n +1)

1 2

E 3 E 2

1

n  /(n +1)

Figure 4.10: Transition diagram for Problem 4.18.

f 3 =

∞

∞

1 = 1, 2n

  (n) f 3

=

n=1

n=1

using the formula for the sum of a geometric series. This means that E 3 is persistent. The mean recurrence time is given by ∞ ∞ 1 1 3 n (n) = + = 2, µ3 = nf 3 = + n − 1 2 2 2 2





n=1

n=3

using the formula for the sum of the geometric series. Hence E 3 is persistent and non-null. 4.19. In Example 4.9, a persistent, null state occurred in a chain with step-dependent transitions: such a  state cannot occur in a finite chain with a constant transition matrix. However, chains over an infinite number of states can have persistent, nul l states. Consider the following chain which has an infinite number  of states E 1 , E 2 , . . . with the transition probabilities  p11 =

1 , 2

1 , 2

p12 =

pj1 =

1 ,  j + 1

pj,j+1 =

j ,  j + 1

( j

≥ 2).

Find the mean recurrence time for  E 1 , and confirm that  E 1 is a persistent, null state. From the transition diagram, the probabilities of first returns to E 1 are given by E 1

1 2

E 2

1 2

E  3

2 3

E 4

3 4

4 5

E 5

1

3 1 4

1 5

1 6

Figure 4.11: Transition diagram for Problem 4.19. f 1(1) =

1 , 2

f 1(2) =

1 , 2.3

f 1(3) =

1 , 3.4

...

, f1(n)   =

1 , n(n + 1)

....

Therefore f 1 =

∞

∞

  n=1

(n) f 1

=

n=1

1 = lim N →∞ n(n + 1)

N 

 n=1

1 n

− n +1 1



= lim N 

which implies that E 1 is p ersistent. However the mean recurrence time is µ1

=

∞

 n=1

(n) nf 1

=

∞

 n=1

66

1 = n+1

∞,

→∞

−  1

1 = 1, N 

the series being divergent. According to the definition, E 1 is a null state. 4.20. A random walk takes place on  1, 2, . . . subject to the following rules. A jump from position  i to position  1 occurs with probability  qi , and from position  i to i + 1 with probability  1 qi for  i = 1, 2, . . ., where 0 < qi < 1. Sketch the transition diagram for the chain. Explain why to investigate the persistence of every state, only one state, say state 1, need be considered. Show that the probability that a first return  to state 1 occurs at some step is

−

f 1 =

∞ j −1



(1

j=1 k=1

− q )q . j

k

If  qj = q ( j = 1 , 2, . . .), show that every state is persistent. The transition diagram is shown in Figure 4.12 with the states labelled E 1 , E 2 , . . .. The chain is irreducible since every state can be reached from every other state. The diagram indicates that every state E 1

1-

q 1

E 2

q 1

1-

E 3

q 2

1-

E 4

q 3

1-

q 4

E 5

q 2 q 3

q 4 q 5

Figure 4.12: Transition diagram for Problem 4.20. is aperiodic since a return to any state can be achieved in any number of steps. Need only consider one state since the others will have the same properties. Consider state E 1 . Then, from the diagram, the probabilities of first returns are j (1) f 1

= q1 ,

(2) f 1

= (1

(3) f 1

− q )q , 1

2

Therefore f 1 =

∞

f 1 =

∞ j −1



(1

j=1 k=1

− q )(1 − q )q , . . . 1

2

∞ j −1

  j=1

If  qj = q, ( j = 1 , 2, . . .), then

= (1

(j)

f 1

=

(1

j=1 k=1

− q)q = q

∞

 j=1

(1

3

(j) , f1 

=

−1



(1

k=1

− q )q , . . . . k

− q )q . k

j

− q) −

1

j

= q/q = 1 ,

using the formula for the sum of geometric series. Hence E 1 and every state is recurrent.

67

j

Chapter 5

Poisson processes 5.1. The number of cars which pass a roadside speed camera within a specified hour is assumed to be a  Poisson process with parameter  λ = 92 per hour. It is also found that 1% of cars exceed the designated  speed limit. What are the probabilities that (a) at least one car exceeds the speed limit, (b) at least two cars exceed the speed limit in the hour. With λ = 92 in the Poisson process, the mean number of cars in the hour is 92 1 = 92. Of these, on average, 0.92 cars exceed the speed limit. Assume that the cars which exceed the limit form a Poisson process with parameter λ1 = 0.92. Let N (t) be a random variable of the number of cars exceeding the speed limit by time t measured from the beginning of the hour. (a) The probability that at least one car has exceeded the limit within the hour is

×

1

− P(N (t) < 1) = 1 − e−

λ1

=1

− 0.398 = 0.602.

(b) The probability that at least two cars have exceeded the limit within the hour is 1

λ1

− P(N (t) < 2) = 1 − e− − λ e− 1

λ1

=1

− 0.398 − 0.367 = 0.235.

5.2. If the between-event time in a Poisson process has an exponential distribution with parameter  λ with  density  λe−λt , then the probabilitythat the time for the next event to occur is at least  t1 is P Qt > t = e

{

Show that, if  t1 , t2

}

−λt .

≥ 0, then  P Qt > t 1 + t2 Qt > t1 = P Qt > t 2 .

{

|

}

{

}

What does this result imply about the Poisson process and its memory of past events?  By formula (1.2) on conditional probability P(Q > t 1 + t2 Q > t1 )

|

=

P(Q > t1 + t2 Q > t1 ) P(Q > t1 + t2 ) = P(Q > t 1 ) P(Q > t1 )

∩

e−λ(t +t ) = e−λt e−λt P(Q > t2 ) 1

= =

2

2

1

The result shows the loss of memory property of the Poisson process. 5.3. The number of cars which pass a roadside speed camera are assumed to behave as a Poisson process with intensity  λ. It is found that the probability that a car exceeds the designated speed limit is σ . (a) Show that the number of cars which break the speed limit also form a Poisson process.

68

(b) If  n cars pass the camera in time t, find the probability function for the number of cars which exceed  the speed limit. (a) Let N (t) be the random variable representing m the number of speeding cars which have occurred in time t. The probability qn (t) = P(N (t) = m) satisfies qn (t + δt)

≈ q − (t)λσδt + q

n (t)(1

n 1

− λσδt),

where λσδt is the probability that a speeding car appears in the time δt . This is the equation for a Poisson process with intensity λσ . (b) Of the n cars the number of ways in which m speeding cars can be arranged is n! = m!(n m)!

−



n . m

The probability that any individual event occurs is



n σ m (1 m

n m

− σ) −

(m = 0 , 1, 2, . . . , n),

,

which is the binomial distribution. 5.4. The variance of a random variable Xt is given by  2

V(Xt ) = E(Xt )

2

− E(X ) . t

In terms of the generating function  G(s, t), show that  V(Xt ) =

 

∂G (s, t) s ∂s

∂  ∂s

−

∂G (s, t) ∂s

 2

.

s=1

(an alternative formula to (5.20)). Obtain the variance for the Poisson process using its generating function  G(s, t) = eλ(s−1)t

given by eqn (5.17), and check your answer with that given in Problem 5.3. The assumption is that the random variable Xt is a function of the time t. Let pn (t) = P(Xt = n). The probability generating function G(s, t) becomes a function of two variables s and t. It is defined by G(s, t) =

∞



 pn (t)sn .

n=0

The mean of  Xt is given by E(Xt ) =

∞

 n=1

The mean

of  Xt2

is given by 2 E(Xt )

∞

n=1

Hence V(Xt ) =

∂  ∂s

 −

2

∂G (s, t) s ∂s



s=1

∂G (s, t) ∂s

− . Then

∂  λ(s−1)t [e ] E(Xt ) = ∂s

69



s=1

.

s=1

2

s=1

λ(s 1)t

For the Poisson process G(s, t) = e

 

∂G (s, t) s ∂s

∂  n  pn (t) = ∂s

  

=

∂G (s, t) npn (t) = ∂s

= λt,

.

and 2 E(Xt )



∂  = [sλteλ(s−1)t ] ∂s

Hence V(Xt ) = λt.

= λt

s=1

2

− (λt) .

5.5. A telephone answering service receives calls whose frequency varies with time but independently of  other calls perhaps with a daily pattern—more during the day than the night. The rate λ(t) 0 becomes a   function of the time t. The probability that a call arrives in the small time interval  (t, t + δt ) when  n calls have been received at time t satisfies

≥

 pn (t + δt ) = pn−1 (t)(λ(t)δt + o(δt )) +  pn (t)(1

− λ(t)δt + o(δt)),

(n

≥ 1),

with   p0 (t + δt ) = (1

− λ(t)δt + o(δt)) p (t). 0

It is assumed that the probability of two or more calls arriving in the interval  (t, t + δt ) is negligible. Find  the set of differential-difference equations for  pn (t). Obtain the probability generating function  G(s, t) for  t the process and confirm that it is a stochastic process with intensity  0 λ(x)dx. Find  pn (t) by expanding  G(s, t) in powers of  s. What is the mean number of calls received at time t? 

 

From the difference equation it follows that  pn (t + δt ) δt

− p

n (t)

 p0 (t + δt ) δt

Let δt

→ ∞. Then

= λ(t) pn−1 (t)

− λ(t) p

n (t) +

o(1),

− p (t) = −λ(t) p (t) + o(1). 0

0

 pn (t) = λ(t) pn−1 (t)

− λ(t) p

n (t),

(i)

 p0 (t) = −λ(t) p0 (t).

Let

∞

(ii)



G(s, t) =

 pn (t)sn .

n=0

n

Multiply (i) by s , sum from n = 1, and add (ii) so that ∂G (s, t) = λ(t)(s ∂t

− 1)G(s, t).

(iii)

The initial value is G(s, 0) = 1. The solution of (iii) (which is essentially an ordinary differential equation) subject to the initial condition is

 −    t

G(s, t) = exp (s

1)

λ(u)du .

0

Expansion of the generating function gives the series

−         −         t

G(s, t) = exp

1

λ(u)du exp s

0

where the probability

µ(t) =

λ(u)du =

0

1 exp  pn (t) = n! The mean of the process is

∞

t

n=0

t

1

n

t

λ(u)du

λ(u)du

0

∂G (s, t) ∂s

 pn (t)sn ,

.

0

t

=

s=1

λ(u)du.

0

5.6. For the telephone answering service in Problem 5.5, suppose that the rate is periodic given by  λ(t) = a + b cos(ωt ) where a > 0 and  b < a. Using the probability generating function from Problem 5.6 find the

||

70

probability that  n calls have been received at time t. Find also the mean number of calls received at time t. Sketch graphs of  p0 (t), p1 (t) and  p2 (t) where a = 0 .5, b = 0 .2 and  ω = 1. Using the results from Problem 5.5,

 −  



t

G(s, t) = exp (s

1)

(a + b cos ωu )du = exp[(s

0

Hence  pn (t) =

− 1)(at + (b/ω)sin ωt)].

1 −[at+(b/ω) sin ωt] e [at + (b/ω )sin ωt ]n , n!

and the mean is µ(t) =

at + (b/ω )sin ωt.

The first three probabilities are shown in Figure 5.1.

p0 (t)

p1(t) p (t) 2

t 

Figure 5.1: Graphs of the probabilities p0 (t), p1 (t) and p2 (t) versus t in Problem 5.6.

5.7. A Geiger counter is pre-set so that its initial reading is n0 at time t = 0. What are the initial  conditions on  pn (t), the probability that the reading is n at time t, and its generating function  G(s, t)?  Find  pn (t), and the mean reading of the counter at time t. The probability generating function for this Poisson process is (see eqn (5.16)) G(s, t) = A(s)eλ(s−1)t .

The initial condition is G(s, 0) = xn . Hence A(s) = sn and 0

0

G(s, t) = sn eλ(s−1)t . 0

The power series expansion of  G(s, t) is G(s, t) =

∞



n=n0

sn λn−n e−λt . (n n0 )! 0

−

Hence

(λt)n−n e−λt , (n n0 )! 0

 pn (t) = 0 ,

(n < n0 ),

pn (t) =

The mean reading at time t is given by ∂G (s, t) µ(t) = ∂s



71

s=1

−

= n0 + λt.

(n

≥ n ). 0

5.8. A Poisson process with probabilities  pn (t) = P[N (t) = n] =

(λt)n e−λt n!

has a random variable N (t). If  λ = 0 .5, calculate the following probabilities associated in the process: (a) P[N (3) = 6]; (b) P[N (2.6) = 3]; (c) P[N (3.7) = 4 N (2.1) = 2]; (d) P[N (7) N (3) = 3].

|

−

6

0.5 3

× 3) e− × = 0 .00353. 6! (0.5 × 2.6) e− × (b) P(N (2.6) = 3) = p (2.6) = = 0 .100. (a) P(N (3) = 6) = p6 (3) =

(0.5

3

3

0.5 2.6

3!

(c) P[N (3.7) = 4 N (2.1) = 2] = P[N (1.6) = 2] = p2 (1.6) = (d) P[N (7)

|

(0.5

2

0.5 1.6

× 1.6) e− 2!

− N (3) = 3] = P[N (4) = 3] = 0.180.

×

= 0 .144.

5.9. A telephone banking service receives an average of 1000 call per hour. On average a customer transaction takes one minute. If the calls arrive as a Poisson process, how many operators should the bank employ  to avoid an expected accumulation of incoming calls?  Let time t be measured in minutes . The intensity of the Poisson process λ = 1000/60 = 50/3. The expected inter-arrival time is 1 3 = = 0 .06 minutes. λ 50 This must be covered 1 = 16 .7 operators. 0.06 Hence 17 operators would be required to cover expected incoming calls. 5.10. A Geiger counter automatically switches off when the nth particle has been recorded, where n is fixed. The arrival of recorded particles is assumed to be a Poisson process with parameter  λt. What is the expected  value of the switch-off times?  The probability distribution for the switch-off times is F (t)

= =

1

− {probabilities that 0 , 1, 2, . . . , n − 1 particles recorded by time t)} (λt) − 1 − e− 1 + λt + · · · + (n − 1)! λt

  n 1

= 1 − e−λt

−

r=0

n 1



(λt)r r!

Its density is, for t > 0, f (t)

=

dF (t) d = dt dt n 1

= λe−λt

−

 r=0

=



λe

(n

n 1

1 − e−λt

(λt)n n! n 1

− −λt (λt)

− 1)! .

72

   − −

r=0

n 1

− e−λt

−

r=1

(λt)r r!

λn tn−1 (n 1)!

which is gamma. Its expected value is µ

=

 

∞

tf (t)dt =

0

=

λ(n

1

− 1)!

 

∞

0

 

∞

λn−1 n λe−λt t dt =

(n

0

− 1)!

 

∞

λn (n 1)!

n! n = . e−s ds = λ(n 1)! λ

−

e−λt tn dt

0

−

5.11. Particles are emitted from a radioactive source, and, N (t), the random variable of the number of  particles emitted up to time t form  t = 0, is a Poisson process with intensity  λ. The probability that any  particle hits a certain target is p, independently of any other particle. If  M t is the random variable of the number of particles that hit the target up to time t, show, using the law of total probability, that  M (t) forms a Poisson process with intensity  λp.

For any two times t1 , t2 , (t2 > t1 P[M (t2 )

− M (t1) = k]

≥ 0), using the law of total probability, with t2 − t1 = t, =

∞

 

P[N (t2 )

− N (t1) = n]

n=k

=

∞

= e−λt

n k  p (1 k

n!

pk (1  p)k

−

n k  p (1 k

 −    −− n

(λt) e−λt

n=k



 p)n−k

∞ (λt)n n

n=k

n!

k

(1

∞ [(1  p)λt]n−k ( pλt) = e−λt k

k!

=

(λpt)k e− k!

which is a Poisson process of intensity λp.

73

n=k λpt

,

(n

− p)n−k

k)!

− p)n

Chapter 6

Birth and death processes 6.1. A colony of cells grows from a single cell. The probability that a cell divides in a time interval  δt is λδt + o(δt ).

There are no deaths. Show that the probability generating function for this death process is G(s, t) =

1

se−λt . (1 e−λt )s

− −

Find the probability that the original cell has not divided at time t, and the mean and variance of population  size at time t (see Problem 5.4, for the variance formula using the probability generating function). This is a birth process with parameter λ and initial population size 1. Hence the probability generating function is (see eqn (6.12)) G(s, t) = se−λt [1 (1 e−λt )s]−1 ,

− −

which satisfies the initial condition G(s, 0) = s. It follows that the probability that the population size is 1 at time t is p1 (t) = e−λt . The mean population size is ∂G (s, t) = eλt . µ(t) = ∂s s=1



Using the generating function, the variance of the population size is σ

2

 

∂  = ∂s

∂G (s, t) s ∂s

−

∂G (s, t) ∂s



= [2 e2λt s=1

λt

2λt

−e ]−e

= e2λt

−e

λt

.

6.2. A simple birth process has a constant birth-rate λ. Show that its mean population size the differential equation  dµ(t) = λµ(t). dt How can this result be interpreted in terms of a deterministic model for a birth process? 

µ(t)

satisfies

From Section 6.3, the mean population of the birth process with initial population n0 is given by = n0 eλt . It can be verified that

µ(t)

dµ dt

−λ

µ

= n0 λeλt

λt

− n λe 0

= 0,

This differential equation is a simple deterministic model for the population in a birth process, so that the mean population size in the stochastic process satisfies a deterministic equation. [However, this is not always the case in the relation b etween stochastic and deterministic models.]

74

6.3. The probability generating function for a simple death process with death-rate µ and initial population  size n0 is given by  n se−µt − µt n G(s, t) = (1 e ) 1+ 1 e−µt

−

0





−

0

(see Equation (6.17)). Using the binomial theorem find the probability  pn (t) for  n n0 . If  n0 is an even  number, find the probability that the population size has halved by time t. A large number of experiments were undertaken with live samples with a variety of initial population sizes drawn from a common source and the times of the halving of deaths were recorded for each sample. What would be the expected time for  the population to halve? 

≤

The binomial expansion of  G(s, t) is given by

  −   − 

G(s, t) = (1 − e−µt )n

0

n0

= (1 − e−µt )n

n0 e−nµt sn n (1 e−µt )n

0

From this series, the coefficient of  sn is  pn (t) = (1

− e−

n=0

n0 , ) −n e−nµt n

µt n0

n0

se−µt 1+ 1 e−µt

(n = 0 , 1, 2, . . . , n0 )

which is the probability that the population size is n at time t. Let n0 = 2 m0 , where m0 is an integer, which ensures that n0 is even. We require = (1 − e−µt )m e−m

 pm

0 µt

0

0

 

2m0 . m0

The mean population size at time t is given by µ

µt

− s(1, t) = n e−

=G

0

.

This mean is half the initial population is n0 eµt = 12 n0 , which occurs, on average, at time t = µ−1 ln2. 6.4. A birth process has a probability generating function  G(s, t) given by  G(s, t) =

eλt

s + s(1

λt )

− e−

.

(a) What is the initial population size?  (b) Find the probability that the population size is n at time t. (c) Find the mean and variance of the population size at time t. (a) Since G(s, 0) = s, the initial population size is n0 = 1. (b) Expand the generating function using the binomial theorem: s G(s, t) = λt e + s(1

− e−λt)

= se−λt

∞



sn (1

n=0

− e−

λt n

) .

The coefficients give the required probabilities:  p0 (t) = 0,

(c) Since ∂G (s, t) ∂s

Hence the mean population size is

µ



s=1

pn (t) = e−λt (1

eλt = λt [e + s(1 eλt )]2

−

λt

=e .

75

λt n 1

− e−



) − .

s=1

= eλt .

(i)

From (i), ∂ 2 G(s, t) ∂s 2

Hence the variance is given by V(t)



λt

−2e

(1 = λt [e + s(1

s=1

= = =



λt

) λt )]3

−e −e



s=1

∂G (s, t) ∂ 2 G(s, t) +s ∂s ∂s 2

e

λt

λt

e

2λt

− 2e −e

λt

(1

.

=

λt

−e )−e

 −

λt

−2e

(1

∂G (s, t) ∂s

2λt

λt

−e

).



s=1

6.5. A random process has the probability generating function 

  2 + st 2+t

G(s, t) =

r

,

where r is a positive integer. What is the initial state of the process? Find the probability  pn (t) associated  with the generating function. What is pr (t)? Show that the mean associated with  G(s, t) is µ(t)

rt . 2+t

=

Since G(s, 0) = 1, this implies that the initial size is 1. Expansion of  G(s, t) using the binomial theorem leads to G(s, t)

                        =

2 + st 2+t

=

2 2+t

r

∞

r

r

2 2+t

=

n=0

1+

1 st 2

r

n

r n

st 2

,

(n = 0 , 1, 2, . . .),

Hence the probability that the size is n at time t is  pn (t) =

With n = r,

2 2+t

 pr (t) =

The mean size is given by µ(t)

=

r

r n

2 2+t

∂G (s, t) ∂s

t 2

r

=

s=1

r r

n

t 2

r

=

rt (2 + st)r−1 (2 + t)r

r

rt 2+t

.

=

s=1

rt . 2+t

6.6. In a simple birth and death process with unequal birth and death-rates λ and  µ, the probability generating function is given by  n µ(1 s) (µ λs)e−(λ−µ)t G(s, t) = , λ(1 s) (µ λs)e−(λ−µ)t



− − − − − −



0

 for an initial population size n0 (see Equation (6.23)). (a) Find the mean population size at time t. (b) Find the probability of extinction at time t. (c) Show that, if  λ < µ , then the probability of ultimate extinction is 1. What is it if  λ > µ?  (d) Find the variance of the population size.

76

(a) Let G(s, t) = [ A(s, t)/B (s, t)]n with obvious definitions for A(s, t) and B (s, t). Then 0

[A(s, t)]n −1 ∂G (s, t) = [( µ + λe−(λ−µ)t )B (s, t) [B (s, t)]n +1 ∂s 0

0

If  s = 1, then A(1, t) = B (1, t) =

−

−(µ − λ)e−

(λ µ)t

−

µ

− (−λ + λe−

(λ µ)t

−

))A(s, t)].

. Therefore the mean population size is given by

= n0 e(λ−µ)t .

(b) The probability of extinction at time t is  p0 (t) = G(0, t) =

− µ λ

−

µe−(λ−µ)t µe−(λ−µ)t



n0

.

(c) If  λ < µ, then e(λ−µ)t 0 as t . Hence p0 (t) 1. If  λ > µ, then p0 (t) (µ/λ)n as t . (d) This requires a lengthy differentiation to obtain the second derivative of  G(s, t): symbolic computation is very helpful. The variance is given by

→

→

0

→∞ →∞

V(t) = Gss (1, t) + Gs (1, t)

→

2

− [G (1, t)] s

= n0

(λ + µ) (λ−µ) [e (λ µ)

−

− 1],

(λ = µ).



6.7. In a population model, the immigration rate λn = λ, a constant, and the death rate µn = nµ. For an  initial population size n0 , the probability generating function is (Example 6.3) G(s, t) = eλs/µ exp[ λ(1

−

− (1 − s)e−

µt

)/µ][1

− (1 − s)e−

µt n0

] .

Find the probability that extinction occurs at time t. What is the probability of ultimate extinction?  The probability of extinction is  p0 (t) = G(0, t) = (1

µt n0

− e−

−µt

) e−λ(1−e

)/µ

.

The probability of ultimate extinction is lim  p0 (t) = e−λ/µ .

t

→∞

6.8. In a general birth and death process a population is maintained by immigration at a constant rate λ, and the death rate is nµ. Using the differential-difference equations (6.26) directly, obtain the differential  equation  dµ(t) + µµ(t) = λ, dt  for the mean population size µ(t). Solve this equation assuming an initial population  n0 and compare the answer with that given in Example 6.3. In terms of a generating function the mean of a process is given by µ(t)

=



npn (t).

n=1

From (6.25) (in the book) the differential-difference equation for this immigration-death model is dpn (t) = λpn−1 (t) dt

− (λ + nµ) p

n (t) +

77

µ(n + 1) pn+1(t),

(n = 1 , 2, . . .).

Multiply this equation by n and sum over over n:

∞

 n=1

dpn (t) n dt

=

λ

∞

 

npn−1 (t)

n=1

=

λ

∞

∞

 − λ

npn (t)

n=1

(n + 1) pn (t)

n=0

=

λµ(t) + λ

=

λ

µ(t)

−λ

µ(t).

−µ

−λ

∞

 −  − µ

npn (t) + µ

n=1

µ(t)

∞

µ

n(n + 1) pn+1 (t)

n=1

n2 pn (t) + µ

n=1

∞

n(n

1) pn (t)

n=2

µ(t)

−µ

∞

  −

Hence

dµ(t) + µµ(t) = λ. dt This is a first-order linear equation with general solution

= Ae−µt +

µ(t)

The initial condition implies A = n0

λ . µ

− (λ/µ). Hence µ(t) =

− λ µ

n0

λ . µ

e−µt +

6.9. In a death process the probability of a death when the population size is n = 0 is a constant  µ but  obviously zero if the population size is zero. Verify that, if the initial population is n0 , then  pn (t), the probability that the population size is n at time t is given by 



0

 p0 (t) =

(n0

 

t

µn

− 1)!

(µt)n −n −µt e , (n0 n)! 0

sn −1 e−µs ds,

pn (t) =

0

0

(1

−

≤ n ≤ n ), 0

Show that the mean time to extinction is n0 /µ. The probability that a death occurs in time δt is a constant µ independently of the population size. Hence (i)  pn (t + δt) = (1 µ) pn0 (t),

−

0

 pn (t + δt ) = µpn+1 (t) + (1

− µ) p

n (t),

(n = 1 , . . . , n0

 p0 (t + δt ) = µp1 (t) +  p0 (t).

− 1),

(ii) (iii)

subject to the initial conditions pn (0) = 1, pn (0) = 0, (n = 0 , 1, 2, . . . n0 1). Divide through each of the eqns (i) and (ii) by δt , and let t to obtain the differential-difference equations

−

0

→∞

 pn (t) = 0

 pn (t) =

−µp

From (iii) and the initial conditions,

n (t) +

n0 (t),

(iv)

−µp

µpn+1 (t),

(n = 1 , 2, . . . , n0

 p0 (t) = µp1 (t).

− 1).

 pn (t) = Ae−µt = e−µt . 0

From (iv) with n = n0

− 1,  pn −1 (t) = 0

−µp

n0

−1 (t) + µpn (t) = 0

−µp

n0

−1 (t) + e

−µt .

Subject to the initial condition pn −1 (0) = 1, this first-order linear equation has the solution 0

 pn (t) = µte−µt . 0

Repeat this process which leads to the conjecture that (µt)n −n −µt e , (n0 n)! 0

 pn (t) =

−

78

(n = 1 , 2, . . . , n0

− 1),

(v) (vi)

which can be proved by induction. The final probability satisfies  p0 (t) = µp1 (t) =

Direct integration gives (n0

0

0

−

 

t

µn

0

 p0 (t) =

µn tn −1 −µt e . (n0 1)!

− 1)!

sn −1 e−µs ds.

(vii)

0

0

It can be checked that p0 (t) 1 as t which confirms that extinction is certain. The probability distribution of the random variable T  of the time to extinction is p0 (t) = P[T  given by (viii). Its density is d p0 (t) µn tn −1 −µt f (t) = e . = dt (n0 1)! Hence the expected value of  T  is

→

→∞

0

≤ t]

0

−

E(T ) =

 

∞

tf (t)dt =

0

 

∞ µn tn 0

(n0

0

0

− 1)! e

−µt = n0 , µ

using an integral formula for the factorial (see the Appendix in the book). 6.10. In a birth and death process the birth and death rates are given by  λn = nλ + α,

µn = nµ,

where α represents a constant immigration rate. Show that the probability generating function  G(s, t) of  the process satisfies ∂G (s, t) ∂G (s, t) = ( λs µ)(s 1) + α(s 1)G(s, t). ∂t ∂s Show also that, if  G(s, t) = ( µ λs)−α/λ S (s, t),

−

−

−

−

then  S (s, t) satisfies

∂S (s, t) ∂S (s, t) . = ( λs µ)(s 1) ∂t ∂s Let the initial population size be n0 . Solve the partial differential equation for  S (s, t) using the method of  Section 6.5 and confirm that 

−

G(s, t) =

(µ

−

α/λ

(λ µ)t n0

− λ) [(µ − λs) − µ(1 − s)e [(µ − λs) − λ(1 − s)e − ]

−

]

(λ µ)t n0 +(α/λ)

.

(Remember the modified initial condition for  S (s, t).) Find  p0 (t), the probability that the population is zero at time t (since immigration takes place even  when the population is zero there is no question of extinction in this process). Hence show that  lim  p0 (t) =

t

→∞

− µ

λ

α/λ

µ

if  λ < µ . What is the limit if  λ > µ?  The long term behaviour of the process for  λ < µ can be investigated by looking at the limit of the probability generating function as t . Show that 

→∞

lim G(s, t) =

t

→∞

− µ µ

−

λ λs

α/λ

.

This is the probability generating function of a  stationary distribution and it indicates that a balance has been achieved the birth and immigration rates, and the death rate. What is the long term mean population?  If you want a further lengthy exercise investigate the probability generating function in the special case λ = µ.

79

The differential-difference equations are  p0 (t) =  pn (t) = [(n

Multiply (ii) by s

n

−αp (t) + µp (t), 0

− 1)λ + α] p − (t) − (nλ + α + nµ) p , sum over n ≥ 1 and add (i) leads to ∞

n 1



 p (t)sn

∞



=

n

n=0

[(n

n=1

n (t) + ( n

+ 1)µpn+1(t), (n = 1 , 2, . . .)

− 1)λ + α] p − (t) − (nλ + α + nµ) p n 1

∞



+

(i)

1

(ii)

n n (t)s

(n + 1)µpn+1 (t)sn

n=0



∞  p (t)sn . Then the summations above lead to n=0 n

Let the probability generating function be G(s, t) = ∂G (s, t) = ( λs ∂t

Let G(s, t) = (µ

− λs)−

α/λ

− µ)(s − 1) ∂G∂s(s, t) + α(α − 1)G(s, t).

S (s, t). Then

∂G (s, t) = (µ ∂s

− λs)−

α/λ ∂S (s, t)

∂s

+ α(µ

(α/λ) 1

− λs)−

− S (s, t),

and

∂G (s, t) ∂S (s, t) = ( µ λs)−α/λ . ∂t ∂t This transformation removes the non-derivative term to leave

−

∂S (s, t) = ( λs ∂t

(s, t) − µ)(s − 1) ∂S ∂s .

Now apply the change of variable defined by ds = ( λs dz

− µ)(s − 1)

as in Section 6.5(a). Integration gives (see eqn (6.21)) s=

λ λ

(λ µ)z

− = h(z ) (say). (λ−µ)z

− µe − λe

The initial condition is equivalent to G(s, 0) = sn or S (s, 0) = (µ 0

S (f (z ), 0) =



λ(µ λ) λ λe(λ−µ)z

−

−

 − α/λ

λ λ

−

(iii) α/λ n0

− λs)

µe(λ−µ)z λe(λ−µ)z



s

. It follows that

n0

= w(z ) (say).

Since S (f (z ), t) = w(z + t) for any smooth function of  z + t, it follows that G(s, t)

= =

− λs)− (µ − λs)− (µ

α/λ

α/λ

S (f (z ), t) = ( µ



λ

−

α/λ

− λs)−

λ(µ λ) λe(λ−µ)(z+t)

−

w(z + t)

 − α/λ

λ λ

µe(λ−µ)(z+t) λe(λ−µ)(z+t)

−



n0

,

where z is defined by s = h(z ) given by (iii). Finally G(s, t) =

(µ

α/λ

(λ µ)t n0

− λ) [(µ − λs) − µ(1 − s)e [(µ − λs) − λ(1 − s)e − ]

−

]

(λ µ)t n0 +(α/λ)

as displayed in the problem. The probability that the population is zero at time t is  p0 (t) = G(0, t) =

(µ

α/λ

(λ µ)t n0

− λ) (µ − µe (µ − λe − )

−

)

(λ µ)t n0 +(α/λ)

80

.

,

(iv)

If  λ < µ, then  p0 (t)

If  λ > µ , then  p0 (t) =

as t

(µ

  − → µ

α/λ

λ

α/λ

(µe−(λ−µ)t µ)n (µe−(λ−µ)t λ)n +(α/λ)

− λ)

.

µ

−

−

0

    − → λ

0

lim G(s, t) =

t

→∞

− µ µ

−

α/λ

λ

. The long term behaviour for λ < µ is determined by letting t

→∞

µ

λ λs

n0

µ λ

,

→ ∞ in (iv) resulting in

α/λ

.

Express G(s, t) in the form α/λ

A(s, t) , B (s, t)

G(s, t) = ( µ

− λ)

(λ µ)t n0

B (s, t) = [(µ

where A(s, t) = [(µ

− λs) − µ(1 − s)e

−

] ,

Then Gs (s, t) = (µ

− λ)

α/λ As (s, t)B (s, t)

− λs) − λ(1 − s)e

(λ µ)t n0 +(α/λ)

−

]

.

− A(s, t)B (s, t) s

[B (s, t)]2

For the mean we require s = 1, for which value A(1, t) = (µ

− λ)

µ(t)

B (1, t) = ( µ

,

(λ µ)t

0

s

=

n0

(λ µ)t

0

=

,

1

n0 +(α/λ) 1

−.

− λ) A (1, t)B(1[,Bt)(1−, tA)](1, t)B (1, t) n [ α + {n ( µ − λ ) − α }e − ] . µ−λ α/λ

Gs (1, t) = ( µ 0

s

s

2

(λ µ)t

0

If  λ < µ , then µ(t)

as t

n0 +(α/λ)

− λ) A (1, t) = n (−λ + µe − )(µ − λ) − , B (1, t) = (n + (α/λ))(−λ + λe − )(µ − λ) s

Hence

n0

→ µn−αλ , 0

→ ∞. If  λ > µ , then the mean becomes unbounded as would be expected.

6.11. In a birth and death process with immigration, the birth and death rates are respectively  λn = nλ + α,

µn = nµ.

Show directly from the differential-difference equations for  pn (t), that the mean population size µ(t) satisfies the differential equation  dµ(t) = ( λ µ)µ(t) + α. dt Deduce the result  α µ(t) µ λ as t if  λ < µ. Discuss the design of a deterministic immigration model based on this equation.

−

→ −

→∞

The difference equations for the probability pn (t) are given by  p0 (t) =  pn (t) = [(n

− 1)λ + α] p

−αp (t) + µp (t), − (t) − [(nλ + α + nµ) p (t) + (n + 1)µp 0

n 1

1

n

81

n+1 (t).

(i) (ii)

The mean µ(t) is given by µ(t) = the sums dµ(t) = dt

∞





∞ np (t). Multiply (ii) by n and sum from n = 1. Then, re-ordering n n=1

npn (t) = λ

n=1

∞



n(n

− 1) p − (t) + α n 1

n=2

−(λ + µ) =

λ

∞



∞



 pn−1 (t)

n=1

n2 pn (t) + µ

n=1

n(n + 1) pn (t) + α

−(λ + µ) α + (λ

− µ)



α

npn (t)

n=1

∞



∞



 pn (t)

n=0

∞

∞

 −

n(n + 1) pn+1 (t)

n=1

n=1

=

∞



n2 pn (t) + µ

n=1

−α

∞



n(n

n=2

µ(t).

∞



npn (t)

n=1

− 1) p

n (t)

The mean of the stochastic process satisfies a simple deterministic model for a birth and death process with immigration. 6.12. In a simple birth and death process with equal birth and death rates λ, the initial population size has a Poisson distribution with probabilities  pn (0) = e−α

αn n!

(n = 0 , 1, 2, . . .),

with intensity  α. It could be thought of as a process in which the initial distribution has arisen as the result  of some previous process. Find the probability generating function for this process, and confirm that the probability of extinction at time t is exp[ α/(1 + λt)] and that the mean population size is α for all  t.

−

In Section 6.5(b), the probability generating function G(s, t) for the case in which the birth and death rates are equal satisfies ∂G (s, t) ∂G (s, t) = λ(1 s)2 . ∂t ∂s To solve the partial differential equation, the transformation

−

z=

1 λ(1

− s)

or

,

s=

λz 1 λz

−

(i)

is used. The result is that G(s, t) = w(z + t) for any smooth function w. The initial condition at t = 0 is w(z ) =

∞



n

 pn (0)s =

n=0

∞

 n=0



αn e−α n λz 1 s = eα(s−1) = exp α n! λz



− −1

= e−α/(λz) ,

usng the transformation (i). Hence G(s, t) = w(z + t) = e−α/[λ(z+t)] = exp

The probability of extinction at time t is

−   α

1

1

− s + λt

  −

α(1 s) = exp 1 + λt(1 s)

− −



 p0 (t) = G(0, t) = exp[ α/(1 + λt)].

−

6.13. A birth and death process takes place as follows. A single bacterium is allowed to grow and assumed  to behave as a simple birth process with birth rate λ for a time t1 without any deaths. No further growth  then takes place. The colony of bacteria is then allowed to die with the assumption that it is a simple death  process with death rate µ for a time t2 . Show that the probability of extinction after the total time t1 + t2 is

∞

 n=1

eλt (1 1

λt1 n 1

− e−

) − (1

82

− e−

µt2 n

) .

Using the formula for the sum of a geometric series, show that this probability can be simplified to 1 eµt . λt µt +e 1 e 2

1

−

−

2

Suppose that at time t = t1 the population size is n. From Section 6.3, the probability that the population is of size n at time t1 entirely through births is  pn (t1 ) = e−λt (1 1

− e−

λt1 n 1

) − .

From Section 6.4 on the death process, the probability that the population b ecomes extinct after a further time t2 is q0 (t2 ) = (1 e−µt )n .

−

2

The probability that the population increases to n and then declines to zero is  pn (t)q0 (t) = e−λt (1

− e−

1

λt1 n 1

) − (1

µt2 n

− e−

)

Now n can take any value equal to or greater than 1. Hence the probability of extinction through every possible n is s(t1 , t2 ) =

∞



e−λt (1 1

n=1

λt1 n 1

− e−

) − (1

− e−

µt2 n

) .

The probability s(t1 , t2 ) can be expressed as a geometric series in the form

∞

=

e−λt 1 e−λt

=

(1 − e−λt )(1 − e−µt ) e−λt · (1 − e−λt ) [1 − (1 − e−λt )(1 − e−µt )]



1

s(t1 , t2 )

−

1

n=1

1

λt1

− e−

)(1

µt2

− e−

1

1

)]n 2

1

2

eµt 1 λt µt e +e 1 2

=

[(1

1

−

2

−

6.14. As in the previous problem a single bacterium grows as a simple birth process with rate λ and no deaths for a time τ . The colony numbers then decline as a simple death process with rate µ. Show that the probability generating function for the death process is (1 e−µt (1 (1 e−λτ )(1

− 1− −

λτ 

− s))e− − e− (1 − s)) , µt

where t is measured from the time τ . Show that the mean population size during the death process is eλτ −µt . During the birth process the generating function is (see eqn (6.12)) G(s, t) =

1

se−λt , (1 e−λt )s

− −

assuming an initial population of 1. For the death process suppose that time restarts from t = 0, and that the new probability generating function is H (s, t). At t = 0, H (s, 0) = G(s, τ ) =

For the death process the transformation is s = 1 H (s, 0) = w(z ) =

− e−

1

se−λτ  . (1 e−λτ )s

− −

µz

, so that

(1 e−µz )e−λτ  . (1 e−λτ )(1 e−µz )

− 1− − 83

−

Then, in terms of  s,

[1 e−µt (1 1 (1 e−λτ )[1 The mean population size in the death process is

− − −

H (s, t) = w(z + t) =

H s (1, t) =

[1

λt

− s)]e− − e− (1 − s)] . µt

λτ 

)]e−µt−λτ  + e−λτ −µt (1 [1 (1 e−λτ ]2

− (1 − e−

− −

− e−

λτ 

)

= eλτ −µt .

6.15. For a simple birth and death process the probability generating function ( equation (6.23)) is given by  G(s, t) =



µ(1 λ(1

(λ µ)t

− s) − (µ − λs)e− − s) − (µ − λs)e−

− (λ−µ)t



n0

 for an initial population of  n0 . What is the probability that the population is (a) zero, (b) 1 at time t? 

G(s, t)

= = =



(λ µ)t



n0

− s) − (µ − λs)e− − − s) − (µ − λs)e− − [{µ − µe− − } − s{µ − λe− − }] [{λ − µe− − } − s{λ − λe− − }] µ − µe− − λ − λe− n s 1 + λ − µe− − λ − µe− µ(1 λ(1

(λ µ)t

(λ µ)t (λ µ)t



(λ µ)t

(λ µ)t

(λ µ)t

n0

(λ µ)t

n0

  n0

(λ µ)t

− (λ−µ)t

0

(λ µ)t

λe− − −− µe −

− (λ−µ)t

µ µ

 · · · +

The probabilities p0 (t) and p1 (t) are given by the first two coefficiemts in this series. 6.16. (An alternative method of solution for the probability generating function) The general solution of  the first-order partial differential equation  A(x , y , z)

∂z ∂z + B (x , y , z) = C (x , y , z) ∂x ∂y

is f (u, v) = 0, where f  is an arbitrary function, and  u(x , y , z) = c1 and  v(x , y , z) = c2 are two independent  solutions of  dx dy dz = = . A(x , y , z) B (x , y , z) C (x , y , z) This is known as Cauchy’s method. Apply the method to the partial differential equation for the probability generating function for the simple birth and death process, namely (equation (6.19)) ∂G (s, t) = ( λs ∂t

by solving  (λs Show that  u(s,t,G) = G = c1 ,

−

ds µ)(1

− µ)(s − 1) ∂G∂s(s, t) , =

dt dG . = 1 0

− s) −

v (s,t,G) = e−(λ−µ)t

and 

− 1 µ λ

−

s s

= c2 .

are two independent solutions. The general solution can be written in the form 



G(s, t) = H  e−(λ−µ)t

 −  1

s

µ λ

−s

.

Here H  is a function determined by the initial condition  G(s, 0) = sn . Find  H  and recover formula (6.22)   for the probability generating function. 0

84

Note that in the birth and death equation the function C  is zero. Comparing the two partial differential equations, we have to solve ds dt dG . = = (λs µ)(1 s) 1 0 The second equality is simply dG = 0. This equation has a general solution which can be expressed as

−

−

−

u(s,t,G)

≡G=c . 1

The first equality requires the solution of the differential equation ds = dt

−(λs − µ)(1 − s).

The integration is given essentially in eqn (6.20) in the text which in terms of  v can be expressed as v (s,t,G) ≡ e−(λ−µ)t



1 s (µ/λ) s





−

−

−

= c2 .

Hence the genetal solution is



or f  G, e−(λ−µ)t

f (u, v ) = 0,

1 s (µ/λ) s

−



= 0.

Alternatively, this can be written in the form





G(s, t) = H  e−(λ−µ)t

1 s (µ/λ) s

−

−



,

where he function H  is determined by initial conditions. Assuming that the initial population size is n0 , then G(s, 0) = sn , which means that H  is determined by 1 s H  = sn . (µ/λ) s 0

Let u = (1



− s)/((µ/λ) − s). Then

−

−



0

−  −  − 

λ µu n , λ λu which determines the functional form of  H . The result follows by replacing u by 0

H (u) =

e−(λ−µ)t

1 s (µ/λ) s

−

,

as the argument of  H . 6.17. Apply the Cauchy’s method outlined in Problem 6.16 to the immigration model in Example 6.3. In  this application the probability generating function satisfies ∂G (s, t) = λ(s ∂t

− 1)G(s, t) + µ(1 − s) ∂G∂s(s, t) .

Solve the equation assuming an initial population of  n0 . Reading the coefficients of the partial differential equation in Problem 6.16, ds dt dG = = . 1 µ(1 s) λ(s 1)

−

−

−

Integration of the second equality gives u(s,t,G)

≡ G + λµ s = c , 1

85

whilst the first gives

v (s,t,G) = eµt (1

− s) = c . 2

The general solution can be expressed in the functional form

− λµ s + H (e

G(s, t) =

µt

(1

− s)).

From the initial condition G(s, 0) = sn . Therefore 0

− µλ s + H (1 − s) = s so that

− s) = λµ s + s

n0

H (1

Let u = 1

− s: then

H (u) =

The result follows by replacing u by eµt (1

n0

λ (1 µ

,

. n0

− u) + (1 − u)

.

− s) in this formula.

6.18. In a population sustained by immigration at rate λ with a simple death process with rate µ (see Example 6.3), the probability  pn (t) satisfies (equation (6. 25)) d p0 (t) = dt

−λp (t) + µp (t), 0

1

dpn (t) = λpn−1 (t) (λ + nµ) pn (t) + ( n + 1)µpn+1 (t). dt Investigate the steady-state behaviour of the system by assuming that 

−

 pn (t)

→  p

dpn (t)/dt

n,

→0

 for all  n, as t . Show that the resulting difference equations for what is known as the corresponding  stationary process λp0 + µp1 = 0 ,

→∞

λpn−1

− (λ + nµ) p

−

n

+ (n + 1)µpn+1 = 0 ,

(n = 1 , 2, . . .)

can be solved iteratively to give

λ  p1 =  p0 , µ



λ2  p0 , 2!µ2

p2 =

pn =

···

λn  p0 , n!µn

···.

∞

Using the condition  n=0 pn = 1, and assuming that  λ < µ, determine p0 . Find the mean steady-state population size, and compare the result with that obtained in Example 6.3. From the steady-state difference equations λ  p1 =  p0 , µ  p2 =  p3 =

λ2 1 , [ λp0 + (λ + 2µ) p1 ] = 2µ 2!µ2

−

1 λ3 [ λp1 + (λ + 2µ) p2 ] =  p0 , 3µ 3!µ3

−

and so on: the result can be confirmed by an induction proof. The requirement

  ∞

n=0

if  p0 = ( µ

λ µ

n

 p0 =

− λ)/µ. 86

µ

µ

− λ p

0

=1



∞  p = 1 implies n=0 n

The mean steady state population is given by µ

=

      − ∞

npn =

n=1

=

∞

=

n=1

λ µ

p0

∞

np0 n!

n=1

eλ/µ =

(µ

λ)λ

∞

n

λ µ

=

n=1

 p0

(n

− 1)!

 λ µ

n

eλ/µ .

µ2

6.19. In a simple birth process the probability that the population is of size n at time t given that it was n0 at time t = 0 is given by   pn (t) =

− n n0

1 e−λn t (1 1 0

−

− e−

λt n n0

) − ,

(n

≥ n ). 0

(see Section 6.3 and Figure 6.1). Show that the probability achieves its maximum value for given  n and  n0 when  t = (1 /λ)ln(n/n0 ). Find also the maximum value of  pn (t) at this time. Differentiating pn (t), we obtain

−

d pn (t) n =λ n0 dt

1

−1

e−λn t (1 0

λt n n0

− e−

) − −1 [ n0 (1

−

The derivative is zero if 

− e−



1 n0 n , or t = ln n λ n0 Substituting this time back into pn (t), it follows e−λt =

max[ pn ] = t

− n n0

1 nn 0 (n 1 nn 0

−

) + λ(n

λt

− n )e− 0

].

.

n n0

−n ) − 0

λt

.

6.20. In a birth and death process with equal birth and death parameters, λ, the probability generating    function is (see eqn(6.24)) n 1 + ( λt 1)(1 s) G(s, t) = . 1 + λt(1 s)



−

−



−

0

Find the mean population size at time t. Show also that its variance is 2n0 λt. The derivative of  G(s, t) with respect to s is given by Gs (s, t) =

n0 [λt (λt 1)s]n −1 . [(1 + λt) λts]n +1

−

− −

0

0

The mean population size at time t is µ(t) =

Gs (1, t) = n0 .

We require the second derivative given by Gss (s, t) =

n0 [λt (λt 1)s]n −2 [n0 [(1 + λt) λts]n +2

−

− −

0

0

2 2

− 1 + 2λts + 2λ t (1 − s)]

The variance of the population size is V(t) = Gss (1, t) + Gs (1, t)

− [G (1, t)] s

2

= [2 λn0 t + n20

2 0

−n ]+n −n 0

0

= 2 n0 λt.

6.21. In a death process the probability that a death occurs in time δt is a time-dependent parameter  µ(t)n when the population size is n. The pgf  G(s, t) satisfies ∂G = µ(t)(1 ∂t

87

− s) ∂G . ∂s

as in Section 6.4. Show that  G(s, t) = [1

where

τ 

− e− (1 − s)]

 

n0

,

t

τ  =

µ(s)ds.

0

Find the mean population size at time t. In a death process it is found that the expected value of the population size at time t is given by  µ(t)

=

n0 , 1 + αt

(t

≥ 0),

where α is a positive constant. Estimate the corresponding death-rate µ(t). Let z=

and

 

ds 1

so that

− s,

 

z

− e− ,

s =1

t

τ  =

µ(u)du.

0

The equation for the probability generating function b ecomes ∂G ∂G . = ∂τ  ∂z

The general solution can be expressed as G(s, t) = w(z + τ ) for any arbitrary differentiable function w. Initially τ  = 0. Hence G(s, 0) = sn = w(z ) = (1 e−z )n ,

−

0

so that G(s, t) = w(z + τ ) = [1

− e−

(z+τ ) n0

]

0

= [1

The mean population size at time t is given by

τ 

n0

− e− (1 − s)]

.

−     −    t

−τ [1 − e−tau(1 − s)]n −1 µ(t) = Gs (1, t) = n0 e 0

Given the mean µ(t) =

n0 = n0 exp 1 + αt

s=1

= n0 e−τ  = n0 exp

µ(u)du .

0

t

µ(u)du ,

(i)

0

it follows that the death-rate is µ(t) = α/(1 + αt), which can be obtained by differentiating both sides (i) with rspect to t 6.22. A population process has a probability generating function  G(s, t) which satisfies the equation  e−t

∂G = λ(s ∂t

2 ∂G

− 1)

∂s

.

If, at time t = 0, the population size is n0 , show that 



1 + (1 s)(λet λ 1) G(s, t) = 1 + λ(1 s)(et 1)

−

− − − −



n0

.

Find the mean population size at time t, and the probability of ultimate extinction. The generating function satisfies e−t

Let

 

∂G = λ(x ∂t

t

τ  =

u

e du,

z=

0

88

2 ∂G

− 1)

 

∂s

.

ds λ(s

2

− 1)

.

The transformations are τ  = et 1 and s = (λz 1)/(λz ). The transformed partial differential equation has the general solution G(s, t) = w(z + τ ). The initial condition is

−

−

G(s, 0) = s

Hence G(s, t)

= =

− − 1

(s

−   − −  − − −

n0

n0

1 λ(z + τ )

= w(z ).

s 1 + λ(s

= 1

1)(λet λ 1) 1 λ(s 1)(et 1) 1

−

n0

1 λz

= 1

n0

−1 − 1)(e − 1) t



n0

− −

as required. The mean population size is µ(t) =

Gs (1, t) = n0 .

The probability of extinction at time t is



 p0 (t) = G(0, t) =

as t

λ(et 1) λ(et 1) + 1

−

−



n0

→1

→ ∞.

6.23. A population process has a probability generating function given by  G(s, t) =

1 µe−t (1 1 + µe−t (1

−

− s) , − s)

where µ is a parameter. Find the mean of the population size at time t, and its limit as t G(s, t) in powers of  s, determine the probability that the population size is n at time t.

→ ∞.

Expand 

We require the derivative Gs (s, t)

µe−t [1 + µe−t (1

=

t

t

2µe−t [1 + µe−t (1

= Then the mean population size is

µ(t)

t

− s)] + µe− (1 − s)[1 − µe− (1 − s)] [1 + µe− (1 − s)]

− s)]

2

2

= Gs (1, t) = 2µe−t .

To find the individual probabilities we require the power series expansion of  G(s, t). Using a binomial expansion G(s, t)

= =

=

− − −

1 µe−t 1 + µe−t 1 µe−t 1 + µe−t 1 µe−t 1 + µe−t

  −    −

µe−t s 1+ 1 µe−t µe−t s 1+ 1 µe−t

∞

n=0

 −       −

µe−t 1 + µe−t

1

∞

n=0

µe−t s 1 + µe−t

−1

µe−t 1 + µe−t

n

n

sn

µe−t 1 µe−t

sn +

∞

n=1

µe−t 1 + µe−t

  n

sn+1

the coefficients of the powers sn give the following probabilities:  p0 (t) =

1 µe−t  pn (t) = 1 + µe−t

−



1 µe−t , 1 + µe−t

−

µn e−nt µn e−nt + (1 + µe−t )n (1 + µe−t )n−1 (1

89

−

1 µe−t)



=

2µn e−nt , (n (1 + µe−t )n+1

≥ 1).

6.24. In a birth and death process with equal rates λ, the probability generating function is given by (see eqn (6.24)) n n 1 + ( λt 1)(1 s) λ(z + t) 1 G(s, t) = , = λ(z + t) 1 + λt(1 s)



−

  0

−

−



−

0

where n0 is the initial population size. Show that  pi , the probability that the population size is i at time t, is given by 

      i

 pi (t) =

m=0

if  i

≤ n , and by  0

n0

 pi (t) =

m=0

if  i > n 0 , where

n0 m

n0 + i m i m

n0 m

n0 + i m i m

1

α(t) =

  −

− −1 − − −

− λt ,

1

α(t)m β (t)n

+i m

α(t)m β (t)n

+i m

0

−

λt . 1 + λt

β (t) =

λt

0

−

Expand G(s, t) as a power series in terms of  s using the binomial expansion: G(s, t)

   −  −      n0

λt 1 + λt

=

1+

n0 n0

λt 1 + λt

=

1

α=

1

n0

1

n0 αk sk k

k=0

where

λt s λt

− λt ,

β  =

λt

∞

n0 + j  j

j=0



−n λt s 1 + λt

0



−1

β j sj ,

λt . 1 + λt

Two cases have to be considered. (a) i n0 .

≤

 pi (t)

=

     −     −  −    −       − −   −      − −  −  − λt 1 + λt

n0

+

=

=

λt 1 + λt

n0 +i

λt 1 + λt

n0 +i

n0 n0 + i α0 i 0

1

β i +

n0 n0 + i 2 i−1 α β  + i 1 1

···

n0 n0 1 0 αi β  0 i i

m=0 i

m=0

n0 m

n0 + i 1 i m

m

α β 

n0 m

n0 + i 1 i m

m

1

m

m

(λt)2 (λt)2

(b) i > n 0 .

 pi (t)

= =

     −  · · ·    −   −        − − − − λt 1 + λt λt 1 + λt

n0

n0 n0 + i α0 0 i

n0 +i n0

m=0

n0 m

1

β i +

n0 + i 1 i m

m

+

1

n0 αn n0

0

(λt)2 (λt)2

i 1 β i−n i n0

0

m

6.25. We can view the birth and death process by an alternative differencing method. Let  pij (t) be the conditional probability   pij (t) = P(N (t) = j N (0) = i),

|

90

where N (t) is the random variable representing the population size at time t. Assume that the process is in  the (fixed) state N (t) = j at times t and  t + δt and decide how this can arise from an incremental change δt in the time. If the birth and death rates are λj and  µj , explain why   pij (t + δt) = pij (t)(1

− λ δt − µ δt) + λ δtp (t) + µ δtp − (t)  for  i = 1 , 2, 3, . . ., j = 0 , 1, 2, . . .. Take the limit as δt → 0, and confirm that  p (t) satisfies the differential  i

i

i

i+1,j

i

i 1,j

ij

equation 

dpij (t) = dt How should  p0,j (t) be interpreted? 

−(λ

i

+ µi ) pij (t) + λpi+1,j (t) + µi pi−1,j (t).

In this approach the final state in the process remains fixed, that is, the j in pij . We now view pij (t + δt) as pij (δt + t) — in other words see what happens in an initial δt. There will be a birth with probability λi in a time δt or a death with probability µi . Then  pij (t + δt ) = pij (t)[1

− λ δt − µ δt] + λ δt + µ δtp − i

i

i

i

i 1,j (t) +

o(δt)

for i = 1 , 2, 3, . . .; j = 1 , 2, 3, . . .. Hence  pij (t + δt ) δt

In the limit δt

→ 0,

− p

ij (t)

d pij (t) = dt

=

−(λ

−(λ

i

i

+ µi ) pij (t) + λi pi+1,j (t) + µi pi−1,j (t) + o(1).

+ µi ) pij (t) + λi pi+1,j (t) + µi pi−1,j (t),

where we require  p0,j (t) = P(N (t) = j N (0) = 0) =

|



0, 1,

j>0 j =0

6.26. Consider a birth and death process in which the rates are λi = λi and  µi = µi, and the initial  population size is n0 = 1. If   p1,j = P(N (t) = j N (0) = 1),

|

it was shown in Problem 6.25 that  p1,j satisfies dp1,j (t) = dt

−(λ + µ) p

where  p0,j (t) =

If 

1,j (t) +

  0, 1,

Gi (s, t) =

∞

λp2,j (t) + µp0,j (t), j>0 j =0

 pij (t)sj ,

j=0

show that 

∂G 1 (s, t) = (λ + µ)G1 (s, t) + λG2 (s, t) + µ. ∂t Explain why  G2 (s, t) = [G1 (s, t)]2 (see Section 6.5). Hence solve what is effectively an ordinary differential  equation for  G1 (s, t), and confirm that 

−

G1 (s, t) =

µ(1 λ(1

− s) − (µ − λs)e− − s) − (µ − λs)e−

(λ µ)t

− , (λ−µ)t

as in eqn (6.23) with  n0 = 1. Given

dp1,j (t) = dt

−(λ + µ) p

1,j (t) +

91

λp2,j (t) + µp0,j (t),

multiply the equation by sj and sum over j = 1 , 2, . . .. Then ∂G (s, t) = ∂t

Also

−(λ + µ)G (s, t) + λG (s, t) + 1

G2 (s, t) = E[sN 

1

(t)

2

µ(t).

]E[sN  (t) ] = E[sN  (t) ]2 = G1 (s, t)2 . 2

1

Therefore

∂G 1 (s, t) = (λ + µ)g1 (s, t) + λG1 (s, t)2 + µ. ∂t This is a separable first-order differential equation with general solution

−

 

(λG1

dG 1 µ)(G1

−

 

− 1) =

dt + A(x) = t + A(x),

where the ‘constant’ is a function of  x. Assume that G < min(1,µ/λ). Then t + A(x)

dG 1 1 + λG1 µ λ µ

λ

=

− λG − µ 1

1

=

λ

−µ

ln



Hence G1 (s, t) =

 | |

−

1 G1 µ λG1

− −



−

 

dG 1 G1 1

−

.

µ + B (x)e−(λ−µ)t , λ + B (x)e−(λ−µ)t

where B (x) (more convenient than A(x)) is a function to be determined by the initial conditions. Initially, G(s, 0) = s, so that B (x) = ( µ λs)/(1 s). Finally G1 (s, t) agrees with G(s, t) in eqn (6.23) with n0 = 1.

−

−

6.27. In a birth and death process with parameters λ and  µ, (µ > λ), and initial population size n0 , show  that the mean time to extinction of the random variable T n is given by  0

E(T n0 ) = n0 µ(µ

2

− λ)

  0

∞ te−(µ−λ)t [µ − µe−(µ−λ)t ]n −1 dt. [µ − λe−(µ−λ)t ]n +1 0

0

If  n0 = 1, using integration by parts, evaluate the integral over the interval  (0, τ ), and then let  τ  show that  µ λ 1 E(T 1 ) = . ln λ µ

−

−

The distribution function for T n is given by 0

−

µ F (t) = p0 (t) = G(0, t) = µ

−

µe−(µ−λ)t λe−(µ−λ)t



n0

,

(put s = 0 in (6.23)). Its density is f (t) =

dF (t) = n0 µ(µ dt

2

− λ) e−

− µe− [µ − λe−

(µ λ)t [µ

−

(µ λ)t n0

− ] −1 , (µ−λ)t ]n +1 0

(t > 0).

The mean time to extinction is E(T n0 ) =

 

∞

0

tf (t)dt = n0 µ(µ

2

− λ)

 

∞

0

as required.

92

te−(µ−λ)t

[µ [µ

(µ λ)t n0

− ] −1 dt, (µ−λ)t ]n +1

− µe− − λe−

0

→ ∞ to

If  n0 = 1, then E(T 1 )

=

µ(µ

− λ) (µ − λ) 2

=

µ

   

∞

2

te−(µ−λ)t dt [µ λe−(µ−λ)t ]2

−

0

∞ [1

0

−

te−(µ−λ)t dt (λ/µ)e−(µ−λ)t ]2

Integrate the following finite integral between t = 0 and t = τ :

 

τ 

0

[1

−

te−(µ−λ)t dt (λ/µ)e−(µ−λ)t ]2

=

λ(µ

− λ) µ

=

λ(µ

− λ)

λ(µ

λ(µ

dt (λ/µ)e−(µ−λ)t

1 1

1 t + ln e(µ−λ)t − (µ − λ)t µ λ (λ/µ)e

1

−

{

−



τ 

0

}

− (λ/µ)

τ 

0

(µ λ)τ 

−

}



µ

→

 

t + (λ/µ)e−(µ−λ)t

1 τ  − − λ) 1 − (λ/µ)τ  + τ  + µ − λ ln{1 − (λ/µ)e − µ −1 λ ln{1 − (λ/µ)}

µ

=

as τ 

− − − − −

µ

− λ) ln{1 − (λ/µ)}, 2

→ ∞. Finally E(T 1 ) =

−

1 ln λ

− µ

λ

µ

.

6.28. A death process (see Section 6.4) has a parameter  µ and the initial population size is n0 . Its probability  generating function is G(s, t) = [1 e−µt (1 s)]n .

−

Show that the mean time to extinction is n0 µ

n0

−1

 k=0

−

0

 −

( 1)k n0 1 . (k + 1)2 k

−

Let T n be a random variable representing the time to extinction. The probability distribution of  T n is given by F (t) = p0 (t) = G(0, t) = (1 e−µt )n . 0

0

−

The mean time to extinction is E(T n0 ) =

 

∞ d p (t) 0 t

0

dt

dt = n0 µ

 

∞

te−µt (1

0

0

− e−

µt n0

) −1 dt.

Replace (1 − e−µt )n −1 by its binomial expnsion, namely ∞ n0 − 1 −kµt e , (1 − e−µt )n −1 = (−1)k 0

    −  −    0

k=0

k

and integrate the series term-by-term: E(T n0 ) = n0 µ

∞

k=0

k

( 1)

∞

1

n0

k

n0 te−(k+1)µt dt = µ

0

∞

k=0

 −

( 1)k n0 1 . (k + 1)2 k

−

6.29. A colony of cells grows from a single cell without deaths. The probability that a single cell divides into two cells in a time interval  δt is λδt + o(δt ). As in Problem 6.1, the probability generating function    for the process is se−λt G(s, t) = . 1 (1 e−λt )s

− − 93

By considering the probability  n 1

−

 −

F (t) = 1

 pk (t),

k=n

that is, the probability that the population is n or greater by time t, show that the expected time T n for the population to first reach  n 2 is given by 

≥

n 1

−

1 λ

E(T n ) =

 k=1

1 . k

The expansion of the generating function is G(s, t) =

∞



e−λt (1

n=1

Consider the probability function F (t) =

∞

n 1



 pk (t) = 1

k=n

Its density is defined by, for n

− e−

−

λt n 1 n

) − s .

n 1

 −

 pk (t) = 1

k=1

−

 −

e−λt (1

k=1

− e−

λt k 1

) − .

≥ 3, (although it is not required)

dF (t) f (t) = = λe−λt + λe−λt dt and for n = 2,

n 1

−



(1

k=2

− ke−

f (t) = λe−λt ,

λt

)(1

λt k 2

− e−

) − ,

(t > 0)

(t > 0).

Then,

    −   −  τ 

E(T n )

=

lim

τ 

→∞

0



n 1

=

lim

τ 

→∞

τ 

−

=

lim

k=1

τ 

→∞

1 λ

n 1

−

k=1

τ 

τ 

−

k=1

e−λτ (1

− e−

F (t)dt

0

n 1

−

τ 

e−λτ (1 − e−λτ )k−1

n 1

=

   −      − − −   − −

dF (t) t dt = lim [tF (t)τ  0 τ →∞ dt

λτ  k 1

) −

1

0

τ  +

e−λt (1

e−λt )k−1

dt

k=1

1 λ

n 1

−

k=1

1 (1 k

e−λτ )k

1 . k

6.30. In a birth and death process, the population size represented by the random variable N (t) grows as a simple birth process with parameter  λ. No deaths occur until time T  when the whole population  dies. The distribution of the random variable T  is exponential with parameter  µ. The process starts with  one individual at time t = 0. What is the probability that the population exists at time t, namely that  P[N (t) > 0]?  What is the conditional probability  P[N (t) = n N (t) > 0] for  n = 1 , 2, . . .? Hence show that 

|

P[N (t) = n] = e

(λ+µ)t

(1

λt n 1

−e

) − .

Construct the probability generating function of this distribution, and find the mean population size at time t. Since the catastrophe is a Poisson process with intensity µ

−µt .

P[N (t) > 0] = e

94

The process must be simple birth conditional on no deaths, namely

−λt (1 − e−λt )n−1 ,

P[N (t) = n N (t) > 0] = e

|

Hence

(n = 1 , 2, . . .). µt

− P[N (t) > 0] = 1 − e− , − (1 − e− P[N (t) = n] = P[N (t) = n|N (t) > 0]P[N (t) > 0] = e P[N (t) = 0] = 1

(λ+µ)t

λt n 1

) − ,

(n = 1 , 2, . . .).

The probability generating function is G(s, t), where G(s, t)

∞



=

n=0

n

−µt + P[N (t) = n]s = 1 − e

= 1 − e−µt + e−(λ+µ)t

1

−

∞



e−(λ+µ)t (1

n=1

− e−

λt n 1 n

) − s

s s(1 e−λt )

−

using the formula for the sum of a geometric series. For the mean, we require



∂G (s, t) = e−(λ+µ)t 1 ∂s

Then the mean

∂G (s, t) µ(t) = ∂s



−



s(1 e−λt ) 1 + . [1 s(1 e−λt )]2 s(1 e−λt )

− − −

−

= e−(λ+µ)t [eλt + e2λt s=1

−e

λt

] = e(λ−µ)t .

6.31. In a birth and death process, the variable birth and death rates are, for  t > 0, respectively given by  λn (t) = λ(t)n > 0, (n = 0 , 1, 2, . . .)

µn (t) = µ(t)n > 0, (n = 1 , 2, . . .).

If  pn (t) is the probability that the population size at time t is n, show that its probability generating function  is G(s, t) =

∞



 pn (t)sn ,

n=0

satisfies

∂G ∂G . = ( s 1)[λ(t)s µ(t)] ∂t ∂s Suppose that  µ(t0 = αλ(t) ( α > 0, α = 1), and that the initial population size is n0 . Show that 

−

G(s, t) =

−  1 αq (s, t) 1 q(s, t)

−



n0

where q (s, t) =

−

Find the probability of extinction at time t.

 −   −    1 α

−

s exp (1 s

t

α)

λ(u)du .

0

Using eqns (6.25), the differential-difference equations are  p0 (t) = µ(t) p1 (t),  pn (t) = (n

− 1)λ(t) p

n 1 (t)

−

− n[λ(t) + µ(t)] p

n (t) + ( n

+ 1)µ(t) pn+1 (t), (n = 1 , 2, . . .).

In the usual way multiply the equations by sn and sum over n:

∞



 p (t)sn = n

n=0

Let G(s, t) =

∞

 n=2



(n

n

− 1)λ(t) p − (t)s − [λ(t) + µ(t)] n 1

∞

 n=1

n

npn (t)s +

∞



(n + 1)µ(t) pn+1(t)sn

n=0

∞  p (t)sn . Then the series can be expressed in terms of  G(s, t) as n=0 n ∂G ∂t

= =

∂G − [λ(t) + µ(t)]s ∂G + µ(t) ∂s ∂s ∂G (s − 1)[λ(t)s − µ(t)] ∂s λ(t)s2

∂G ∂s

95

Let µ(t) = αλ(t), (α = 1). Then



∂G = (s ∂t

− 1)(s − α) ∂G . ∂s

Let dτ  = λ(t)dt so that τ  can be defined by

 

t

τ  =

λ(u)du.

0

Let ds/dz = ( s

− 1)(s − α) and define z by z

= =

 

ds (s 1)(s 1 1 ln α 1 α

−

1

− α) = 1 − α −s , −s

 

−

 

1

α



1

−s − 1−s

ds

where s < min(1, α). Inversion of this equation gives s=

1 αe(1−α)z = q (z ), 1 e(1−α)z

− −

say. Let G(s, t) = Q(z, τ ) after the change of variable. Q(z, τ ) satisfies ∂Q ∂Q = . ∂τ  ∂z

Since the initial population size is n0 , then

 − −  −  −   −   

n0

Q(z, 0) = s

Hence Q(z, τ ) =

Finally

− 

1 αq (s, t) G(s, t) = 1 q (s, t)

n0

−

1 αe(1−α)(z+τ ) 1 e(1−α)(z+τ )

where q (s, t) =

−

as required. The probability of extinction is

G(0, t) =

where q (0, t) =

n0

1 αe(1−α)z = 1 e(1−α)z

1 α

−

.

n0

.

t

s exp (1 s

α)

λ(u)du

0

−  −  −    n0

1 αq (0, t) 1 q(0, t)

1 exp (1 α

,

t

α)

λ(u)du .

0

6.32. A continuous time process has three states E 1 , E 2 , and  E 3 . In time δt the probability of a change  from  E 1 to E 2 is λδt , from  E 2 to E 3 is also λδt , and from  E 2 to E 1 is µδt . E 3 can be viewed as an  absorbing state. If  pi (t) is the probability that the process is in state E i ( i = 1 , 2, 3) at time t, show that   p1 (t) =

−λp (t) + µp (t), 1

2

p2 (t) = λp1 (t)

− (λ + µ) p (t), 2

p3 (t) = λp2 (t).

Find the probabilities p1 (t), p2 (t), p3 (t), if the process starts in  E 1 at  t = 0. The process survives as long as it is in states E 1 or  E 2 . What is the survival probability of the process?  By the usual birth and death method  p1 (t + δt ) = µδtp2 (t) + (1

96

2

− λδt) p (t) + O((δt) ), 1

 p2 (t + δt) = λδtp1 (t) + (1

2

− λδt − µδt) p (t) + O((δt) ), 2

 p3 (t + δt) = λδtp2 (t) + O((δt )2 ).

Let δt

→ 0 so that the probabilities satisfy  p1 (t) = µp2 (t)  p2 (t) = λp1 (t)

− λp (t),

(i)

− (λ + µ) p (t),

(ii)

1

2

 p3 (t) = λp2 (t).

(iii)

Eliminate p2 (t) between (i) and (ii) so that p1 (t)  p1 (t) + (2λ + µ) p1 (t) + λ2 p1 (t) = 0 .

This second-order differential equation has the characteristic equation m2 + (2λ + µ)m + λ2 = 0 ,

which has the solutions m1 m2

where α =

−

1 2 (2λ

1 2

+ µ) and β  =



=α

± β,

√(4λ + µ). Therefore, since p (0) = 1, then 1

 p1 (t) = Aem

1

t

+ (1

− A)e

m2 t

.

From (i), since p2 (0) = 0,  p2 (t)

= = =

1  [ p (t) + λp1 (t)] µ 1 1 [A(m1 + λ)em t + (1 A)(m2 + λ)em t ] µ (m1 + λ)(m2 + λ) [ em t + em t ] µ(m1 m2 )

−

1

−

−

It follows that

1

1

[ (m2 + λ)em m1 m2 The survival probability at time t is p1 (t) +  p2 (t).  p1 (t) =

−

−

2

1

t

2

+ (m1 + λ)em t ]. 2

6.33. In a birth and death process, the birth and death rates are given respectively by  λ(t)n and  µ(t)n in  eqn (6.25). Find the equation for the probability generating function  G(s, t). If  µ(t) is the mean population  size at time t, show, by differentiating the equation for  G(s, t) with respect to s, that  µ

(assume that  (s

2

− 1)∂  G(s, t)/∂s

2

 (t) = [λ(t) − µ(t)]µ(t),

= 0 when  s = 1)). Hence show that 

 



t

µ(t)

= n0 exp

[λ(u)

0

− µ(u)]du

,

where n0 is the initial population size. The differential difference equations for the probability pn (t) are (see eqn (6.25))  p0 (t) = µ(t) p1 (t),  pn (t) = λ(t)(n

− 1) p

n 1 (t)

−

− [λ(t)n + µ(t)n] p

n (t) +

µ(t)(n + 1) pn+1 (t).

Hence the probability generating function G(s, t) satisfies ∂G (s, t) = [ λ(t)s ∂t

− µ(t)](s − 1) ∂G∂s(s, t) , 97

(i)

(the method parallels that in Section 6.5). Differentiating (i) with respect to s: ∂ 2 G(s, t) = λ(t)(s ∂s∂t

2

− 1) ∂G∂s(s, t) + [λ(t)s − µ(t)] ∂G∂s(s, t) + [λ(t)s − µ(t)](s − 1) ∂  G∂s(s, t) .

Put s = 1 and remember that

2

µ(t) =

Gs (s, t). Then µ

 (t) = [λ(t) − µ(t)]µ(t).

Hence integration of this differential equation gives

 

t

µ(t) =

n0 exp

[λ(u)

0

98



− µ(u)]du

.

Chapter 7

Queues 7.1. In a single-server queue a Poisson process for arrivals of intensity  12 λ and for service and departures of intensity  λ are assumed. For the corresponding stationary process find  (a) pn , the probability that there are n persons in the queue, (b) the expected length of the queue, (c) the probability that there are not more than two persons in the queue, including the person being  served in each case. (a) As in Section 7.3, with µ = 12 λ  pn = (1

− ρ) ρ

n

ρ=

,

λ 1 = . µ 2

In this case pn = 1 /2n+1. (b) If  N  is the random variable of the number n of persons in the queue (including the person being served), the its expected value is (see Section 7.3(b)) 1

E(N ) =

− ρ = 1. ρ

(c) The probability that there are not more than two persons in the queue is  p0 + p1 + p2 =

1 1 1 7 + + = . 2 4 8 8

7.2. Consider a telephone exchange with a very large number of lines available. If  n lines are busy the probability that one of them will become free in small time δt is nµδt. The probability of a new call is λδt (that is, Poisson), with the assumption that the probability of multiple calls is negligible. Show that  pn (t), the probability that  n lines are busy at time t satisfies  p0 (t) =  pn (t) =

−(λ + nµ) p

n (t) +

−λp (t) + µp (t), 0

1

λpn−1 (t) + ( n + 1)µpn+1 (t),

In the stationary process show by induction that  e−λ/µ  pn = lim  pn (t) = t→∞ n!

 λ µ

(n

≥ 1).

n

.

Identify the distribution. If  pn = limt→∞ pn (t) (assumed to exist), then the stationary process is defined by the difference equations λp0 + µp1 = 0 ,

−

99

(n + 1)µpn+1 Assume that

− (λ + nµ) p

e−λ/µ  pn = n!  pn+1

=

1 [(λ + nµ) pn µ(n + 1)

  

=

1 µ(n + 1)

=

e−λ/µ µ(n + 1)!

=

e−λ/µ (n + 1)!

λ µ

+ λpn−1 = 0 .



n

λ µ

.

− λp − ] n 1

λ + µn −λ/mu e n! λ µ

n

n

[λ + nµ

− λ µ

n

λ

(n

− 1)!

  λ µ

n 1

−

− nµ]

n+1

Hence if the formula is true for pn and pn−1 , then it is true for pn+1 . It can be verified that that p1 and  p2 are correct: therefore by induction on the positive integers the formula is proved. The distribution is Poisson with parameter λ/µ. 7.3. For a particular queue, when there are n customers in the system, the probability of an arrival in the small time interval  δt is λn δt + o(δt ). The service time parameter  µn is also a function of  n. If  pn denotes the probability that there are n customers in the queue in the steady state queue, show by induction that   pn = p0

λ0 λ1 . . . λn−1 , µ1 µ2 . . . µn

(n = 1 , 2, . . .)

and find an expression for  p0 . If  λn = 1 /(n + 1) and  µn = µ, a constant, find the expected length of the queue. Let pn (t) be the probability that there are n persons in the queue. Then, by the usual arguments,  p0 (t + δt ) = µ1 p1 (t) + (1  pn (t + δt ) = λn−1 pn−1 (t)δt + µn+1 pn+1

Divide through by δt , and let δt

→ 0:

− µ δt) p (t), (t)δt + (1 − λ δt − µ δt ) p 1

0

n

n

n (t),

(n = 1 , 2, . . .).

 p0 (t) = µ1 p1 (t),

 pn (t) = µn+1 pn+1 (t)

− (λ

n

+ µn ) pn (t) + λn−1 pn−1 (t).

Assume that a limiting stationary process exists, such the pn = limt→∞ pn (t). Then pn satisfies µ1 p1 µn+1 pn+1

− (λ

n

− λ  p

0 0

= 0,

+ µn ) pn + λn−1 pn−1 = 0 .

Assume that the given formula is true for pn and pn−1 . Then, using the difference equation above,  pn+1

= =

1 µn+1



λ0 λ1 . . . λn−1 (λn + µn ) µ1 µ2 . . . µn

−

λ0 λ1 . . . λn−2 λn−1 µ1 µ2 . . . µn−1

λ0 λ1 . . . λn  p0 , µ1 µ2 . . . µn+1

Showing that the formula is true for pn+1 . It can be verified directly that  p1 =

λ0 , µ1 p0

p2 =

Induction proves the result for all n.

100

λ0 λ1  p0 . µ1 µ2



The probabilities satisfy



∞  p = 1. Therefore n=0 n ∞



1 = p0 + p0

n=1

λ0 λ1 . . . λn−1 , µ1 µ2 . . . µn

provided that the series converges. If that is the case, then  p0 = 1

  ∞

1+

n=1



λ0 λ1 . . . λn−1 . µ1 µ2 . . . µn

If  λn = 1 /(n + 1) and µn = µ, then  pn =

where  p0 = 1

p0 µn

· 11 · 12 · · · n1 = µp n! , 0 n

   ∞

1+

1 µn n!

n=1

Hence

 pn =

= e−1/µ .

e−1/µ . µn n!

which is Poisson with parameter 1 /µ. The expected length of the queue is the usual result for the mean: µ

=

∞



npn = e−1/µ

n=1

∞

 n=1

1 µn (n

−

1 1 = e−1/µ e1/µ = . µ µ 1)!

7.4. In a baulked queue (see Example 7.1) not more than  m 2 people are allowed to form a queue. If  there are m individuals in the queue, then any further arrivals are turned away. If the arrivals form a  Poisson process with parameter  λ and the service distribution is exponential with parameter  µ, show that  the expected length of the queue is ρ (m + 1)ρm+1 + mρm+2 , (1 ρ)(1 ρm+1 )

≥

−

−

−

where ρ = λ/µ. Deduce the expected length if  ρ = 1. What is the expected length of the queue if  m = 3 and  ρ = 1?  How doeas the expected length of the baulked queue behave as ρ becomes large? From Example 7.1, the probability of a baulked queue having length n is  pn =

ρn (1 ρ) , (1 ρm+1 )

−

−

(n = 0 , 1, 2, . . . , m),

(ρ = 1) .



The expected length is m

E(N ) =

∞

  npn =

n=1

n=1

nρn (1 ρ) = (1 ρm+1 )

−

−

Let



1 1

−ρ

−ρ

m

 

S  =

nρn .

n=1

Then

m

(1

− ρ)S  =

ρn

n=1

101

− mρ

m+2

.

m+1

 m

n=1

nρn .

(i)

Further summation of the geometric series gives S  =

m+1

m+2

− (m + 1)ρ + mρ (1 − ρ)

ρ

,

2

so that the expected length of the queue is ρ

E(N ) =

m+1

− (m + 1)ρ + mρ (1 − ρ)(1 − ρ )

m+2

(ρ = 1) .

,

m+1



(ii)

If  ρ = 1, then, applying l’Hˆ opital’s rule in calculus to (i)





d n  pn = [ρ (1 dρ

− ρ)]

d [1 dρ

−ρ

m+1



]

= ρ=1

1 . m+1

In this case the expected length is m

E(N ) =

 n=1

1 npn = m+1

m



1 (m + 1)

n=

n=1

· 12 m(m + 1) = 12 m,

using an elementary formula for the sum of the first m integers. If  ρ = 1 and m = 3, then E(N ) = 3 /2. The expected length in (ii) can be re-arranged into E(N ) =

as ρ

ρ−m−1 (m + 1)ρ−1 + m (ρ−1 1)(ρ−1 1)

− −

−

→ m,

→ ∞. For the baulked queue there is no restriction on ρ.

7.5. Consider the single-server queue with Poisson arrivals occurring with parameter  λ, and exponential  service times with parameter  µ. In the stationary process, the probability  pn that there are n individuals in  the queue is given by  n λ λ  pn = 1 , (n = 0 , 1, 2, . . .). µ µ

 −   

Find its probability generating function 

G(s) =

∞

 pn sn .

n=0

If  λ < µ, use this function to determine the mean and variance of the queue length. The probability generating function G(s, t) is defined by G(s, t) =

   −    ∞

n=0

λ µ

n

λ µ

sn

n+1

sn .

Summation of the two geometric series G(s, t) =

µ

µ λ µ µ−λ − λs − µ · µ − λs = µ − λs .

The first two derivatives of  G(s, t) are Gs (s, t) =

λ(µ λ) , (µ λs)2

− −

Gss (s, t) = 2

Hence the mean and variance are given by E(N ) = Gs (1, t) =

102

λ

µ

− λ,

λ2 (µ λ) (µ λs)3

−

−

V(N ) = Gss (1, t) + Gs (1, t)

2

− [G (1, t)] s

2λ2 λ + 2 (µ λ) µ λ

=

2

λ − − ( µ − λ)

−

2

=

λµ (µ λ)2

−

7.6. A queue is observed to have an average length of 2.8 individuals including the person being served. Assuming the usual exponential distributions for both service times and times between arrivals, what is the traffic density, and the variance of the queue length?  With the usual parameters λ and µ and the random variable N  of the length of the queue, then its expected length is, with ρ = λ/µ, ρ E(N ) = , 1 ρ which is 2.8 from the data. Hence the traffic density is ρ = 2 .8/3.8 0.74. The probability that the queue length is n in the stationary process is (see eqn (7.5)) pn = (1 ρ)ρn . The variance of the queue length is given by

−

V(N )

= = =

If  ρ = 0 .74, then V(N )

2

E(N  )

− [E(N )]

ρ(1 + ρ) (1 ρ)2 ρ (1 ρ)2

− −

2

= (1

≈

− ρ)

2

− (1 −ρ ρ)

∞



−

n 2 ρn

n=1

2

− (1 −ρ ρ)

2

2

≈ 10.9.

7.7. The non-stationary differential-difference equations for a queue with parameters λ and  µ are (see equation (7.1)) dp0 (t) = µp1 (t) λp0 (t), dt dpn (t) = λpn−1 (t) + µpn+1 (t) (λ + µ) pn (t), dt where pn (t) is the probability that the queue has length  n at time t. Let the probability generating function  of the distribution   pn (t) be

−

−

{

}

G(s, t) =

∞



 pn (t)sn .

n=0

Show that  G(s, t) satisfies the equation  s

∂G (s, t) = (s ∂t

− 1)(λs − µ)G(s, t) + µ(s − 1) p (t). 0

Unlike the birth and death processes in Chapter 6, this equation contains the unknown probability  p0 (t) which complicates its solution. Show that it can be eliminated to leave the following second-order partial  differential equation for  G(s, t): 2

s(s

G(s, t) − 1) ∂  ∂t∂s − (s − 1) (λs − µ) ∂G∂s(s, t) − G(∂ts, t) − λ(s − 1) G(s, t) = 0 . 2

2

This equation can be solved by Laplace transform methods. Multiply the second equation in the question by sn , sum over all n from n = 1 and add the first equation to the sum resulting in

∞



 pn (t) = µp1 (t) − λp0 (t) + λ

n=0

= =

∞



n

 pn−1 (t)s + µ

n=1

∞



 pn+1 (t)s

n=1

µ µp1 (t) λp0 (t) + λsG(s, t) + [G(s, t) sp1 (t) s µ 1 (s 1)(λs µ)G(s, t) + (s 1) p0 (t). s s

−

−

−

−

−

103

n

− (λ − µ)

∞



 pn (t)sn

n=1

− p (t)] − (λ + µ)[G(s, t) − p (s, t)] 0

0

Hence the differential equation for G(s, t) is s

∂G (s, t) = (s ∂t

− 1)[(λs − µ)G(s, t) + µp (t)]. 0

Write the differential equation in the form ∂G (s, t) = ( λs ∂t 1

s s

−

− µ)G(s, t) + µp (t). 0

Differentiate the equation with respect to s to eliminate the term p0 (t), so that ∂  ∂s

or

− (s −1 1)

2



s s

−

∂G (s, t) ∂t 1



=

∂  [(λs ∂s

− µ)G(s, t)],

∂G (s, t) s ∂ 2 G(s, t) + = λG(s, t) + ( λs ∂t s 1 ∂s∂t

−

The required result follows.

− µ) ∂G∂s(s, t) .

7.8. A call centre has r telephones manned at any time, and the traffic density is λ/(rµ ) = 0 .86. Compute how many telephones should be manned in order that the expected number of callers waiting at any time should not exceed  4? Assume a stationary process with inter-arrival times of calls and service times for all  operators both exponential with parameters λ and  µ respectively (see Section 7.4). From (7.11) and (7.12), the expected length of the queue of callers, excluding those being served, is E N

8

6

4

2

r 2

4

6

8

10

Figure 7.1: Expected queue length E(N ) versus number r of manned telephones.

E(N ) =

where

  r 1

 p0 = 1

−

n=0

(r

−

n

ρ + n! (r

p0 ρr+1 , 1)!(r ρ)2 r

−

(i)

−

ρ ρ)(r

− 1)!



,

ρ=

λ . µ

(ii)

Substitute for p0 from (ii) into (i) and compute E(N ) as a function of  r with ρ = 0 .86r . A graph of  E(N ) against r is shown in Figure 7.1 for r = 1 , 2, . . . , 10. From the graph the point at r = 6 is (just) below the line E(N ) = 4. The answer is that 6 telephones should be manned. 7.9. Compare the expected lengths of the two queues M (λ)/M (µ)/1 and  M (λ)/D (1/µ)/1 with  ρ = λ/µ < 1. The queues have parameters such that the mean service time for the former equals the fixed service time in the latter. For which queue would you expect the mean queue length to be the shorter? 

104

From Section 7.3(b), the expected length of the M (λ)/M (µ)/1 queue is, with ρ = λ/µ < 1, ρ

E1 (N ) =

1

− ρ.

Since λτ  = λ/µ = ρ (τ  is the fixed service time), the expected length of the M (λ)/D (1/µ)/1 is (see end of  Section 7.5) ρ(1 12 ρ) E2 (N ) = . 1 ρ It follows that 1 2 ρ ρ ρ 2 = E1 . E2 (N ) = 1 ρ 1 ρ 1 ρ In this case the queue with fixed service time has the shorter expected length.

− −

− − − ≤ −

7.10. A queue is serviced by  r servers, with the distribution of the inter-arrival times for the queue being  exponential with parameter  λ and each server has a common exponential service time distribution with  parameter  µ. If  N  is the random variable for the length of the queue including those being served, show  that its expected value is

 −   r 1

E(N ) = p0

−

n=1

where ρ = λ/µ < r , and 

ρn

(n

r 1

 p0 = 1

−

n=0

(see equation (7.11)). If  r = 2, show that 



ρr [r 2 + ρ(1 r)] , + 1)! (r 1)!(r ρ)2

− −

−

ρn + n! (r

−

ρr ρ)(r

− 1)!



.

4ρ . 4 ρ2 For what interval of values of  ρ is the expected length of the queue less than the number of servers?  E(N ) =

−

For the M (λ)/M (µ)/r queue, the probability that there n persons in the queue is  pn =



ρn p0 /n! ρn p0 /(rn−r r !)

n< r , n r

≥

where p0 is given in the question. The expected length of the queue including those being served is E(N ) =

∞



       −   − −   − − − − r 1

npn = p0

n=1

−

n=1

Consider the series

nρn + n!

∞

n=r

R=

r 1

nρn rn−r r !

∞

n=1

ρ r

n

n=r

−

= p0

ρn

(n

rr + 1)! r!

n

.

Using the method for summing geometric series twice ρ r

R=

Hence

r 1

E(N ) = p0

−

n=1

as required. If  r = 2 (two servers) then

 p0 = 1



r

r(r2 + ρ ρr ) . (r ρ)2

ρn

(n

1)!

1+ρ+

+

ρr [r 2 + ρ(1 r)] , (r 1)!(r ρ)2

ρ2

(2

105

− ρ)



=

2 ρ , 2+ρ

−

∞

n=r

n

ρ r

n

.

and E(N ) = p0



ρ2 (4 ρ) ρ+ (2 ρ)2

− −



=

4ρ . 4 ρ2

−

The expected length of the queue is less than the number of servers if  4ρ − ρ < 2, or ρ + 2ρ − 4 < 0. √ √ + 2ρ − 4 = 0 are ρ = −1 ± 5. The required range for ρ is 0 < ρ < 5 − 1. 2

4

The roots of  ρ2

2

probability  ility  pn that there are n servers in the queue, including those 7.11. For a queue with two servers, the probab being served, is given by  2 ρ , 2+ρ

−

 p0 =

p1 = ρp0 ,

pn = 2

 ρ 2

n

(n

 p0 ,

≥ 2),

where ρ = λ/µ (see (see Section Section 7.4). If the random random variable X is the number of people in the queue, find its probability generating function. Hence find the mean length of the queue including those being served. The probability probability generating generating function for this queue with two servers is, summing summing the geometric geometric series, G(s)

   −   −  

=

p0 + p0 ρs + 2 p0

∞

ρ 2

n=2

2 ρ 2+ρ

=

2 + ρs 2 ρs

2 ρ 2+ρ

n

s =

ρ2 s2 1 + ρs + 2 ρs

−



,

−

assuming that 0 ρs < 2. For the mean we require

n

≤

G (s) =

Then the mean length of the queue µ

− 2 ρ 2+ρ

= G (1) =

4ρ . (2 ρs)2

−

4ρ , (4 ρ2 )

−

which agrees with the result from Problem 7.10.

7.12. The queue M (λ)/D(τ )/1, which has a fixed service time of duration  τ  for every customer, has the probability generating function  (1 ρ)(1 s) G(s) = , 1 seρ(1−s) where ρ = λτ  (0 < ρ < 1) (see Section 7.5). (a) Find the probabilities p0 , p1 , p2 . (b) Find the expected value and variance of the length of the queue. (c) Customers are allowed a service time τ  which is such that the expected length of the queue is two individuals. Find the value of the traffic density  ρ.

− −

−

(a) The Taylor aylor series for G(s) is G(s) = (1

ρ

Hence  p0 = 1

ρ

ρ 2

− ρ) + (1 − ρ)(e − 1)s + (1 − ρ)(e − ρ − 1)e s

− ρ,

p1 = (1

ρ

− ρ)(e − 1),

p2 = (1

(b) The first two derivatives of  G(s) are



esρ (−1 + ρ) esρ + eρ G (s) =

(esρ

106

−

ρ

2

ρ

− ρ)(e − ρ − 1)e .

1 + sρ

− e s)

ρ

+ O (s3 ).

2



−s ρ

,

 

 −

eρ+sρ (−1 + ρ) esρ 2 + (2 − 4s)ρ + (−1 + s)sρ2 + eρ  G (s) =

(esρ

3

ρ

− e s)

It follows that the mean and variance are



E(N ) = G (1) =

ρ(2 2(1

2 + 2 sρ

2 2

−s ρ

+ s 3 ρ2



.

− ρ) , − ρ)

− [G (1)] = 12(1 ρ− ρ) (12 − 18ρ + 10ρ − ρ ). (c) We require G (1) = 2, or 2 − 2ρ = ρ − ρ . Henc Hencee ρ − 6ρ + 4 = 0. The roots roots of this this quadrati quadraticc √ √ equation are ρ = λτ  = 3 ± 5: the density is ρ = 3 + 5 since ρ must be less than 1. 

2



V(N ) = G (1) + G (1)

2

3

2

1 2 2

2

7.13. For the baulked queue which has a maximum length of  m beyond which customers are turned away, The probabilities that there are n individuals in the queue are given by   pn =

ρn (1 ρ) , 1 ρm+1

−

−

(0 < ρ < 1),

pn =

1 m+1

(ρ = 1) ,

 for  n = 0 , 1, 2, . . . , m. Show that the probability probability generating generating functions are G(s) =

(1 ρ)[1 (ρs)m+1 ] , (1 ρm+1 )(1 ρs)

− −

−

and 

(ρ = 1) ,



−

1 sm+1 , (ρ = 1) . (m + 1)(1 s) Find the expected value of the queue length including the person being served.

−

G(s) =

−

The probability generating function G(s) of the baulked queue is defined by, for ρ = 1,



  −−   m

G(s) =

1

n

 pn s =

ρm+1

1

n=0

m

ρ

ρn sn =

n=0

(1 ρ)(1 ρm+1sm+1) , (1 ρm+1 )(1 ρs)

−

−

−

−

using the formula for the sum of the geometric series. The first derivative of  G(s) is G (s) =

(1

m

m

− ρ)ρ (1 − (sρ) − m(sρ) (1 − sρ)) . (1 − sρ) (1 − ρ ) 2

1+m 1+m

The expected length of the queue is



E(N ) = G (1) =

If  ρ = 1, then G(s) =

∞

n

n=0

Its first derivative is

1+m 1+m

1  pn s = m+1



G (s) =

1

1+m 1+m

(1 + m)ρ − 1−ρ 1−ρ ρ

m



sn =

n=0

1 sm+1 . (m + 1)(1 s)

−

−

m

1+m + ms1+m . (1 + m)(1 s)2

− (1 + m)s

The expected length of the queue is E(N ) = G (1) = 12 m.

−

7.14. In Section 7.3(ii), the expected length of the queue with parameters λ and  µ, including the person  being served was shown to be ρ/(1 ρ). What is the expected length of the queue excluding the person being  served? 

−

107

The expected length of the queue excluding the person being served is given by E(N )

∞

∞

− − −  − − 

=

(n

1) pn =

n=2

∞

n

nρ

n=2

2 ρ2

=

ρn+1 )

n=2

∞

=

1)(ρn

(n

ρ

∞

n

n=2

n

nρ + 2

n=3

2

∞

ρn

n=3

3

− 1 ρ− ρ + 12−ρ ρ

ρ2

=

1

−ρ

observed over a long period period of time. Regular Regular sampling sampling indicates indicates that the 7.15. An  M (λ)/M (µ)/1 queue is observed mean length of the queue including the person being served is 3, whilst the mean waiting time to completion  of service by any customer arriving is 10 minutes. What is the mean service time?  From Section 7.3, the mean length of the queue is 3, and the mean time to service is 10 minutes: hence ρ

1 Hence ρ=

−ρ 3 4

ρ

= 3,

µ(1

and µ =

− ρ) = 10 .

3 ρ = . 10(1 ρ) 10

−

7.16. An  M (λ)/G/1 queue has a service time with a gamma distribution with parameters n and  α where n is a positive integer. The density function of the gamma distribution is f (t) =

αn tn−1 e−αt , (n 1)!

−

t > 0.

Show that, assuming  ρ = λn/α, the expected length of the queue including the person being served is given  by  ρ[2n (n 1)ρ] . 2n(1 ρ)]

− − −

The formula for the expected length of queue is given by (7.25): E(X ) =

2ρ

+ λ2 V(S ) , 2(1 ρ)

−ρ

2

−

where ρ = λE(S ), ), and S  is the random variable variable of the service time. time. Since S  has a gamma distribution with parameters n and α, then E(S ) =

n , α

V(S ) =

Hence E(X ) =

n , α2

ρ = λE(S ) =

λn . α

2λnα λ2 n2 + λ2 n . 2α(α λn)

−

This result assumes λ < α/n.

−

7.17. The service in a single-server queue must take at least 2 minutes and must not exceed 8 minutes, and  the probability that the service is completed at any time between these times is uniformly distributed over  the time interval. The average average time betwee between n arrival is 4 minutes, and the arrival distribution distribution is assumed  assumed  to be exponentia exponential. l. Calculate Calculate expecte expected d length of the queue.

108

The density of the uniform distribution is



f (t) =

1/(τ 2 0

− τ  )

≤ ≤

τ 1 t τ 2 elsewhere

1

The expected length of the queue is given by (7.25). In this formula, we require E(S ) and V(S ). They are given by ∞ τ  1 tdt = (τ 2 + τ 1 ). E(S ) = tf (t)dt = 2 τ 2 τ 1 −∞ τ 

 

 

2

−

1

By (7.21), it follows that the traffic density is ρ = λE(S ) =

 

τ 2

V(S )

=

2

E(S  )

− [E(S )]

2

=

τ 1

1 λ τ  2 ( 2

− τ  ). The variance is given by 1

t2 dt τ 2 τ 1

1 − (τ  − τ  ) − 4 2

1

2

1 2 1 = (τ 2 + τ 1 τ 2 + τ 12 ) (τ 2 + τ 1 )2 3 4 ρ2 1 = (τ 2 τ 1 )2 = 2 12 3λ The formula is simpler if (7.25) is expressed as a function of the traffic density ρ. Thus, substituting for V(S ), the expected queue length is for ρ < 1,

−

−

E(X ) =

In the particular case τ 2

− τ 

1

2ρ

2ρ ρ2 + 13 ρ2 + λ2 V(S ) ρ(3 = = 2(1 ρ) 2(1 ρ) 3(1

−ρ

2

−

−

− ρ) − ρ) .

(i)

= 4 minutes. Hence the expected time between arrivals is

 

∞

λte−λt dt =

0

1 4

−

1 . λ

(minutes)−1 . Since τ 2 − τ 1 = 6 minutes, then ρ =

Hence λ = in eqn (i) becomes

3 (1 4

− E(X ) = 3(1 −

3 ) 4 3 ) 4

=

1 2

1 4

· ·6=

3 . 4

Finally, the expected length

9 . 4

7.18. A person arrives at an  M (λ)/M (µ)/1 queue. If there are two people in the queue (excluding customers being served) the customer goes away and does not return. If there are there are fewer than two queueing  then the customer joins the queue. Find the expected waiting time for the customer to the start of service. This a is a baulked queue with baulking after 2 customers in the queue but not being served. The method is similar to that in Section 7.3(c). Let T i , (i = 1 , 2) be the random variable representing the time for customer i to reach the service position. Let S 1 and S 2 be the random variables defined by S 1 = T 1 ,

S 2 = T 1 + T 2 .

As explained in Section 7.3(c), the densities of  S 1 and S 2 are gamma with parameters respectively µ, 1 and µ, 2, are (i) f 1 (t) = µe−µt (t > 0), f 2 (t) = µ2 te−µt , (t > 0). From Example 7.1, the probability that there are n persons in the queue is  pn =

Then, using (i) and (ii) P(S 1 + S 2 > T  )

ρn (1 ρ) , 1 ρ3

− −

(n = 0 , 1, 2).

=

P(S 1 > t) p1 + P(S 2 > t ) p2

=

ρ(1 ρ) 1 ρ3

= =

   

(ii)

 

− ∞ f  (s)ds + ρ (1 − ρ) ∞ f  (s)ds − 1−ρ ∞ ∞ ρ(1 − ρ) µe− ds + ρ µ se− ds 1−ρ ρ(1 − ρ) − e [1 + ρµt + ρ]. 1−ρ 3

2

1

3

t

t

µt

3

109

µs

  t

2

t

2

µs



The density associated with this probability is

−

d g(t) = 1 dt



ρ(1 ρ) −µt ρ(1 ρ) e µ(1 + ρµt)e−µt . [1 + ρµt + ρ] = 3 3 1 ρ 1 ρ

− −

− −

Finally, the expected waiting time for the customer to start service is E(S 1 + S 2 ) =

 

∞

0

ρ(1 tg (t)dt = 1

− ρ)µ −ρ 3

 

∞

ρ(1

(t + ρµt2 )e−µt dt =

− ρ)(1 + 2ρ) = ρ(1 + 2ρ) 1−ρ 1+ρ+ρ 3

0

2

7.19. A customer waits for service in a bank in which there are four counters with customers at each counter  but otherwise no one is queueing. If the service time distribution is negative exponential with parameter  µ  for each counter, for how long should the queueing customer have to wait on average?  For each till, the density is f (t) = µe−µt . The corresponding probability function is

 

t

F (t) =

 

t

f (s)ds =

0

µe−µs ds = 1

0

− e−

µt

.

Let T  be random variable of the time to complete service at any till. Then the probability that T  is greater than time t is P(T > t )

=

P(T > t for till 1)P(T > t for till 2)P(T > t for till 3)P(T > t for till 4)

=

e−µt e−µt e−µt e−µt = e−4µt .

Hence the density of  T  is g (t) = 4 µe−4µt (t > 0), which is exponential with parameter 4µ. The mean time for a till becoming available is ∞ 1 E(T ) = , 4µte−4µt dt = 4µ 0

 

after integrating by parts.

7.20. A hospital has a waiting list for two operating theatres dedicated to a particular group of operations. Assuming that the queue is stationary, and that the waiting list and operating time can be viewed as an  M (λ)/M (µ)/1 queue, show that the expected value of the random variable N  representing the length of the queue is given by  ρ3 λ E(N ) = , ρ = < 2. 2 4 ρ µ The waiting list is very long at 100 individuals. Why will  ρ be very close to 2? Put  ρ = 2 ε where ε > 0 is small. Show that  ε 0.02. A third operating theatre is brought into use with the same operating  parameter  µ. What effect will this new theatre have on the waiting list eventually? 

−

−

≈

In Section 7.4, apply eqns (7.11) and (7.12) (in the book) with r = 2 giving for the expected length of  the queue of patients as p0 ρ3 E(N ) = , (2 ρ)2 where ρ2 2 ρ  p0 = 1 . 1+ρ+ = 2 ρ 2+ρ

−



−

Elimination of  p0 between these equations leads to E(N ) =

For E(N ) to be large, 4

−ρ

2

ρ3 , 4 ρ2

−



−

λ < 2.

is small. Since 0 < ρ < 2, let ρ = 2 E(N ) =

2(1 ε(1

− −

1 ε 2 ) 1 ε 4 )

110

=

− ε with ε small. Then

1 + O(1). ε

as ε 0. If  E(N ), then ε 2/100 = 0.02. If  r = 3 (which will require λ/µ < 3), then

→

≈

p0 ρ4 , 2(3 ρ)2

E(N ) =

where  p0 = 1

Therefore





ρ3 1 1 + ρ + ρ2 + . 2 2(3 ρ)

E(N ) =

If  ρ = 2

−

−

(3

− ε, then the expected length is E(N )

−

ρ4 . ρ)(6 + 4ρ + ρ2 )

≈ 89 + O(ε),

which is a significant reduction. 7.21. Consider the M (λ)/M (µ)/r queue which has r servers such that  ρ = λ/µ < r . Adapting the method   for the single-server queue (Section 7.3 (iii)), explain why the average service time for  (n r +1) customers to be served is (n r + 1)/(µr ) if  n r. What is it if  n < r? If  n r , show that the average value of the waiting time random variable T  until service is

−

≥

−

≥

E(T ) =

∞



n

− r +1p

n.

rµ

n=r

What is the average waiting time if service is included?  If  n r 1, then the arriving customer has immediate service. If  n r, the customer must wait until n r + 1 preceding customers have been served, and they take time (n r + 1)/(rµ) since the mean service time is 1/(rµ ) for r servers. The expected value of the service time T  until service is

≤ −

−

−

E(T ) =

∞



n

≥

− r +1p

n,

rµ

n=r

where (see Section 7.4)

  r 1

ρn p0  pn = n−r , r r!

p0 = 1

−

n=0

ρn + (r n!

−

ρr ρ)(r

− 1)!



.

Then E(T )

= =

ρr p0 µrr ! µ(r

∞

 n=r r

−

(n

ρ  p0 1)!(r

−



ρ r + 1) r

n r

−

     · · ·

ρr p0 = 1+2 µrr !

ρ r

+3

ρ r

2

+

2

− ρ)

The mean time represented by random variable T  until service is completed is



E(T  ) = E(T ) +

1 . µ

7.22. Consider the queue M (λ)/M (µ)/r queue. Assuming that  λ < rµ, what is the probability in the long  term that at any instant there is no one queueing excluding those being served? 

111

The probability that no one is queueing is

∞

 −

qr = 1

 pn ,

n=r

where (see Section 7.4) using the formula for the sum of a geometric series,

∞

p0 r r  pn = r!

 n=r

and

∞

    n=r

r 1

 p0 = 1

ρ r

−

n=0

n

=

ρn + (r n!

−

(r

−

ρr ρ)(r

p0 ρr 1)!(r

− 1)!

− ρ) ,



.

7.23. Access to a toll road is controlled by a plaza of  r toll booths. Vehicles approaching the toll booths choose one at random: any toll booth is equally likely to be the one chosen irrespective of the number of  cars queueing (perhaps an unrealistic situation). The payment time is assumed to be negative exponential  with parameter  µ, and vehicles are assumed to approach as Poisson with parameter  λ. Show that, viewed  as a stationary process, the queue of vehicles at any toll booth is an  M (λ/r )/M (µ)/1 queue assuming  λ/(rµ ) < 1. Find the expected number of vehicles queueing at any tol l booth. How many cars would you  expect to be queueing over all booths?  One booth is out of action, and vehicles distribute themselves randomly over the remaining booths. Assuming that  λ/[(r 1)µ] < 1, how many extra vehicles can be expected to be queueing overall? 

−

It is assumed to be an M (λ/r/M (µ)/1) queue. For a random number N n of cars at toll booth n (n = 1 , 2, . . . r) is λ E(N n ) = . µr λ For all booths, the expected number of cars is

−

E(N 1 + N 2 +

· · · N  ) = µrrλ− λ , r

including cars at any booth. If one booth is out of action (say booth r) then the expected number of cars queueing is E(N 1 + N 2 +

· · · N  − ) = µ((rr−−1)1)−λ λ , r 1

Hence the expected extra length of the queue is 2

(r 1)λ µ(r 1) λ

− rλ λ − = − − µr − λ [µ(rt − 1) − λ](µr − λ) .

7.24. In an  M (λ)/M (µ)/1 queue, it is decided that the service parameter  µ should be adjusted to make the mean length of the busy period 10 times the slack period to allow the server some respite. What should  µ be in terms of  λ?  From Section 7.3(d), in the stationary process, the expected length of the slack period is 1 /λ and that of the busy period 1/(µ λ). Therefore the busy period is 10 times of the slack period if 

−

1

µ

−λ =

10 , λ

or

µ=

11 λ. 10

7.25. In the baulked queue (see Example 7.1) not more than  m 2 people (including the person being  served) are allowed to form a queue, the arrivals having a Poisson distribution with parameter  λ. If there are m individuals in the queue, then any further arrivals are turned away. It is assumed that the service

≥

112

distribution is exponential with rate µ. If  ρ = λ/µ = 1, show that the expected length of the busy periods is given by  (1 ρm )/(µ λ).

−



−

For the baulked queue with a limit of  m customers (including the person being served) the probability that the queue is of length n is given by ρn (1 ρ)  pn = , 1 ρn+1

−

−

(n = 0 , 1, 2, . . . , m).

In the notation of Section 7.3(d) for slack and busy periods,

 

1 lim j →∞  j and



1 lim j →∞  j

j

bi =

i=1

1

− p

0

 p0

j

si =

i=1

1 lim j →∞  j

1 , λ

j

si =

i=1

1

− p

λp0

0

=

1 ρm . µ λ

− −

7.26. The M (λ)/D(τ )/1 queue has a fixed service time τ , and from Section 7.5, its probability generating   function is (1 λτ )(1 s) G(s) = . 1 seλτ (1−s) Show that the expected length of its busy periods is τ /(1 λτ ).

− −

−

−

The average length of the slack periods is, in the notation of Section 7.3(d),



1 lim j →∞  j

j

si =

i=1

1 , λ

since it depends on the arrival distribution. Also the average length of the busy periods is



1 lim j →∞  j

j

bi =

1

− p

0

 p0

i=1

From the given generating function p0 = G(0) = 1



1 lim j →∞  j

si =

i=1

1  p0 .  p0 λ

−

− λτ , so that the average length of busy periods is



1 lim j →∞  j

j

j

bi =

i=1

τ 

1

− λτ  .

7.27. A certain process has the (r +1) states E n , (n = 0 , 1, 2, . . . r). The transition rates between state n and  state n + 1 is λn = ( r n)λ, (n = 0 , 1, 2, . . . , r 1), and between  n and  n 1 is µn = nµ, (n = 1 , 2, . . . , r). These are the only possible transitions at each step in the process. [This could be interpreted as a ‘capped’  birth and death process in which the population size cannot exceed  r.]  Find the differential-difference equation for the probabilities pn (t), (n = 0 , 1, 2, . . . , r), that the process is in state n at time t. Consider the corresponding staionary process in which  dpn /dt 0 and  pn (t)  pn as t . Show that  n λ r µr  pn = , (n = 0 , 1, 2, . . . , r). µ n (λ + µ)r

−

−

−

→∞

→∞

 

In (6.25), let λn = ( r

− n)λ,

(n = 0 , 1, 2, . . . , r

− 1),

113

µn = nµ,

(n = 1 , 2, . . . , r),

→

so that the finite system of differential-difference equations are  p0 (t) = λrp0 (t) + µp1 (t),  pn (t) = (r

− n + 1)λp − (t) − [(r − n) + nµ] p n 1

 p (t) = λp r

n (t) + ( n

r 1 (t)

−

+ 1)µpn+1 (t),

(n = 1 , 2, . . . , r

− rµp (t). r

The time-independent stationary process satisfies the difference equations λrp0 + µp1 = 0 ,

(r

− n + 1)λp − − [(r − n) + nµ] p n 1

n

+ (n + 1)µpn+1 = 0 ,

− rµp

λpr−1

Let un = λ(r

− n) p − (n + 1)µp n

u0 = 0 ,

The the difference equations become

un−1

−u

n

= 0,

 pn+1

(n = 1 , 2, . . . , r

− 1). Hence λ r−n =

λ µ

n

r (r + 1)

n+1

µ

· · · (r − n + 1) p

0

n!

− 1),

   

Repeated application of this formula gives  pn =

= 0.

n+1 .

We conclude that un = 0 for (n = 0 , 1, 2, . . . , r



r

(n = 1 , 2, . . . , r

λ µ

=

ur−1 = 0 .

 pn .

n

r  p0 , n

(n = 0 , 1, 2, . . . , r).

The probability p0 is defined by

       r

1=

r

 pn =

n=0

n=0

λ µ

Hence  p0 =

so that, finally  pn =

n

r  p0 = p0 n

µr , (λ + µ)r

  λ µ

n

r µr . n (λ + µ)r

114

− 1),

λ 1+ µ

r

.

− 1),

Chapter 8

Reliability and renewal 8.1. The lifetime of a component has a uniform density function given by  f (t) =



1/(t1 0

−t ) 0

0 < t0 < t < t 1 otherwise

For all  t > 0, obtain the reliability function  R(t) and the failure rate function  r (t) for the component. Obtain the expected life of the component. For the given density, the distribution function is F (t) =



0 (t 1

0 t t0 t0 < t < t 1 t t1

≤ ≤ ≥

− t )/(t − t ) 0

1

0

Therefore the reliability function R(t) is R(t) = 1



1 (t1 0



0 1/(t1 t) does not exist

− F (t) =

0 t t0 t0 < t < t 1 , t t1

≤ ≤ ≥

− t)/(t − t ) 1

0

and the failure rate function r(t) is f (t) r(t) = = R(t)

−

0 t t0 t0 < t < t 1 . t t1

≤ ≤ ≥

Failure of the component will not occur for 0 t t0 since R(t) = 1 in this interval. The component will not survive beyond t = t1 . If  T  is a random variable of the lifetime of the component, then the expected lifetime is given by

≤ ≤

E(T ) =

 

∞

 

t1

tf (t)dt =

0

t0

tdt t2 = 1 t1 t0 2(t1

−

2 0

− t = 1 (t −t ) 2 0

0

+ t1 ).

8.2. Find the reliability function  R(t) and the failure rate function  r (t) for the gamma density  f (t) = λ2 te−λt ,

t > 0.

How does r(t) behave for large t? Find the mean and variance of the time to failure. For the given gamma density,

 

t

F (t) = λ

2

se−λs ds = 1

0

115

− e−

λt

(1 + λt).

Hence the reliability function is R(t) = 1

The failure rate function is

− F (t) = (1 + λt)e−

r (t) =

For fixed λ and large t,

as t

→ ∞.

.

f (t) λ2 t . = R(t) 1 + λt

  

r (t) = λ 1 +

λt

1 λt

−1

= λ + O(t−1 ),

If  T  is a random variable of the time to failure, then its expected value is E(T ) =

 

∞

sf (s)ds = λ

2

0

 

∞

s2 e−λs ds =

0

2 . λ

Also the variance V(T ) =

 

∞

2

s f (s)ds

0

−    2 λ

2

∞

= λ2

s3 e−λs ds

0

− λ4

2

=

6 λ2

− λ4

2

=

2 . λ2

Both these are results for gamma mean and variance. 8.3. A failure rate function is given by  r (t) =

t , 1 + t2

t

≥ 0,

The rate of failures peaks at  t = 1 and then declines towards zero as t : failure becomes less likely  with time (see Figure 8.1 ). Find the reliability function, and the corresponding probability density.

→∞

Figure 8.1: Failure rate distribution r(t) with a = 1 and c = 1

In terms of  r (t) (see eqn (8.5), the reliability function is given by

−    −    t

R(t) = exp

t

r(s)ds = exp

0

for t

0

s ds 1 + s2

= exp[

−

≥ 0. Hence the probability function 1 F (t) = 1 − R(t) = 1 − √ , (1 + t )

(t

2

Finally, the density is given by f (t) = F  (t) =

116

t

(1 + t2 )

3 2

1 2

.

ln(1 + t2 )] =

≥ 0).

√(1 1+ t ) , 2

8.4. A piece of office equipment has a piecewise failure rate function given by 



r(t) =

2λ1 t, 2(λ1 λ2 )t0 + 2λ2 t,

0 t0

−

≤t , 0

λ1 , λ2 > 0.

Find its reliability function. The reliability function is given by

−              − t

R(t) = exp

r (s)ds ,

0

where, for 0 < t

≤t , 0

t

t

sds = λ1 t2 ,

r(s)ds = 2 λ1

0

and, for t > t0 ,

 

t

0

t0

r(s)ds

=

0

t

r(s)ds +

r(s)ds

0

t0

t

=

λ1 t20

+

[2(λ1

λ2 )t0 + 2λ2 s]ds

t0

=

λ1 t20 + 2(λ1

=

t0 (λ1

2

2 0

− λ )t (t − t )] + λ (t − t ) − λ )(2t − t ) + λ t 2

2

0

0

2

0 2

2

Hence the reliability function is R(t) =



e−λ e−[t

1

t2

0

(λ1

−λ

2

0 < t t0 t > t0 .

≤

)(2t t0 )+λ2 t2 ]

−

8.5. A laser printer is observed to have a failure rate function  r(t) = 2λt (t > 0) per hour whilst in use, where λ = 0.00021(hours)−2 : r(t) is a measure of the probability of the printer failing in any hour given  that it was operational at the beginning of the hour. What is the probability that the printer is working  after  40 hours of use? Find the probability density function for the time to failure. What is the expected  time before the printer will need maintenance?  Since r (t) = 2 λt, the reliability function is

−    t

R(t) = exp

2

sds = e−λt .

2λ

0

Hence R(40) = e−0.00021×40×40 = 0.715. The probability of that the printer is working after 40 hours is 0.715. The probability of failure is F (t)1 R(t), so that F (t) = 1 e−λt . By (8.8), the expected time T  of the random variable to failure is

−

E(T ) =

 

∞

−

R(t)dt =

0

 

∞

2

e−λt dt =

0

(see the Appendix for the value of the integral).

1 2

2

 

π = 61 .2 hours, λ

8.6. The time to failure is assumed to be gamma with parameters α and  n, that is, f (t) =

α(αt)n−1 e−αt , (n 1)!

−

t > 0.

Show that the reliability function is given by  n 1

R(t) = e−αt

−

 r=0

117

αr t r . r!

Find the failure rate function and show that  limt→∞ r (t) = α. What is the expected time to failure?  The gamma distribution function is

 

t

F (t,α,n)

=

0

=

α(αs)n−1 e−αs αn ds = (n 1)! (n 1)! n 1

α −

−

n 1

αt

− (n − 1)! t − e−

= e−αt =

−

1

−

− e−

αn−1 tn−1 (n 1)!

αt

−

n 1

−

 r=0

n 1

+

α −

 

t

sn−1 e−αs ds

0

− 2)! F (t,α,n − 1) − − − α(n −t2)! − · · · − αt + 1! (n

  

n 2 n 2

t

e−αs ds

0

αr t r . r!

after repeated integration by parts. The reliability function is therefore n 1

−

αr t r . r!

    −  −     −   →  −  R(t) = 1 − F (t,α,n) = e−αt

r=0

The failure rate function r (t) is defined by r (t)

=

n 1

f (t) α(αt)n−1 e−αt = R(t( (n 1)!

e−αt

r=0

n 1

=

αn tn−1

(n

1)!

−

−

r=0

αr t r r!

αr tr r!

For the limit, express r (t) in the form

n 1

r (t) = αn

(n

1)!

−

r=0

as t

αr tr−n+1 r!

αn

(n

αn (n 1)!

· αn!

1)!

αn−1 (n 1)!

−

=α

. The expected time to failure is

→∞

E(T ) =

 

∞ αn tn e−αt dt

0

(n

− 1)!

=

−

n+1

=

n . α

8.7. A electrical generator has an exponentially distributed failure time with parameter  λf  and the subsequent repair time is exponentially distributed with parameter  λr . The generator is started up at time t = 0. What is the mean time for generator to fail and the mean time from  t = 0 for it to be operational again?  As in Section 8.4, the mean time to failure is 1 /λf . The mean repair time is 1 /λr , so that the mean time to the restart is 1 1 + . λf  λr 8.8. A hospital takes a grid supply of electricity which has a constant failure rate λ. This supply is backed  up by a stand-by generator which has a gamma distributed failure time with parameters (2, µ). Find the reliability function  R(t) for the whole electricity supply. Assuming that time is measured in hours, what  should the relation between the parameters λ and  µ be in order that R(1000)=0.999?  For the grid supply, the reliability function is Rg (t) = e−λt . For the stand-by supply, the reliability function is (see Problem 8.6) 1

Rs (t) = e−µt

 r=0

µr tr = e−µt (1 + µt). r!

118

The reliability function for the system is λt

− [1 − R (t)][1 − R (t)] = 1 − [1 − e− e− − (1 + µt)e− + (1 + µt)e− .

=

R

1

g

s

λt

=

(λ+µ)t

− (1 + µt)e−

][1

µt

µt

]

Let T  = 1000 hours. Then solving the equation above for λ at time t = T , we have λ=

−





R(T ) (1 + µT )e−µT  1 ln , 1 (1 + µT )e−µT  T 

−

−

where R(T ) = R(1000) = 0.999. 8.9. The components in a renewal process with instant renewal are identical with constant failure rate λ = (1 /50)(hours)−1 . If the system has one spare component which can take over when the first fails, find  the probability that the system is operational for at least  24 hours. How many spares should be carried to ensure that continuous operation for  24 hours occurs with probability  0.98?  Let T 1 and T 2 be respectively random variables of the times to failure of the components. Let S 2 be the time to failure of these components. If  τ  = 24 hours is the operational time to be considered, then, as in Example 8.5, −λτ. P(S 2 < τ ) = 1 (1 + λτ )e

−

Hence P(S 2 < 24) = 1

−

 

− 0.916.

  

 

1+

24 −24/50 =1 e 50

The required probability is 0.916. This is the reverse problem: given the probability, we have to compute n. If  S n is the time to failure, then τ 

P(S N  < τ )

=

F n (τ ) =

0

τ 

λn f n (s)ds = (n 1)!

sn−1 e−λs ds

0

(λτ ) (λτ )n−1 = 1 − e−λτ  1 + λτ  + +···+ The smallest value of  n is required which makes 1 1

2

−

2!

(n

− 1)!



− F  (24) > 0.98. Conputation gives n

− F  (24) = 0.916,

F 3 (24) = 0.987 > 0.98.

2

Three components are required. 8.10. A device contains two components c1 and  c2 with independent failure times T 1 and  T 2 from time t = 0. If the densities of the times to failure are f 1 and  f 2 with probability distributions F 1 and  F 2 , show  that the probability that  c1 fails before c2 is given by 

   

P  T 1 < T 2 =

{

}

∞

y

y=0

x=0

f 1 (x)f 2 (y )dxdy, =

 

∞

F 1 (y )f 2 (y)dy.

y=0

Find the probability  P  T 1 < T 2 in the cases: (a) both failure times are exponentially distributed with parameters λ1 and  λ2 ; (b) both failure times have gamma distributions with parameters (2, λ1 ) and  (2, λ2 ).

{

}

The probability that c1 fails befors c2 is

    

P(T 1 < T 2 ) =

f 1 (x)f 2 (x)dx,

A

where the region A is shown in Figure 8.2. As a repeated integral the double integral can be expressed as P(T 1 < T 2 ) =

   

y

∞

0

0

f 1 (x)f 2 (y )dxdy =

∞

[F 1 (y )

0

119

− F  (0)]f  (y)dy = 1

2

 

∞

0

F 1 (y )f 2 (y )dy,

y 

A

x  O 

Figure 8.2: Region A in Problem 8.10. since F 1 (0) = 0. (a) For exponentially distributed failure times f 2 (y ) = λ2 e−λ y ,

F 1 (y) = 1

2

− e−

λ1 y

.

Therefore P(T 1 < T 2 )

  − −  − ∞

=

(1

e−λ y )λ2 e−λ y dy 1

2

0

=

1 −λ e λ2

λ2

=

λ2

1 λ2

1 e−(λ λ1 + λ2 1 λ1 = λ1 + λ2 λ1 + λ2 2

y

+

1

+λ2 )y





∞ 0

(b) For gamma distributions with parameters ( λ1 , 2) and (λ1 , 2), f 2 (y ) = λ22 ye −λ y ,

F 1 (y ) = 1

2

Hence P(T 1 < T 2 ) =

 

∞

(1

0

λ1 y

− e− − λ ye− 1

λ1 y

λ1 y

− (1 + λ y)e− 1

)(λ22 y )e−λ y dy = 2

.

λ21 (λ1 + 3λ2 ) . (λ1 + λ2 )3

8.11. Let  T  be a random variable for the failure time of a component. Suppose that the distribution function  of  T  is F (t) = P(T  t), t 0, with density 

≤

f (t) = α1 e−λ

1

t

≥

+ α2 e−λ t ,

α1 , α2 > 0,

2

λ1 , λ2 > 0,

where the parameters satisfy 

α1 α2 + = 1. λ1 λ2 Find the reliability function  R(t) and the failure rate function  r(t) for this ‘double’ exponential distribution. How does r(t) behave as t ? 

→∞

The probability distribution

  −

t

F (t)

=

t

f (s)ds =

0

= = =

 

(α1 e−λ

1

s

+ α2 e−λ s )ds

0



α1 −λ s α2 −λ s t e e λ1 λ2 0 α 1 −λ t α 2 −λ t α 1 α2 + + e e λ1 λ2 λ1 λ2 α1 −λ t α2 −λ t 1 e e λ1 λ2

− −

1

1

− − −

1

120

2

2

2

2

The reliability function is therefore R(t) = 1

− F (t) = αλ

1

e−λ

1

t

+

1

α2 −λ t e . λ2 2

The failure rate function is f (t) = α1 e−λ r (t) = R(t)

As t



→ ∞,

r (t)

1

→

t

+ α2 e−λ



2

t



α 1 −λ e λ1

1

t

+

α 2 −λ e λ2

2

t



.

if  λ2 > λ1 if  λ2 < λ1 if  λ1 = λ2 = λ

λ1 λ2 λ

8.12. The lifetimes of components in a renewal process with instant renewal are identically distributed with  constant failure rate λ. Find the probability that at least three components have been replaced by time t. In the notation of Section 8.7

 

t

P(S 3 < t) = F 3 (t) =

F 2 (t

− y)f (y)dy,

0

where F 2 (t) = 1

Then

− (1 + λt)e−

 

λt

f (t) = λe−λt .

,

t

P(S 3

≤ t)

=

[1

0

=

1

λ(t y)

− ]λe−λy dy

− (1 + λ(t − y)e−

− (1 + tλ +

1 2 2 t λ )e λt . 2

−

8.13. The lifetimes of components in a renewal process with instant renewal are identically distributed with  a failure rate which has a uniform distribution with density 



f (t) =

1/k 0

0
Find the probability that at least two components have been replaced at time t. For the uniform density F 1 (t) =

As in Example 8.5,

  

0 t/k 1

t<0 0k

F 1 (t

− y)F (y)dy.

t

F 2 (t) =

0

Interval 0 < t < k .

1 F 2 (t) = k

 

t

F 1 (t

0

−

1 y )dy = 2 k

 

t

2

(t

0

− y)dy = 2tk

2

Interval t > k

F 2 (t)

= = =

1 k

 

k

F 2 (t

0

−

1 y)dy = 2 k

1 [ 1 (t k )2 + 12 k 2 ] k2 2 t (2k t) 2k2

− − −

121

 

k

t k

−

(t

− y)dy

.

To summarize F 2 (t) =



t2 /(2k2 )

0
t(2k

− t)/(2k )

,

t >k

which is the probability that at least two components have failed. 8.14. The lifetimes of components in a renewal process with instant renewal are identically distributed each  with reliability function  1 R(t) = (e−λt + e−2λt ), t 0, λ > 0. 2 Find the probability that at least two components have been replaced by time t.

≥

Given the reliability function, it follows that the distribution function and its density are given by F (t) = 1

− R(t) = 1 −

f (t) = F  (t) =

1 e λt 2

−

1 e 2λt , 2

−

−

1 λe λt 2[

− + 2λe−2λt ].

In the notation of Example 8.5,

 

t

P(S 2 < t)

=

F 2 (t) =

F (t

0

 

t

1 [1 2 0 1 [4 + (1 4

= =

−

− s)f (s)ds

λ(t s) 1 2e

− −

2λ(t s) 1 ][λe λs 2e

−

−

−

2λt

−

λt

− 2λt)e− − (5 + λt)e−

+ 2λe−2λt ]ds

],

which is the probability that at least two components have failed by time t. 8.15. The random variable T  is the time to failure from  t = 0 of a system. The distribution funmction for  T  is F (t), (t > 0). Suppose that the system is still functioning at time t = t0 . Let  T t be the conditional  time to failure from this time, and let  F t (t) be its distribution function. Show that  0

0

F t (t) = 0

F (t + t0 ) F (t0 ) , 1 F (t0 )

−

−

and that the mean of  T t is 0

E(T t0 ) =

1

−

1 F (t0 )

 

∞

(t

[1

t0

≥ 0, t ≥ 0), 0

− F (u)]du.

The distribution function for the conditional time to failure is F t (t) 0

=

P(T t0

≤ t) = P(T  − t ≤ t|T > t ) P(T  − t ≤ t ∩ T > t ) (by eqn (1.2)) 0

0

=

0

0

P(T > t0 )

P(t0 < T 

=

≤t+t ) 0

∂ (T > t0 ) F (t + t0 ) F (t0 ) , 1 F (t0 )

=

−

−

as required. For the mean E(T t0 )

=

   − ∞

1

0

= =

1 1

− −

1 F (t0 ) 1 F (t0 )



F (t + t0 ) F (t0 ) dt 1 F (t0 )

−

− ∞ [1 − F (t + t )]dt

    0

∞

t0

0

[1

− F (u)]du, 122

(where u = t + t0 ).

8.16. Suppose that the random variable T  of the time to failure of a system has a uniform distribution for  t > 0 given by  t/t1 , 0 t t1 F (t) = . 1 t > t1



≤ ≤

Using the result from Problem 8.15, find the conditional probability function assuming that the system is still working at time t = t0 . Thare are two cases to consider: t0 t1 and t0 > t1 . t0 t1 . In the formula in Problem 8.15

≤

• ≤

F (t + t0 ) =

Hence F t (t) = 0

•t

0

> t1 . F t = P(T  0

 

(t + t0 )/t1 1

0 t + t0 t + t0 > t 1

≤

[(t + t0 )/t1 ] [t0 /t1 ] t = t1 t0 1 [t0 /t1 ] 1 [t0 /t1 ] =1 1 [t0 /t1 ]

− t ≤ t|T > t 0



− −

0)

−

−

−

= 1.

123

≤t , 1

0

.

≤t≤t −t t >t −t 1

1

0

0

.

Chapter 9

Branching and other random processes 9.1. In a branching process the probability that any individual has j descendants is given by   p0 = 0 ,

1 , 2j

pj =

( j

≥ 1).

Show that the probability generating function of the first generation is s

G(s) =

2

− s.

Find the further generating functions G2 (s), G3 (s) and  G4 (s). Show by induction that  s

Gn (s) =

2n

− (2 − 1)s . n

Find  pn,j , the probability that the population size of the n-th generation is j given that the process starts with one individual. What is the mean population size of the n-th generation?  The generating function is given by G(s) =

∞

∞

   j

 pj s =

j=0

s 2

j=1

j

=

s + 2



2

s 2

+

· · · = 2 −s s ,

using the geometric series formula for the sum. For the second generation G2 (s) = G(G(s)), so that G2 (s) =

s/(2 s) s = 2 [2/(2 s)] 2(2 s)

−

−

−

− −s

=

s

4

− 3s .

Repeating this procedure, G3 (s) = G(G(G(s))) = G2 (G(s)) = G4 (s) = G3 (G(s)) =

Then Gn+1 (s) = G(Gn (s)) =

− − 3s

s

8(2

Consider the formula Gn (s) =

4(2

2n

−

− s) −

=

s

8

− 7s ,

s

− s) − 7s = 16 − 15s . s (2n

− 1)s .

s

2n (2

s s)

(2n

124

− 1)s

=

2n+1

−

s (2n+1

− 1)s .

Hence if the formula is correct for Gn (s) then it is true for Gn+1 (s). The result has been verified for G2 (s) and G3 (s) so it is true for all n by induction on the integers. integers. Using the binomial binomial expansion expansion Gn (s) =

2n

−

s (2n

−

−

s = n 1 1)s 2

(2n 1) s 2n

−

  −  −1

∞

=

j =1

2n 1 2n

j 1

−

sj .

Hence the probability that the population size of the n-th generation is j is given by the coefficient of  sj in this series, namely 2n 1 j −1  pn,j = . 2n Since G(s) = s/(2 s), then G (s) = 2/(2 s)2 , so that the mean of the first generation is µ = G (1) = 2. Using result (9.7) in the text, the mean size of the n-th generation is

−

 −

−

µn

= Gn (1) = µn = 2 n .

9.2. Suppose in a branching process that any individual has a probability given by the modified geometric distribution  ( j = 0 , 1, 2, . . .),  pj = (1  p) pj ,

−

of producing  j descendants in the next generation, where p (0 < p < 1) is a constant constant.. Find the proba probability  bility  generating function of the second and third generations. What is the mean size of any generation?  The probability probability generating generating function function is G(s) =

∞

∞

  j

 pj s =

j =0

(1

j =0

j j

− p) p s

1  p , 1  ps

− −

=

using the formula for the sum of the geometric series. In the second generation G2 (s)

= =

1  p = 1  pG(s) 1 (1  p)(1  ps) , (1  p + p2 )  ps

G(G(s)) =

− −

−

−

−

1 − p − p[(1 − p)/(1 − ps)]

−

and for the third generation G3 (s)

=

 p(1  p)/(1  ps) ] (1  p)[1 2 (1  p + p )  p(1  p)/(1  ps)

− −{ − − −{ − (1 − p)(1 − p + p − ps) (1 − 2 p + 2 p ) − p(1 − p + p )s

G2 (G(s)) =

2

=

2

− } − }

2

The mean size of the first generation is (1  p) p  µ = G (1) = (1  ps)2

− −

From (9.7) in the book, it follows that generation.

µ2

=

µ

2

,

µ3

=



=

s=1

µ

3

p

1

− p .

, and, in general, that

9.3. A branching process has the probability generating function  G(s) = a + bs + (1

− a − b)s

2

 for the descendants of any individual, where a and  b satisfy the inequalities 0 < a < 1,

b > 0,

125

a + b < 1.

µn

=

µ

n

for the n-th

Given that the process starts with one individual, discuss the nature of the descendant generations. What is the maximum possibl possiblee populati population on of the n-th generation? Show that extinction in the population is certain  if  2a + b 1.

≥

Each descendant produces 0,1,2 individuals with probabilities a, b, 1 a b respectively. If Xn represents a random variable of the population size in the n-th generation, then the possible values of  X1 , X2 , . . . are

−−

{X } = {X } = {X } = ··· ··· {X } =

{0, 1, 2} {0, 1, 2, 3, 4} {0, 1, 2, 3, 4, 5, 6, 7, 8} ··· {0, 1, 2, . . . , 2 }

1 2 3

n

n

The maximum possible population of the n-th generation is 2n . The probability of extinction is the smallest solution of  G(g ) = g , that is, a + bg + (1

− a − b)g

2

= g,

or (g

− 1)[(1 − a − b)g − a] = 0 .

The equation always has the solution g = 1. The other possible solution is g = a/(1 a b). Extinction Extinction is certain if  a 1 a b, that is if 2 a + b 1. The region in the a, b where extinction is certain is shown in Figure 9.1. If  a < 1 a b, then extinction occurs with probability a/(1 a b).

≥ − − − −

− −

≥

− −

b 

1

extinction certain

a  0.5

O 

1

Figure 9.1: Extinction probability region in the a, b plane for Problem 9.3. 9.4. A branching process starts with one individual. Subsequently any individual has a probability (Poisson)  pj =

λj e−λ ,  j !

( j = 0 , 1, 2, . . .)

of producing  j descendants. Find the probability generating function of this distribution. Obtain the mean  and variance of the size of the n-th generation. generation. Show that the proba probability bility of ultimate ultimate extinction is certain  certain  if  λ 1.

≤

The probability probability generating generating function is given given by G(s) =

∞

∞

  j

 pj s =

j =0

j =0

λj e−λ j s = eλs−λ .  j !

As expected for this distribution, the mean and variance of the population of the first generation are µ σ

2

= G (1) = λeλs−λ

= G (1) + µ

−

µ

2

|

s=1

= λ,

= λ2 + λ

−λ

2

= λ.

By Section 9.3, the mean and variance of the population of the n-th generation are µn

= µn = λn ,

126

2 σn

µ

=

µn

= 1 and

σn

(σ 2

+ µ2 )(µn 1) µ(µ

−

µ

−

− 1)

(λ = 1)



λn+1(λn 1) . λ 1

= If  λ = 1, then

n

−

= n.

−

9.5. A branching process starts with one individual. Any individual has a probability   pj =

λ2j sech λ , (2 j )!

( j = 0 , 1, 2, . . .)

of producing  j descendants. Find the probability generating function of this distribution. Obtain the mean  size of the n-th generation. generation. Show that ultimate extinction extinction is certain if  λ is less than the computed value 2.065. The probability probability generating generating function of this distribution distribution is given given by G(s, λ) = sechλ

∞

 j =0

Its derivative is

λ2j j s = sechλ cosh(λ s), (2 j )!

√

Gs (s, λ) =

(s

≥ 0).

√ λ √ sechλ sinh(λ s). 2 s

Hence the mean size of the population of the first generation is 1 λ tanh λ, 2 which implies that the mean population of the n-th generation is µ

= Gs (1, λ) =

µn

=

λn tanhn λ. 2n

λ

6

4 (1, 2.065) 2

g  1

0.5

O 

1.5

Figure 9.2: Graph of  g = G(g ) for Problem 9.5. Ultimate extinction occurs with probability g where g is the smallest solution of 

√

g = G(g, λ) = sechλ cosh(λ g ).

This equation aways has the solution g = 1, which is the only solution if  λ < 2.065, approximately. approximately. This is a numerically computed value. The graph of the equation above is shown in Figure 9.2. branching proces processs starts starts with two individuals. individuals. Either individual individual and any of their descendant descendantss has 9.6. A branching probability  pj , ( j = 0, 1, 2, . . .) of producing  j descendant descendantss independently independently of any other. Explain Explain why the probabilities of  0, 1, 2, . . . descendants in the first generation are n

 p20 ,

p0 p1 + p1 p0 ,

p0 p2 + p1 p1 + p2 p0 ,

...



 pi pn−i , . . . ,

i=0

127

respectively. Hence show that the probability generating function of the first generation is G(s)2 , where G(s) =

∞



 pj sj .

j=0

The second generation from each original individual has generating function  G2 (s) = G(G(s)) (see Section 9.2). Explain why the probability generating function of the second generation is G2 (s)2 , and of the n-th generation is Gn (s)2 . If the branching process starts with  r individuals, what would you think is the formula for the probability  generating function of the n-th generation?  For each individual, the probability generating function is G(s) =

∞



 pj sj ,

j=0

and each produces descendants with populations 0 , 1, 2, . . . with with probabilities p0 , p1 , p2 , . . .. The combined probabilities that populations of the generations are  p20 ,

p0 p2 + p21 + p2 p0 ,

p0 p1 + p1 p0 ,

....

These expressions are the coefficients of the powers of  s in 2

G(s) =

∞

∞

k

∞

    pj s

j

j=0

k

 pk s =

k=0

 pj pk−j sk

k=0 j=0

(this is known as the Cauchy product of the power series). Hence the probability that the population of the first generation is of size k is k

∞



 pj pk−j .

k=0 j=0

Repeating the argument, each original individual generates descendants whose probabilities are the coefficients of  G2 (s) = G(G(s)). Hence the probabilities of populations 0, 1, 2, . . . descandants are coefficients of  G22 . This process is repeated for succeeding generations which have the generating functions Gn (s)2 . 9.7. A branching process starts with two individuals as in the previous problem. The probabilities are  pj =

1 2j+1

( j = 0 , 1, 2, . . .).

,

Using the results from Example 9.1, find  H n (s), the probability generating function of the n-th generation. Find also (a) the probability that the size of the population of the n-th generation is m 2; (b) the probability of extinction by the n-th generation; (c) the probability of ultimate extinction.

≥

For either individual the probability generating function is G(s) =

∞

 j=0

sj

2j+1

=

Then G2 (s) = G(G(s)) =

1

. 2−s

2 s , 3 2s

and, in general, Gn (s) =

− −

n (n 1)s . (n + 1) ns

− − −

128

According to Problem 9.6, the generating function for the combined descendants is H n (s)

2

− − −

n (n 1)s (n + 1) ns

=

Gn (s) =

=

n2 1 (n + 1)2

=

n2 1 (n + 1)2

2

2 2

2(n

2 2

− 1)s + (n − 1) s n2

n

n 2n s+ + (n + 1)2 (n + 1)3

∞



 −     1

n2

n

2

=



− − 2(n − 1)s (n − 1) s +

∞

ns −2 n+1

(r + 1)

r=0

[(r − 1)n − + 2nr ]

r=2

r 2

(n + 1)r+2

n n+1

r

sr

sr

after some algebra: series expansion by computer is helpful to confirm the formula. (a) From the series above, the probability pn,m that the population of the n generation is m is the coefficient of  sm in the series, namely  pn,m =

[(m

m 2

− + 2nm ]

− 1)n

(n + 1)m+2

(m

,

≥ 2)

(b) From the series above, the probability of extinction by the n-th generation is  pn,0 =

n2 . (n + 1)2

(c) The probability of ultimate extinction is lim  pn,0

n

→∞



n2 = lim n→∞ (n + 1)2



= 1,

which means that is is certain. 9.8. A branching process starts with  r individuals, and each individual produces descendants with probability  distribution   pj , ( j = 0, 1, 2, . . .), which has the probability generating function  G(s). Given that the probability of the n-th generation is [Gn (s)]r , where Gn (s) = G(G(. . . (G(s)) . . .)), find the mean population  size of the n-th generation in terms of  µ = G (1).

{ }

Let Q(s) = [Gn (s)]r . Its derivative is Q (s) = rG n (s)[Gn (s)]r−1 ,

where

d Gn−1 (G(s)) = Gn−1 (G(s))G (s). ds Hence, the mean population size of the n-th generation is Gn (s) =

µn

= Q (1) = r[Gn (1)]r−1 Gn−1 (1)G (1) = rµn .

9.9. Let  Xn be the random variable of the population size of a branching process starting with one individual. Suppose that all individuals survive, and that  Z n = 1 + X1 + X2 +

···+X

n

is the random variable representing the accumulated population size. (a) If  H n is the probability generating function of the total accumulated population, Z n , up to and including  the n-th generation, show that  H 1 (s) = sG(s),

H 2 (s) = sG(H 1 (s)) = sG(sG(s)),

(which perhaps gives a clue to the probability generating function of  H n (s)),

129

(b) What is the mean accumulated population size E(Z n ) (you do not require H n (s) for this formula)?  (c) If  µ < 1, what is limn→∞ E(Z n ), the ultimate expected population?  (d) What is the variance of  Z n ?  (a) Let pj be the probability that any individual in any generation has j descendants, and let the probability generation function of   pj be

{ }

∞



G(s) =

 pj sj .

j=0

The probabilities of the accumulated population sizes are as follows. Since the process starts with one individual P(Z 0 = 1) = 1 P(Z 1 = 0) = 0 , P(Z 1 = n) = pn−1 (n = 1 , 2, 3, . . .). Hence the generating function of  P(Z 1 = n) is given by H 1 (s), where H 1 (s) =

∞



r

P(Z 1 = r)s =

r=1

∞



P(X1 = r

r=1

r

− 1)s

=

∞



 pr−1 sr = sG(s).

r=1

For the probability of  Z 2 , use the identity P(Z 2 = n) =

∞



P(Z 2 = n Z 1 = r

|

r=1

− 1)P(Z 

1

=r

− 1).

Then the probability generating function H 2 (s) has the series H 2 (s)

=

∞

    

n

P(Z 2 = n)s =

n=1

=

∞ ∞

∞ ∞



∞

=

|

∞



 pr−1 E(sZ ) = 2

 pr−1 E(s(Z

1

=

+Y 1 )+(Z1 +Y 2 )+

···+(Z

1

+Y r )

)

r=1

 pr−1 E(sZ

1

+Y 1

)E(sZ

1

+Y 2

r=1

∞

n

 pr−1 P(Z 2 = n Z 1 = r )sn

r=1

∞

|

n=1 r=1

n=1 r=1

=

P(Z 2 = n Z 1 = r)P(Z 1 = r)s

)

· · · E(s

Z1 +Y r

)

 pr−1 [sG(s)]r = sG[sG(s)],

r=1

using a method similar to that of Section 9.2. In this analysis, in the second generation, it is assumed that X2 = Y 1 + Y 2 +

· · · + Y  , r

where Y j are iid. (b) The mean of the accumulated population is (see eqn (9.7))

{ }

E(Z n )

= = =

E(1 + X1 + X2 + 2

···+X

n)

= E(1) + E(X1 ) + E(X2 ) +

n

1+ µ+µ + +µ n+1 1 µ (µ = 1) . 1 µ

− −

after summing the geometric series. If  µ = 1, then E(Z n ) = n + 1. (c) If  µ < 1, then from (b) E(Z n )

···



→ 1/(1 −

µ).

130

· · · + E(X

n)

(d) The variance of  Z n is, from Section 9.3(i), V(Z n )

=

V(1 + X1 + X2 +

· · · + X ) = V(1) + V(X ) + V(X ) + · · · + V(X − ( − 1) − − −) 0+ = ( −1 −1 (1 − ) 1− − 1− (  = 1) −1 1− (1 − )(1 − ) (1 − )(1 − ) n

=

 

σ

2

µ σ

=

2

1

2 r 1 r σ µ µ

µ

µ

µ

2n

µ

µ2

µ

2

σ

µ

r=1

=

n

n

n

µ

µ

n+1

2

n)

n



µ

2r 1

µ

r 1

r=1



µ

µ2

µ

9.10. A branching process starts with one individual and each individual has probability  pj of producing  j descendants independently of every other individual. Find the mean and variance of   pj in each of the  following cases, and hence find the mean and variance of the population of the n-th generation:

{ }

(a)

(b)

(c)

e−µ µj ,  j !

pj =

pj = (1

pj =



j

−

−

1

− p) −  p



r + j 1 j  p (1 r 1

(Poisson);

( j = 0 , 1, 2, . . .)

(geometric);

( j = 1 , 2, . . . ; 0 < p < 1)

r

− p) ,

(negative binomial).

( j = 0 , 1, 2, . . . ; 0 < p < 1)

where r is a positive integer, the process having started with one individual (a negative binomial distribution). (a) For the Poisson distribution with intensity µ,  pj =

e−µ µj .  j !

Its probability generating function is G(s) = e−µ(1−s) . Therefore G (s) = µe−µ(1−s) ,

G (s) = µ2 e−µ(1−s) ,

and the mean and variance of the first generation are µ

= µ,

σ

2

= G (1) + G (1)

2

− [G(1)]

= µ2 + µ + µ2 = µ.

The mean and variance of the n-th generation are (see Section 9.3), for µn

= µn = µn ,

2 σn

=

2 n 1 σ µ (µn

−

µ

−1

−

G (s) =

pq , (1  ps)2

G (s) =

−

n

− 1) = µ

(b) The geometric distribution is pj = qj −1 p, where q = 1 function G(s) = q/(1  ps). Then

= 1, (µ − 1) . µ−1

µ

n

− p, which has the probability generating 2 p2 q . (1  ps)3

−

The mean and variance of the first generation are µ

= G (1) =

p , q

σ

2

= G (1) + G (1)

2

− [G (1)]

=

p . q2

The mean and varistion of the n-th generation are µn

=

  p q

n

,

2 σn

=

2 n 1 σ µ (µn

−

µ

−1

− 1) = 131

1 2 p

−1

    −   p q

n

 p q

n

1

( p = 12 ).



(c) The negative binomial distribution is  pj =



  

r + j 1 j r  p q r 1

−

−

Its probability generating function is G(s) =

The derivatives are

(q = 1

q 1  ps

− p).

r

.

−

2 r rpq r  (s) = r (r + 1) p q . , G (1  ps)r+1 (1  ps)r+2 Hence, the mean and variance of the first generation are

G (s) =

µ

=

−

−

rp , 1  p

σ

−

2

=

rp , (1  p)2

−

( p = 1) .



The mean and variance of the populations of the n-th generation are µn

σn

=

2 n 1 σ µ (µn

−

µ

−1

− 1) =

=

rp

  −     −  rp 1  p

1 1 +  p

n

,

n

rp 1  p

−

rp 1  p

−

−

n

1 .

9.11. A branching process has a probability generating function  G(s) =

− 1  p 1  ps

r

(0 < p < 1),

,

−

where r is a positive integer (a negative binomial distribution), the process having started with one individual. Show that extinction is not certain if  p > 1/(1 + r). We need to investigate solutions of  g = G(g ) (see Section 9.4). This equation always has the solution g = 1, but does it have a solution less than 1? For this distribution the equation for g becomes g (1

r

− gp)

= (1

r

− p) . and the curve y = g (1 − gp )

Consider where the line y = (1  p)r r. The curve has a stationary value where

−

dy = (1 dg

r

r

r 1

− gp) − rp(1 − gp) −

intersect in terms of  g for fixed p and

= 0,

which occurs at g = 1/[ p(1 + r)], which is a maximum. The line and the curve intersect for a value of  g between g = 0 and g = 1 if  p > 1/(1 + r), which is the condition that extinction is not certain. Graphs of  the line and curve are shown in Figure 9.3 for p = 12 and r = 2. 9.12. Let  Gn (s) be the probability generating function of the population size of the n-th generation of a  branching process. The probability that the population size is zero at the n-th generation is Gn (0). What  is the probability that the population actually becomes extinct at the n-th generation?  In Example 9.1, where pj = 1 /2j+1 ( j = 0 , 1, 2, . . .), it was shown that  n + Gn (s) = n+1

∞

 r=1

nr−1 sr . (n + 1)r+1

Find the probability of extinction, (a) at the n-th generation, (b) at the n-th generation or later. What is the mean number of generations until extinction occurs? 

132

y 

g1

p =

0.5

r =

2

g 

Figure 9.3: Graphs of the line y = (1  p)r and the curve y = g(1 Problem 9.11.

−

− gp)r with p = 12 and r = 2 for

The probability that the population is extinct at the n-th generation is Gn (0), but this includes extinction of previous generations at r = 1, 2, . . . , n 1. The probability is therefore Gn (0) given that individuals have survived at the (n 1)-th generation, namely Gn (s) Gn−1 (s). (a) In this example Gn (s) = n/(n + 1). Hence probability of extinction at the n-th generation is

−

−

Gn (0)

−

− G − (0) = n +n 1 − n −n 1 = n(n1+ 1) . n 1

(b) Since ultimate extinction is certain, the probability that extinction occurs at or after the n-th generation is n 1 1 1 Gn−1 (0) = 1 = . n n The mean number of generations until extinction occurs is

− −

−

∞



n[Gn (0)

n=1

−G

n 1 (0)]

−

=

∞

 n=1

n = n(n + 1)

This series diverges so that the number of generations is infinite.

∞

 n=1

1 . n+1

9.13. An annual plant produces N  seeds in a season which are assumed to have a Poisson distribution with  parameter  λ. Each seed has a probability  p of germinating to create a new plant which propagates in the  following year. Let  M  the random variable of the number of new plants. Show that  pm , the probability that  there are m growing plants in the first year is given by   pm = ( pλ)m e−pλ /m!

(m = 0 , 1, 2, . . .),

that is Poisson with parameter  pλ. Show that its probability generating function is G(s) = epλ(s−1) .

Assuming that all the germinated plants survive and that each propagates in the same manner in succeeding  years, find the mean number of plants in year  k . Show that extinction is certain if  pλ 1.

≤

Given that plant produces seeds as a Poisson process of intensity λ, then f n =

λn e−λ . n!

Then  pm

=

  ∞

r=m

=

r  pm (1 m

r mλ

− p) −

n

( pλ)m e−pλ . m!

133

e−λ = ( λp)m e−λ n!

∞

 i=0

(1

i i

− p) λ i!

Its probabilit probability y generating generating function is G(s) = e−pλ

∞



m=0

The mean of the first generation is µ

( pλ)m m pλ+pλs s = e−pλ+ . m!

pλ(s−1) = G (1) = pλepλ(

|

s=1

= pλ.

The mean of the n-th generation is therefore therefore µn

= µn = ( pλ)n .

Extinction occurs with probability g , where g is the smaller solution of  g = G(g), that is, the smaller solution of  g = e−pλ epλg . Consider the line y = g and the exponential curve y = e−pλ epλg . On the curve, its slope is y 1.0

0.8

0.6

0.4

0.2

g 0.2

0.4

0.6

0 .8

1.0

Figure 9.4: Graphs of the line y = g and the curve y = e− pλ e pλg with λ = 2 and p = 1 for Problem 9.11.3 dy = pλe−pλ epλg , dg and its slope at g = 1 is pλ. Sinc Sincee e−pλ epλg 0 as g , and e−pλ epλg and its slope decrease as g decreases, then the only solution of  g = G(g ) is g = 1 if  pλ 1. Extinc Extinction tion is certai certain n in this case. case. If   pλ > 1 then there is a solution for 0 < g < 1. Figure 9.4 shows such a solution for λ = 2 and p = 1.

→

→ −∞ ≤

version n of Exampl Examplee 9.1 with a gener general al geome geometric tric distrib distributio ution n is the branc branchin hingg proc process ess with  9.14. The versio j  pj = (1  p) p , (0 < p < 1; j = 0 , 1, 2, . . .). Show that 

−

G(s) =

1  p . 1  ps

− −

Using an induction method, prove that  Gn (s) =

(1

n

n

n 1

n+1

n

n 1

− p)[ p p − (1 − p) − ps{ p − − (1 − p) − }] , − (1 − p) − ps{ p − (1 − p) }] [ p  p n+1

n

( p =



1 ). 2

Find the mean and variance of the population size of the n-th generation. What is the probability of extinction by the n-th generation? generation? Show that ultimate extinction is certain  1 1 if  p < 2 , but has probability  (1  p)/p if  p > 2 .

−

As in Problem 9.2, the generating function for the first generation is G(s) =

1  p . 1  ps

134

− −

Consider Gn (G(s))

=

n

n

n 1

n+1

n

n+1

n

=

n

n+1

=

n 1

n 1

n+1

n+2

n

n 1

n+1

n+1

=

n 1

− p)[ p p − (1 − p) − p[(1 − p)/(1 − ps)]{ p − − (1 − p) − }] − (1 − p) − p[(1 − p)/(1 − ps)]{ p − (1 − p) }]  p [ p (1 − p)[{ p − (1 − p) }(1 − ps) − p(1 − p){ p − − (1 − p) − }] { p − (1 − p) }(1 − ps) − p(1 − p){ p − − (1 − p) − } − (1 − p) − ps{ p − (1 − p) }]  p (1 − p)[ p − (1 − p) − ps{ p − (1 − p) }  p (1

n+2

n

n 1

n

n+1

n+1

Gn+1 (s).

Hence if the formula is true for Gn (s), then it is true for Gn+1 (s). It can be verified for G2 (s), so that by induction on the integers, it is true for all n. The probability of extinction by the n-th generation is Gn (0) =

 pn (1  p)n ] (1  p)[ p .  pn+1 (1  p)n+1

−

− − − −

If  p > 12 , express in the following form Gn (0) =

(1  p)[1 ((1  p)/p)n ]  p[1 ((1  p)/p)n+1 ]

−

− − − −

→ 1  p− p 1 2

as n

→ ∞, which is the probability of ultimate extinction. If  p < , then [( p/(1 − p)) − 1] G (0) = − 1] → 1, [( p/(1 − p)) as n → ∞: extinction is certain. n

n

n+1

9.15. A branching process starts with one individual, and the probability of producing  j descendants has the distribution  distribution   pj , ( j = 0, 1, 2, . . .). The same prob probabi abilit lityy distrib distributio ution n applie appliess indep independ endent ently ly to all  descendant descendantss and their descendant descendants. s. If  Xn is the random variable of the size of the n-th generation, show  that  E(Xn ) 1 P(Xn = 0) .

{ }

In Section 9.3 it was shown that  E(Xn ) = eventually is certain if  µ < 1. By definition definition E(Xn ) =

∞

µ

n

, where



 j P(Xn = j )

j =1

Hence

≥ −

µ

∞

 ≥

Deducee that the proba probability bility of extinction  extinction  = E(X1 ). Deduc

P(Xn = j ) = 1

j =1

P(Xn = 0) = 1

Therefore, if  µ < 1, then P(Xn = 0)

−

µ

n

− P(X

n

= 0) .

.

→ 1 as n → ∞. This conclusion is true irrespective of the distribution.

9.16. In a branching process starting with one individual, the probability that any individual has j descendants is pj = α/2j , ( j = 0 , 1, 2, . . . , r), where α is a constant and  r is fixed. This means that any individual  can have a maximum of  r descendant descendants. s. Find  α and the probability generating function  G(s) of the first  generation. Show that the mean size of the n-th generation is µn



2r+1 2 r = 2r+1 1

− − −



n

.

What is the probability of ultimate extinction?  Given pj = α/2j , then for it to be a probability distribution r

  j =0

α 1 1 = α 1+ + 2 + 2j 2 2

···

  −  

1 + r = 2α 1 2

135

1 2

r+1

= 1.

Therefore the constant α is defined by α=

2[1

The probability probability generating generating function is given given by

  r

G(s) = α

−

1 . ( 12 )r+1 ]

j =0

sj s s2 = 1 + + + α 2j 2 22

···

(i)

sr + r 2



=α

[1

r+1

− (s/2) (1 − s) 1 2

]

,

(ii)

using the formula for the sum of the geometric series: α is given by (i). The derivative of  G(s) is G (s) =



1+r 2−r α 21+r

− 2(1 + r)s (s − 2)

r

1+r + rs1+r

2



.

Hence the mean value of the first generation is µ



1+r = G (1) = 2−r α 21+r

By (9.7), the mean of the n-th generation is µn

=µ

n

− 2 − r)





=

2r+1 2 r = 2r+1 1

Since

− − −

2r+1 2 r . 2r+1 1

− − −



n

.

2r+1 2 r < 1, 2r+1 1 then, by Problem 9.15, ultimate extinction is certain. µ

=

− − −

9.17. Extend the tree in Figure 9.3 for the gambling martingale in Section 9.5 to Z 4 , and confirm that  E(Z 4 Z 0 , Z 1 , Z 2 , Z 3 ) = Z 3 .

|

confirm also that  E(Z 4 ) = 1. Extension of the gambling martingale to Z 4 is shown in Figure 9.5. The values for the random variables Z0

Z1

Z2

Z3 8

4 0

Z4 16 0 8 -8

2

12 0

4 -4

1 6 2 -2 0

-4 4 -12 14 -2 6 -10 10

2 -2 -6

-6 2 -14

Figure 9.5: Martingale for Problem 9.17. Z 4 are: Z 4 = even numbers between ”-14” and ”16” inclusive

{

The mean value of  Z 4 is given by 15

E(Z 4 ) =



m=0

1 ( 24 + 2m + 2) = 1 , 4 2

−

136

}

or the mean can be calculated from the mean of the final column of numbers in Figure 9.5. 9.18. A gambling game similar to the gambling martingale of Section 9.5 is played according to the following  rules: (a) the gambler starts with £1, but has unlimited resources; (b) against the casino, which also has unlimited resources, the gambler plays a series of games in which  the probability that the gambler wins is 1/p and loses is ( p 1)/p, where p > 1; (c) at the n-th game, the gambler either wins £( pn  pn−1 ) or loses £ pn−1 . If  Z n is the random variable of the gambler’s asset/debt at the n-th game, draw a tree diagram similar  to that of Figure 9.3 as far as Z 3 . Show that 

−

−

Z 3 =

2

2

3

2

3

2

3

3

{− p − p , − p , − p, 0, p − p − p,p − p , p − p,p }

and confirm that  E(Z 2 Z 0 , Z 1 ) = Z 1 ,

E(Z 3 Z 0 , Z 1 , Z 2 ) = Z 2 ,

|

|

which indicates that this game is a martingale. Show also that  E(Z 1 ) = E(Z 2 ) = E(Z 3 ) = 1 .

Assuming that it is a martingale, show that, if the gambler first wins at the n-th game, then the gambler  will have an asset gain or debt of £( pn+1 2 pn + 1)/( p 1). Explain why a win for the gambler can only  be guaranteed for all  n if  p 2.

−

≥

−

The tree diagram for this martingale is shown in Figure 9.6. From the last column in the Figture 9.6, Z

Z1

Z0

Z3

2

3

p p

2

0

p

3

p -p

0

-p 1

2

2

3

p -p 2

p -p

-p

0 3

2

p -p -p -p

2

-p -p

Figure 9.6: Tree diagram for Problem 9.18. it can be seen that the elements of  Z 3 are given by Z 3 =

2

2

3

2

3

2

3

3

{− p − p , − p , − p, 0, p − p − p,p − p , p − p,p }.

For the other conditional means, E(Z 2 Z 0 , Z 1 ) =  p, 0 = 0, p ,

|

{ } { } E(Z  |Z  , Z  , Z  , Z  ) = { p , 0, p − p, − p} = {− p, 0, p − p,p}. 3

0

1

2

2

3

For the other means,

2

2

1  p 1 +0 = 1,  p  p

−

E(Z 1 ) = p

1 1  p 1 + ( p2 E(Z 2 ) = p 2 + 0  p  p  p 2

−

−

etc.

137

 p  p)

− 1 1 − p

 p  p

− 1

 p

 p

2

= 1,

Suppose the gambler first wins at the nth game: on the tree the path will be lowest track until the last game. Generalising the path for Z 3 , the gambler has an asset of 

   

 pn − 1 £( p − 1 − p − p − · · · − p − ) = £  pn − n

2

n 1

 p



 pn+1 2 pn + 1 . =£  p 1

−1

− −

To guarantee winnings requires  pn+1

− 2 p

n

+ 1 = pn ( p

− 1)+ > 0

for all n. This will certainly be true if  p > 2. A smaller value of  p will guarantee winnings but this depends on n. 9.19. Let  X1 , X2 , . . . be independent random variables with means Z n = X1 + X2 +

···+X

µ1 , µ2 , . . .

respectively. Let 

n,

and let  Z 0 = X0 = 0. Show that the random variable n

Y n = Z n

 −

(n = 1 , , 2, . . .)

µi ,

i=1

is a martingale with respect to Xn . [Note that  E(Z n X1 , X2 , . . . , Xn ) = Z n .]

{ }

|

The result follows since n+1

E(Y n+1 X1 , X2 , . . . , Xn )

|

=

E(Z n+1

=

−



µi

i=1

n)

|X , X , . . . , X 1

2

n+1

 −  −

E(Z n + Xn+1

µi

i=1

n)

|X , X , . . . , X 1

2

n+1

=

Z n + µn+1

µi

i=1

n

=

Z n

 −

µi

= Y n

i=1

Hence the random variable Y n is a martingale. 9.20. Consider an unsymmetric random walk which starts at the origin. The walk advances one position  with probability  p and retreats one position with probability  1  p. Let  Xn be the random variable giving  the position of the walk at step n. Let  Z n be the random variable given by 

−

Z n = Xn + (1

− 2 p)n.

Show that  E(Z 2 X0 , X1 ) =

{−2 p, 2 − 2 p} = Z  . Generally show that  {Z  } is a martingale with respect to {X }. |

1

n

n

The conditional mean E(Z 2 X0 , X1 )

|

=

E(X2 + (1

=

2

=

− 2 p)2|X , X ) E(X |X , X ) + 2(1 − 2 p) {1 + 1 − 2 p, −1 + 1 − 2 p} = {2 − 2 p, −2 p} = Z  0

0

1

1

1

By the Markov property of the random walk, E(Z n+1 X0 , X1 , . . . , Xn ) = E(Z n+1 Xn ) = E(Xn+1 + (1

|

|

138

− 2 p)(n + 1)|X

n)

Suppose that Xn = k . Then the walk either advances one step with probability p or retreats one step with probability 1  p. Therefore

−

E(Xn+1 + (1

− 2 p)(n + 1)|X

n)

=

p(k + 1) + (1

=

k + (1

− p)(k − 1) + (1 − 2 p)(n + 1)

− 2 p)n = X

n.

9.21. In the gambling martingale of Section 9.5, the random variable Z n , the gambler’s asset, in a game against a casino in which the gambler starts with £ 1 and doubles the bid at each play is given by  Z n =

n

{−2

(m = 0 , 1, 2, . . . , 2n

+ 2m + 2 ,

}

− 1).

Find the variance of  Z n . What is the variance of  E(Z n Z 0 , Z 1 , . . . , Zn  −1 )?

|

Then the sum of the elements in the set Z n = that is, 2n

−1



2n

Z n

−1

−

=

n

{−2

+ 2 m + 2 , m = 0, 1, 2, . . . , 2n

}

2n n

n

n

( 2 + 2m + 2) = ( 2 + 2)2 + 2

m=0

−

m=0

−1



− 1) is required,

m = 2n

m=1

Since all the elements in are equally likely to occur after n steps, then 1 E(Z n ) = n 2

2n

−1



Z n =

m=0

1 n 2 = 1. 2n

The variance of  Z n is given by V(Z n ) =

2 E(Z n )

−

since

1 [E(Z n )] = n 2 2

2n

2n

−1

−

( 2n + 2m + 2)2

m=0

−1

−

( 2n + 2m + 2)2 =

m=0

Since

− 1 = 13 (2 − 1), 2n

2n (2 + 22n ). 3

E(Z n Z 0 , Z 1 , . . . , Zn  −1 ) = Z n−1 ,

|

then

V[E(Z n Z 0 , Z 1 , . . . , Zn  −1 )] = V(Z n−1 ) =

|

1 2(n−1) [2 3

− 1]

by the previous result. 9.22. A random walk starts at the origin, and, with probability  p1 advances one position and with probability  q1 = 1  p1 retreats one position at every step. After  10 steps the probabilities change to p2 and  q2 = 1  p2 respectively. What is the expected position of the walk after a total of  20 steps? 

−

−

After 10 steps the walk could be at any position in the list of even positions

{−10, −8, −6 . . . 6, 8, 10}, which are the random variable Xr . Let the random variable Y n be the position of the walk after 20 steps so that Y n = 20, 18, 16, . . . , 16, 18, 20 .

{− − −

139

}

Then the position after a further 10 steps is E(Y n Xr ) = Xr + 10( p2

− q ).

|

2

Its expected position is E[E(Y n Xr )] = E[Xr + 10( p2

|

− q )] = 10( p 2

1

+ p2

− q − q ). 1

2

9.23. A symmetric random walk starts at the origin  x = 0. The stopping rule that the walk ends when the position  x = 1 is first reached is applied, that is the stopping time T  is given by  T  = min n : Xn = 1 ,

{

}

where Xn is the position of the walk at step n. What is the expected value of  T ? If this walk was interpreted  as a gambling problem in which the gambler starts with nothing with equal odds of winning or losing £ 1 at each play, what is the flaw in this stopping rule as a strategy of guaranteeing a win for the gambler in  every game? [Hint: the generating function for the probability of the first passage is G(s) = [1

1 2

2

− (1 − s )

]/s :

see Problem 3.11.]  The probability generating function for the first passage to x = 1 for the walk starting at the origin is G(s) =

1 [1 s

2

− (1 − s )

1 2

]=

s s3 s5 + + + O (s7 ), 2 8 16

which imples that the probability that the first visit to x = 1 occurs at the 5-th step is 1 /16. The mean of the first visits is

 µ = G (s)|

s=1

=

2

1

− (1 − s ) s (1 − s ) 2

1 2

1 2

2



=

s=1

∞.

It seems a good ploy but would take, on average, an infinite number of plays to win £1. 9.24. In a finite-state branching process, the descendant probabilities are, for every individual,  pj =

2m−j 2m+1

( j = 0 , 1, 2, . . . , m),

− 1,

and the process starts with one individual. Find the mean size of the first generation. If  Xn is a random  variable of the size of the n-th generation, explain why  Z n =



2m+1

−1 −m−2

2m+1



n

Xn

defines a martingale over  Xn .

{ }

In this model of a branching process each descendant can produce not more than m individuals. It can be checked that m m 2m−j = 1,  pj = 2m+1 1

  j=0

−

j=0

using the formula for the sum of the geometric series. The probability generating function for the first generation is m

G(s) =



 pj sj =

j=0

m

2m 2m+1

  − 1

j=0

140

s 2

j

=

2m+1 (2m+1

m+1

−s − 1)(2 − s) .

Its first derivative is

2m+1

G (s) =

m+1

m+1

Therefore the mean of the first generation is µ

m

− 2(m + 1)s + ms − 1)(2 − s) (2

.

2

2m+1 m 2 . 2m+1 1

− − −

= G (1) =

The random variable Z n is simply Z n = Xn /µn (see Section 9.5). 9.25. A random walk starts at the origin, and at each step the walk advances one position with probability   p or retreats with probability  1  p. Show that the random variable

−

Y n = Xn2 + 2(1

− 2 p)nX

n

2

+ [(2 p

2 2

− 1) − 1]n + (2 p − 1) n ,

where Xn is the random variable of the position of the walk at time n, defines a martingale with respect to Xn .

{ }

Let α( p,n) = 2(1 E(Y n+1 Xn )

|

2

2 2

− 2 p)n and β ( p,n) = [(2 p − 1) − 1]n + (2 p − 1) n =

p[(Xn + 1)2 + α( p,n + 1)(Xn + 1) + β ( p,n + 1)]

+(1

=

in the expression for Y n . Then

Xn2

2

− p)[(X − 1) + α( p,n + 1)(X − 1) + β ( p,n + 1)] + X [4 p − 2 + α( p,n + 1)] + [1 + (2 p − 1)α( p,n + 1) + β ( p,n + 1)] n

n

n

The coefficients in the last expression are 4 p

− 2 + α( p,n + 1) = 4 p − 2 + 2(1 − 2 p)(n + 1) = 2(1 − 2 p)n = α( p,n),

and 1 + (2 p

− 1)α( p,n + 1) + β ( p,n + 1) = 1 + (2 p − 1)2(1 − 2 p)(n + 1) + [(2 p − 1) − 1](n + 1) + (2 p − 1) (n + 1) [(2 p − 1) − 1]n + (2 p − 1) n = β ( p,n) 2

=

2

Hence

2

2 2

2

E(Y n+1 Xn ) = Xn + 2(1

|

2

− 2 p)nX

+ [(2 p

n

so that, by definition, Y n is a martingale.

2

2 2

− 1) − 1]n + (2 p − 1) n

= Y n ,

9.26. A simple epidemic has n0 susceptibles and one infective at time t = 0. If  pn (t) is the probability that  there are n susceptibles at time t, it was shown in Section 9.7 that  pn (t) satisfies the differential-difference equations (see eqns (9.15 and (9.16)) dpn (t) = β (n + 1)(n0 dt

− n) p

n+1 (t)

− βn (n

0

+1

− n) p

 for  n = 0 , 1, 2, . . . n0 . Show that the probability generating function  n0

G(s, t) =



 pn (t)sn

n=0

satisfies the partial differential equation  ∂G (s, t) = β (1 ∂t

−



∂G (s, t) s) n0 ∂s

−



∂ 2 G(s, t) s . ∂s 2

Nondimensionalize the equation by putting  τ  = βt . For small  τ  let  G(s,τ/β ) = G0 (s) + G1 (s)τ  + G2 (s)τ 2 +

141

···.

n (t),

Show that  nGn (s) = n0 (1

2

− s) ∂G ∂s− (s) − s(1 − s) ∂  G∂s− (s) , n 1

n 1 2

 for  n = 1, 2, 3, . . . n0 . What is G0 (s)? Find the coefficients G1 (s) and  G2 (s). Hence show that the mean  number of infectives for small  τ  is given by 

− n τ  − 12 n (n − 2)τ 

n0

0

0

2

0

+ O(τ 3 ).

In Example 9.9, the number of susceptibles initially is given by  n0 = 4. Expand  p0 (t), p1 (t) and  p2 (t) in powers of  τ  and confirm that the expansions agree with  G1 (s) and  G2 (s) above. Multiply the difference equation by s and sum from s = 0 to s = n0 giving n0

n0



 p (t)sn = β 

n=0

or,

−1



n0

(n + 1)(n0

n=0

n0

Gt (s, t)

=

βn 0



mpm (t)s −

 − β 

m=2

n0

+β 



n(n

n=1

=

n+1 (t)

n(n0 + 1

β 

n=1

− n) p

n0

m 1

m=1

=

− n) p

 −

βn 0 Gs (s, t)

− 1) p

n0

m 1

m(m − 1) pm (t)s −

n (t)s

0

ss (s, t)

0

s

− βn

0



npn (t)sn

n=1

n

− βn sG (s, t) + βs βn (1 − s)G (s, t) + βs (s − 1)G (s, t), − βsG

n n (t)s ,

s

2

Gss (s, t)

ss

as required. Let τ  = βt . Then the equation for H (s, τ ) = G(s,τ/β ) is ∂H (s, τ ) = (1 ∂τ 

For small τ , let

−



∂H (s, τ ) s) n0 ∂s

−



∂ 2 H (s, τ ) s . ∂s 2

H (s, τ ) = G(s,τ/β ) = H 0 (s) + H 1 (s)τ  + H 2 (s)τ 2 +

···,

and substitute this series into the partial differential equation for H (s, τ ), so that H 1 (s) + 2 H 2 (s)τ  +

· · · = (1 − s)n [H  (s) + H  (s)τ  + · · ·] − s(1 − s)n [H  (s) + H  (s)τ  + · · ·]. 0

0

0

1

0

1

Equating powers of  τ , we obtain nH n (s) = (1

− s)n H  − (s) − s(1 − s)n H  − (s), 0

0

n 1

For τ  = 0,

(n = 1 , 2, 3, . . .).

n 1

n0

H 0 (s) = G(s, 0) =



 pn (0)sn .

n=0

Since the number of susceptibles is n0 at time t = 0. Therefore  pn (0) = n0 , 0

pn (0) = 0,

(n = n0 ).



Hence H 0 (s) = sn . From (i), 0

H 1 (s)

n0 (1

=

2 0

=

2

(s) − s) ∂H ∂s(s) − s(1 − s) ∂  H  ∂s − − s(1 − s)n (n − 1)s n (1 − s)s − n (1 − s)s ,

=

0

0

n0

1

n0

1

142

0

0 2

0

n0

−2

(i)