Solution scott l. miller, donald g. childers probability and random processes

Probability and Random Processes Instructor’s Manual

Probability and Random Processes With Applications to Signal Processing and Communications

Instructor’s Manual Scott L. Miller Professor Department of Electrical Engineering Texas A&M University

Donald G. Childers Professor Emeritus Department of Electrical and Computer Engineering University of Florida

Amsterdam • Boston • Heidelberg • London • New York • Oxford Paris • San Diego • San Francisco • Singapore • Sydney • Tokyo

Senior Acquisition Editor: Barbara Holland Project Manager: Troy Lilly Associate Editor: Tom Singer Marketing Manager: Linda Beattie Cover Design: Eric DeCicco Elsevier Academic Press 200 Wheeler Road, Burlington, MA 01803, USA 525 B Street, Suite 1900, San Diego, California 92101-4495, USA 84 Theobald's Road, London WC1X 8RR, UK Copyright © 2004, Elsevier Inc. All rights reserved. No part of this publication may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopy, recording, or any information storage and retrieval system, without permission in writing from the publisher. Permissions may be sought directly from Elsevier's Science & Technology Rights Department in Oxford, UK: phone: (+44) 1865 843830, fax: (+44) 1865 853333, e-mail: [email protected]. You may also complete your request on-line via the Elsevier homepage (http://elsevier.com), by selecting "Customer Support" and then "Obtaining Permissions." ISBN: 0-12-088475-5 For all information on all Elsevier Academic Press Publications visit our Web site at www.books.elsevier.com Printed in the United States of America 04 05 06 07 08 09 9 8 7 6 5 4 3 2 1

Contents

Solutions to Chapter 2 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1 Solutions to Chapter 3 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .25 Solutions to Chapter 4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .45 Solutions to Chapter 5 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .75 Solutions to Chapter 6 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .107 Solutions to Chapter 7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .121 Solutions to Chapter 8 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .133 Solutions to Chapter 9 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .153 Solutions to Chapter 10 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .169 Solutions to Chapter 11 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .000 Solutions to Chapter 12 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .000

Solutions to Chapter 2 Exercises

Problem 2.1 Given M events A1, A2, . . . , AM : Ai ∩ Aj = φ ∀ i 6= j prove that M [

Pr

!

Ai =

i=1

M X

P r (Ai) .

i=1

We shall prove this using induction. For the case M=2 2 [

Pr

!

Ai =

i=1

2 X

P r (Ai) .

i=1

This we know to be true from the axioms of the probability. Let us assume that the proposition is true for M=k. Pr

k [

!

Ai =

i=1

k X

P r (Ai)

i=1

We need to prove that this is true for M=k+1. Define an auxiliary event B : B=

k [

!

Ai .

i=1

Then the event

k+1 [

!

Ai = (B ∪ Ak+1 )

i=1

Pr

k+1 [

!

Ai = P r (B ∪ Ak+1 ) .

i=1

Since the propostion is true for M=2 we can rewrite the above equation as Pr

k+1 [

!

Ai = P r (B) + P r (Ak+1 ) .

i=1

1

Since the proposition is true for M=k we can rewrite this as =

k X

P r (Ai ) + P r (Ak+1)

=

i=1 k+1 X

P r (Ai ) .

i=1

Hence the propostion is true for M=k+1. And by induction the proposition is true for all M.

Problem 2.2 First, note that the sets A ∪ B and B can be rewritten as A ∪ B = {A ∪ B} ∩ {B ∪ B} = {A ∩ B} ∪ B B = B ∩ {A ∪ A} = {A ∩ B} ∪ {A ∩ B}. Hence, A ∪ B can be expressed as the union of three mutually exclusive sets. A ∪ B = {B ∩ A} ∪ {A ∩ B} ∪ {A ∩ B}. Next, rewrite B ∩ A as B ∩ A = {B ∩ A} ∪ {A ∩ A} = A ∩ {A ∪ B} = A ∩ {A ∩ B}. Likewise, A ∩ B = B ∩ {A ∩ B}. Therefore A ∪ B can be rewritten as the following union of three mutually exclusive sets: A ∪ B = {A ∩ (A ∩ B)} ∪ {A ∩ B} ∪ {B ∩ (A ∩ B)}. Hence

P r (A ∪ B) = P r A ∩ (A ∩ B) + P r (A ∩ B) + P r B ∩ (A ∩ B) . Next, write A as the union of the following two mutually exclusive sets A = {A ∩ {A ∩ B}} ∪ {A ∩ B}. Hence,

P r (A) = P r A ∩ {A ∩ B} + P r (A ∩ B)

2

and

P r A ∩ {A ∩ B} = P r (A) − P r (A ∩ B) . Likewise,

P r B ∩ {A ∩ B} = P r (B) − P r (A ∩ B) . Finally,

P r (A ∪ B) = P r A ∩ (A ∩ B) + P r (A ∩ B) + P r B ∩ (A ∩ B)

= (P r (A) − P r (A ∩ B)) + P r (A ∩ B) + P r (B) − P r (A ∩ B) = P r (A) + P r (B) − P r (A ∩ B) .

Problem 2.3

P r(A ∪ B ∪ C) = P r((A ∪ B) ∪ C) = P r(A ∪ B) + P r(C) − P r((A ∪ B) ∩ C) = P r(A) + P r(B) − P r(A ∩ B) + P r(C) − P r((A ∩ C) ∪ (B ∩ C)) = P r(A) + P r(B) + P r(C) − P r(A ∩ B) − P r((A ∩ C) ∪ (B ∩ C)) = P r(A) + P r(B) + P r(C) − P r(A ∩ B) − (P r((A ∩ C) + P r(B ∩ C)) − P r(A ∩ C ∩ B ∩ C)) = P r(A) + P r(B) + P r(C) − P r(A ∩ B) − P r(A ∩ C) − P r(B ∩ C) + P r(A ∩ B ∩ C)

Problem 2.4 Since A ⊂ B and A ∩ B = A, B = A ∪ {B ∩ A}. Since A and B ∩ A are mutually exclusive, we have P r(B) = P r(A) + P r(B ∩ A) . Hence, by (1) and considering P r(B ∩ A) ≥ 0, P r(A) ≤ P r(B).

Problem 2.5 Pr

M [

!

≤

i=1

M X i=1

3

P r(Ai )

(1)

We shall prove this by induction. For k = 1 this reduces to P r(A1 ) ≤ P r(A1) which is obviously true.For k = 2 we have P r(A1 ∪ A2) = P r(A1 ) + P r(A2 ) − P r(A1 ∩ A2) by the axioms of probability. ⇒ P r(A1 ∪ A2) ≤ P r(A1 ) + P r(A2 ) The equality holding when the events are mutually exclusive. Assume that the propostion is true for M = k. Then we have Pr

!

k [

Ai ≤

k X

i=1

P r(Ai )

i=1

We shall prove that this is true for M = k + 1. Let k [

B=

Ai

i=1

Then the following holds P r(B) ≤

k X

P r(Ai )

(2)

i=1

Since the proposition is true for M = 2 P r(B ∪ Ak+1 ) ≤ P r(B) + P r(Ak+1 )

(3)

Using (2) in (3) we have P r(B ∪ Ak+1 ) ≤

k X

P r(Ai ) + P r(Ak+1 )

i=1

P r(

k+1 [

Ai ) ≤

i=1

P r(

k X

P r(Ai ) + P r(Ak+1 )

i=1 k+1 [

Ai) ≤

i=1

k+1 X

P r(Ai )

i=1

Thus the proposition is true for M = k + 1 and by the principle of induction it is true for all finite M.

4

Problem 2.6 Assume S is the sample space for a given experiment. Axiom 2.1: For an event A in S, nA . P (A) = lim n→∞ n Since nA ≥ 0, and n > 0, P (A) ≥ 0. Axiom 2.2: S is the sample space for the experiment. Since S must happen with each run of the experiment, nS = n. Hence nS =1. P (S) = lim n→∞ n Axiom 2.3a: Suppose A ∩ B = 0. For an experiment that is run n times, assume the event A ∪ B occurs n0 times, while A occurs nA times and B occurs nB times. Then we have n0 = nA + nB . Hence n0 nA + nB nA nB = lim = lim + lim = P (A) + P (B) . P (A ∪ B) = lim n→∞ n n→∞ n→∞ n→∞ n n n Axiom 3.b: For an experiment that is run n times, assume the event Ai occurs nAi times, i = 1, 2, · · ·. Define event C = A1 ∪ A2 · · · ∪ Ai · · ·. Since any two P events are mutually exclusive, event C occurs ∞ i=1 nAi times. Hence, P(

∞ [

Ai) = lim

i=1

n→∞

P∞

i=1

n

nAi

=

∞ X

∞ X nAi P (Ai ) . = n→∞ n i=1 i=1

lim

Problem 2.7 Axiom 2.1:

P r(A, B) ≥0 P r(B) since both P r(A, B) and P r(B) are greater than zero. Axiom 2.2: P r(B) P r(S, B) = =1 P r(S | B) = P r(B) P r(B) Axiom 2.3: P r((A ∪ B) ∩ C) P r((A ∩ C) ∪ (B ∩ C)) P r(A ∪ B | C) = = P r(C) P r(C) P r(A ∩ C) P r(B ∩ C) P r(A ∩ B ∩ C) + − = P r(C) P r(C) P r(C) = P r(A | C) + P r(B | C) − P r(A ∩ B | C) P r(A | B) =

5

Problem 2.8 Show that if P r(B | A) = P r(B) then (a) P r(A, B) = P r(A)P r(B) P r(A, B) P r(A) P r(B | A) = P r(B)

P r(B | A) =

(4) (5)

Equating (4) and (5) we get P r(A, B) P r(A) P r(A, B) = P r(A)P r(B) P r(B) =

(6) (7)

(b) P r(A | B) = P r(A) P r(A | B) =

P r(A, B) P r(B)

(8)

Since (5) ⇒ (7) we get using (7) P r(A)P r(B) P r(B) P r(A | B) = P r(A) P r(A | B) =

(9) (10)

Problem 2.9 Let Ai = ith dart thrown hits target. Method 1: P r(at least one hit out of 3 throws) = = − +

P r(A1 ∪ A2 ∪ A3 ) P r(A1) + P r(A2 ) + P r(A3 ) − P r(A1 ∩ A2) P r(A1 ∩ A3) − P r(A2 ∩ A3) P r(A1 ∩ A2 ∩ A3 ).

P r(Ai ) = 1/4, P r(Ai ∩ Aj ) = 1/16, and P r(Ai ∩ Aj ∩ Ak ) = 1/64. P r(at least one hit) = 3 ·

6

1 1 37 1 −3· + = . 4 16 64 64

Method 2: P r(at least one hit) = 1 − P r(no hits) = 1 − P r(A1 ∩ A2 ∩ A3 ) 3 3 37 = 1 − P r(A1) · P r(A2 ) · P r(A3 ) = 1 − = . 4 64 Note 1: To complete this problem, we must assume that the outcome of each throw is independent of all others. Note 2: The sample space is not equally likely.

Problem 2.10 Let Di = ith diode chosen is defective. P r(D1 ∩ D2 ) = 1 − P r(D1 ∩ D2 ) = 1 − P r(D1 )P r(D2 | D1 ) P r(D1 ) =

25 30

24 29 (after the first diode is selected and found to be not defective, there are 29 diodes remaining of which 5 are defective and 24 are not) P r(D2 | D1 ) =

P r(D1 ∩ D2 ) = 1 −

9 25 24 · = 30 29 29

Problem 2.11 (a) P r(1st = red, 2nd = blue) = P r(1st = red)P r(2nd = blue | 1st = red) 3 P r(1st = red) = 12 5 P r(2nd = blue | 1st = red) = 11 5 3 5 · = . P r(1st = red, 2nd = blue) = 12 11 44 (b) P r(2nd = white) = (c) Same as part (b).

4 12

= 13 .

7

Problem 2.12 (a) There are 2n distinct words. (b) Method 1: 3 P r(2 ones) = P r({110}∪{101}∪{011}) = P r(110)+P r(101)+P r(011) = . 8 Method 2: ! 2 3−2

3 1 · P r(2 ones) = 2 2

·

1 2

3 = . 8

Problem 2.13 (a) There are mn distinct words. (b) For this case there are 43 distinct words. ! 2 4−2

4 1 · P r(2 pulses of level 2) = 3 2

·

2 3

=

8 . 27

Problem 2.14 (a)

!

21 9 1 3 1 9−3 = 0.1641. ( ) ( ) = P r (3 heads) = 128 3 2 2 (b) P r (at least 3 heads) =

9 X i=3

= 1−

!

9 1 i 1 9−i ( )( ) i 2 2 2 X i=0

!

9 1 i 1 9−i 233 = 0.9102 . ( )( ) = i 2 2 256

(c) P r (at least 3 heads and at least 2 tails) = P r (3 ≤ number of heads ≤ 7) =

7 X i=3

8

!

9 1 i 1 9−i 57 = 0.8906 . ( )( ) = i 2 2 64

Problem 2.15 (a) 1 6 5 P r(5) = 6 25 5 5 · = P r(5, 5) = 6 6 36 P r(5) =

(b) Pr(sum = 7) The sum of 7 can occur in the following 6 possible ways. sum = {(1;6), (2;5) , (3;4) , (4;5) , (5;2) , (6;1) }. And there are a total of 36 outcomes in the sample space. P r(sum = 7) =

1 6 = 36 6

(c) A= { (3;5) , (5;3) } P r(A) =

1 2 = 36 18

(d) 1 6 2 P r(B = (5 | 4)) = 6 1 1 2 · = P r(A, B) = 6 6 18 Alternatively the desired event is given by the following set of outcomes X ={ (5;4) , (5,5) } P r(A = 5) =

P r(X) =

1 2 = 36 18

(e) 1 6 1 1 1 · = P r(5, 5) = 6 6 36 P r(5) =

9

(f) P r(A = 6) = P r(B = 6) = P r(A ∪ B) = = =

1 6 1 6 P r(A) + P r(B) − P r(A ∩ B) 1 1 1 1 + − · 6 6 6 6 11 36

Problem 2.16 (a) P r ({1st ∈ (2, 3, 4)} ∩ {2nd ∈ (2, 3, 4)}) = P r (1st ∈ (2, 3, 4)) × P r (2nd ∈ (2, 3, 4)) 1 3 3 = × = . 6 6 4 (b) The possible combinations of the two die can be (6, 4), (5, 3), (4, 2), (3, 1). Hence, 1 1 1 P r (1st − 2nd = 2) = 4 × × = . 6 6 9 (c) Since one roll is 6, all possible combinations of two die are (1, 6), (2, 6), (3, 6), (4, 6), (5, 6), (6, 6), (6, 5), (6, 4), (6, 3), (6, 2), (6, 1). Since only two combinations satisfy the requirement, (4, 6) and (6, 4), we have 2 . 11 (d) Similarly, all possible combinations of two die are (1, 5), (2, 5), (3, 5), (4, 5), (5, 5), (6, 5), (5, 1), (5, 2), (5, 3), (5, 4), (5, 6). Since four combinations satisfy the requirement, (5, 2), (5, 3), (2, 5) and (3, 5), we have P r (sum = 10 | one roll = 6) =

4 . 11 (e) Since the sum is 7, all possible combinations of two die are (4, 3), (5, 2), (6, 1), (3, 4), (2, 5), (1, 6). Hence, we have P r (sum ∈ (7,8) | one roll = 5) =

P r (one roll = 4 | sum = 7) =

10

1 2 = . 6 3

Problem 2.17

P r(defective) = P r(defective | A) · P r(A) + P r(defective | B) · P r(B) 0.15 1 + (0.05) · = 0.137. = (0.15) · 1.15 1.15

Problem 2.18 (a) !

4 3 48 3 · · = 0.0130. · P r A, A, A = 3 · P r (two Aces) = 52 51 50 2 P r (two of any kind) = 13 · P r (two Aces) = 0.1694.

(b) 4 3 2 · · = 0.000181. 52 51 50 P r (three of any kind) = 13 · P r (three Aces) = 0.00235. P r (three Aces) = P r (A, A, A) =

(c) 13 12 11 · · = 0.0129. 52 51 50 P r (three of any suit) = 4 · P r (three Hearts) = 0.0518. P r (three Hearts) = P r (H, H, H) =

(d)

P r (straight) = P r (2, 3, 4) + P r (3, 4, 5) + . . . + P r (Q, K, A) 4 4 4 · · = 0.003092. = 11 · P r (2, 3, 4) = 11 · 52 51 50

11

Problem 2.19 (a) !

5 · P r A, A, A, A, A P r (two Aces) = 2 ! 4 3 48 47 46 5 · · · · = 0.03993. = · 2 52 51 50 49 48 P r (two of any kind) = 13 · P r (two Aces) = 0.5191.

Note that the above calculations also allow that the hand may have two pair or a full house. Hence to be completely accurate (in the poker sense) we must subtract these probabilities. P r (two of a kind) = 0.5191 − P r (two pair) − P r (full house) = 0.5191 − 0.04754 − 0.001441 = 0.4701. Note that Pr(2 pair) is calculated in part (c) and Pr(full house) is calculated in part (e). (b) !

5 · P r A, A, A, A, A P r (three Aces) = 3 ! 4 3 2 48 47 5 · · · · = 0.001736. = · 3 52 51 50 49 48 P r (three of any kind) = 13 · P r (three Aces) = 0.02257.

Note that the above calculations also allow that the hand may have a full house and hence this probability must be subtracted. P r (three of a kind) = 0.02257 − P r (full house) = 0.02257 − 0.001441 = 0.02113. (c) P r (two Aces, two Kings) ! ! 5 3 · · P r A, A, K, K, A ∪ K = 2 2

12

= P r (two pair) =

!

!

4 3 4 3 44 5 3 · · · · = 0.0006095. · · 2 2 52 51 50 49 48 ! 13 · P r (two Aces, two Kings) = 0.04754. 2

(d) 13 12 11 10 9 · · · · = 0.0004952. 52 51 50 49 48 P r (flush) = 4 · P r (five Hearts) = 0.001981.

P r (five Hearts) =

Note that the above calculations also allow that the hand may have a straightflush and hence this probability must be subtracted. 4·9 . P r (straight-flush) = 52 · 51 · 50 · 49 · 48 4 · 9 · (13 · 12 · 11 · 10 − 1) P r (flush) = = 0.001981. 52 · 51 · 50 · 49 · 48 (e) P r (three Aces, two Kings)

!

5 = · P r (A, A, A, K, K) 3 4 3 2 4 3 · · · · = 9.235 × 10−6 . = 10 · 52 51 50 49 48 P r (full house) = 13 · 12 · P r (three Aces, two Kings) = 0.001441. (f) 4 4 4 4 4 · · · · = 3.283 × 10−6 . 52 51 50 49 48 P r (straight) = 4 · P r (10,J,Q,K,A) = 2.955 × 10−5 .

P r (10,J,Q,K,A) =

Problem 2.20 (a)

P r (1 Heart) = 13 · P r H, H, H, . . . , H

28 13 39 38 · · ... = = 13 · 52 51 50 40

13

13 1

·

52 13

39 12

= 0.08006

(b) We can choose anywhere between 7 to 13 cards of a given suit to satisfy the given condition. All these events are mutually exclusive. Let Ai denote the event of having i Hearts. Following a procedure similar to part (a), it is found that 13 39 · i 13−i . P r (Ai) = 52 13

Therefore P r (at least 7 Hearts) = P r

13 [

Ai

!

13 X

=

i=7

= Pr

13 [

i=7

Ai

!

13 X

=

i=7

P r(Ai ) 13 i

i=7

·

39 13−i

52 13

.

The probability of at least 7 cards from any suit is simply 4 times the probability of at least 7 Hearts. Hence !

!

13 4 X 13 39 · = 0.0403. P r (at least 7 cards from any suit) = 52 i 13 − i i=7 13

(c)

P r (no Hearts) = P r H, H, H, . . . , H

39

27 39 38 37 13 · · ... = 52 = 0.01279 = 52 51 50 40 13

Problem 2.21 (a) Let p = P r (Ace is drawn) = 4/52 = 1/13. P r (1st Ace on 5th selection) = (1 − p)4 p =

12 4 1 = 0.0558. · 13 13

(b) P r (at least 5 cards drawn before Ace) = (1 − p)5 =

14

12 5 = 0.6702. 13

(c) P r (1st Ace drawn on 5th selection) =

48 47 46 45 4 · · · · = 0.0599 . 52 51 50 49 48

P r (at least 5 cards drawn before Ace) =

48 47 46 45 44 · · · · = 0.6588 . 52 51 50 49 48

Problem 2.22 (a) = 14 . We If the cards are replaced the probability of drawing a club is p = 13 52 need to find the probability that we ! draw 2 clubs in 7 trials and a club on 7 · p2 (1 − p)5 · p. the 8th trial. This is given by 2 P r(3rd club drawn on 8th selection) =

7 2

!

1 3 · ( )3 ( )5 = 0.0779. 4 4

(b) We need to find the probability that we draw either 0, 1 or 2 clubs in 8 trials. This follows the binomial distribution and this probability is given by P r(at least 8 cards drawn before 3rd club) =

2 X

8 i

i=0

!

1 3 729 = 0.7119. ·( )i ( )8−i = 4 4 1024

(c)

P r(3rd club drawn on 8th selection) = P r(2 clubs drawn in 7 selections)P r(8th card = club | 2 clubs in first 7 selections).

P r(2 clubs in 7 selections) =

13 2

39 5

52 7

= 0.3357. 2 . 45

P r(8th card = club | 2 clubs in first 7 selections) = 2 P r(3rd club drawn on 8th selection) = · 45

15

13 2

39 5

52 7

= 0.0149.

P r(at least 8 cards drawn before 3rd club) = P r(less than 3 clubs in 7 selections) 2 X

=

P r(k clubs in 7 selections)

k=0 2 X

=

13 k

k=0

39 7−k 52 7

= 0.7677.

Problem 2.23 P r(all n versions erased) =

n

1 2

Problem 2.24 (i) (I, M) is not permissible. If A and B are independent, then P r(A, B) = P r(A)P r(B). By assumption, P r(A) 6= 0 and P r(B) 6= 0 and hence for independent events A and B it follows that P r(A, B) 6= 0 and therefore they are not mutuallly exclusive. (ii) (I, NM) is permissible. For example let A = {throw a 1 on first roll of a die} and B = {throw a 2 on second roll of a die}. These two events are independent, but not mutually exclusive. (iii) (NI, M) is permissible. For example let A = {throw a 1 on first roll of a die} and B = {sum of two rolls of a die is 8}. These two events are not independent, but they are mutually exclusive. (iv) (NI, NM ) is permissible. This is the most general case. For example let A = {throw a 2 on first roll of a die} and B = {sum of two rolls of a die is 8}. These two events are not independent, nor are they mutually exclusive.

Problem 2.25 (a) P r(X = k) = (b) n X

n k

pk (1 − p)n−k .

P r(X = k) =

k=0

n X k=0

!

n k p (1 − p)n−k = (p + (1 − p))n = 1n = 1. k

(c) Binomial

16

Problem 2.26 (a) Since ∞ X

PX (k) = 1 ,

k=0

i.e., ∞ X

c

0.37k = 1 .

k=0 c 1−0.37

= 1, which gives c = 0.63. Hence (b) Similarly, ∞ X

c

0.82k = 1 .

k=1 c 1−0.82

= 1 + c, which gives c = 0.2195. Hence (c) Similarly, c

24 X

0.41k = 1

k=0

which gives c = 0.5900. (d) Similarly, c

15 X

0.91k = 1

k=1

which gives c = 0.1307. (e) Similarly, c

6 X

0.412k = 1

k=0

which gives c = 0.8319.

Problem 2.27 (a) The probability mass function for a binomial random variable is given by P r(X = k) =

n k

!

pk (1 − p)n−k

Tabulating the probabilities for various values of k we get the following

17

k 0 1 2 3 4 5 6 7 8 9 10

Probability 0.10737418240000 0.26843545600000 0.30198988800000 0.20132659200000 0.08808038400000 0.02642411520000 0.00550502400000 0.00078643200000 0.00007372800000 0.00000409600000 0.00000010240000

Refer to Figure 1 for the plot.

(b) The probability mass function for a Poisson distribution is given by αk k! Tabulating the probabilities for various values of k we get the following P r(X = k) = e−α

k 0 1 2 3 4 5 6 7 8 9 10

Probability 0.13533528323661 0.27067056647323 0.27067056647323 0.18044704431548 0.09022352215774 0.03608940886310 0.01202980295437 0.00343708655839 0.00085927163960 0.00019094925324 0.00003818985065

Refer to Figure 1 for the plot. k 10−k P 1 4 = 0.0328 (c) Binomial: P r(X ≥ 5) = 1 − P r(X < 5) = 1 − 4k=0 10 k 5 5 P

k

Poisson: P r(X ≥ 5) = 1 − P r(X < 5) = 1 − 4k=0 2k! e−2 = 0.0527 The Poisson approximation is not particularly good for this example.

18

Problem 2.28 (a) λt = P r(X < 3) =

2 X

1 5 1 −1 1 e = (1 + 1 + ) = = 0.9197. k! e 2 2e k=0

(b) λt = P r(X < 3) =

!

10calls 1 minute = 1 · minute 10

!

10calls · (6 minutes) = 60 minute

2 X

3600 60k −60 e ) = 1861·e−60 = 1.627×10−23 . = e−60 (1+60+ k! 2 k=0

Problem 2.29 (a) 1 P r(win with one ticket) = p = 50 = 6.29 ∗ 10−8 . 6

(b)

!

6 ∗ 106 4 6 p (1 − p)6∗10 −4 = 5.8055 ∗ 10−4 . P r(4 winners) = 4 (c) The probability that n people win is !

6 ∗ 106 n 6 p (1 − p)6∗10 −n . P r(n winners) = n Note that when n = 0, the above probability evaluates to 0.6855. Since this probability is greater than 1/2, it is impossible for any other number of winers to be more probable. Hence the most probable number of winning tickets is zero. (d) Here, for the Possion approximation, α = np = 501 ∗ 6 ∗ 106 = 0.3774. (6) Hence, α4 −α 0.37744 −0.3774 e = e = 5.7955 ∗ 10−4 . P (4) = 4! 4!

19

Compared with results in (b), the Possion distribution is an accurate approximation in this example. For the case of (c), P (n) =

αn −α 0.3774n −0.3774 e = e . n! n!

When n = 0, P (n) will be maximum, which is the same as the result in c).

Problem 2.30 4 3 2 · · = 6 5 4 2 4 3 P r(X = 1) = 3 · · · 6 5 4 4 2 1 P r(X = 2) = 3 · · · 6 5 4 P r(X = 0) =

1 . 5 3 = . 5 1 = . 5

Problem 2.31 Let p = P r(success) =

1 . 10

!

10 · p · (1 − p)9 = 0.3874. P r(1 success) = 1 P r(≥ 2 successes) = 1 − P r(≤ 1 success) = 1 − P r(0 successes) − P r(1 success). P r(0 successes) = (1 − p)1 0 = 0.3487. P r(≥ 2 successes) = 1 − 0.3487 − 0.3874 = 0.2639.

Problem 2.32 (a) n k n n−k

! !

=

n! k!(n − k)!

=

n! (n − k)!(n − (n − k))!

20

n n−k n n−k

! !

n! (n − k)!(k)!

=

!

n k

=

(b) n k

!

+

n k+1

!

= = = = = =

n! n! + k!(n − k)! (k + 1)!(n − k − 1)! 1 n! 1 + k!(n − k − 1)! n − k k + 1 ! n! k+1+n−k k!(n − k − 1)! (n − k)(k + 1) ! n! 1+n k!(n − k − 1)! (n − k)(k + 1) (n + 1)! (k + 1)!(n − k)! ! n+1 k+1

(c) n X

!

n k

k=0

= 2n

Consider the binomial expansion of (p + q)n. We have n

(p + q) =

n X

n k

k=0

!

· pk · q (n−k)

Put p = q = 1 . Then we get n

2 =

n X k=0

(d)

21

n k

!

n X

n k

k=0

!

(−1)k = 0

Consider the binomial expansion of (p + q)n. We have n

(p + q) =

n X

n k

k=0

!

· pk · q (n−k)

Put p = −1, q = 1 . Then we get n X

n k

k=0

!

(−1)k = 0

(e) n X

n k

k=0

!

k = n2n−1

Once again consider the binomial expansion (p + q)n . n

(p + q) =

n X

n k

k=0

!

· pk · q (n−k)

Differentiate both sides with respect to p. n d X d (p + q)n = dp dp k=0

n(p + q)

n−1

=

n X

n k

k=0

n k !

!

· pk · q (n−k)

· k · pk−1 · q (n−k)

Once again put p = q = 1. We get the following relation. n−1

n2

=

n X k=0

n k

!

·k

The above relation can be rewritten as follows. n−1

n2

=

n X k=1

n k

!

n 0

· k + 0.

n2n−1 =

n X k=1

22

!

n k

!

!

·k

(11)

(f) n X k=0

!

n · k · (−1)k = 0 k

For this we proceed in the same way as in (e) but we substitute p = −1, q = 1 in (11). This gives us n X

n k

k=0 n X

!

n k

k=0

· k · pk−1 · q (n−k) = 0 !

· k · (−1)k−1 = 0

Multiplying by -1 through out gives the required identity. n X k=0

n k

!

· k · (−1)k = 0

Problem 2.33 (a) If more than 1 error occurs in a 7-bit data block, the decoder will be in error. Thus, the decoder error probability is Pe =

7 X i=2

!

7 (0.03)i (1 − 0.03)7−i = 0.0171 . i

(b) Similarly, Pe =

15 X i=3

!

15 (0.03)i (1 − 0.03)15−i = 0.0094 . i

(c) Similarly, Pe =

31 X i=4

!

31 (0.03)i (1 − 0.03)31−i = 0.0133 . i

23

Probablity Mass Function of Binomial & Poisson Random Variables 0.35 Binomial n=10, p=0.2 Poisson n=10, np= 0.2 0.3

Pr(X=k) −>

0.25

0.2

0.15

0.1

0.05

0

0

1

2

3

4

5 k−>

6

7

Figure 1: Probability Mass Function for Binomial and Poisson Dist.

24

8

9

10


Problem 3.1 fX (x) =

1 b−a

a≤x≤b otherwise.

0

The CDF is given by FX (x) =

Z

x

fX (x) dx −∞

Integrating the above function for various regions of x we get the following  x≤a  0 x−a a≤x≤b FX (x) =  b−a 1 x≥b A plot of this function is shown in Figure 1

CDF(x)

(c,1)

(a,0)

(c,0)

x-->

Figure 1: CDF for Problem 3.1

25

(b,0)

Problem 3.2 (a)   0 x<0 fX (x)dx = x 0≤x<1 FX (x) =  −∞ 1 x≥1 . Z

x

(b)  0 x<0    R x xdx Rx 0≤x<1 Rx R01 FX (x) = −∞ fX (x)dx =  xdx + 1 (2 − x)dx 1 ≤ x < 2   0 x≥2  1 0 x < 0    x2 0≤x<1 2 = x2  − + 2x − 1 1 ≤ x < 2   2 1 x≥2 . CDF for Problem 2(a)

CDF for Problem 2(b)

0.8

0.8

0.6

0.6

FX(x)

1

FX(x)

1

0.4

0.4

0.2

0.2

0

0

−1

0

1

2

−1

0

x

Figure 2: CDFs for Problem 3.2

26

1 x

2

3


a−bx 0

x≤0 otherwise.

(a) Since this is a pdf the following integral should evaluate to 1. Z ∞ fX (x) dx = 1 −∞ Z ∞ Z 0 fX (x) dx + fX (x) dx = 1 −∞ 0 Z 0 Z ∞ −bx a dx + 0 dx = 1 −∞

0

x=0 −a =1 b ln(a) x=−∞ −bx

−1 −a−bx + lim =1 b ln(a) x→−∞ b ln(a)

Here two cases arise a > 1 and 0 < a < 1. The case a < 0 is not possible because then fX (x) can become negative or imaginary and will no longer be a valid pdf. First considering the case a > 1 This evaluates to a finite value only if b < 0. And in the case a < 1 the limit will be finite only if b > 0. In both cases the limit evaluates to 0 and the above eqn reduces to −1 =1 b ln(a) a=e

−1 b

We can see that this relation satisfies the requirements we laid on a and b earlier. Also using this relation the pdf can be written as fX (x) = a−bx 1 −bx = e− b fX (x) = ex

27

(1)

(b) The CDF is given by FX (x) =

Z

FX (x) =

Z−∞ x

x

fX (x) dx ex dx

−∞

Integrating the above function for various regions of x we get the following x x≤0 e FX (x) = 1 0≤x

Problem 3.4 (a) Since

Z

∞

fX (x)dx = 1 , −∞

i.e., c∗

Z

∞

e−2xdx = 1 , 0

which gives c=2. (b) Z

Pr (X > 2) =

∞

fX (x)dx ,

2

Hence,

Z

Pr (X > 2) =

∞

2e−2x dx = e−4 .

2

(c) Pr (X < 3) =

Z

3

2e−2x dx = 1 − e−6 . 0

(d) Since Pr (X < 3 | X > 2) = and Pr (2 < X < 3) =

Z

Pr (2 < X < 3) , Pr (X > 2)

3

2

28

2e−2x dx = e−4 − e−6 ,

and from part (b) Pr (X > 2) = e−4 , we have Pr (X < 3 | X > 2) =

e−4 − e−6 = 1 − e−2 . e−4


x2

c +4

(a) Since this is a probability function the Z ∞ fX (x) dx Z −∞ ∞ c dx 2 −∞ x + 4 x ∞ c arctan( ) 2 2 −∞ c π π ( + ) 2 2 2

following integral evaluates to 1. = 1 = 1 = 1 = 1

c =

2 π

(b) P r(X > 2) Z

∞

2 dx + 4) 2 x=∞ 2 x = · arctan 2π 2 x=2 1 1 π π ·( − )= = π 2 4 4

P r(X > 2) =

π(x2

(c) P r(X < 3) P r(X < 3) = =

Z

3

2 dx + 4) −∞ x=3 x 2 · arctan 2π 2 π(x2

x=−∞

29

3 π 1 · (arctan + ) π 2 2 3 π 1 · (arctan + ) = 0.8128. = π 2 2 =

(d)P r(X < 3|X > 2) P r(2 < X < 3) P r(X > 2, X < 3) = P r(X > 2) P r(X > 2) Z 3 2 dx P r(2 < X < 3) = 2 2 π(x + 4) x=3 x 2 · arctan = 2π 2

P r(X < 3|X > 2) =

x=2

3 2 1 · (arctan − arctan ) = π 2 2 3 π 1 · (arctan − ) = π 2 4 1 3 · (arctan − π4 ) 2 P r(X < 3|X > 2) = π 1 4

4 3 π = · (arctan − ) = 0.2513. π 2 4

Problem 3.6 (a) FX (x) =

Z

x

fX (x)dx = c ∗ −∞

Z

x −∞

π x 1 √ dx = c ∗ arcsin( ) + c ∗ , 2 5 2 25 − x

since FX (∞)=1, we have c=

1 . π

and, FX (x) =

1 x 1 arcsin( ) + . π 5 2

(b) Pr (X > 2) = 1 − FX (2) = 1 −

30

1 2 1 arcsin( ) − = 0.369 . π 5 2

(c) Pr (X < 3) = FX (3) =

1 3 1 arcsin( ) + = 0.7048 . π 5 2

(d) Since Pr (X < 3 | X > 2) =

Pr (2 < X < 3) , Pr (X > 2)

and Pr (2 < X < 3) = FX (3) − FX (2) = 0.07384 , and from part (b) Pr (X > 2) = 0.369 , we have Pr (X < 3 | X > 2) =

0.07384 = 0.2001 . 0.369

Problem 3.7

P r(|S − 10| > 0.075) = 2 = 2 = 2

Z

9.925

9.9 Z 9.925 9.9 Z 0.025

fS (s)ds 100 · (s − 9.9)ds 100 · u · du

0

= 100 · (0.025)2 = 0.0625.

Problem 3.8 Z

∞

x2 ) dx 2 −∞ Z ∞ y2 exp(− ) dy I = 2 −∞ Z ∞ Z ∞ x2 y2 2 exp(− ) dx exp(− ) dy I = 2 2 −∞ −∞ I =

exp(−

31

Since these are totally independent and the operation is linear we can rewrite the above equation as Z ∞Z ∞ y2 x2 2 exp(− ) dy exp(− ) dx I = 2 2 −∞ −∞ Z ∞Z ∞ 2 2 x +y ) dx dy = exp(− 2 −∞ −∞ Using polar coordinates to do this integral we make the substitution x = ρ cos θ and y = ρ sin θ. Then the integral can be written as Z 2π Z ∞ ρ2 = exp(− ) ρ dρ dθ 2 0 0 ∞ 2 Z 2π exp(− ρ2 ) = − dθ 2 0 0 Z 2π 1 dθ = 2 0 2π =π = √2 ⇒I = π

Problem 3.9 Since (x + 2 exp(−(ax +bx+c)) = exp −

b 2 ) 2a

then we have Z Z ∞ 2 exp(−(ax + bx + c))dx =

∞

+

c a

−

b2 4a2

1 a

!

2

= exp(

2

b −c) exp − 4a

(x +

b 2 ) 2a

!

(x +

b 2 ) 2a

1 a

b − c) ∗ exp − dx 1 4a −∞ a ! r Z ∞ b 2 2 (x + ) 1 b π 2a ∗ pπ = exp( − c) exp − dx . 1 4a a a a −∞

−∞

Since 1 pπ

a

Z

∞

exp − −∞

32

exp(

(x + 1 a

b 2 ) 2a

!

dx = 1,

!

,

then

Z

∞

exp(−(ax2 + bx + c))dx = −∞

r

b2 π exp( − c) . a 4a

Problem 3.10 fX (x) = ce−2x

2 −3x−1

(a) As usual the integral of the pdf evaluates to 1. Z ∞ fX (x) dx −∞ Z ∞ c exp{(−2x2 − 3x − 1)} dx −∞ Z ∞ 3x 1 + )} dx c exp{(−2(x2 + 2 2 −∞ Z ∞ 3 3 1 c exp{(−2(x + )2 − ( )2 + )} dx 4 4 2 Z−∞ ∞ 3 3 1 c exp{(−2(x + )2 − ( )2 + )} dx 4 4 2 −∞ Z ∞ 3 1 c exp{(−2(x + )2 − ( )} dx 4 16 −∞ Z ∞ 3 1 c e8 exp{(−2(x + )2 } dx 4 −∞ r 1 π c e8 2 r 2 −1 c= e 8 = 0.7041 π

= 1 = 1 = 1 = 1 = 1 = 1 = 1 = 1

(b) The previous pdf can be rewritten in the form of a standard gaussian pdf as follows fX (x) = c = =

exp{−(2x2 + 3x + 1)} r 2 −1 e 8 exp{−(2x2 + 3x + 1)} π r 3 2 −1 1 e 8 exp{−2(x + )2 } e 8 π 4

33

=

r

3 2 exp{−2(x + )2 } π 4

q (x−µ)2 1 This is in the form of standard gaussian pdf given by 2πσ 2 exp{− 2σ 2 } and we can easily identify the mean m and the standard deviation σ as 1 1 3 m = − , 2σ 2 = ⇒ σ = 4 2 2

Problem 3.11 For the given Gaussian pdf, m = 10 and σ = 5. Hence, (a) P r(X > 17) = Q(

7 17 − 10 ) = Q( ) = 0.0808 . 5 5

(b) P r(X > 4) = Q(

4 − 10 −6 6 ) = Q( ) = 1 − Q( ) = 0.8849 . 5 5 5

(c) P r(X < 15) = 1 − Q(

15 − 10 ) = 1 − Q(1) = 0.8413 . 5

(d) P r(X < −2) = 1 − Q(

−12 12 −2 − 10 ) = 1 − Q( ) = Q( ) = 0.0082 . 5 5 5

(e) P r(|X − 10| > 7) = P r(X < 3) + P r(X > 17) 3 − 10 17 − 10 7 = 1 − Q( ) + Q( ) = 2Q( ) = 0.1615 . 5 5 5 (f) P r(|X − 10| < 3) = P r(7 < X < 13) 13 − 10 3 7 − 10 ) − Q( ) = 1 − 2Q( ) = 0.4515 . = Q(( 5 5 5

34

(g) P r(|X − 7| > 5) = P r(X < 2) + P r(X > 12) 12 − 10 8 2 2 − 10 ) + Q( ) = Q( ) + Q( ) = 0.3994 . = 1 − Q( 5 5 5 5 (h) P r(|X − 4| < 7) = P r(−3 < X < 11) −3 − 10 11 − 10 13 1 = Q(( ) − Q( ) = 1 − Q( ) − Q( ) = 0.5746 . 5 5 5 5

Problem 3.12 (a) Γ(n) = (n − 1)! Z

Γ(x) =

∞

e−t tx−1 dt

Z0 ∞

Γ(x + 1) =

e−t tx dt

0

Integrating by parts we get Γ(x + 1) = =

∞ −e−t tx 0

Z

−

Z

∞

(−e−t ) x tx−1 dt 0

∞ −t

e

xt

x−1

dt

0

= x

Z

∞

e−t tx−1 dt 0

= x Γ(x) Z ∞ Γ(1) = e−t t1−1 dt Z0 ∞ e−t dt = 0 ∞ = −e−t 0

= 1 Γ(2) = (2 − 1) Γ(1) Γ(2) = 1 = 1!

35

Γ(3) = 2 Γ(2) = 2! Γ(4) = 3 Γ(3) = 3! · · · Γ(n) = (n − 1)! (b) Γ(n) = nΓ(n − 1) Refer to (a) (c) Γ( 12 ) =

√

π 1 Γ( ) = 2

Z

∞ 1

e−t t− 2 dt 0

√ Put

t = y. Then dt = 2ydy 1 Γ( ) = 2

Z

∞

e−y 0

= 2

Z

2

1 2y dy y

∞ 2

e−y dy

√0 π = 2 √2 = π

Problem 3.13 (a) FX|A (−∞) = P r(X < −∞|A) = 0 since it’s impossible for anything to be less than −∞. FX|A (∞) = P r(X < ∞|A) =

P r(X < ∞, A) P r(A)

Since {X < ∞} ∩ A = A, then P r(X < ∞|A) = 1. (b) P r(X < x, A) . FX|A(x) = P r(X < x|A) = P r(A) Since all probabilities are ≥ 0, the ratio is also ≥ 0. ⇒ P r(X < x|A) ≥ 0.

36

Since {{X ≤ x} ∩ A} ⊆ A, P r(X ≤ x, A) ≤ P r(A). ⇒ P r(X < x|A) ≤ 1. (c) FX|A (x) =

P r(X < x, A) . P r(A)

For x1 < x2 , {X ≤ x1 } ⊆ {X ≤ x2}. ⇒ {{X ≤ x1} ∩ A} ⊆ {{X ≤ x2 } ∩ A}. ⇒ Pr({X ≤ x1} ∩ A) ≤ P r{X ≤ x2} ∩ A). ⇒ FX|A(x1 ) ≤ FX|A (x2. (d) P r({x1 < X ≤ x2} ∩ A) . P r(A) P r(X ≤ x2, A) − P r(X ≤ x1, A) = P r(A) = FX|A(x2 ) − FX|A (x1).

P r(x1 < X ≤ x2 ) =

Problem 3.14 (a) fX|X>0(x) =

( (

fX (x) Pr (X>0)

0 fX (x) Q(0)

x>0 otherwise

x>0 0 otherwise 2fX (x) x > 0 = 0 otherwise ( x2 √ 2 exp(− 2σ x>0 2) 2 2πσ = 0 otherwise

=

(b)

37

fX||X|<3(x) = = =

( (

fX (x) Pr (−3
−3 < x < 3 otherwise

0 fX (x) 3 1−2Q( σ )

( 0

−3 < x < 3 otherwise x2 exp(− 2σ −3 < x < 3 2)

1 √ 1 3 1−2Q( σ ) 2πσ 2

0

otherwise

(c) fX||X|>3(x) = = =

( (

fX (x) Pr (|X|>3)

0 fX (x) 2Q( σ3 )

|X| > 3 otherwise

|X| > 3

( 0

otherwise 1 x2 √ 1 exp(− 2σ |X| > 3 2) 2Q( 3 ) 2πσ 2 σ

0

otherwise

Problem 3.15 r1 = 1.5 ft = radius of target. r2 = 0.25 ft = radius of bulls-eye. σ = 2 ft. r r2 fR (r) = 2 · exp − 2 · u(r) σ 2σ (a) Z

r1 r2 P r(hit target) = P r(R ≤ r1 ) = fR (r)dr = 1 − exp(− 12 ) 2σ 0 2 1 1.5 = 1 − exp(− · ) = 0.2452. 2 2

Maybe Mr. Hood isn’t such a good archer after all!!

38

Conditional PDF

PDF for Problem 14(a)

PDF for Problem 14(b)

PDF for Problem 14(c)

1.6

1.6

1.6

1.4

1.4

1.4

1.2

1.2

1.2

1

1

1

0.8

0.8

0.8

0.6

0.6

0.6

0.4

0.4

0.4

0.2

0.2

0.2

0 −5−4−3−2−1 0 1 2 3 4 5 x

0 −5−4−3−2−1 0 1 2 3 4 5 x

0 −5−4−3−2−1 0 1 2 3 4 5 x

Figure 3: PDF plots for Problem 3.14; (σ 2 = 1). (b) Z

r2

r2 fR (r)dr = 1 − exp(− 22 ) P r(hit bulls-eye) = P r(R ≤ r2 ) = 2σ 0 2 1 0.25 = 1 − exp(− · ) = 0.0078. 2 2 (c) P r(hit bulls-eye) P r({hit bulls-eye} ∩ {hit target}) = P r(hit target) P r(hit target) 1 0.25 2 1 − exp(− 2 · 2 ) 0.0078 = 0.0317. = = 1 1.5 2 0.2452 1 − exp(− · )

P r(hit bulls-eye|hit target) =

2

39

2

Problem 3.16 Let P r(M = 0) = p0 and P r(M = 1) = p1 1 x2 fX|M =0(x) = √ exp − 2 2σ 2πσ 2 1 (x − 1)2 fX|M =1(x) = √ exp − 2σ 2 2πσ 2 fX (x) = fX|M =0(x).P r(M = 0) + fX|M =1P r(M = 1) p0 p1 x2 (x − 1)2 = √ exp − 2 + √ exp − 2σ 2σ 2 2πσ 2 2πσ 2 fX|M =0(x) P r(M = 0|X = x) = fX (x) 2 x p0 exp − 2σ 2 = (x−1)2 x2 p0 exp − 2σ exp − + p 1 2 2σ 2 = =

1+

p1 exp p0

1+

p1 exp p0

1 2

− (x−1) + 2σ 2

1 x σ2

−

1 2σ 2

x2 2σ 2

We have to plot this function for the following combinations (a) p0 = 12 , p1 = 12 , σ 2 = 1 P r(M = 0|X = x) =

1 1 + exp x − 12

p0 = 12 , p1 = 12 , σ 2 = 5 P r(M = 0|X = x) =

1 1 + exp

x 5

−

1 10

(b) p0 = 14 , p1 = 34 , σ 2 = 1 P r(M = 0|X = x) =

40

1 1 + 3exp x − 12

p0 = 14 , p1 = 34 , σ 2 = 5 P r(M = 0|X = x) =

1 1 + 3exp

x 5

−

1 10

Plot of all the distributions 1 po=p1=0.5,var=1 po=p1=0.5,var=5 po=0.25,p1=0.75,var=1 po=0.25,p1=0.75,var=5

0.9

0.8

0.7

0.6

0.5

0.4

0.3

0.2

0.1

0 −40

−30

−20

−10

0

10

20

Figure 4: P r(M = 0|X = x) for various values of σ and p0

41

30

40

Problem 3.17 (a) We decide a “0” was sent if Pr (M = 0|X = x) ≥ 0.9 . Since Pr (M = 0|X = x) = = =

fX|M =0 (x)Pr (M =0) fX|M =0 (x)Pr (M =0)+fX|M =1 (x)Pr (M =1) fX|M =0 (x) fX|M =0 (x)+fX|M =1 (x) 2 − x2 2σ 2 (x−1)2 − x − 2σ 2 e 2σ2 +e

e

,

we have x≤

1 − σ 2 ln 9 . 2

(2)

Similarly, we decide a “1” was sent if x≥

1 + σ 2 ln 9 . 2

(3)

Finally, the symbol will be erased if neither (2) nor (3) hold, that is if 1 1 − σ 2 ln 9 < x < + σ 2 ln 9 . 2 2

(4)

Hence, we have the following decision rules  x ≤ −1.6972 ,  0, 1, x ≥ 2.6972 ,  erased , −1.6972 < x < 2.6972 . (b) The probability that the receiver erases a symbol is given by P (erased) = = = = = =

P (erased|M = 0)P (M = 0) + P (erased|M = 1)P (M = 1) P (erased|M = 0)/2 + P (erased|M = 1)/2 P (−1.6972 < x < 2.6972|M = 0)/2 + P (−1.6972 < x < 2.6972|M = 1)/2 [Q(−1.6972) − Q(2.6972)] /2 + [Q(−1.6972 − 1) − Q(2.6972 − 1)] /2 1 − Q(1.6972) − Q(2.6972) 0.95167 .

42

(c) The probability that the receiver makes an error is given by P (error) = = = = = =

P (error|M = 0)P (M = 0) + P (error|M = 1)P (M = 1) P (error|M = 0)/2 + P (error|M = 1)/2 P (x ≥ 2.6972|M = 0)/2 + P (x ≤ −1.6972|M = 1)/2 Q(2.6972)/2 + [1 − Q(−2.6972)] /2 Q(2.6972) 0.0034963 .

Problem 3.18 rN (t) = λn =

1 days−1 . 100

The overall failure rate for a serial connection is r(t) =

10 X

rn (t) = 10 · λn =

n=1

1 days−1 , r. 10

(a) The reliability function is R(t) = exp(−rt) and the probability that the system functions longer than 10 days is 1 −1 days = e−1 = 0.3679. R(10 days) = exp −(10 days) 10 (b) Rn (t) = exp(−rn · t) 1 1 −1 = e− 10 = 0.9048. Rn (10 days) = exp −(10 days) days 100 (c) Rn (t) = exp(−rn t), R(t) = exp(−rt),

1 days−1 (component) 100 1 r = days−1 (system) 10

rn =

43


Problem 4.1 (a) Mean of the Distribution !

n k

PX (k) =

n X

µ =

pk (1 − p)n−k , k = 0, 1, 2, . . . n. !

n k

k

k=0

pk (1 − p)n−k

Consider the binomial expansion of (p + q)n n X

n

(p + q) =

n k

k=0

!

pk q n−k

(1)

Differentiating (1) wrt to p we get n(p + q)

n−1

!

n X

n k−1 n−k = k p q . k k=0

Multiplying both sides by p we get np(p + q)n−1

n X

!

n k n−k = k p q k k=0

Substituing q = 1 − p we get !

n X

n k np = k p (1 − p)n−k k k=0 The mean µ is therefore µ = np Second moment of the Distribution E[X 2 ] =

n X 2

k

k=0

n k

45

!

pk (1 − p)n−k

(2)

Starting with (2) and differentiating it again wrt to p we get np(n − 1)(p + q)

n−2

+ n(p + q)

n−1

=

n X

k

2

k=0

!

n k−1 n−k p q k

Multipilying throughout by p we get 2

np (n − 1)(p + q)

n−2

+ np(p + q)

n−1

=

n X 2

k

k=0

!

n k n−k p q . k

Substituting q = 1 − p in the above equation we get 2

np (n − 1) + np =

n X

k

!

n k p (1 − p)n−k k

2

k=0 n X

!

n k n p + np(1 − p) = k p (1 − p)n−k . k k=0 The Variance of the distribution is given by using the relation 2 2

2

σ 2 = E[X 2 ] − E[X]2 σ 2 = n2p2 + np(1 − p) − (np)2 σ 2 = np(1 − p) (b) Poisson Distribution PX (k) = ∞ X

αk −α e k! ∞ X −α

∞ X αk αk −α =e k k! (k − 1)! k=1 k=1

αk −α E[X] = k e =e k! k=0 = e−α

∞ X

∞ X αm+1 αm = α e−α = α e−αeα = α m! m! m=0 m=0

µ = α The Second Moment of the Distribution ∞ ∞ k X X αk 2 2 α −α −α k k2 E[X ] = e =e k! k! k=0 k=1 = e−α −α

= e

σ2 σ2

∞ X

k

k=1 ∞ X

∞ X αk αm+1 = e−α (m + 1) (k − 1)! m! m=0

∞ X αm+1 αm+1 + m m! m=0 m=0 m!

!

−α

= αe

= α e−α (αeα + eα ) = α (α + 1) = α2 + α = E[X 2 ] − E[X]2 = α2 + α − α2 = α

46

∞ X

∞ X αm αm + m m! m=0 m! m=0

!

(c) Laplace distribution |x| 1 exp(− ) 2b b Z ∞ |x| x E[X] = exp(− ) dx b −∞ 2b

fX (x) =

This is an odd function therefore this integral will evaluate to zero. µ=0 Second Moment 2

E[X ] =

Z ∞ −∞

|x| x2 exp(− ) dx = 2 2b b

Z ∞ 0

x x2 exp(− ) dx 2b b

Putting t = xb , E[X 2 ] = 2

Z ∞ 0

(bt)2 exp(−t) b dt = b2 2b

Z ∞

t2exp(−t) b dt = b2 Γ(3).

0

Hence, σ 2 = E[X 2] − E[X]2 = b2 Γ(3) − 0 = 2 b2 . (d) Gamma Distribution fX (x) = E[X] = = = µ =

( xb )c−1 exp(− xb ) U (x) b Γ(c) Z ∞ ( x )c−1 exp(− xb ) dx x b b Γ(c) 0 Z Γ(c + 1) ∞ ( xb )c exp(− xb ) dx b Γ(c) b Γ(c + 1) 0 Γ(c + 1) b Γ(c) bc

Second Moment E[X 2 ] =

Z ∞ −∞

x2

( xb )c−1 exp(− xb ) U (x) dx b Γ(c)

47

= = = σ2 = =

Z ∞

( xb )c+1 exp(− xb ) dx b Γ(c) 0 Z b2 Γ(c + 2) ∞ ( xb )c+1 exp(− xb ) dx Γ(c) b Γ(c + 2) 0 b2 Γ(c + 2) = c (c + 1). Γ(c) E[X 2] − E[X]2 b2c (c + 1) − b2 c2 = b2 c. b2

Problem 4.2 Let N = number of trips until escape T = time to escaspe = 2 N hours. Then E[T ] = 2E[N ] hours. Pr(N = n) = p(1 − p)n−1 , E[N ] =

∞ X

np(1 − p)

n−1

n=1

n≥1

1 (p = ). 3

∞ d X d = −p (1 − p)n = −p dp n=0 dp

1 p

!

=

1 = 3. p

Therefore, E[T ] = 6 hours.

Problem 4.3 Let N = number of packets transmitted until first success. Pr(N = n) = q n−1 (1 − q), E[N ] =

∞ X

nq

n−1

n = 1, 2, 3, . . . .

(1 − q) = (1 − q)

n=1

∞ X

nq n−1

n=1

∞ 1 d X d 1 = . = (1 − q) q n = (1 − q) dq n=1 dq 1 − q 1−q

48

Problem 4.4

T = total transmission time = (N − 1)Ti + N Tt = N (Ti + Tt) − Ti . Ti + Tt − Ti . E[T ] = E[N ](Ti + Tt ) − Ti = 1−q

Problem 4.5 Let

X−µX σX

= Y , then Y ∼ N (0, 1) cs = E

ck = E

"

"

X − µX σX X − µX σX

3 #

4#

=E Y3 .

h

i

(3)

h

i

(4)

=E Y4 .

The coefficient of skewness is calculated as follows: E[Y 3 ] =

Z ∞ −∞

y2 y3 √ exp(− )dy = 0. 2 2π

The integral is zero because the integrand is an odd function of y integrated over an interval syymetric about the origin. The coefficient of kurtosis is calculated as follows: E[Y 4 ] =

Z ∞ −∞

y2 y4 √ exp(− )dy. 2 2π

Perform integration by parts once with u = y 3 and dv = y exp(−y 2/2) resulting in: Z ∞ −∞

y2 y4 √ exp(− )dy = 3 2 2π

Z ∞ −∞

y2 y2 √ exp(− )dy. 2 2π

The remaining integral represents the variance of a standard normal random variable and hence evaluates to 1. Therefore, ck = 3.

49

Problem 4.6 1 (x − µ)2 fX (x) = √ exp − 2σ 2 2πσ 2

!

The central moments can be written as follows. 1 mk = E[(X − µ) ] = √ 2πσ 2 k

Z ∞

(x − µ)2 (x − µ) exp − 2σ 2 −∞ k

!

dx

Making a change of variable x−µ =t σ ⇒ mk

1 = σ √ 2π k

Z ∞

tk exp(−t2/2) dt

−∞

When k is odd, the integrand is an odd function and the total area enclosed is zero. This implies the odd central moments are zero. For even k ≥ 2, let Ik =

Z ∞

tk exp(−t2/2) dt.

−∞

Performing integration by parts with u = tk−1 and dv = t exp(−t2/2)dt produces Ik = (k − 1)

Z ∞ −∞

tk−2 exp(−t2/2) dt = (k − 1)Ik−2 .

With this recursion, it is seen that Ik = (k − 1) · (k − 3) · (k − 5) . . . 3 · 1 · I0. √ Noting that I0 = 2π we get that for even k ≥ 2 E[(X − µ)k ] = σ k (k − 1) · (k − 3) · (k − 5) . . . 3 · 1. This can be written in a more compact form as E[(X − µ)k ] =

50

σ k k! . (k/2)! 2k/2

mk = 2σ k

s

Z ∞

2k π

2

tk e−t dt

0

Substituting t2 = y we can rewrite this equation as mk = 2σ = σ

k

= σk = σk

k

s

s s s

2k Z ∞ k/2 −y dy y e √ π 0 2 y

2k π

Z ∞

2k π

Z ∞

y

k−1 2

e−y dy

0

y

k+1 −1 2

e−y dy

0

2k k + 1 Γ( ) π 2

Finally we can see that if we put k = 2 we should get the variance of the gaussian which is the same as σ 2 m2 = σ

2

s

√ 22 π = σ2 π 2

Problem 4.7 The Cauchy random variable has a PDF given by fX (x) =

b/π . + x2

b2

Since the PDF is symmetric about zero, the mean is zero. The second moment (and the variance) is 2

E[X ] =

Z ∞ −∞

bx2/π dx. b2 + x2

As x → ∞, the integrand approaches a constant (b/π) and hence this integral will diverge. Therefore the varinace of the Cauchy random variable is infinite (undefined).

51

Problem 4.8

µn = µ = µ1 = cn =

Z ∞ −∞ Z ∞ −∞

Z ∞

−∞

xn fX (x) dx xfX (x) dx (x − µ)n fX (x) dx

=

( Z ∞ X n

=

−∞ k=0 ( n X Z ∞

=

k=0 n X k=0

−∞

n k

!

n k n k

! !

)

xk (−µ)n−k fX (x) dx k

x (−µ)

n−k

fX (x) dx

)

µkk (−µ)n−k

Problem 4.9 (a) E[2X − 4] = 2E[X] − 4 = −2 . (b) E[X 2 ] = E[X]2 + V ar(X) = 5 . (c) E[(2X − 4)2 ] = E[(4X 2 + 16 − 16X)] = 4E[X 2 ] − 16E[X] + 16 = 20 .

Problem 4.10 1 (x − µ)2 exp(− ) 2σ 2 2πσ 2 Z ∞ 1 (x − µ)2 E[|X|] = |x| √ exp(− ) dx 2σ 2 −∞ 2πσ 2 fX (x) = √

52

= = = + =

Z ∞

1 t2 |µ + σt| √ exp(− ) dt 2 −∞ 2πσ 2 Z −µ/σ Z ∞ 1 t2 1 t2 (µ + σt) √ exp(− ) dt + (µ + σt) √ exp(− ) dt − 2 2 −∞ −µ/σ 2πσ 2 2πσ 2 "Z # Z ∞ −µ/σ σ t2 t2 √ t exp(− ) dt − t exp(− ) dt 2 2 −∞ 2π −µ/σ "Z # Z −µ/σ 2 ∞ t t2 1 1 √ exp(− ) dt − √ exp(− ) dt µ 2 2 −µ/σ −∞ 2π 2π " !# 2 σ µ √ 2 exp − 2 + µ [Q(−µ/σ) − (1 − Q(−µ/σ))] 2σ 2π s

E[|X|] =

µ2 2σ 2 exp − 2 π 2σ

!

+ µ 1 − 2Q

µ σ

Problem 4.11 Z a/2

E[X] =

−a/2

Z a/2

E[X 2] =

−a/2

Z a/2

E[X 3] =

−a/2

Z a/2

E[X 4] =

−a/2

(a) cs = E (b) ck = E

"

"

X −µ σ

x dx = 0. a a2 a2 x2 dx = ⇒ σ2 = . a 12 12 3 x dx = 0. a a4 x4 dx = . a 80

X −µ σ

4#

=

3 #

=

E[X 3 ] = 0. σ3

144 9 E[X 4 ] a4 /80 = = . = 4 4 σ a /144 80 5

(c) For a Gaussian random variable, cs = 0, ck = 3.

53

Problem 4.12 Z ∞ 0

Z ∞Z ∞

[1 − FX (x)]dx =

0

y

fX (x)dxdy

By interchanging the two integrals, we have Z ∞ 0

[1 − FX (x)]dx =

Z ∞ 0

Z x

fX (x)

dydx =

Z ∞

0

0

xfX (x)dx = E(x)

Problem 4.13 We know the pdf of a distribution can be written as sum of the conditional pdfs. fX (x) = E[X] = =

n X

fX|Ai (x)P r(Ai)

i=1 Z ∞ −∞ Z ∞ −∞

xfX (x) dx x

n X

fX|Ai (x)P r(Ai ) dx

i=1

We can interchange the operations of the integration and summation as they are linear and rewrite the above equation as E[X] =

n Z ∞ X

E[X] =

−∞ i=1 n X

E[X] =

i=1 n X

xfX|Ai (x)P r(Ai ) dx

P r(Ai )

Z ∞ −∞

xfX|Ai (x) dx

P r(Ai )E[X|Ai ]

i=1

Problem 4.14 A convex function is one which has the property that for any α such that 0 ≤ α ≤ 1, and any x0 < x1 g(αx0 + (1 − α)x1 ) ≤ αg(x0 ) + (1 − α)g(x1 ).

54

Applying this property repeatedly, we can show that for any x0 < x1 < x2 < . . . < xn and any discrete distribution ~p = [p0 , p1 , p2 , . . . , pn ] (i.e., 0 ≤ pi ≤ 1 P and ni=0 pi = 1) a convex function g(x) will satisfy g

n X

!

pi xi ≤

n X

i=0

pi g(xi ).

i=0

To start with, suppose X is a discrete random variable. Then we can choose pi such that pi = P r(X = xi ). In that case n X

pi xi = E[X]

i=0 n X

pi g(xi ) = E[g(X)]

i=0

Hence, for discrete random variables g(E[X]) ≤ E[g(X)]. Next, suppose X is a continuous random variable. In this case, let the xi be a set of points evenly spaced by ∆x and let pi = P r(|X − xi| < ∆x/2). As ∆x → 0, pi → fX (xi)∆x. Therefore, lim g

∆x→0

n X

fX (xi )xi∆x

!

i=0

⇒g

Z

xfX (x)dx

≤

lim

∆x→0

≤

Z

n X

fX (xi )g(xi)∆x

i=0

fX (x)g(x)dx

⇒ g (E[X]) ≤ E[g(x)]dx

Problem 4.15

fΘ (θ) =

1 2π

(a) Y = sin θ. This equation has two roots. At θ and π − θ fY (y) =

X θi

dθ fΘ (θ) dy

55

θ=θi

1 1 = + 2π cos θ θ=θ 2π cos θ θ=π−θ 1 = π cos θ 1 √ = π 1 − y2

(b) Z = cos θ. This equation has two roots. At θ and 2π − θ fZ (z) = =

dθ fΘ (θ) dy θi θ=θi 1 + X

2π sin θ θ=θ 1 = π sin θ 1 √ = π 1 − y2

1 2π sin θ θ=2π−θ

(c) W = tan θ. This equation has two roots. At θ and π + θ fW (w) = =

dθ fΘ (θ) dy θi θ=θi 1 + 2 X

2π sec θ θ=θ 1 = π sec2 θ 1 = 2 π(tan θ + 1) 1 = 2 π(w + 1)

1 2 2π sec θ θ=π+θ

Problem 4.16 √ For any value of 0 ≤ y ≤ a2, y = x2 has two real roots, namely x = ± y Then, " √ √ # i fX ( y) fX (− y) h U (y) − U (y − a2 ) + fY (y) = √ √ 2 y 2 y

56

√ √ i fX ( y) + fX (− y) h 2 = U (y) − U (y − a ) √ 2 y i 1 h 2 = U (y) − U (y − a ) . √ 2a y

Problem 4.17 fX (x) = 2 e−2x u(x). (a) X ≥ 0, Y = 1 − X ⇒ Y ≤ 1. (b)

2 e−2x u(x) fY (y) = = 2 exp(−2(1 − y)) u(1 − y). | − 1| x=1−y

Problem 4.18 1 x2 fX (x) = √ exp(− ) 2 2π Y = |X|

The equation Y = |X| has two roots at ±X at all x except at x = 0 fY (y) =

X xi

=

X xi

= = =

s s s

dx fX (x) dy

x=xi 2

x 1 √ exp(− 2 2π

x2 1 exp(− ) + 2π 2 x2 2 exp(− ) π 2 y2 2 exp(− ) π 2

57

dx ) dy x=xi s

x2 1 exp(− ) 2π 2

Strictly speaking we cannot find the pdf for fY (y = 0) using the above equation because Y = |X| is q not differentiable at x = 0. But we can see that 1 fY (y = 0) = fX (x = 0) = 2π fY (y) =

 q 2 2  exp(− y2 ) qπ 1  2π

y 6= 0 y=0

Problem 4.19 For X > 0 (and hence Y > 0), Y = X and thus fY (y) = fX (y). For X < 0, Y = 0 and hence Pr(Y = 0) = Pr(X < 0) = 1 − Q(0) = 1/2. So the PDF of Y is of the form !

1 y2 1 u(y). fY (y) = δ(y) + √ exp − 2 2 2π

Problem 4.20

fY (y) = Pr(X > 1)δ(y − 2) + Pr(X < 1)δ(y + 2) + 1/2fX (y/2)u(y + 2)u(2 − y) ! 1 y2 exp − 2 . = Q(1/σX )[δ(y − 2) + δ(y + 2)] + q 8σX 2 2πσ 2 X

Problem 4.21

fY (y) =

fX (x) dy dx

=

√1 2 2πσX

exp

2 − 2σx 2 X

|3x2 |

x=y 1/3

Problem 4.22 h

(a) X ∈ − 12 , 12 .

h

(b) X is uniform over − 12 , 12 . (c) E[X 2] =

R 1/2

−1/2

x2dx =

1 . 12

58

x=y 1/3

!

y 2/3 q = exp − 2 . 2 2σX 3y 2/3 2πσX 1

0.25

0.2 Q(1)

Q(1)

fY(y)

0.15

0.1

0.05

0 −2.5

−1.5

−0.5

0.5

1.5

2.5

y

Figure 1: PDF for Problem 4.20; σX = 1.

Problem 4.23 (a) 1 Pr(Y = 0) = Pr(X < 0) = . 2 1 Pr(Y = 1) = Pr(X > 0) = . 2 (b)

1 1 =Q = 0.3085. 2 2 1 1 =1−Q = 0.6915. Pr(Y = 1) = Pr(X > 0) = Q − 2 2

Pr(Y = 0) = Pr(X < 0) = 1 − Q −

59

Problem 4.24

fY (y) =

dx fX (x) dy

x=g −1 (y)

b/π b2 +1/y 2 y2

1 = 2 fX (x) = 1 y x=

=

b/π . +1

b2 y 2

y

Problem 4.25 xc−1 exp(− x2 ) 2c Γ(c) √ X =

fX (x) = Y

fY (y) =

dx fX (x) dy

xc−1 exp(− x2 ) √ (2 x) 2c Γ(c) xc−1 exp(− x2 ) √ x = 2c−1 Γ(c) =

2

(y 2 )c−1 exp(− y2 ) y = 2c−1 Γ(c) 2

y 2c−1 exp(− y2 ) u(y) fY (y) = 2c−1 Γ(c)

Problem 4.26 For the transformation from arbitrary to uniform, we want

fX (x)

fY (y) = dg dx

= 1. x=g −1 (y)

One obvious way to achieve this would be to select the transformation such that dg = fX (x) ⇒ g(x) = FX (x). dx

60

Hence Y = FX (x) transforms X ∼ fX (x) to Y ∼ uniform(0, 1). For the transformation from uniform to abitrary, just use this result in reverse. Y = FY−1(X), will transform X ∼ uniform(0, 1) to Y ∼ fY (y).

Problem 4.27 From the previous problem, the transformations should be chosen according to Y = FY−1 (X). (a) Exponential Distribution fY (y) = b e−b y u(y) ⇒ FY (y) = 1 − exp(−b y). 1 X = 1 − exp(−b Y ) ⇒ Y = − ln(1 − X). b Note that since X is uniform over (0, 1), 1 − X will be as well. Hence Y = − 1b ln(X) will work also. (b)Rayleigh Distribution y y2 fY (y) = 2 exp − 2 σ 2σ Y2 X = 1 − exp − 2 2σ

!

!

!

y2 u(y) ⇒ FY (y) = 1 − exp − 2 . 2σ

⇒Y =

q

−2σ 2 ln(1 − X) or

q

−2σ 2 ln(X).

(c)Cauchy Distribution

b/π 1 1 y ⇒ FY (y) = + tan−1 . fY (y) = 2 2 b +y 2 π b

Y 1 1 X = + tan−1 2 π b (d)Geometric Distribution

. ⇒ Y = b tan(πx − π/2) = −b cot(πx).

fY (y) =

∞ X

(1 − p)pk δ(y − k)

FY (y) =

k=0 ∞ X

βk u(y − k),

k=0

where βk =

k X m=0

Pr(X = m) =

k X m=0

61

(1 − p) pk = 1 − pk+1 .

1

0.9

0.8

0.7

x=FY(y) −−>

0.6

0.5

0.4

0.3

0.2

0.1

0

0

1

2

3

4

5 y −−>

6

7

8

9

10

Figure 2: X vs Y for the Geometric Dist.

This is a staircase function and its inverse is also a staircase function as shown in Figure 2. The inverse function can be expressed in terms of step functions as: ∞ FY−1 (x) =

X

u(y − βk ).

k=0

Y can be written in a little more convenient form as follows: Y =

∞ X

u(X − (1 − pk )).

k=0

Note: This transformation works for any integer valued discrete rand variable. The only change would be the values of the βk . (e) Poisson Distribution - Using the results of part (d): Y = FY−1(x) =

∞ X

u(y − βk ),

k=0

where in this case, βk =

k X

Pr(X = m) =

m=0

62

k X

bm −b e . m=0 m!

1

0.9

0.8

0.7

x=FY(y) −−>

0.6

0.5

0.4

0.3

0.2

0.1

0

0

1

2

3

4

5 y −−>

6

7

8

9

10

Figure 3: X vs Y for the Poisson Dist.

Problem 4.28 h

i

h

i

h

i

ΦY (ω) = E ejωY = E ejω(aX+b) = ejωb E ejωaX = ejωb ΦX (aω).

Problem 4.29 ΦX (ω) =

Z ∞ −∞

fX (x) ejωx dx

(a) Φ∗X (ω)

=

Z ∞ −∞

fX∗ (x) e−jωx dx

Since the pdf is a real function its complex conjuate will be the same as itself. Using this we can rewrite the above equation as Φ∗X (ω) =

Z ∞

−∞

fX (x) e−jωx dx

Φ∗X (ω) = ΦX (−ω) (b) ΦX (ω) = ΦX (0) =

Z ∞ −∞ Z ∞ −∞

ΦX (0) = 1

63

fX (x) ejωx dx fX (x) dx

The integral on the RHS evaluating to 1 as it is a valid pdf. (c) ΦX (ω) = |ΦX (ω)| = ≤ ≤

Z ∞ Z−∞ ∞

fX (x) ejωx dx

jωx fX (x) e dx −∞ Z ∞ jωx fX (x) e dx −∞ Z ∞ −∞

fX (x) dx because |ejωx| ≤ 1

≤ 1 because

R∞

−∞

fX (x) = 1 as fX (x) is a pdf

(d) If fX (x) is even, fX (−x) = fX (x) Z ∞

ΦX (ω) =

−∞ Z ∞

Φ∗X (ω) =

−∞

fX (x) ejωx dx fX (x) e−jωx dx

If we make a change of variable and put t = −x we get Φ∗X (ω) Φ∗X (ω)

= − =

Z

Z −∞ ∞ ∞

−∞

fX (−t) ejωt dt

fX (t) ejωt dt

Φ∗X (ω) = ΦX (ω) Since both the ΦX (ω) and its complex conjugate are equal it must be real. (e) Let us assume that the ΦX (ω) is purely imaginary. Then ΦX (0) is also purely imaginary. But we know that ΦX (0) = 1. This cannot be possible if ΦX (ω) were purely imaginary. So our assumption about ΦX (ω) being purely imaginary must be false and ΦX (ω) cannot be purely imaginary.

64

Problem 4.30 For an integer valued discrete random variable X, h

i

φX (2πn) = E ej2πnX =

X j2πnk

e

Pr(X = k).

k

Since ej2πnk = 1 for any integers n and k, we have φX (2πn) =

X

Pr(X = k) = 1.

k

To prove the reverse, note that since |ej2πnX | ≤ 1, the only way we can get E[ej2πnX ] = 1 would be if ej2πnX = 1 with probability 1. This means that 2πnX must always be some multiple of 2π or equivalently nX must be an integer for any integer n. This will only happen if X takes on only integer values.

Problem 4.31 (a) Characteristic Function |x| 1 exp(− ) 2b b Z ∞ 1 − |x| jωx e b e ΦX (ω) = −∞ 2b Z ∞ 1 − |x| jωx = e b e −∞ 2b Z 0 Z ∞ 1 x jωx 1 − x jωx b e e + e be = 2b −∞ 2b 0  fX (x) =

"

x

1  e b +jωx = 1 2b + jω b =

1 2b

1 = 2b =

# x=0

"

x

e− b +jωx + − 1b + jω x=−∞

1 b

1 1 − 1 + jω − b + jω

1 b

1 + + jω

1 b

1 1 + b2ω 2

65

1 − jω

!

!

# x=∞   x=0

(b) Taylor Series Expansion of ΦX (ω). ΦX (ω) = =

1 1 + b2ω 2 ∞ X

(−1)k (bω)2k

k=0

(c) k th Moment of X.

dk ΦX (ω) E[X k ] = (−j)k dω k ω=0 ∞ X

1 ΦX (ω) = k! k=0

!

∞ X dk ΦX (ω) k ω = (−1)m b2mω 2m dω k ω=0 m=0

Since there are no odd powers in the Taylor series expansion of ΦX (ω), all odd moments of X are zero. For even values of k, we note from the above expressions that ! 1 d2k ΦX (ω) = (−1)k b2k (2k)! dω 2k ω=0

dk ΦX (ω) ⇒ = (k!)j k bk dω k ω=0 ⇒ E[X k ] = (k!)bk .

Problem 4.32 h2 = E[X(X − 1)] = E[X 2 ] − E[X] = E[X 2 ] − h1 . Then we have E[X 2] = h1 + h2 . Hence the variance is 2 = E[X 2 ] − µ2X = h1 + h2 − h21 . σX

Problem 4.33 ∞ X

1 HX (z) = k=0 k!

!

∞ X dk HX (z) 1 k hk (z − 1)k . (z − 1) = k dz k! k=0 z=1

66

If the Taylor Series coefficients are ηk , that is HX (z) =

∞ X

ηk (z − 1)k ,

k=0

then hk = (k!)ηk .

Problem 4.34 Poisson: HX (z) =

∞ X µk −µ k (µz)k e z = e−µ = e−µ eµz = eµ(z−1) k! k! k=0 k=0 ∞ X

. Binomial: HX (z) =

n X k=0

!

!

n X n k n p (1 − p)n−k z k = (pz)k (1 − p)n−k k k k=0

= (pz + 1 − p)n = (1 + p(z − 1))n . Let µ = np. µ(z − 1) ⇒ HX (z) = 1 + n

!n

As n → ∞ lim HX (z) = lim

n→∞

n→∞

µ(z − 1) 1+ n

!n

= exp(µ(z − 1))

Problem 4.35 (a) HX (z) =

∞ X

=

k=0 ∞ X

PX (k)z k

αk −α k e z k=0 k! −α

= e

∞ X

(αz)k k! k=0

= eα(z−1) .

67

(b) ∞ X

αk (z − 1)k . k=0 k!

eα(z−1) = (c) Noting that

∞ X

hk (z − 1)k , k! k=0

HX (z) = we get hk = αk .

Problem 4.36 1 1 − zn HX (z) = n 1−z X 1 n−1 zk = n k=0 We know that ∞ X

HX (z) =

PX (X = k)z k

k=0 n−1 X

1 n

=

zk =

k=0

n−1 X k=0

1 k z n

Recognizing that the coeffecient of z k in the above equation is the PX (X = k) we get the PMF of the distribution as PX (X = k) =

(

1 n

k = 0, 1, 2, . . . n − 1 0 otherwise

Problem 4.37 MX (u) = E[euX ] = =

Z ∞

x2

Z ∞ 0

fX (x)eux dx

xe− 2 eux dx

0

68

Z ∞

=

1

2

1

2

xe− 2 (x−u) e 2 u dx

0

(5) Let t = x − u, we have MX (u) = e

1 2 u 2

1

= e2u

2

Z ∞

te

− 21 t2

−u 1

2

dt + u Z ∞

Z ∞

e

−u 1 2

− 12 t2

e− 2 u + u e− 2 t dt −u 1 2 √ = 1 + e 2 u u 2πQ(−u) 1 2 √ = 1 + e 2 u u 2π[1 − Q(u)]

dt

(6)

Problem 4.38 x2 + a2 ) Io(ax) U (x) 2 Z ∞ x2 + a2 uX 2 ux2 ) Io(ax) dx E[e ] = e x exp(− 2 0 Z ∞ x2 − 2ux2 + a2 ) Io(ax) dx = x exp(− 2 0 fX (x) = x exp(−

Put t2 = x2 (1 − 2u). 2

Z ∞

at t2 + a2 dt t ) Io( √ exp(− )√ 2 0 1 − 2u 1 − 2u 1 − 2u 2 a a2 2 2 Z ∞ t + (1−2u) a − (1−2u) 1 at ) dt t exp(− = exp(− ) ) Io( √ 1 − 2u 2 2 0 1 − 2u

E[euX ] =

√

ua2 1 exp( ) = 1 − 2u 1 − 2u ua2 1 exp( ) = 1 − 2u 1 − 2u

Z ∞

t exp(−

0

t2 +

a2 (1−2u)

2

at ) Io( √ ) dt 1 − 2u

The last step is accomplished by noting that the remaining integrand is a properly normalized Rician PDF and hence must integrate to one.

69

Problem 4.39 Note that |X − µ| ≥ is equivalent to |X − µ|n ≥ n . Applying Markov’s inequality results in E[|X − µ|n ] n n . Pr (|X − µ| ≥ ) ≤ n

Problem 4.40 Pr(X ≤ x0) = ≤

Z x0 −∞

Z ∞

−∞

fX (x)dx =

Z ∞ −∞

fX (x)u(x0 − x)dx

fX (x) exp(u(x − x0))dx

= exp(−uxo)

Z ∞ −∞

(u ≤ 0)

fX (x) exp(ux)dx

= exp(−uxo)MX (u). The bound is tightened by finding the value of u which makes the right hand side as small as possible. Therefore, P r(X ≤ x0) ≤ min exp(−uxo)MX (u). u≤0

Problem 4.41 ∞ αk −α sk X (αes )k MX (s) = e e = = exp(αes ) exp(−α) = exp(α(es − 1)). k! k=0 k! k=0 ∞ X

Pr(X ≥ n0 ) ≤ min e−sn0 MX (s) s≥0

−sn0

e

MX (s) = exp(−sn0 + α(es − 1))

d −sn0 e MX (s) = exp(−sn0 + α(es − 1))[−n0 + αes ] = 0 ds ⇒ n0 = αes s = ln(n0/α). This will be the minimizing value of s provided that n0 ≥ α. In which case, Pr(X ≥ n0 ) ≤ exp(−n0 ln(n0/α) + n0 − α) n0 α exp(n0 − α) (for n0 ≥ α). = n0

70

Problem 4.42 MX (u) =

1 bΓ(c)

Z ∞ c−1 x

b

0

e−x/b eux dx =

1 bΓ(c)

Z ∞ c−1 x 0

b

exp(−x(1/b−u))dx.

Let y = x(1/b − u) ⇒ dy = dx(1/b − u). Then the moment generating function is given by MX (u) =

Z ∞ 1 1 y c−1 exp(−y)dy = . c (1 − bu) Γ(c) 0 (1 − bu)c

The Chernoff bound is then computed as follows: Pr(X ≤ xo ) ≤ min e−sxo MX (s) s≥0 −sxo

e (1 − bu)c " # e−sxo bc d −sxo e MX (s) = −x0 + =0 ds (1 − bu)c 1 − bs e−sxo MX (s) =

Hence, the minimizing value of s is given by s=

x0 − bc x0 b

provided that (x0 > bc). Plugging in this value produces

x0 = exp − + c e b c 1 x0 c = MX (s) = x0 −bc bc 1 − x0 c x0 x0 Pr(X ≤ xo) ≤ exp − + c bc b −sxo

(x0 > bc).

Problem 4.43 1 (from results of the last exercise). n (1 − u) Ψ(u) = ln(MX (u)) = −n ln(1 − u)

MX (u) =

71

λ(u) = Ψ(u) − ux0 − ln(u) = −n ln(1 − u) − ux0 − ln(u) n 1 − x0 − λ0 (u) = 1−u u n 1 λ00 (u) = 2 + 2 u (1 − u) The saddlepoint is given by n 1 − x0 − = 0 1−u u This is a quadratic equation in u whose roots are given by λ0 (u) = 0 ⇒

u0 =

x0 − 1 − n ±

q

(n + 1 − x0 )2 + 4x0 . 2x0

Note we take the negative square root so that the saddlepoint satisfies u0 < 0. In summary, the saddlepoint approximation is Pr(X ≤ x0 ) ≈ −

u0 =

MX (u0 ) exp(−u0 x0) q

u0 2πλ00(u0 )

n + x0 − 1 ±

q

(n + x0 − 1)2 + 4x0 2x0

1 (1 − u)n n 1 λ00 (u) = 2 + 2 u (1 − u)

MX (u) =

The exact value of the tail probability is given by Pr(X ≤ x0 ) = 1 − e−x0

n−1 X k=0

xk0 . k!

Figure 4 shows a comparison of the exact probability and the saddlepoint approximation.

Problem 4.44 The tail probability can be evaluated according to Pr(X > x0 ) = Pr(X 2 > x20) ≈

72

MX 2 (u0) exp(−ux20) q

u0 2πλ00 (u0)

.

Tail Probabilities − Erlang distribution, n=50 1 Exact Tail Probability Saddlepoint Approximation

0.6

0

Pr(X
0.8

0.4

0.2

0 25

30

35

40

45

50

x

0

Figure 4: Tail Probabilities for Problem 4.43; n = 50. The necessary functions are calculated as follows: MX 2 (u) =

exp

ua2 1−2u

(from results of Problem 4.38)

1 − 2u

ua2 − ux20 − ln(u) 1 − 2u a2 2 1 0 2 + − x − λ (u) = 0 1 − 2u (1 − 2u)2 u 2 4 4a 1 λ00 (u) = + + 2 2 3 (1 − 2u) (1 − 2u) u λ(u) = − ln(1 − 2u) +

The saddlepoint is the solution to λ0 (u) =

a2 2 1 + − x20 − = 0 2 1 − 2u (1 − 2u) u

⇒ u[2(1 − 2u) + a2 − x20 (1 − 2u)2 ] − (1 − 2u)2 = 0. Since this equation is cubic in u, we would need to solve it numerically. Once the necessary root is found (we must take care to find a root which is positive), the saddlepoint approximation is then found according to: Pr(X > x0) ≈

exp

u(1 − 2u) 2π

r

73

ua2 1−2u

− ux20

4 (1−2u)2

+

4a2 (1−2u)3

+

1 u2

exp

=r 2π 4u2 +

ua2 1−2u

− ux20

4a2 u2 (1−2u)

+ (1 − 2u)2

.

Problem 4.45 (a) H(X) =

1 1 3 log2(2) + 2 · log2(4) = = 1.5bits/symbol. 2 4 2

(b) Symbol Probability Codeword 1 1/2 (1) 2 1/4 (01) 3 1/4 (00)

Length 1 2 2

The average codeword length is L=

1 1 · 1 + 2 · · 2 = 1.5bits/symbol. 2 4

74


Problem 5.1 fX,Y (x, y) = ab e−(ax+by) u(x) u(y) Z xZ y

FX,Y (x, y) =

0

ab e−(ax+by) dx dy

0

We can write this integral as follows FX,Y (x, y) = ab

Z x

e−axdx

0

Z y

# 0 −ax x=x

"

e

= ab −

a −ax

= (1 − e

e−by dy

"

x=0

e−by − b

−by

)(1 − e

# y=y

y=0

)

(b) Marginal PDFs fX (x) = =

Z ∞

fX,Y (x, y) dy

Z0∞

ab e−(ax+by) dy

0 −ax

= ab e

"

e−by − b

# y=∞ y=0

= a e−ax Because of the symmetry we see in the equation we can say that the marginal pdf of Y would be fY (y) = b e−by (c) P r(X > Y ) P r(X > Y ) =

Z ∞Z ∞ 0

y

75


=

Z ∞

−by

ab e

0

= =

Z ∞ Z0∞ 0

−by

be

"

e−ax dy − a h

dy −e

x=y

i x=∞ x=y

b e−by dye−ay

"

e−(by+ay) = b − a+b =

−ax

# x=∞

# y=∞ y=0

b a+b

(d) P r(X > Y 2 ) P r(X > Y ) =

Z ∞Z ∞

=


y2

0

Z ∞

−by

ab e

0

=

Z ∞

"

e−ax dy − a

b e−by dy

h

=

x=y

i x=∞ −e−ax 2 x=y

0

Z ∞

# x=∞ 2

b e−by dy e

−ay 2

0

=

Z ∞

b e−by dy e−ay

2

0

= b

Z ∞ 0

= be

b2 4a

= be

b2 4a

= be

b2 4a

2

e−(by+ay ) dy

Z ∞

b

2

e−a(y+ 2a ) dy

0

r r

π 0 + b/2a ) Q( q a 1/2a b π Q( √ ) a 2a

Problem 5.2 (a) Since Z ∞Z ∞ 0

0

d u(x)u(y)dxdy = 1 , (ax + by + c)n

76

(1)

we should consider the cases for different n. When n = 1 or n = 2, the integration in (1) can not be 1. Hence we only consider when n ≥ 3. Since Z ∞Z ∞ 0

0

d d u(x)u(y)dxdy = , n (ax + by + c) (n − 1)(n − 2)abcn−2

we have d = (n − 1)(n − 2)abcn−2 , n ≥ 3. (b) Z ∞

d u(y)dy (ax + by + c)n 0 d = b(n − 1)(ax + c)n−1 (n − 2)acn−2 = . (ax + c)n−1

fX (x) =

Z ∞

d u(x)dx (ax + by + c)n 0 (n − 2)bcn−2 . = (by + c)n−1

fY (y) =

(c) P r(X > Y ) =

Z ∞Z ∞ 0

=

Z ∞ 0

=

d u(x)u(y)dxdy (ax + by + c)n y d dy a(n − 1)(ay + by + c)n−1

b . (a + b)

Problem 5.3 fX,Y (x, y) = d exp(−(ax2 + bxy + cy 2 ))

We note immediately, that we must have a > 0 and c > 0 for this joint PDF to go to zero as x, y → ±∞.

77

(a) Relation between a, b, c, d To be a valid pdf, the following integral must evaluate to 1: Z ∞Z ∞ −∞ −∞

Z ∞Z ∞

d exp(−(ax2 + bxy + cy 2 )) dx dy = 1

−∞ −∞

d

Z ∞Z ∞ −∞ −∞

fX,Y (x, y)dx dy = 1

!

b2 y 2 − cy 2) dy dx exp( 4a Z ∞ r b2 y 2 π exp( − cy 2) dy d a 4a −∞ ! r Z ∞ b2 2 π d exp −(c − )y dy a −∞ 4a r s π π d b2 a (c − 4a )

by exp −a(x + )2 2a

2πd

s

= 1 = 1

(for a > 0)

= 1

(for 4ac > b2)

= 1

1 = 1 4ac − b2 √ 4ac − b2 ⇒d = 2π

The restriction on a, b, c are that a > 0, c > 0 and 4ac > b2. (b) Marginal pdfs of X, Y fY (y) =

Z ∞

fX,Y (x, y)dx

−∞ Z ∞

= d

exp(−(ax2 + bxy + cy 2)) dx

−∞

Z ∞ by b2 y 2 2 − cy ) exp −a(x + )2 = d exp( 4a 2a −∞ r 2 2 b y π − cy 2) = d exp( 4a a √ ! r 2 2 b 4ac − b π exp −y 2(c − ) = 2π a 4a s

fY (y) =

b2 4ac − b2 exp −y 2(c − ) 4πa 4a

78

!

!

dx

Because of the symmetry in the equation in X, Y we can write the pdf of fX (x) as follows s

fX (x) =

b2 4ac − b2 2 exp −y (a − ) 4πc 4c

!

(c) P r(X > Y ) =

Z ∞ Z ∞ −∞

y

fX,Y (x, y) dx dy 







!! Z !2 r 2 ∞ by   πZ∞ a 2 4ac − b   exp −a x + = d exp −y dx dy a −∞ 4a π 2a y r r

π = d a

Z ∞ −∞

exp −y

2

4ac − b2 4a

!!

√ Q

! !

b 2a 1 + y dy 2a

In order to evaluate this last integral we rewrite the Q-function by defining P (x) = Q(x) − 12 . Then !! "

r

! !#

Z √ b π ∞ 4ac − b2 1 P r(X > Y ) = d exp −y 2 2a 1 + y +P a −∞ 4a 2 2a !! r Z 2 4ac − b d π ∞ exp −y 2 dy = 2 a −∞ 4a !! ! ! r Z ∞ 2 √ b π 2 4ac − b + d exp −y P 2a 1 + y dy a −∞ 4a 2a

The second integral in this last expression is zero because the integrand is odd. The first integral is evaluated using the Gaussian normalization integral. Hence, r s 1 d π 4πa = . P r(X > Y ) = 2 2 a 4ac − b 2

Problem 5.4 (a) Let x = r cos(θ) and y = r sin(θ). Then we have Z Z

fX,y (x, y)dxdy = c

79

Z 2π Z 1 0

0

√ r 1 − r2 drdθ

dy

Z 1

= 2πc

√ r 1 − r2 dr

0

2πc = 3 3 . ⇒c = 2π (b) P r(X 2 + Y 2 > 1/4)

= =

3 2π

P r(r > 1/2) √ r 1 − r2 drdθ

Z 2π Z 1 0

Z 1

=

3

=

√ 3 3 . 8

1/2

√ r 1 − r2 dr

1/2

(c) Since the joint PDF is symmetric with respect to the line y = x, 1 Pr(X > Y ) = . 2

Problem 5.5 1 (x − 1)2 + (y + 1)2 fX,Y (x, y) = exp − 8π 8

!

(a)P r(X > 2, Y < 0) P r(X > 2, Y < 0) = = = =

!

Z ∞Z 0

(x − 1)2 + (y + 1)2 1 exp − 8 2 −∞ 8π Z ∞ Z 0 2 (x − 1) (y + 1)2 1 1 √ √ ) dx ) dy exp(− exp(− 8 8 2 −∞ 8π 8π 2−1 0+1 1 1 Q Φ =Q Φ 2 2 2 2 Q(1/2) (1 − Q(1/2))

80

(b)P r(0 < X < 2, |Y + 1| > 2) = P r({0 < X < 2, Y > 1} ∪ {0 < X < 2, Y < −3}) =

Z 2 Z −3 0

= = = =

−∞

fX,Y (x, y)dx dy +

Z 2Z ∞

Z 2

0

1

fX,Y (x, y)dx dy

Z

−3 1 (x − 1)2 (y + 1)2 1 √ exp(− √ exp(− ) dx ) dy + 8 8 0 −∞ 8π 8π 0−1 2−1 −3 + 1 1+1 Q −Q Φ +Q 2 2 2 2 1 1 Q − −Q (Φ (−1) + Q (1)) 2 2 1 2Q(1) 1 − 2Q 2

Z ∞ 1

(y + 1)2 1 √ exp(− ) dy 8 8π

(c) P r(Y > X) P r(Y > X) =

Z ∞Z y −∞ −∞

fX,Y (x, y)dx dy

This integral is easier to do with a change of variables. Let us use the substitution

J " det

u = x−y v = x+y ! ! u v 1 −1 = x y 1 1 #

u v = 2 x y fX,Y (x, y) fU,V (u, v) = 2 P r(Y > X) = P r(U < 0) ! Z ∞Z 0 (x − 1)2 + (y + 1)2 1 1 exp − du dv = 8 2 −∞ −∞ 8π ! Z ∞Z 0 1 (u − v − 2)2 + (u + v + 2)2 = exp − du dv 16π −∞ −∞ 32 ! (u2 + v 2 + 4 − 4u 1 Z ∞Z 0 exp − du dv = 16π −∞ −∞ 16

81

!

= = = =

Z

!

Z

∞ 0 1 (u − 2)2 + v 2 exp − du dv 16π −∞ −∞ 16 ! ! Z ∞ Z 0 v2 (u − 2)2 1 exp − dv exp − du 16π −∞ 16 16 −∞ ! √ 1 √ 0−2 16π 16π Φ √ 16π 8 1 Q( √ ) 2

Problem 5.6 (a) XX m

PM,N (m, n) =

n

L−1 X L−1−m X

c

m=0 n=0 L−1 X

= c

= c

L−m

m=0 m X

k

k=1

L(L + 1) 2 2 . ⇒ c = L(L + 1) = c

(b) PM (m) =

L−1−m X

c=

2(L − m) , L(L + 1)

0 ≤ m < L.

c=

2(L − n) , L(L + 1)

0 ≤ n < L.

n=0

Similarly, PN (n) =

L−1−n X m=0

(c) If L is an even integer, then L/2−1 L/2−1−m

P r (M + N < L/2) =

X

X

m=0

n=0

82

c=

L+2 . 4(L + 1)

If L is an odd integer, then (L−1)/2 (L−1)/2−m

P r (M + N < L/2) =

X

X

m=0

n=0

c=

L+3 . 4L

Hence, we have P r (M + N < L/2) =

 L+2   4(L+1)   L+3 4L

L even, L odd.

Problem 5.7 1 x2 + 2xy + 4y 2 √ exp − fX,Y (x, y) = 6 2π 3 fX (x) = = = = =

Z ∞

−∞

fX,Y (x, y) dy !

Z ∞

x2 + 2xy + 4y 2 1 √ exp − dy 6 −∞ 2π 3 ! Z ∞ x2 + 2xy + 4y 2 1 √ exp − dy 6 2π 3 −∞ ! Z ∞ 1 (2y + x/2)2 + 3x2 /4 √ exp − dy 6 2π 3 −∞ ! ! Z ∞ (y + x/2)2 x2 1 √ exp −2 exp − dy 3 8 2π 3 −∞ s

1 x2 3π √ exp − = 8 2π 3 2 ! 2 1 x = √ exp − 8 8π fY (y) =

!

Z ∞

−∞

!

fX,Y (x, y) dx !

Z ∞

x2 + 2xy + 4y 2 1 √ = dx exp − 6 −∞ 2π 3 ! (x + y)2 + 3y 2 1 Z∞ √ exp − dx = 6 2π 3 −∞

83

Z

∞ 1 (x + y)2 √ = exp − 6 2π 3 −∞ ! y2 1 √ √ 6π exp − = 2 2π 3 ! 2 1 y = √ exp − 2 2π

!

y2 exp − 2

!

dx

(b) From the results of part (a), the marginal PDFs of X and Y are both Gaussian. Hence, the required means and variances can be found simply by inspecting the PDFs. E[X] V ar(X) E[Y ] V ar(Y )

= = = =

0, 4, 0, 1.

(c) Conditional pdf fX|Y (x|y) fX|Y (x|y) = =

fX,Y (x, y) fY (y) 1√ 2π 3

exp − x

2 +2xy+4y 2

6 2 y 1 √ exp − 2 2π 2

1 x + 2xy + y 2 = √ exp − 6 6π ! 1 (x + y)2 = √ exp − 6 6π

!

(d)E[XY ] E[XY ] = = = =

EY [Y EX [X|Y ]] EY [Y (−Y )] Mean of fX|Y (x|y) as found in (c) EY [−Y 2 ] −V ar(Y ) = −1

84

Problem 5.8 (a) Since x and y are uniformly distributed, over x2 + 4y 2 ≤ 1.

fX,Y (x, y) = c,

The constant c is the inverse of the area of the ellipse. Recall, that the area of an ellipse is given by πr1r2, where r1 and r2 are the radii of the major and minor axes of the ellipse. In this case we get, c=

2 1 = . πr1r2 π

Hence, fX (x) =

Z

Z √1−x2 /2

fX,Y (x, y)dy =

2√ = 1 − x2 π

√

−

1−x2 /2

2 dy π

− 1 ≤ x ≤ 1.

Similarly, fY (y) = =

Z

fX,Y (x, y)dx =

4q π

1 − 4y 2

Z

√ 1−4y 2

√ −

1−4y 2

2 dx π

− 1/2 ≤ y ≤ 1/2.

(b) Since both marginal PDFs are symmetric about the origin, E[X] = E[Y ] = 0. Hence, the variances are equal to the second moments. 2 V ar(X) = E(X ) = π 2

Z 1

√ x2 1 − x2dx = 1/4.

−1

Similarly, V ar(Y ) = E(Y 2) =

4 π

Z 1/2 −1/2

85

q

y 2 1 − 4y 2dy = 1/16.

(c) fX|Y (x|y) = fX,Y (x, y)/fY (y) =

2/π 1 √ √ = 4 1 − 4y 2 2 1 − 4y 2 π

2/π 1 √ = fY |X (y|x) = fX,Y (x, y)/fX (x) = 2 √ 1 − x2 1 − x2 π

|x| ≤

q

1 − 4y 2 .

√ 1 − x2 . |y| ≤ 2

(d) E[XY ] = EY [Y EX [X|Y ]]. From the results of part (c), the conditional PDFs are symmetric about the origin and hence, E[X|Y ] = 0. It follows that E[XY ] = 0, Cov(X, Y ) = 0, 0. ρX,Y =

Problem 5.9

ρX,Y =

(

1 if a > 0 −1 if a < 0

Y

ρX,Y µY σy2

= aX + b Cov(X, Y ) σx σy = E[Y ] = aE[X] + b = aµX + b = E[Y 2 ] − µ2Y = E[(aX + b)2 ] = E[a2X 2 + 2abX + b2 ] − (aµX + b)2 =

= = = ⇒ σy = Cov(X, Y ) = =

a2E[X 2 ] + 2abE[X] + b2 − (a2µ2X + 2abµX + b2 ) a2(σx2 + µ2X ) + 2abµX + b2 − (a2µ2X + 2abµX + b2 ) a2σx2 |a|σx E[(X − µX )(Y − µY )] E[(X − µX )(aX + b − aµX − b)]

86

ρX,Y ρX,Y

= E[a(X − µX )2 ] = aE[(X − µX )2 ] = aσx2 aσx2 = |a|σxσx a = |a| ( 1 if a > 0 = −1 if a < 0

Now we shall prove the converse of the statement that is if ρX,Y = 1 then we will be able to express Y as a linear function of X. That means we will be able to find constants a, b such that Y = aX + b and (a 6= 0.) |ρX,Y | = 1 ⇒ Cov(X, Y ) = ±σX σY = E[XY ] − µX µY If as we mentioned earlier Y = aX + b then E[(Y − aX − b)2] = 0 ⇒ E[Y 2 + a2X 2 + b2 − 2aXY − 2abX − 2bY ] = 0 E[Y 2 ] + a2 E[X 2] + b2 − 2aE[XY ] − 2abE[X] − 2bE[Y ] = 0 2 + µ2X ) + b2 − 2a(±σX σY + µX µY ) − 2abµX − 2bµY = 0 σY2 + µ2Y + a2(σX (σY ± aσX )2 + (µY − aµX − b)2 = 0 σY σX σY b = µ Y ± µX σX ⇒a=±

Thus we can find two constants a, b such that Y is a linear function of X if ρX,Y = ±1 and this proves what we set out to prove.

Problem 5.10 From Theorem 5.4. and its proof, we are aware that (E[XY ])2 ≤ E[X 2]E[Y 2] ,

87

which gives E[XY ] ≤

q

E[X 2]E[Y 2] .

Note that E[X 2 ]E[Y 2 ] ≥ 0. Hence, q

E[X 2 ] + E[Y 2 ] + 2E[XY ] ≤ E[X 2 ] + E[Y 2 ] + 2 E[X 2 ]E[Y 2 ] , which means

q

E[(X + Y )2] ≤ ( E[X 2 ] + Therefore,

q

E[(X + Y )2 ] ≤

q

q

E[X 2 ] +

E[Y 2 ])2 .

q

E[Y 2 ] .

Problem 5.11

µX σX ⇒ E[X 2 ] µY σY ⇒ E[Y 2 ]

= = = = = =

ρX,Y

=

⇒ Cov(X, Y ) ⇒ E[XY ] U V

= = = =

2 1 2 σX + µ2X = 1 + 4 = 5 −1 4 σY2 + µ2Y = 16 + 1 = 17 1 4 ρX,Y σX σY = 1 Cov(X, Y ) + µX µY = 1 − 2 = −1 X + 2Y 2X − Y

(a) E[U ] and E[V ] E[U ] = E[X + 2Y ] = E[X] + 2E[Y ] = 2 − 2 = 0 E[V ] = E[2X − Y ] = 2E[X] − E[Y ] = 4 − (−1) = 5

88

(b)E[U 2 ], E[V 2], V ar(U ) and V ar(V ) E[U 2 ] E[V 2 ] V ar(U ) V ar(V )

= = = =

E[(X + 2Y )2] = E[X 2] + 4E[XY ] + 4E[Y 2 ] = 5 + 4(−1) + 4 · 17 = 69 E[(2X − Y )2] = 4E[X 2 ] + E[Y 2 ] − 4E[XY ] = 4 · 5 − 4(−1) + 17 = 41 E[U 2 ] − µ2u = 69 − 0 = 69 E[V 2 ] − µ2v = 41 − 25 = 16

(c) E[U V ], Cov(U, V ) and ρU,V E[U V ] = E[(X + 2Y )(2X − Y )] = E[2X 2 − 2Y 2 + 3XY ] = 2E[X 2 ] − 2E[Y 2 ] + 3E[XY ] = 2(5) − 2(17) + 3(−1) = −27 Cov(U, V ) = E[U V ] − µU µV = −27 − (0)(3) = −27 Cov(U, V ) 27 = −√ ρU,V = σU σV 67 · 16

Problem 5.12 Since fX (x) is symmetric about the origin, we know µX = 0, and E[XY ] = E[aX 3] = a

Z ∞ −∞

x3 fX (x) = 0 .

Hence, ρX,Y =

E[XY ] − E[X]E[Y ] =0 . σX σY

Problem 5.13

fX,Y (x, y) =

(

+ yb2 ≤ 1 otherwise

0

q

fX (x) =

Z b

2

1− x2

q

−b

2

x2 a2

1 πab

a

2 1− x2 a

1 dy πab

s

2 x2 = 1− 2 πa a Similary we can write s

2 y2 fY (y) = 1− 2 πb b

89

We can see that the product of the marginal pdfs is not equal to the constant 1 which means that the two variables X, Y are not independent. Let us πab calculate the correlation of X, Y . q Z a Z b 1− x2 2 q a

E[XY ] =

−a −b

s

Z a

1 = πa

xy

2 1− x2 a

x 1−

−a

1 dy dx πab

x2 dx a2

The above integral is odd and hence it evaluates to zero. Thus we have found a case where two variables which are uncorrelated and yet are dependent.

Problem 5.14 The marginal PDF of X can be found as follows: fX (x) = =

Z ∞ −∞

fX,Y (x, y)dy

Z ∞ −∞



1

2πσX σY

fX (x) =

Z ∞ −∞

1 − ρ2X,Y X−µX σX

q

u2 = exp − 2(1 − ρ2X,Y )

u2 exp − = q 2 2 2πσX 1

Y −µY σY

!Z

!Z

1

−∞

2πσX 1 − ρ2X,Y

−∞

!

q

q

1

2π(1 − ρ2X,Y )

exp −

#!

" 2 v − 2ρ

dv

uv + ρ2X,Y u2 2(1 − ρ2X,Y ) X,Y

!

The last step is accomplished by noting that the integrand is a properly normalized Gaussian PDF and hence integrates to 1. Following identical

90

 dy

dv

v 2 − 2ρX,Y uv exp − 2(1 − ρ2X,Y )

(x − µX )2 =q exp − 2 2 2σX 2πσX 1

#! "

∞

∞



. The above equation then

u2 − 2ρX,Y uv + v 2 exp − 2(1 − ρ2X,Y )

2πσX 1 − ρ2X,Y

u2 exp − = q 2 2 2πσX

and v =

2(1 − ρ2X,Y )

"

1

1

X 2 X Y Y 2 ( x−µ ) − 2ρX,Y ( x−µ )( y−µ ) + ( y−µ ) σX σX σY σY

exp − 

q

To simplify notation, let u = simplifies to



#!

dv

steps, the marginal PDF of Y is found to be !

(y − µY )2 exp − . fY (y) = q 2σY2 2πσY2 1

Problem 5.15

fX,Y (x, y) =



1

q

2πσxσy 1 −

ρ2X,Y



exp −

x−µx 2 σx

− 2ρX,Y

x−µx σx

y−µy σy

+

y−µy σy

2(1 − ρ2X,Y )

2   

From the results of the previous exercise, the marginal PDF of Y is (y − µy )2 exp − fY (y) = q 2σy2 2πσy2 1

!

The conditional pdf fX|Y (x|y) is then given by fX|Y (x|y) =

fX,Y (x, y) fY (y) 1 √ 2πσx σy

fX|Y (x|y) =



1−ρ2X,Y



exp −

(

x−µx σx

√1 2 2πσy



exp

1  exp − = q 2 2 2πσx (1 − ρX,Y ) 

)

2

−2ρX,Y

(

2 y) − (y−µ 2σy2

x−µx 2 σx

x− 1  exp − = q 2πσx2 (1 − ρ2X,Y )

y−µy x−µx σx σy 2(1−ρ2X,Y )

)

+

y−µy σy

2   

− 2ρX,Y

x−µx σx

2(1 − µx − ρX,Y σσxy (y − 2 2σX (1 − ρ2X,Y )

y−µy σy 2 ρX,Y )

+ ρ2X,Y

2 

µy )  

The above equation is in the form of a Gaussian random variable whose mean and variance are given by σx µX|Y = µx + ρX,Y (y − µy ), σy 2 2 σX|Y = σX (1 − ρ2X,Y ).

91

y−µy σy

2   

Problem 5.16 (a) d d E[(Y − aX)2 ] = E[ (Y − aX)2] = E[2(Y − aX)(−X)] = 0 da da ⇒ E[XY ] = aE[X 2 ] ρσX σY + µX µY E[XY ] = ⇒a = 2 2 E[X ] σX + µ2X (b) E[(Y − aX)2 ] = E[Y 2 ] − 2aE[XY ] + a2E[X 2 ] = = = =

!

!2

E[XY ] E[XY ] E[XY ] + E[X 2 ] E[Y ] − 2 2 2 E[X ] E[X ] 2 E[XY ] E[Y 2 ] − E[X 2] (ρσX σY + µX µY )2 σY2 + µ2Y − 2 σX + µ2X 2 (σY2 + µ2Y )(σX + µ2X ) − (ρσX σY + µX µY )2 . 2 σX + µ2X 2

Problem 5.17 (a) Characteristic function of the jointly distributed Gaussian

fX,Y (x, y) =



1

q

2πσxσy 1 −

ρ2X,Y



exp −

x σx

2

− 2ρX,Y

x σx

y σy

2(1 − ρ2X,Y )

+

ΦX,Y (ω1 , ω2 ) = E[ejω1x+jω2 y ] = =

Z ∞ Z ∞ −∞ Z ∞ −∞

−∞

fX,Y (x, y) exp(j(ω1 x + ω2 y)) dx dy

fY (y)ejω2 y

Z ∞ −∞

fX|Y (x|y)ejω1x dx dy

= EY [ejω2 y ΦX|Y (ω1 )] From Problem 5.15 we know that conditioned on Y , X is Gaussian with a 2 2 = σX (1 − ρ2). Hence the mean of µX|Y = ρyσX /σY and a variance of σX|Y

92



y 2 σy  

conditional characteristic function of X given Y is

1 σX 2 (1 − ρ2 ) + jω1 ρy ΦX|Y (ω1 ) = exp − ω12 σX 2 σY

The joint characteristic function is then given by ΦX,Y (ω1 , ω2 ) =

Z ∞ −∞

fY (y)ejω2 y ΦX|Y (ω1 )dy

Z

∞ 1 σX 2 = exp − ω12 σX (1 − ρ2 ) fY (y) exp j ω2 + ω1 ρ y dy 2 σY −∞ 1 2 2 σX 2 = exp − ω1 σX (1 − ρ ) ΦY ω2 + ω1 ρ 2 σY ! 1 2 2 1 σX 2 2 2 = exp − ω1 σX (1 − ρ ) exp − ω2 + ω1 ρ σY 2 2 σY 1 2 2 1 2 2 = exp − (ω1 σX + 2ρσx σY ω ω2 + ω2 σY ) . 2

(b) First, find the first few terms in the 2-D Taylor series expansion of the joint characteristic function as follows:

1 2 ΦX,Y (ω1 , ω2 ) = exp − ω12 σX 2 1 2 2 = 1 − ω 1 σX + 2 × × = + +

1 exp (−ρω1 ω2 σX σ2) exp − ω22 σY2 2 1 4 4 ω σ + ... 8 1 X 1 2 2 2 2 1 − ρω1 ω2 σX σY + ρω1 ω2 σX σY + . . . 2 1 2 2 1 4 4 1 − ω 2 σY + ω 2 σY + . . . 2 8 1 2 2 1 2 2 1 − ω1 σX − ω2 σY − ρω1 ω2 σX σ2 2 2 1 4 4 1 4 4 1 1 2 2 2 2 2 ω 1 σX + ω 2 σY + + ρ σX σY ω 1 ω 2 8 8 4 2 higher order terms

The correlation is found from the characteristic function as

∂ 2ΦX,Y (ω1 , ω2 ) . E[XY ] = − ∂ω1∂ω2 ω1 =ω2 =0

93

(2)

The required mixed partial derivative is the coefficient of ω1 ω2 in the series expansion given in (2) above. Hence, E[XY ] = ρσ1 σ2. (c) Proceeding as in part (b),

∂ 4ΦX,Y (ω1 , ω2 ) . E[X 2 Y 2 ] = ∂ω12∂ω22 ω1 =ω2 =0 This fourth order mixed partial derivative can also be found from the series expansion given in (2) above. That is

∂ 4ΦX,Y (ω1 , ω2 ) = 4 × (coefficient of ω12 ω22 ). 2 2 ∂ω1 ∂ω2 ω1 =ω2 =0 Hence, 2 2 σY E[X 2 Y 2 ] = (1 + 2ρ2 )σX

Problem 5.18 First we note that (see solution to Problem 5.17 for details) h

i

ΦX,Y (ω1 , ω2 ) = EY ejω2 Y EX [ejω1X |Y ] .

(3)

Conditioned on Y, X is a Gaussian random variable with a mean of 2 (1 − ρ2 ). Hence µX + ρ(σX /σY )(Y − µY ) and a variance of σX

EX [ejω1 X |Y ] = exp jω1 µX + ρ

σX 1 2 (Y − µY ) − ω12 σX (1 − ρ)2 ) σY 2

(4)

By (3), (4), we have

σX 1 2 (Y − µY ) ω1 − ω12 σX (1 − ρ2 ) σY 2 ! ω12 2 σX 2 = exp jω1 µX − jρµY − σX (1 − ρ ) σY 2 σX × EY exp j(ω2 + ρ ω1 )Y σY ! ω12 2 σX σX 2 = exp jω1 µX − jρµY − σX (1 − ρ ) ΦY (ω2 + ρ ω1 ) σY 2 σY

ΦX,Y (ω1 , ω2 ) = EY exp jω2 Y + j µX + ρ

94

Since

ΦY ω2 + ρ



σX  ω1 = exp jµY σY

ω2 + ρ

σX ω1 − σY

ω2 + ρ σσXY ω1



2

2



σY2  , (5)

the joint characteristic function is then (after some algebraic manipulations) ω 2 σ 2 + 2ρσX σY ω1 ω2 + ω22 σY2 ΦX,Y (ω1 , ω2 ) = exp j(ω1 µX + µY ω2 ) − 1 X 2

!

.

Problem 5.19 fR (r)dr = Pr(r ≤ R < r + dr) ! Z 2π Z r+dr r2 r exp − 2 drdθ = 2πσ 2 2σ 0 r ! Z r+dr r2 r exp − dr = σ2 2σ 2 r r2 = − exp − 2 2σ 2

!

! r+dr r

!

r (r + dr)2 = exp − 2 − exp − 2σ 2σ 2 !" !# r2 rdr dr2 = exp − 2 1 − exp − 2 − 2 . 2σ 2σ 2σ As dr → 0 the terms involving dr2 will be small compared to those involving dr. Hence, fR(r)dr → = → ⇒ fR (r) =

!"

!#

r2 rdr exp − 2 1 − exp − 2 2σ σ !" !# r2 rdr r2 dr2 exp − 2 1 − 1 − 2 + + ... 2σ σ 2σ 4 ! r r2 exp − 2 dr σ2 2σ ! r r2 exp − 2 σ2 2σ

95

Problem 5.20

fX (x) fY (y) Z FZ (z)

= = = = = = =

be−bx be−by X −Y P r(Z ≤ z) = P r(X − Y ≤ z) P r(X ≤ Y + z) Z Z Z Z

X≤Y +z

fX (x)fY (y)dx dy b2e−b(x+y) dx dy

X≤Y +z

Here we need to consider two cases(Z > 0 and Z < 0) because the limits for the area defined by Z will be different for these cases (i) Z < 0 2

FZ (z) = b

Z ∞ Z y+z −z

2

= b

Z ∞" −z

= b

Z ∞

e−b(x+y) dx dy

0

e−bx − −b

# x=y+z e−by dy x=0

(1 − e−(by+bz) )e−by dy

−z

"

e−by e−(2by+bz) + = b − b 2b bz

ebz −

= =

e 2

!

# y=∞

y=−z

ebz 2

(ii) Z > 0 2

FZ (z) = b

= b2

Z ∞ Z y+z 0

0

Z ∞" 0

e−b(x+y) dx dy

e−bx − −b

96

# x=y+z e−by dy x=0

= b

Z ∞

(1 − e−(by+bz) )e−by dy

0

"

e−by e−(2by+bz) + = b − b 2b −bz

=

e

1−

!

# y=∞ y=0

2

In summary the CDF of Z is given by FZ (z) =

(

ebz 2

1−

e−bz 2

Z<0 Z ≥0

And the corresponding pdf is given by fZ (z) =

(

bebz 2 be−bz 2

Z <0 Z ≥0

Problem 5.21 This was proven in the text. In Example 5.25, a 2x2 linear transformation of two jointly Gaussain random variables X and Y produce two new jointly Gaussian random variables Z and W . Since Z and W are jointly Gaussian, Z and W are individually Gaussian as well. Hence Z is a Gaussian random variable.

Problem 5.22 X ∼ N (1, 4) Y ∼ N (−2, 9) Z = 2X − 3Y − 5 1 ρX,Y = 3 3×2 =2 ⇒ Cov(X, Y ) = 3 We will make use of the fact that a linear transformation of the Gaussian Random variables results in a Gaussian Random variable. E[Z] = E[2X − 3Y − 5] = 2(1) − 3(−2) − 5 = 3

97

V ar(Z) = = = = =

E[(Z − 3)2 ] = E[(2X − 3Y − 8)2 ] E[(2X − 2 − 3Y − 6)2 ] E[4(X − 1)2 + 9(Y + 2)2 − 12(X − 1)(Y + 2)] 4V ar(X) + 9V ar(Y ) − 12Cov(X, Y ) 4(4) + 9(9) − 12(2) = 73

Hence Z is a Gaussian distributed as follows Z ∼ N (3, 73)

Problem 5.23 (a) We first find the CDF of Z and then differentiate to get the PDF. Pr(Z ≤ z) = Pr(max(X, Y ) ≤ z) = Pr(X ≤ z, Y ≤ z) = FX (z)FY (z) !!2

z2 1 − exp − . = 2 ! !! z2 z2 1 − exp − . fZ (z) = 2z exp − 2 2 (b) Proceeding in a simalar manner as in part (a),

Pr(W ≥ z) = Pr(min(X, Y ) ≥ z) = Pr(X ≥ z, Y ≥ z) = (1 − FX (z))(1 − FY (z)) = = fZ (z) = =

!!2

z2 exp − 2 exp(−z 2). d (1 − Pr(W ≥ z) dz 2z exp(−z 2 ).

98

Problem 5.24 fX (x) = e−x U (x) fY (y) = e−y U (y) For the equation z 2 + Xz + Y = 0 to have real roots the following condition must hold X2 X − 4Y ≥ 0 ⇒ Y ≤ 4 X2 ) P r(real roots) = P r(Y ≤ 4 2

=

Z ∞ Z x2 /4 0

=

Z ∞

e−x e−y dy dx

0

(1 − e−x

2 /4

0

=

Z ∞ 0

= 1−

−x

e

dx −

Z ∞

)e−x dx

Z ∞

e−x−x

2 /4

dx

0

e−x−x

2 /4

dx

0

Z ∞

(x + 2)2 − 4 ) dx 4 0 √ Z∞ 1 (x + 2)2 √ = 1 − e 4π ) dx exp(− 4 0 4π √ 0+2 = 1 − e 4πQ( √ ) 2 √ √ = 1 − e 4πQ( 2) ≈ 0.77 = 1−

exp(−

(b) For the equation to have imaginary roots the condition is X 2 − 4Y > 0 ⇒ Y <

X2 4

X2 ) = 1 − P r(real roots) 4√ √ √ √ = 1 − (1 − e 4πQ( 2)) = 1 − e 4πQ( 2) ≈ 0.23

P r(imaginary roots) = P r(Y >

(c) For the equation to have equal roots the condition is X 2 − 4Y = 0

99

This is the equation of a parabola in the (x,y) plane. The probability of the point (X,Y) falling on a line or a curve is zero because its area is zero.

Problem 5.25 Conditioned on X=x, the transformation Z=xY is a simple linear transformation and from fX (x) we know that x > 0, fZ|x (z|x) =

1 1 z 1 q fY ( ) = |x| x xπ 1−

z2 x2

, |z| < x

The PDF of Z can be found according to Z

fZ (z) = fZ|x (z|x)fX (x)dx =

 x2 R   ∞ 1 2 e− 2σ2 √ x dx z πσ x2 −z2 R ∞ 1 − x2  x  2σ 2 √ dx 2e −z πσ

x2 −z2

z>0 z<0

And, Z ∞ z

1 − x22 x e 2σ √ 2 dx = 2 πσ x − z2

Z ∞

1 − x22 x e 2σ √ 2 dx 2 −z πσ x − z2 Z ∞ 1 − u2 1 = e 2σ √ du 2 2 2πσ z u − z2 Z 1 − z22 ∞ − t 2 1 2σ e e 2σ √ dt = 2πσ 2 0 t z2 1 e− 2σ2 = √ 2πσ 2

Hence, Z is a Gaussian random variable with a zero mean and unit variance.

Problem 5.26 1 2π 1 fY (y) = 2π Z = (X + Y ) mod 2π P r(Z ≤ z) = P r(X + Y < z) + P r(2π < X + Y < 2π + Z) fX (x) =

100

Note that the above two events are mutually exclusive because Z < 2π, hence the first event includes the the condition X + Y < 2π Z zZ x Z z Z 2π Z 2π Z 2π+z−x 1 1 1 1 1 1 dy dx + dy dx + dy dx P r(Z ≤ z) = 2π 2π 0 0 2π 2π 0 2π−x 2π 2π z 2π−x Z z Z z Z 2π 1 2 = ( ) x dx + x dx + z dx 2π 0 0 !z 1 2 z2 z2 z = ( ) + + 2πz − z 2 = 2π 2 2 2π 1 ⇒ fZ (z) = 2π

Problem 5.27 Conditioned on Y=y, the transformation Z=Xy is a simple linear transformation and 1 z z fX ( ) = fX ( ) . fZ|Y (z|y) = |y| y y The PDF of Z can be found according to fZ (z) =

Z

fZ|Y (z|y)fY (y)dy = pfX (z) + (1 − p)fX (−z) .

Z is a Gaussian random variable if fX (x) is symmetric about the origin, i.e. µX = 0. Also, if p = 0 or p = 1, Z will be Gaussian.

Problem 5.28 Since the transformation is linear and X and Y are jointly Gaussian, U and V will be jointly Gaussian with E[U ] = E[X] cos(θ) − E[Y ] sin(θ) = 0 E[V ] = E[X] sin(θ) + E[Y ] cos(θ) = 0 V ar(U ) = E[U 2 ] = E[X 2 ] cos2(θ) + E[Y 2 ] sin2(θ) − 2E[XY ] cos(θ) sin(θ) = cos2 (θ) + sin2 (θ) = 1 V ar(V ) = E[V 2 ] = E[X 2 ] sin2 (θ) + E[Y 2 ] cos2(θ) + 2E[XY ] cos(θ) sin(θ) = cos2 (θ) + sin2 (θ) = 1 Cov(U, V ) = E[U V ] = E[X 2 ] cos(θ) sin(θ) − E[Y 2 ] cos(θ) sin(θ) + E[XY ](cos2 (θ) − sin2(θ)) = cos(θ) sin(θ) − cos(θ) sin(θ) = 0

101

Hence, U and V are independent standard Normal random variables.

Problem 5.29 Note that any linear transformation of jointly Gaussian random variables produces jointly Gaussian random variables. So U, V are both Gaussian random variables. And if ρU,V = 0, U and V are independent. E[U ] = aE[X] + bE[Y ] = 0 . E[V ] = cE[X] + dE[Y ] = 0 . E[U V ] = acE[X 2] + bdE[Y 2 ] + (ad + bc)E[XY ] = ac + bd + ρ(ad + bc) . Thus, ρU,V = 0 if ac + bd + ρ(ad + bc) = 0 . So, U and V are independent, if ac + bd = −ρ(ad + bc) .

Problem 5.30

fX,Y (x, y) = R = Θ =

J "

x y r θ

X = Y! =

!#

=

1 (x − µx )2 + (y − µy ) exp − 2πσ 2 2σ 2 √ X2 + Y 2 Y tan−1 ( ) X R cos θ R sin θ ! cos θ sin θ −r sin θ r cos θ

!

x y det J = r r θ fR,Θ (r, θ) = fX,Y (x, y)r ! (x − µx )2 + (y − µy ) r exp − = 2πσ 2 2σ 2

102

!

r (r cos θ − µx )2 + (r sin θ − µy ) = exp − 2πσ 2 2σ 2 ! (r2 + µ2x + µ2y − 2rµx cos θ − 2rµy sin θ) r = exp − 2πσ 2 2σ 2 ! ! (r2 + µ2x + µ2y ) r rµx cos θ + rµy sin θ = exp − exp 2πσ 2 2σ 2 σ2 ! ! r (r2 + a2) ra cos(θ + φ) = exp − exp 2 2 2πσ 2σ σ2 where a =

q

µ2x + µ2y and tan φ =

fR (r) =

Z 2π 0

µy . µx

fR,Θ (r, θ) dθ !

Z 2π

!

(r2 + a2 ) r ra cos(θ + φ) = exp − exp dθ 2 2 2πσ 2σ σ2 0 !Z ! 2π r (r2 + a2 ) ra cos(θ + φ) = exp − exp dθ 2πσ 2 2σ 2 σ2 0 ! (r2 + a2 ) ar r exp − I ( ) = o 2πσ 2 2σ 2 σ2 This is the general form of a Rician distribution and therefore fR (r) follows a rician distribution. Please note in the above integral we have made use of the following property of periodic functions. Z 2π

ecos(θ+φ) dθ =

0

Z 2π

ecos(θ) dθ

0

This follows directly because of the periodicity of the cos function.

Problem 5.31 The inverse transformation is given by s

X=±

Z +W 2

103

(6)

s

Y =±

J=

    



Z −W . 2

z w    = det x y 

"

2x 2y 2x −2y

(7) #

= −8xy .

Plugging these results into the general formula results in ( z+w ) ( z−w ) 1 2 2 exp − + 2 2πσX σY 2σX 2σY2

1

!!

q fZ,W (z, w) = 4 ∗ q z−w 8 z+w 2 2 1 z+w z−w 1 = √ 2 exp − ( + ) , z > w, z > 0 . 2 4σX 4σY2 z − w2 2πσX σY

Problem 5.32

J " det J

Z = X2 + Y 2 W = XY √ √ Z + 2W + Z − 2W X = 2√ √ Z + 2W − Z − 2W Y = 2 1 fX (x) = π(1 + x2) 1 fY (y) = π(1 + y 2) ! ! z w 2x 2y = x y y x z w x y

!#

= 2|x2 − y 2 | fX (x)fY (y) 2|x2 − y 2| 1 1 = 2 2 2π (1 + x )(1 + y 2 )|x2 − y 2| 1 1 = 2π 2 (1 + x2 + y 2 + x2y 2)|x2 − y 2 |

fZ,W (z, w) =

104

=

1 1 √ 2 2 2π (1 + z + w2 ) z 2 − 4w2

Problem 5.33 (a) E[Z] =

ZZ

Z

Z 2π

Aejθ fA,θ (A, θ)dθdA = AfA (A)

ejθ

0

1 dθdA = 0 . 2π

(b) The variance is defined as i 1 h 1 1 V ar(Z) = E |Z − µZ |2 = E[|Z|2 ] = E[A2] . 2 2 2

Problem 5.34 2 X 2 X

qi,j I(X; Y ) = pj qi,j log pj i=0 j=0

!

1 [3 · 0.8 · log(2.4) + 6 · 0.1 · log(0.3)] 3 = 0.8 · log(2.4) + 0.2 · log(0.3) = 0.663 bits

=

Problem 5.35 2 X 2 X

qi,j I(X; Y ) = pj qi,j log pj i=0 j=0

!

1 [3 · 0.9 · log(2.7) + 3 · 0.1 · log(0.3)] 3 = 0.9 · log(2.7) + 0.1 · log(0.3) = 1.116 bits

=

105

Problem 5.36 2 2 X X

qi,j I(X; Y ) = pj qi,j log pj i=0 j=0 1 1 1/3 = 9 · · log 3 3 1/3 = 0 bits

!

!

The output is independent of (tells us nothing about) the input, hence the information carried by the channel is zero.

106


Problem 6.1 (a) (0, 0, 0), (0, 0, 1), (0, 1, 0), (0, 1, 1), (1, 0, 0), (1, 0, 1), (1, 1, 0), (1, 1, 1) (b) Each outcome is mutually exclusive (and therefore not independent).

Problem 6.2 (a) fX (x) = Z

(

c kxk ≤ 1 0 kxk > 1

fX (x) dx = 1 = c

Z Z Z

dx1 dx2 dx3

4π (This integral is the volume of a unit sphere) 3 3 ⇒c = 4π = c

(b) Marginal pdf fX1 ,X2 (x1, x2) fX1 ,X2 (x1, x2 ) =

Z

fX1 ,X2 ,X3 (x1 , x2, x3) dx3 √ Z 1−x2 −x2 1 2 = √ 2 2 c dx1 −

1−x1 −x2

q 3 = 2 1 − x21 − x22 4π 3 q = 1 − x21 − x22 2π

(c) Marginal pdf fX1 (x1 ) fX1 (x1) =

Z

fX1 ,X2 (x1, x2) dx2

107

Z

=

√

3 q 1 − x21 − x22 dx2 2π

1−x21

√ −

1−x21

3 (1 − x21) 4

= (d) Conditional pdfs

fX1 ,X2 ,X3 (x1 , x2, x3) fX2 ,X3 (x2, x3 )

fX1 |X2 ,X3 (x1|x2 , x3) = =

3 2π

q

=

3 4π

q

1 − x22 − x23 1

2 1 − x22 − x23 fX1 ,X2 ,X3 (x1 , x2, x3) fX3 (x3)

fX1 ,X2 |X3 (x1, x2 |x3) =

3 4π

=

3 (1 4

q

=

− x23) 1

π 1 − x23

Problem 6.3 (a) Define In (z) =

Z Z

Z

. . . Pn

i=1

xi ≤z

dx1 dx2 . . . dxn .

Then c = IN−1(1). Note that In (z) = = =

Z z "Z Z 0

Z z Z0z 0

#

Z

. . . Pn−1 i=1

xi ≤z−xn

In−1 (z − xn )dxn In−1 (u)du.

Using this iteration, we see that I1 (z) =

Z z

du = z,

0

108

dx1 dx2 . . . dxn−1 dxn

Z z

1 udu = z 2 , 2 0 Z z 1 1 2 u du = z 3, I3 (z) = 3! 0 2 .. . 1 n z . In (z) = n! I2 (z) =

Then, c = IN−1 (1) = N !. (b) fX1 ,X2 ,...,XM (x1 , x2, ..., xM ) =

Z Z

...

Z xM +1 +...+xN ≤1−x1 −x2 −...−xM

c dxN dxN −1 . . . dxM +1

= c IN −M (1 − x1 − x2 − . . . − xM ) N! (1 − x1 − ... − xM )N −M , = (N − M)!

M X

xM ≤ 1, xi ≥ 0

m=1

(c) From part(b), we have fXi (xi ) = N (1 − xi )N −1 . Then, fX1 (x1 )fX1 (x1 )...fXN (xN ) = N N (1−x1)N −1 ...(1−xN )N −1 6= fX1 ,X2 ,...,XN (x1, x2 , ..., xN ) = N ! Thus, Xi are identically distributed, but they are not independent.

Problem 6.4 X = [X1 , X2 , . . . , XN ]T 1 = [1, 1, . . . , 1]T n × 1 column vector Z =

N X

bi Xi = bT X

i=1 N X

bi = 1 = bT 1

i=1

We know that V ar(Z) = V ar(bT X) = bT CXX b

109

To minimize σZ2 we use lagrange multiplier ∇(bT CXXb) + λ∇(bT 1 − 1) ∇(bT )CXXb + bT CXX ∇(b) + λ∇(bT )1 2∇(bT )CXX b + λ∇(bT )1 ∇(bT ) ⇒ λ1 λC−1 XX 1 T −1 λ1 CXX1

= = = = = = =

0 0 0 In×n −2CXX b −2b −21T b = −2 2 λ = − T −1 1 CXX1 Using this value of λ in (1) we get 2 1 −2b = − T −1 C−1 1 CXX1 XX C−1 1 b = T XX −1 1 CXX 1

(1)

(2)

This solution must then be checked against the costraint that each bi must be non-negative. If any of the bi specified by (2) are negative, then those bi should be set equal to zero and the problem reworked with the lower dimensionality.

Problem 6.5 A matrix R is non-negative definite if zT Rz ≥ 0 for any vector z. If R is a correlation matrix, then zT Rz = zT E[XXT ]z = E[zT XXT z] 2

= E[(zT X) ] ≥ 0.

Problem 6.6 (a) The 36 possible realizations of the random vector (X, Y ) are: (1, 1) (1, 2) (1, 3) (1, 4) (1, 5) (1, 6)

110

(2, 1) (2, 2) (2, 3) (2, 4) (2, 5) (2, 6) (3, 1) (3, 2) (3, 3) (3, 4) (3, 5) (3, 6) (4, 1) (4, 2) (4, 3) (4, 4) (4, 5) (4, 6) (5, 1) (5, 2) (5, 3) (5, 4) (5, 5) (5, 6) (6, 1) (6, 2) (6, 3) (6, 4) (6, 5) (6, 6) (b) Each realization has a probability of 1/36. (c) E[(X, Y )] = (3.5, 3.5). (d) 35 0 C = 12 35 0 12

Problem 6.7 E[(X, Y, Z)] = (3.5, 3.5, 3.5), C =

 35  12  0

0

0 35 12

0



0  0

35 12

Problem 6.8

5 1

1 = QΛQT 2 " √ # 7+ 13 0√ 5.3028 2 Λ = = 7− 13 0 0 2 0.9571 −0.2898 Q = 0.2898 0.9571 C =

0 1.6972

The appropriate transformation is given by √ 2.2040 T=Q Λ= 0.6673

111

−0.3775 1.2469

Problem 6.9 The covariance matrix of Y will be of the form, CY Y = ACXX AT , Note that CXX can be decomposed into CXX = QΛQT , where Λ is a diagonal matrix of the eigenvalues of CXX and Q is an orthogonal matrix whose columns are the corresponding eigenvectors of C. So if we let CY Y be Λ, then Y will be uncorrelated, where A should be QT . 



3−λ 1 −1   CXX − λI =  1 5 − λ −1  . −1 −1 3 − λ ⇒ det(CXX − λI) = −(λ − 2)(λ − 3)(λ − 6) = 0 So, λ1 = 2, λ2 = 3, λ3 = 6 . And, the eigenvector matrix is calculated to be  √ 

√

2 2

Q=  √0

2 2



√

3 3√

6 √6 6 3√

− √33 − 33 −

  

6 6

.

Hence, the appropriate transformation matrix is  √ 

A = QT =  

√

2 √2 3 √3 6 6

The covariance matrix of Y is 

CYY



2 0√ 2√  −√ 33 − √33   . 6 − 66 3



2 0 0  =Λ= 0 3 0   . 0 0 6

112

Problem 6.10 (a) E[X1 |X2 = x2, X3 = x3] XT = [X1 X2 X3 ]   1 ρ ρ   CXX = σ 2  ρ 1 ρ  ρ ρ 1

(3)

det[CXX] = σ 6 (1 − ρ)2 (1 + 2ρ)   1 + ρ −ρ −ρ −2 σ  1 + ρ −ρ  C−1  −ρ  XX = (1 − ρ)(1 + 2ρ) −ρ −ρ 1 + ρ

(4)

We know that fX1 |X2 X3 (x1|x2, x3 ) is also a Gaussian random variable and the mean of a Gaussian Random Variable with pdf d exp(−ax2 − bx − c) is given b . So all we need to do is get the coefficient of x21 and x1 in the pdf of by − 2a fX1 |X2 X3 (x1|x2, x3 ) fX1 |X2 X3 (x1|x2, x3 ) =

fX1 ,X2 ,X3 (x1 , x2, x3) fX2 ,X3 (x2, x3 )

1 = q (2π)3 det[CXX]

(5)

exp − 12 XT C−1 XX X fX2 ,X3 (x2, x3 )

(6)

We know that fX2 ,X3 (x2, x3) is a function comprising only of x2, x3 and so it will not affect the coefficient of x21 or x1 in the above equation. Let us calculate the coefficient of x21 ( say a) and x1 (say b) in the above equation. Using 4 and the above eqn we get σ −2 (1 + ρ) 2(1 − ρ)(1 + 2ρ) σ −2 2ρ(x2 + x3) b = 2(1 − ρ)(1 + 2ρ)

a =

Mean of the distribution given by eqn 6 ρ b = (x2 + x3) = E[X1|X2 = x2, X3 = x3 ] 2a 1+ρ

113

(b) E[X1 X2 |X3 ] E[X1 X2 |X3 ] = E [(X2 E[X1 |X2 , X3 ])|X3] ρ (X2 + X3 )|X3 ] E[X1 X2 |X3 ] = E[X2 1+ρ ρ = E[(X22 + X2 X3 )|X3 ] 1+ρ We know from problem 5.15 that for a pair of jointly distributed Gaussian Random Variables X,Y the conditonal pdf fX|Y (x|y) is given by 2 (1 − ρ2XY )) N ∼ (µX + ρXY ( σσXY )(Y − µY ), σX fX2 |X3 (x2|x3) = N ∼ (ρX3 , σ 2(1 − ρ2)) ρ E[X22|X3 ] + E[X2 X3 |X3 ] ⇒ E[X1X2 |X3 ] = 1+ρ ρ 2 σ (1 − ρ2 ) + (ρx3 )2 + x3ρx3 = 1+ρ ρ 2 = σ (1 − ρ2 ) + ρx23 (1 + ρ) 1+ρ = ρσ 2 (1 − ρ) + (ρx3 )2 (c) E[X1 X2 X3 ] E[X1 X2 X3 ] = E [X3 E[(X1 X2 |X3 )]] = E[X3 (ρσ 2(1 − ρ) + (ρX3 )2 )] = 0 We notice the expectation is a linear combination of the odd moments of a Gaussian Random Variable and each of them is zero and consequently the above expectation evaluates to zero

Problem 6.11 TX

ΦX (Ω) = E[ejΩ

] = E[ejZ ] = ΦZ (1) .

where Z = ΩT X, And, ΦZ (ω) = exp(−

ω 2 ΩT CXX Ω ω 2σZ2 ) = exp(− ). 2 2

114

Hence, ΦX (Ω) = ΦZ (1) = exp(−

ΩT CXX Ω ). 2

Problem 6.12 X = [X1, X2 , . . . , XN ]T Ω = [ω1, ω2 , . . . , ωN ]T From problem 6.6 we know ! ΩT CxxΩ ΦX (Ω) = exp − 2 

1 ΩT CxxΩ + ⇒ E[ej(ω1 x1 +ω2 x2 +ω3 x3 +ω4 x4 ) ] = 1 − 2 2!

ΩT CxxΩ 2

!2



· · ·

Expanding both the exponentials and equating the coeffts of ω1 ω2 ω3 ω4 we get E [· · · + X1 X2 X3 X4 ω1 ω2 ω3 ω4 + · · ·] =

1 1 ··· + (C12C34 + C13 C24 + C14C23) 8ω1 ω2 ω3 ω4 + · · · 2! 22

Where Cij = Cov(Xi , Xj ) 1 1 (C12C34 + C13C24 + C14C23 ) 8 2! 22 ⇒ E[X1 X2 X3 X4 ] = (C12C34 + C13 C24 + C14C23) ⇒ E[X1 X2 X3 X4 ] =

Because Xi are zero mean Random Variables Cij = E[Xi Xj ] E[X1 X2 X3 X4 ] = E[X1 X2 ]E[X3X4 ] + E[X1 X3 ]E[X2X4 ] + E[X1 X4 ]E[X2X3 ]

Problem 6.13 (a) The PDF of the median sequence is fYk (y) =

(2k − 1)! fX (y)[FX (y)]k−1[1 − FX (y)]k−1 . [(k − 1)!]2

Then, we have "

y (2k − 1)! 1 y exp(− ) 1 − exp(− ) fYk (y) = 2 [(k − 1)!] µ µ µ

115

#k−1 "

y exp(− ) µ

#k−1

U (y)

(7)

(b) µ1 Y

MY (µ1 ) = E[e

] =

Z ∞

µ1 y

e

0

"

yk y (2k − 1)! 1 exp(− ) ) 1 − exp(− [(k − 1)!]2 µ µ µ

(2k − 1)! 1 = [(k − 1)!]2 µ

dy

!

Z ∞ k−1 X 0

#k−1

k m k−1 (−1)m ey(µ1 − µ − µ ) dy m

m=0

!

k−1 X

(2k − 1)! 1 k−1 1 m = (−1) k m [(k − 1)!]2 µ m=0 m + µ − µ1 µ

(8)

!

X k−1 d (2k − 1)! k−1 µ E[Y ] = MY (µ1 )|µ1 =0 = (−1)m 2 dµ1 [(k − 1)!] m=0 (k + m)2 m

for k 6= 1 , E[Y ] 6= µ. Hence, the median is a biased estimate of the mean of the underlying exponential distribution. (c) From part(b), we have !

X k−1 d2 (2k − 1)! k−1 2µ2 m (−1) . E[Y ] = 2 MY (µ1 )|µ1 =0 = dµ1 [(k − 1)!]2 m=0 (k + m)3 m 2

Then, V ar(Y ) = E[Y 2 ] − E[Y ]2

!

X k−1 (2k − 1)! k−1 2µ2 m = (−1) [(k − 1)!]2 m=0 (k + m)3 m

−

!

X k−1 (2k − 1)! k−1 µ m (−1) [(k − 1)!]2 m=0 (k + m)2 m

!2

.

Problem 6.14 !

N d X d N (FX (y))k (1 − FX (y))N −k FYm (y) = dy dy k=m k

=

N X

−

k=m N X k=m

!

N kfX (y)(FX (y))k−1 (1 − FX (y))N −k k !

N (N − k)fX (y)(FX (y))k (1 − FX (y))N −k−1 k

116

N X

N! fX (y)(FX (y))k−1 (1 − FX (y))N −k (k − 1)!(N − k)! k=m

=

N X

N! fX (y)(FX (y))k (1 − FX (y))N −k−1 k!(N − k − 1)! k=m

−

In the second sum, let n = k + 1. Then fYm (y) = −

N X

N! fX (y)(FX (y))k−1 (1 − FX (y))N −k (k − 1)!(N − k)!

k=m N −1 X

n=m+1

N! fX (y)(FX (y))n−1 (1 − FX (y))N −n (m − 1)!(N − m)!

The second series cancels the first except the k = m term. Hence, fYm (y) =

N! fX (y)(FX (y))m−1 (1 − FX (y))N −m (m − 1)!(N − m)!

Problem 6.15 fym (y)dy = Pr(y < Ym < y + dy) Define the following events: A = {m − 1 of the X’s are < y} B = {one of the X’s ∈ (y, y + dy)} C = {N − m of X’s are > y + dy} Then, Pr(y < Ym < y + dy) = Pr(A, B, C) = Pr(A) Pr(B|A) Pr(C|A, B) These various probabilities work out as follows: !

N m−1 (FX (y)) Pr(A) = m−1 Pr(B|A) = (N − m + 1)fX (y)dy Pr(C|A, B) = (1 − FX (y + dy))N −m

117

Note that as dy → 0, FX (y + dy) → FX (y) for any continuous random variable. Hence, !

N m−1 N −m Pr(y < Ym < y + dy) = (FX (y)) (N − m + 1)fX (y)dy(1 − FX (y + dy)) m−1 N! ⇒ fYm (y) = fX (y)(FX (y))m−1 (1 − FX (y + dy))N −m (m − 1)!(N − m)!

Problem 6.16 (a) αk −α 1011 −10 e = e = 0.1137. k! 11! (b) Let Xi be the number of cars that approach the ith booth in one minute. Pr(X = k) =

Pr(X1 ≤ 5, X2 ≤ 5, . . . , XN ≤ 5) = Pr(X1 ≤ 5) Pr(X2 ≤ 5) . . . Pr(XN ≤ 5) = Pr(Xi ≤ 5)N . Pr(Xi ≤ 5) =

5 X

αm i −αi e m! m=0

Since the traffic splits evenly between all booths, we take αi = α/N . The following table illustrates the relevant calculations. N 1 2 3 4 5 6 7 8

αi 30 15 10 7.5 6 5 4.2857 3.75

Pr(Xi ≤ 5) 2.2573 × 10−8 0.0028 0.0671 0.2414 0.4457 0.6160 0.7390 0.8229

Pr(Xi ≤ 5)N 2.2573 × 10−8 7.7977 × 10−6 3.0192 × 10−4 0.0034 0.0176 0.0546 0.1204 0.2102

From these numbers we conclude that the manager must keep 6 booths open.

Problem 6.17 (a)

1 1 Xn + Xn−1 E[Yn ] = E = E[Xn ] + E[Xn−1 ] = 0. 2 2 2

118

E[Yn Yk ] = E

Xn + Xn−1 2

Xk + Xk−1 2

1 (E[Xn Xk ] + E[Xn Xk−1 ] + E[Xn−1 Xk ] + E[Xn−1 Xk−1 ]) 4 1 [cn,k + cn,k−1 + cn−1,k + cn−1,k−1 ] = 4 =

Using ci,j = δi,j we get E[Yn Yk ] =

(1

n = k, n = k ± 1, otherwise.

2 1 4

0

The correlation matrix is then of the form 

2  1 1 0 R=  4 0  .. .

1 2 1 0 .. .

0 1 2 1 .. .

0 0 1 2 .. .

0 0 0 1 .. .



... ...   ... . ...  .. .

2 (b) In this case, ci,j = σX δi,j so that



2  1 2   σX  0 R= 4  0  .. .

1 2 1 0 .. .

0 1 2 1 .. .

0 0 1 2 .. .

0 0 0 1 .. .



... ...   ... . ...  .. .

Problem 6.18 (a)

1 1 Xn − Xn−1 = E[Xn ] − E[Xn−1 ] = 0. 2 2 2 Xn − Xn−1 Xk − Xk−1 E[Yn Yk ] = E 2 2 1 (E[Xn Xk ] − E[Xn Xk−1 ] − E[Xn−1 Xk ] + E[Xn−1 Xk−1 ]) = 4 1 [cn,k − cn,k−1 − cn−1,k + cn−1,k−1 ] = 4 E[Yn ] = E

119

Using ci,j = δi,j we get ( 1

n = k, n = k ± 1, otherwise.

2

E[Yn Yk ] =

− 14 0

The correlation matrix is then of the form 

2 −1 0 0 0  2 −1 0 0  −1 1  0 −1 2 −1 0 R=  4 0 0 −1 2 −1  .. .. .. .. .. . . . . .



... ...   ... . ...  .. .

2 δi,j so that (b) In this case, ci,j = σX



2 −1 0 0 0  −1 2 −1 0 0  σ2  0 −1 2 −1 0 R= X  4   0 0 −1 2 −1  .. .. .. .. .. . . . . .



... ...   ... . ...  .. .

Problem 6.19 (a)

1 c c 1

a1 c a c ˆ n = cXn−1 . = 2 ⇒ 1 = ⇒X a2 a2 c 0

(b) ˆ n )2 ] = E[(Xn − X = = =

E[(Xn − cXn−1 )2] 2 ] − 2cE[Xn Xn−1 ] E[Xn2 ] + c2E[Xn−1 2 2 1 + c − 2c 1 − c2.

120


Problem 7.1 (a)

"

#

n n n 1X 1X 1X E[ˆ µ] = E Xi = E[Xi ] = 5 = 5. n i=1 n i=1 n i=1

(b) V ar(ˆ µ) = E[(ˆ µ − 5)2 ] 

= E 

= E = =

1 n2 1 n2

1 n

n X

1 n

n X

i=1

i=1

n n X X

!2  Xi − 5 

!2  (Xi − 5) 

E[(Xi − 5)(Xj − 5)]

i=1 j=1 n X

E[(Xi − 5)2 ]

i=1

1 σ2 = X = . n 10 2 (c) Since the sample variance is unbiased, E[ˆ s2] = σX = 1.

Problem 7.2 2 In this case, µX = 1 and σX = 1.

E[ˆ µ] = µX = 1 (since the sample mean is unbiased) 1 σ2 V ar(ˆ µ) = X = . n 2

121

Problem 7.3 Since the Ni are Gaussian, µ ˆ is also Gaussian with E[ˆ µ] = µN = 0 2 0.01 σN = = 10−4 . V ar(ˆ µ) = n 100 ⇒µ ˆ ∼ N (0, 10−4 ).

Problem 7.4 Consider a discrete random variable, X ∈ {−1, 0, 1}, whose PMF is PX (k) =

(

k = ±1, 1 − 2 k = 0.

2 = 2. Both the sample mean For this random variable, µX = 0 and σX and the median will be unbiased in this case. The variance of these two estimators are as follows: Sample Mean: σ2 2 V ar(ˆ µ) = X = . n n Median: Suppose n = 2k − 1 samples are taken. Then Yk is the median.

Pr(Yk = 1) = Pr(k or more X’s = 1) ! n X n m (1 − )n−m = m m=k Pr(Yk = −1) = Pr(Yk = 1) Pr(Yk = 0) = 1 − 2 Pr(Yk = 1) V ar(Yk ) = 2 Pr(Yk = 1) = 2

n X m=k

!

n m (1 − )n−m m

Note that for small , V ar(Yk ) ∼ n/2. Hence while the variance of the sample mean decays in an inverse linear fashion with n, the variance of the median decays exponentially in n. Hence in this case, the median would give a better (lower variance) estimate of the mean.

122

Problem 7.5 Since Xm (m from 1 to n) are IID sequence, assume the expected value and the variance are µ and σ respectively. Moreover, since µ ˆ=

n 1 X Xm , n m=1

n 1 X 2 ˆ (Xm − µ ˆ )2 σ = n m=1

=

n 1 X X2 − µ ˆ2 n m=1 m

From that, we have E(σˆ2) = = = = =

n 1 X 2 E(Xm ) − E(ˆ µ2 ) n m=1 n 1 X (µ2 + σ 2) − E(ˆ µ2 ) n m=1 n X 1 µ2 + σ 2 − 2 E[( Xm )2] n m=1 1 µ2 + σ 2 − 2 (nσ 2 + n2 µ2 ) n n−1 2 σ n

Hence, this estimate is biased.

Problem 7.6 n 1 X Xm n m=1 n 1 X = (Xm − µ ˆ )2 n − 1 m=1

µ ˆ = sˆ2

h

i

h

i

h

V ar(ˆ s2 ) = E (ˆ s2 )2 − E sˆ2 = E (ˆ s2 )2 − σ 4

123

i2

We can write the equation for sˆ2 a little differently using matrix tranformations. First make the following definitions: Y m = Xm − µ ˆ 1 n−1 1 1 · · · − ][X1 · · · Xm · · · Xn ]T Ym = [− − · · · n n n n T Y = [Y1 , Y2 , · · · Yn ] = AX  n−1  − n1 · · · − n1 n  − 1 n−1 · · · − 1   n n   A =  n  ··· ··· ··· ···  − n1 − n1 · · · n−1 n n×n Note that the covariance matrix of Y is 2 2 CY = E[YYT ] = E[AXXT AT ] = ACX AT = σX AAT = σX A. 2 n−1 2 From this we see that V ar(Yk ) = σX and Cov(Yk , Ym ) = −σX /n. Furn thermore, since the Yk are Gaussian, we have the following higher order moments (which will be needed soon):

(n − 1)2 , n2 E[Yk2 ]E[Ym2 ] + 2E[Yk Ym ]2 σY4 + 2Cov(Yk , Ym )2 2 4 (n − 1) 4 1 σX + 2σX 2 n n2 2 4 n − 2n + 3 σX . n2

4 E[Yk4 ] = 3σY4 = 3σX

E[Yk Ym ] = = = =

The variance of the sample variance is then found according to E[(ˆ s2)2 ] =

n n X X 1 E[Yi2 Yj2 ] (n − 1)2 i=1 j=1

n o 1 4 2 2 2 nE[Y ] + (n − n)E[Y Y ] i i j (n − 1)2 ( ) 2 2 1 (n − 1) n − 2n + 3 4 4 = 3nσX + (n2 − n)σX (n − 1)2 n2 n2 4 n+1 . = σX n−1

=

124

V ar(ˆ s2) = E[(ˆ s2)2 ] − E[ˆ s2 ]2 4 n+1 4 − σX = σX n−1 2 σ4 . = n−1 X

Problem 7.7 First, define Ym = Xm − µ ˆ for m = 1, 2, . . . , n. Note that both the {Ym } and µ ˆ are Gaussian random variables. Furthermore, µ − µ)(Xm − µ ˆ )] Cov(ˆ µ, Ym ) = E[(ˆ 2 µ ] − µE[Xm ] + µE[ˆ µ] = E[ˆ µXm ] − E[ˆ ! ! 2 2 σX σX + µ2 − + µ2 − µ2 + µ2 = n n = 0. Hence, µ ˆ is independent of all the Ym . Since sˆ2 is a function of the Ym , 1 P 2 Ym , then sˆ2 must also be independent of µ ˆ. namely n−1

Problem 7.8 Given x1 , x2, . . . , xN are observed, we want to minimize 2 =

N 1 X (xn − a − bn)2 . N n=1

Taking derivatives with respect to a and b and setting equal to zero produces N ∂2 1 X (−2)(xn − a − bn) = 0 = ∂a N n=1

!

N N N 1 X 1 X 1 X ⇒ xn = a 1 +b n N n=1 N n=1 N n=1

!

N 1 X ∂2 = (−2n)(xn − a − bn) = 0 ∂b N n=1

!

N N N 1 X 1 X 1 X ⇒ nxn = a n +b n2 N n=1 N n=1 N n=1

125

!

To simplify the notation, define the following: N 1 X n = n, N n=1

n2

N 1 X = n2 , N n=1

xn

N 1 X = xn , N n=1 N 1 X = nxn . N n=1

nxn

Then, the optimum values of a and b will satisfy the following matrix equation: 1 n a xn = n n2 b nxn . The solution is

2 n

· xn − n · nxn nxn − n · xn . n2 − (n)2

a = b

Problem 7.9 (a) A sequence converges in the mean square sense(MSS) if h

lim E |Sn − S|2

n→∞

i

= 0,

(1)

and it converges in probability if lim P r (|Sn − S| > ) = 0.

n→∞

(2)

Applying Markov’s ineqaulity to the LHS of (2) we get limn→∞ E [|Sn − S|2 ] n→∞ 2 ≤ 0 (since the sequence converges in MSS) ⇒ lim P r (|Sn − S| > ) = 0 (since probabilities cannot be negative) lim P r (|Sn − S| > ) ≤

n→∞

126

(b) Consider the sequence that takes two values 0, n with probabilities 1−1/n and 1/n respectively. The sequence tends to 0 as n tends to infinity. Consider the convergence in probability P r(Sn = 0) = 1 −

1 n

1 n lim P r (|S − S| > ) = lim P r (|Sn − 0| > ) n n→∞ n→∞ P r(Sn = n) =

= =

lim P r (Sn 6= 0)

n→∞

lim

n→∞

1 =0 n

So the sequence converges in probability.If we consider the convergence in MSS h

lim E |Sn − S|2

n→∞

i

= = =

h

lim E |Sn − 0|2

n→∞

lim 0(1 −

n→∞

i

1 1 ) + n2 n n

lim n 6= 0

n→∞

The sequence does not converge in MSS.

Problem 7.10 P

Z Let Zi = log Xi , then √1n ni=1 Ziσ−u converges to a standard normal random Z variable as n approaches infinity. And,

log Yn =

√

n 1 X Z i − uZ √ nσZ + nuZ . n i=1 σZ

Hence, Yn converges to a lognormal distribution with a mean of nE[log(Xi )] and a variance of nV ar(log(Xi )) . !

1 (log(y) − nη)2 exp − u(y), fYn (y) = √ 2ns2 2πns2y 2 where η = E[log(Xi )] and s2 = V ar(log(Xi )).

127

Problem 7.11 (a) Because Xi is a Bernoulli RV 2 = pA (1 − pA ) σX n 1X Xi pÂ = n i=1 E[pÂ ] = pA pA (1 − pA ) σ2 V ar(pÂ ) = X = n n

By virtue of the central limit theorem, we can write pÂ ∼ (pA ,

pA (1 − pA ) ) n





pA + ε − pA  P r(|pÂ − pA | < ε) = 1 − 2Q  q pA (1 − pA )/n =

v u u  1 − 2Q t



nε2  pA (1 − pA )

(b) P r(|pÂ − pA | < 0.1pA ) = 0.95 Using the result from (a) we get v u u  1 − 2Q t

⇒Q



n(0.1pA )2  = 0.95 pA (1 − pA ) s

⇒

0.01pA n (1 − pA )

s

!

≤ 0.025

0.01pA n ≥ 1.9597 ≈ 1.96 (1 − pA )

Note in the last step, the inequality is reversed since Q(x) is a decreasing function of x. ⇒ n ≥ 19.62

1 − pA pA

128

(c) Yn =

n X

Xi = npÂ

i=1

E[Yn ] = npA Since the value of n was chosen to satisfy the constraints of (b), we can write E[Yn ] = 19.62

1 − pA pA = 19.62 (1 − pA ). pA

Strictly speaking we will have E[Yn ] ≥ 19.62 (1 − pA ). If we assume that pA << 1 we can approximate it as E[Yn ] ≥ 19.62 ≈ 384.

Problem 7.12 (a) The probability mass function for the random variable N is !

k−1 m p (1 − pA )k−m . PN (k) = m−1 A Hence, !

∞ X

∞ X m−1 m−1 k−1 m PN (k) = pA (1 − pA )k−m E[pÂ ] = k−1 k−1 m−1 k=m k=m

=

∞ X k=m

!

k−2 m p (1 − pA )k−m . m−2 A

Using the substitutions, i = k − 2 and j = m − 2, the above expression is rewritten as ! ∞ X i m (1 − pA )i−j . E[pÂ ] = pA j i=j This series is evaluated using identity (E.12) in Appendix E of the text. The results is pm A E[pÂ ] = = pA . (1 − (1 − pA ))m+1

129

Hence, the estimate of pA is unbiased. (b) If pÂ = m , N !

∞ X

∞ X m m k−1 m PN (k) = pA (1 − pA )k−m 6= pA . E[pÂ ] = k k m − 1 k=m k=m

The estimate would be biased and hence not as good of a choice.

Problem 7.13 2 For this uniform random variable, µX = 0 and σX = 1/36. With a sufficiently large number of samples, the sample mean will be Gaussian. The mean and 1 . Hence the PDF of variance of the sample mean are E[ˆ µ] = 0 and σµ2ˆ = 36n the sample mean will be:

fµˆ (u) =

s

18n exp(−18nu2 ). π

Problem 7.14 µX = 5volts, σX = 0.25volts. For n = 100 samples, the sample mean will have E[ˆ µ] = 5volts, σµˆ =

1 volts. 40

The 99% confidence interval will be (µX − , µX + ) where = c0.99σµˆ = 2.58 ·

1 = 0.0645volts. 40

Hence, the 99% confidence interval is (4.9355,5.0645) volts. None of the estimates in (a)-(c) fall in this range.

Problem 7.15 √ 90% confidence: = 1.64σX / √ N1 . 99.9% confidence: = 3.29σX / N2 .

130

For these confidence intervals to be the same length we must have q

q

1.64σX / N1 = 3.29σX / N2

N2 3.29 2 = = 4.02. N1 1.64 N2 should be about 4 times larger than N1. ⇒

Problem 7.16 If M is an exponential random variable, then E[M] = µM and V ar(M) = 2 = µ2M . It is desired that the confidence interval have a width of = σM 0.2µM . Hence, the number of samples is determined from = c0.9σµˆ . This results in √ √ 0.2µM = 1.64σM / n = 1.64µM / n ⇒ n = 67.24. At least 68 failures need to be observed.

Problem 7.17 E[SN ] = E

"∞ X

#

Yi Z i =

i=1

∞ X

E[Yi Zi ]

i=1

Note the value of Yi is determine by whether or not the test terminates before timei. In particular, the values of {Z1 , Z2 , . . . , Zi−1 } determine Yi . In other words, Yi is dependent on {Z1 , Z2 , . . . , Zi−1 } but not on {Zi , Zi+1 , . . .}. Therefore, Yi and Zi are independent. ⇒ E[YiZi ] = E[Yi ]E[Zi ]. E[SN ] =

∞ X

E[Yi ]E[Zi]

i=1

Note that Zi is independent of i since the Zi are IID. Hence, E[SN ] = E[Zi ]

∞ X i=1

131

E[Yi ]

= E[Zi ]E

"∞ X

= E[Zi ]E

i=1 "N X i=1

= E[Zi ]E[N ].

132

Yi

#

#

1


Problem 8.1 (a) µX (t) =

1 1 2 1 1 · 1 + (−3) + · sin(2πt) = − + sin(2πt). 3 3 3 3 3

(b) RX,X (t1, t2) = E[X(t1)X(t2 )] 1 1 1 · 1 + (−3)(−3) + · sin(2πt1) sin(2πt2) = 3 3 3 8 1 = − + sin(2πt1) sin(2πt2). 3 3 (c) The process is not WSS since the mean is not constant nor is the autocorrelation a function of t2 − t2 .

Problem 8.2 This problem is worked out assuming that each member function occurs with equal probability. (a) 1 [−2 cos(t) − 2 sin(t) + 2(cos(t) + sin(t)) 5 + (cos(t) − sin(t)) − (cos(t) − sin(t))] = 0.

µX (t) =

(b) RX,X (t1, t2 ) = E[X(t1)X(t2)] 1 [4 cos(t1 ) cos(t2 ) + 4 sin(t1 ) sin(t2) + 4(cos(t1) + sin(t1))(cos(t2 ) + sin(t2)) = 5 + 2(cos(t1 ) − sin(t1 ))(cos(t2) − sin(t2))] 1 [10 cos(t1) cos(t2) + 10 sin(t1) sin(t2 ) + 2 cos(t1) sin(t2 ) + 2 sin(t1) cos(t2) = 5 2 = 2 cos(t2 − t1) + sin(t1 + t2). 5

133

(c) The process is not WSS since the autocorrelation is not a function of t2 − t2 only. Hence, the process is also not strictly stationary.

Problem 8.3 (a) 1 µX [n] = E[X[n]] = (1 + 2 + 3 + 4 + 5 + 6) = 3.5. 6 (b) RX,X [k1 , k2 ] = E[X[k1 ]X[k2]] If k1 6= k2 then RX,X [k1 , k2 ] = E[X[k1]]E[X[k2]] = (3.5)2 . If k1 = k2 then RX,X [k1, k2 ] = E[X 2 [k1 ]] = 16 (12 + 22 + 32 + 42 + 52 + 62) = RX,X [k1 , k2 ] =

(

49 4 91 6

k1 = 6 k2 , k1 = k2 .

Problem 8.4 Case 1: n1 even, n2 even

n1 n2 X RY,Y [n1 , n2] = E X 2 2

=

(

Case 2: n1 odd, n2 odd RY,Y [n1, n2 ] = E [X [n1 + 1] X [n2 + 1]] = Case 3: n1 odd, n2 even

n2 RY,Y [n1, n2 ] = E X [n1 + 1] X 2

=

Case 4: n1 even, n2 odd

n1 X [n2 + 1] = RY,Y [n1, n2 ] = E X 2

(

(

1 n1 = n 2 , 6 n2 . 0 n1 = (

1 n1 = n2 , 6 n2 . 0 n1 =

1 0

n2 = 2(n1 + 1), n2 6= 2(n1 + 1).

1 0

n1 = 2(n2 + 1), n1 6= 2(n2 + 1).

In summary,

RY,Y [n1 , n2] =

(

1 1 1 0

n1 = n 2 , n2 = 2(n1 + 1) (n1 odd), n1 = 2(n2 + 1) (n2 odd), otherwise.

134

91 . 6

Problem 8.5 (a) RY,Y [n1, n2 ] = E[(X[n1]+c)(X[n2]+c)] = RX,X [n1, n2 ]+cµX [n1]+cµX [n2 ]+c2 Since X[n] is WSS, µX [n] = µX and RX,X [n1, n2 ] = RX,X [n2 − n1 ]. ⇒ RY,Y [n1, n2 ] = RX,X [n2 − n1 ] + 2cµX + c2 . (b) E[X[n1]Y [n2]] = E[X[n1 ](X[n2] + c)] = RX,X [n2 − n1 ] + cµX . E[X[n1]]E[Y [n2]] = µX (µX + c) = µ2X + cµX . The processes are not orthogonal (since RX,Y [n1 , n2] 6= 0). The processes are not uncorrelated (since RX,Y [n1 , n2] 6= µx µY ). The processes are not independent (since not uncorrelated and since Y [n] = X[n] + c).

Problem 8.6 E[Y [n]] = E[(X[n + m] − X[n − m])2] = E[X 2 [n + m]] + E[X 2[n − m]] − 2E[X[n + m]X[n − m]] = 2RX,X (0) − 2RX,X (2m)

Problem 8.7 (a) µX (t) = µA cos(ωt) + µB sin(ωt) = 0. (b) RX,X (t1, t2 ) = E[A2] cos(ωt1) cos(ωt2) + E[B 2] sin(ωt1) sin(ωt2 ) + E[AB] cos(ωt1 ) sin(ωt2) + E[AB] sin(ωt1) cos(ωt2) E[A2] − E[B 2] E[A2] + E[B 2] cos(ω(t2 − t1 )) + cos(ω(t1 + t2 )). = 2 2 2 . (c) X(t) will be WSS if E[A2] = E[B 2] ⇒ σA2 = σB

135

Problem 8.8 Define X(t) and Y (t) according to: X(t) Y (t) µA (t) RA,A (τ ) RA,B (τ )

= = = = =

A(t) cos(t), B(t) sin(t), µB (t) = 0, RB,B (τ ) = R(τ ), 0.

Then, E[X(t)] = 0. E[Y (t)] = 0. RX,X (t1, t2 ) = E[A(t1)A(t2) cos(t1 ) cos(t2 )] = R(t2 − t1 ) cos(t1) cos(t2) RY,Y (t1, t2 ) = E[B(t1)B(t2) sin(t1) sin(t2 )] = R(t2 − t1 ) sin(t1) sin(t2 ) RZ,Z (t1, t2 ) = E[(X(t1) + Y (t1))(X(t2 ) + Y (t2))] = RX,X (t1, t2) + RY,Y (t1, t2 ) + RX,Y (t1, t2) + RY,X (t1, t2 ) = R(t2 − t1 )[cos(t1 ) cos(t2 ) + sin(t1) sin(t2)] (sinceRX,Y (τ ) = RY,X (τ ) = 0) = R(t2 − t1 ) cos(t2 − t1 ). Therefore, for this example, Z(t) = X(t) + Y (t) is WSS, while X(t) and Y (t) are not WSS.

Problem 8.9

X(t) = A(t) cos(ω0 t + Θ) E[X(t)] = E[A(t)]E[cos(ω0 t + Θ)] = 0 RX,X (t1, t2 ) = E[A(t1)A(t2)]E[cos(ω0 t1 + Θ) cos(ω0 t2 + Θ)] 1 RA,A (t2 − t1){E[cos(ω0 (t2 − t1 ))] + E[cos(ω0 (t1 + t2 ) + 2Θ)]} = 2 1 RA,A (t2 − t1) cos(ω0 (t2 − t1)). = 2

136

Similarly, E[Y (t)] = 0 1 RA,A (t2 − t1) cos((ω0 + ω1 )(t2 − t1 )). RY,Y (t1 , t2) = 2 Hence, both X(t) and Y (t) are WSS. If Z(t) = X(t) + Y (t), E[z(t)] = E[X(t)] + E[Y (t)] = 0, RZ,Z (t1 , t2) = RX,X (t1, t2) + RY,Y (t1, t2 ) + RX,Y (t1, t2) + RY,X (t1, t2 ), RX,Y (t1 , t2) = E[A(t1)A(t2)]E[cos(ω0 t1 + Θ) cos((ω0 + ω1 )t2 + Θ)], 1 RA,A (t2 − t1) cos(ω0 (t2 − t1) + ω1 t2), = 2 RY,X (t1 , t2) = RA,A (t2 − t1) cos(ω0 (t1 − t2) + ω1 t1 ). Since RZ,Z (t1, t2) is not a function of t1 − t2, Z(t) is not WSS.

Problem 8.10 (a) Pr(X(t) = 1) = + = +

Pr(X(t) = 1|X(0) = 1) Pr(X(0) = 1) Pr(X(t) = 1|X(0) = −1) Pr(X(0) = −1) p · Pr(even number of switches in (0, t)) (1 − p) · Pr(odd number of switches in (0, t)) 1 1 1 1 + exp(−2λt) + (1 − p) − exp(−2λt) = p 2 2 2 2 1 1 − 2p = − exp(−2λt). 2 2 1 1 − 2p exp(−2λt). Pr(X(t) = −1) = 1 − Pr(X(t) = 1) = + 2 2 (b) E[X(t)] = 1 · Pr(X(t) = 1) + (−1) · Pr(X(t) = −1) = (1 − 2p) exp(−2λt). (c) First assume t2 > t1. To calculate E[X(t1)X(t2 )], note that X(t1 )X(t2 ) =

(

1 if even number of switches in (t1, t2), −1 if odd number of switches in (t1, t2).

137

Then, RX,X (t1 , t2) = 1 · Pr(even number of switches in (t1, t2)) + (−1) · Pr(odd number of switches in (t1, t2)) 1 1 1 1 + exp(−2λ(t2 − t1)) − + exp(−2λ(t2 − t1 )) = 2 2 2 2 = exp(−2λ(t2 − t1)). Similarly, if t1 > t2 , RX,X (t1, t2) = exp(−2λ(t1 − t2 )). Putting these results together, we get that RX,X (t1, t2) = exp(−2λ|t1 − t2|) (d) Although RX,X (t1 , t2) is only a function of t2 −t1 , the mean is not necessarily constant. Hence, the process is WSS only if p = 1/2 (so that µX (t) = 0).

Problem 8.11 (a) Since T is uniformly distributed over one period of s(t), for any time instant t, X(t) = s(t − T ) will be equally likely to take on any of the values in one period of s(t). Since s(t) is 1 half of the time and -1 half of the time, we get 1 Pr(X(t) = 1) = Pr(X(t) = −1) = . 2 (b) E[X(t)] = (1) · Pr(X(t) = 1) + (−1) · Pr(X(t) = −1) = 0. This can also be seen in an alternative manner: E[X(t)] = E[s(t − T )] =

Z

s(t − u)fT (u)du =

Z 1

s(t − u)du.

0

Since the integral is over one period of s(t), E[X(t)] is just the d.c. value (time average) of s(t) which is zero. (c) RX,X (t1, t2) = E[s(t1 − T )s(t2 − T )]

138

=

Z 1 0

=

Z 1 0

=

s(t1 − u)s(t2 − u)du s(v)s(v + t2 − t1 )dv

s(t) ∗ s(−t)

.

t=t2 −t1

This is the time correlation of a square wave with itself which will result in the periodic triangle wave shown in Figure 1.

1

0

R

X,X

(τ)

0.5

−0.5

−1 −1

−0.5

0.5 τ

0

1

1.5

2

Figure 1: Autocorrelation function for process of Exercise 8.11

(d) The process is WSS.

Problem 8.12 (a) Since T is uniformly distributed over one period of s(t), for any time instant t, X(t) = s(t − T ) will be equally likely to take on any of the values in one period of s(t). Given the linear functional form of s(t), X(t) will be uniform over (−1, 1). fX (x; t) =

(

1 2

−1 ≤ x ≤ 1, 0 otherwise.

139

(b) E[X(t)] = 0 since the PDF above is symmetric about zero. (c) RX,X (t1, t2) = E[s(t1 − T )s(t2 − T )] =

Z 1 0

=

Z 1 0

=

s(t1 − u)s(t2 − u)du s(v)s(v + t2 − t1 )dv

s(t) ∗ s(−t)

.

t=t2 −t1

This is the time correlation of a triangle wave with itself which will result in the periodic signal shown in Figure 2. 0.4

RX,X(τ)

0.2

0

−0.2

−0.4 −1

−0.5

0

0.5 τ

1

1.5

2

Figure 2: Autocorrelation function for process of Exercise 8.12

(d) The process is WSS.

Problem 8.13 (a) E[X(t)] = Pr(X(t) = +1) − Pr(X(t) = −1) =

1 1 − = 0. 2 2

(b) X(t1 )X(t2) =

(

1 ±1

if no pulse transitions in (t1, t2), with equal prob. if 1 or more pulse transitions in (t1, t2).

140

E[X(t)] = 0 · Pr(≥ 1 transitions in (t1, t2 )) + 1 · Pr(no transitions in (t1, t2)). For |t2 − t1 | < ∆, Pr(no transitions in (t1, t2)) = 1 − Therefore, RX,X (t1 , t2) =

(

1−

|t2 −t1 | ∆

|t2 − t1| . ∆

|t2 − t1| < ∆, |t2 − t1| > ∆.

0

(c) X(t) is WSS.

Problem 8.14 (a)

fA (a)

fX (x; t) = dX

dA A=− 1 ln(x) t

(b) E[X(t)] = E[e−At ] =

Z ∞

=

fA (− 1t ln(x)) tx

e−at e−ada =

0

1 . 1+t

RX,X (t1, t2) = E[X(t1)X(t2 )] = E[e−A(t1 +t2 ) ] =

1 . 1 + t1 + t2

The process is not WSS.

Problem 8.15 (a) E[X[n]] = E[pX[n − 1]] + E[Wn ] ⇒ µX [n] = pµX [n − 1] ⇒ µX [n] = pn µX [0]

Since µX [0] = E[W0 ] = 0, then µX [n] = 0 for all n.

141

(b) Assume n2 > n1 . Then, X[n2 ] = pX[n2 − 1] + Wn2 = p2 X[n2 − 2] + Wn2 + pWn2 −1 = p3 X[n2 − 3] + Wn2 + pWn2 −1 + p2 Wn2 −2 .. . = pn2 −n1 X[n1 ] +

n2 −n X1 −1

pm Wn2 −2

m=0

.. . = E[X[n1]X[n2]] = =

n2 X

m

p Wn2 −m =

m=0 n1 X n2 X

n2 X

pn2 −i Wi

i=0

pn1 −i pn2 −j E[Wi Wj ]

i=0 j=0 n1 X n1 +n2 −2i

p

E[Wi2]

i=0

=

2 n1 +n2 σW p

n1 X

(p−2 )i

i=0 −2 n1 +1 2 n1 +n2 1 − (p ) = σW p 1 − p−2 n1 +n2 +2 − pn2 −n1 2 p = σW . p2 − 1 Similarly, if n1 > n2 , then

pn1 +n2 +2 − pn1 −n2 . p2 − 1 Putting these two results together gives 2 E[X[n1]X[n2 ]] = σW

2 RX,X [n1 , n2] = σW

pn1 +n2 +2 − p|n2 −n1 | . p2 − 1

Problem 8.16 E[X 2 (t1)Y 2 (t2)] = E[X 2(t1 )]E[Y 2 (t2)] + 2E[X(t1 )Y (t2)]2 = RX,X (0)RY,Y (0) + 2R2X,Y (t2 − t1).

142

Problem 8.17 Using the result of Exercise 6.12, E[X(t1)X 3 (t2)] = 3E[X(t1)X(t2 )]E[X 2(t2 )] = 3RX,X (t2 − t1)RX,X (0).

Problem 8.18

RZ,Z [k] = RX,X [k] + RY,Y [k] + RX,Y [k] + RY,X [k]. RX,Y [k] = E[X[n]Y [n + k] = µX [n]µY [n + k] = 0. ⇒ RZ,Z [k] = RX,X [k] + RY,Y [k]. Figure 3 shows the various autocorrelation functions.

RX,X[k] RY,Y[k] RZ,Z[k]

Autocorrelation

2

1.5

1

0.5

0 −8

−6

−4

−2

0 k

2

4

6

8

Figure 3: Autocorrelation functions for processes of Exercise 8.18

Problem 8.19 Since lim RX,X [k] = µX = 0,

k→∞

the process is ergodic in the mean.

143

Problem 8.20

hY (t)i = hC · X(t)i = ChX(t)i = CµX . E[Y (t)] = E[C · X(t)] = E[C]µX . Since these two quantities are not equal, the process is not ergodic in the mean. hY (t1 )Y (t2)i = hC 2 · X(t1)X(t2 )i = C 2hX(t1 )X(t2 )i = C 2RX,X (t1 , t2). E[Y (t1)Y (t2 )] = E[C 2 · X(t1 )X(t2)] = E[C 2]RX,X (t1, t2 ). Since these two quantities are not equal, the process is not ergodic in the mean.

Problem 8.21 The first equation is a first order constant coefficient differential equation whose solution is PX (0; t) = c · e−λt. The constant c is resolved by noting that since PX (i; t) is a counting process, at time t = 0, ( 1 i = 0, PX (i; 0) = Pr(X(0) = i) = 0 i > 0. Since PX (0; 0) = 1, we get c = 1 ⇒ PX (0; t) = e−λt. The second differential equation is also first order, contant coefficient (although not homogeneous). It can be solved using the integrating factor technique to be PX (i; t) = λ

Z t 0

PX (i − 1; u) exp(−λ(t − u))du + c · e−λt .

Using the initial condition PX (i; 0) = 0 for i > 0 we get c = 0. Therefore, PX (i; t) = λ

Z t 0

PX (i − 1; u) exp(−λ(t − u))du

144

For

R

i = 1, PX (1; t) = λ 0t e−λtdu = λte−λt, R 2 e−λt , i = 2, PX (2; t) = λ 0t λue−λtdu = (λt) 2 R t (λu)2 −λt 3 e−λt, i = 3, PX (3; t) = λ 0 2 e du = (λt) 3! .. .

In general PX (i; t) =

(λt)i −λt e . i!

Problem 8.22 (a) Consider a time instant, t, such that 0 < t < to . Pr(X(to ) = 1|X(t) = 1) Pr(X(t) = 1) Pr(X(to ) = 1) λte−λt Pr(no arrivals in (0, t)) = λto e−λto t = exp(−λ(t − to )) exp(−λ(to − t)) to t = . to

Pr(X(t) = 1|X(to ) = 1) =

Let S1 be the arrival time of the first arrival. Then {X(t) = 1} ⇔ {S1 ≤ t}. Hence, given that there is one arrival in (0, to), that is X(to ) = 1, Pr(X(t) = 1|X(to ) = 1) = Pr(S1 ≤ t|X(to ) = 1) = FS1 (t|X(to) = 1) = ⇒ fS1 (t|X(to) = 1) =

1 , to

t to

0 ≤ t < to .

(b) Let 0 ≤ t1 ≤ t2 ≤ to . Also define S1 = arrival time of first arrival, S2 = arrival time of second arrival. The joint distribution of the two arrival times is found according to: fS1 ,S2 (t1 , t2|X(to ) = 2) = fS1 |S2 (t1|S2 = t2, X(to ) = 2)fS2 (t2|X(to ) = 2) = fS1 |S2 (t1|S2 = t2)fS2 (t2|X(to ) = 2)

145

To find fS2 (t2), proceed as in part (a). FS2 (t2|X(to ) = 2) = Pr(X(t2 ) = 2|X(to ) = 2) = Pr(X(to ) = 2|X(t2 ) = 2) = exp(−λ(to − t2 ))

2

t2 to 2t2 , ⇒ fS2 (t2|X(to ) = 2) = t2o =

Pr(X(t2) = 2) Pr(X(to ) = 2)

(λt2 )2 −λt2 e 2 2 (λt0 ) −λto e 2

. 0 ≤ t2 ≤ to .

Given S2 = t2 there is one arrival between 0 and t2 . From the results of part (a), we know S1 is uniform over (0, t2 ) given S2 = t2. Therefore fS1 |S2 (t1 |t2) =

1 , t2

0 ≤ t1 ≤ t2 .

Putting the two previous results together we get fS1 ,S2 (t1, t2|X(to ) = 2) = fS1 |S2 (t1|S2 = t2)fS2 (t2|X(to ) = 2) 2t2 1 = · t2o t2 2 = 2, 0 ≤ t1 ≤ t2 ≤ to . to The two arrival times S1 and S2 are uniformly distributed over 0 ≤ t1 ≤ t2 ≤ to .

Problem 8.23

Pr(N (t) = k|N (t + τ ) = m) = Pr(N (t + τ ) = m|N (t) = k) = =

(λτ )m−k −λτ (λt)k −λt e e (m−k)! k! m (λ(t+τ )) exp(−λ(t + τ )) m! ! k m−k

m t τ . k (t + τ )m

146

Pr(N (t) = k) Pr(N (t + τ ) = m)

Problem 8.24 (a) E[Y (t) =

1 µ X − µX (E[X(t + to )] − E[X(t)]) = = 0. to to

(b) "

#

(X(t1 + to ) − X(t1 )(X(t2 + to ) − X(t2 )) RY,Y (t1, t2) = E t2o RX,X (t2 − t1 ) − RX,X (t2 − t1 − to ) − RX,X (t2 − t1 + to) + RX,X (t2 − t1) = t2o 2RX,X (τ ) − RX,X (τ − to ) − RX,X (τ + to) = t2o The process Y (t) is WSS.

Problem 8.25 Using probability generating functions: HX (z) = E[z

X(t)

] = E[z

Pn

Xi (t) i=1

]=E

" n Y

z

Xi (t)

#

i=1

=

n Y

E[z Xi (t)] =

i=1

(λi t)k −λi t e . k! ∞ X (λi zt)k −λit e = eλizt e−λi t = eλit(z−1) . HXi (z) = k! k=0

Pr(Xi (t) = k) =

HX (z) =

n Y

λit(z−1)

e

= exp

i=1

n X

!

!

λi t(z − 1) .

i=1

P

Define λ = ni=1 λi . Then HX (z) = exp(λt(z − 1)) which is the probability generating function of a Poisson random variable. Therefore X(t) is a Poisson P proces with arrival rate λ = ni=1 λi .

Problem 8.26 (a) Pr(N (t) = k) = Pr(k of the Ti are < t, n − k are > t)

147

n Y i=1

HXi (z).

= =

!

n (Pr(Ti < t))k (Pr(Ti > t))n−k k ! n (1 − e−λt )k (e−λt )n−k k

Therefore, N (t) is a Binomial process. (b) !

n −nλt e = 1 − e−nλt. Pr(N (t) ≥ 1) = 1 − Pr(N (t) = 0) = 1 − 0 Using n = 10, λ = (250 days)−1 , and t = 90 days, we get Pr(N (t) ≥ 1) = 1 − exp(−

900 ). 250

Problem 8.27 Pr(N (t) < 10) =

9 X

(λt)k −λt e k=0 k!

(a) λ = 0.1, t = 10 ⇒ P r(N (t) < 10) =

9 X

(1)k −1 e ≈ 1. k! k=0

(b) λ = 10, t = 10 ⇒ P r(N (t) < 10) =

9 X

(100)k −100 ≈ 0. e k! k=0

(c) Pr(1 call in 10 minutes) = 1 · e−1 = 0.3679. 12 −1 · e = 0.1839. Pr(2 calls in 10 minutes) = 2! Pr(1 call, 2 calls) = Pr(1 call) Pr(2 calls) 13 −2 e = 0.0677. = 2!1!

148

Problem 8.28 Expected number of strikes is st. (a) In one minute, st = 13 · 1 = 13 . . (b) In ten minute, st = 13 · 10 = 10 3 (c) Average time between strikes is

1 s

= 3 minutes.

Problem 8.29 (a) Figure 4 shows a typical realization.

0.08 0.07 0.06

X(t)

0.05 0.04 0.03 0.02 0.01 0

20

15 10 time, t (minutes)

5

0

25

Figure 4: Sample realization for process of Exercise 8.29

(b) E[X(t)] = s

Z t

t exp(−at)dt =

0

s [1 − (1 + at)e−at]. a2

(c) CX,X (t, t + τ ) = s

Z t

t exp(−at)(t + τ ) exp(−a(t + τ ))dt

0 −aτ

= se

Z t

2 −2at

te

dt + τ

0

Z t

te

−2at

dt

0

τ 1 = se [1 − (1 + 2at + 2a2 t2)e−2at ] + 2 [1 − (1 + 2at)e−2at] 3 4a 4a RX,X (t, t + τ ) = CX,X (t, t + τ ) + E[X(t)]E[X(t + τ )] −aτ

149

Problem 8.30 Following the same procedure used in the text, approximate the shot noise process as X(t) =

∞ X

An Vn h(t − n∆t),

n=0

where the An are IID and independent of the Vn ∈ (0, 1) which are also IID with Pr(Vn = 1) = λ∆t. (a) E[X(t)] =

∞ X

E[An ]E[Vn ]h(t − n∆t)

n=0

= λE[A] = λE[A]

∞ X

h(t − n∆t)∆t

n=0 Z ∞

h(t − u)du

0

= λE[A]

Z t

h(v)dv.

0

(b) RX,X (t, t + τ ) = E =

"∞ X

An Vn h(t − n∆t)

n=0 ∞ ∞ X X

∞ X

Am Vm h(t + τ − m∆t)

#

m=0

E[AnAm ]E[Vn Vm ]h(t − n∆t)h(t + τ − m∆t)

n=0 m=0

Note that E[Vn Vm ] =

(

λ∆t n = m, 2 6 m. (λ∆t) n =

Using this, the autocorrelation becomes: RX,X (t, t + τ ) = E[A]2

∞ ∞ X X

+ E[A2]

n=0 m=0 ∞ X

− E[A]2

n=0 ∞ X

(λ∆t)2h(t − n∆t)h(t + τ − m∆t)

(λ∆t)h(t − n∆t)h(t + τ − n∆t) (λ∆t)2 h(t − n∆t)h(t + τ − n∆t).

n=0

150

In the limit as ∆t → 0, the last term will be insignificant because of the (∆t)2 term. The autocorrelation then becomes: RX,X (t, t + τ ) = λ2 E[A]2 + λE[A2]

Z ∞

0 Z ∞

h(t − u)du

Z ∞

h(t + τ − v)dv

0

h(t − u)h(t + τ − u)du

0 2

= µX (t)µX (t + τ ) + λE[A ]

151

Z ∞ 0

h(v)h(v + τ )dv.


Problem 9.1 P=

"

1−p p q 1−q

#

.

Performing an eigen decomposition, we get P = QΛQ−1 where Q=

"

1 p 1 −q

#

,Λ =

"

1 0 0 1−p−q

#

,Q

−1

1 = p+q

"

q p 1 −1

#

.

Then Pn = QΛn Q−1 " #" #" # 1 1 p 1 0 q p = 0 (1 − p − q)n 1 −1 p + q 1 −q " # 1 q + p(1 − p − q)n p − p(1 − p − q)n = . p + q q − q(1 − p − q)n p + q(1 − p − q)n

Problem 9.2 (a) Pr(X2 = 1, X1 = 0) = Pr(X2 = 1|X1 = 0) Pr(X1 = 0) = p · Pr(X1 = 0). Pr(X1 = 0) = Pr(X1 = 0|X0 = 0) Pr(X0 = 0) + Pr(X1 = 0|X0 = 1) Pr(X0 = 1) = (1 − p)s + q(1 − s). ⇒ Pr(X2 = 1, X1 = 0) = p(1 − p)s + pq(1 − s). (b) Pr(X1 = 1|X0 = 0, X2 = 0) =

Pr(X2 = 0|X1 = 1, X0 = 0) Pr(X1 = 1|X0 = 0) Pr(X2 = 0|X0 = 0) (1)

=

(1)

P1,0 P0,1 (2)

P0,0 qp . = (1 − p)2 + pq 1 153

(c) Pr(X2 = X1 ) = = = = Pr(X1 = X0 ) = =

Pr(X2 = 0, X1 = 0) + Pr(X2 = 1, X1 = 1) (1 − p) Pr(X1 = 0) + (1 − q) Pr(X1 = 1) (1 − p)[(1 − p)s + q(1 − s)] + (1 − q)[ps + (1 − q)(1 − s)] (1 − p)2 s + q(1 − p)(1 − s) + p(1 − q)s + (1 − q)2 (1 − s). Pr(X0 = 0, X1 = 0) + Pr(X0 = 1, X1 = 1) (1 − p)s + (1 − q)(1 − s).

These expressions are not the same, so Pr(X1 = X0 ) 6= Pr(X2 = X1 ).

Problem 9.3 To simplify notation, define Y = [Xk+2 , Xk+3 , . . . , Xk+m ]. Then, Pr(Xk |Xk+1 , Xk+2 , . . . , Xk+m ) = Pr(Xk |Xk+1 , Y) Pr(Xk+1 , Y|Xk ) Pr(Xk ) = Pr(Xk+1 , Y) Pr(Y|Xk+1 , Xk ) Pr(Xk+1 |Xk ) Pr(Xk ) = Pr(Y|Xk+1 ) Pr(Xk+1 ) Because the process is a Markov chain, Pr(Y|Xk+1 , Xk ) = Pr(Y|Xk+1 ). Therefore, Pr(Xk |Xk+1 , Y) =

Pr(Xk+1 |Xk ) Pr(Xk ) = Pr(Xk |Xk+1 ). Pr(Xk+1 )

The statement is true. That is, a Markov chain viewed in reverse time still possesses the memoryless property.

Problem 9.4 For a general 2x2 transition matrix, P=

"

1−p p q 1−q

154 2

#

,

the two-step transition matrix is 2

P =

"

(1 − p)2 + pq p(1 − p) + p(1 − q) q(1 − p) + q(1 − q) (1 − q)2 + pq

#

.

Note that P2 is a stochastic matrix as each element is ≤ 1 (for 0 ≤ p, q ≤ 1) and each row sums to 1. Summing the diagonal elements we note that (2)

(2)

P1,1 + P2,2

= = = = ≥

(1 − p)2 + pq + (1 − q)2 + pq 2 − 2p − 2q + p2 + q 2 + 2pq 1 + 1 − 2(p + q) + (p + q)2 1 + (1 − p − q)2 1.

Since any two-step transition matrix must be of this general form, any twostep transition matrix must have the sum of its diagonal elements ≥ 1. Next, consider a general stochastic matrix A=

"

1−α α β 1−β

#

.

The matrix A will have an eigen decompostion of the form A = QΛQ−1 where the matrix of eigenvalues is of the form Λ=

"

1 0 0 1−α−β

#

. √ A must be a stochastic

For this to be a valid matrix, √ transition √ two-step −1 matrix. Note that A = Q ΛQ where √ Λ=

"

1 √ 0 1−α−β 0

#

.

√ This will result in a valid stochastic matrix if 1 − α − β is real. Hence, we must have α + β ≤ 1 or equivalently (1 − α) + (1 − β) ≥ 1. Therefore the matrix A will be a valid two-step transition matrix if the sum of its diagonal components is ≥ 1.

155 3

Problem 9.5 (a) The states of the system can be the values of the voltage between switches. Hence there are three states, namely -1, 0, and +1. With this representation, the process is a random walk with absorbing boundaries. The corresponding state transition matrix is 



0 1 0   P =  q 0 p . 0 1 0 (n)

(b)For this process Pi,i > 0 only if n is even. Therefore, the process is periodic with period 2. (c) For (i − 1)ts ≤ t < its and i odd, X(t) = 0. For (i − 1)ts ≤ t < its and i even, Pr(X(t) = 1) = p, Pr(X(t) = −1) = q = 1 − p.

Problem 9.6 Given there are i black balls (and n − i white balls) in Urn A, there must be n − i black balls (and i white balls) in Urn B. Represent this state by Xt = i. Given Xt = i, there are three possibilities for Xt+1 : Xt+1 =

(

i−1 i+1 i

if black selected from A and white selected from B, if white selected from A and black selected from B, same color selected from both urns.

The transition probabilites are then calculated as follows: Pr(Xt+1 = i − 1|Xt = i) = Pr(A=black, B=white|Xt = i) 2 i i i · = , = n n n Pr(Xt+1 = i + 1|Xt = i) = Pr(A=white, B=black|Xt = i) n−i 2 n−i n−i · = , = n n n 2 i n−i 2 Pr(Xt+1 = i|Xt = i) = 1 − − . n n

156 4

Problem 9.7 (a) h

π(1) = π(0)P =

"

i

1 2

1 2

Therefore, Pr(+1 after 1 step) = 38 . (b) h

π(1) = π(0)P =

1 3

"

i

2 3

1 2 1 4

1 2 3 4

1 2 1 4

1 2 3 4

#

#

=

h

=

h

3 8

5 8

1 3

2 3

Therefore, Pr(+1 after 1 step) = 23 . (c) π(2) = π(0)P2 , where 2

P = With π(0) =

h

1 2

1 2

"

1 2 1 4

i

#"

1 2 3 4

1 2 1 4

1 2 3 4

#

=

"

3 8 5 16

5 8 11 16

#

.

,

π(2) =

h

1 2

1 2

i

Therefore, Pr(+1 after 1 step) = h i 1 2 With π(0) = 3 3 , π(2) =

h

1 3

2 3

"

3 8 5 16

5 8 11 16

#

=

h

11 32

21 32

1 3

2 3

i

11 . 32

i

"

3 8 5 16

5 8 11 16

#

=

h

i

Therefore, Pr(+1 after 1 step) = 13 .

Problem 9.8 (a) go skip go P= skip

1 2 3 4

1 2 1 4

!

(b) 1 Pr(go Friday|went Wednesday) = . 2

157 5

.

.

i

i

(c) 2

P =

1 2 3 4

#"

1 2 1 4

1 2 3 4

1 2 1 4

#

=

"

5 8 9 16

3 8 7 16

#

5 (2) Pr(go Friday|went Monday) = P0,0 = . 8 where

(d) P = QΛQ−1 1 Q= 4

"

"

4 2 4 −3

#

1 ,Λ = 4

"

#

4 0 0 −1

,Q

−1

1 = 5

"

3 2 4 −4

#

.

The limiting form of the eigenvalue matrix is 

lim Λ = lim  k

k→∞

k→∞

1 0

0 − 14



k 

=

"

1 0 0 0

#

.

Hence, the limiting form of the k-step transition matrix is 1 lim Pk = k→∞ 20

"

4 2 4 −3

#"

1 0 0 0

#"

3 2 4 −4

#

1 = 5

"

3 2 3 2

#

.

Since both rows are equal, a steady state distribution does exist and it is given by h i π = 35 25 .

Problem 9.9 (a)





0.25 0.5 0.25   P =  0.4 0.6 0  = QΛQ−1 . 1 0 0 Using MATLAB we found, 



−0.4197 0.5774 0.2792   Q =  0.1570 0.5774 −0.3981  , 0.8940 0.5774 0.8738

158 6





−0.4695 0 0   Λ= 0 1 0 . 0 0 0.3195

The limiting form of the k-step transition probability matrix is then found as follows: 

lim Λk

k→∞



0 0 0   =  0 1 0 . 0 0 0

lim Pk = Q lim Λk Q−1

k→∞

k→∞





0.4 0.5 0.1   =  0.4 0.5 0.1  . 0.4 0.5 0.1

The steady-state distribution is then π=

h

i

0.4 0.5 0.1 .

(b) Using MATLAB to calculate P100, (a):  0.4  100 P =  0.4 0.4

we get the same matrix found in part

(100)

⇒ P1,3



0.5 0.1 0.5 0.1  . 0.5 0.1 = 0.1.

The interpretation of this result is that for all practical purposes, the process has reached steady state after 100 steps. (c) π(3) = π(0)P3 =

h

1 3

1 3

1 3

i

3



0.25 0.5 0.25 h i   0  = 0.4274 0.4808 0.0919  0.4 0.6 1 0 0

Pr(in state 3 after 3rd step) = 0.0919.

Problem 9.10 1 · 0.6 · 0.2 = 0.04. 3 1 Pr(C A T) = · 0.7 · 0.3 = 0.07. 3 1 Pr(T A T) = · 0.8 · 0.3 = 0.08. 3 Pr(A C T) =

159 7

Therefore, Pr(proper english word) = 0.04 + 0.07 + 0.08 = 0.19.

Problem 9.11 (a) 

0 1

0  5 1  12 2  0  P = 3 0  4  0 5  0 6 0

1 0 0 5 12

0 0 0 0

2 0 7 12

0 5 12

0 0 0

3 0 0 7 12 1 6 7 12

0 0

4 0 0 0 5 12

0 7 12

0

5 0 0 0 0 5 12

0 0

6 0 0 0 0 0



          5  12 

1

(b) Pr(loses in 3 tosses) = Pr(3 → 2 → 1 → 0) = (5/12)3 = 0.0723. (c)

1 5 Pr(loses in 4 tosses) = Pr(3 → 3 → 2 → 1 → 0) = 6 12

3

= 0.0121.

Pr(loses in 5 tosses) = + + +

Pr(3 → 3 → 3 → 2 → 1 → 0) Pr(3 → 4 → 3 → 2 → 1 → 0) Pr(3 → 2 → 3 → 2 → 1 → 0) Pr(3 → 2 → 1 → 2 → 1 → 0) 2 3 7 5 4 1 5 +3 = 0.0548. = 6 12 12 12

Pr(loses in 5 or fewer tosses) = 0.0723 + 0.0121 + 0.0548 = 0.1392. We can verify this solution using MATLAB by noting that the probability of interest is found by finding the entry in the 4th row and 1st column of P5 .

160 8

Problem 9.12 (a) 

0 0

0 1 1  n2 n P= 2 .. ..  .   . n

1

1 1 0 0 .. .

2 0 n−1 n

0

0

0 .. .

3 0 0 n−2 n

.. . 0

... n  ... 0  ... 0   ... 0   ..  .. . .  ... 0

(b) Pr(M = m|X0 = 0)

= Pr(X1 = 1|X0 = 0) Pr(X2 = 2|X1 = 1) . . . . . . Pr(Xm−1 = m − 1|Xm−2 = m − 2) Pr(Xm = 0|Xm−1 = m − 1) n−1 n−2 n−m+2 m−1 = 1· · ··· · n n n n (m − 1)n! = nm (n − m + 1)!

Problem 9.13 pi,j = Pr(1 person Pr(2 people Pr(4 people Pr(8 people

infected) infected) infected) infected)

(

pi j = 2i i 1−p j =0

= = = = .. . k Pr(2 people infected) =

1−p p(1 − p2 ) p · p2 (1 − p4 ) = p3 (1 − p4 ) p · p2 · p4 (1 − p8 ) = p7 (1 − p8 ) p2

k −1

k

(1 − p2 )

Problem 9.14 (a) Let Yn = Xn mod 3 be the states of a Markov chain. Then {Xn is a multiple of 3} ⇔ {Yn = 0}.

161 9

The transition matrix for this Markov chain is 

1 1 P = 1 3 1



1 1  1 1. 1 1

Since P2 = P, Pn = P ⇒ π = 13 [1, 1, 1]. Therefore 1 lim Pr(Yn = 0) = . 3

n→∞

(b) Let Yn = Xn mod 5 be the states of a Markov chain. Then {Xn is a multiple of 5} ⇔ {Yn = 0}. The transition matrix for this Markov chain is 

1 1 1  P = 1 6 1 2

2 1 1 1 1

1 2 1 1 1

1 1 2 1 1



1 1   1.  2 1

The steady state distribution is π = 15 [1, 1, 1, 1, 1] since with this distribution π = πP. Therefore 1 lim Pr(Yn = 0) = . n→∞ 5

Problem 9.15 The eigendecomposition of the given transition matrix is 

0.4 0.5  P =  0.05 0.7 0.05 0.5  −0.9931  =  0.0828 0.0828



0.1  0.25  0.45  0.5774 0.6509 0.35 0  0.5774 −0.3906   0 1 0.5774 0.6509 0 0



0 −0.9295  0   0.1332 0.2 0

0 1.0825 −0.9601

From this it is easily determined that the limiting form on the n-step transition matrix is 

lim Pn

n→∞

−0.9931  =  0.0828 0.0828

0.5774 0.5774 0.5774





0.6509 0 0 0 −0.9295   −0.3906   0 1 0   0.1332 0.6509 0 0 0 0

162 10

0 1.0825 −0.9601



0.9295  0.5163  . 0.9601



0.9295  0.5163  0.9601



0.0769  =  0.0769 0.0769

0.6250 0.6250 0.6250



0.2981  0.2981  . 0.2981

Since all rows are identical, there is a steady state distribution and it is given by π = [ 0.0769 0.6250 0.2981 ] .

Problem 9.16 (a) If the process is currently in state (X,Y) it must transition to a state of the form (?,X). This eliminates half of the possible transitions and hence the transition matrix must have at least half of its entries equal to zero. (b) π = 14 [ 1 1 1 1 ] is the solution to πP = π. (c) Pr(process is in state A) = Pr((A, A) ∪ (A, B)) = Pr(A, A) + Pr(A, B) =

1 1 1 + = . 4 4 2

Problem 9.17 (a) The first three states communicate with period 1. (b) All states communicate with period = 3. (c) All states communicate with period = 2.

Problem 9.18 Suppose d(j) = n. Further suppose i ↔ j and a transition for i to j is possible in k steps and from j to i in m steps. Then transitions from i to i are possible in steps of k + m + n, k + m + 2n, k + m + 3n, ... Thus d(i) = GCD{numbers which include k + m + n, k + m + 2n, . . .} ⇒ d(i) ≤ n or d(i) ≤ d(j). Using a similar argument, we conclude that d(j) ≤ d(i) and therefore we must have d(i) = d(j).

163 11

Problem 9.19 Starting with (9.19) in the text, (n) pi,i

=

n X

(n−m) (m) fi,i ,

pi,i

n = 1, 2, 3, . . . .

m=0

Multiplying each equation by z n and summing over n results in ∞ X

n ∞ X X

(n)

pi,i z n =

n=1

(n−m) (m) n fi,i z .

pi,i

n=1 m=0 (0)

We can add the n = 0 term to the series on the right hand side since fi,i = 0 and hence the n = 0 term contributes nothing. On the left hand side, the (0) n = 0 term is pi,i = 1 and hence we must adjust the left hand side by 1 if we include the n = 0 term. Doing so we get ∞ X

(n) pi,i z n

−1 =

n=0

n ∞ X X

(n−m) (m) n fi,i z .

pi,i

n=0 m=0

Changing the order of sums on the right hand side produces ∞ X

∞ ∞ X X

(n)

pi,i z n − 1 =

(n−m) (m) n fi,i z

pi,i

m=0 n=m ∞ ∞ X (m) m X (n−m) n−m fi,i z pi,i z n=m m=0 ∞ X (m) fi,i z m Pi,i (z) m=0

n=0

= Pi,i (z) − 1 =

= Fi,i (z)Pi,i (z).

Problem 9.20 Start with the relationship (n)

pi,j =

n X

(m) (n−m)

fi,j pj,j

,

n = 1, 2, 3, . . . .

m=0 (0)

(0)

Taking fi,j = 0 and pi,j = 0 (for i 6= j), the above relationship holds for n = 0 as well. Multiplying by z n and summing results in ∞ X n=0

(n)

pi,j z n =

n ∞ X X n=0 m=0

164 12

(m) (n−m) n

fi,j pj,j

z .

Exchanging order of sums on the right hand side and also writing z n = z m z n−m results in ∞ X

pi,j z n =

∞ X

Pi,j (z) =

m=0 ∞ X

(n)

n=0

∞ X

(m)

fi,j z m

(n−m) n−m

pj,j

z

n=m (m)

fi,j z m Pj,j (z)

m=0

Pi,j (z) = Fi,j (z)Pj,j (z).

Problem 9.21 Suppose i ↔ j such that i can be reached from j in m steps and j can be (m) (k) reached from i in k steps. That is pj,i > 0 and pi,j > 0. Then, (n)

≥ pj,i pi,i

(n)

≥

pj,j ⇒

∞ X

pj,j

n=1

(m) (n−m−k) (k) pi,j ∞ (m) (k) X (n−m−k) pj,i pi,j pi,i n=1

If i is recurrent, then ∞ X

(n−m−k)

pi,i

∞ X

=

n

(n)

pi,i = ∞

n

⇒

∞ X

(n)

pj,j

= ∞.

n=1

Therefore state j is recurrent as well. Alternatively, suppose state j is tranP (n) sient ⇒ ∞ n=1 pj,j < ∞, then ∞ X

(n−m−k) pi,i

≤

n=1

P∞

(n) n=1 pj,j (m) (k) pj,i pi,j

< ∞.

Therfore state i is also transient.

Problem 9.22 (a) pi,j =

(

p 1−p 0

165 13

j = i+1 j=0 otherwise



1−p  1 −p P= 1 −p  .. .



p 0 0 ··· 0 p 0 ···  . 0 0 p ···  .. .. .. . . . . . .

(b) (n)

f0,0 = Pr(n − 1 successes followed by 1 failure) = pn−1 (1 − p). (c) f0,0 =

∞ X

(n)

f0,0 =

n=1

∞ X

pn−1 (1 − p) =

n=1

1−p = 1. 1−p

Therefore state 0 is recurrent.

Problem 9.23 We start by noting that the n-step transition probabilities are given by (n) pi,i

=

(

(1 − p)pi n = i + 1, i + 2, . . . , 1 n = 0, 0 n = 1, 2, . . . , i.

From this we can determine the generating function Pi,i (z) =

∞ X

(n)

pi,i z n = 1+

n=0

∞ X

pi (1−p)z n = 1+

n=i+1

1 − z + (1 − p)pi z i+1 (1 − p)pi z i+1 = . 1−z 1−z

Using the relation of (9.22) we get Fi,i (z) = 1 −

1−z 1 (1 − p)pi z i+1 =1− = . Pi,i (z) 1 − z + (1 − p)pi z i+1 1 − z + (1 − p)pi z i+1

The mean time to first return is then

(1 − p)pi z i+1 dfi,i (z) d 1 µi = . = = i i i+1 dz z=1 dz 1 − z + (1 − p)p z z=1 p (1 − p)

From this the steady state distribution is obtained: πi =

1 = pi (1 − p). µi

166 14

Problem 9.24 Starting with (9.33) pi,j (t + s) =

X

pi,k (t)pk,j (s).

k

Letting s = t and t = ∆t produces pi,j (t + ∆t) =

X

pi,k (∆t)pk,j (t).

k

Applying (9.34) results in pi,j (t + ∆t) = λi ∆tpi+1,j (t) + (1 − (λi + µi )∆t)pi,j (t) + µi ∆tpi−1,j (t) + o(∆t) pi,j (t + ∆t) − pi,j (t) o(∆t) = λi pi+1,j (t) − (λi + µi )pi,j (t) + µi pi−1,j (t) + . ⇒ ∆t ∆t Passing to the limit as ∆t → 0 gives the desired result dpi,j (t) = λi pi+1,j (t) − (λi + µi )pi,j (t) + µi pi−1,j (t). dt

Problem 9.25 (a) Given Φ(ω, t) = Z

∞ −∞

R∞

−∞

f (x, t)ejωxdx, then using integration by parts ∞

∂ f (x, t)ejωxdx = ejωx f (x, t) − jω ∂x −∞

Similarly

Z

Z

∞

f (x, t)ejωxdx = −jωΦ(ω, t).

−∞

∂2 f (x, t)ejωxdx = (−jω)2Φ(ω, t). 2 ∂x −∞ Now, multiply both sides of (9.63) by ejωx and integrate with respect to x over (−∞, ∞) to produce Z

∞ −∞

∞

∂ f (x, t)ejωxdx == −2c ∂t

Z

∞ −∞

∂ f (x, t)ejωxdx + D ∂x

Z

∞ −∞

∂2 f (x, t)ejωxdx. ∂x2

Using the above identities, this can be written in terms of Φ(ω, t) as ∂ Φ(ω, t) = (2cjω − Dω 2 )Φ(ω, t). ∂t

167 15

Given that X(0) = 0, then Φ(ω, 0) = E[ejωX(0)] = 1. (b) Since this is a first order constant coefficient differential equation, the solution is of the form Φ(ω, t) = k exp((2cjω − Dω 2 )t). Applying the initial condition, we determine that k = 1 so that Φ(ω, t) = exp((2cjω − Dω 2 )t). (c) The characteristic function found in part (b) is that of a Gaussian random variable with mean µ = 2ct and variance σ 2 = 2Dt (see Example 4.20 for the characteristic function of a Gaussian random variable). Hence, 1 (x − 2ct)2 f (x, t) = √ ). exp(− 4Dt 4πDt

Problem 9.26 (a) Let N = number of transmissions. Pr(N = n) = q n−1 (1 − q)n = 1, 2, 3, . . . E[N ] =

∞ X

nq n−1 (1 − q) =

n=1

1 . 1−q

(b) T = (Tt + Ta)N − Ta E[T ] = (Tt + Ta )E[N ] − Ta =

Tt + Ta − Ta . 1−q

(c) T = (Tt + Ta)(N1 + N2 ) − 2Ta E[T ] = (Tt + Ta )(E[N1] + E[N2]) − 2Ta =

168 16

2(Tt + Ta ) − 2Ta . 1−q


Problem 10.1 RX,X (t, t + τ ) = E[X(t)X(t + τ )] = b2 E[cos(2πΨt + Θ) cos(2πΨ(t + τ ) + Θ)] b2 b2 E[cos(2πΨτ )] + E[cos(2πΨ(2t + τ ) + 2Θ)] = 2 2 b2 b2 E[exp(j2πΨτ )] + E[exp(−j2πΨτ )] = 4 4 b2 b2 ΦΨ (2πτ ) + ΦΨ (−2πτ ) RX,X (τ ) = 4 4 b2 b2 F T [ΦΨ(2πτ )] + F T [ΦΨ(−2πτ )] SX,X (f ) = 4 4 where F T refers to the Fourier Transform operator. Note that ΦΨ (2πτ ) =

Z

∞ −∞

fΨ (φ)ej2πφτ dφ = F T −1[fΨ (φ)].

Therefore, F T [ΦΨ(2πτ )] = F T [F T −1[fΨ (φ)]] = fΨ (f ). Likewise, F T [ΦΨ(−2πτ )] = fΨ (−f ). Thus, SX,X (f ) =

b2 b2 fΨ (f ) + fΨ (−f ). 4 4

Hence for any valid PSD, S(f ), we can construct a process X(t) = b cos(2πΨt+ Θ) which will have a PSD equal to S(f ) by choosing Ψ to have a PDF whose even part is proportional to S(f ). The constant b is adjusted to make the total power match that specified by S(f ).

Problem 10.2 X(t) = A cos(ωt) + B sin(ωt) 1 169

E[X(t)] = E[A] cos(ωt) + E[B] sin(ωt) = 0 RX,X (t, t + τ ) = E[(A cos(ωt) + B sin(ωt))(A cos(ω(t + τ )) + B sin(ω(t + τ )))] = E[A2] cos(ωt) cos(ω(t + τ )) + E[B 2] sin(ωt) sin(ω(t + τ )) Since A and B are identically distributed, E[A2] = E[B 2] = σ 2, so that RX,X (t, t + τ ) = σ 2 cos(ωτ ). Therefore, X(t) is WSS. E[X 3(t)] = E[(A cos(ωt) + B sin(ωt))3] = E[A3] cos3 (ωt) + E[B 3] sin3(ωt) = E[A3]{cos3 (ωt) + sin3 (ωt)}. Since cos3 (ωt) + sin3 (ωt) is not constant, the process will not be strictly stationary for any randomvariableA such that E[A3] 6= 0.

Problem 10.3 (a) RX,X (t, t + τ ) = E[( =

N X

an cos(ωn t + θn ))(

n=1 N N X X

N X

am cos(ωm (t + τ ) + θm ))]

m=1

an am E[cos(ωn t + θn ) cos(ωm (t + τ ) + θm )]

n=1 m=1

The expected value in the above expression is zero for all m 6= n. Therefore RX,X (t, t + τ ) = RX,X (τ ) =

N X

a2n E[cos(ωn t + θn ) cos(ωn (t + τ ) + θn )]

n=1 N X

1 a2 cos(ωn τ ). 2 n=1 n

(b) SX,X (f ) = F T [RX,X (τ )] =

N 1X a2 {δ(f − fn ) + δ(f + fn )}. 4 n=1 n

2 170

Problem 10.4 (a) RX,X (t, t + τ ) = E[(

∞ X

An cos(nωt) + Bn sin(nωt))

n=1 ∞ X

Am m=1 ∞ ∞ X X

(

cos(mω(t + τ )) + Bm sin(mω(t + τ )))]

{E[An Am] cos(nωt) cos(mω(t + τ ))

=

n=1 m=1

+ E[An Bm ] cos(nωt) sin(mω(t + τ )) + E[Bn Am ] sin(nωt) cos(mω(t + τ ) + E[Bn Bm ] sin(nωt) sin(mω(t + τ )} ∞ X

=

{E[A2n ] cos(nωt) cos(nω(t + τ )) + E[Bn2 ] sin(nωt) sin(nω(t + τ ))}

n=1

(b) If

E[A2n ]

= E[Bn2 ] = σn2 , then the above simplifies to RX,X (τ ) =

∞ X

σn2 cos(nωt),

n=1

and the process is WSS.

Problem 10.5 SX,X (f ) = F T [RX,X (τ )] = F T [1] = δ(f ). That is, all power in the process is at d.c.

Problem 10.6 (a) RY,Y (t, t + τ ) = E[Y (t)Y (t + τ )] = E[X 2 (t)X 2 (t + τ )] = E[X 2(t)]E[X 2(t + τ )] + 2E[X(t)X(t + τ )]2 = R2X,X (0) + 2R2X,X (τ ) SY,Y (f ) = R2X,X (0)δ(f ) + SX,X (f ) ∗ SX,X (f ) =

Z

∞ −∞

SX,X (f )df 3 171

2

(0)δ(f ) + SX,X (f ) ∗ SX,X (f )

(b) Z

RX,X (0) =

∞ −∞

SX,X (f )df = 2B. !

f SX,X (f ) ∗ SX,X (f ) = 2Btri . 2B

!

f SY,Y (f ) = 4B 2 δ(f ) + 4Btri . 2B (c) 2 E[Y (t)] = E[X 2 (t)] = σX = constant 2 2 RY,Y (t, t + τ ) = RX,X (0) + 2RX,X (τ )

Since E[Y (t)] is constant and RY,Y (t, t+τ ) is a function of τ only, the process is WSS.

Problem 10.7 RX,X (t, t + τ ) = E[b2 cos(ωt + Θ) cos(ω(t + τ ) + Θ)] b2 b2 = cos(ωτ ) + E[cos(ω(2t + τ ) + 2Θ)] 2 2 b2 RX,X (τ ) = hRX,X (t, t + τ )i = cos(ωτ ) 2 2 2 b b δ(f 0 − f ) + δ(f 0 + f ) SX,X (f 0 ) = 4 4 This PSD is independent of the distribution of Θ. This is expected because the process has all its power at frequency, f , regardless of the phase Θ.

Problem 10.8 Let s(t) =

∞ X

sk exp(j2πfo t).

k=−∞

4 172

Then ∞ X

s(t − T ) =

sk exp(j2πfo (t − T ))

k=−∞

RX,X (τ ) = E

"

XX k

=

XX k

sk s∗m

exp(j2πkfo (t − T ) exp(−j2πmfo (t + τ − T )

m

sk s∗m exp(j2π(k − m)fo t) exp(−j2πmfoτ )

m

E[exp(j2π(k − m)fo T )] Z 1 to E[exp(j2π(k − m)foT )] = exp(j2π(k − m)fo u)du t 0 (o 0 k 6= m = 1 k=m RX,X (τ ) =

∞ X

SX,X (f ) =

k=−∞ ∞ X

|sk |2 exp(−j2πkfoτ ) |sk |2 δ(f − kfo )

k=−∞

Hence the process X(t) = s(T − T ) has a line spectrum and the height of each line is given by the magnitude squared of the Fourier Series coefficients.

Problem 10.9 Write Y (t) = b cos(ωo t + Ωt + Θ) where Ω = 2πfo V/c and ωo = 2πfo . Note that Ω is uniformly distributed over (ωo νo /c, −ωo νo /c). For simplicity, define zo = ωo νo /c. RY,Y (t, t + τ ) = b2 E[cos((ωo + Ω)t + Θ) cos((ωo + Ω)(t + τ ) + Θ)] b2 b2 E[cos((ωo + Ω)τ )] + E[cos((ωo + Ω)(2t + τ ) + 2Θ)] = 2 2 Assuming Θ is uniform over [0, 2π) the second expectation is zero. Hence Z

zo b2 1 RY,Y (τ ) = cos(ωo + ω)τ )dω 2 2zo −zo b2 [sin((zo + ωo )τ ) + sin((zo − ωo )τ )] = 4ωo τ

5 173

#

b2νo sin(ωo τ ) cos(ωo τ ) c ωo τ ( ! !) b2 π(f − fo ) π(f + fo ) SY,Y (f ) = rect + rect 4fo zo zo =

The power of the signal is now spread over a range of frequencies around ±fo .

Problem 10.10 (a)

|k|

1 1 |k| + RZ,Z [k] = RX,X [k] + RY,Y [k] = 2 3 (see Exercise 8.18 for details) (b) For a funcion of the form R[k] = p|k| , the Fourier Transform is (to is the time between samples of the discrete time process) S(f ) =

X

R[k]e−j2πkf to

k

= 1+

∞ X

pk {e−j2πkf to + ej2πkf to }

k=1

pej2πkf to pe−j2πkf to + 1 − pe−j2πkf to 1 − pej2πkf to 1 − p2 = 1 + p2 − 2p cos(2πfto )

= 1+

Therefore, 3/4 5/4 − cos(2πfto ) 8/9 SY,Y (f ) = 10/9 − (2/3) cos(2πfto ) SZ,Z (f ) = SX,X (f ) + SY,Y (f ).

SX,X (f ) =

Problem 10.11 For a discrete time random process: S(f ) = 6 174

P∞

k=−∞

R[k]e−j2πkf to .

The inverse relationship is: R[k] = to

R 2t1o

− 2t1

S(f )ej2πkf to df .

o

The average power in the process is: 1 1 = E[X 2 [k]] = RX,X [0] = to to

Pavg

Z

1 2to

− 2t1

S(f )df.

o

Problem 10.12 RX,X (t1 , t2) = E[cos(ωc t1 + B[n1]π/2) cos(ωc t2 + B[n2]π/2)], where n1 and n2 are integers such that n1 T ≤ t1 < (n1 + 1)T and n2 T ≤ t2 < (n2 + 1)T /. For t1 , t2 such that n1 6= n2 , RX,X (t1 , t2) = E[cos(ωc t1 + B[n1]π/2)]E[cos(ωc t2 + B[n2]π/2)] = 0, while for t1, t2 such that n1 = n2 , 1 cos(ωc (t2 − t1)) + 2 1 = cos(ωc (t2 − t1)) − 2

RX,X (t1, t2) =

1 E[cos(ωc (t2 + t1) + πB[n1])] 2 1 cos(ωc (t2 + t1)) 2

Since this autocorrelation depends on more than just t1 − t2, the process is not WSS. (b) From part (a), RX,X (t, t+τ ) =

(

0 1 2

if t, t + τ are in different intervals, cos(ωc τ ) − 12 cos(ωc (2t + τ )) if t, t + τ are in in the same intervals.

Since the process is not WSS we must take time averages. 1 1 RX,X (τ ) = hRX,X (t, t+τ )i = (1−p(τ ))h0i+p(τ )h cos(ωc τ )− cos(ωc (2t+τ ))i, 2 2 where p(τ ) is the fraction of the values of t that lead to t and t + τ being in the same interval. This function is given by p(τ ) =

(

0 1−

|τ | T

7 175

|τ | > T, |τ | < T.

Therefore, p(τ ) = tri(t/T ) 1 RX,X (τ ) = tri(τ /T ) cos(ωc τ ) 2 1 F T [tri(τ /T )] ∗ F T [cos(ωc τ ] SX,X (f ) = 2 using Table E.1 in Appendix E in the text, 1 T sinc2 (fT ) ∗ (δ(f − fc ) + δ(f + fc )) 4 T = (sinc2 ((f − fc )T )) + sinc2 ((f + fc )T )) 4

SX,X (f ) =

Problem 10.13 2 Brms

Recall that R(τ ) = Also note that

R∞

−∞

R∞

f 2 S(f )df −∞ S(f )df

R∞ = −∞

S(f )ej2πf τ df . Thus the denominator is simply R(0).

d2 R(τ ) = (j2πf )2 dτ 2 Therefore, the numerator is

Z

Z

∞

∞

1 f S(f )df = − 2π −∞ 2

Therefore, 2 Brms

S(f )ej2πf τ df.

−∞

2

d2 R(τ ) . dτ 2 τ =0

d2 R(τ ) 1 =− . (2π)2R(0) dτ 2 τ =0

Problem 10.14 (a) The absolute BW is ∞ since S(f ) > 0 for all |f | < ∞. (b) The 3dB BW, f3 satisfies 1 1 = 2 3 (1 + (f3/B) ) 2

q

⇒ f3 = B 21/3 − 1 = 0.5098B. 8 176

(c) Z

∞

2

f S(f )df = −∞

Z

∞

S(f )df = −∞ 2 Brms =

Brms =

Z

Z

∞ π f2 z2 3 df = B dz = B 3 2 3 2 3 8 −∞ (1 + (f/B) ) −∞ (1 + z ) Z ∞ Z ∞ 3π 1 1 B df = B dz = 2 3 2 3 8 −∞ (1 + (f/B) ) −∞ (1 + z ) π 3 B B2 8 = 3π B 3 8 B √ . 3 ∞

Problem 10.15 (a) The absolute BW is ∞ since S(f ) > 0 for all |f | √ < ∞. (b) The peak value of the PSD occurs at f = B/ 2 and has a value of Smax = 4/27. Next we seek the values of f which satisfy 1 (f /B)2 Smax = 2 3 (1 + (f/B) ) 2 2 (1 + (f/B)2 )3. ⇒ (f/B)2 = 27 q √ √ The two solutions are f1 = 3 23−5 B and f2 = 2B. The 3dB bandwidth is then s √   √ 3 3 − 5 f3 = f2 − f1 =  2 − B = 1.1010B. 2 (c) Since this is a bandpass process, the definition of (10.24) in the text is used. First we must find R∞ fS(f )df fo = R0 ∞ 0 S(f )df Z

Z

Z

∞ 1 2 f (f/B)2 z3 2 B df = B dz = (1 + (f/B)2 )3 (1 + z 2 )3 4 0 0 0 Z ∞ Z ∞ Z ∞ π (f/B)2 z2 S(f )df = df = B dz = B 2 3 2 3 (1 + (f/B) ) (1 + z ) 16 0 0 0 1 2 B 4 fo = 4π = B. B π 16 ∞

fS(f )df =

∞

9 177

Next, we find the RMS bandwidth according to R∞

(f − fo )2 S(f )df R∞ 0 S(f )df Z ∞ Z ∞ (z − 4/π)2 z 2 (f − fo )2 S(f )df = B 3 dz = 0.2707B 3 (1 + z 2 )3 0 0 4 · 0.2707B 3 2 = 5.5154B 2 Brms = π/16 · B Brms = 2.3485B. 2 Brms

4

=

0

Problem 10.16 RY,Y (t, t + τ ) = E[X(t)X(t + τ )]E[cos(ωo t + Θ) cos(ωo (t + τ ) + Θ)] 1 RY,Y (τ ) = RX,X (τ ) · cos(ωo τ ) 2 1 1 1 SY,Y (f ) = SX,X (f ){ δ(f − fo ) + δ(f + fo )} 2 2 2 1 1 = SX,X (f − fo ) + SX,X (f + fo ) 4 4 0.5

PSD for B=1, f =5 o

0.4 SY,Y(f)

0.3 0.2 0.1 0 −0.1

−10

−5

0

5

10

f

Figure 1: PSD for Problem 10.16; B = 1, fo = 5.

Problem 10.17 1 X[n] = X[n − 1] + E[n]. 2 10 178

(1)

Taking expectations of both sides of (1) results in 1 µ[n] = µ[n − 1], 2

n = 1, 2, 3, . . . .

Hence µ[n] = (1/2)n µ[0]. Noting that X(0) = 0, then µ[0] = 0 ⇒ µ[n] = 0. Multiply both sides of (1) by X[k] and then take expected values to produce 1 E[X[k]X[n]] = E[X[k]X[n − 1]] + E[X[k]E[n]]. 2 Assuming k < n, X[k] and E[n] are independent. Thus, E[X[k]E[n]] = 0 and therefore 1 RX,X [k, n − 1]. 2 n−k 1 RX,X [k, k], ⇒ RX,X [k, n] = 2 RX,X [k, n] =

n = k, k + 1, k + 2, . . . .

Following a similar procedure, it can be shown that if k > n RX,X [k, n] =

k−n

1 2

RX,X [k, k].

Hence in general RX,X [k, n] =

|n−k|

1 2

RX,X [m, m], where m = min(n, k).

Note that RX,X [m, m] can be found as follows: 1 RX,X [m, m] = E[X 2 [m]] = E[( X[m − 1] + E[m])2] 2 1 RX,X [m − 1, m − 1] + E[X[m − 1]E[m]] + E[E 2[m]]. = 4 Since X[m − 1] and E[m] are uncorrelated, we have the following recursion 1 RX,X [m − 1, m − 1] + σE2 4 m m−1 X 1 i 1 2 ⇒ RX,X [m, m] = RX,X [0, 0] + σE . 4 4 i=0 RX,X [m, m] =

11 179

Note that since X(0) = 0, RX,X (0, 0) = 0. Therefore 4σ 2 1 − (1/4)m = E (1 − (1/4)m ) 1 − 1/4 3 |n−k| 2 4σE 1 (1 − (1/4)m ) . ⇒ RX,X [k, n] = 3 2 RX,X [m, m] = σE2

Since m =min(n, k) is not a function of n − k, the process is not WSS.

Problem 10.18 (a) Given Y [n] = a1 Y [n − 1] + a2 Y [n − 2] + X[n],

(2)

then multiplying (2) by Y [n − k] and taking expectations produces E[Y [n]Y [n − k]] = a1E[Y [n − 1]Y [n − k]] + a2E[Y [n − 2]Y [n − k]] + E[X[n]Y [n − k]].

(3)

Note that (for k > 0) X[n] and Y [n − k] are independent since Y [n − k] depends on X[n − k], x[n − k − 1], . . . , but not on X[n]. Hence, E[X[n]Y [n − k]] = E[X[n]]E[Y [n − k]] = 0. Then (3) becomes RY,Y [k] = a1 RY,Y [k − 1] + a2RY,Y [k − 2]. (b) Squaring both sides of (2) and taking expectations gives E[Y 2 [n]] = E[(a1Y [n − 1] + a2Y [n − 2] + X[n])2] RY,Y [0] = (a21 + a22)RY,Y [0] + 2a1 a2RY,Y [1] + RX,X [0] 2 = (1 − a21 − a22)RY,Y [0] − 2a1 a2RY,Y [1]. ⇒ σX From (4) with k = 1, we get RY,Y [1] = a1RY,Y [0] + a2RY,Y [−1]. Since RY,Y [1] = RY,Y [−1] we have (1 − a2 )RY,Y [1] − a1RY,Y [0] = 0. 12 180

(4)

Thus we have the following 2x2 linear equations:

1 − a21 − a22 −a1

2 RY,Y [0] σX = 0 RY,Y [1] 2 σX 1 − a2 RY,Y [0] ⇒ = a1 RY,Y [1] ∆

2a1a2 1 − a2

where ∆ = (1 − a2)(1 − a21 − a22). This then provides the initial conditions for the difference equation in (4). (c) The general solution of (4) will be of the form RY,Y [k] =

|k| k1 b1

+

|k| k2 b2 ,

where b1 , b2 =

a1 ±

q

a21 + 4a2 2

The constants k1 and k2 are found using the initial conditions: 2 σX (1 − a2) ∆ σ2 RZ,Z [1] = k1 b1 + k2 b2 = X a1 ∆ 2 σX 1 − a2 1 k1 = b2 k2 a1 ∆ 1 k b2 (1 − a2) − a1 ⇒ 1 = . k2 ∆(b2 − b1) −b1 (1 − a2) + a1

RZ,Z [0] = k1 + k2 =

⇒

1 b1

(d) Taking the DFT of (5) gives k2 (1 − b22) k1 (1 − b21) SY,Y (f ) = + . 1 + b21 − 2b1 cos(2πf ) 1 + b22 − 2b2 cos(2πf )

Problem 10.19 (a) E[2] = E[(Y [n + 1] − a1Y [n] − a2 Y [n − 1])2 ] = RY,Y [0](1 + a21 + a22 ) − 2a1(1 − a2)RY,Y [1] − 2a2 RY,Y [2] (b) ∂E[2] ∂a1

= 2a1 RY,Y [0] − 2(1 − a2)RY,Y [1] = 0 13 181

(5)

⇒ RY,Y [0]a1 + RY,Y [1]a2 = RY,Y [1] 2

∂E[ ] ∂a2

= 2a2 RY,Y [0] − 2RY,Y [2] + 2a1 RY,Y [1] = 0

⇒ RY,Y [1]a1 + RY,Y [0]a2 = RY,Y [2] RY,Y [0]RY,Y [1] a1] RY,Y [1] = ⇒ RY,Y [0] RY,Y [1] a2 RY,Y [2] 1 a] RY,Y [0]RY,Y [1] − RY,Y [1]RY,Y [2] ⇒ 1 = a2 RY,Y [0]RY,Y [2] − R2Y,Y [1] R2Y,Y [0] − R2Y,Y [1]

Problem 10.20 (a) 2

E[ ] = E[(Y [n + 1] −

p X

ak Y [n − k + 1])2]

k=1

= E[Y 2 [n + 1]] − 2

p X

ak E[Y [n + 1]Y [n + 1 − k]]

k=1

+

p p X X

ak am E[Y [n + 1 − k]Y [n + 1 − m]]

k=1 m=1

= RY,Y [0] − 2

p X

ak RY,Y [k] +

p X p X

ak am RY,Y [m − k]

k=1 m=1

k=1

To simplify the notation, introduce the following vectors and matrices: r = [ RY,Y [1] RY,Y [2] . . . RY,Y [p] ]T , a = [ a1 a2 . . . ap ]T , R = p x p matrix whose (k, m)th element is RY,Y [m − k]. Then the mean squared error is E[2] = RY,Y [0] − 2rT a + aT Ra. (b) ∇a = −2r + 2Ra = 0 ⇒ a = R−1r

14 182

Problem 10.21 See solutions to Problem 10.18

Problem 10.22

SN,N (f ) = ez

ktk z , 2 ez − 1

where z =

h|f | ktk

z z = 2 −1 (1 + z + z /2 + z 3/3! + . . .) − 1 1 = . 1 + z/2 + z 2 /6 + . . .

As |f | → 0, z → 0. Clearly, as z → 0, ezz−1 → 1 so that lim sN,N (f ) =

|f |→0

No ktk = . 2 2

Problem 10.23 Noting that V = V1 + V2 , 2 2 2 = E[V 2] = E[(V1 + V2 )2 ] = E[V12 ] + E[V22 ] + 2E[V1 V2 ] = V1,rms + V2,rms Vrms

⇒ Vrms = 4kte (r1 + r2 )∆f =

q

2 2 V1,rms + V2,rms

q

(4kt1 r1 ∆f )2 + (4kt2 r2 ∆f )2 q

= 4k∆f (t1r1 )2 + (t2 r2)2 q

te =

(t1 r1)2 + (t2r2 )2 r1 + r2

Note if t1 = t2 = to , then the effective temperature is q

te = to

r12 + r22 r1 + r2

which in general is not equal to the physical temperature (unless r1 = r2 ).

15 183

Problem 10.24 In the case of parallel resistors, V =

V1 V2 + r1 r2

r1 r2 . r1 + r2

Let r be the paraalel combination of the resistances (i.e., r = 2 Vrms

(4kte r∆f )2 ⇒ t2e

!

E[V12] E[V22 ] 2 = E[V ] = + r = r12 r22 = ((4kt1 ∆f )2 + (4kt1 ∆f )2)r2 = t21 + t22

te =

2

r1 r2 ). r1 +r2

Then !

2 2 V2,rms V1,rms + r2 2 2 r1 r2

q

t21 + t22 .

In this case, the effective temperature is the same as the physical temperature if t1 = t2.

16 184

Solution scott l. miller, donald g. childers probability and random processes

Recommend Documents