M3S11: Games, Risks and Decisions

Games,
Risks,
and
Decisions
-
M4S11 Taught
by
Lynda
White
-
[email protected]. Autumn
2014

Contents 0 Introduction 0.1 Game Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 0.2 Utility Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 0.3 Decision Theory . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

2 2 2 3

1 Game Theory 1.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.2 Two-Person Zero-Sum Games . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1.3 Non-Zero-Sum Games . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

3 3 4 38

2 Utility 2.1 Introduction . . . . . . . . 2.2 The Lottery Axioms . . . 2.3 The Existence of a Unique 2.4 The Utility of Money . . .

51 51 51 53 54

. . . . . . . . . . . . . . . . . . . . Utility Function . . . . . . . . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

3 Bayesian Methods

55

4 Decision Theory 4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.2 Decision Rules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4.3 Decision Trees . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

57 57 58 59

5 Mastery Exam Component

63

6 Appendix: Further Material of Use 6.1 On Risk and Loss Functions . . . . 6.2 Mid-Term 1 - 13/11/14 . . . . . . 6.3 Mid-Term 2 - 11/12/14 . . . . . . 6.4 Further Problem Sheet Questions .

67 67 69 70 72

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

Distribution of this1document is illegal

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

. . . .

0
Introduction 0.1
Game
Theory Started
in
the
1920’s.
Borel
1921,
Von
Neumann
1928.
Developed
in
WWII
for
logistics,
submarine
search,
and air
defence.
1944
gave
us
von
Neumann
and
Morgenstern’s
“The
Theory
of
Games
and
Economic
Behaviour”. Later
on
John
Nash
[film:
A
Beautiful
Mind]. 0.1.1
Aim The
aim
of
Game
Theory
is
to
develop
optimal
strategies
in
competitive
situations
involving
two
or
more intelligent
antagonists.
Key
points: • Conflicting interests • Playing rationally Areas of application: 1. Games: • Chess: 2 players, no random element, perfect information • Monopoly:

2 players, random element, perfect information

• Bridge: 2 against 2, random element, non-perfect information Can you think of a game with no random element and “not perfect information”? 2. Military / political situations, countries with conflicting interests. Note that coalitions can form 3. Biology: Competing species. Evolutionary stable strategies [i.e. balance of species in an environment such that none has an incentive to change its behaviour] 4. Economics: Companies compete for the market share in a product. Auctions, how much would you bid? We will only look at two-person games. [MSci students will consider n-person games.] There are two types: 1. Strictly competitive games where the players act independently 2. Games in which players may cooperate to their mutual advantage

0.2

Utility Theory

We will need to assess the value of the outcomes of any decisions we make. Not all outcomes are monetary. We need a common scale [of value] to compare the values of, say, a kidney machine and an incubator in a hospital. We will develop a mathematical theory based on axioms representing how a reasonable person makes decisions. This leads to a utility function, a measure of value. Risk = E[loss] =

E[utility]

We will look at risk aversion. How risk averse are you? Consider choosing one of the following: • £1 guaranteed ( £2 if a fair coin shows H • £0 if a fair coin shows T N.B. these have the same expected outcome. Attitudes change when sums are multiplied by 10, 100, 1000 &c. 2

0.3
Decision
Theory We
make
decisions
all
the
time;
the
crucial
element
is
the
uncertainty.
To
limit
the
uncertainty
we
collect
data. Decision
making
is
like
playing
a
game
with
the
unknown.
We
will
study
Bayesian
inference
to
tell
us
how
to incorporate
this
data
into
the
decision
making
process
in
order
to
maximise
expected
utility. Exempli
gratia:
Marriage
problem.
We
neither
want
to
choose
too
soon
nor
too
late.
Whom
can
you
choose as
your
long-term
partner?
Roughly,
look
around
for
about
1
3 of
the
time
and
then
choose
the
first
one
after that who is better than all those seen so far. In fact, it’s not actually 13 but 1e . [End lecture 1, 09/10/14]

1

Game Theory

1.1

Introduction

We assume 2 players, A and B, both are rational and greedy! Each player has available a number of strategies, or “recipes”, for playing the game. A strategy for a player [sometimes called a pure strategy] involves a complete description of all the moves that the player will make, including responses to the opponent’s moves and any random moves [e.g. using dice or coins]. I.e. a strategy is a program that can be followed mechanically by the player. It usually comprises a sequence of moves, some of which may be random. Examples: 1. Two companies, A and B. Each produces a product. To capture more of the market each will decide to do one of three things: • Spend 10% extra on advertising • Reduce the product price • Give away a free product

These are the 3 pure strategies for each player 2. A fair coin is tossed. The result is shown to A and B. Each then has a choice of n possible moves, {m1 , . . . , mn }. Consider player A. A strategy for A is a map: {H, T } ! {m1 , . . . , mn }. An example of a strategy would be: if the coin shows H, play move m4 ; if the coin shows T , play move m6 . H and T can both be mapped to any of the n moves, so A has n2 strategies. N.B. for each player there are 2n outcomes. These are not strategies 3. GOPS [game of pure strategy]. Each player has 10 cards numbered 1, 2, . . . , 10. The dealer has a similar set of cards and deals them onto the table one by one, at random, so both players can see them [one by one]. When a card is dealt on the table in random order, so both players can see, the players then simultaneously bid for each card, one by one. The higher bid wins the dealt card. The cards that have been used for bidding are discarded. If both players play the same card, the dealt card is discarded. The winner is the one with the highest total score on the dealt cards they have won. Let’s think about GOPS with 2 cards: A strategy for A tells A which card [1 or 2 to play in response to the first dealt card. The rest of the game is then determined. A strategy is a map: {dealt cards} ! {A’s first card} i.e. a map {1, 2} ! {1, 2}. The number of strategies is thus = 22 = 4. Question: How many strategies does each player have in a game of GOPS with 3 cards each? Players may have a finite or infinite number of pure strategies. Here’s an example where players have an infinite number of strategies: ( A chooses a real number, a 2 [0, 1] B chooses a real number, b 2 [0, 1] A wins [and B loses] an amount |a

b|.

3

In
general,
if
A
plays
a
and
B
plays
b
we
write
down
the
ordered
pair
(expected
gain
to
A,
expected
gain
to
B). N.B.
“expected”
is
to
take
account
of
possible
random
moves.
Write
this
as
(gA (a,
b),
gB (a,
b)).
We
call
gA (a,
b)
and
gB (a,
b)
the
gains
or
pay-o↵s
to
A
and
B
respectively
when
A,
B
play
a,
b
respectively. If
gain
to
A
+
gain
to
B
=
0
8
a,
b
we
call
the
game
zero-sum.
In
such
a
game,
whatever
one
player
wins
the other
loses.
The
game
is
strictly
competitive;
there’s
no
point
in
collaborating.
If
gain
to
A
+
gain
to
B
=
C [=
constant,
6=
0]
then
we
make
this
into
a
zero-sum
game: ( c ⇤ (a, b) = g (a, b) gA A ⇤ ⇤ 2 Let then gA (a, b) + gB (a, b) = 0 8 a, b c ⇤ gB (a, b) = gB (a, b) 2 If the gain to A + the gain to B 6= constant, i.e. in a non-constant-sum game, there are two possibilities: 1. The players can’t collaborate, there’s no pre-play communication. E.g. Prisoners’ Dilemma 2. The players can collaborate for their mutual advantage. This is sometimes called bargaining Listing the strategies for each player and forming a table of pay-o↵s [or gains] is called the normal form of the game. Another form of the game, the extensive form, describes the game as a tree. E.g. Nim, Piles of stones. Players, in turn, take stones from one pile only. The aim is to take the last stone. The two approaches are equivalent with strategies being described using paths through the tree in the extensive form, see section 4.4 for an example. We will look only at the normal form. [End lecture 2, 14/10/14]

GOPS with 3 Cards 7,077,888 pure strategies. We need two “sub-strategies” here. A strategy for A must tell her/him: 1. Which card to play in response to the first card dealt 2. Having played her/his first card, which second card to play in response to B’s first card and the second dealt card 1. is determined by a map {1, 2, 3} ! {1, 2, 3} ⌘ {first dealt card} ! {A’s first GOPS pay-o↵}. 9 33 such maps. For each such map, A needs to know how to respond at the next step. I.e. A has 2 choices for each ((i, j), k, l) where: i = first card dealt; j = A’s first card; k = B’s first card; l = second card dealt. For a given map [in 1.] there are 3 possibilities for the [ordered] pair (i, j), 3 for k, and 2 for l. Hence for each map [in 1.] there are 3 ⇥ 3 ⇥ 2 = 18 possibilities for stage 2. We have a map from this set of 18 to 2 choices for A. The total number of strategies for A is 33 ⇥ 218 , as there are 3 maps in 1., 2 choices, and 18 possibilities.

1.2 1.2.1

Two-Person Zero-Sum Games Introduction Let

(

AS = the set of pure strategies for A BS = the set of pure strategies for B

Assume gain to A + gain to B = 0 for all a 2 AS , b 2 BS . AS , BS can be finite or infinite. A and B choose their strategies independently and simultaneously with no collaboration. If A chooses a 2 AS and B chooses b 2 BS let: g(a, b) = gain to A so g(a, b) = gain to B or g(a, b) = loss to B. N.B. g(a, b) could be negative! g(a, b) can be monetary or simply a score, e.g. A wins =) g = +1 and A loses =) g = 1. If AS = {a1 , . . . , am } and BS = {b1 , . . . , bn } are finite, the game is called a finite or matrix game. In this case, we write: gij = g(ai , bj ) 4

Then G = (gij ) is called the pay-o↵ matrix. For analytical reasons, we assume g(a, b) is bounded below. Note the problem of choosing “optimal” strategies is not altered by the following: 1. Adding a constant to each pay-o↵ 2. Multiplying each pay-o↵ by a positive constant Examples: 1. Rock, paper, scissors. Win = +1: lose =

1. Rock beast scissors, paper beats rock, scissors beat paper: R R P S

2

0 41 1

P

S

1 0 1

3 1 15 0

2. Matching pennies. A and B each has a coin and simultaneously each shows a side, H or T , of her coin. If both show the same then B pays A one unit [i.e. A wins, B loses]. Otherwise, B wins one unit from A. a1 = H, a2 = T , b1 = H, b2 = T :  1 1 G= 1 1 3. Matching pennies with imperfect spying. Like matching pennies but after A has made a choice of H or T [without showing it to B] a “spy-coin” is tossed by a third party. The spy is unreliable and: P (spy-coin shows A’s choice) = p P (spy-coin shows opposite of A’s choice) = 1

p

Both A and B know the value of p. The result of the spy coin toss is then shown to B who then makes a choice. A has 2 pure strategies: a1 = H; a2 = T . B has 4 pure strategies: a strategy is a map {spy-coin tosses} ! {H, T }. We can label these strategies: (H, H)

(H, T )

(T, H)

(T, T )

Here, the first entry in each ordered pair tells B how to play if the spy-coin shows heads and the second entry tells B how to play if the spy-coin shows tails. E.g. (H, H) tells B to play H in either case. (H, T ) tells B to copy the spy-coin. Let’s take a look at the pay-o↵ matrix:

A

B

2

3 (H, H) (H, T ) (T, H) (T, T ) 4a1 = H 1 2p 1 1 2p 1 5 a2 = T 1 2p 1 1 2p 1

These are the expected gains to A. For example:

g(a1 , b2 ) ⌘ g(H, (H, T )) = 1 ⇥ P (B chooses H) + ( 1) ⇥ P (B chooses T ) = 1 ⇥ P (spy-coin shows H) + ( 1) ⇥ P (spy-coin shows T ) =p

(1

p) = 2p

1 [since B copies the spy-coin]

Here’s another example from the 2013/14 course: 4. A man, X, arrives in his office at a random time T between 4 : 00 and 5 : 00 p.m. I.e. T ⇠ U (4, 5), a uniform distribution. X will leave the office at 5 : 00 p.m. There are two clients, A and B. Each wants to meet X for dinner that evening but must phone X while X is in the office to make the arrangements. Each client only has time to make one phone call, and whoever calls first, provided X is in the office, will get the dinner appointment. If both call before X arrives in the office, each gets payo↵ = 0. Gain from getting dinner appointment = +1: gain from not getting the dinner appointment = 1. Note that this is a zero-sum game. Let A call at time a, B call at b, and a, b 2 [4, 5], a 6= b. If T = t, A gains: 8 > t < a < b or b < t < a <1 1 t < b < a or a < t < b > : 0 a < t and b < t So:

g(a, b) = ET (gain to A) ( 1 · P (T < a) + ( 1) · P (a < T < b) = 1 · P (b < T < a) + ( 1) · P (T < b) ( 2a b 4 a < b = [N.B. a 6= b] a 2b + 4 a > b 1.2.2

Pure Strategy Saddle-Points

Question: How should A and B play in a two-person zero-sum game?

(

ab

A wants to maximise g(a, b) B wants to minimise g(a, b)

Consider player A and pure strategy a 2 AS . We know that A cannot get less than: inf g(a, b)

b2BS

[Recall that inf = greatest lower bound. In a finite game, this inf is simply the smallest entry in row a.] A cannot guarantee getting more than this. Hence: L

= sup a2AS

⇢

inf g(a, b)

b2BS

is the upper limit to what A can guarantee getting. [sup = least upper bound. In a finite game, maxa minb g(a, b).]

L

=

[End lecture 3, 15/10/14] Similarly, for any b 2 BS , B will not lose more than: sup g(a, b) a2AS

In a finite game, this is the largest entry in column b. B can’t guarantee losing less if they play b. B wants to choose b to minimise this guarantee level, so if: ( ) U

then

U

= inf

b2BS

sup g(a, b)

a2AS

= upper limit of what B can guarantee getting.

Definition:

L

= lower pure value of the game:

U

= upper pure value of the game.

Lemma 1.



L

U

Proof. For any a 2 AS , b 2 BS we have: inf0 g(a, b0 )  g(a, b)  sup g(a0 , b) b

a0

and this yields: ⇢ sup inf0 g(a0 , b0 ) a0

Definition: If

L

=

b

U

⇢  sup g(a , b) 8b =) sup inf0 g(a0 , b0 ) 0

a0

a0

b

⇢

 inf0 sup g(a0 , b0 ) b

a0

then their common value is called the pure value of the game.

Suppose 9(a⇤ , b⇤ ) with a⇤ 2 AS , b⇤ 2 BS such that: ( ⇤ L = supa inf b g(a, b) = inf b g(a , b) 1: ⇤ U = inf b supa g(a, b) = supa g(a, b ) Then, in a finite game, b⇤ minimises the column maxima in the pay-o↵ matrix and a⇤ maximises the row minima [in the pay-o↵ matrix]. If L = U = the pure value of the game, then 1 =) supa g(a, b⇤ ) = inf b g(a⇤ , b). Since g(a⇤ , b⇤ ) lies between the the two values, they are both equal to g(a⇤ , b⇤ ). If, in a finite game, g(a⇤ , b⇤ ) is simultaneously the smallest value in its row and the largest in its column in the pay-o↵ matrix then the game has a pure value = g(a⇤ , b⇤ ). Example: 2

A

col. max.

6 a1 6 4 a2 a3

B

3 b 1 b 2 b3 1 3 27 7 5 4 65 3 2 4

5

4⇤

row min. 1 4⇤ 3

6

Largest row min. = smallest col. max. = 4. Pure value = 4. A should play a2 and B should play b2 . If such a⇤ , b⇤ exist [as in 1] we say that g(a⇤ , b⇤ ) is a pure strategy saddle-point at g(a⇤ , b⇤ ) [Similarly (a⇤ , b⇤ )]. We sometimes say that a⇤ and b⇤ are good strategies for A and B respectively. We use the abbreviation PSSP. If a game has a PSSP [i.e. an element of the pay-o↵ matrix that is the smallest in its row and largest in its column] then we have solved the game. Definition: A pure strategy a 2 AS is said to be dominated by another pure strategy a0 2 AS if: g(a, b)  g(a0 , b) 8b 2 BS [I.e. a0 is always better than a, whatever B plays.] Similarly b 2 BS is dominated by another pure strategy b0 2 BS if g(a, b) g(a, b0 ) 8a 2 AS . Warning: It is possible for a game to have a PSSP at strategy! Exempli gratia: 2 b1 6 a1 3 6 4 a2 3 a3 1

(a⇤ , b⇤ ) but for a⇤ to be dominated by another pure 3 b2 b3 5 47 7 6 55 2 0

3 is a PSSP. However a2 dominates a1 , the PSSP. 3 = (a2 , b1 ) is also a PSSP. Downloaded by:: chrismcleod3 | [email protected] Distribution of this7document is illegal

Exercises 1. Show that if a game has two PSSPs, given by (a, b) and (a0 , b0 ), then g(a, b) = g(a0 , b0 ) 2. Show that if a game has a unique PSSP, given by (a⇤ , b⇤ ), then a⇤ is not dominated by any other a0 2 As with a similar result for B. [Hint: WLOG let (a1 , b1 ) be a unique PSSP in a finite game. Suppose, for a contradiction, that a2 dominates a1 . Show that (a2 , b1 ) also gives a PSSP] If A has a pure strategy, a, that is dominated by a pure strategy a0 2 AS then we delete the row of the pay-o↵ matrix corresponding to a. We may lose a PSSP by deleting a row [or column] like this, but it will not a↵ect the existence [or otherwise] of a PSSP in the game. See 2. above. It is good practice to delete dominated rows and columns and to look for PSSP’s. Not all games have a PSSP. Here’s an example of 2 b1 6 a1 1 6 4 a2 5 a3 3 col. max.

5

a game without a PSSP [with 3 b2 b3 row min. 3 27 1 7 5 4 6 4 7 4 3 7

L

= 4,

U

= 5]:

6

We can delete a1 by comparing it with a2 or with a3 to give:  5 4 6 3 7 4 We can now delete b3 [compare it with b1 ] to give: 2

4 a2 a3

3 b1 b2 5 45 3 7 [End lecture 4, 16/10/14]

The Duellists’ Problem 2 duellists X, Y . Each has a gun with only one bullet. They start at 1 unit distance apart and simultaneously start walking towards each other.

They walk towards each other. Each must decide when [i.e. at what distance] they should fire. Rules: 1. A duellist who hits the other one wins 1 point and the loser loses 1 point 2. If a duellist fires and misses, the other can walk up and shoot at point blank range, therefore winning 3. If both fire at the same time and both hit or both miss, each gets a pay-o↵ of 0 This
is
a
two-person
zero-sum
game.
So
the
pure
strategies
for
X
and
Y
are
XS
=
YS
=
[0,
1].
Let
X
fire
at
a
distance
x
apart:
let
Y
fire
at
a
distance
y
apart.
x,
y
2
[0,
1]. 8

We’ll
assume
P
(X
hits
when
distance
apart
is
x)
=
p1 (x)
and
P
(Y
hits
when
distance
apart
is
y)
=
p2 (y).
Both
X
and
Y
know
the
functions
p1 (x)
and
p2 (y).
p1 (x)
and
p2 (y)
are
monotonic
decreasing
and
continuous. 8 > x>y : Both fire together x = y 8 > x>y <1 · P (X hits) + ( 1) · P (X misses) We get g(x, y) = 1 · P (Y misses) + ( 1) · P (Y hits) x : 1 · P (X hits) · P (Y misses) + ( 1) · P (X misses) · P (Y hits) x = y 8 > x>y <2p1 (x) 1 Using P (X hits) = p1 (x) and P (Y hits) = p2 (y) =) g(x, y) = 1 2p2 (y) x : p1 (x) p2 (x) x = y Consider the pure strategy, d, that is to fire at distance apart d where p1 (d) + p2 (d) = 1. [9 such a solution.] N.B. d satisfies p1 (d) = 1 p2 (d) i.e. P (X hits) = P (Y misses) and vice-versa. Suggest X and Y both play this pure strategy < d >. We find that (< d >, < d >) gives a PSSP. To see this, calculate: 8 > x>d <2p1 (x) 1 g(x, < d >) = 1 2p2 (d) x : p1 (d) p2 (d) x = d Note that 2p1 (x) 1 2p2 (d) = p1 (d)

1 < 2p1 (d) 1 [when x > d because p1 is mon. decreasing] = p1 (d) p2 (d). So supx g(x, < d >) = p1 (d) p2 (d) = g(< d >, < d >).

Similarly inf y g(< d >, y) = p1 (d) and (< d >, < d >) gives a PSSP.

1.2.3

p2 (d). Also

p2 (d) = g(< d >, < d >) so the pure value of the game is p1 (d)

p2 (d)

Randomised Strategies

If 9 a PSSP, it’s clear to both players what they need to do; we’re home and dry. If @ a PSSP then we will find that it is beneficial to introduce randomised strategies. Consider the following example: a1 a2

b1 b2 | 3 4 | 5 2

No PSSP

Suppose the game is to be played many times and suppose A always plays a1 . Then B would always play b1 and A would get 3. If A always plays a2 then B would play b2 and A would get 2. I.e. if A always sticks to one pure strategy, they can’t get more than 3. With randomised strategies A can do better than 3 = max(2, 3). In fact, they can get 3 12 on average. Also, randomised strategies introduce an element of surprise. Definition: A randomised [or mixed] strategy for A is a probability distribution over AS , A’s pure strategies [with a similar definition for B]. If the game is finite and AS = {aP 1 , . . . , am }, BS = {b1 , . . . bn } then a randomised strategy is a set of probabilities {p1 , . . . , pm } where pi 0 and i pi = 1 such that P A chooses pure strategy ai with probability pi . We write ↵ = (p1 , . . . , pm ), = (q1 , . . . , qm ) where qi 0, i qi = 1. Sometimes we write ↵ = p1 a1 + p2 a2 + · · · + pm am or ↵ = p1 < a1 > + · · · + pm < am >. N.B. Pure strategies are just special cases of a randomised strategies. E.g. the pure strategy a1 corresponds to ↵ := (1, 0, . . . , 0) =: (p1 , p2 , . . . , pm ). A and B don’t reveal their strategies [randomised] to each other pre-play. If A chooses ↵ and B chooses then the pay-o↵ to A, in a finite game, is: XX E[gain to A] = gij pi qj i

9

j

Here
G
=
(gij
)
is
the
pay-o↵
matrix.
Notice
that
the
term
piqj
occurs
because
the
players
choose
their
randomised
strategies
independently.
Since
the
strategy
choices
are
independent
this
can
also
be
written
as: X X X pi gij qj = pi g(ai , ) i

j

i

In the continuous case: E[gain to A] =

Z Z a

where fA , fB are P.D.F’s.

g(a, b)fA (a)fB (b)dbda b

[End lecture 5, 21/10/14]

Question 1 Part I B b 1 b 2 b 3 b4 | a a b b | c d c d | c e c e

a1 a2 a3

A

WLOG. a  b. If we look at column b3 and compare it with b1 we see that b3 is dominated by b1 and similarly b4 is dominated by b2 . This leaves us with the following: a1 a2 a3

b1 b2 | a a | c d | c e

WLOG let’s take d  e. So a2 is dominated by a3 . Notice that we have not “gained” any PSSPs along the way. We now have: a1 | a3 |

b1 b 2 a a c e

WLOG c  e. The second strategy for B in this 2 ⇥ 2 game is dominated by the first. We are then left with

a c

which has a PSSP at max{a, c}.

Part II

2

1 G=4 0 0

3 2 2 2 15 0 3

So, no. This is a counterexample. GT does not have a PSSP

Question 2 Part I WLOG we can assume the pay-o↵s are 1, 2, 3, 4 in some order and that g(a1 , b1 ) = 1. We can write down the 6 possible pay-o↵ matrices: 1 4

2 3

1 3

2 4

All of these are equally likely so P (PSSP) =

1 3 4 2 4 6

=

2 3

1 2

3 4

1 4 3 2

1 2

4 3

Part II 3 ⇥ 3 game.

P (PSSP) =

X

P (PSSP in cell (i, j))

(i,j)

since the pay-o↵s are disjoint [if there were more than one PSSP, the entries in the pay-o↵ matrix would be the same; this contradicts the fact that all entries are disjoint], we have the union of 9 disjoint events. Consider: P (PSSP in cell (i, j)| Entries in cells in row i and col. j consist of, say, a, b, c, d, e) =

4 5!

E.g. (i, j) is the middle cell. The entry in the cell (i, j) must be the middle value in {a, b, c, d, e}

The 4 remaining values must be placed such that the 2 largest values go in row i and the 2 smallest values go in column j [2 orders in each case]. 9 5! possible arrangements. This conditional probability is the same for all selections of 5 symbols out of the 9, so unconditionally, P (PSSP in cell (i, j)) = 5!4 and so: 4 3 P (PSSP) = 9 ⇥ = 5! 10

Question 3

Assume that the volume of available business is proportional to the population and that between them the two companies take all of the business [so it becomes a constant sum game].

Small Company

Large Company

W X Y Z

| | | |

W 60 72 64 56

X 48 60 56 52

Y 56 64 60 48

Z 64 68 72 60

E.g. g(X, Y ) = (0.8 ⇥ 20) + (0.8 ⇥ 40) + (0.4 ⇥ 20) + (0.4 ⇥ 20) = 64, since L is closer to X and W but farther from Y and Z than S is. 9 a PSSP at (X, X) and L can expect to get 60% of the business. Constant-sum game. When the population distribution changes, let p = 3q. The population distribution becomes: W : 20(1 + 2q)% N.B. that the sum is 100.

X : 40(1

3q)%

Y : 20(1 + 2q)%

Z : 20(1 + 2q)%

We recalculate the pay-o↵ matrix: • If p <

3 8

then (X, X) is still a PSSP

• If p =

3 8

then (X, X), (X, Y ), (Y, X) and (Y, Y ) all become PSSPs

• When p > 38 , (Y, Y ) is a PSSP [and (X, X) no longer is a PSSP Small Company

Large Company

W X Y Z

W X Y Z | 60 48 + 16q 56 8q 64 32q | 72 16q 60 64 32q 68 24q | 64 + 8q 56 + 32q 60 72 16q | 56 + 32q 52 + 24q 48 + 16q 60

[If the stores can locate anywhere between W and Z, the 9 a PSSP at (X, X).]

Extra Solutions Let’s also answer the exercises posed in lecture 4: 1. If a game has 2 PSSPs then they have the same pay-o↵. WLOG 9 PSSPs at distinct locations in the pay-o↵ matrix at (a1 , b1 ) and (a2 , b2 ). If these are in the same row or column they must have the same pay-o↵. If not, then we have: g(a1 , b1 )  g(a1 , b2 ) because (a1 , b1 ) is a PSSP  g(a2 , b2 ) because (a2 , b2 ) is a PSSP

Similarly g(a2 , b2 )  g(a1 , b1 ) using comparisons with g(a2 , b1 )

2. If (a1 , b1 ) is a PSSP and a2 dominates a1 , show that (a2 , b1 ) also gives a PSSP. [Hint: Show g(a1 , b1 ) = g(a2 , b1 ).] Clearly: g(a2 , b1 )

a2 dom. a1

g(a1 , b1 )

(a1 ,b1 ) is a PSSP

g(a2 , b1 )

So g(a1 , b1 ) = g(a2 , b1 ) and g(a2 , b1 ) g(a, b1 ) 8a. Also, g(a2 , b1 ) = g(a1 , b1 )  g(a1 , b) 8b [as (a1 , b1 ) is a PSSP]  g(a2 , b) 8b [a2 dominates a1 ]. So g(a2 , b1 ) is the smallest in its row and largest in its column [End lecture 6, 23/10/14] We only consider the cases where pairs of random strategies have finite pay-o↵s. ( A = {↵ : g(↵, ) < 1 8 } Let B = { : g(↵, ) < 1 8↵} Example:

2

3 ( b1 b2 ↵ = 13 , 23 4 a1 3 4 5 Let = 34 , 14 a2 5 1 ✓ ◆ ✓ ◆ ✓ ◆ ✓ ◆ 1 3 1 1 2 3 2 1 g(↵, ) = 3 ⇥ ⇥ + 4⇥ ⇥ + 5⇥ ⇥ + 1⇥ ⇥ 3 4 3 4 3 4 3 4 ✓ ◆ ✓ ◆ 1 3 1 2 3 1 = 3⇥ +4⇥ + 5⇥ +1⇥ 3 4 4 3 4 4 What are the analogues of L , U when we allow random strategies?

Lemma 2. For any ↵⇤ 2 A , 1. sup↵2A g(↵, 2. inf

2B

⇤)

⇤

2 B:

= supa2AS g(a,

⇤)

g(↵⇤ , ) = inf b2BS g(↵⇤ , b)

Proof. of 1. only, in the discrete case. Clearly supa2AS g(a, Conversely, if ↵ = (p1 , . . . , pm ) then: g(↵,

⇤

)=

X i

pi · g(ai ,

= sup g(a, a2AS

⇤

⇤

)·

)

X i

X i

pi · sup g(a, a2AS

⇤)

⇤

⇤)

for all

) [definition of sup and pi

pi 8↵ =) sup g(↵, ↵2A

 sup↵2A g(↵,

⇤

)  sup g(a,

⇤

⇤

because AS ✓ A .

0] =

)

a2AS

Definitions: A maximin strategy for A is a randomised strategy ↵⇤ such that: inf g(↵⇤ , ) = sup inf g(↵, ) ↵

Or equivalently [by lemma 2] such that inf b g(↵⇤ , b) = sup↵ inf b g(↵, b). RHS is called the lower value of the game and is denoted by VL [compare this with L where we only look at pure strategies]. Clearly sup↵ inf b g(↵, b) supa inf b g(a, b) [AS ✓ A ] i.e. VL L. Similarly a minimax strategy for B is a randomised strategy ⇤ such that sup↵ g(↵, ⇤ ) = inf sup↵ g(↵, ). RHS = upper value of the game = VU . We showed [lemma 1] that L  U . The same argument shows VL  VU . We also have VU  U , so we have L  VL  VU  U . If VL = VU then their common value, denoted by V , is called the value of the game and (↵⇤ , saddle-point of the game [where ↵⇤ represents the maximin and ⇤ a minimax] and we have: inf g(↵⇤ , ) = sup g(↵, ↵

⇤

) [= g(↵⇤ ,

⇤

⇤)

define a

)]

[Compare this with PSSP.] If ↵⇤ and ⇤ define a saddle-point then we say that ↵⇤ and ⇤ are good strategies for A and B respectively. In this case, the quantity V = value = amount A should pay B for the privilege of playing the game. V = 0 defines a fair game. Let’s find a minimax strategy for B. [We’ll see a better method later.] Example: 2

4 a1 a2

3 b1 b 2 1 35 2 1

L U

=1 =) VU , VL 2 [1, 2] =2

Let

=
(q1 ,
q2 )
where
qi

0
and
q1
+
q2
=
1.
We
will
find
q1
and
q2
to
make

minimax
for
B.
Calculate: g(a1 ,
)
=
q1
+
3q2
=
3

2q1 g(a2 ,
)
=
2q1
+
q2
=
1
+
q1 We
need
to
choose
q1
so
that
the
larger
of
these
two,
3

2q1
and
1
+
q1
is
minimised.
[N.B.
we
only
need
to
consider
g(a1,
)
and
g(a2,
)
by
lemma
2.]

At x: q1 = 23 , q2 = 13 . So

⇤

=

2 1 3, 3

is minimax for B and VU = 1 +

2 3

= 53 , which is the y coordinate at x.

Similarly, we find ↵⇤ = 13 , 23 is maximin for A and VL = 53 . As these two are equal, the game has value 53 . This is a very long-winded method, and later on we’ll see a better method of solving 2 ⇥ 2 games. Note if AS and/or BS are/is not finite, then there is no guarantee that the suprema and infima above are actually attained by any randomised strategies. Definition: If a game has a value and if the maximin and minimax strategies for A, B respectively exist, then the game is called strictly determined. We will prove later that all finite games are strictly determined. All perfect information games have a PSSP [including chess] but we won’t prove this. To “solve a game” means to find VL , VU and maximin and minimax strategies for A, B when these exist. The following result enables us to check whether a guessed solution to a game is actually a solution. [End lecture 7, 28/10/14] In the class on Thursday 6th November, we’ll go over questions 4, 7, 12, and 11. The following lemma tells us how to check a guessed solution to a game is actually a solution: Lemma 3. If ↵⇤ ,

⇤

are randomised strategies for A, B respectively such that for all a 2 AS , b 2 BS we have: ⇤

g(a, then the game has value V = g(↵⇤ ,

⇤)

and ↵⇤ ,

⇤

)  g(↵⇤ , b)

are maximin and minimax respectively.

Proof. VU = inf sup g(↵, )  sup g(↵, ↵

= sup g(a, a

↵

⇤

⇤

)

) [by lemma 2]

 inf g(↵⇤ , b) [by the hypothesis] [largest on LHS  smallest on RHS] b

= inf g(↵⇤ , ) [by lemma 2]  sup inf g(↵, ) = VL ↵

Distribution of this14 document is illegal

Hence VU  VL . But we know that VL  VU , so VL = VU [= V , say], which is the value of the game: V = sup g(↵, ↵

⇤

) = inf g(↵⇤ , )

[because the inequalities above have become equalities]. Hence ↵⇤ is maximin and since g(↵⇤ , ⇤ ) lies between the terms in the above equation, we have V = g(↵⇤ , ⇤ ).

⇤

is minimax. Finally,

Examples: 1.

2

4 a1 a2

3 b1 b2 b3 1 3 35 2 1 2

Let ↵⇤ = 13 , 23 , ⇤ = 23 , 13 , 0 . We calculate: g(a1 , ⇤ ) = 53 , g(a2 , ⇤ ) = 53 ; g(↵⇤ , b1 ) = 53 , g(↵⇤ , b2 ) = 53 , g(↵⇤ , b3 ) = 73 . Using lemma 3 we deduce that ↵⇤ is maximin, ⇤ is minimax and V = 53 ( a b a b 2. AS = BS = [0, 1], a, b 2 [0, 1], g(a, b) = |a b| i.e. g(a, b) = b a a
⇥

This tells us that If the game has a value V then V 2 0,

1 2

⇤

. One might guess

Using lemma 3.: ( As a

1 2



1 2

⌦ ↵ g(a, ⇤ ) = g a, 12 = a 12 , a 2 [0, 1] g(↵⇤ , b) = 12 g(< 0 >, b) + 12 g(< 1 >, b) = 12 (b + 1

8a 2 [0, 1] we can apply lemma 3. So ↵⇤ is maximin,

⇤

1 2

(

= inf b supa g(a, b). First which would yield 1 b.]

U

↵⇤ = 12 < 0 > + 12 < 1 > ⌦ ↵ ⇤ = 1 2

b) =

1 2

is minimax and V =

1 2

1.2.4
Admissible
Strategies Definition:
A
randomised
strategy
↵⇤
for
A
is
called
inadmissible
if
9
a
random
strategy
↵
2
A
such
that
g(↵⇤ ,
b)

g(↵,
b)
8b
2
BS
where
at
least
one
inequality
is
strict
[so
↵
6=
↵⇤ ].
In
this
case,
↵
is
said
to
dominate
↵⇤ ,
i.e.
whatever
B
plays,
↵
is
better
[or
equal]
than
[or
to]
↵⇤
for
A. Definition:
A
randomised
strategy
↵⇤
is
called
admissible
if
no
such
↵
exists.
[Similar
definitions
for
B
with
the
inequalities
the
other
way
round.] We
can
delete
inadmissible
strategies
without
a↵ecting
the
existence
or
otherwise
of
a
saddle-point.
We
may,
by
doing
this,
miss
one
or
more
solutions
to
the
game,
but
this
doesn’t
matter
as
we
only
want
one
solution.

15

Example. 5 targets; 2 armies.

Army A can attack one and only one target. Army B must choose one target to defend but any target connected immediately to that chosen target is also automatically defended. E.g. if B chooses 2 then 1, 3 and 4 are also defended. If A attacks a defended target he/she loses a point [i.e. receives 1] and B receives +1. If A attacks an undefended target then A receives +1 and B receives 1. This is a zero-sum game [always check]! B defends

A attacks

1 2 3 4 5

2 6 6 6 6 4

1

2

3

4

5

1 1 1 1 1

1 1 1 1 1

1 1 1 1 1

1 1 1 1 1

3 1 17 7 17 7 15 1

Delete the inadmissible a2 by comparing it with a1 ; similarly delete a4 by comparing it with a5 . Delete b1 by comparing it with b2 ; delete b3 by comparing it with b2 . [End lecture 8, 29/10/14] Doing this yields the following matrix: B defends 2 A attacks

2 1 1 3 4 1 5 1

4

5

1 1 1

3 1 15 1

4 and 5 are the same so we can delete one; 1 and 3 are the same so we can also delete one of those. This yields: B defends

A attacks

3 5



Try to guess a solution. Use lemma 3 to show that ↵S⇤ = maximin and S⇤ is the minimax in the sub-game.

2

4

1 1

1 1

⇤ S

=

1 1 2, 2

Subgame, S is a solution. S = “sub-game”. ↵S⇤ is the

Maximin and minimax strategies for the whole game are ↵⇤ = 0, 0, 12 , 0, 12 , = 0. [Exercise.] Is this solution reasonable? [Exercise.]

⇤

= 0, 12 , 0, 12 , 0 . Value? Value

1.2.5
Equaliser
Strategies Definition:
A
randomised
strategy
↵⇤
for
A
is
called
an
equaliser
strategy
[ES]
if
g(↵⇤ ,
b)
=
constant
8
b
2
BS
and
there’s
a
similar
definition
for
B. Example:
Two
committees,
A
and
B,
are
at
war.
B
has
two
airfields,
but
can
only
defend
one. ( b1 : defend airfield 1 B’s pure strategies are: b2 : defend airfield 2 A only has resources to attack one airfield. A’s pure strategies are a1 and a2 . If A attacks a defended airfield it withdraws and there is no loss to either side. If A attacks an undefended airfield, then the airfield is destroyed. Airfield 1 is twice as valuable, to both A and B, as airfield 2. b1 b2  0 2 1 0

a1 a2

Consider ↵⇤ = 13 , 23 . We find g(↵⇤ , b1 ) = g(↵⇤ , b2 ) = 23 . So ↵⇤ is an equaliser strategy for A. g(↵⇤ , ) = Also, ⇤ = 23 , 13 is an equaliser strategy for B. Exercise: Show that ↵⇤ ,

⇤

2 3

8 .

are maximin and minimax for A, B respectively, using lemma 3. Find V .

If we had instead the following pay-o↵ matrix: b1 b 2 b 3  0 2 2 1 0 1

a1 a2

We would find that ↵⇤ = 13 , 23 would still be maximin and ⇤ would still be the minimax [check using lemma 3] but ↵⇤ would no longer be an equaliser strategy because g(↵⇤ , b3 ) = 43 i.e. a maximin strategy is not necessarily an equaliser strategy [same for minimax]. Likewise, an equaliser strategy is not necessarily maximin or minimax. Exempli gratia: b1 b2  2 4 2 4

a1 a2

b2 is an equaliser strategy, but b2 is not minimax. However... Lemma 4. If both A and B have equaliser strategies, ↵⇤ and are maximin and minimax respectively.

⇤,

say, then the game has a value and ↵⇤ ,

⇤

Proof. For all ↵ 2 A we have that: inf g(↵⇤ , )

=

↵⇤ is ES

g(↵⇤ ,

⇤

)

⇤

=

is ES

g(↵,

⇤

)

inf g(↵, ) =) inf g(↵⇤ , )

sup inf g(↵, ) ↵

By definition, because the reverse inequality, , trivially holds ↵⇤ is maximin. Exercise: Show Also inf g(↵⇤ , ) = g(↵⇤ ,

This also shows that VL = g(↵⇤ ,

⇤)

⇤

) = sup g(↵, ↵

= VU so the game has value g(↵⇤ ,

If a game has a pair of equaliser strategies ↵⇤ ,

⇤

⇤

⇤

is minimax.

)

⇤ ).

for A, B respectively, then we say they form a simple solution. 17

1.2.6

Solving 2 ⇥ 2 Games

Consider the two-person zero-sum game with pay-o↵s:

a1 a2

b1 b2  x y z w

Neither player can have pure strategy which is an ES, i.e. x 6= y, x = 6 z, z 6= w, y 6= w. Show that if x = y then 9 a PSSP. ( x y and z w have opposite signs If the game has a PSSP, we are done! If the game has no PSSP then x z and y w have opposite signs If they don’t have opposite signs then the game has a PSSP. None can be 0 because otherwise 9 PSSP. Consider the following: ✓ ◆ |z w| |x y| ⇤ ↵ = , |x y| + |z w| |x y| + |z w| N.B. the denominator 6= 0. ↵⇤ is proper vector of probabilities.

Firstly, notice that ↵⇤ represents a randomised strategy. Secondly, suppose x < y and z > w. Then: g(↵⇤ , b1 ) =

x|z |x

w| + z|x y| x(z = y| + |z w| |x

and g(↵⇤ , b2 ) =

y|z |x

w) + z(y x) zy wx = y| + |z w| |x y| + |z w|

w| + w|x y| zy wx = y| + |z w| |x y| + |z w|

So ↵⇤ is an equaliser strategy for A if x < y and z > w. If x > y and z < w we find, in a similar fashion, that z·y ⇤ g(↵⇤ , b1 ) = g(↵⇤ , b2 ) = |x w·x y|+|z w| . In both cases ↵ is an ES for A. Similarly: ⇤

=

✓

|y

|y w| w| + |x

, z| |y

|x z| w| + |x

z|

◆

is an ES for B. So the game has equaliser strategies for A and for B and, by lemma 4, ↵⇤ is maximin and is minimax. Exercise: Compare with the result for the airfields example in section 1.2.5.

⇤

[End lecture 9, 30/10/14] The test will be very basic - solving a series of games. There’ll be a class on Thursday 6th November. This will be the last examinable lecture for the test on the 13th November. 1.2.7

The G

1

Method

If m = n, G, the pay-o↵ matrix, is square. If G 1 exists we can use it to check if the game has a simple solution. If the game has a simple solution this method gives it. Let G = (gij ) be the n ⇥ n pay-o↵ matrix. Calculate G 1 if such exists. If G 1 does not exist try adding a constant to each entry of G to give an invertible matrix. [The game theory problem will be una↵ected. This will work, i.e. 9 such a constant provided the rows of G and the row vector of all ones spans Rn .] Assume that G

1

exists. We will look for a pair of ES’s [one for A, one for B]. Let: ↵⇤ = (p1 , . . . , pn )

↵⇤ is an ES if g(↵⇤ , bj ) =

Pn

i=1 pi gij

⇤

= (q1 , . . . , qn )

= k 8j.

Distribution of this18 document is illegal

1. I.e. if ↵⇤ · G = k · 1T where 1T = row-vector of all ones. Similarly for B we have G · 1 = coumn-vector of all ones. From 1.: 2.

↵⇤ = k · 1T G ⇤ =k·1 G

⇤T

= k · 1 where

1 1 T

We need to show that these two, ↵⇤ and ⇤ , represent proper randomised strategies. This will be so provided the entries of ↵⇤ are all the same sign [similarly for ⇤ ]. [0 is allowed.] ( ↵⇤ consists of k ⇥ row vector of the column sums of G 1 From 2. ⇤ consists of k ⇥ row vector of the row sums of G 1 In order for the expressions in 2. to be randomised strategies, G 1 must have all its row sums with the same sign [or 0] and all its column sums with the same sign [or 0]. If this property does not hold then 6 9 a simple solution. If these conditions are met, we have:  1T · G 1 1T · G 1 1T · G 1 ⇤ ↵ = T = = 1 ·G 1·1 sum of the entries of 1T · G 1 sum of all entries of G 1 Similarly: ⇤

[Sum of entries of G

1

=

1T · (G 1 )T sum of entries of 1T · (G

= sum of those of (G

1 )T .]

↵⇤ ,

1 )T ⇤

=

1T · (G 1 )T sum of the entries of G

1

are maximin and minimax respectively by lemma 4.

Exercise: Find the value of v: k · 1T · 1 ↵⇤ · G · 1 1T · G 1 · G · 1 1 = = = T T 1 ·1 1 ·1 n ⇥ sum of entries of G sum of entries of G 1 0 1 0 5 7 25 15 33 1 @ 31 3 1A =) G 1 = 185 5 40 A 14 5 4 44 25 15 4 6 5 40

V =k= 0

5 Example: G = @7 3 ⇥

1

⇤ 1 We ignore 185 . We add the numbers outside the G 1 matrix. The row and column sums are all positive, so the game has a simple solution: ✓ ◆ ✓ ◆  2 5 40 11 14 4 185 1 ⇤ ⇤ ↵ = , , , = , , ,V = = 17 51 51 17 51 51 51 sum of entries of G 1 Warning: this is not necessarily the best method. Unless G is very easy to invert it is often better to seek equaliser strategies by writing down the relevant equations.

1.2.8

Equilibrium Pairs

Definition: A pair of strategies (↵⇤ , g(↵,

⇤) ⇤

is said to be in equilibrium [or form an equilibrium pair] if:

)  g(↵⇤ ,

⇤

)  g(↵⇤ , ) 8↵ 2 A ,

2B

In an equilibrium pair, neither player has an incentive to change strategy. From lemma 3 we see that an equilibrium pair has ↵⇤ , g(↵⇤ , ⇤ ). However, the converse also holds:

⇤

maximin [for A] and minimax [for B] and V =

Distribution of this19 document is illegal

Lemma
5.
If
↵⇤ ,
⇤
are
maximin
and
minimax
and
if
the
game
has
a
value
then
V
=
g(↵⇤ ,
⇤ )
and
(↵⇤ ,
⇤ )
is
an
equilibrium
pair. Proof. VL

=

↵⇤ is maximin

inf g(↵⇤ , )  g(↵⇤ ,

⇤

)  sup g(↵, ↵

⇤

)

⇤

=

is minimax

VU

Since the game has a value, we know VL = VU = V so everything in the line above is equal and V = g(↵⇤ , Now to show (↵⇤ ,

⇤)

is an equilibrium pair, let ↵0 , g(↵0 ,

so (↵⇤ , 1.2.9

⇤)

⇤

)  sup g(↵, ↵

⇤

0

⇤ ).

2 A , B respectively. Then we have:

) [= g(↵⇤ ,

⇤

)] = inf g(↵⇤ , )  g(↵⇤ ,

0

)

is an equilibrium pair.

S-Games

Pre-amble If we have a set of point x1 , . . . , xn 2 Rm we can form their convex P hull, C(x1 , . . . , xn ), which consists of all points of the form q1 x1 + q2 x2 + · · · + qn xn where qi 0 8i and ni=1 qi = 1.

E.g. with m = 2, imagine a rubber band closing in on the points shaded. The convex hull is the triangle and its interior, C(x1 , x2 , x3 , x4 ):

In general, C(x1 , . . . , xn ) is a convex set i.e. for any 2 points in C, the line segment joining them is also in C. [Exercise.]

[End lecture 10, 04/11/14] If we have 2 disjoint [i.e. non-intersecting] convex sets in Rm it can be shown that there is a hyperplane [i.e. (m 1)-dimensional subspace of Rm ]: a1 x1 + a2 x2 + · · · + am xm = k, ai 2 R 20

which
separates
the
two
convex
sets.
Here,
(x1 ,
.
.
.
,
xm )
is
a
point
in
Rm .
E.g.
with
m
=
2
one
can
obviously
always
draw
a
line
[of
dimension
1
=
(2

1)]
that
separates
the
two
disjoint
convex
sets
C1
and
C2 :

If (x1 , . . . , xm ) 2 Rm then

(

a · x > k x 2 C2 a · x < k x 2 C1

[>, < could be the other way round], a = (a1 , . . . , am ), k const.

In our diagram, C1 and C2 include their boundaries. However not all convex sets do include their boundaries, e.g. {(x1 , x2 ) 2 R2 : x21 + x22 < 1}. We could have 2 disjoint convex sets, C1 and C2 , where C2 includes its boundary but C1 does not, but one of the boundary points of C1 is in C2 :

We still get a separating hyper-plane, but

(

a·x k a·xk

x 2 C2 x 2 C1 [ (boundary of C1 )

Application to Finite Games Finite game AS = {a1 , . . . , am }, BS = {b1 , . . . , bn }. Let 2 B be a randomised strategy for B. We represent by a point in Rm : (g(a1 , ), g(a2 , ), . . . , g(am , )) 2 Rm ✓ ◆ 3 2 1 2 Exempli gratia: G = Let = 23 b1 + 13 b2 : 4 1 5 3

21

Recall from M1GLA:

(

g(a1 , ) = 23 g(a1 , b1 ) + 13 g(a1 , b2 ) = 83 g(a2 , ) = 23 g(a2 , b1 ) + 13 g(a2 , b2 ) = 3

⇥

⇤ = 23 b1 + 13 b2 lies on the line segment joining b1 and b2 and lies in C(b1 , b2 , b3 , b4 ) . [Any other also lie in the convex hull of b1 , b2 , b3 , b4 .] More generally, = q1 b1 + · · · qn bn , [qi 2 [0, 1], q1 , . . . , qn at b1 , . . . , bn respectively.

P

2 B will

qi = 1] is represented by the centre of gravity of weights

Let S be the set of all points in Rm that represent randomised strategies for B. S is called the risk set for B. [A can have a risk set. If we want to distinguish between the 2 risk sets, we call them S(B) and S(A)]. S [for B] is a convex set. It’s the convex hull of the points in Rm that represent the pure strategies for B. In our example, S is the triangle with vertices b1 , b2 , b3 and its interior. If m = 2, S is a convex polygon and its interior. In higher dimensions, if m > 2, S is a convex multi-dimensional polyhedron and its interior. B must choose a point in S as her randomised strategy. What about A? A typical randomised stratP m egy for A is (↵1 , . . . , ↵m ) where ↵i 2 [0, 1] and i ↵i = 1. This defines a set of hyperplanes in R : Hk : ↵1 x1 + . . . ↵m xm = k, for Hk , k a constant. So we have a set of hyperplanes = {Hk }. As k varies the Hk form a set of parallel hyperplanes. Example [m = 2] [the arrow indicates the direction, which follows from k increasing]:

A typical line is ↵1 x1 + ↵2 x2 = k. Exercise: Why is the slope of the lines negative? [All points in S on the same hyperplane have the same pay-o↵ to A [and to B]]. In general, a typical ↵ = (↵1 , . . . , ↵m ) defines a direction in Rm . If ↵ 2 A ,

2 B, then the payo↵ to A is:

g(↵, ) = ↵1 g(a1 , ) + · · · + ↵m g(am , ) =
(↵1
⇥
first
coordinate
of
s)
+
·
·
·
+
(↵m
⇥
mth
coordinate
of
s)
=
↵
·
s
[dot
product]

where
s
2
S
is
the
point
representing

in
S
[as
long
as
Hk
intersects
S].

N.B.
all
points
on
the
same
hyperplane
have
the
same
pay-o↵
[namely
k].
Think
of
this
as
a
new
game
in
which B’s
randomised
strategies’re
points
of
S,
the
risk
set,
and
A’s
pure
strategies’re
the
m
coordinates
in
Rm .
The
payo↵
to
A
is
↵
·
s.
This
is
called
an
S-game.

22

E.g.: G =

3 2 1 2 4 1 5 3

= 23 b1 + 13 b2 :

1. If ↵ = (1, 0) then Hk is the line g(a1 , ) = k 2. If ↵ = (0, 1) then Hk is the line g(a2 , ) = k 3. If ↵ =

1 2 3, 3

then Hk is the line 13 g(a1 , ) + 23 g(a2 , ) = k

We will represent maximin [for A] and minimax [for B] geometrically. [End lecture 11, 05/11/14]

Question 4 R = rock, S = scissors, P = paper.

R P S ↵⇤ and

⇤

R 0 41 1 2

are both ESs. So ↵⇤ is maximin,

P S 3 1 1 0 15 1 0

⇤

↵⇤ = ⇤ =

1 3, 1 3,

1 3, 1 3,

1 3 1 3

is minimax. V = 0.

Question 7 Zero-sum game because all targets are equally valuable to 2 0 0 1 60 0 0 6 6 1 0 0 G=6 6 1 1 0 6 4 1 1 1 1 1 1

A and to B. The pay-o↵ matrix is [for m = 6]: 3 1 1 1 1 1 17 7 0 1 17 7 0 0 17 7 0 0 05 1 0 0

The obvious thing to do is to delete inadmissible pure strategies. This leads to: a2 a5 Consider ↵S⇤ = 12 , 12 , S⇤ = the whole game we have:

1 1 2, 2 ⇤

| |

b1 0 1

b6 1 0

. Both are ES’s so ↵S⇤ ,

↵ =

✓

1 1 0, , 0, 0, , 0 2 2

as maximin, minimax respectively. V =

1 2.

◆

Exercise ⇤ S ⇤

are maximin and minimax in the sub-game. For =

✓

1 1 , 0, 0, 0, 0, 2 2

◆

Check this using Lemma 3.

m = 8: Delete inadmissible pure strategies. This leads to the 4 ⇥ 4 matrix: a2 a4 a5 a7

| | | |

b1 0 1 1 1

b4 1 0 0 1

b5 1 0 0 1

b8 1 1 1 0

a4 = a5 , b4 = b5 in the sub-game. Delete, for example, a4 and b4 . We then get: a2 | a5 | a7 |

b1 0 1 1

b5 1 0 1

b8 1 1 0

which has a simple solution: ↵S⇤

=

✓

1 1 1 , , 3 3 3

◆

We then extend this to the whole game.

⇤ S

1

=

2

a1 a2

✓

1 1 1 , , 3 3 3

3

4

◆

5

2 3

V = 6

7

8.

b1 b2  x y z w

Question 11 Suppose A bids £a and B bids £b. 8 > <400 a A gets: b > : 200

8 > : 200

a>b a
[This gives g(a, b)]

b

a>b a
[This is a constant-sum game]

The table of pay-o↵s [below, there are 4 PSSP’s]: B [£100]

A [£100]

0 2 63 6 62 6 41 0 2

0 1 2 3 4

1 1 2 2 1 0

2 2 2 2 1 0

3 3 3 3 2 0

4 3 4 47 7 47 7 45 2

If we change this to a zero-sum game by subtracting £200 from each pay-o↵, Value = £0. This is a fair game. If any real-numbered bid from £0 to £400 is allowed, consider A and B bot playing / bidding < 200 >. This is a PSSP. Check this using Lemma 3.

Question 12 By contradiction. Suppose that:

⇤

is a unique minimax strategy for B that is not admissible. Then 9 6= g(↵, )  g(↵,

⇤

) 8↵ 2 A

[with at least one strict inequality]. Hence: sup g(↵, )  sup g(↵, ↵

By the definition of

⇤

↵

)

⇤

)

as minimax, we must have: sup g(↵, ) = sup g(↵, ↵

which contradicts the uniqueness of

⇤

↵

as minimax.

Question 8 Part I Here we have a PSSP:

⇤

2

3 G = 42 5

4 6 4

3 1 05 3

⇤

such

Part II

2

3 x 0 0 G = 4 0 y 05 0 0 z

Are there any PSSP’s? If any of x, y, z is 0 we get a PSSP and V = 0. Look for ES’s. Suppose ↵ = (p, q, r) is an ES for A. p, q, r 2 [0, 1], p + q + r = 1. Then: xp = yq = zr

!

p=

yz xz xy , q= , r= D D D

where D = xy +xz +yz. If x, y, z all have the same sign then (p, q, r) represents a randomised strategy. V =

xyz D

If they didn’t all have the same sign, delete inadmissible strategies ! a game with value 0.

Part III Add 1 to each entry in Part II. [End lecture 12, 06/11/14] Thursday 13th November. 40 minutes long. Justify your answers! [Test 2: 11th December.] We will represent minimax and maximin strategies geometrically. Let: Q✏ = {(x1 , . . . , xm ) 2 Rm : xi  ✏

i 2 {1, . . . , m}}

Q✏ is the set of points in Rm whose maximum coordinate is  ✏. [✏ may not be small or even positive!] m = 2:

Q✏
is
convex
and
it
includes
its
boundary.
[The
interior
of
Q✏
is
also
convex.] Let
S✏
=
S
\
Q✏ .
S✏
contains
those
points
of
the
risk
set,
S,
whose
maximum
coordinate
is

✏. If
✏
is
very
large
and
positive
then
the
whole
of
S
will
lie
in
Q✏
and
S✏
=
S:
if
✏
is
large
but
negative,
then
Q✏ may
not
intersect
S
at
all.
Somewhere
in-between
we
have
the
smallest
✏
such
that
Q✏
intersects
S
[S✏
6=
;]. [Since
we
assumed
that
the
pay-o↵s
in
A
are
bounded
from
below,
this
✏
is
finite.]

25

We let: ✏M = inf{✏ : S✏ = S \ Q✏ 6= ;} Claim: S✏M = S \ Q✏M is the set of minimax points for B. This is because it contains those points of S that minimise the maximum coordinate, i.e. they minimise supa g(a, ) [= sup↵ g(↵, )]. Typically, for m = 2, we have:

Here, S✏M consists of the single point SM alone. Note that other situations can arise, e.g. multiple minimax points and minimax point for B that is a pure strategy for B:

What
about
A?
Consider
the
hyperplane
Hk
where: ↵1 x1
+
↵2 x2
+
·
·
·
+
↵m xm
=
k For
fixed
↵
=
(↵1 ,
.
.
.
,
↵m )
[i.e.
a
randomised
strategy
for
A]
and
fixed
k,
all
points
on
the
hyperplane,
all
points
of
S,
have
the
same
pay-o↵
[=
k].
For
fixed
↵,
the
hyperplanes
Hk
are
parallel.
As
we
increase
k
the
pay-o↵
increases. Hk
must
intersect
S
so
the
most
A
can
guarantee
getting
by
playing
↵
is
given
by: inf{k
:
Hk
\
S
6=
;}
[=
k ⇤ ,
say] A
wants
to
choose
↵
[i.e.
a
direction]
to
maximise
this
infimum.
We
will
see
soon
that
the
optimal
↵
[i.e.
maximin]
is
given
by
a
hyperplane
separating
two
convex
sets:
S
and
the
interior
of
Q✏M
.

26

Typically,
for
m
=
2,
we
have
[the
direction
of
the
lines
is
determined
by
↵,
the
solid
line
is
the
separating
line
that
yields
the
limiting
k]:

The coefficients of the separating line give the maximin strategy for A. In many situations the separating line in the case m = 2 will lie along an edge of S. Suppose we have:

Hk⇤ has equation ↵1 x1 + ↵2 x2 = k ⇤ . The point of contact P has x1 > x2 . So we can increase the pay-o↵ to A by increasing ↵1 [and so decreasing ↵2 ]. This rotates the line through P clockwise [P remains fixed]. However we cannot go beyond the edge P Q because otherwise, as it reaches Q, the point of contact changes. Similarly if Q were the point of contact we could increase the pay-o↵ to A by increasing ↵2 [as x2 > x1 for Q] and thereby rotating the line anti-clockwise. The limiting [optimal] situation is that the optimal direction for A is parallel to P Q. Notes [for m = 2]: 1. The slope of the separating line is never positive [↵1 + ↵2 = 1] 2. The separating line passes through (✏M , ✏M ) [proved later] 3. Typically [m = 2], we get the diagram in the top below. The direction of the separating line gives us the direction for A’s maximin strategy. Sometimes the separating line [m = 2] is not always along an edge of S and there may be more than one maximin strategy. [Illustrated in the other two diagrams below] 27

We will show [later] that the separating line [m = 2] passes through (✏M , ✏M ). And, as we’ve already seen, sometimes the separating line is not along an edge of the risk set S. [End lecture 13, 11/11/14] Class Thursday 20th November. 1.2.10

The Minimax Theorem for Finite Games

Recall: B is trying to minimise supai g(ai , ) = maxai g(ai , ). [ai are pure strategies for A.] Notation: In an S-game g(↵, ) = g(↵, s) where s is the point representing in the S-game. We can write the pay-o↵ to A as g(↵, ) = g(↵, s) = a · s [a dot product]. ↵ = (↵1 , · · · , ↵m ), s = (s1 , · · · , sn ). Theorem 1 (The Minimax Theorem (A very famous theorem proved by Von Neumann)). Every finite game is strictly determined [i.e. has a value and maximin [for A] and minimax [for B] strategies]. Proof. BS is finite so S is closed [i.e. it includes all its limit points] and S is bounded. Let sM 2 S be a minimax point for B [sM 2 S because S contains its limit points]. By definition of minimax:  ✏M [= maximum coordinate of sM ] = sup g(↵, sM ) = max g(a, sM ) as the game is finite ↵

i.e. ↵ · sM  ✏M for all ↵.

a2AS

Recall:
Q✏M
=
{(x1 ,
.
.
.
,
xm )
2
Rm
:
xi

✏M
,
i
=
1,
2,
.
.
.
,
m}.
Let
T
be
the
interior
of
Q✏M
.
So
T
=
{(x1 ,
.
.
.
,
xm )
2
Rm
:
xi
<
✏M
,
i
=
1,
2,
.
.
.
,
m}.
T
is
a
convex
set
and
T
\
S
=
;
because
all
points
in
S
have
at
least
one
coordinate

✏M
[by
definition
of
Q✏M
].
S
is
also
convex
[and
includes
its
boundary]
so
9
a
separating
hyperplane
a
·
x
=
k
where
a
=
(a1 ,
.
.
.
,
am ).
[N.B.
these
are
not
pure
strategies
for
A,
they’re
components
of
a
row
vector,
a.
x
=
(x1 ,
.
.
.
,
xm ).]
This
separates
S
and
T
: (

1

[We’ll see that if

a·x k a·xk

x2S x 2 T [ boundary of T [i.e. Q✏M ]

,  are the other way round - the proof still works!]

Let i 2 Rm have ith coordinate = 1 and the rest = 0. Then sM are  ✏M . Then: ↵ · sM

sM 2S

k

Hence a · i 0 8i i.e. the ith coordinate of a is then this would yield ai  0 8i = 1, . . . , m.] Let ↵⇤ =

Pa i

ai .

sM

i 2Q✏M

i

2 Q✏M because all components of sM

a · (sM

0 8i = 1, . . . , m. [If

i

i)

,  were the other way round in 1

[This is a vector of probabilities whose entries sum to 1.] The ↵⇤ represents a randomised

strategy for A. Let V =

Pk i

ai .

We’ll show that ↵⇤ is maximin for A and V = value of the game. From 1 : (

2

↵⇤ · s V ↵⇤ · x  V

s2S x 2 Q✏M

Consider the point x⇤ = (✏M , . . . , ✏M ) 2 Q✏M [by definition] so: n

3

✏ M = ↵ ⇤ · x⇤  V

For any ↵ 2 Q we have: 4

(

↵ · sM



definition of minimax

✏M  V  ↵⇤ · s for all s 2 S 3 2

Looking at the 2 extreme ends of this inequality we can use lemma 3 to show that ↵⇤ is maximin [we know already that sM is minimax and V is the value of the game because if we put ↵ = ↵⇤ and s = sM in 4 this gives V = value and V = ✏M . We get equality through 4 .]

Corollary: From 3 we get ↵⇤ · x⇤ = ✏M = V so the separating hyperplane passes through x⇤ = (✏M , . . . , ✏M ). Examples: 1. a1 a2

b 1 b2 b 3 b 4 b 5 | 4 5 8 2 6 | 1 8 5 6 6 29

In this diagram, the dotted lines denote the boundaries of Q✏M . sM is the point of the line x1 = x2 and the line joining b1 and b4 ]. V = 22 7 . The minimax for B is 47 b1 + 37 b4 [M1GLA!] or

4 3 7 , 0, 0, 7 , 0

22 22 7 , 7

[the intersection

.

The separating line joins b1 and b4 and has equation: 5x1 2x2 22 + = 7 7 7

✓

x1 = g(a1 , ) x2 = g(a2 , )

◆

So A’s maximin strategy is 57 a1 + 27 a2 . The only admissible strategies for B are those on the line segment joining b1 and b4 . In fact, from the pay-o↵ matrix b2 , b3 , and b5 are inadmissible. We could have deleted these to give:

a1 a2

b1 b4 | 4 2 | 1 6

[End lecture 14, 12/11/14]

2. A has 3 strategies and B has 2. There are no obvious dominated pure [inadmissible] strategies. Plot the risk set S(A) for A. n = 2: a1 | a2 | a3 |

b1 b2 3 5 4 1 0 8

The dotted lines mark the boundary of the limiting Q✏ for A. tM , the intersection of the line x1 = x2 and the segment from a1 to a2 gives the maximin point for A: 35 a1 + 25 a2 . [Exercise.] The separating line is along a1 a2 . This has equation x2 + 4x1 = 17. We just have to normalise this:

⇤

3. a1 a2

x2 4x1 17 + = =V 5 5 5 ✓ ◆ ✓ ◆ 4 1 3 2 ⇤ = , ↵ = , ,0 5 5 5 5

b1 b2 b3 3 3 5 5 2 3

PSSP at (a1 , b2 )

Here there are multiple minimax points for B; they join (3, 3) to b2 . The separating line here is simply x1 = 3 i.e. g(a1 , ) = 3 ! ↵⇤ = a1 . b2 is the only admissible strategy for B; the other minimax points are inadmissible

1.2.11
Bayes
Strategies
in
S-Games Definition:
A
randomised
strategy
↵⇤
for
A
is
a
Bayes
strategy
for
A
with
respect
to

2
B
if
g(↵⇤ ,
)
=
sup↵
g(↵,
)
i.e.
↵⇤
maximises
the
expected
pay-o↵
to
A
if
B
plays
. Definition:
If
↵⇤
is
Bayes
with
respect
to

then
g(↵⇤ ,
)
is
called
the
Bayes
loss
of

to
B. Similarly


⇤
is
Bayes
with
respect
to
↵
2
A


if
g(↵,
⇤ )
=
inf
g(↵,
).

Typically,
for
m
=
2:

b here is Bayes with respect to ↵. ↵ is represented by the parallel lines. Exercise: Show that ↵⇤ , and minimax respectively () ↵⇤ is Bayes with respect to ⇤ and vice-versa.

⇤

are maximin

Admissibility in S-Games Intuitively, admissible strategies for B are those points of S in the south west corner of our diagrams. More precisely (assuming a finite game): Assume S is bounded and includes its boundary. For s 2 S let Q(s) = {x 2 Rm : xi  si , i = 1, . . . , m} where x = (x1 , . . . , xm ), s = (s1 , . . . , sm ). N.B. this is di↵erent from Q✏ earlier. Below are two diagrams with m = 2 in which Q(s) \ S 6= {s} i.e. s is inadmissible and Q(s) \ S = {s} i.e. s is admissible.

Let (S) = {s 2 S : Q(s) \ S = {s}}. (S) consists precisely of all admissible points for B because s 2 (S) () there is no point s0 2 S [s0 6= s] such that si  s0i 8i [with at least one strict inequality].

Lemma
6.
If
a
Bayes
strategy
s⇤
2
S
with
respect
to
↵
2
A
is
unique
then
it
is
admissible. Proof.
Suppose
s⇤
is
inadmissible
and
let
↵
=
(↵1 ,
.
.
.
,
↵m ).
Then
9s
6=
s⇤
such that
for all
i
we
have
si

si⇤
with
at
least
one
strict
inequality.
Hence,
by
multiplying
by
↵i
and
summing,
P ↵i si

P ↵i s⇤
i [we
need

in
case
one
of
the
↵i ’s
is
0]
i.e.
g(↵,
s)

g(↵,
s⇤ ). But
s⇤
is
Bayes
with
respect
to
↵
so
g(↵,
s)

g(↵,
s⇤ )
so
g(↵,
s)
=
g(↵,
s⇤ ),
which
contradicts
the
uniqueness
of
s⇤
as
the
unique
Bayes
strategy
for
B
with
respect
to
↵. Lemma
7.
If
s⇤
is
Bayes
with
respect
to
↵
=
(↵1 ,
.
.
.
,
↵m )
where
each
↵i
>
0
8i
then
s⇤
is
admissible. (

g(ai , s)  g(ai , s⇤ ) 8i Proof. Suppose s⇤ is inadmissible. Then 9s 2 S such that g(aj , s) < g(aj , s⇤ ) for some j P P ⇤ ⇤ ⇤ i ↵i g(ai , s) < i ↵i g(ai , s ) = g(↵, s ) contradicting s as Bayes with respect to ↵.

hence g(↵, s) =

↵i >0

Lemma 8. If s is admissible then it is Bayes with respect to some ↵ 2 A .

Proof. If s is admissible then s 2 (S) and S and Q(s)/{s} are disjoint convex sets so 9 a separating hyperplane between them. The coefficients of this hyperplane give the required ↵. 1.2.12

Solving Finite Games

1. Cheat! There are many game solvers on the web that solve finite games. See the following site which will only work with numerical values: http://banach.lse.ac.uk/form.html [End lecture 15, 18/11/14] 2. Linear programming for finite games. Simplex method algorithm. We won’t look at this approach. 3. (a) Look for a PSSP (b) Delete any obvious inadmissible strategies. These are usually the pure strategies but not necessarily:

Exempi gratia: Consider

a1 a2

b1 b2 b3 3 2 4 4 5 1

I.e. b1 is inadmissible. Exercise: show ↵⇤ =

2 1 3, 3

,

and notice that

⇤

1 1 3 b2 + b3 gives 3 2 2

= 0, 12 , 12 are maximin and minimax

(c) Look for a pair of equaliser strategies: • Use ideas of symmetry. In the following, the first and third rows are mirror images and the second row is a mirror image of itself [similarly for columns]: 1 2 6 3 4 3 6 2 1 We should try ↵⇤ = (p, q, p), ⇤ = (r, s, r) where 2p + q = 1 [similarly for B] • If the pay-o↵ matrix, G, is symmetric then an equaliser strategy for A, if such exists, will also be an equal strategy for B • If the game is symmetric [i.e. G = GT ] then an equaliser strategy for A, if such exists, will also be an ES for B. A symmetric game has value 0: If ↵⇤T = (p1 , . . . , pm ) is maximin for A then ⇤ = (p1 , . . . , pm ) must be minimax for B [by symmetry: both players have the same options] and V = ↵⇤T G↵⇤ = ↵⇤T GT ↵⇤ = ↵⇤T G↵⇤ = V =) V = 0. [Here we used the fact that V is scalar so is equal to V T .] This is a fair game

4. Look at sub-games [i.e. we have a subset of A’s pure strategies and a subset of B’s]: Sometimes we can find a solution to a sub-game that, when extended to the whole game, can give a solution to the whole game. [One can simply try putting 00 s for other pure strategies in an attempt to solve the whole game. Use lemma 3 to check.] In fact, It can be shown that 9 at least one solution that can be obtained from extending a simple solution [i.e. a pair of ESs] to some sub-game: [Q✏M intersects S on a “face” or “edge” or point of S [in some linear subspace of S] and the pay-o↵ on the separating hyperplane is the same at all points of this intersection]. Example: 3 5 2 4 1 3 0 8 2 The G 1 method does not work here, there’s no simple solution [all the row sums are > 0, but this is not true for all the column sums]. We need to look at sub-games. Try the following: b1 b2 b3 3 5 2 4 1 3

a1 a2 b1 is inadmissible !

5 2 ! ↵S⇤ = 25 a1 + 35 a2 , 1 3

⇤ S

= 15 b2 + 45 b3 . Try ↵⇤ =

2 3 5, 5, 0

,

⇤

= 0, 15 , 45

So lemma 3 tells us we do not have a solution. However: a2 a3 gives ↵⇤ = 0, 34 , 14 ,

⇤

b1 b2 b3 4 1 3 0 8 2

= 0, 18 , 78 , which works [using lemma 3]. V =

11 4

[exercise]

[End lecture 16, 19/11/14]

Question 5 g(↵, )



(↵, ) in equil.

g(↵, )  g( , )  g( , )  g(↵, ) ( , )

( , )

(↵, )

Hence g(↵, ) = g(↵, ) = g( , ) = g(↵, ). To show (↵, ) is in equilibrium we need to show that for all 2 A , µ 2 B, we have: g( , )  g(↵, )  g(↵, µ) by the definition of an equilibrium pair To see this, note: g( , )  g( , ) (↵, )

=

From above Also = g(↵, )

g(↵, )  g(↵, µ) (↵, )

Distribution of this34 document is illegal

Question 6 b 1 b 2 b3 1 2 3 2 3 2 3 2 x

a1 | a2 | a3 | Part I Two methods: • The G

1

method. What makes the row and column sums have the same sign? Answer: 1  x  3

• Let ↵ = (p, q, r) where p, q, r

0, p + q + r = 1. Write down the equations for ↵ to be an ES and solve: p=

3

x 4

x

,

1 4

,

r=

1 2

We see that (p, q, r) is a randomised strategy because 1  x  3. Value = (1 ⇥ p) + (2 ⇥ q) + (3 ⇥ r) =

x+7 4

Part II 1 2 3 G=2 3 2 3 2 x 1. x > 3: b3 is inadmissible [compare it with b1 ]. Now delete a1 [compare it with a2 ]. We are left with: a2 | a3 |

b1 b2 2 3 3 2

⌦ ↵ Let’s guess: ↵⇤ = 12 , ⇤ = 12 , 12 . ↵S⇤ , S⇤ are ES’s for the sub-game so are maximin and minimax for the sub-game. Extend this to the whole game. We get that the value = 52 . ⇥ ⇤ 2. x < 1: b2 is inadmissible compare it with 12 b1 + 12 b3 . We are left with: b 1 b3 1 3 2 2 3 x

Distribution of this35 document is illegal

We see that a2 is maximin [and it happens to be an ES]. There are many separating lines all going through (2, 2). B has minimax strategy qb1 + (1 q)b3 in the sub-game for 12  q  23 xx [one way of seeing this is by asking the question: For which values of q does Lemma 3. tell us we have a solution to the sub-game? Another way of forming this inequality is noting that the separating line is of the form (y 2) = m(x 2) where m is the gradient of a line between the lines joining (2, 2) and (3, x), and (2, 2) and (1, 3), so x 2  m  1. ✓ ◆ m 1 ⇤ = , m 1 m 1 Extend this to the whole game. The value = 2.

Question 9 Let

⇤

be an admissible ES for B. Proof by contradiction. Suppose sup g(↵, ↵

⇤

⇤

is not minimax. Then 9 such that:

) > sup g(↵, ) ↵

⇤

is an ES, so suppose g(↵, ⇤ ) = c(= constant) 8↵. So c > sup↵ g(↵, ). g(↵, ⇤ ) > g(↵, ) 8↵, which contradicts the admissibility of ⇤ .

So c > g(↵, ) 8↵.

So

Question 14 0 a b

G=

a b 0 c c 0

This has no PSSP’s. If a = 0 we look at possibilities for b and c. If both are negative, then (a3 , b3 ) is a PSSP and V = 0. If at least one of b and c is 0 then [exercise] show that 9 a PSSP. So a 6= 0. Similarly b, c 6= 0. To show a, b have opposite signs, look at these two cases: • a, b are both > 0 • a, b are both < 0 In both cases we get PSSP’s. Show a, c have the same sign, again by contradiction. Look for equaliser strategies. Note that the pay-o↵ matrix is anti-symmetric (G = and V = 0. Write down the equations for an ES for A [same for B]. We find: ↵⇤ = to be a pair of ES’s and hence ↵⇤ , strategies.

⇤

⇤

=

GT ) so the game is fair

(c, b, a) a b+c

are maximin and minimax respectively. Check that these are randomised

Distribution of this36 document is illegal

Question 13 Lynda only gave numerical solutions to this question. Here’s my solution: Part I G = a1 | a2 |

b 1 b 2 b 3 b4 1 3 2 7 8 2 4 1

So the line x1 = x2 intersects the line segment b2 b3 . So the separating line encompasses the segment b2 b3 . y = mx + c: ( ( ✓ ◆ 2 = 3m + c m= 2 y 8 2 1 8 normalising 2x ⇤ =) =) y + 2x = 8 =) + = =) ↵ = , ,V = 3 3 3 3 3 3 4 = 2m + c c=8 As for

⇤:

✓ ◆ 3 + 2 2

( ✓ ◆ ✓8◆ 3 2 = 38 =) 3 4 2 3

+ 2(1 2 + 4(1 2

2)

= 2) =

8 3 8 3

=)

2

2 = , 3

3

=

1 =) 3

⇤

=

✓

2 1 0, , , 0 3 3

◆

Part II 1 4 3 G=3 2 4 5 1 4 ⇥ ⇤ ⇥ ⇤ b3 is inadmissible compare it with 12 b1 + 12 b2 . Then a2 is inadmissible compare it with 12 a1 + 12 a3 , giving 1 4 the sub-game , which you can solve using the formula for 2 ⇥ 2 games with no PSSP and extend this to 5 1 the whole game !: ✓ ◆ ✓ ◆ 4 3 3 4 19 ⇤ ⇤ ↵ = , 0, , = , ,0 , V = 7 7 7 7 7 [End lecture 17, 20/11/14]

Distribution of this37 document is illegal

1.3 1.3.1

Non-Zero-Sum Games Introduction

In a zero-sum game [i.e. a strictly competitive game]: 1. It’s never advantageous to divulge your strategy to your opponent 2. There is no point in discussing a joint strategy with your opponent 3. If (↵1 , equal

1)

and (↵2 ,

2)

are both equilibrium pairs then so are (↵1 ,

4. (↵, ) is an equilibrium pair () ↵,

2)

and (↵2 ,

1)

and the pay-o↵s are

are maximin / minimax. V = g(↵, ). [Proved in lemmas 3, 5]

In non-zero-sum games, one or more of these may not hold: ( gA (↵, ) = gain to A when A plays ↵ and B plays Let gB (↵, ) = gain to B when A plays ↵ and B plays A wants to maximise gA (↵, ) B wants to maximise gB (↵, ) Non-zero-sum games are of two types: 1. Non-cooperative games: A and B cannot confer. They both know gA and gB and the table of pay-o↵s. They choose their strategies independently and simultaneously 2. Cooperative games: The 2 players can confer in advance to decide on a joint strategy to their possible mutual advantage. Their strategy choices are not independent. All pre-play agreements are binding. “Bargaining” 1.3.2

Non-Cooperative Games

Examples: 1. Prisoners’ Dilemma: Two suspects, A and B, are held in di↵erent cells and they cannot communicate. The officer in charge suspects that they are guilty of major crime but does not have enough evidence to convict them. Each prisoner is given two choices: DON’T CONFESS = a1 for A = b1 for B

CONFESS = a2 for A = b2 for B

If both were to confess, (a2 , b2 ), they each would receive eight years in prison. If neither confesses, they get one year each in prison because they can be convicted on some other minor crime. If one confesses and the other does not, the one who confesses gets three months and the other gets ten years: B b1= don’t confess b2 = confess ( 1, 1) ( 10, 0.25) A a1 = don’t confess a2 = confess ( 0.25, 10) ( 8, 8) Multiply by 4 and add 40: a1 a2



b 1 b2 (36, 36) (0, 39) (39, 0) (8, 8)

Whatever B chooses, A would prefer a2 . Similarly, whatever A chooses, B would prefer b2 . However if A plays a2 and B plays b2 then both do rather badly. What appears to be best for them, as individuals, is not good when they both choose to confess Distribution of this38 document is illegal

2. The battle of the sexes: A is a man, B is a woman. Each has a choice: cinema [a1 and b1 ] or tennis [a2 and b2 ]. A prefers the cinema, B prefers tennis, but most importantly they want to be together. Consider:

a1 a2

A

B b b2 1  (2, 1) ( 1, 1) ( 1, 1) (1, 2)

Assume there’s no communication about the decision. The man says to himself: “I want a1 , and she wants b2 , but we both do badly like that. So, if I choose a2 [i.e. give in to her] and she chooses b2 then we both do pretty well”. However B will argue in a similar way and give in to him by playing b1 . This gives the combination (a2 , b1 ), which is bad for both. Moral: communicate in advance! 3. Evolution: Dawkins [“Selfish Gene”] Maynard-Smith. Worked models for animal behaviour. In a population of hawks and doves, each member behaves either like a hawk [H] or a dove [D]. When two members confront each other, we have the rules: • If 2 hawks meet they always fight until one gets badly injured [cost = 50]. On average, the loss to a hawk in an H H confrontation is 25 [half the time they win, half the time they lose] • When a hawk and a dove meet, the hawk wins 50 because the dove runs away [dove gets 0] • When 2 doves meet no one gets hurt, but one [who gets 0] runs away, the other gets 50. However both lose 10 for wasting time in a staring match. In a D D confrontation the average gain = 12 ⇥ 50 10 = 15. We can model this as a two-person game: 2 players, A and B. Each can choose H or D. We get a pay-o↵ table:

A

H D

B H D  ( 25, 25) (50, 0) (0, 50) (15, 15)

Question: is there a mixture [i.e. a randomised strategy] of H and D that is stable? I.e. a mixture such that any small deviation from the stable state is soon brought back to it. Looking for an “equilibrium”. [End lecture 18, 25/11/14] 4. Cuban missile crisis in 1962: Russians wanted to put nuclear weapons on Cuba, threatening the USA. The US had two strategies: • Natural blockade to stop Russians sending further weapons • Air strike to wipe out existing weapons on Cuba, followed by an invasion of Cuba Russia also had two strategies: • Withdraw (W) • Not withdraw Russians W Not W  USA Blockade Compromise for both Soviet victory, more powerful weapons Airstrike USA victory Nuclear war In the end, Russia withdrew and persuaded Kennedy not to invade Cuba. ‘Game Theory in the Humanities’ by S.J. Brams MIT Press Distribution of this39 document is illegal

Definitions: 1. The pair (↵⇤ ,

⇤)

is said to be in [Nash] equilibrium if

(

gA (↵⇤ , gB (↵⇤ ,

⇤) ⇤)

gA (↵, ⇤ ) 8↵ 2 A gB (↵⇤ , ) 8 2 B

The definition is due to Nash [1950] ! Nobel prize. Neither A nor B has an incentive to change strategy. N.B. This definition agrees with our earlier definition for zero-sum games, why? We call (↵⇤ ,

⇤)

an equilibrium pair

2. Two equilibrium pairs (↵1 , equilibrium pairs

1)

and (↵2 ,

2)

are called interchangeable if (↵1 ,

2)

and (↵2 ,

1)

are also

3. A non-cooperative game is called Nash solvable if every pair of equilibrium pairs is interchangeable. By convention, any game with 0 or 1 equilibrium pair is Nash solvable. Examples of equilibrium pairs: 1. Chicken [the starred entries in the following are the E. pairs [pure]]:

We have a long straight road on a single track. To swerve or not to swerve? That is the question. It’s of course more macho not to swerve, but one doesn’t want to get killed! B swerve don’t swerve  swerve (2, 2) (1, 4)⇤ A ⇤ don’t swerve (4, 1) ( 100, 100) 2. Prisoners’ Dilemma: B b1 = don’t  confess b2 = confess (36, 36) (0, 39) A a1 = don’t confess a2 = confess (39, 0) (8, 8)⇤ 3. Battle of the sexes: a1 a2



b1 b2 (2, 1)⇤ ( 1, 1) ( 1, 1) (1, 2)⇤

Here are two equilibrium pairs but 9 a pair of randomised strategies in equilibrium:

(

↵⇤ = 35 a1 + 25 a2 ⇤ = 2b + 3b 5 1 5 2

[N.B. These are NOT maximin and minimax respectively.] Exercise: Check this from the definition that (↵⇤ , ⇤ ) are an EP. We need to verify: gA (↵⇤ , gA (ai , ⇤ ) 8i; gB (↵⇤ , ⇤ ) gB (↵⇤ , bj ) 8j. [We only need to check pure strategies by lemma 2]

Distribution of this40 document is illegal

⇤)

There
is
a
situation
where
it
is
easy
to
find
equilibrium
pairs: Lemma
9.
In
a
non-zero-sum
game
played
non-cooperatively
[or
cooperatively],
if
↵⇤
is
an
equaliser
strategy
for
A
using
B’s
pay-o↵s,
and
⇤
is
an
equaliser
strategy
for
B
using
A’s
pay-o↵s,
then
(↵⇤ ,
⇤ )
is
an
equilibrium
pair. ( ( gA (ai , ⇤ ) = CA 8i gA (↵⇤ , ⇤ ) = CA = gA (ai , ⇤ ) 8i Proof. C , C are constants so A B gB (↵⇤ , bj ) = CB 8j gB (↵⇤ , ⇤ ) = CB = gB (↵⇤ , bj ) 8j Examples: 1. Battle of the Sexes

a1 a2



b1 b2 ⇤ (2, 1) ( 1, 1) ( 1, 1) (1, 2)⇤

2 1 1 and in this game B has equaliser strategy 25 , 35 . B’s pay-o↵s are 1 1 1 and in this game A has equaliser strategy 35 , 25 . N.B. Not all equilibrium pairs occur this way. A’s pay-o↵s are

1 2

2. Evolutionary Model H D



H D ( 25, 25) (50, 0) (0, 50) (15, 15)

25 50 25 7 5 7 5 ⇤ which gives ⇤ = 35 60 , 60 = 12 , 12 . Similarly ↵ = 12 , 12 . So a popula0 15 5 hawks and 12 doves is stable or in equilibrium. This is called an evolutionary stable strategy

A’s pay-o↵s are tion with

7 12

Definition: An equilibrium pair (↵1 ,

1)

is said to be equivalent to an equilibrium pair (↵2 , ( gA (↵1 , 1 ) = gA (↵2 , 2 ) gB (↵1 , 1 ) = gB (↵2 , 2 )

2)

if

[In a zero-sum game any 2 equilibrium pairs are both equivalent and interchangeable.] Examples: Look at Chicken and the Battle of the Sexes. Equivalent? Interchangeable? In both Chicken and the Battle of the Sexes the equilibrium pairs of pure strategies are neither equivalent nor interchangeable. Geometrical Interpretation of Non-Zero Sum, Non-Cooperative Games ( x = pay-o↵ to A We plot the points (x, y) where Suppose ↵ = pa1 + (1 p)a2 , y = pay-o↵ to B a game where each player has two pure strategies.

= qb1 + (1

q)b2 in

[End lecture 19, 26/11/14] These represent randomised strategies when 0  p, q  1. We have: x = gA (↵, ) = pqgA (a1 , b1 ) + p(1

q)gA (a1 , b2 ) + q(1

p)gA (a2 , b1 ) + (1

p)(1

q)gA (a2 , b2 )

with a similar expression for y. Question: which points (x, y) 2 R2 correspond to randomised strategies (↵, )? Alternatively, for which points (x, y) 2 R2 do there exist p, q 2 [0, 1] with 0  p, q  1 that give rise to (x, y)?

The
set
of
such
points
corresponding
to
randomised
strategies
for
A
and
B
is
called
the
pay-o↵
set,
S. Firstly,
any
point
of
S
must
be
in
the
convex
hull
of
the
4
points
(ai ,
bj
)
[i,
j
2
{1,
2}]
because
0

pq

1,
0

p(1

q)

1
&c.
and
pq
+
p(1

q)
+
q(1

p)
+
(1

p)(1

q)
=
1. However
not
every
point
in
the
convex
hull
is
necessarily
an
element
of
S
because
the
coefficients
pq,
p(1

q),
q(1

p)
and
(1

p)(1

q)
are
constrained
[i.e.
they
have
a
particular
structure];
this
arises
from
the
fact
that
the
players
choose
their
strategies
independently. Example:
Battle
of
the
Sexes: a1 a2



b1 b2 (2, 1)⇤ ( 1, 1) ( 1, 1) (1, 2)⇤

Then (x, y) 2 S () for some p, q 2 [0, 1] we have: (x, y) = pq(2, 1) + p(1

q)( 1, 1) + q(1

p)( 1, 1) + (1

p)(1

q)(1, 2)

= q[p(2, 1) + (1

p)( 1, 1)] + (1 q)[p( 1, 1) + (1 p)(1, 2)] ( L = p(2, 1) + (1 p)( 1, 1) which lies on the line segment joining two points But L is a point on the M = p( 1, 1) + (1 p)(1, 2) line segment joining (2, 1) and ( 1, 1) and M is a point on the line segment joining ( 1, 1) and (1, 2). The line segment LM is in S. In the below diagrams, S is the shaded region of the parabolic envelope.

Here, we let p vary in [0, 1]. S consists of the 2 straight line segments [( 1, 1) to (2, 1) and ( 1, 1) to (1, 2)], the parabolic arc and the region interior to these. Note that there are points in the convex hull that are not in S. Note the following: • We would have produced the same S if we had used p instead of q in the expressions for (x, y) • All points on the line segment ( 1, 1) to (2, 1) are in S [take q = 1] and all points on the line segment joining ( 1, 1) to (1, 2) are in S [take q = 0] • The line segment joining (1, 2) and (2, 1) is not in S. Note that ( 1, 1) and (2, 1) are in the same row of the table of pay-o↵s, as are ( 1, 1) and (1, 2), whereas (1, 2) and (2, 1) have no row or column in common Exercise: Show that the equation of the parabolic arc is 5(x

y + 1)2 = 4(3x

2y + 1).

In general, S depends on the orientation and location of the four points representing the pure strategies.

Example [2 ⇥ 2 game]: a1 a2 Possibilities [we will plot ⌘, ✓,

b1 b2 ⌘ ✓ ⇠

and ⇠]:

Here, the line segments joining ⌘ and ✓, and ⇠, ⌘ and , ✓ and ⇠ all lie in S [set p = 1, p = 0, q = 1, q = 0 respectively]. However the line segments joining ⌘ and ⇠ or ✓ and may not be in S. Sometimes S comprises the whole of the convex hull of the four points representing pairs of pure strategies. E.g. if the four points ⌘, ✓, , ⇠ are the vertices of a convex quadrilateral, labelled in the order ⌘, ✓, , ⇠ we have diagrams (a) or (b). If M is in the convex hull, we can find p, q 2 [0, 1] such that ↵ = (p, 1 p), = (q, 1 q) are represented by the point M . In general, this will not be so and various di↵erent diagrams can result. In (d) we have a curve on the lower part and two straight lines on the upper part. The whole of the shaded area = S. The lines drawn are in equal increments. [In (c) there are two straight lines on the lower part, and a curve on the upper.] Other possibilities can occur. [End lecture 20, 27/11/14]

Theorem 2 (Nash 1950). Every finite 2-person game has at least one equilibrium pair of strategies. Preamble: Let ↵ = (↵1 , . . . , ↵m ), = ( 1 , . . . , n ) be any randomised strategies for A, B respectively. ( ri = max {gA (ai , ) gA (↵, ), 0} i = 1, . . . , m sj = max {gB (↵, bj ) gB (↵, ), 0} j = 1, . . . , n Example: b2  b1 (4, 0) (0, 2) (3, 1) (4, 0)

a1 a2 gA (↵, ) =

17 6 ,

so r1 = max gA (a1 , )

17 6 ,0

Let: ↵ =

✓

1 2 , 3 3

◆

=

= 0 and r2 = max gA (a2 , )

✓

1 3 , 4 4

◆

17 6 ,0

=

11 12 .

[Exercise.]

Proof (Outline). Let ↵i⇤ = ⇤ ) and and ↵⇤ = (↵1⇤ , . . . , ↵m cise].

⇤

=(

⇤ 1, . . . ,

⇤ n ).

↵ i + ri P 1 + ri

⇤ j

=

j

1+

+ sj P sj

Check that these are randomised strategies for A and B [Exer-

Define f (↵, ) = (↵⇤ , ⇤ ). We now use Brouwers’ Fixed Point Theorem [without proof]: a continuous function from a closed bounded convex set C into C has a fixed point. f satisfies the requirements of this theorem. f is a map from Rm+n to itself. So f has a fixed point, i.e. 9(↵, ) such that f (↵, ) = (↵, ). We will show that the pair (↵, ) is in equilibrium () f (↵, ) = (↵, ). Firstly =) : If (↵, ) is an equilibrium pair then, by definition, gA (ai , )  gA (↵, ) 8i so ri = 0 8i and hence ↵i = ↵i⇤ 8i and so ↵ = ↵⇤ . Similarly = ⇤ , so f has a fixed point. Secondly (= : By contradiction. Suppose (↵, ) is not an equilibrium pair. Then either [or both]: ( 0 9↵ such that gA (↵0 , ) > gA (↵, ) 0 9 such that gB (↵, 0 ) > gB (↵, ) 0 ). We know We’ll just consider the first of these situations as the second is very similar. Let ↵0 = (↵10 , . . . , ↵m P that gA (↵0 , ) = i ↵i0 gA (ai , ) hence 9 at least one gA (↵, ). Otherwise, there would P i such that gA (ai , ) > P be a contradiction with the first case: gA (↵0 , )  i ↵i0 gA (↵, ) = gA (↵, ) i ↵i0 = gA (↵, ).

P P For P this i, we have ri > 0 and, as another consequence, we know ri > 0. However, gA (↵, ) = i ↵i gA (ai , ) = hence gA (ak , )  gA (↵, ) for some k with ak > 0 otherwise gA (↵, ) > gA (↵, ), k:↵k >0 ↵k gA (ak , ) and h i P clearly a contradiction uses ↵ = 1 . k k:↵k >0 For this particular k, we have rk = 0 and ↵k⇤ = ↵⇤ 6= ↵ and (↵, ) is not a fixed point of f .

↵ Pk 1+ i ri

< ↵k since ↵k > 0 and

P

ri > 0. Hence ↵k⇤ 6= ↵k , so

Solving
Non-Cooperative
Non-Zero-Sum
Games Difficult!
We’ve
met
Nash
solvable
already. Definition:
A
non-cooperative
game
is
called
Nash
solvable
if
every
pair
of
equilibrium
pairs
is
interchangeable. Convention:
If
a
game
only
has
one
equilibrium
pair,
it
is
called
Nash
solvable.
Exempli
gratia:
Prisoners’
Dilemma
is
Nash
solvable
[as
it
only
has
one
equilibrium
pair].
The
Battle
of
the
Sexes
is
not.

44

0

Definition: A pair of randomised strategies (↵, ) is called jointly inadmissible if 9(↵ , (

0

gA (↵ , 0 gB (↵ ,

0 0

) )

0

) such that

gA (↵, ) gB (↵, )

with at least one strict >. Otherwise (↵, ) is jointly admissible. Definition: A non-cooperative game is said to have a solution in the strict sense if: 1. 9 an equilibrium pair amongst the jointly admissible pairs 2. All jointly admissible equilibrium pairs are both equivalent and interchangeable [if 9 only one jointly admissible equilibrium pair, by convention it is solvable in the strict sense] Very few interesting games satisfy these stringent requirements. E.g. the following does have a solution in the strict sense:

A



B (1, 1)⇤ (0, 0) (0, 0) (2, 2)

(1, 1) is in equilibrium, but is not jointly admissible. (2, 2) is a solution in the strict sense. Prisoners’ Dilemma has no jointly admissible equilibrium pairs, so it is not solvable in the strict sense. Repetition of Non-Cooperative Non-Zero-Sum Games [or Supergames] Richard Dawkins “The Selfish Gene”, Poundstone: “Prisoners’ Dilemma”. Consider: B[you] cooperate defect  (3, 3) (0, 5) A[me] cooperate [nice] defect [nasty] (5, 0) (1, 1) If you only play one game, it’s best to defect [though both do badly]. But how should we play if we repeat the game a large number of times? You need to think about your strategy. Generally, both players do reasonably well if both cooperate, but there is always the temptation to defect to gain more. Would your opponent forgive you? I somehow doubt it. In 1981 Robert Axelrod [American political scientist] conducted an experiment. He held a competition and asked people to submit strategies for playing repeated Prisoners’ Dilemma games. The computer programs he added randomly. The participants weren’t told how many rounds, but they played each other for about 200 games!

[End lecture 21, 02/12/14] Strategies: 1. Tit for Tat: Cooperate on first move then copy previous moves of opponent. Never 1st to defect 2. Nave Prober: Same as Tit for Tat, but occasionally play defect, then copy your opponent’s last move 3. Grudger: Always played defect once the opponent has defected There are two kinds of strategy: a nice strategy is one where you are never the first to defect; a nasty strategy is the opposite. Axelrod found that all nice strategies did better than all nasty ones.

Second
experiment:
62
submitted
strategies.
All
were
told
results
of
the
first
experiment.
Many
in
the
second
experiment
were
nice
strategies
because
these
had
worked
in
round
one.
Others
submitted
nasty
strategies
because
they
wanted
to
take
advantage
of
a
lot
of
“softies”.
Tit
for
Tat
won
in
both
experiments.
For
one
game,
defect
is
the
only
sensible
strategy.
If
the
number
of
moves
is
known
in
advance,
then
each
player
would
play
defect
on
the
last
move
as
there’s
no
chance
of
retaliation.
So
it
is
best
to
play
defect
on
round
n

1
[n
total
moves]
as
the
opponent
will
play
defect
on
round
n
anyway.
Backwards
induction
&c.
The
game
becomes
trivial. 1.3.3
Cooperative
Games Most
games
are
like
this.
All
agreements
are
binding
and
players
decide
in
advance,
after
discussion,
what
each
will
play.
Assume
AS
,
BS
are
finite.
We
have
a
pay-o↵
table
with
entries: {gA (ai ,
bj
),
gB (ai ,
bj
)}
2
R2


i
=
1,
.
.
.
,
m


j
=
1,
.
.
.
,
n

Definition: A joint strategy (↵, ) is a joint probability distribution over AS ⇥ BS . I.e. we attach probability pij to the point representing (ai , bj ). Because P the players cooperate, their strategy choices are not independent and so pij 6= pi qj necessarily. 0  pij  1, i,j pij = 1. The pay-o↵ set, the set of joint strategies, is now the convex hull of the points representing (ai , bj ), the pij . Let us call it S. Example with the Battle of the Sexes. S is the pay-o↵ set [the isosceles triangle and its interior]:

0

0

Definition: A point (x, y) 2 S is called inadmissible if 9 a distance point (x , y ) 2 S such that x Otherwise (x, y) is called admissible. In The Battle of the Sexes, the admissible points are shaded above.

0

x, y

0

y.

Definition: The set of admissible points is called the pareto optimal set. S is a convex polygon. Typically:

In this diagram, LM is the pareto optimal set. B would prefer L, but A would prefer M . However we can find: ⇢ sA = sup inf gA (↵, ) ↵

I.e. look at A’s pay-o↵s alone and calculate the value of the game to A. This is the maximum A can guarantee getting, ignoring B’s pay-o↵s. A won’t settle for less than this. sA is A’s security level. Similarly: n o sB = sup inf gB (↵, ) ↵

is B’s security level. Remember that B is trying to make gB big here. Definition: The points of the pareto optimal set with x sometimes called the bargaining set.

sA and y 0

sB are called the negotiation set, N , 0

Referring to the above diagram, N = two line segments from L to M . Examples: 1. Battle of the Sexes: A’s pay-o↵s

2 1

1 1

Solve as a zero-sum game ! sA = 15 , careful with B as pay-o↵ table has gains to B. We may need to transpose B’s pay-o↵s ! sB = 15 . N = whole of the line segment from (1, 2) to (2, 1)

2. Prisoners’ Dilemma:

0

0

The set N is from L to M . 

(36, 36) (0, 39) (39, 0) (8, 8)

A!

36 0 39 [8]

B!

sA = 8

36 39 0 [8]

sB = 8

Again, in general, be careful as the numbers in the pay-o↵ table are gains to B.

⇤ 0 ⇥ [PSSP’s]. A cannot reasonably expect to get more than the x-coordinate of M = 38 13 and B cannot 0 reasonably expect to get more than the y-coordinate of L . Exercise: What is this value? 0

0

Question: What is a “fair solution”? I.e. which points on L M should represent a solution? The Nash Arbitration Procedure [Also Called the Shapley Solution] Choose s⇤ 2 S to maximise (s1 sA )(s2 sB ) [the excess over sA multiplied by the excess over sB ] over all points in N where s⇤ = (s1 , s2 ), with respect to s1 , s2 . Why choose this function? The advantages are: • Symmetric on s1

sA and s2

sB

• Scale invariant [End lecture 22, 03/12/14]

Question 10 Part I ↵⇤ =

⇤

=

1 1 1 1 4, 4, 4, 4

are ES for the 2 players, so maximin and minimax.

Part II 1 2 3 6 1 2 2 3 2 There are two approaches: 1. Write down equations which we need for a pair of ES’s 2. G

1

method. Both lead to ↵⇤ =

1 1 1 3, 6, 2

,

⇤

=

1 1 1 6, 3, 2

,v=

7 3

Question 15 b2  b1 (x, z) (y, z) (x, w) (y, w)

a1 a2

All pairs of pure strategies are in equilibrium. We need to show that every pair of randomised strategies is in equilibrium: Let ↵⇤ = pa1 + (1 that:

p)a2 ,

⇤

= qb1 + (1 q)b2 be any pair of strategies 0  p  1, 0  q  1. We must show ( gA (↵⇤ , ⇤ ) gA (↵, ⇤ ) 8↵ gB (↵⇤ , ⇤ ) gB (↵⇤ , ) 8

For A (B is similar): gA (↵⇤ ,

⇤

) = pqgA (a1 , b1 ) + p(1

q)gA (a1 , b2 ) + q(1

.. . ⇤

p)(1

q)gA (a2 , b2 )

q)gA (↵, b2 ) + &c. 8↵

pqgA (↵, b1 ) + p(1

gA (↵,

p)gA (a2 , b1 ) + (1

) 8↵

Question 16 Let (↵1 ,

1)

and (↵2 ,

be two equilibrium pairs. For 0  1. Let ↵0 = ↵1 + (1

2)

We need to show that (↵0 , gA (↵0 ,

0

)=

0) 2

0

)

0

=

1 + (1

)

2.

is in equilibrium. For A (B is similar):

gA (↵1 ,

1)

+ (1

)[gA (↵1 ,

Since the game is Nash solvable, we know that (↵1 , ↵ 2 A , we have: gA (↵0 ,

)↵2 ,

2

gA (↵,

= gA (↵, = gA (↵,

1) 1)

0

+ (1

+ (1

2)

)[gA (↵, )gA (↵,

2)

+ gA (↵2 ,

and (↵2 , 2)

1)

+ gA (↵,

1 )]

+ (1

)2 gA (↵2 ,

2)

are also equilibrium pairs, so for any 1 )]

+ (1

)2 gA (↵,

2)

2)

) 8↵

A similar result holds for gB and so (↵0 ,

0)

is also an equilibrium pair. Nash equilibria form a convex set.

Question 17 a1 a2

b2  b1 (1, 4) (9, 0) (7, 1) (3, 3)

gA (a1 , b1 ) = 1 < 7 = gA (a2 , b1 ), so (a1 , b1 ) is not in equilibrium. Check the other three pairs of pure strategies are not in equilibrium. Look for ES’s for each player using the other’s pay-o↵s. For A:

1 9 [these are gains to B] ! 7 3

=

1 1 2, 2

is an ES for B using A’s pay-o↵s.

For B:

4 0 [these are gains to A] ! ↵⇤ = 1 3

1 2 3, 3

is an ES for A using B’s pay-o↵s.

[Find gA (ai ,

⇤)

⇤

= 5 8i, gB (↵⇤ , bj ) = 2 8j, hence (↵⇤ ,

⇤)

is in equilibrium].

Question 18



Suppose that ↵ = (p, 1

p),

= (q, 1

(36, 36) (0, 39) (39, 0) (8, 8)⇤

q) are in equilibrium.

gA (↵, ) = 36pq + 39q(1 = 5pq + 31q We know gA (↵, )

gA (a1 , ) and gA (↵, )

p) + 8(1

p)(1

q)

8p + 8

gA (a2 , ) ? [we only need compare with pure strategies].

Using the second of these, ?, 5pq + 31q 8p + 8 gB (↵, ) = 8(1 q) gB (↵, b2 ) = 8 =) q = 0.

31q + 8 so p(5q

8)

0

=)

p = 0. If p = 0,

[End lecture 23, 04/12/14] Example [Shapley solution]:

A

a1 a2

B b2  b1 (2, 1) (3, 0) (0, 4) (2, 5)

The dot on the pareto optimal set, i.e. the line segment between (2, 5) and (3, 0), represents a Shapley solution. [2] 3 [1] 0 A: PSSP ↵⇤ = a1 , SA = 2. B : gains to B. PSSP ⇤ = b1 , SB = 1. 0 2 4 5 The Shapley solution is (s1 , s2 ) where (s1 2)(s2 1) is maximised over N , i.e. along the relevant part of the line 5s1 + s2 = 15. We find (s1 2)(s2 1) = (s1 2)(14 5s1 ) = 5s21 + 24s1 28, a quadratic in s1 [[]. Di↵erentiation tells us that the maximum occurs when s1 = 12 Check that this is in N . N.B. It is 5 , s2 = 3. a global maximum. Exercises [Cooperative Games]: Give examples: 1. Not all equilibrium pairs are jointly admissible 2. Not all jointly admissible pairs are in equilibrium 3. It is possible for ↵ to be inadmissible [ignoring B’s pay-o↵s] and vice-versa for , yet (↵, ) is in equilibrium

2 2.1

Utility Introduction

Consider the choice between: decision d1 = accept £100; decision d2 = get £200 with probability with probability 12 . Most people prefer d1 even though both have the same expected pay-o↵.

1 2

and £0

Now consider the St. Petersburg Paradox [Bernoulli]. Toss a fair coin until you get tails. You can get £2n if the first tail occurs at toss n. E(pay-o↵) =

1 X

2n P (first tail is on toss n)

n=1

=

1 X

n=1

2n ·

1 !1 2n

Although the expected pay-o↵ is infinite, people won’t pay much to play it. So the expected pay-o↵ is not a good measure to compare decisions about money. In addition, the results of making decisions may not even be monetary; they may be prestige, goodwill &c., which are hard to quantify. We need a scale for “value”. We also may have more than one objective. We need to make a comprehensive list of possible decisions and, for each decision, a list of consequences. Let R = the set of all possible consequences of a set of decisions. The unknown events need to be taken into account. The elements of R may not be numerical, and may involve unknown events. We have R = {C}, C is a consequence. Example: You are invited to your friends’ house one evening at 8p.m., but you do not know if they will provide dinner. Two decisions: d1 = eat beforehand; d2 = do not eat beforehand. Your friends will either provide dinner [event ✓1 ] or not [event ✓2 ]. The true value of ✓ is unknown. There are four consequences: C1 = (d1 , ✓1 ), C2 = (d1 , ✓2 ), C3 = (d2 , ✓1 ), C4 = (d2 , ✓2 ). We have a set of consequences {C1 , . . . , C4 }P and, P, a probability distribution over these. An element of P is a set of probabilities p1 , p2 , p3 , p4 . [pi 0, i pi = 1.]

The notation (p1 C1 , p2 C2 , p3 C3 , p4 C4 ) denotes a gamble or a lottery in which Ci occurs with probability pi . In our first example, R = {£0, £100, £200} ⌘ {C0 , C100 , C200 }. We were asked to compare the two lotteries (1 · C100 ) and 12 · C0 , 12 · C200 .

2.2

The Lottery Axioms

By comparing lotteries we will develop a common scale for utility [i.e. “value”]. We can think of decisionth making as comparing P lotteries. R = {C1 , . . . , Cm }, Ci = i consequence. A lottery is L = (p1 C1 , . . . , pm Cm ) where 0  pi  1, i pi = 1. We can form a compound lottery such as L = (pL1 , (1 p)L2 ) where L1 and L2 are lotteries, which means you receive the result of L1 with probability p and the result of L2 with probability 1 p. Definition: We write L1 < L2 if the decision-maker prefers L2 to L1 . L1 > L2 [L1 preferred to L2 ]. L1 ⇡ L2 [indi↵erent between L1 and L2 ]. L1 . L2 [i.e. L1 < L2 or L1 ⇡ L2 ]. Example at the friends’ house: d1 d2



✓1 ✓2 C1 [full / costs] C2 [costs] C3 [best] C4 [starve]

Each
Ci
is
a
degenerate
lottery. We
can
order
theses,
e.g.:
C4
<
C1
<
C2
<
C3 .
We
will
now
set
out
some
axioms
for
how
a
rational
decision-maker
should
compare
lotteries.

51

Stuvia.co.uk - The Marketplace for Revision Notes & Study Guides

Axiom 1. For any two lotteries L1 and L2 , either L1 < L2 , L1 > L2 , or L1 ⇡ L2 . I.e. the decision-maker must express their feelings about L1 and L2 . Axiom 2 (Transitivity). For any three lotteries L1 , L2 , L3 such that L1 . L2 , L2 . L3 we must have L1 . L3 . There are some criticisms of axiom 2: • Many people do not behave transitively in their preferences, especially when the consequences are multidimensional • Three non-transitive dice problem: Red 1, 4, 4, 4, 4, 4; Blue 3, 3, 3, 3, 3, 6; Green 2, 2, 2, 5, 5, 5 P (R > B) =

25 = 0.694 36

P (B > G) =

21 = 0.583 36

P (G > R) =

21 = 0.583 36

All >

1 2

Non-transitive [End lecture 24, 09/12/14] Axiom 3 (Substitutability of Lotteries). 1. If L = (p1 L1 , . . . , pn Ln ) and L1 ⇡ L01 , then L ⇡ (p1 L01 , p2 L2 , . . . , pn Ln ) 2. If L =P(p1 L1 , . . . , pn LP n ) and Li ⇡ (qi1 C1 , . . . , qim Cm ) [q’s are probabilities] then we should be indi↵erent: L ⇡ [ i pi qi1 C1 , . . . , i pi qim Cm ] [like the law of total probability]

Criticism: Some people like a multi-stage lottery [those who like gambling]. Axiom 3 reflects this fact. Here’s an example from the 2013/14 course: Ci = £i, L =

1 4 5 L1 , 5 L2

where L1 =

1 1 2 C2000 , 2 C0

, L2 = C200 . The we should believe that: ✓ ◆ 1 1 4 L⇡ C2000 , C0 , C200 10 10 5

Axiom 4 (Principle of Irrelevant Alternatives). If L1 , L2 , and L are 3 lotteries then for any ↵ 2 (0, 1) we should have L1 < L2 () (↵L1 , (1 ↵)L) < (↵L2 , (1 ↵)L). I.e. introducing a new lottery, L, should not change our preferences between L1 and L2 . Axiom 5 (Continuity). For any consequences C1 < C < C2 9↵ 2 (0, 1) such that C ⇡ ((1 However this may be difficult.

↵)C1 , ↵C2 ).

Here’s an example from the 2013/14 course: A doctor is to treat a patient. The patient will either die or survive. There are 3 treatments available: T1 , T2 , T . T always cures but is more expensive than T2 . T1 causes half the patients to die, the cost is the same as T . T2 is cheap, and always cures. Clearly T1 < T < T2 . But the doctor cannot find ↵ > 0 such that T ⇡ ((1 ↵)T1 , ↵T2 ) because the survival of the patient is more important than any cost. Exercise: C0 get nothing, C1000 get £1000, C2000 get £2000. Choose ↵ so you are indi↵erent between C1000 and L = ((1 ↵)C0 , ↵C2000 ).

Distribution of this52 document is illegal

2.3
The
Existence
of
a
Unique
Utility
Function First
notice
that
axioms
1
and
2
together
enable
us
to
order
the
consequences
C1 ,
.
.
.
,
Cm .
C
.
·
·
·
.
C.
To
avoid
trivialities
we
assume
C
<
C. Lemma
10.
For
any
consequence
C
such
that: C
↵)C, ↵C).

Proof. Axiom 5 tells us that 9 at least one such ↵. Suppose [for a contradiction] that 9↵1 < ↵2 such that: C ⇡ ((1

↵1 )C, ↵1 C) and C ⇡ ((1

↵2 )C, ↵2 C)

Axiom 2 now tells us: ↵1 )C, ↵C) ⇡ ((1

((1 Let L be the lottery ((1 Let

= ↵2

( C, (1 ⇡ ((1

)C, C) where

✓

(↵2

↵1 1+↵1 ↵2 .

2 (0, 1). Notice that 1

↵1 . Check

)L) ⇡

=

↵1 )C, (1 + ↵1

↵2 )

↵1 )C, ↵1 C) [by axiom 3] ⇡ ((1

Hence by axiom 2 ( C, (1

)L) ⇡ ( C, (1

✓

=

↵2 )C, ↵2 C)

Check

1 ↵2 . 1

2 (0, 1).

Consider:

1 ↵2 ↵1 C C, 1 + ↵1 ↵2 1 + ↵1 ↵2

◆◆

= ((↵2 ↵1 )C, (1

↵2 )C, ↵2 C) [by axiom 2] ⇡ ( C, (1

)((1

)C, C))

)L) [by axiom 3]

)L), which violates axiom 4 because C < C. So ↵1 = ↵2 .

Corollary: Lemma 10 works with C, C replaced by any C1 , C2 where C1 < C < C2 . Defintion: A utility function u is a map u : R ! R such that: 1. If C1 < C2 then u(C1 ) < u(C2 ) 2. If C1 < C < C2 and if C ⇡ ((1 to Lemma 10] then u(C) = (1

↵)C1 , ↵C2 ) [note that 9↵ and it’s unique by Axiom 5 and the corollary ↵)u(C1 ) + ↵u(C2 )

I.e. u is preference preserving and linear. Lemma 11. If u is a utility function then u(Ci ) = a↵i + b where a and b are the constants such that a > 0 and ↵i is the unique value in (0, 1) such that Ci ⇡ ((1 ↵i )C, ↵i C). Proof. By linearity, u(Ci ) = (1 ↵i )u(C) + ↵i u(C) = a↵i + b where a = u(C) preserves our preferences, we have a > 0 because C < C.

u(C) > 0, b = u(C). Since u

N.B. The converse of Lemma 11 holds too [not proved here]. We can extend this toP lotteries. Let L =P(p1 C1 , . . . , pm Cm ) be a lottery and let Ci ⇡ ((1 ↵i )C, P ↵i C), ↵i unique. Hence L ⇡ ( i pi (1 ↵i )) C, ( pi ↵i ) C [by axiom 3] = ((1 ) ✓ C) where = pi ↵i . We define u(L) = a + b, a > 0, a, b constants. This is a well-defined function because the ↵i ’s are well-defined and so is . It can also be shown that u(L) has the properties of a utility function. Note that: u(L) = a

X i

pi ↵ i + b =

X i

pi (a↵i + b) =

X

pi u(Ci )

i

so u(L) can be thought of as expected utility. If we accept axioms 1 to 5 then we should base our preferences on maximising the expected utility.

2.4
The
Utility
of
Money What
does
u(z)
look
like?
Assume
each
element
of
R
is
a
sum
of
money
in
R Typically
we
have
the
following
concave
u(z):

• u(z)

0 .


Consider
u(z)
for
z
2
R

0 .


0 [u(0) = 0]

• u(z) is increasing in z • u(z) is twice di↵erentiable • u(z) is concave. u00 (z)  0 because u0 (z) decreases as z increases, which reflects the idea that an extra £10 to a rich person means very little compared with what it means to a poor person • u(z) is linear near z = 0 • u(z) is bounded above; after £101000 no one would want any more [End lecture 25, 10/12/14] u(z) is very di↵erent for z < 0. Typical functions used to model u(z) are: • u(z) = a log(1 + bz), a, b > 0 constants and b small (to get linearity near the origin) z • u(z) = z+✓ , ✓ > 0 constant. ✓ represents willingness to take risks: a high ✓ ⌘ love risk; a low ✓ ⌘ risk averse with this u(z).

• u(z) = 1

e

✓z ,

✓ > 0 constant. Risk averse ⌘ large ✓: risk loving ⌘ small ✓.

A risk-averse person prefers the certainty of small amounts of money. A risk-loving person prefers speculative gains of large amounts to certain gains of small amounts.

3

Bayesian Methods

If X, Y are continuous random variables then the conditional P.D.F. of Y given X = x: where: " # fX|Y (x|y)fY (y) fX,Y (x, y) fY |X (y|x) = =R fX (x) y fX|Y (x|y)fY (y)dy If x is fixed, fX (x) = constant and fY |X (y|x) / fX,Y (x, y) = fX|Y (x|y)fY (y).

Example: Y ⇠ Exp( ), > 0. X|Y = y ⇠ Exp(y). We will calculate the conditional distribution of Y |X. fY |X (y|x) / fX|Y (x|y)fY (y) = ye xy e y / ye (x+ )y , y 2 (0, 1). I.e. given X = x, Y has a Gamma(2, x + ) distribution.

Bayesian Inference Suppose we have some observations x1 , . . . , xn , which are realisations of random variables X1 , . . . , Xn that have a joint P.D.F. depending on an unknown parameter ✓ [✓ may be vector-valued but will be scalar in an exam]. E.g. X1 , . . . , Xn independent but identically distributed N (✓, 1) random variables where ✓ is unknown [this is called a random sample]. The Bayesian approach to estimating ✓ assumes that ✓ itself has a P.D.F. before we collect the data. I.e. we regard ✓ as a random variable. Denote this P.D.F. ⇡(✓). We call it a prior P.D.F. for ✓. It represents our knowledge and beliefs about ✓ before we collect the data. Here’s an example form the 2013/14 course: A patient sees a doctor. The patient either has a disease, D, or not. The doctor must diagnose which it is: ✓ = 1 is the event that the patient has D; ✓ = 0 is the event that the patient does not have D. The doctor believes P (✓ = 1) = ⇡, P (✓ = 0) = 1 ⇡. The doctor now does some tests, i.e. collects data. We will call this data X. We get: ( P (✓ = 0|X = x) / fX|✓ (x|✓ = 1) ⇥ ⇡ P (✓ = 0|X = x) / fX|✓ (x|✓ = 0) ⇥ (1 ⇡) Updated beliefs about ✓. In general, we use Bayes’ Theorem as follows: ⇡✓|X (✓|x) / fX|✓ (x|✓)⇡(✓) x = (x1 , . . . , xn ), X = (X1 , . . . , Xn ), ⇡✓|X (✓|x) is called the posterior P.D.F. of ✓. fX|✓ (x|✓) is the joint P.D.F. of the data given a particular ✓. As a function of ✓, this joint P.D.F. is called the likelihood function, i.e. posterior P.D.F. of ✓ / likelihood ⇥ prior P.D.F. of ✓. Examples: 1. X ⇠ Bin(n, ✓). n known, ✓ unknown. We’re told ⇡(✓) = 6✓(1 ✓), ✓ 2 [0, 1]. Using Bayes: ✓ ◆ n x ⇡(✓|x) / ✓ (1 ✓)n x · 6✓(1 ✓) / ✓x+1 (1 ✓)n x+1 x ✓ 2 [0, 1]. Hence the posterior P.D.F. of ✓ given X = x, is that of a Beta(x+2, n x+2) random variable. The prior distribution looks like something resembling a semi-circle: its posterior counterpart is more peaked - the variance has decreased. Posterior P.D.F. updates our feelings about ✓ in light of the data 2. X ⇠ N (✓, 02 ), 02 is known, ✓ is unknown. We want to estimate ✓. The prior distribution of ✓ [we are told] is ⇡(✓) = N (µ0 , ⌧02 ) where µ0 , ⌧02 are both known. Use Bayes: ✓  ◆ 1 (x ✓)2 (✓ µ0 )2 =) ⇡(✓|x) / exp + [= prior ⇥ likelihood] 2 2 ⌧02 0 The first term in the [ ] comes from the likelihood, and the second term from the prior. Think of this as a function proportional to another normal P.D.F. ⇣ of ✓.⌘ Complete the⇣square in⌘ ✓. Get an 2expression ⌧0 + 02 µ 1 1 x ! ✓|x ⇠ N m, p where m = p ⌧ 2 + 2 and p = ⌧ 2 2 0

0

0

0

How do we find ⇡(✓)? • From historical data • Subjectively • Non-informative priors. How do we express ignorance about ✓?

Non-Informative Priors If ✓ can only take a finite set of k values we might set ⇡(✓) = k1 . If ✓ 2 [a, b] then take ⇡(✓) =

1 b a,

✓ 2 [a, b].

Problem: Any non-linear function of ✓, say g(✓), is not uniformly distributed over [g(a), g(b)]. There’s a worse problem if the range of ✓ values is infinite. R If, for example, ⇡(✓) = constant [finite] then ✓ ⇡(✓)d✓ is infinite. This is an example of an improper prior as it doesn’t integrate to 1. However, we can still use Bayes’ Theorem as we often get a proper posterior distribution for ✓ [integrates to 1]. E.g. taking X1 , . . . , Xn a random sample from Exp( ), unknown [ > 0] [random sample means X1 , . . . , Xn are independently and identically distributed]. We assume ⇡( ) / 1 , an improper prior. ⇡( |x) / Provided n

n

e

Pn

i=1

xi

⇥

1

[by independence] =

1, so this gives a proper posterior P.D.F. for

n 1

e

Pn

i=1

xi

[ > 0]

given X = x, that is Gamma (n,

Pn

i=1 xi ).

[End lecture 26, 16/12/14]

4
Decision
Theory 4.1
Introduction Decision
making
involves
uncertainty.
Firstly,
make
a
list,
D,
of
all
possible
decisions
or
actions.
Secondly,
make
a
list
of
the
unknowns.
These
are
called
the
states
of
nature,
⇥. E.g.
do
you
take
an
umbrella?
You
don’t
know
if
it
will
rain
or
not.
⇥
=
{0,
1}.
✓
=
0
it
rains:
✓
=
1
it
does
not
rain.
For
any
d
2
D
and
✓
2
⇥
we
assume
that
we
can
evaluate
the
consequences
and
the
utility
of
this
consequence.
In
decision
theory,
we
talk
of
losses;
let
us
define
the
following
loss
function: L(✓,
d)
=
utility
of
the
action
d
when
✓
is
the
true
state
of
nature n from
which
we
can
form
a
table
of
the
states
of
nature
{✓i}m i=1
against
the
actions/decisions
{dj
}j =1
with
entries
equal
to
L(✓i,
dj
).
We
assume
that
all
the
L’s
are
finite.
It
will
look
a
bit
like
the
pay-o↵
matrix
of
a
two-person
zero-sum
game.
For
the
rows,
the
“states
of
nature”
represent
the
set
of
pure
strategies
for
player
A
and
“actions”
for
the
columns
of
player
B.
We
are
trying
to
minimise
L.
However,
there
are
di↵erences:

• “Nature” is not playing like an intelligent opponent, i.e. it is not trying to maximise L • What are the “randomised strategies” for both “players”? For “Nature” a randomised strategy is a probability distribution over ⇥, i.e. a prior P.D.F. [or P.M.F.] ⇡(✓) for ✓ • We (i.e. player B) do not need to use any element of surprise on Nature so there’s less need for a randomised strategy for the decision maker • We know ⇡(✓), so we are trying to choose the best d 2 D in the light of “Nature” playing the randomised strategy ⇡(✓). I.e. we need to look for a Bayes strategy for the decision maker with respect to ⇡(✓) We should also note that: • Even if we consider randomised actions [i.e. probability distributions that: inf L(✓, d) = inf L(✓, )

over D] then lemma 2 tells us

d

So we lose nothing by restricting ourselves to pure strategies for player the decision maker Definitions: 1. r(⇡, d) = E⇡ L(✓, d) =

R

L(✓, d)⇡(✓)d✓ is called the Bayes Loss of d with respect to ⇡(✓)

2. A decision d⇤ 2 D is called a Bayes Decision or Bayes Action with respect to ⇡(✓) if it minimises r(⇡, d) i.e. if r(⇡, d⇤ )  r(⇡, d) 8d 2 D 3. If d⇤ is a Bayes Decision then r(⇡, d⇤ ) is called the Bayes Loss of ⇡ Example: Let ⇡(✓1 ) = 14 , ⇡(✓2 ) = 34 : ✓1 ✓2 ✓

d1 d2 0 3 5 2

◆ ✓ 1 r(⇡, d1 ) = 0 ⇥ + 5⇥ 4 ✓ ◆ ✓ 1 r(⇡, d2 ) = 3 ⇥ + 2⇥ 4

◆ 3 = 4 ◆ 3 = 4

So d2 is the Bayes Action with respect to ⇡, the Bayes Loss of ⇡ is 94 .

15 4 9 4

4.2
Decision
Rules In
practice,
we
collect
data
to
inform
out
decision
making.
These
data
give
us
information
about
the
state
of
nature.
Let
X
represent
the
data.
Let
X
=
sample
space
of
all
possible
x’s.
The
distribution
of
X
depends
on
the
unknown
✓.
D
is
the
set
of
actions. Definitions: 1. A decision rule d is [now] a map d : X ! D. I.e. if we observe x 2 X then we choose d(x) 2 D 2. The risk function of a decision rule, d, is: R(✓, d) = EX|✓ L (✓, d(X)) = expected value over the distribution of X for a particular fixed ✓ If we have no data then the risk is equivalent to the loss i.e. R(✓, d) = L(✓, d) 3. The Bayes Risk of d 2 D with respect to ⇡(✓) is r(⇡, d) = E⇡ R(✓, d). [Compare with E⇡ L(✓, d) in the case of having no data, as on the previous page] 4. A Bayes Decision Rule, d⇤ , with respect to ⇡(✓) minimises the Bayes Risk of d with respect to ⇡(✓). I.e. if d⇤ is a Bayes Rule, then r(⇡, d⇤ )  r(⇡, d) 8d(x) 5. If d⇤ is a Bayes Rule, then r(⇡, d⇤ ) is called the Bayes Risk of ⇡, denoted by r(⇡) Lemma 12. The Bayes Risk is minimised by minimising the expected posterior loss of d(x). Proof. We want to choose d(x) to minimise the Bayes Risk. I.e. we want to choose d(x) to minimise: Z r(⇡, d) = ⇡(✓)R(✓, d)d✓ ✓

=

Z

⇡(✓) ✓

Z

f (x|✓)L(✓, d(x))dx d✓ x

where the integral in the square bracket is R(✓, d). The next step assumes we can interchange we can do as we’re dealing with realty and expect not to have any pathological example: Z Z = fX (x) ⇡(✓|x)L(✓, d(x))d✓ dx x

using ⇡(✓|x) =

f (x|✓)⇡(✓) fX (x)

R

✓

and

R

x,

which

✓

from Bayes’ Theorem.

Looking at the last integral, once we have observed x we should minimise [. . . ] inside the main integral. If we were to do this for each x then we would have minimised r(⇡, d). I.e. once we know x we need to minimise for any particular x: Z ⇡(✓|x)L(✓, d(x))d✓

✓

= the expected posterior loss of choosing the decision rule d(x)

So minimising the Bayes Risk ⌘ minimising the expected posterior loss ⌘ maximising the expected posterior utility. Example: We observe x from X where X ⇠ N (✓, 1) [✓ is unknown]. The prior distribution of ✓ is N (0, ⌧ 2 ). ✓ 2 ◆ ⌧ x ⌧2 ✓|x ⇠ N , [See Section 3] 1 + ⌧2 1 + ⌧2 We need a loss function. The most common in inference is square error loss: L(✓, d(x)) = (✓

d(x))2

There was some more material on this covered in the 2013/14 lecture series, which I’ll include in an appendix in section 6.1.

Lemma
13.
Under
square
error
loss
[and
a
proper
posterior
distribution
for
✓],
the
Bayes
Decision
Rule
is
to
estimate
✓
by
the
mean
of
the
posterior
distribution
for
✓. Proof.
The
expected
posterior
loss
of
a
decision
rule
d(x)
is: Z ⇡(✓|x)(✓ d(x))2 d✓

This is a [-shaped quadratic in d(x). We di↵erentiate with respect to d(x) and set it = 0: Z Z d 2 ⇡(✓|x)(✓ d(x)) d✓ = 2 ⇡(✓|x) · (✓ d(x))d✓ = 0 d{d(x)} Z Z =) ⇡(✓|x) · ✓d✓ = ⇡(✓|x)d(x)d✓ = d(x) ⇥ ⇤ R The mean of the posterior distribution of ✓. N.B. ⇡(✓|x)d✓ = 1. Examples:

2 1. In our ⇣X ⇠ N (✓, 1) ⌘ example, with square error loss, the prior is N (0, ⌧ ). The posterior distribution of 2 2 ⌧ x ⌧ ✓ is N 1+⌧ [see Section 3]. 2 , 1+⌧ 2

The mean of the posterior distribution for ✓ i.e. the Bayes Rule, d⇤ , for estimating ✓ with respect to ⇡(✓) ⌧ 2x is d⇤ (x) = 1+⌧ 2 2. X ⇠ Bin(n, ✓). ⇡(✓) = 6✓(1 ✓), ✓ 2 [0, 1]. Squared erros loss. We have ⇡(✓|x) / ✓x+1 (1 ✓)n x+1 i.e. the posterior distribution for ✓ is Beta(x + 2, n x + 2). The Bayes Rule d⇤ for estimating ✓ with respect x+2 to ⇡(✓) is the mean of this beat distribution, i.e. d⇤ (x) = n+4 . If instead we were interested in

= ✓2 then we would calculate:

(x + 2)(n x + 2) E✓|X (✓ |X = x) = var(✓|X = x) + [E(✓|X = x)] = + (n + 4)2 n + 5) 2

2

✓

x+2 n+4

◆2

from Beta

[End lecture 27, 17/12/14]

4.3

Decision Trees

Decision-making is usually sequential; most decision tress have many steps, which we use to represent decision problems. There are 2 kinds of nodes [vertices]: decision nodes and chance nodes. Quite often these alternate. Examples: 1. Dinner example: C1 [overfull]; C2 [okay but costly]; C3 [great]; C4 [starve]:

In this diagram, ⇡ = P (meal), the square represents the decision node and the circles, the chance nodes. It is drawn from left to right ! time goes from left to right. Ci are consequences [in practice with attached utilities, sometimes called “terminal utilities”, which sum to 1]

2. Inference [an example from the 2013/14 course]:

3. A company is thinking of launching a new product that it has developed. The marketing executive estimates the profits / losses resulting from di↵erent market shares. The market share level, ✓, is equal to one of 2% or 10% with prior probabilities [with the numbers in utility units] of 0.3 and 0.7: Market Share Level (✓) Prior Decisions

Launch Don’t launch



10% 0.7

500 0

2% 0.3

250 0

[Cost for “scrap” is 0 because the research and development costs have already been sunk.] E(utility of ”launch”) = 0.7 ⇥ 500 E(utility of ”scrap”) = 0

0.3 ⇥ 250 = 275

This is called a prior analysis of the problem. We might do better if we knew more about the market share [i.e. 10% or 2%]. We would launch if 10% and scarp if 2%. E(utility under perfect information) = 500 ⇥ 0.7 + 0 ⇥ 0.3 = 350 The di↵erence [350 275 = 75] is called the expected value of perfect information. It is not worth spending more than this to get information on the unknown market share. Suppose now that the company is o↵ered a market research proposal at a cost of 10 utility units. The market researchers will report one of: • Market share will be “high” • Market share will be “low”

Unfortunately “high” and “low” do not correspond exactly with 10% and 2% as the M.R. company may get things wrong. Market Research Report  High Low Actual Share 10% 0.85 (0.15) 2% 0.25 0.75 (0.15) = P (The market research report says low|the share is actually 10%).

Distribution of this60 document is illegal

Let’s draw the decision tree:

The Market Research Problem

Note the terminal utilities on the right hand side. All probabilities are conditional on everything to the left. Exempli gratia: i. P (M R says “high”) = P (high|10%)P (10%) + P (high|2%)P (2%) = 0.85 ⇥ 0.7 + 0.25 ⇥ 0.3 = 0.67 ii. P (10%| MR says “high”) =

P (MR says “high” |10%)⇥P (10%) P (M R says “high”)

= 0.89 by Bayes

How do we solve this? We fold back the tree from right to left. The purple numbers are obtained from this folding back. The quadruple purple lines are indicating to us not to take these decisions. We take expectations at chance nodes: we maximise at decision nodes. In the top right corner, we take expectations: 0.89(500 10) + 0.11( 250 10). Repeat this for other chance nodes on the right. Attach these expectations to the relevant chance nodes. Then, maximise at decision nodes you meet next. For a more algorithmic specification: • Work from right to left

• Take expectations at a chance node [use probabilities to the right] • Maximise utility at a decision node

• Block o↵ the non-optimal lines at the decision nodes at each stage

4. The Marriage Problem. When should one get married?! Assume n suitable partners will be presented to you in a random order, one by one [n is known]. Having seen person j you can rank persons 1, 2, . . . , j. You then must decide to reject person j [and move on to person j + 1] or accept person j. You cannot back track; you cannot accept one seen earlier. [If you were able to see all at once you could rank them all.] If you reach person n you must accept them. pi > pj denotes person i is preferred to person j. There are no ties. Terminal utility = P (best person has been selected). We want to maximise this. Consider the case n = 4. We need to calculate the terminal “utilities”. For example: P (p3 is best |p3 > p2 > p1 ) =

P (p3 > p2 > p1 and p3 > p4 ) = P (p3 > p2 > p1 )

1 8 1 6

=

3 4

The Marriage Problem

If p2 > p1 then accept p2 . Otherwise see p3 and only accept p3 if p3 > p2 > p1 .

[End lecture 28, 18/12/14] We didn’t cover the case for general n in lectures, but I’ll include it here anyway. If uj = utility of the chance node immediately following seeing pj , exempli gratia un = n1 = “utility” of choosing the final person. Find: ✓ ⇢ ◆ 1 n k un k = max , un k+1 + (n k + 1)un k+1 n k n ✓ ◆ n (k + 1) 1 1 1 = + + ··· + n n (k + 1) n k n 1 as long as un

k+1



n k n

holds.

The threshold occurs when un k+1 = n n k . Set n n k = approximate the sum of reciprocals: ✓ ◆ n 1 log ⇡1!1 n (k + 1)

n k n

⇣

1 n k

k n

1

+ ··· +

⇡e

1

1 n 1

⌘

. Use Euler’s construct to

⇡ 0.368

After the threshold, choose the best so far, id est the first one better than all those seen so far

5

Mastery Exam Component

The M4S11 mastery question will be on n-person cooperative games. The following books may be useful: • Introducing Game Theory and its Applications by Mendelson, Chapman and Hall • Intorduction to Game Theory by Morris, Springer Verlag • The Theory of Games by Wang, Oxford We’re expected to study the following topics: 1. Coalitions 2. Characteristic Functions 3. Imputations 4. The core of an n-person cooperative game Definition. Let P be the set consisting of all N players. A coalition, S, is a subset of P. The corresponding counter-coalition to S is simply S c = P\S. Clearly there are 2N coalitions. We call P the grand coalition and its complement is the empty coalition, ;. With a coalition S, it’s natural to think of the game as having two players: S and S c . We can rewrite this game in bi-matrix form with the tuples representing the sum of the payo↵s to players in the coalition in the first coordinate and which’s second coordinate is the sum of the payo↵s to the players in S c . Definition. The maximin value for the coalition, denoted v (S), is called the characteristic function. Its domain is the set of all coalitions. Note: Obviously v (P) = the largest total payo↵ which the set of all players can achieve and v(;) = 0. Theorem (Superaditivity). Let S and T be two disjoint coalitions. Then: v (S [ T )

v (S) + v (T )

Proof. By the definition of a characteristic function, there’s a joint strategy for the members of S such that the total payo↵ to the members of S is v (S). Similarly, there’s a joint strategy for the members of T such that the total payo↵ to them is v (T ). Since S and T are disjoint, if players in the two coalitions play according to these strategies, then the total payo↵ to the union is guaranteed to be at least v (S) + v (T ) and hence the maximin value for the union is this value, i.e. superaditivity holds. Definition. A game in characteristic function form comprises a set of players P = {P1 , . . . , PN } together with a function v defined on all subsets of P such that v(;) = 0 and superaditivity holds, i.e. v(S [ T ) v(S) + v(T ) for disjoint subsets S and T of P. Definition. An N -person game, v, in characteristic function form is said to be inessential if: v (P) =

N X i=1

v ({Pi })

and a game which is not inessential is said to be essential If a game is inessential all it means is that there’s no reason for coalitions to form. We can see this by the below theorem: Theorem. Let S be any coalition of the players in an inessential game. Then: X v(S) = v({Pi })

Proof. Suppose not, then by superaditivity we must have v(S) > v(S) + v(S c ) >

v(P)

N X i=1

P

Pi 2S

v({Pi }). But now, by superaditivity:

v({Pi })

which contradicts the game being inessential. Theorem. A two-person-zero-sum game in its normal form is inessential in its characteristic function form. Proof. As the game is zero-sum, v (P) = 0. Further, v ({P1 }) =

v ({P2 }). Hence:

v (P) = v ({P1 }) + v ({P2 }) However, zero-sum games with more than two players can be essential. Definition. Let v be an N -person game in characteristic function form with players P = {P1 , . . . , PN }. An N -tuple, x, of real numbers is said to be an imputation if both of the following conditions hold: 1. Individual Rationality: For all players Pi : v ({Pi })

xi 2. Collective Rationality: We have: N X

xi = v (P)

i=1

The first condition is an intuitively obvious imposition. If xi were < v ({Pi }) then player Pi would do better o↵ on their own. For the second condition, consider: PN PN • v(P): If the sum were < v(P), then = v(P) i=1 xi i=1 xi > 0 and the players would form the grand coalition and distribute the total payo↵ v(P) as x0i = xi + /N so must hold PN • i=1 xi  v(P): suppose x occurs, i.e. a coalition with such a profit split occurs. Then, using superaditivity: N X X X xi = xi + xi = v(S) + v(S c )  v(P) i=1

Pi 2S

Pi 2S c

Theorem. Let v be an N -person game in characteristic function form. If v is inessential, then it only has one imputation: x = (v({P1 }), . . . , v({PN })) and if v is essential then it has infinitely many imputations. Proof. Suppose that v is inessential and that x is an imputation. If for some j we had xj > v({Pj }) then PN PN i=1 xi > i=1 v({Pi }) = v(P), contradicting collective rationality. Now suppose that v is essential and let:

= v(P)

N X i=1

v({Pi }) > 0

then
for
any
N -tuple
↵
of
positive
numbers
summing
to

we
have
xi
=
v({Pi })
+
↵i ,
which
clearly
defines
an
imputation.
So,
with
infinitely
many
choices
for
↵,
there
are
infinitely
many
imputations
when
a
game
is
essential. For
an
essential
game,
there
are
too
many
imputations.
So
we
need
a
way
to
single
out
the
ones
which
merit
the
title
of
a
“solution”.
The
following
definition
attempts
to
formalise
the
definition
of
one
imputation
being
preferred
over
another. 64

Definition. Let v be a game in characteristic function form and let S be a coalition with imputations x and y. We say that x dominates y through the coalition S if the following two conditions hold: 1. xi > yi 8Pi 2 S P 2. Pi 2S xi  v(S)

and we denote this by x

S

y

The second condition merely tells us that such an x is feasible, that the coalition can attain enough payo↵ so as to distribute the payo↵s as prescribed. Now we’ll meet a solution concept, albeit one that’s fundamentally flawed in that it is sometimes empty! Definition. Let v be the game in characteristic form. The core of v consists of all imputations which are not dominated by any other imputation through any coalition. So if x is in the core, then no group of players has reason to form a coalition and replace x with a di↵erent imputation. At first, it looks difficult to decide whether x is in the core, but the following theorem will help make this easier. Theorem. Let v be a game in its characteristic function form with N players and let x be an imputation. x is in the core of v if and only if: X xi v(S) Pi 2S

for every coalition S.

Proof. Suppose this formula holds for every coalition S. If some other imputation w dominates x through a coalition S then: X X wi > xi v(S) Pi 2S

Pi 2S

which violates the feasibility condition in the definition of domination.

Now suppose that x is in the core and suppose that S is a coalition such that: X xi < v(S) Pi 2S

Note that S = 6 P otherwise collective rationality in the definition of an imputation would be violated. Next, there has to exist a Pj 2 S c such that xj > v({Pj }). If this weren’t true then by superaditivity: N X i=1

xi < v(S) +

X

Pi 2S c

xi  v(P)

which violates collective rationality. So we can indeed choose Pj 2 S c such that 9↵ with 0 < ↵  xj v({Pj }) P and ↵  v(S) Pi 2S xi . Now, with k denoting the number of players in S, we define a new imputation, w, by: ↵ wi = xi + for Pi 2 S k w j = xj ↵ wi = xi for all other i Then w dominates x through S and so the assumption that x is in the core is contradicted. The next Corollary will give a more convenient form of the above result, which’ll allow us to calculate the core of a game more easily. Corollary. Let v be a game in characteristic function form with N payers and let x be an N -tuple of numbers. Then x is an imputation in the core if and only if the following two conditions hold: PN 1. i=1 xi = v(P) P 2. v(S) for every coalition S Pi 2S xi

Proof.
Certainly
an
imputation
in
the
core
satisfies
the
two
above
conditions.
Now
let
x
satisfy
both
conditions.
The
second
condition
applied
to
one-player
coalitions
shows
that
individual
rationality
holds.
The
first
condition
is
collective
rationality,
and
so
x
is
an
imputation.
And
it’s
certainly
in
the
core. Definition.
Let
v
be
a
game
in
characteristic
function
form.
We
say
that
v
is
constant-sum
if,
for
every
coalition
S,
we
have: v(S)
+
v(S c )
=
v(P) Further,
it’s
zero-sum
if
it’s
constant-sum
and
v(P)
=
0. The
concepts
of
zero-sum
and
constant-sum
are
not
the
same
in
normal
and
characteristic
function
forms;
the
two
aren’t
equivalent.
It’s
possible
for
a
game
which
is
not
constant-sum
in
its
normal
form
to
be
constant-sum
in
its
characteristic
function
form. Theorem.
If
an
N -person
game
is
zero-sum
in
its
normal
form,
then
it
is
also
zero-sum
in
its
characteristic
form. Proof.
Suppose
that
the
game
weren’t
zero-sum
in
its
characteristic
function
form,
i.e.
for
some
coalition
S
we
have
v(S)
6=
v(S c ).
Then,
if
the
players
in
the
coalition
S
and
the
counter
coalition
S c
adopt
the
strategies
that
give
them
such
the
maximin
payo↵s
v(S)
and
v(S c )
in
the
normal
form
game
then
the
total
payo↵
to
all players would P be v(S) + v(S c ) 6= 0. But this is a contradiction, since the game is zero-sum in the normal form we know that N i=1 ⇡i (x1 , . . . , xN ) = 0 for all strategies x1 , . . . , xN for players P1 , . . . , PN . Theorem. If an N -person game is constant-sum in its normal form, then it is also constant-sum in its characteristic form. P ! Proof. Let c be the constant value of the normal-form game, ⇡ , i.e. N i=1 ⇡i (x1 , . . . , xN ) = c for all choices of ! strategies x1 , . . . , xN for players P1 , . . . , PN respectively. We define a new game ⌧ by subtracting c/N from ! every payo↵ in ⇡ . Then ⌧ (x1 , . . . , xN ) = ⇡(x1 , . . . , xN ) c/N for every choice of i and for all choices of ! ! strategies. Then ⌧ is zero-sum, and thus the characteristic function form of ⌧ , u, is zero-sum. But it’s easy ! to see that the characteristic function v of ⇡ is related to u by the formula: v(S) = µ(S) + kc/N where k is the number of players in the coalition S. So clearly v is constant sum. Theorem. If v is both essential and constant-sum, then its core is empty. Proof. Suppose v has players {Pi }N i=1 . We’ll prove this by showing that if v is constant sum and there’s an imputation in its core then v must be inessential. We know, for any player Pj , we have xj v({Pj }) by individual rationality. Since x is in the core, we also have: X xi v({Pj }c ) i6=j

Summing these inequalities we see, using collective rationality, that: v(P) =

N X

xi

i=1

v({Pj }) + v({Pj }c ) = v(P)

by the constant sum property. Hence the inequality is actually an equality and so xj = v({Pj }). Since it holds for every j, the game is inessential. Definition. A game v in characteristic function is form is called simple if all of the below hold: • v(S) is either 0 or 1 for every coalition S • v(P) = 1 • v({Pi }) = 0 8Pi 2 P In a simple game, a coalition S with v(S) = 1 is called a winning coalition and a coalition with v(S) = 0 is called a losing one.

6
Appendix:
Further
Material
of
Use 6.1
On
Risk
and
Loss
Functions These
are
a
couple
of
examples
given
in
last
year’s
course,
which
Lynda
didn’t
have
time
to
go
over
this
year: 1. Oil company. Should they drill for oil at a particular site? There is an unknown probability that there is oil there. They have drillings at other similar sites to inform about ✓. ⇥ = [0, 1], ✓ = P (oil at the site). d1 = drill at the site: d2 = do not drill at the site. Let A = profit if d1 is chosen and oil is found. B = loss if d1 is chosen and no oil is found. So the expected losses are

(

L(✓, d1 ) = A✓ + B(1 L(✓, d2 ) = 0

✓)

In n similar wells [n, known], x have oil. So X = {0, 1, . . . , n} and X ⇠ Bin(n, ✓) (a) Calculation of risk function. Consider the decision rules {dc } of the form: ( d1 if nx c dc (x) = d2 if nx < c Where c 2 [0, 1] is a constant. R(✓, dc ) = expected loss over distribution of X for a particular ✓ = L(✓, d1 )P (X

cn|✓) + L(✓, d2 )P (X < cn|✓) n ✓ ◆ X n t (A + B)✓) ✓ (1 ✓)n r r

= (B

r cn

(b) Calculation of the Bayes Decision Rule. Suppose now that we have a prior distribution for ✓ and that it is Beta(↵, ) where ↵, are known. So: ⇡(✓, x) / ✓x (1

✓)n

Hence the distribution of ✓|x is Beta(↵ + x, n

x ↵ 1

✓

(1

✓)

1

x + ).

We want to choose the action [d1 or d2 ] to minimise the expected posterior loss, so we calculate: E✓|X L(✓, d1 ) = = Where

↵+x ↵+n+

(A + B)E✓|X (✓|X = x) + B (A + B)

↵+x ↵+n+

+B

is the mean of the posterior distribution. E✓|X L(✓, d1 ) = 0

↵+x So, we drill if (A + B) ↵+n+

B, i.e. when x

(n+ )B ↵A . A+B

This is the Bayes Decision Rule

2. Observe a random sample x1 , . . . , xn from N (✓, 1) where ✓ is unknown. xi is an observation on Xi . X1 , . . . , Xn are independent N (✓, 1) random variables. ⇥ = R. We want to estimate ✓, i.e. we are asking which function d(x) of the data best estimates ✓? We need a loss function, L. Some common ones are: • Squared error loss: L(✓, d(x)) = (d(x) • Absolute loss: L(✓, d(x)) = |d(x)

✓)2

✓|

Example to show risk function [assume squared error loss]: R(✓, dc ) = EX|✓ L(✓, dc (X)) where dc (x) = cx, P c = constant and x = n1 n1 xi , X similarly. R(✓, dc ) = EX|✓ (cX

E X = ✓, var X =

1 n

✓)2 = var(cX

✓) + (mean(cX

✓))2 =

c2 + ✓2 (c n

1)2

In the range [A, B], d 1 is better than d1 . but we don’t know ✓! Note that if c > 1 then: 2

R(✓, d1 ) =

1 c2 < + ✓2 (c n n

1)2 = R(✓, dc ) > 0 8✓

If c > 1 we can improve on dc by using d1 . We say that dc is inadmissible. Suppose now that n = 1, i.e. one observation, X, and we have a prior distribution for ✓ that is N (0, ⌧ 2 ) where ⌧ is known. Squared error loss. R(✓, dc ) = c2 + ✓2 (c

1)2 as above

Bayes Risk of dc with respect to ⇡ = r(⇡, dc ) = E⇡ R(✓, dc ) = E⇡ c2 + ✓2 (c 1)2 = c2 +(c 1)2 E⇡ (✓2 ) = c2 + (c 1)2 ⌧ 2 since the variance of the prior is equal to ⌧ 2 . Amongst the rules of this kind (i.e. of the ⌧2 kind dc (x) = cx) this gives the best one, i.e. we minimise c2 + (c 1)2 ⌧ 2 with respect to c ! c⇤ = 1+⌧ 2 This rule is best overall [i.e. the Bayes Rule]. Lemma. If we have absolute loss, the Bayes Rule is to estimate ✓ by the median of the posterior distribution of ✓ [i.e. that value m, say] such that: Z m 1 ⇡(✓|x)d✓ = 2 1 Proof. Expected posterior loss of decision rule d ⌘ d(x) is: Z

1

⇡(✓|x)|✓

d(x)|d✓ =

1

Z

d

(d

✓)⇡(✓|x)d✓ +

1

Z

1

(✓

d)⇡(✓|x)d✓

d

Now let us di↵erentiate with respect to d [N.B. the limit of the integrals involve d itself]:

"

Z

d

⇡(✓|x)d✓ = 1

@ We have used: @a

Z

Z

1

⇡(✓|x)d✓ Exercise

d

g(a)

f (a, x)dx = 1

Z

g(a) 1

@f dx + g 0 (a)f (a, g(a)) @a

#

6.2

Mid-Term 1 - 13/11/14

Solve the following games: Part I G=

3 6 1 0

(a1 , b1 ) is a PSSP because 3 < 6 [row 1] and 3 > 1 [column 1]. The game thus has value V = 3. Part II 1 0 2 G=0 2 1 2 1 0 Consider ↵⇤ =

⇤

1 1 1 3, 3, 3

=

(

g(a1 , ⇤ ) = g(a2 , ⇤ ) = g(a3 , ⇤ ) = 1 g(↵⇤ , b1 ) = g(↵⇤ , b2 ) = g(↵⇤ , b3 ) = 1

Hence A and B both have ES’s implying that ↵⇤ is maximin and

⇤

is minimax. V = 1.

Part III 1 4 6 G=2 0 3 1 2 8 b3 is inadmissible [compare it with b1 or b2 ]. So we delete b3 and after doing so we see that a3 is now inadmissible [compare it with a1 ]. So we then delete a3 . We’re left with: 1 4 2 0 A has ES ↵S⇤ =

2 3 5, 5

and B has ES

⇤ S

=

4 1 5, 5

.

Hence ↵S⇤ is maximin in this sub-game and S⇤ is minimax. For the whole game, ↵⇤ = are maximin and minimax respectively. V = 85 .

2 3 5, 5, 0

and

Part IV x, y 2 R>0 : G= (a1 , b1 ) is a PSSP if When

1 2

<

x y

x y

x 2x 2y y

2. In this case V = x. (a2 , b2 ) is a PSSP if

x y

 2. In this case V = y.

< 2 then there’s no PSSP. ⇤

↵ =

⇤

=

✓

|y

|y 2x| 2x| + |2y

, x| |y

✓

y x , x+y x+y |2y x| 2x| + |2y

◆ x|

is an ES for A ◆

=

✓

2x y 2y x , x+y x+y

◆

is an ES for B

=

3xy x+y .

Check that the latter is indeed a randomised strategy. As both ↵⇤ and

⇤

are ESs, ↵⇤ is maximin and

⇤

is minimax. V =

xy x+y

+

2xy x+y

⇤

=

4 1 5, 5, 0

Part V G=

1 x x2 x x 2 x3

1. x > 0, x 2 R: • 0  x  1 =) (a1 , b3 ) is a PSSP since x2  1, x2  x and x2

x3 . V = x2

• When x > 1, (a2 , b1 ) is a PSSP as x < x2 , x < x3 and x > 1. V = x 2. x =

2 [considering subgames]: G=

1 2

2 4

4 8

9 any obvious inadmissible strategies. The idea here is to solve the 3 sub-games to see if any can be ◆ extended to give a solution to the whole game. Consider: 1 2 Here ↵S⇤ =

2 1 3, 3

=

⇤ S

2 4

are ES’s in the sub-game.

For the whole game, we calculate: ( g(a1 , ⇤ ) = g(a2 , ⇤ ) = 0 g(↵⇤ , b1 ) = g(↵⇤ , b2 ) = g(↵⇤ , b3 ) = 0 where ↵⇤ = 23 , 13 and ⇤ = 23 , 13 , 0 . Hence ↵⇤ and maximin, minimax respectively. V = 0.

⇤

are both ES’s for the whole game and ↵⇤ ,

⇤

are

If we had to disregard b1 instead of b3 we would have obtained an alternate solution to the whole ⇥ ⇤ chosen ⇤ 2 1 2 1 ⇤ game ↵ = 3 , 3 , = 0, 3 , 3 . The other 2 ⇥ 2 sub-game has a PSSP at (a1 , b1 ), but Lemma 3 shows [very easily] that this cannot be 2 1 extended to a solution to the whole game. In fact, any ⇤ = ) 0, 23 , 13 with ↵⇤ = 23 , 13 3 , 3 , 0 + (1 for 2 [0, 1] gives a solution

6.3

Mid-Term 2 - 11/12/14

Part I a. Note that (3, 8) lies on the line joining (2, 9) and (7, 4). (x + y = 11). S, the risk set, is the triangle with vertices (2, 9), (7, 4) and (8, 2).

b. The lines segments (2, 9) ! (7, 4) and (7, 4) ! (8, 2) are the admissible strategies for A. c. sM = (7, 4) + (1 )(3, 8) = (4 + 3, 8 4 ) lies on y = x if = 58 . So ↵⇤ = 58 a3 + 38 a4 is maximin for ⇤ = 1 , 1 is minimax and V = 11 . An alternative A. The line joining (3, 8) and (7, 4) is x2 + y2 = 11 2 so 2 2 2 7 3 maximin strategy for A is ↵⇤⇤ = 10 a3 + 10 a1 [or any strategy of the form p↵⇤ + (1 p)↵⇤⇤ with p 2 [0, 1]] d. The maximum of the blow four is

31 5 .

Hence a1 is Bayes with respect to

2 5 2 g(a2 , ) = 5 2 g(a3 , ) = 5 2 g(a4 , ) = 5 g(a1 , ) =

3 5 3 ⇥8+ 5 3 ⇥7+ 5 3 ⇥3+ 5 ⇥2+

and the Bayes loss for

is

31 5 :

31 5 22 ⇥2= 5 26 ⇥4= 5 30 ⇥8= 3 ⇥9=

Part II

Part III 3 > 2 [column 1] and 6 > 5 [row 3] for A and B respectively. (a1 , b2 ) is an equilibrium pair because 3 > 1 and 3 > 2 [column 2] and 2 = 2 [row 1] for A and B respectively. A’s pay-o↵s are: 1 3 2 2 1 3 3 2 1 An equaliser strategy for B using these pay-o↵s is

⇤

= 13 b1 + 13 b2 + 13 b3 .

B’s pay-o↵s are: 2 2 2 3 5 1 6 5 5 An equaliser strategy for A using these pay-o↵s is ↵⇤ = 1a1 as g(a1 , bj ) = 2 for j = 1, 2, 3. Since ↵⇤ and

⇤

are equaliser strategies using each others pay-o↵s, (↵⇤ ,

⇤)

is an equilibrium pair.

6.4

Further Problem Sheet Questions

The answers that’re to follow were all written by Lynda White, and have been copied into this document.

Question 19 The pairs of pure joint strategies form a rectangle and the pay-o↵ set consists of two triangles:

(a1 , b2 ) and (a2 , b1 ) are both admissible and in equilibrium, but they are not interchangeable so the game is not solvable in the strict sense.

Question 20 The pay-o↵ set is the whole of the convex set determined by the four points representing pairs of pure strategies. The pairs (ai , b2 ) [i = 1, 2] are both equilibrium pairs and so, therefore, is any pair of the form (↵, b2 ). Show that there are no other equilibrium pairs by considering x = 2pq q + 4, y = p q + 2. The game is Nash solvable. The security levels for A and B are 3 and 2 respectively and the Shapley solution is (a1 , b2 ) as (a1 , b2 ) is the only admissible pair of strategies. Draw a diagram!

Question 21 a.

i. Firstly consider the non-cooperative game. Note that if ↵ = (p, 1 x = gA = 4pq + 3p + 2q + 1, y = gB = 4pq p 2q + 3. Hence: p=

x+y 2

4

1

p=

6

x 2

y

q=

10 x 3y 20 4x 4y

p) and

1

q=

= (q, 1

q) then

10 3x y 20 4x 4y

[provided x + y 6= 5]. If x + y = 5, p = 12 , q is arbitrary. So p 0 and 1 p 0 () (x, y) lies between the parallel lines formed by the join of (1, 3) and (3, 1), and the join of (4, 2) and (2, 4). Also q 0 and 1 q 0 () 10 x 3y and 10 3x y have the same sign. The jointly admissible strategies are those corresponding to the line segment joining (2, 4) and (4, 2). No pair of pure strategies is in equilibrium [check]. If ↵⇤ = (p, 1 p) and ⇤ = (q, 1 q) are in equilibrium then 4pq + 3p + 2q + 1 gA (a1 , ) = 4 2q =) (4q 3)(1 p) 0 =) p = 1 or q 43 . Also, 4pq + 3p + 2q + 1 gA (a2 , )2q + 1 =) p(3 4q) 0 =) q  34 or p = 0.

Hence either p = 1, q  34 or q 34 , p = 0 or q = 34 . • If p = 1: 4pq p 2q + 3 gB (↵, b1 ) =) 2q + 2 4 =) q 1 > 34 • If p = 0: 4pq p 2q + 3 gB (↵, b2 ) =) 3 2q 3 =) q  0 < 34 Hence q = 34 and gB (↵, ) = 2p + 32 . But gB (↵, b1 ) = 3p + 1 and gB (↵, b2 ) = 3 p =) p = 12 . That is p = 12 , q = 34 is the only equilibrium pair and the game is Nash solvable ii. If the game is played cooperatively sA = sB = 52 . The negotiation set is the line joining 72 , 52 to 5 7 2 , 2 and the Shapley solution is given by the point (3, 3) by symmetry. This corresponds to ↵ = a1 and = 12 b1 + 12 b2 b.

i. Non-cooperative game: since each edge of the convex hull of the four pure strategy points can be achieved by suitable p and q, the pay-o↵ set is the whole of the convex hull [draw a diagram]. The jointly admissible strategies are those corresponding to the line segment joining (2, 5) and (3, 0). The point (2, 1) represents a pair of pure strategies in equilibrium. Using a method similar to that in 26. show that this is the only equilibrium pair. The game is Nash solvable ii. Cooperative game: show that sA = 2, sB = 1. The negotiation set N is the line segment joining (2, 5) to 11 5x] and the Shapley solution is (s1 , s2 ) where (s1 2)(s2 1) 4 , 1 [which’s equation is y = 15 is maximum over N . Di↵erentiation gives s1 = 12 5 , s2 = 3, which is in fact in N . The Shapley 2 3 solution is ↵ = 5 a1 + 5 a2 , = b2

Question 22 Draw a diagram. The payo↵ set is a square. The admissible strategy pairs correspond to points on the line segment joining (a b, a + b) to (a, b). The security levels for A and B are respectively 0 and b. The Shapley 2 2 solution maximises s1 (s2 b) such that as1 +bs2 = a2 +b2 . This gives s1 = a2 and s2 = a +2b by di↵erentiation. 2b ⇥ ⇤ a a a a2 +2b2 If 2 > a b i.e. if b > 2 the Shapley solution is given by x = 2 and y = 2b , which is in the negotiation set. Otherwise the Shapley solution is at (a b, ab ).

Question 23 L1 > L2 =) u(5000) > 0.1u(25000) + 0.89u(5000) + 0.01u(0) and L4 > L3 =) 0.1u(25000) + 0.9u(0) > 0.11u(5000) + 0.89u(0). Rearrange each of these to give two contradictory inequalities. Many people would say L1 > L2 and L4 > L3 , but if you believe the Lottery axioms these are contradictory.

Question 24 Plot a graph! Note that u(z) and u0 (z) are both continuous at z = 1 . h i u0 (z) = 2 2 2 z 0 0  z  1 and zero otherwise. Hence u(z) is a non-decreasing function of z reflecting h i the fact that people prefer more money to less. u00 (z) = 2 2 0  z  1 and zero otherwise. So u(z) is a concave function of z, reflecting the fact that people tend to avoid taking risks. Also u(z) ! 1 as z ! 1, which reflects the fact that for most people there is an upper limit to the amount of money they want. High values of correspond to high risk aversion. u(L1 ) = 0.4(20 100 2 ) + 0.4(10 25 2 ) and u(L2 ) = 0.1(20 2 From this we deduce that L1 > L2 i↵ < 35 .

100

2)

+ 0.9(10

25

2 ),

since

1

> 10.

When = 0.01, 1 = 100 and 100¡10↵ < 200, 50 < 5↵  100. Hence u(L1 ) = 0.4u(10↵) + 0.4u(5↵) = (0.4 ⇥ 1) + 0.4(0.1↵ 0.0025↵2 ) and u(L2 ) = 0.1u(10↵) + 0.9u(5↵) = (0.1 ⇥ 1) + 0.9(0.1↵ 0.0025↵2 ). 2

u(L1 ) < u(L2 ) if 0.3¡0.5(0.1↵ 0.0025↵2 ). I.e. if g(↵) = ↵8 5↵ + 30 < 0. But for 10¡↵ < 20, 8g(↵) = (10

↵)(20

↵)

10↵
+
40.
So
g(↵)
<
0
for
↵
2
[10,
20].
We
conclude
that
L2
is
preferred
to
L1
and
u(L2 )
=
0.1
+
0.09↵

0.0025↵2 ,
which
is
minimised
[over
↵
2
[10,
20]]
when
↵
=
10
or
↵
=
20.
Direct
calculation
of
u(L2)
for
these
two
values
of
↵
leads
to
the
required
result. 73

Question 25 1. u(L) = (p ⇥ 0.67) + ((1

p) ⇥ 0.46) > 0.59 when p > 0.62

2. u(L0 ) = (p2 ⇥ 0.72) + (2p(1 p > 0.59]

p) ⇥ 0.32) > 0.59 if 0.14p2

p) ⇥ 0.59) + ((1

0.54p + 0.27 < 0 [i.e. if

Question 26 Let X be the number of people out of the 5 who win the lottery. Then X ⇠ Bin 5, 23 . The reward to each 1 person is X)) = 4000X 10000. The expected utility for each person is therefore Pthen 5 (10000X 10000(5 1 j = 0.5 u(4000j)P (X = j) = 35 {u(0) + 10u(4000) + 40u(8000) + 80u(12000) + 80u(16000) + 32u(20000)}. Apart from u(0) and u(20000), we do not know any of these utilities exactly, but if we assume that the utility function is concave we have: u(4000)

0.4u(2500) + 0.6u(5000)

u(8000)

0.4u(5000) + 0.6u(10000)

u(12000)

0.6u(10000) + 0.4u(15000)

u(16000)

0.8u(15000) + 0.2u(20000)

Hence E(utility) 0.89, which is greater than the utility [= 0.85] of each person’s current assets, so it is worth each person taking part in the lottery.

Question 27 a. For xi = m, m + 1 . . . we have P (Xi = xi ) =

xi 1 m 2

⇡(✓|x) / ✓mn+↵

1

✓m (1 (1

✓)xi P

✓) P Hence the posterior distribution of ✓ is Beta(mn + ↵, xi mean and variance.

m.

xi mn+

By Bayes: 1

mn + ). Hence you can find the posterior

P

P

a b. ⇡(✓|x) / ✓n e ✓ Pxi ✓a 1 e a✓/b = ✓a+n 1 e ✓( xi + b ) . Hence the posterior distribution of ✓ is Gamma a + n, xi + ab and we can then find the posterior mean and variance.

Question 28

1 bab for ✓ > a and 0 < x1 , . . . , xn < ✓ ✓n ✓b+1 These two conditions on ✓ can be expressed as ✓ > max{a, max{xi }}. The result follows immediately. ⇡(✓|x) /

Question 29 ⇡(✓|x) / exp

1X (yi 2

✓xi )2

Now complete the square in ✓ and show that the posterior distribution of ✓ is normal with mean variance

P1 2 . xi

P

xi y i x2i

and

Question 30 Let M be the random variable representing the smaller amount, m, and let X and Y be the amounts in your envelope and the other envelope respectively. For any m, we have: P (X = m|M = m) = P (X = 2m|M = m) = since the two envelopes are allocated at random.

Distribution of this74 document is illegal

1 2

The posterior distribution of M given we observe X = x is given by: P (M = x|X = x) = and P M = x2 |X = x = 1

P (X = x|M = x)⇡(x) P (X = x|M = x)⇡(x) + P X = x|M =

P (M = x|X = x) =

⇡ ( x2 )

⇡(x)+⇡ ( x2 )

x 2

⇡

=

x 2

⇡(x) ⇡(x) + ⇡

x 2

.

⌘ x ⇣ x P Y = |X = x + 2xP (Y = 2x|X = x) 2 2 ⇣ ⌘ x ⇡ x2 + 4⇡(x) x x = P M = |X = x + 2xP (M = x|X = x) = 2 2 2 ⇡(x) + ⇡ x2

E(reward|swap) = E(Y |X = x) =

You should swap () E(reward|swap) > x () 2⇡(x) > ⇡

x 2

.

Question 31 ✓)2 = c2 var(X) + (c

a. dc (x) = cx =) R(✓, dc ) = EX|✓ (cX dc (x) c2 ✓ > ✓ = d1 (x) when c > 1.

1)2 ✓2 = c2 ✓ + (c

1)2 ✓2 . We see that

b. Under the square error loss the Bayes Rule for estimating ✓ is the posterior mean. The posterior density of ✓ is ⇡(✓|x) / e ✓ ✓x ✓↵ 1 e ✓ . Hence the posterior distribution of ✓ is Gamma(↵ + x, 1 + ) and E(✓|x) = is dB (x) = ↵+x 1+ . ⇣ We have R(✓, dB (x)) = EX|✓ ✓ Bayes Risk of ⇡ is: E⇡

↵+X 1+

1 ✓+( ✓ (1 + )2

⌘2

⇣ = varX|✓ ↵+X + ✓ 1+

↵)2 =

⇣↵ 1 + (1 + )2

↵+✓ 1+

2

⌘2

=

↵+x 1+ .

✓ (1+ )2

⌘ var(✓) =

Hence the Bayes Rule

+

( ✓ ↵)2 . (1+ )2

Hence the

↵ (1 + )

Question 32 [This material wasn’t covered in lectures.]

Question 33 R(✓, dc ) = EX|✓ (cX ✓)2 = c2 var(X) + [(c where = 1✓ . We therefore get: R(✓, dc ) = c2 ✓(✓

1)✓]2 . For the geometric distribution var(X) =

1) + [(c

1)✓]2 = [c2 + (c

1)2 ]✓2

1

2

= ✓(✓

1),

c2 ✓

Now: R(✓, dc ) = R(✓, d1 ) = (c2

1)✓(✓

1) + [(c

1)✓]2 > (c2

1)✓(✓

1) > 0

for all ✓ > 1 when c > 1. Hence dc is inadmissible for c > 1. x 1

When ✓ has a prior P.D.F proportional to ✓12 you can find ⇡(✓|x) = x(x+1) 1 1✓ by Bayes Theorem ✓3 [and some integration to get the normalising constant]. You can then find the Bayes Rule by calculating the expected posterior mean of ✓: ✓ ◆ Z 1 x(x + 1) 1 x 1 E(✓|x) = 1 d✓ ✓2 ✓ 1 which [after an obvious substitution] gives E(✓|x) = x + 1. However, it is easier to note that the prior distribution of is UR nif orm[0, 1] and so the posterior P.D.F. of 1 is x(x + 1) (1 )x 1 . We then calculate E(✓|x) = E 1 |x = 0 x(x + 1)(1 )x 1 d = x + 1.

Question 34 [This material wasn’t covered in lectures.]

Question 35 P P 2 R(✓, dc ) = EX|✓ c Xi2 ✓ = var c Xi2 + [(cn 1)✓]2 = [2c2 n + (cn 1)2 ]✓2 , which is minimised with 1 respect to c when c = c0 = n+2 . ⇣ P ⌘ m 1 exp( Let = 1✓ . Then ⇡( |x) / n/2 exp 2 x2i ↵). So the posterior distribution of is: ✓

n 1X 2 m + ,↵ + xi 2 2

The Bayes Rule for estimating ✓ is E(✓|x) = E(1/ |x) = As ↵ ! 0, dB (x) becomes becomes inadmissible.

P

x2i 2m+n 2

=c

P

x2i with c =

P 2↵+ x2i 2m+n 2

1 2m+n 2

◆

[exercise].

6=

1 n+2

unless m = 2. Hence the Bayes Rule

Question 36 If 12 < ✓ < 32 we find, on folding back, that choosing box A gives expected utility 2✓ + max 2✓ , 3 43✓ and that choose box B gives expected utility ✓+2 4 . Hence the utility of box A is less than that of box B if and only if: ⇢ ✓ 3 3✓ 1 ✓ max , < 2 4 2 4 By plotting the three relevant lines, we see that this will always be the case and we therefore choose box B. If ✓ > 23 , we find that the utilities of choosing the two boxes are both ✓ and so it doesn’t matter which box we choose.

P (R from A) = P (R from A|W = A)P (W = A) + P (R from A|W = B)P (W = B) =

P (Correct) = P (W = A|R from A) =

P (R from A|W = A)P (W = A) = P (R from A)

✓ 1 ✓ 1+✓ + = 2 4 4

1 2 ·✓ 1+✓ 4

=

2✓ 1+✓

Question
37 Let
F
denote
a
faulty
machine,
R
denote
the
number
of
alarms
that
ring
and
H
denote
overhaul. With
perfect
information,
the
expected
utility
of
H
is
0.8
⇥
800
+
0.2
⇥
700
=
780
and
the
expected
utility
of
H
is
0.8
⇥
1000
+
0
⇥
700
=
800. Hence
the
expected
value
of
perfect
information
is
940

max{780,
800}
=
140.
Since
each
scanning
device
costs
50,
we
can
take
n
=
0,
1,
2. 1. For n = 1 show that P (R = 0) = P (R = 1) = 0.5 2. For n = 2 show that P (R = 0) = 0.29, P (R = 1) = 0.42, P (R = 2) = 0.29 Hence calculate the conditional probabilities of F and F given the possible values of R for the cases n = 1 and n = 2. Taking into account the costs of the scanning devices, the optimal decision is n = 1 [utility = 812]. One should use one scanning device and then overhaul if the alarm sounds. [See the below diagram, shamelessly copied form Lynda’s notes.]

Question
38 Let
M
=
minor
fault;
S
=
serious
fault;
T M
=
examination
indicates
M ;
T
S
=
examination
indicates
S.
We
have: P (M ) = 0.9 P (T M ) = 0.8 P (M |T M ) = 36/37 P (M |T S) = 9/13

P (S) = 0.1 P (T S) = 13/15 P (S|T M ) = 1/37 P (S|T S) = 4/13

Folding back the decision tree, we see that the detailed examination should be selected if 7 c < 50 i.e. if the cost of the examination is less than £140, 000.

13 50

+c <

4 10

i.e. if

n = 1: P (R = 0) = P (R = 0|F )P (F ) + P (R = 0)|F )P (F )

P (F |R = 0) =

P (R = 0|F )P (F ) P (R = 0)

&c.

n = 2: P (R = 0) = P (R = 0|F )P (F ) + P (R = 0|F )P (F )

P (R = 1) = P (R = 1|F )P (F ) + P (R = 1|F )P (F )