3.2 Vector Space Properties
13. IV = Ix: x =
X, ] . [ Xl
=
XI
-3x~.
20. IV
~
{w, w
~ ~ , [
or Rtf
167
'aay «al "mbe,}
].
.'"2 any real number}
~
14. W = {w: w = [
].
15. Il' = (u: u = [ : ]. 16. Il'
= {x: x = t [ ~
17.1l'={X:x=
].
b any real number}
C
+(1
21. IV
~ OJ
~
lu, " c
~
~
a' + b' +0'
[ : ].
~
I ,"d
OJ
In Exercises 22-26. give a set-theoretic description of the given points as a subset IV of R 2 .
I any real number}
22. The points on the line
a] , , [ b . a-+b-=4j
X -
2)' = I
23. The points on the ,r-axis 24. The points in the upper half-plane 25. The points on the line)" = 2
In Exercises 18-21. interpret the subset IV of RJ geometrically by sketching a graph for IV.
on ..
= I
•
18. IV
19. IV
l ~ [:J
~ {~ ~ ~ ,
~ I"
[
26. The points on the parabola y =
x::!
In Exercises 27-30. ghe a set·theoretic description of the given points as a subset IV of R3 .
a > OJ
x,
27. The points on the plane
~ -x, -
X .;.. -' -
2;: = 0
28. The points on the line ....ith parametric equations x = 21. y = -31. and;: = t 29. The points in the y;:-plane
h,j
30. The points in the plane ). = 2
VECTOR SPACE PROPERTIES OF R n Recallihat R" is the se.t..of aU n-dimensional \ectors with real components:
'" "
'f-
'> R~
x. If x and .y
R~ geo-
XI.X~ ••••• X .. real numbers}.
[
r~
f
X, ]
'7 .
= {x: x =
are elements of W with
X, ]
[
", ]
'~:: andY~::' [
168
Chapter 3
The Vector Space Nil
A~ -YftMJ A ,. ~1"'M.,(.,J..
then (see Section 1.5) the \ ector x
~'0~ ~6'1 ~~.r.?~;~.4tdf
.l'W ta'I1 !( M 01
@It b(
_'j'
is defined by
[ x, _ y, x-y=
~~
]
X2+)2 x.
.
+ ).
YVlMiA.4d I,.. :-~.w., . de fi ne d to be .. '1 and if a is a real number. then the vector ax IS
1hi IJU;(,.n ~ ,1M.
[
1<1M.(.{~~WIo~ ,,/1,.._..
ax=
1~1AJ"" . - I ..... ~~IN,
ax, ]
a~2. ax"
.
In the context of R~. scalars are always real numbers. In panicular. throughout thl~ chapter. the tenn scalar always means a real number. The following theorem gives die anthmeiic propenies of \'ector addition and scalar multiplication. Note that the statements in this theorem are already familiar from Section 1.6. which discusses the arithmetic properties of matrix operations (a vector in R n is an (n x I) matrix, and hence the propenies of matrix addition and scalar multiplication listed in Section 1.6 are inherited by vectors in R"), ~ As we will see in Chapter 5. any set that satisfies the properties of Theorem 1 is called a "trtor space: thus for each positi,-e integer n, R~ is an example of a vector space
-
-ltI£olU \\ 1 If x. y. and z are vectors in hold,
~J1~~
~
and a and b are scalars. then the following propenie:. \Wo~(" ;0 ~
(el) x + y i, io R". (e2) oxisinR~.
~
nIl,
( Properties ofadditiOIl:
~1MW. tH.;1w •
~. ~1l ~1-i", ItJ
.rf /) '11 . ['1.1 ~ J \ If , e..) lit . J I S 1
') ~ t
R~
Closure properties:
i/.ut.n wrur W/n M
£~11
v -oftn '<""1«
'81
II vO"WJtI I "1 "'>1m // r) --'-"\ ~I"..t~ vf if'
(81) (a2)
x + y ~ y + x. X..j.. ()'
+ z) =
(x
~y)...j..
z.
(a3) R It contains the zero vector. 8. arid x ~ 8 = x for all x in R", (a4) For each "ector x in R". there is a vector -x in R" such that x + (-x) == 8. Properties ofscalar multiplication:
(ml) a(bx) = (ab)x. (m2) a(x + y) = ax + ay. (013) (a
+ b)x = ax + bx.
(mol) Ix = x for all x In R",
•
Subspaces of R" In this chapter we are interested in subsets. \V, of R" that satisf) all the properties of Theorem I (with R" replaced by \V throughout). Such a subset W is called a subspou
3.2 Vector Space Properties of RIO
169
OI\IGI W Of- tt IGttEI\-DI \ \E~fIO~ALfPACFJ In addition to Gtassmaon (see Section 1.7). Sir William Hamilton (1805-1865) also envisioned algebras of /I-tuples (which he called po/}plets). In 1833. Hamilton gave rules for the addition and multiplication of ordered pairs, (a. b), which became the algebra of complex numbers, z = a + hi. He searched for years for an extension to 3-tuples. He finally discovered, in a flash of inspiration while crossing a bridge, lhatthe extension was possible if he used 4-lUples (a. b. c, d) = a + hi + cj + dk. In this algebra of quaternions, however, multiplication is not commutative; for example. ij = k. but ji = -k. Hamilton stopped and carved the basic formula. i 2 = j2 = k 2 = ijk. on the bridge. He considered the quaternions his greatest achievement. even lhough his so-called Hamiltonian principle is considered fundamentallo modem physics.
iIhroughout Ihis tion and scalar from Section or in RIO is an mUltiplication
~
rTheorem I is III \eclor space. fig properties
,1 /
/1
,
Figurf' 3.5
of R". For example, consider the subset W of R) defined by
-,. &
IV = {so x = [
,.
\4-. as a subset of Rl
TllIL\I~t-\\
2
~
].
x, aod x, "aI n"mbe,,}.
Viewed geometrically, 1V is the .1")·-plane (see Fig. 3.5). so it can be represented by R2. Therefore. as can be easily shown. W is a subspace of R 3. The following lheorem provides a convenient way of delermining when a subset IV of R" is a subspace of R". A subset IV of R" is a subspace of R" if and only if the following condilions are mel: (51)" The zero vector. 8. is in U'.
'.lM 1"~1"'1
~ Ff u, il:l (I ~ Proof
o1JrOlA~ 1).'tl4 i.l.
-x)=8.
• properties of a subspace
(52) x
+ y is in W
whenever x and " are in tv.
(53) ax is in lV whenever x is in 1V and a is any scalar. Suppose that lV is a subset of R" Ibat satisfies conditions (sIHs3). To show that 1V is a subspace of R". we must show that the 10 properties of Theorem 1 (with R" replaced by IV throughout) are satisfied. But properties (al). (a2). (ml). (m2). (mJ). and (m4) are satisfied by e\'ery subsel of R- and so hold in 1V. Condition (aJ) is satisfied by 1V because the hypothesis (s I) guarantees thai 8 is in 1V. Similarly. (c 1) and (c2) are given by the hypotheses (s2) and (s3). respecth'ely. The only remaining condition is (a4). and we can easily see that -x = (-l)x. Thus if x is in W. then. by (s3). -x is also in W. Therefore. all the conditions of Theorem I are satisfied by 1V. and 1V is a subspace of R". For the con\erse. suppose 1V is a subspace of R". The conditions (a3). (e1). and (c2) of Theorem 1 imply that properties (sl). (s2), and (s3) hold in IV. • The neXI example illustrates the use of Theorem 2 to verify that a subset IV of R" is a subspace of R". "TIle usual stalemem orThcorem 2li.l. only conditions (s2) and (.3) bul as~ume. thaI the subset IV is nonemplY. Thus (51) replaces the assumplion thal IV is nonempty. The twO \ersions are equivalenl (su Exercise 34).
170
Chapter 3
E\. "'PI
The Vector Space R"
r 1 Lei IV be lhe subset of RJ defined by IV == fx; x = [
:~ ].
X~.
XI =.\"2 -
.\"1 and .\') any real numbers}.
XJ
Verify that IV is a subspace of R J and give
Solutioll
geometric interpretation of IV.
il
To show that IV is a subspace of R J , we must check that properties (51)-(s3) of Theorem 2 afC satisfied by IV. Clearly the zero vector, 6. salisfies the condition .t1 = X2 - .\']. Therefore. 8 is in IV. showing that (sl) holds. Now let u and \' be in IV. where
=
U
['" ]
and \. =
112
"3
[" ] t'!
•
'3
and let a be an arbitrary scalar. Since u and \' are in 1V. 1/1
The sum u
=
112 -
and
113
VI
=
t'2 -
"
t.'3·
+ v and the scalar product au are given by u + \' =
1
'" + v, /12 + Vz [ Ilj + U3 J
and au =
au, ] (/l/2
[
.
all)
To see Ihal u + \' is in IV. note that (1) giles "l ..... t'.
=
(112 -
If)
+ (1'2
- 1'3)
= (uz
+ t'2) -
(113 T V3)·
Thus if the components of u and " salisf) the condition XI = X2 - XJ. then so do the components of the sum U T '>. This argument shows that condilion (s2) is mel by IV. Similarl). from (I).
../
alii
=
a(lll - IIJ)
=
so au i!'i in IV. Therefore. nr is a sub~pace of R 3. Geometrically. IV is the plane whose equation is.r - y
:=y-x
')
.(0. I.
(
, x
Figure 3.6
/
.'
alll - allJ.
I)
, ·(\.1.0)
A portion of t~ plane.r - y +.: = 0
+.: =
0 (see Fig. 3.6).
•
3.2 Vector Space Properties of R ff
171
Verifying that Subsets are Subspaces Example 1 illustrates the typical procedure for verifying thaI a subset IV of R" is a subspace of RI!, In general such a verification proceeds along the following lines: ~
Verifying that W Is a Subspace of R n '"Theorem
kX1 -
Step J.
An algebraic specification for the subset \V is given, and this specification serves as a test for determining whether a vector in R" is or is not in IV.
Step 2.
Test the zero vector. 9, of R" to see whether it satisfies the algebraic specification required to be in W. (This shows that IV is nonempty.)
Step 3.
Choose two arbitrary \"e<:tors x and J from IV. Thus x and yare in R", and both vectors satisfy the algebraic specification of \Y.
Step 4. Step 5.
Test the sum x .... )' to see whether it meets the specification of IV,
.l'J.
(I,
For an arbilfary scalar, Q. test the scalar multiple ax to see whether it meets the specification of W.
The next example illustrates again the use of the procedure described above to verify that a subset lV of R" is a subspace. E'..... '\I'I' l
lei IV be the subset of R J defined by
IV .0
do t:he
~ I>' ~ X
x,
[ :: ].
~ 2.". x, ~ 3x,.
x, "'y ""I numbe,) .
llel by lV.
Verify thai IV is a subspace of R J and give a geometric interpretation of IV.
Solution :;.6).
•
For clarity in this initial example. we explicitly number the five steps used 10 verify that IV is a subspace. I. The algebraic condition for x to be in W is
•
X1
t/
=
2Xt
and
XJ
= 3XI.
H)
In \\'oreis. x is in W if and only if the second component of x is twice the first component and the third component of x is three times the first. 2. Note that the zero vector, 9, clearly satisfies (4). Therefore, 8 is in W. 3. Next, let u and v be two arbitrary vectors in W:
U
=
[ ] "ndV~[::] '"
1/1
"J
172
Chapler 3
The \'ector Space R"
Because That is.
U
and v are in W. each musl satisfy the algebraic specification of \V = 2uI
and
u)
tI:2 = 2vI
and
v)
1/2
= 3// 1
(Sal
= 3vl.
(5b]
4. Next, check whether the sum, U + v, is in W. (That is, does the veclor u + \ satisfy Eq. (4)1) Now. the sum u + v is given by
u,+u,] u+v=
[
112+ti2 1I:1+ V)
By (5a) and (5b). we have U2
+ Vl = 2(ul
+ vd
and
(U)
+ v) = 3(//1
+ Vt).
Thus u + v is in W whenever u and v are both in 1V (see Eq. (4».
5. Similarly. for any scalar a. Ibe scalar multiple au is given by au, ] au =
aU2
[
.
au,
Using (5a) gives aUl = a(2111) = 2(a/ll) and aUJ = 0(3/11) = 3(OUl). Therefore. au is in W whene\'er u is in W (see Eq. (4)).
:u Thus. by Theorem 2. \V is a subspace of RJ . G....:.?me.!!!cally. W is a line-lhrough the origin with parametric equalions x =
XI
y=2x 1
:.= 3.1'1.
•
The graph of the line is given in Fig. 3.7.
Exercise 29 shows that any line in three-space through the origin is a subspace of R J • and Example 3 of Section 3.3 shows Ibat in three-space any plane through the origin is a subspace. Also note thai for each positive integer n. R" is a subspace of itself and {OJ is a subspace of R". We conclude this section with examples of subsels that are nOI subspaces.
Ij
E,-\\lPL[
3
Let \V be the subset of R J defined by
IV
~
(x,"
~ ~: [
l
Show that \V is not a subspace of R).
Ex, ,"dx, any real numbe,,}.
3.2 Vector Space Properties or R"
tication of
w.
173
"
(5a)
,,, (I. 2. 3) ,, ,,,
(5h)
e vector u
+v y
r
,,:~.
---------. (I. 2. 0)
x Figun 3.7 A geometric representation of the subspace IV (see Example 2)
Solution
To show that IV is not a subspace of R', we need only verify that at least one of the properties (~IHs3) of Theorem 2 fails. NOle Ihal geometrically IV can be interpreted as the plane:; = I, which does not contain the origin. In other words, the zero vector. 8, is not in IV. Because condition (sl) of Theorem 2 is not mel. IV is not a subspace of R', Although it is nOI necessary to do so. in this example we can also sho,"" that both conditions (s2) and (s3) of Theorem 2 fail. To see this. let x and y be in IV, where
iiJUd. There-
x=
[ x, ]
"7
~ghthe
ond'~[~,:l
Then x + ~' is given by X, -
x+~'=
•
[
,., ]
X1~Y1
'
In particular, x - ~' is nOI in W _ because the third component of x + Ydoes not have the value 1. Similarly.
I~ubspace of Ih the origin ,f 1l~lf and that are not
ax =
So if a E\A\\PLE ,I
[
ax, ] a;1 .
•
i= I. then ax is not in IV.
Let IV be the subset of R 2 defined by \V
= Ix: x = [ :: ].
Xl and
Demonstrate that IV i<; not a subspace of R 2 •
Xl
any integers}.
174
The Veclor Space R"
Chaprer 3
In this case 8 is in IV. and it is easy to see that if x and }' are in IV. then so is x + y, If we set
Solution
= " =il:
x~[:]
=11: ~&'=I:I:
=- and a =
=J:I:
J/2. then x is in IV but ax is not. Therefore. condition (s3) of Theorem 2 is nOi • --met by IV. and hence IV is not a subspace of R2 ,
= x !..e Ji ~
r \A \\[JLI::: S
Let IV be the subspace of R2 defined by IV
= {x; x = [
X, ] '
X2
0 or X2 = OJ.
where either XI =
"'" , tct ". be
Show that IV is not a subspace of R2. Let x and y be defined by
Solution
x=[~]
,nd J~[~l
G,,"e.~
1lell" tet
Then x and)' are in IV, Bur
'=[:]
X+ J
is not in IV. so IV is nor a subspace of R2. Note that 8 is in Uf. and for any "ector x in IV and any scalar a. ax is again in IV. Geometrically, \V is the sel of points in the plane thai lie either on the x-axis or on lhe J.axis. Eilher of lhese axes alone is a subspace of R 2 • bul. as lhis example demonstrates. their union is not a subspace. •
"31
d.
a'
p,,,,,
~ ~
EUT:'1
EXERCISES
E.ter.:!<.e ~ I. f
In Exercises 1--8. IV is a subset of R~ consisting of"ec· lars of lhe fonn
Je-.;.."'n;Oor
7. IV = {x: xi +X2 = I} 8. IV = Ix: XjXl = OJ
:2.. =
.~ [ x,x, ]
In Exercises 9-17. IV is a subset of R 3 consisting of \cctors of the fonn
In each case detennine whether W is a subspace of Rl . If '" is a subspace. Ihen give a geometric descriplion of IV.
X~[::].
X
•
= 2x~J
I. IV = Ix:
.(1
2. IV = Ix:
Xl -
J. IV = 4. IV = 5. IV = 6. IV =
Gne3~
:1. let. aOC
2J
X2 =
{lo::
Xl
= X2
or
{x:
Xl
and
X2
{x:
Xl
= OJ
Ix: lxll
+ IXII
.tj
= -X2}
are rational numbers}
x,
In each case. detennine whethc,r IV is a subspace of R), If IV is a subspace. then give a geomclric description of IV. 9. IV = Ix:
X]
= lxl -
10. IV = Ix: X2 =X] = O}
:3.• =
11. IV = Ix:
XIX!
X2)
+Xl}
= x31
~~.•
=
LI
~"=L
3.2 Yectol' Space Properties of R"
~
'\: + y.
If
=
12. \V = {x: Xl = 2x 31
13. IV = {x: .If =.11 +.121 14. \V = {x: .12 = 01 15. \V = {x: Xl = 2Xl, .12 = -.Ill
em2isnot
•
16, IV = Ix: Xl = .12 = 2Xli
Prove that IV is a subspace of R3 . 19, Let IV be the subspace defined in Exercise 18. where
a~
[:
l
Give a geometric description for \V. 20. Let \V be the subspace defined in Exercise 18. where
a=Ul
f
-IOCX
;n
the plane p3Ce of
•
Give a geometric description of IV. 21. Let a and b be fixed VKtOrs in R3 , and let IV be the subset of Rl defined b) \V = {x: aTx = 0 and bTx =O}. Prove thaI W is a subspace of RJ • In Exercises 22-25. W is the subspace of Rl defined in Exercise 21. For each choice of a and b. give a geo~t ric description of IV.
~lOg
of
"eof RJ. ~nption
11a~[-J b~[-:]
n.a~Ul b~[i] ~.a~[:J b~m 15. a =
Ul ~ -n b
[
=
26. In R 4 , let x [I. -3. 2. I)T, Y [2, I. 3. 2jT. and z = [-3. 2. -1,4jT. Seta = 2 andb = -3. illustrate that the ten properties ofTheorero I are satisfied by x, y. z, a. and b. 27. In R 2. suppose that scalar multiplication were defined by
17. IV = {x: .12 =Xl =OJ 18. Let a be a fixed vector in RJ , and define IV to be the subset of Rl given by
IV = {x: aTx =OJ,
175
ax
~a[
X, ]
.12
~[
z"x, ] 2aX2
for e\'ery scalar a. Illustrate with specific examples those properties of Theorem I that are not satisfied. 28. Let IV = {x x = [ ::
l
.12
~ 01
In the statement of Theorem I, replace each occurrence of R~ with W. Illustrate with specific examples each of the ten properties of Theorem 1 that are not satisfied. 29. In R]. a line through the origin is the set of all points in R] whose coordinates satisfy XI = at, .12 = bt, and .13 = ct.whe~ r is a variable and u. b, and c are not aU zero. Show that a line through the origin is a subspace of RJ • 30. [f U and V are subsets of R". then the sel U + V is defined by U+V={x:x=u-,', uinU and ,'in VI . Prove that if U and V are sUbspaces of R". then U + V is a subspace of R". 31. leI U and V be sUbspaces of R". Pro'e thai the intersection. U n V. is also a subspace of R~. 32. Let U and V be the sUbspaces of Rl defined by U = Ix: aT x = OJ and V = Ix: b T" = 01. whe~
'~[i] ~d b~[J Demonstrate that the union. U U V, is not a subspace of R3 (sec Exe,rcise 18). 33. Let U and V be subspaces of R n . a) Show that the union, U U V. satisfies properties (sl) and (s3) of Theorem 2. b) If neither U nor V is a subset of the olher, show that U U V does not satisfy condition (s2) of
176
Chapter 3
The Vector Space R"
Theorem 2. [Hint: Choose vectors u and y such that u is in U but not in V and y is in V but not in U. Assume that u + \' is in either U or V and reach a contradiction.]
34. Let W be a nonempty subsel of R" thai satisfies conditions (52) and (s3) of Theorem 2. Prove that 8 is in Wand conclude that W is a subspace of R". (Thus property (s I) of Theorem 2 can be replaced with the assumption thaI W is nonempIY.)
EXAMPLES OF SUBSPACES In this section we introduce several important and particularly useful examples of subspaces of R".
The Span of a Suhset To begin, recall that if VI ...• , \', are vectors in R", then a vector y in R" is a linear combination of VI, ...• y" provided that there exist scalars al •... ,0, such that y = alYI
+ ... +a,v,.
The next theorem shows that the set of all linear combinations of subspace of R". Til rORE\\
1
If
VI •••.•
VI ••••• V,
Proof
VI •...• V,
is a
V, are \'ectors in R". then the set \V consisting of all linear combinations of is a subspace ofR".
To show that \V is a subspace of R", we must verify that the three conditions of Theorem 2 are satisfied. Now 8 is in W because
8 =OVI + ···+Ov,. Jext, suppose that)' and 1 are in \V. Then lhere exist scalmal .... , a" bl ..... b, such that ~'=
a!vl
+ ... +a,v,
and
l=bIVI+···+b,v,. Thus.
Y + 1 = (al + bdvi + ... + (a, + b,)v,. so Y+ 1 is a linear combination of VI •...• v,: that is,)' + 1 is in W. Also, for any scalar
c. cy = (cailvi
+ ... + (ca,)v,.
In particular, C)' is in W. It follows from Theorem 2 that W is a subspace of R".
•
If S = {VI t • • • • v, I is a subset of R" , then the subspace W consisting of all linear combinations of Vl •... V, is called the subspace spanned by S and will be denoted by t
SpeS) or Sp{VI .... ,V'}.
3.3 Examples of Subspaces
oil satisfies con-
av'
rmethat8isin e of R~. (Thus - aced with the
,
For a single vector \' in RI!. Spiv} is the subspace Spiv} = {a\': a is any real number}.
If v is a nonzero vector in R2 or R 3 • then Sp{\'} can be interpreted as the line determined by v (see Fig. 3.8). As a specific example, consider
,~[
Figure 3.8 SpIv}
t'Jples of sub-
l'
Then Spiv}
is a linear that
177
~ ~ {, [
J
i, ,ny ,,,1 numbe,}.
Thus Spiv} is the line with parametric equations
x = r )' = 2r
.. \', is a
z = 3r. 'inations of
Equivalently, SpIv} is the line that passes through the origin and through the point with coordinates J. 2, and 3 (see Fig. 3.9).
lflditions of
: l
" (I. 2. 3)
. b, such
,, ,, ,,
V
r
-~~------
:_
,:
J
.. (I. 2. 0)
x
Hgu" 39 ~ any
scalar
Sp
I[i ]I
If u and \' are noncollinear geometric vectors, then R~.
•
alllinear ,~noted by
,
Sp{u. v} = {au
+ bv: a. b any real numbers}
is the plane containing u and \' (see Fig. 3.10). The following example illustrates this case with a subspace of R 3.
178
Chapter 3
The Vector Space Rn ,.4
au/
,
, ,,
, ,,
(lu+bv~
,-
,, ,,
"
,,
,
oIe' •
• -------,"
\'
Figure 3.10
EAM"IPLI::: 1
,
"
bv
Sp[u. v)
Let u and v be the three-dimensional vectors
U~[n
V~[:J
and
, A
Detennine IV = Sp{u. v) and give a geometric interpretation of W. Solutio"
Let y be an arbitrary vector in R 3, where
y~ [~J Then y is in IV if and only if there exist scalars Xl and y =
That is, y is in
W
X2
such that (I)
Xlll +X2V,
if and only if there ex.ist scalars Xl and Yl
=
2Xl
)'2
=
Xl
)'3
=
+
Xl
such that
(2)
X2 2X2.
The augmented matrix for linear system (2) is
20"'] [ I
I
Y2
a
2
}'3
PtlYflCAL I\tPI\Eft~rrATIO~fOf'VECTOrv
,
Tho v"'"' space wo,k of
Grassmann and Hamilton was distilled and popularized for the case of R 3 by a Yale University physicist, Josiah Willard Gibbs (1839-1903). Gibbs produced a pamphlet, "Elements of Vector Analysis," mainly for the use of his students. In it, and subsequent articles, Gibbs simplified and improved Hamilton's work in multiple algebras with regard to three-dimensional space. This led to the familiar geometrical representation of vector algebra in terms of operations on directed line segmenlS,
~
3.3 Examples of Subspaces
179
and this matrix is row equivalent to the matrix
(I/2)Y,]
1 0
o o
[
,
0/2))'3
0
+ (l/2)YI -
J2
in echelon form. Therefore, linear system (2) is consistent if and only if (1/2).'1'1 (1/2))'3 = 0, or equivalently, if and only if
,.
)'1 -
•y x
13)
)'2 - (1/2»)',
1
\V =
x~2)'+;:=O
+
(4)
Thus It! is the subspace given by
"
Figure 3.11 A ponion of the plane
2n + )'3 = O.
}'2
Iy =
y, ] )'2
[
:)'1 -
2)'2 + Y3 = OJ.
(5)
n
It also follows from Eq. (5) that geomelrically \tI is the plane in three-space wilh equation :c - 2)'+:: = (see Fig. 3.11). •
o
The Null Space of a Matrix We no\\, introduce two subspaces that ha\e panicular relevance 10 the linear system of equations Ax = b. where A is an (m x n) matrix. The first of these subspaces is called the null space of A (or the kernel of A) and consists of all solutions of Ax = 8. (1 )
O£:t-!" 1110' I
Let A be an (m x n) matrix. The null spau of A (denoted X(A» is the set of \'ectors in R" defined by ."(A) = {x: Ax = 8.
(ll
x in R"}.
th~x§\lCh
In words. the null space consists of all that Ax is the zero vector. The next theorem shows that the null space of an (m x n) matrix A is a subspace of R".
£
lill
()I~1 \I ~ Proof
If A is an (m x n) matrix. then. '~(A) is a subspace of R"":'
--
TOSilOWlliat Ii. (Alis a subspace of R". we must verify that the three conditions of Theorem 2 hold. Let 8 be the zero vector in R". Then A8 =8.
'of ysicist,
mainly
(6)
and so 8 is in N(A). (Nole: In Eq. (6). the left 8 is in R" but the right 8 is in R"'.) Now let u and" be vectors in N(A). Then u and v are in R" and
's work
Au=8 and Av=8.
(1)
To see that u + v is in !'I(A). we must test u + v against the algebraic specification of N(A); that is, we must show that A(u +,"') -= 8. But it follows from Eq. (7) that A(u
+ v) -= Au + A\' -= 8 + 8 -= 8,
180
Chapter 3
The Vector Space R n
and therefore u
+ v is in N(A).
Similarly, for any scalar a. it follows from Eq. (7) that
A(au) = aAu = a8 = 9.
•
Therefore, au is in N(A). By Theorem 2. N(A) is a subspace of R n • EX,.\,\1PLf
2
Describe N(A). where A is the (3 x 4) matrix
A~[i Solution
3 1]
1 1
5
4
2
4
-1
.
N(A) is determined by solving the homogeneous system Ax = 9.
(8)
This is accomplished by reducing the augmented matrix [A 18]10 echelon form. It is easy to verify that LA 181 is row equivalent to
u n o
2
1
1 -2
o
o
3
0
Solving the corresponding reduced system yields Xl
= -2x) - 3x-t
X2
= -
X)
+ 2X4
as the solution to Eq. (8). Thus a veclor x in R 4 •
x
~ ~1 [
is in N(A) if and only if x can be wriuen in the form
-2X;-3X,] -x) +2X4
x=
[
.\")
[_2] [-3] =x, .
-I 1
+X4
2 0
.
4 0 1
where.\"3 and.\"4 are arbitrJry; thai is.
N(A)
~ ~ X; [ Ix x
-2 -1
1
o
]+x'[-~l
.\") and.\"4 any real numbers}.
..
3.3 Examples of Subspaces
f7Hhar
•
181
As the next example demonstrates. the fact that ...\ "(A) is a subspace can be used to show that in three-space e\'el) plane through the origin is a subspace.
L\.\\\PLE] Solution
Verify that any plane through the origin in R 3 is a subspace of R3.
t
The equation of a plane in three-space through the origin is
ax
-
+ by + c.;; = 0,
(9,
where a. b. and c are specified constants not all of which are zero. Now. Eq. (9) can be ~ritten as
Ax =8. where A is a (I x 3) matrix and x is in R3 :
II
i~
A
~ I.
b
cJ ood x = [
J
Thus x is on the plane defined by Eq. (9) if and only if x is in N(A). Since N(A) is a subspace of R3 by Theorem 4. any plane through the origin IS a subspace ~R3.
•
The Range of a Matrix Another important subspace associated with an (m x n) matrix A is the rallge of A. defined as follows.
OH1'ITIO'
2.
Let A be an (m x 11) matrix. The ra1lge of A (denoted 'R(A)] is the set of \'ectors in Rill defined by 'R(A)
= b:)' = Ax for some x in
R~J.
In words. the range of A consists of the set of all vectors y in R'" such that the linear system Ax = Y
•
is consistent. As another way to view R(A). suppose that A is an (m x n) matrix. We can regard multiplication by A as defining a function from W to Rm. In this sense. as x varies through R~. the set of all vectors
y = Ax produced in R'" constitutes the "range" of the function.
182
Chapter 3
The Vector Space R n
x II) matrix A has colu
We saw in Section 1.5 (see Theorem 5) that if the (m At, A 2 , .... A" and if
X~[J then the matrix equation Ax =y
is equivalent to the vector equalion XtAt
+ X2A2 + ... + xnA"
= y.
Therefore, it follows that R(A) = Sp{A t .A2..... All}'
By Theorem 3, Sp{AJ, A 2, ... ,All} is a subspace of R m . (This subspace is also called the columll space of matrix A.) Consequently, R(A) is a subspace of R m , and we ha\~ proved the following theorem.
+-J
Til I OIU- \\
~
If A is an (III x II) matrix and if R(A) is the range of A. then R(A) is a subspace of~. • The next example illustrates a way to give an algebraic specification for R(A).
I \, ,\PI r I
Describe the range of A. where A is the (3 x 4) matrix
A~[~ Solurion
1
3 1] 5
4
2
4
-1
1
.
Let b be an arbitrary vector in R 3.
b~
[:J
Then b is in R.(A) if and only if the system of equations Ax = b
is consistent. The augmented matrix for the system is
[AlbJ~[~
1
3
1
1
5
4
2
4
-1
::
b, ] '
3.3 Examples of Subsp8ces
183
which is equivalent to
4 has columns
[~
0
2
3
1
1
-2
0
0
0
b,-b, 2b 1 -b2
] .
-3bl +b2+ b 3
It follows that Ax = b has a solution [or equivalently, b is in 'R(A)} if and only if -3b l + b~ + bJ O. or b3 3b 1 - b2, where hi and b2 are arbitrary. Thus
=
7«A)
=
= Ib, b = [
::
]
3b 1 - b1,
= b, [
~ also called j.nd we ha\e
•
-
~ubspace
t 'CIAI.
~~' LJl"'Ao f/vJ ~tA
•. Jo~wl1
')
~
]
[
-:J
b, ,nd b, aoy
",1
n"m~,,).
-
The Row Space of a Matrix If A is an (m x n) matrix with columns AI. A 2..... A~. then we have already defined the column space of A 10 be Sp{AI.A2 ..... A~} .
'"
.""--
+ b,
In a similar fashion. the rows of A can be regarded as vectors ai, a2' ... , a.. in R~, and the row space of A is defined to be Sp{al.a2 ..... a.. }. For example. if
A=[:~~l then the row space of A is Sp{a\. all. where al
= II
2 3J and
82
= [1
0 1].
The following theorem shows that row-equivalent m8lrices have the same row space. TUEOI\(\\
b
Let A be an (m x n) matrix. and suppose thai A is row equivalent to the (m x 11) matrix B. Then A and B have the same row space. The proof of Theorem 6 is given at the end of this section. To illustrate Theorem 6, let A be the (3 x 3) matrix
1 -1 A= [
~
-:
:]
184
Chapter 3
The Vector Space RBy perfonning the elementary row operations R2 - 2R 1, R3 - RI . R 1+ R2•and R3 -2R: we obtain the matrix
J]
1 0
B~ [
0
1 2
o
0 0
By Theorem 6, matrices A and 8 have the same row space. Clearly the zero row of E
contributes nothing as an element of the spanning SCI, so the row space of B is Sp{b l , b: where bl ::: [1 0 3] and b2::: [0 I 2].
If the rows of A are denoted by 3]. 82. and 83. then Sprat. 82. 83}::: Sp{b]. b2J.
More generally. given a subset S ::: {"j ....• "", 1of Rn , Theorem 6 allows us to obtai.::. a "nicer" subset T ::: {WI •••.. Wk} of R" such that Sp(S)::: Sp(T). The next example illustrates mis. \ \ ,\PI Ie")
Let S ::: {VI, \'2, \'3, V4} be a subset of R 3. where
"I
~
[
:
l
v, = [ :
l ~ [-:l ~d v,
v, = [ _;
l
Show that there exists a set T = {WI. W2) consisting of two veclQrs in R3 such that
SpeS) = Sp(T).
Solutio"
Let A be the (3 x 4) matrix A = [VI, v2. vJ, V41:
that is.
A=[~
2
1
J
4
5
-5
]
~
.
-I
The matri:'l: AT is the (4 x 3) matrix
A
T _
_
I 32 51]
2 I
[2
5-1
and the row vectors of AT are precisely the vectors "f, ward to see that AT reduces to Ihe matri:'l: T _
B -
,
4-5
,'r
1 0 7]
0 0
1-3 0 0
[000
.
t
vf, and vI. It is straightfor-
3.3 Examples or Subspaces and R3-2R1.
uro row of B is 5p{b]. b:ll.
So. by Theorem 6. AT and B T have the same row space. Thus A and B have the same column space, where
B~[~
pt'u example
:]. l -I
, such that
U]
o
o
1
o o
~3
In particular. 5p(S) = Sp(T), where T =
w, ~ ~ us to obtain
185
{WI. W2},
aod W,
~
n
[
-;]
~
Proof of Theorem 6 (Optional) Assume that A and Bare row·equivalenl (m x n) matrices. Then there is a sequence of matrices A = AI.A1, ... ,Ak_I,A,t = B
such that for 2 :5 j ~ k, A j is obtained by performing a single elementary row operalion on Aj _ l . It suffices, then. 10 show that Aj_1 and A j have the same row space for each j. 2 :5 j :5 k. This means thai it is sufficient to consider only the case in which B is oblained from A by a single elementary row operation. Let A have rows 81 .... , a",; that is, A is the (m x n) mauix
a, a, A~
a•
.
where each ai is a (l x n) row vector;
a; = [ail ai2
...
a,~].
Clearly the order of the rows is immaterial: that is, if B is obtained by interchanging the jth and kth rows of A,
a,
~ght(or.
a. B~
, aJ
am
186
Chapter 3
The Yeetor Space R-
then A and B ha\ e the same row space because
Sp{al' .... a J . . . . . 8t- .... a",} = Sp{al' .... at- .... aj' .... a",}.
+ C Rj
Next. suppose that B is oblained by performing the row operalion Rt
on A; thu~
a, aj
B= at
+ C8j am
If lhe vector x is in Ihe row space of A. then there exist scalars bl ..... b". such that
x = b18, + ... .,-bJaJ + ... -b t 8j; _
...
+h"'a....
10
The vector equation (10) can be rewritten as
+ (bJ -
x = biB. _ ...
cbt)a J
_ ••.
+ bj;(llt -
ca J )
... •••
+ b",a",.
and hence x is in the row space of B. Conversel)'. if the \'eetor)' is in the row space of B, then theTe exist scalars d., ' ... d", such that
.Y = dl81
+ ... + djaJ + ... -dt(at + eaj) +, .. + dill&"'.
(1~
But Eq. (12) can be rearranged as y
= dial
+ ···+(dJ +cdt)aj +···-dt84 + ... + d",8", ,
(U,
so Yis in the row space of A. Therefore, A and B have the same row space. The remaining case is the one in which B is obtained from A by multiplying the jth row by the nonzero scalar c. This case is left as Exercise 54 at the end of this section.
') EXERCISES Exercises 1-11 refer to lhe \'eclOl"S in Eq. (14).
a
~[
_:
l
b
~[
-:
l
,~[
-: l C141
In Exercises 1-11. either shoy. that SpeS) = R~ or gi,-e an algebraic specification for SpeS). If Sp(S) #= R~. then give a geometric description of SpeS). I. s ~ {al 2. S ~ {bl 3. s ~ I.) 5. S = lao dl
8. S = {a. b. d}
9, S= {b,c,d}
10. S = lao b. e}
11, S = {a. c, e} Exercises 12-19 refer (0 the \CClors in Eq. (15).
d~[~l·~[a
4. S = {a. b}
7. S = {b. e}
6. S = lao cl
[J w~ -:J x~ J y~ [=n ,~U] v~
[
[
(IS)
3.3 Examples or Subspaces
l .. on A; thus.
In Exercises 12-19. either show that Sp(S) = R) or give an algebraic specification for SpeS). If Spes) "1= R). then gi\'e a geometric description of SpeS).
In Exercises 26-37. give an algebraic specification for the null space and the range of the given matrix A.
12. s= (vI
13. S= (wI
26. A =
14.S=(v.wl
IS.S=(v.xl
16.S=I\'.w.xl
17.S=(w,x.zl
IS.S=(v.w.z}
19.5=(w,x.YI
20. Let S be the set given in Exercise 14. For each vector given below. determine whether the vector is in SpeS). Express those vectONi lhilt are in SpeS) as a linear combination of "and w.
uch that
J ,) [~ ]
.) [ : ]
b) [
d)[:]
')[-~]
(lOj
( II)
space of
0[:]
21. Repeat Exercise 20 for the set 5 gh en in Exertill
cise 15.
H. Determine which oflhe \ectors listed in Eq. (14) is
[ 1-2 ] -3
,.mgthejth seCtion.
A~[:
:]
23. Determine which of the "ectOT'llisted in Eq. (1-1) is
l
24. Determine which of the vectors listed in Eq. (15) is in the null space of the matrix
A = [-2
15).
J ( 15)
I
I).
25. Determine which of the vectors listed in Eq. (15) is in the null space of the matrix
A =
[
1-I 0]
2 -I
I
3 -5 -2
:]
1 -I 30.A= [ 2-1
32.A~ [i
;]
I -,
34. A = [
37. A = [
']
5
1
7
0
I 2
2 5 I 3
[-I 3] 2 -6
31. A =
1 2 '] [ 3 6 4
33A=[~~]
2 -3
36.A=[ -;
=
29.A=[~~]
28.A=[: ~]
o -I 1
2
2
2
[
I 2 3]
35.A=131 2 2
10
]
:]
38. Let A be the matrix given in Exercise 26.
in the null space of the matrix
A~U :
27. A
6
in the null space of the matrix (13
187
,
a) For each "ector b that follo....s. determine .... hether b is in 'R(A).
b) Ifb is in "R.(A). then exhibit a vector x: in R 2 such that Ax: = b. c) If b is in R(A). then write b as a linear combination of the columns of A.
])b=[_:] JU)b: [ : ]
~b~[-:]
~b~[~]
')b~[_~] ~b=[:]
188
The Vector Space R n
Chapter 3
39. Repeat Exercise 38 for the matrix A given in Exercise 27.
45. Let 5 be the set of vectors given in Exercise 17. Exhibit a matrix A such that Sp(S) = R(A).
40. Let A be the matrix given in Exercise 34.
In Exercises 46--49. use the technique illustrated in Example 5 to find a set T = (WI. W2) consisting of two vectors such that SpeS) = Sp(T).
a) For each vector b that follows. detennine whether b is in R(A).
b) If b is in R(A), then exhibit a vector x in R 3 such that Ax = b. c) If b is in R(A), then write b as a linear combination of the columns of A.
i)b
~
U]
ii)b~
iii)b~[;] ,)b~
[J
i,) b~ [ ; ]
U]
,i)b~
[n
41. Repeat Exercise 40 for the matrix A given in Exercise 35.
46.s~!UHn[nl 47
48. h
49. S
2Xl -
2XI+
3X2
+
following. a) The (n x II) identity matrix b) The (II x II) zero matrix c) Any (II x n) nonsingular matnx A
X, ] : Xl, X2, X, real}.
X2+ 4x,
Exhibit a (3 x 3) matrix A such that W = R(A). Conclude that W is a subspace of R'.
51. Let A and B be (n x 11) matrices. Verify that N(A) nN(B) S; N(A + B). 52. Let A be an (m x r) matrix and B an (r x II) matrix. a) Show that N(B) S; N(AB). b) Show that R(AB) S; R(A).
43. Lei W={x= [ : : ]:3XI-4X2+2X3=O}.
53. Let W be a subspace of R n . and let A be an (m x II) matrix. Let V be the subset of R ffl defined by
X3
V = {y: y = Axforsomexin WI.
Exhibit a (I x 3) matrix A such that IV = N(A). Conclude that W is a subspace of R 3. 44. Let S be the set of vectors given in Exercise 16. Exhibit a matrix A such that Sp(S) = R(A) .
•
H_~ H-~ ]I ![~ H~~ H;H-:]I ~ l[ ~ H:H:J[-: ]I
~ \ [ -;
SO. Identify the range and the null space for each of the
42. Let
\V~{y~[ -Xl + 4X2 - 2X3
s
Prove that V is a subspace of Rffl • 54. Let A be an (III x II) matrix, and let B be obtained by multiplying the kth row of A by the nonzero scalarc. Prove that A and B have the same row space.
BASES FOR SUBSPACES Two of the most fundamental concepts of geometry are those of dimension and the use of coordinates to locate a point in space. In this section and the nex.t, we extend these notions to an arbitrary subspace of Rn by introducing the idea of a basis for a
3.4 Bases for Subspaces
~Urclse
,-
":t I
rated in Exling of [\\0
189
subspace. The first pan of this section is devoted to developing the definition of a basis. and in the latter part of the section. we present techniques for obtaining bases for the subspaces introduced in Section 3.3. We will consider the concept of dimension in Section 3.5. An example from R 2 will serve to illustrate the transition from geometry 10 algebra. We have already seen that each vector \' in R 2• v
~
l
[ :
(I)
can be interpreted geometrically as the point with coordinates a and b. Recall that in R2 the vectors el and e2 arc defined by
(J) ,
e,~[~]
.nd
Clearly the vector \' in (1) can be expressed uniquely as a linear combination of e, and e2: v = ael
])
e'~[~l
+ b~.
(2) 2
-h of tl1c
As we will see later. the set Ie" e2} is an example of a basis for R (indeed. it is called the lIatural basis for R 2). In Eq. (2), the vector \' is detennined by the coefficients a and b (see Fig. 3.12). Thus the geometric concept of characterizing a point by its coordinates can be interpreted algebraically as detennining a vector by itS coefficientS when the ,ector is expressed as a linear combination of "basis" vectors. (In facl. the coefficients obtained are often referred to as the coordinates of the vector. This idea will be developed fuoher in Chapter 5.) We rum now to the task of making these ideas precise in the context of an arbitrary subspace \V of R" .
main\[.
yi
(b, L
.m
•
~a. b)
,.
~~ ~~·a ~
XII)
tty
b) 'alare.
++
ae, Figure 3.12
\' = ael ..... be~
Spanning Sets lnd the IC1[Cend ~ for a
,
Let \V be a subspace of R". and let S be a subset of \V. The discussion above suggests that the first requirement for 5 to be a basis for \V is that each vector in \V be expressible as a linear combination of the vectors in S. This leads to the following definition.
190
Chapter 3
DEFt" n 10'
J
The Vector Space R"
Let W be a subspace of R n , and let S = {WI ••.. , wml be a subset of W. We say that S is a spanning set for \V. or simply that S spans W. if every ..ector w in W can be expressed as a linear combination ofveclors in S:
w=a\,,',
+··'TQ",W.,.
A restatement of Definicion 3 in the notation of the previous section is that 5 is a spanning set of W provided that SpeS) = W. It is evident that the set S = feJ, e2. e31. consisting of the unit vectors in R J • is a spanning set for R J • Specifically. if v is in RJ.
"=[:l then v = ael
f\
\\\NL 1
+ be2 + eeJ.
(.\1
The next two examples consider other subsets of RJ.
In R).let S = {UI. U2. u31. where
", = [
"1= [ - ]
-n,
and ", = [
~
l
Detennine whether S is a spanning set for R 3. SQll/tiol/
We must detennine whether an arbitrary vector ,. in RJ can be expressed as a linear combination of Ul. U2. and U3. in other words. we must decide whether the veclor equation X1Ul
+ X2U~ + X3U3 = V.
"
where v is the vector in (3), always has a solution. The vector equation (4) is equivalent to the (3 x 3) linear system with the matrix equation Ax = v.
where A is the (3 x 3) malrix A =
[UI' U2. U3].
[AI'l~[ -l
(5)
The augmented matrix for Eq. (5) is
-2
I
3
2
I
4
and this matrix is row equivalent to
:J
0
0
lOa+9b-7C]
I
U
0
4a+4b-3c
0
I
-a -b+c
.
3.4 Bases for Subspaces
We say 'torwin W
· U'.
191
Therefore. Xl
7c
4a+4b-3c
X2= XJ
+ 9b -
= lOa
= -a -
b
+
c
is the solution of Eq. (4). In particular, Eq. (4) always has a solution, so 5 is a spanning m~RJ. _
r
iSlhatSisa
= lel.e•. eJ}. IfvlsinR J •
E\\\\PLl
2
LetS = {\'I.Vl.VJJ be the subsetofR J defined by
v, =
(3)
Ul
v,
~ [~l
and v,
~
[
il
Does S span RJ ? of RJ.
Solutioll
Let \' be the vector given in Eq. (3). As before, the vector equation XlVI -l- Xl\'l
+ xJ\'J
= \'
(6)
is equivalent to the (3 x 3) system of equations Ax = \'.
where A = [vI. V2,
\'J]'
(7)
The augmented matrix for Eq. (7) is
[AI'I~ U o -I
Fd as a Ii near
Itt the \-eclor
o
I\'1is row equivalem to I 0
'"
equivaJenl
Eq. (5) is
7
-7
and the matrix [A
(5)
2
[
o o
7/2
1 3/2 0
0
:l
b/2 -0 ......
]
(I/2)b
.
-70 +2b-c
It follows that Eq. (6) has a solution if and only if -70 - 2b - c = O. In panicular, S does not span R3. Indeed.
SpeS)
~ {,.". ~ [ :
l
wh,.e - 7.
+ 2b + c = O}.
For example, the vector
w~[:] is in R3 but is not in SpeS); that is. w cannot be expressed as a linear combination of VI. ~,and~.
_
192
Chapter 3
The Vector Space R II
The next example illustrates a procedure for constructing a spanning set for the null space. N(A), of a matrix A.
E\ \ '\Ill r 3
Let A be the (3 x 4) matrix
1
) I]
2
4
1
A=U
54. -I
Exhibit a spanning set for ...'~(A). the null space of A.
Solutio"
The firs[ Step toward obtaining a spanning set for N(A) is 10 obtain an algebraic specification for N(A) by solving the homogeneous system Ax = 9. For the given matrix A, this was done in Example 2 of Section 3.3. Specifically,
.V(A) = {x: x =
- 2X 3-)X'] + 2x4 -XJ
XJ
[
Xl and x~ any real numbers}.
.
x,
Thus a vector x in .,r(A) is totally determined by me unconstrained parameters XJ and X.t. Separating mose parameters gives a decomposition of x:
-2-" -)x, -XJ.U4 x=
]
[
--2x, XJ
]
=
[
x3
X3
+
[ -lx, h 4] 0
0
X4
[ -I -2 ] =X3
I
0
X.t
[ -)2 ] +X4
0
.(8)
I
Let Ul and U2 be the vectors
-l
-2 ]
u, = [
~d
u,
~
[ -) ] ;.
By setting X3 = I and .1'4 = 0 in Eq. (8). we obtain u" so Ul is in X(A). Similarly, U2 can be obtained by setting X3 = 0 and X4 = I. so U2 is in Ar(A). Moreover, it is an immediate consequence of Eq. (8) that each vector x in }leA) is a linear combination of Ul and U2. Therdore. N(A) = Sp{Uj. U2}: that is. {Ul. til} is a spanning set for N(A). • The remaining subspaces introduced in Section 3.3 were either defined or characterized by a spanning set. If S = {VI ••••• "r} is a subset of R". for instance, men obviously S is a spanning set for SpeS). If A is an (m x n) matrix,
A = [At •...• AIlJ.
3,4 Bases for Subspaces
et for the null
193
then. as we saw in Section 3.3, (AJ, ... , An} is a spanning set for'R.(A), the range of A. Finally, if
A~[:~l where a/ is the ;th-row vector of A, then, by definition, {at. ... , am} is a spanning set for the row space of A.
Minimal Spanning Sets
1m
algebraic III" the given
If lV is a subspace of R", W i= {O}, then spanning sets for W abound. For example. a vector v in a spanning set can always be replaced by av, where a is any nonzero scalar. It is easy 10 demonstrate, however, that not all spanning sets are equally desirable. For example, define u in R 2 by
u~ [
l
:
The set S = {el,~, u} is a spanning set for R 2. Indeed, for an arbitrary vector" in Rl ,
-ers Xj and
,=
-3]~
[:l
\' = (0 - c)el - (b - c)el -r CU, where C is any real number whatsOever. But the subset {el. ell already spans R 1, so the vector u is unnecessary. Recall that a set {VI •...• v",} of vectors in R" is linearly independent if the vector equation
,',
C9,
XI\'I-r"'+X",Y", =9
has only the trivial solution Xl = ... = x'" = 0; ifEq. (9) has a nontrivial solution. then the set is linearly dependent. The set S = lei, el. ul is linearly dependent because
el-r e2- u =9. Our nexl example illustrates that a linearly dependent set is not an efficient spanning set: Ihat is, fewer vectors will span the same space. I Similarly.
er, it is an 'ination :g set for
I
\,.\\lPll ~
-
LetS={VI,Vl,Vj}bethesubsetofR 3 ,where
v, =
~h.aracter
[:l
v,
~
[ ;
J
'nd v,
~
[
:l
Show that S is a linearly dependent set. and exhibit a subset T of S such that T contains only two vectors but Sp(T) = SpeS).
Pb\iously
Solution
The vector equation Xl VI
+ X2V! + X3V, = 8
( 10)
194
Chapter 3
The Vector Space
R~
is equivalent to the (3 x 3) homoge,neous system of equations with augmented matrix
A=
[
'230] I 3 5 0
I
I
I
o
-I
1
2
o
0
.
0
Matrix A is row equivalent to
B=[~
n
in echelon form. Solving the system with augmented matrix B gives Xl
X2
= =
X3 -2X3.
Because Eq. (10) has nontrivial solutions. the set S is linearly dependent. Taking X3 = I. for example. gi\'es x. = I. Xl = -2. Therefore. \'1 - 2\'2
+ v)
= 8.
(11 )
Equation (11) allows us to express \'3 as a linear combination of VI and \'2: \'3 = -VI
+ 2\'2'
(!'\OIe lhal we could juSt as easily have soh'ed Eq. (11) for either \', or \'2.) It now follows thaI
Sp{v], \'2) = SP{VI. "2, v31. To illustrate. let \' be in the subspace SP{VI. vl. V3}:
v = o,v. ~1aking the
+ al"l + a3\'3.
+ 2Vl. yields \' = aivi + 0lV2 + a3( -VI + 2\'2).
substitution V3 = -\',
This expression simplifies to v = (al - a3)v. T (al
in panicular. \' is in SP{VI. because
\'2).
b,vI
+ 2a3}Vl:
Clearly any linear combination of \'1 and
\'2
is in Sp(S)
+ b2V2 = blv, + b 2\'2 + 0"3.
Thus if T = {VI. \'l}. then Sp(T) = SpeS).
•
The lesson to be drawn from Example 4 is thai a linearly dependent spanning set contains redundant infonnation. That is, if 5 = (w, .... , w,} is a linearly dependent spanning set for a subspace W, then at least one vector from S is a linear combination of the other r - I vectors and can be discarded from S to produce a smaller spanning set. On the other hand. if B = {\'" ... , VIII} is a linearly independent spanning set for W. lhen no vector in B is a linear combination of the other m - 1 \"ectO~ in B. Hence if a
3.4 Bases for Subspaces
195
vector is removed from B, this smaller sel cannOI be a spanning sel for \V (in particular. the veclor removed from B is in \V but cannot be expressed as a linear combinalion of the vectors retained). In this sense a linearly independent spanning sel is a minimal spanning sel and hence represents the mosl efficient way of characterizing the subspace. This idea leads to the following definition.
~nted matrix
Dft-1'\lrlO\, ,I
lei W be a nonzero subspace of R". A basis for W is a linearly independent spanning sel for IV.
Note Ihat the zero subspace of R". IV = {B}. contains only the vector B. Although it is the case that (B) is a spanning set for IV, Ihe set {B} is linearly dependent. Thus the concept of a basis is not meaningful for \V = {B}. gX3
= I,
Uniqueness of Representation
(II)
lei B = {"I. "2 ..... v p } be a basis for a subspace W of R~. and lei x be a veclor in W. Because B is a spanning set. we know that then: are scalars 01. a2 • •.. . op such Ihat
x=
follows
al"\
+ 02"2 + ... + 0l"'p-
12)
Because B is also a linearly independent set. v. e can show Ihal the representation of x in Eq. (12) is unique. That is. if we have any n:prescmalion of the fonn x = hi VI + h2"2 + ... + bp"p.lhen al = bl. 02 =~ ..... a p = hp- To eSlablish lhis uniqueness. suppose thai bJ, b2 . .... b p are any scalars such thai
x = h 1"l
+ h2 "2 + ... + bp"p,
Subtracting the preceding equalion from Eq. (12). we obtain
8=
(al - hl)"1
Then, using the factthat a2 - ~ =
O..... a p
-
+ (a2
- b2)V2
+ ... + (a p -
bp)"p,
vpJ is linearly independent. we see thai OJ -hi = O. bp = O. This discussion of uniqueness leads 10 the following
{"~I. "2 .... ,
remark.
Remork lei B = (VI. v2 ..... v p ) be a basis for W, where \V is a subspace of RI!. If x is in W. then x can be represented uniquely in terms of the basis B. Thai is. there are unique scalars al. 02 •.... 01' such that in SpeS)
x=
01"1 -02"2
+ ... -0,."1"
As we see later. these scalm are called Ihe coordinates of x with respect to the basis.
•
Examples of Bases II is easy 10 show that Ihe unil "ceIOTS
·,=Ul
"~
[!l
'Od
"~ [n
196
Chapter 3
The Vector Space R"
constitute a basis for R3 . In generaL the II-dimensional vectors el. e2 .... , e" form a basis for R", frequently called the natural basis. In Exercise 30, the reader is asked to use Theorem 13 of Section 1.7 to prove that any linearly independent subset B = {Vj. V2. ,')} of R J is actually a basis for R 3, Thus. for example, the vectors
"~Ul
'2
[il
~
ond
"~
[ : ]
provide another basis for R3 . In Example 3. a procedure for detennining a spanning set for N(A), the null space of a matrix A, was illustrated. Note in Example 3 that the spanning set {Ul. Ul} obtained is linearly independent so it is a basis for NCA). Oftentimes. if a subspace W of R" has an algebraic specification in tenns of unconstrained variables. the procedure illustrated in Example 3 yields a basis for W. The next example provides another illustration. EXA\\PLf 5
Solution
Let A be the (3 x 4) matrix given in Example 4 of Section 3.3. Use the algebraic specification of R(A) derived in that example to obtain a basis for R(A). In Example 4 of Section 3,3, the range of A was determined to be
R(A) = {b b = [
::
3b l
].
b l and b2 any real numbers].
b2
-
Thus b l and b2 are unconstrained variables, and a vector b in R(A) can be decomposed
" • b
~
If Ul and
[
::
3b 1 U2
-
]
b2
~ ~ [
]
+[
3b l
~,] =
-bl
b, [
~
]
3
+ b,
[
~]
.
(BI
-1
are defined by
u,
~ U]
ond U
~
[
-:J
then UI and U:l are in R(A). One can easily check that {Uj. U2} is a linearly independent set. and it is evident from Eq. (13) that R(A) is spanned by Ul and U2. Therefore, [UI. U:l} is a basis for R(A). • The previous example illustrates how to obtain a basis for a subspace W, given an algebraic specification for W. The last two examples of this seclion illustrate two different techniques for constructing a basis for W from a spanning set.
3.4 Bases Cor Subspaces
· e~ fonn a
D..\\\PlC
b
lei \V be the subspace of R4 spanned by the set
~
to prove Ihal for R J . Thus.
v, [ )
v, = [ _
S
= (\'\. \'2.
197
V). V4. v51, where
l ~ tl -J ~l ~ n v, [
"= [
'nd v,
[
Find a subset of S that is a basis for \V.
Solution
The procedure is suggested by Example 4. The idea is to solve the dependence relation XIVI-~~+~~+~~+~~=8
algebraic
and then determine which of the
Vj'S
v= then the augmemed matrix (V
'omposed
114)
can be eliminaled. If V is the (4 x 5) matrix [\'1. V2.
v).
vs1.
V4.
8J reduces 10
[~
0
-2
0
1
1
3
0
2
0
0
I
-I
0
0
0
0
f]
(IS)
The system of equations with augmented matrix (15) has solution
1
2x) -
=
X2
= -3x) - 2,rs
X4
?
Xs
XI
3 1.j""O
xs.
=
, -
b
(16)
0
where xJ and Xs are unconstrained variables. In panicular. the set S is linearly dependent. :\1oreover. taking XJ = J and Xj = 0 yields x\ = 2, X2 = -3. and X4 = O. Thus Eq. (14) becomes
2\'1 - 3\"2 Since Eq. (17) can be solved for
+ \') = 9,
(171
"J. \'J
= ~2"1
..j..
3"2.
it follows that v) is redundam and can be remo\ed from Ihe spanning set. Similarly. setting x) = 0 and Xs = I gh'es .1'\ = ~ L X2 = -2. and .1'4 = I. In this case, Eq. (14) becomes -\'1 ~ 2"2
+ V4 + v!
=
8.
198
Chapter 3
The Veclor Space R"
and hence V5
=
\'1 T 2\"2 -
\'4.
Since both V3 and Vs are in Splvi' V2. v~l. it follows (as in Example 4) thai VI, \'2. anc. \'~ span w. To see that the set {"J, V2. v~1 is linearly independent. nOie that the dependen,,;c relation Xl\'] T XlV!
is JUSt Eq. (14) with Eq. (18) reduces to
\'~
+ .t~\'4
= 8
and Vs removed. Thus the augmented matrix ["I. V2. v418]. for
1000] J 0 0
o o
[ o 00 01 00
.
which is matrix (15) with the third and fifth columns removed. From matrix (19). it i" clear that Eq. (18) has only the lrivial solution: so {\'l' \'2. \'~} is a linearly independent set and therefore a basis for W. • The procedure demonstrated in the preceding example can be outlined as follo\\-,,: I. A spanning set 51\'1 ... " \'",} for a subspace IV is ghen.
1. Solve the ,ector equation XlVI
+ ",+x.v", =8.
3. If Eq. (20) has only the trivial solution XI = ... = x'" = O. then 5 is a independent set and hence is a basis for tv.
Iinearl~
4. If Eq. (20) has nontrivial solutions. then there are unconstrained variables. For each:rJ thai is designated as an unconstrained \'anable, delete the \'ector \'j from the set S. The remaining vectors constitute a basis for w.
,
Our final technique for constructing a basis uses Theorem 7. TIIEORl\\ -
Proof
r \.\ \\PLE -
If the nonzero matrix A is row equivalent to the matrix B in echelon form, then the nonzero rows of B form a basis for the row space of A. B} Theorem 6. A and B ha\e the same row space, II follows that the nonzero rows of B span Ihe row space of A. Since the nonzero rows of an echelon matrix are Iinearl} independent vectors. it follows thallhe nonzero rows of B form a basis for the row space ofA. • Let IV be the subspace of R 4 gi\cn in Example 6. Use Theorem 7 to construct a basis for IV.
3.4 Bases (or Subspaces
Solution , .
"~.
As in Example 6. let V be the (4 x 5) matrix
y =
and
[VI. V2. "3. V~. "3].
Thus \V can be viewed as the row space of the matrix yT. where
ikpendence
,~
199
" 8J.
V' =
for
'9
1
1
2
1
2
1
I
4
-1
5
1
0
4
-1
2
5
0
2
1
0
0 -9
0
I
0
4
I0
0
1
2
0
0
0
0
0
0
0
0
I
-1 1
Since yT is row equivalenllo the matrix
119). it is
pendent
B'
•
~
iollO\H:
in echelon form. it follows from Theorem 7 that tbe nonzero rows of BT form a basis for the row space of yT. Consequently the nonzero columns of
la Iinearl~
•
o' 0I 00 00 00 ] o 0 I 0 0
B~
,Ie, For
[
~9
,. from
4
2 0
0
are a basis for W. Specifically. the set {Ul. u~. U3) is a basis of W. where
then the
~
u, [ fOVo.) of Iinearl} 'p3.ce
•
~ 3 basi!>
j1 ~ ~ 1~d u, [
~
u, [
!1
•
The procedure used in the preceding example can be summarized as follows:
1. A spanning set 5 := {"I ..... VIII} for a subspace W of R" is given. 2. Let Y be the (n x m) matrix Y := l"! .... \'",]. Use elementary row operations to transform V T to a matrix
aT
in echelon form.
3. The nonzero columns of B are a basis for lv.
200
Chapter 3
D..±
The Veclor Space R"
EXERCISES
In Exercises 1-8. let W be the subspace of R4 consisting of vectors of the fonn
,=[J
0)
Find a basis for IV when the components of x satisfy the given conditions. 1.
2.
XI
-X4=0
+ X2 -
X3
5.x] +X2 =0
=0
X2-2x)
=0 X3- X 4=0 -X4 =0
Xl-X'
,) , =
[
i]
3 -I ] 8 -2 2
=0
b)
1I.A~[: 1 0 [I 1 5
9. Let IV be the subspace described in Exercise I. For each veclor Xthai follows, determine if x is in \V. If x is in \V, then express x as a linear combination of the basis vectors found in Exercise I.
~
d) , = [
2
=0
8,XI- X 2 - X 3 - X 4=0
X2+X)
]
,= [ -
i]
~~
]
b) Find a basis for the null s~ of A. c) As in Example 6. find a basis for the range of A that consists of columns of A. For each column, Aj. of A that does not appear in the basis. express AJ as a linear combination of the basis vectors. d) Exhibit a basis for the row space of A.
4. x] - x2 - x) = 0
7. -x] +2xl
~
b), = [ _
a) Find a matrix 8 in reduced echelon form such that 8 is row equivalent to the given matrix A.
=0
3. XI -Xl +X3 -3X4 =0
6,Xl-Xl
-~]
In Exercises 11-16:
+ X4 = 0
Xl - 2.\) -X4
0) ,
,) , = [
=0
XI TX2-X3
X2
,= [
i]
[j iJ d), = [
10. Let \V be the subspace deo;crjbed in Exercise 2. For each vector x that follows, delermine if x is in IV. If x is in IV, then express x as a linear combination of the basis vectors found in Exercise 2.
12, A =
I
I
2 3
:]
U 2
1 0] 13. A= o 1-I [22 0] 5
3 -I
2
2
1
14. A =
2
I
1
2 3 0
IS, A = [ '2 4,
1I ]
362
3.4 Bases for Subspaces
22 2I
16..-1.=
[
'J
27. Let S =
{\'J.
1
2 3 0 -. Cse the technique illustrated in Example 7 10 obtain
" =[
a basis for the range of A. where A is the matrix given in Exercise 11 J
Repeat Exercise 17 for the matrix given in Exercise 12. 19. Repeat Exercise 17 for the matrix given in Exercise 13. ~. Repeat Exercise 17 for the matrix gi\'en in Exercise 14.
,
'0
e
eOfAth31 • A .of A. asa
"~ [il
,
25. Find a basis for the null space of each of the follow-
~ ~]
I I 0]
b) [ I
I
~
29. Let 5 = (VI. \'2, V3, ".. 1,
~s=l[ -iH~;Hj[ -nl 'J [
-J
'0
=[
=:J
"d
'0
= [
~
l
"d
Find every subset of S thaI is a basis for R2.
D.s=![iH][;JtJ! ing matrices.
il
l .,=[ -: l
" = [
.' Find a subset of 5 that is a basis for Sp(S) using Ihe technique illustrated in Example 6. b' Find a basis for Sp(S) using the technique illustrated in Example 7.
Ol.h![: H~]l 1Lh![:H:H:]1
\'2. "Jl. where
Show thaI S is a linearly dependent sel. and verify thatSp{\'J. \'2· \'31 = Sp{vl,v2l, 28, Let S = {v]. \'2, v31. where
Exercises 21-24 for the given set 5:
~_SUCh that
= [
201
0
~]
26. Find a basis for the range of each matrix in Exercise 25.
"=
[-;J
.... here
'0
= [
=:J
'"d ", =
[
=:
l
Find e\ery subset of 5 that is a basis for R). 30, ut B = {VI, "2, ")1 be a seloflinearly independent vectors in R), Prove that B is a basis for R3. [Hint: Use Theorem 13 of Section 1.7 to show that B is a spanning set for R3,J 31. ut B = {"J, "2. V3} be a subset of R3 such that
Sp(B) = R 3. Prove that B is a basis for R3. [Hint: esc Theorem 13 of Section 1.7 to show thaI B is a
linearly independent set.) In Exercises 32-35, detennine whether the given set S is a basis for R3.
32. h
![J [: H=n!
202
Chapter 3
The Vector Space R n
33.hlUH1]U]) 34. ~ I[=; H;H=: H:])
37. Prove that every basis for R 2 contains exactly two vectors. Proceed by showing the following: a) A basis for R 2 cannot have more than two
3;,s~I[J[1])
38. Show that any spanning set for R n must contain at least n vectors. Proceed by showing that if Uj, U2 ..... u p are vectors in R". and if p < n. then there is a nonzero vector v in R n such that v Tu, = O. 1 sis p. [Him: Write the conSlrainli. as a (p x II) system and use Theorem 4 of Section 1.3.J Given v as above. can v be a linear combination ofuj. Ul, . . . up?
vectors. b) A basis for R 2 canriot have one vector. [Hilll: Suppose that a basis for R 2 could contain one vector. Represent ej and e2 in terms of the basi~ and obtain a contradiction.J
5
36. Find a vector w in R 3 such that w is not a linear combination of \'J and "2:
"~ [J ~
ood
"~
[
J
39. Recalling Exercise 38, prove that every basis for R" contains exactly 11 vectors.
DIMENSION In this section we translate the geometric concept of dimension into algebraic terms. Clearly R2 and R3 have dimension 2 and 3. respectively. since these vector spaces are simply algebraic interpretations of two-space and three-space. It would be natural to extrapolate from these two cases and declare that Rn has dimension f1 for each positive integer II: indeed, we have earlier referred to elements of R" as II-dimensional vectors. But if IV is a subspace of Rn , how is the dimension of IV to be detennined? An examination of the subspace. IV. of R3 defined by
. [,,-2
\V = {x: x =
X
:;
)]
,Xl
and
X3
any real numbers}
suggests a possibility. Geometrical.ly. IV is the. plane with equation x = J - 2;:. so naturally the dimension of IV is 2. The techniques of the previous section show that IV has a basis {VI, V2} consisting of the two vectors
'. ~ [i]
and
"~
[
-n
Thus in this case the dimension of IV is equal to the number of vectors in a basis for W.
The Definition of Dimension More generally. for any subspace \V of W, we wish to define the dimension of \V 10 be the number of vectors in a basis for IV. We have seen, however. that a subspace IV
3.5 Dimension n.actly
203
IWO
may have many different bases. In facl. Exercise 30 of Section 3.4 shows Ihat any sel of three linearly independent vectors in R 3 is a basis for R l . Therefore. for the concept of dimension to make sense, we must show that all bases for a given subspace IV contain the same number of vectors. This fact will be an easy consequence of the following theorem.
a-lJ1g
two '.or {Him: lam one .fthe ba!oh
litt Oil.! U'-I ContalO ing that if lJ P < fl. . ,-uch thai
"l
Let W be a subspace of W. and let B = {"'I. "'2, .... ",pI be a spanning set for W comaining p vectors. Then any set of p + 1 or more vcctors in W is linearly dependcllt.
Prouf
Let {Sl. S2, .... Sill} be ally set of 111 vectors in W. where III > p. To show that this set is linearly dependent, we first express each Sj in terms of the spanning set B:
\I
constram~
r. of Section ~binatlon
biorR To ~how Ihat solution of
Sj
=
S2
=
+ a21"'2 + 012"'1 + 022W2 +
Sill
=
al",wl + U:lmW2
(SI, 52 .. , .• Sill)
011"'1
+ apl"'p + Op:lw p
(I)
+ ... + upmw p '
is linearly dependent. we must show that there is a nontrivial
CjSI ..l. C2S2
+ ... + C"'S'"
= 8.
'21
Now using system (I), \\Oe can rewrite Eq. (2) in terms of the vectors in B as CI(OIl \~ I ..&.- a21 \\'2 -
+ U:!mw2 +
C2(OnWI - 022w2
... + c.. (aj .. wl
-
+ Opl W p ) + - ap 2\\'p) .,.. up"'W p )
(3a)
= 8.
Equation (3a) can be regrouped as
+ C2UI2 + e",aj.)wj + (CIU21 + C2U22 T + C... U~)w2 T (CIUpl + C2Up2 + - c..u.-)w p = 8. (CIUIl
... -
-
~.:.
so
thatU
Nov. finding Cj. C2 • •••• c.. to satisfy Eq. (2) is the same as finding CI. C2 • ...• C. to satisfy Eq. (3b). Funherrnore. we can clearly satisfy Eq. (3b) if v.e can choose zero for each coefficient of each w,. Therefore. 10 obtain one solution of Eq. (3b). it suffices 10 sohe the system
+ ... +UI",C.
= 0
U21CI Ta~Jc2--1.···+a:lmc ..
=0
allCt ..l. UI2C2
.. for n'
aplCI + Up2C2 + ... + 0p",e",
(4
= O.
[Recall thai each ail is a specified constant determined by system (1), whereas each is an unknown parameter of Eq. (2).] The homogeneous system in (4) has more unknowns than equations. so by Theorem 4 of Seclion 1.3 there is a nontrivial solution to system (4). But a solution to system (4) is also a solution to Eq. (2). so Eq. (2) has a .... nontrivial solution, and the theorem is established,
Ci
of IV 10 ,j).lce U'
13hl
204
Chapter 3
The Vector Space Rfl
As an immediate corollary of Theorem 8. we can show that all bases for a subspa.;c contain the same number of \'ectors. I
t
COI\Oll.\I\'
I
Proof
Let W be a subspace of Rfl • and let B = {WI. W2 ..... w p } be a basis for IV containin,:,: p veclOrs. Then every basis for \V contains p "ectors. Let Q = {Ul. U2, .. , , Ur I be any basis for IV. Since Q is a spanning set for IV. b) Theorem 8 any set of r + I or more vectors in \f is linearly dependent. Since 8 is. linearly independem set of p vecwrs in IV. we know that p :::: r. Similarly, since B ia spanning set of p vectors for W, any set of p + I or more vecwrs in IV is linear!. dependent. By assumption, Q is a set of r linearly independent vectors in IV; so r :::: p Now. since we have p :::: rand r S p, it must be that r = p. • Given that e\'ery basis for a subspace comains the same number of vectors, we can make the following definition without any possibility of ambiguity.
DH1 .... llIO .... :)
Let IV be a subspace of Rfl • If IV has a basis B = {WI. "'2..... w p } of p vectors, then we say that \V is a subspace of dimension p. and we write dim(IV) = p.
In Exercise 30. the reader is asked to show that every nonzero subspace of R fl does ha\'e a basis. Thus a value for dimension can be assigned to any subspace of Rfl • where for completeness we define dime W) = 0 if IV is the zero subspace. Since R3 has a basis lei. e2. e3} containing three vectors, we see that dim(R3) = 3. In general. Rfl has a basis lei, e2 ..... efl} that contains n veclOrs: sodim(R = n. Thus the definition of dimension-the number of vectors in a basis-agrees with the usual tenninology: R3 is three-dimensional. and in general. Rfl is Il..-dimensional. fI
)
F\,\\\PII: I
Let W be the subspace of R 3 defined by X, ]
IV = {x: x =
X2
[
.
XI
=
-2X3.
-"2 =
X3. X3
x,
Exhibit a basis for IV and detennine dim(1V). 50lulion
A vector Xin \V can be wri!ten in the fonn
-lx, ]
x= [
::
[ -2 ] =
Therefore. the set {u} is a basis for \V. where
u~
XJ
[-:J
:
•
arbitrary}.
3.5 Dimension
It follows that dim(IV) = I. Geometrically. IV is the line through the origin and through the point with coordinates (-2. I. I). so again thedefinition of dimension coincides with our geometric intuition. •
f~)r a subspa~
1\ comainin!
for 1\'. b: ince B i\ '~. since B .n is Jinearl.
iott
205
The next example illuSlTates the importance of the corollary 10 Theorem 8. EX,\MPI
r2
Let
IV
be the subspace of R3 •
u, =
I";sor:::;:. :-tors.\lte
[]
IV
= span[u].
~ ~
u, [
l
u41. where
U2, U3.
~
u;
[ :
l
anh = [ _: ]
Use the techniques illustrated in Examples 5. 6. and 7 of Section 3.4 to find three different bases for IV. Give the dimension of IV.
Solution
,
of e wme w
(a) The lechnique used in Example 5 consisted of finding a basis for IV by using the algebraic specificalion for IV. In panicular.let b be a vector in R3:
b~
[:
l
Then b is in W if and only if the veclor equation
R') = 3 = n. Th lh
XIUj
+ .'f2U2 -
X3U3
+ .'f414 =
b
s..
is consistenl. The mauix equalion for (5a) is Ux = b. "" here U is the (3 x -I.) matrix U = [Ul. U2. U3. 14]. Now. the augmemed matrix [U I b] is TO\It equivalent to the matrix
1 0 [
o o
1
-1
1 1 3/2
0 0
0
20 - b ] -0/2 + b/2 . --ta+2b ...... c
(Sh'
Thus b is in 1V if and only if -40 + 2b + c = 0 or. equivalently. c = -ta - lb. The subspace IV can then be described by
IV
= (b: b = [
:
].
1I
and b any real numbers).
40 - 2b From Ihis description il follows that IV has a basis {VI. \'2}. where
"~ U] ~
ond
"~
[ _:
l
206
Chapter 3
The Vector Space R"
(b) The technique used in Example 6 consisted of discarding redundam vectors from a spanning set for W. In particular since {Ul' U2. U3. u.1I spans lV, this technique gives a basis for \V that is a subset of {Ul. U2. U3. ~}. To obtain such a subset, solve the dependence relation X1Ul +X2U2 +X3U3
+X414
= 8.
eSc
Note that Eq. (5c) is just Eq. (5a) with b = 8. It is easily seen from matri't (5b) that Eq. (5c) is equivalent \0 the reduced system
x,
X4
~O
+ X3 + (3/2)X4
~O.
-4X3 .1"2
-
(Sd I
Backsolving (5d) yields Xl = X2
-X3
+
X4
= -X3 - (3/2)X4.
"'here X3 and X4 are arbilrary. Therefore. the vectors U3 and 14 can be deleted from the spanning set for \Y. lea\"ing {UI. u21 as a basis for W. (c) Let U be the (3 x 4) matrix whose columns span lV. U = [Ul' U2. U3.14). Following the technique of Example 7. reduce U T to the matrix
I 01-24]
CT =
0
o 0 0 [ 000 in echelon form. In this case the nonzero columns of
C=
[
I
0
0
I
o o
4
-2
o
n
fonn a basis for W; that is. (WI. w21 is a basis for W. where
W,
~ ~] [
'nd
w,
~ [ -!J
In each case the basis obtained for \V contains two vectors. sodim(W) = 2. Indeed. viewed geometrically. W is the plane with equation -4x + 2)' + z = O. •
Properties of a p.Dimensional Subspace An imponam feature of dimension is that a p-dimensional subspace W has many of the same properties as R'. For example. Theorem 11 of Section 1.7 shows that any set of p + I or more vectors in R' is linearly dependent The following theorem shows that this same property and others hold in W when dim(1V) = p.
3.5 Dimension
IHI OK.r\\ 9
\ector.
n. tJu
Let W be a subspace of R" wilh dim(W) = p. 1. Any set of p
osuch
207
+ 1 or more vectors in W is linearly dependent.
2. Any set of fewer than p vectors in W does not span W. 3. Any set of p linearly independent \'ectors in W is a basis for W.
~..:
marnx
4. Any set of p vectors that spans W is a basis for Proof
'"
Property I follows immediately from Theorem 8. because dim(W) = p means that W has a basis (and hence a spanning set) of p vectors. Property 2 is equivalent to the statement that a spanning set for W must contain at least p veclOrs. Again, this is an immediate consequence of Theorem 8. Toestablishpropeny 3, let {UI. U! ..... up) be a set of p linearly independent vectors in W. To see that the given set spans W.let v be any veclOr in W. By propeny 1. the set {\'. UI, U!, .... up} is a linearly dependent set of vectors because the set contains p + I vectors. Thus there are scalars ao. al ..... ap (nol all of which are zero) such that 00\'
deleted
u
+ alUI + a2U2 + ... + opu p =
In addition. in Eq. (6). Go cannot be zero because dent set. Therefore, Eq. (6) can be rewritten as
Q;J.
w.
\' = (-I/ao)[a]ul
8.
lUi. U! .....
(6)
up} is a Ijnearly indepen-
+ 02"2 + ... +opup].
(7)
It is clear from Eq. (7) that any vector in W can be expressed as a linear combination of up, so the given linearly independent set also spans W. Therefore, the set is a basis. The proof of property 4 is left as an exercise. UI. U2 ..•••
E\-\ \\1>1 E
J
Let W be the subspace of R 3 given in Example 2. and let lVI. v2. V3) be the subset of W defined by
VI~
[
-J
v,
~ ~ [
l
"d v)
~
[ ]
Determine which of the subsets {\'d. {'"21, {Vl. v21, {v], v31. {v2' v31, and {v], V2, "3) is a basis for W.
Solution
lndeed.
• I~
of the set of ~ that
In Example 2, the subspace \V was described as
W = {b: b = [
:
]. a and b any real numbers},
(8)
4a - 2b Using Eq, (8), we can easily check thai v], v2. and v3 are in W. We saw further in Example 2 that dim(W) = 2. By Theorem 9, propeny 2. neither of the sets {,,]} or {V2} spans IV. By Theorem 9, propeny 1, the set {v]. V2, v3} is linearly dependent We can easily check that each of the sets {v]. "21, {VI, V3}, and {V2. V3}. is linearly independent. so by Theorem 9. property 3. each is a basis for IV. •
208
Chapter 3
The Vector Space R n
The Rank of a Matrix In this subsection we use the concept of dimension 10 characterize nonsingular matrices and 10 detennine precisely when a system of linear equations Ax = b is consistent. For an (m x n) matrix A, the dimension of the null space is called the nullity of A. and the dimension of the range of A is called the rank ofA. The following example will illustrate the relationship between the rank of A and the nullity of A. as well as the relationship between the rank of A and the dimension of the row space of A. -Ex"" \\PLI:: I
Find the rank. nullity, and dimension of the row space for the matrix A, where
A
Solution
~ -~
1 2]
1
[
o
2 -3
4
8
.
5
To find the dimension of the row space of A, observe that A is row equivalent to the matrix
B=
[i
o
-2
I
3
o
0
n
and B is in echelon form. Since the nonzero rows of B fonn a basis for the row space of A, the row space of A has dimension 3. To find the nullity of A, we must detennine the dimension of the null space. Since the homogeneous system Ax = 6 is equivalent to Bx = 6, the null space of A can be detennined by solving Bx = 6. This gives Xl
=
X2
= -3X3
X~
=
2x)
O.
Thus N(A) can be described by
N(A) ~
2Xl] OXl' Xl ,"y ",I ,umbe'l·
-3x)
Ixo> =
[
It now follows that the nullity of A is 1 because the vector
v
~ -r] [
forms a basis for N(A). To find the rank of A, we must detennine the dimension of the range of A. Recall that 'R(A), the range of A, equals the column space of A, so a basis for 'R(A) can be
3.5 Dimension
209
found by reducing AT to echelon form. It is straightforward to show that AT is row equivalent to the matrix C T , where
gular matrices )Osistenl. For 'of A, and the ~ \\.i11 illustrate ~ relationship
CT =
re
I 0 0]
0
[o
o
I 0
0
I
0
0
.
The nonzero columns of the matrix C,
C~[~ inlent 10 the
0 0 I
0
0
I
fonn a basis for neAl. Thus the rank of A is 3.
n
-
Note in the previous example that the row space of A is a subspace of R~. whereas the column space (or range) of A is a subspace of R 3, Thus they are entirely different subspaces: even so, the dimensions are the same, and the next theorem states that this is ahl,'3)'s the case. ~
row space T+'t:OI~t:\\ 10
fpace. Since l'fA can be
If A is an (III x II) matrix. then the rank of A is equal to the rank of AT.
-
The proof of Theorem 10 will be gi\'en at the end of this section. Note that the range of AT is equal to the column space of AT. But the column space of AT is precisely the row space of A. so the following corollaJ) is actually a restatement of Theorem 10.
COl\llll \In
If A is an (m x II) dimension.
matri~.
then the row space and the column space of A ha\e the same _
This corollary pro\'ides a useful way to determine the rank of a matrix A. Specifi.. call}. if A is row equh-alent to a matriJi: B in echelon form. then the number. r. of nonzero rows in B equals the rank of A. The null space of an (m x /I) matrix A is determined by soh ing the homogeneous s},tem of equations Ax = 8. Suppose the augmented matrix [A 18J for the system is row equivalent to the matri~ IB 8J. which is in echelon form. Then clear!} A is row equivalent to B. and the number. r. of nonzero rows of B equals the rank of A. But r is also the number of nonzero rows of [B 18J. It follows from Theorem 3 of Section 1.3 that there are " - r free variables in a solution for Ax = 8. But the number of vectors in a basis for ..\~(A) equals the number of free variables in the solution for Ax = 8 (see Example 3 of Section 3.4): that is. the nullity of A is II - r. Thus we ha\'e show n. informally. Ihat the following formula holds. Remark .-\. Recall .-\) can be
If A is an (Ill x II) malrix.lhen Il
= rank(A)
+
nullily(A) .
This remark will be proved formally in a more general COntexl in Chapter 5.
210
Chapter 3
The Vector Space R n
Example 4 illustrates the argument preceding the remark. If A is the matrix givetl in Example 4,
A
~
[ -;
I
I
0
2
4
8
-~
l
then the augmented matrix fA 19] is row equivalent 10
rBle)~ U
0
-2
0
1
3
0
0
0
1
n
Since A is row equivalent 10 8. the corollary to Theorem 10 implies that A has rank 3. Further. in the notation of Theorem 3 of Section 1.3. the system Ax = 9 has 11 = ~ unknowns. and the reduced matrix [8181 has r = 3 nonzero rows. Therefore, tht' solution for Ax = 8 has n - r = 4 - 3 = I independent variables, and it follows thai the nullity of A is I. In particular, rank(A)
+
nullity(A) = 3 + I = 4,
as is guaranteed by the remark. The following theorem uses the concept of the rank of a matrix to establish necessal) and sufficient conditions for a system of equations. Ax = b. 10 be consistent. TH-EOll,t\\
11
An-(m x II) system of linear equations. Ax = b. is consistent if and only if rank(A) = rank([A I bJ).
Proof
Suppose that A = fA J • A 2, .... An]. Then the rank of A is the dimension of the column space of A. that is. the subspace Sp{A j .A2, ... ,A n )·
(9'
Similarly, the rank of fA I b1 is the dimension of the subspace SprAt. A 2 •... ,An' b}.
(lOJ
But we already know that Ax = b is consistent ifand only ifb is in the column space of A. It follows that Ax = b is consistent if and only if the sub:o>paces given in Eq. (9) and _ Eq. (10) are equal and consequently have the same dimension. Our final theorem in this section shows that rank can be used 10 detennine nonsin· gular matrices. -l~! rORr \\ 11
Proof
An
(11
x
11)
matrix A is nonsingular if and only if the rank of A is
I!.
Suppose that A = fA J • A2' .... A"]. The proof of TIleorem 12 rests on the observation that the range of A is given by R(A)~
Sp{A"A"""A"I,
(11
If A is nonsingular then. by Theorem 12 of Section 1.7, the columns of A are linearl} independent. Thus {A J • A 2..... All} is a basis for R(A). and the rank of A is II.
3.5 Dimension
matrix given
211
Conversely. suppose that A has rank n: that is, R.(A) has dimension n. It is an immediate consequence of Eq. (II) and Theorem 9. property 4. that {AI. A2 ... " A~} is a basis for 'R,(A). In particular, the columns of A are linearly independent. so. by Theorem 12 of Section 1.7. A is nonsingular. ~
Proof of Theorem 10 (Optional) To prove Theorem 10. let A = (au) be an (m x n) matrix. Denote Ihe rows of A by Ill. a2_···. am' Thus,
a, = [ail.ai2 •.. .. ai~l. Similarly, lei AI. A:l .... , A~ be the columns of A. where
~
h" "'nk
hasn=4 fore. the )lJows Ihal
aJ)
Aj =
]
a2J
•
[
a.)
Suppose thai AT has rank k. Since the columns of AT are af.
aI..... a~. it follows that
if '1eCess
\V = Sp{al.32 ..... 3"'1.
then dim(W) = k. Therefore, W has a basis {WI. W2 •.••• w.J and. by Theorem 9. property 2, III ::: k. For I .::: j .::: k. suppose thai Wj is the (I x n) ,ector Wj
column
•
(a.. l. a.. 2,
a.
aJ)
~J
[
..
alii;
l
11 r linear!)
a2.llJ
...•
= 32 = C21wI + C22W2 + ... + C2tWt
a..~J = a", = C.. lwl +C",lw2 + ......
(Ill
C",t,",'t.
Equating the jth component of the left side of system (12) with the jth component of the right side yields
, nonsin-
~nation
.U.'i"]·
al~J = 81 = CIlWI + CI2,",'2 -r-'" + CltWt
au (all. an
[all.
10
pace of
[Wjl. Wj2····
Writing each a, in terms of the basis yields
• «9) and
=
]
=
[
Wlj
c" C21 ] . ..
+ W:2J
[
C22 CJ2
•
..
e",2
C",l
]
+ ... + Wtj
[
C" C2t ] .
..
C",J:
for I .::: j .::: n. For I .::: i .::: k, define c, to be lhe (m x I) column vector
C.. c.,
C" ]
c., = [
(1J)
212
Chapter 3
The Vector Space R"
Then system (13) becomes A J = WljCI
+ lL'2jC2 + ... + WtJCb l.:s
j ::: n.
14
It follows from the equations in (14) that 'R.(A) = Sp{A l . A1.... AII } : ; SpfCI' C2· .. ·. Ctl·
It follows from Theorem 8 that the subspace
V = Sp(CI. C2 .....
cd
has dimension k. at most. By Exercise 32. dim[R(A)) .:s dim(V) ::: k: thtlt is. rank(A) .:s rank(A T ). Since (AT)T = A. the same argument implies that rank(A T ) .:s rank(A). TIm" rank(A) = rank{A T ). _
~
EXERCISES
Ex.ercises 1-14 referto the vectors in (l5l.
u, ~ [ :
1u, ~ [ : 1u, 1 = [ -:
"'=[a u'~[:l v,~[ -;]
10. S = lUI. Ul} for R 2 11. 5 = {U2. U3} for R 2 12.5= IVI. V2. v31 for RJ 13.5= IVI. "2. v~} for RJ !-t.
5=
{"2. "3. "41 for R3
In Exercises 15-20. \V is a subspace of R 4 consisting of vectors of the fonn
v,=[;] "~[ -] ,.,~[-:] "
In Exercises 1-6. detennine by inspection .... hy the given set 5 is not a basis for R2• (That is. either S is linearly dependent or S does not span R2 .)
LS={uJ!
= 5. S = 3. S
{Ul' U2. U3}
{U!.14}
2.S={Ul} 4. S = {U2. U3. us} 6. S = {UI. usl
In Exercises 7-9. detennine b} inspection .... hy the given set S is nOl a basis for R3 . (That is. either S is linearly dependent or S does not span R3 .)
7. S= {Vl."2} 9. S = {"l. V2. "J. V4}
8. S= {VI. V)}
In Exercises 10-14. use Theorem 9. propeny 3. to detennine .... hether the gi,en set is a basis for the indicated \ector space.
X, ] _ X2 x_ . x, [
•
x,
Determine dim("') when the components of x satisfy the given conditions. IS. Xl - 2.t2
+ XJ -.\"4 =
16,XI-2t] =0 17. Xl = -X2 - 2x4 X3 = -X4 19.
Xl X2
X3
0
18.
+
Xl
.\"3-2t 4=0
.1'2+2x J
= -X4 = 3X4 = 2x 4
20. .1'1-.1'2
-3x4=0 =0
X2-21"J
=0 ·1"3- X4=0
Tn Exercises 21-24. find a basis for N(A) and give the nullity and the rank of A. 21. A _ [
_~ _~
]
22. A
= [
1 , 2 -.5
~]
"..
3.5 Dimension
[
-I
"A~ [; that ie;:. Thu_~
.-4,
•
:]
I-I 2-1
13.A=
•
2
0
3
1
3 -I
;]
In Exercises 25 and 26. find a basis fOT R(A) and give the nullity and Ihe rank of A .
25. A~ [ -:
21]
o
3
1 5 7
'6'A=[~ 27.
1
• 1
.
2, 0] 5 -2
ut W be a subs~. and let S be- a spanning sel for W. Find a basis for \Y. and calculate dim(W) for each sel S.
",-"'tingof
.) h
\ salisfy
-:!..r~
=0 Jr~ = 0 ~O ~O
-_f~=O
gi\'e the
, .;
~]
I[J [=; H_][-~])
b)S=I[
-i]{H~iH -f]l
28. Let W be the subspace of R~ defined by \V = fx: \,T x = 01. Calculate dim(\V). where
,.~
[J
29. Let W be- !he subspace of R~ defined by IV = Ix: aT x = 0 and b Tx = 0 and e Tx = 01. Calculate dim{W) for
.=[
-J b~[
213
_] ood
,~[ -:J 30. LeI IV be a nonzero subspace of R~. Show Ihal IV has a basis. [Hint: LeI WI be any nonzero \eclOl" in \Y. If (WI I is a spanning set for \V. then we are done. IfnOl.lhere isa \ectOr""! in IV such that (\II. \Ill is linearly independent. Why? Continue by asking whether this is a spanning set for IV. Why must this process eventually stop?]
31. Suppose thaI {UI. U2 •.•.. up} is a basi~ for a subspace IV. and suppose that x is in IV with x = 0IU] + a2D! + ... - OpDp- Show thallhis representation for x in lenns of !he basis is uniqueIhal is. if x = blUI + b1U2 + ... - bpu p. then b l = al. b 1 = Ul • •... b p = up32. LeI U and V be subspaces of Rn • and suppose that U is a subset of V. Prove Ihat dim(U) ::: dim(V). If dim(U) = dim(V). prove Ihal V is contained in U. and thus conclude thaI U = v. 33. For each of the following. determine lhe largest possible \'alue for the rank of A and the smallest possible \'alue for Ihe nullily of A. a) A is(3 x 3) b) A is(3 x 4) c) A is(5 x 4) 34. If A is a (3 x 4) matrix. prove that the columns of A are linearly dependent. 35. If A is a (4 x 3) m:urh:. prove Ihal the rows of A are linearly dependenl.
36. Let A bean (m x n) matrix. Pro\e Ihal rank(A) .::: m and I1lnk(A) ::: n. 37. Let A be a (2 x 3) matrix with Tank 2. Show thai the (2 x 3) system of equations Ax = b is consistent for e\ cry choice of b in R 2. 38. lei A be a (3 x 4) matrix wilh nullity 1. Prove Ihat the(Jx4)systemofequationsAx = bisconsislenl fore\erychoiccofbin RJ .
214
Chapter 3
The Vector Space W
39. Prove that lin (n x II) matri;( A is nonsingular if and only if the nullil}' of A is zero.
'""'-0-1 •...• wpl· Finall). use Theorem 8 to reach a comrndiction.
40. lei A be an (m x m) nonsingular mlltrix, and let B be an (m x II) mlltrix. Prove that X(AB) = ;\'(B) and conclude that rank (A B) = rank (B).
-U. Suppose that S = {Ul. u! ..... up} is a set of linearly independent vectors in II subspace IV. where dim(W) = 11/ and If/ > p. Prove that there is II \-ector Up_I in IV such that {UI. U2 ..... up. Up_II is linearl) independelll. Use this proof to shO\\ that II basis including all the veclOrs in S can be constructed for W.
41. Prove propeny 4 of Theorem 9 as follows: Assume that dim(W) = p and let S = ("'·1 ..... '""pI be a set of p vectors that spans W. To see that S is linearly independent. suppose that CIWI + ... + cpw p = 8. If Cj :F O. show that W = Sp{Wl ..... WI-J.
i3,E,
ORTHOGONAL BASES FOR SUBSPACES We have seen that a basis provides a very efficient way to chamcterize a subspace. Also. given a subspace IV. we know that there are many different ways to construct a basis for IV. In this '\eCtion we focus on a particular type of basis called an onllOgollaf basis.
Orthogonal Bases The idea of onhogonality is a generalization of the vector geometT) concept of perpendicularit). If u and \' are twO vectors in R2 or R3 • then ",e know that u and \' are perpendicular if u r \' = (see Theorem 7 in Section 2.3). For example. consider the vectors U and v given by
a
u
~ [ _~]
and
"= [ : ],
Clearly u r v = 0, and these two vectors are perpendicular when viewed as directed line segmems in the plane (see Fig. 3.13).
F;gu,~ J.IJ
In R~. non.tero vectors u and ,'are perpendicular if and
onl)ifur,,=O.
In general. for vectors in R n. we use the te.rm orthogonal mther than the tenn perpendicular. Specifically. if u and \' are vectors in R n. we say that U and \' are orthogonal if
uTv=O. We will also find the concept of an onhogonal set of vectors to be useful.
3.6 Orthogonal Bases ror Subspaces ~m
215
8 to reach a Dt:t-l'lTIo, b
1~
a set of lin,pace IV. where ,; that there is a
Let 5 = {Ut. U2 ••... up} be a set of \'ectors in R~. The set S i!. said to be an orthogollal set if each pair of distinct \'ectors from S is orthogonal; that is. u'["Uj =Owheni f:.j.
... up. up... d
I:
1'\'01' to show that a 5 can be con-
~
E,\\\\PI r I
Verify that S is an orthogonal set ofveclOrs. where
s~l[rH -iH =iJ!
~pace. Also. 't a basis for
#U11 basis.
Soilltion
ocept of peru and \' are ,. .:on~ider the
If we use the notation S =
UJ
uJ}, then
I
21 [
= (I 0 I
21 [
u[u1=[1 0
ur
{Ut. U2.
directed line
uIU; = [I
-I
_i]~'_o-,+o=o
=i]~'_o-,+o=o 01 [
-21 ] = 1-2"t"I+O=O. -I
o Therefore. 5 = {u].
U1.
uJ} is an orthogonal set of "ectors in R4 •
•
An important property of an onhogonal set 5 is that S is necessarily linearly independent (so long as S does not contain the zero vector). IllrOR.r \I the term II and v are
13
Proof
Let S = lUI. U2 ••••• upl be a set of nonzero vectors in R~. If S is an orthogonal set of vectors. Ihen S is a linearly independent set of veclOrs. leI (1. q ..... Cp be any scalars that satisfy CtUt -
C1U1
+ ... + CpU p = O.
Form the scalar product UiCctUI
+ C2U1 +.. ·c"U p ) = uio
11
-2J6
Chapter 3
The Vector Space Nit
0' Cj(U[ lIl)
+ C~(Ur U~) +, .. -l- Cp(u'i Up) =
O.
Since uf llj = 0 for 2.::: j :5 p. the expression abo\c reduccslo ("j
(Uf"l) =
O.
'-
Kexi. !>e1:al.L<;e uj lit > 0 when"j is nonzero. we see from Eq. (2) that Cl = O. Similarly. forming the scalar product of both sides of Eq. (I) with lli. we sec that cs(u; ll,) = 0 or Ci = 0 for I .::: i .::: p. Thus S is a linearly independent sel 01 '·ecto~. • By Theorem 13, any orthogonal SCI S containing p nonzero vectors from a pdimensional subspace \V will be a basis for W (since S is a linearly independent subset of p ,'eeton; from W. where dim{lV) = pl. Such a bash. is called an orthogonal basis. In the following definition. recall that the symbolll'-U denotes the length oC\'.II\'1I = ~
DHI'drro, -
Let IV be a sub~pace of RI/. and let B = {UJ. "2, .... Up} be a basis for W. If B is an orthogonal set of \cctors. then B is called an orthogonal basis for W. Funhennore. if u, I = I for I .::: i .::: p. then B i<; said to be an orthonormal basis for HI.
The "ord ort/wnonl/o! suggesls both orthogonal and normalized. Thus an orthonormal basis is an orthogonal basis consisling of vectors having length I. where a lector of length 1 is a unit vector or a nonnaliLed vector. Observe that Ihe unit lectors c,. C1 ..... ell fonn an onhononnal basis for R·. E\--\ \IPlt:
2
Verif) that the set B = {\ I. \'2.
v, ~ Solution
\'J}
is an onhogonal basis for R3 • where
[il [=:J v,=
and
",= [ - ]
We first verify that B is an orthogonal sel by calculating
vi "1 = 3 - 2 vi \'J = I - 8 vf \"3 = 3 + 4 -
..j..
I= 0
7=0
7 = O.
Now. R 3 has dimemion 3. Thus. since 8 is a set of three \ectors and is also a linearly independent sel (see Theorem 13). it follows that B is an orthogonal basis for R J • • These observations are stated fOrolany in the following corollary of Theorem 13.
--,
3.6 Orthogonal Bases for Subspaces
COl\OLL "I'
217
Let IV be a subspace of R". where dim(lV) = p. If S is an orthogonal set of p nonzero vectors and is also a subset of IV. then S is an orthogonal basis for W. •
Orthonormal Bases If B = {UI, U2' ...• upl is an orthogonal set. then C = l{/j Ul. {/2U2 ....• apu p } is also an orthogonal set for any scalars al. a2 ..... a p ' If B contains only nonzero vectors and if wc define the scalars (/, by
(2)
c = O. :h U , we see that .:Iependent set of
aj =
•
XlOrs from a
p_
.:Iependent subset bot!-onal basis. In \-.- vII = JvTv.
for IV. If sis for IV. onhollormal
c'b
d. Thus an or~[lgth I, where a the unit vectors
• also a linearly _n for R 3 • ~
. Theorem 13.
Ju, u, T
.'
then C is an orthol/ormal set. That is. we can convert an orthogonal set of nonzero vectors into an orthonormal set by dividing each vector by its length.
J
Recall that the set B in Example 2 is an orthogonal basis for R3. Modify B so that it is an orthonormal basis.
Soilltion
Given that B = {v). V2. V3} is an orthogonal basis for R 3. we can modify B to be an orthonormal basis by dividing each vector by its length. In particular (see Example 2). the lengths of VI, "2. and V3 arc
1 \+\ "PLE
IlvJlI =
J6. IIv211 =./!T,
and IIv311 =
../66.
Therefore. the set C = lWI. ""2. W3} is an orthononnal basis for R 3 • where
WI
=
I \' 1 = .J6
J66 V3 =
_ _1
w3 -
1/)6 ] 2/)6 . [
W2
1/)6
1 \',. = = ji'1
3/JIT] -l/JIT . [ -l/JIT
1/./66] -4/./66 . [ 7/./66
'nd
•
Determining Coordinates Suppose that IV is a p-dimensional subspace of R". and B = {WI. ""2. , ... wpl is a basis for IV. If v is any vector in IV. then v can be written uniquely in the form v=al\\'1 +{/2w2+···+{/p"",,·
(In Eq. (3), the fact that the scalars al. a2' .... (/p are unique is proved in Exercise 31 of Section 3.5.) The scalars {/J. {/2 • . . . . {/p in Eq. (3) are called the coordinates of I' with respect to the basis B . As we will see, it is fairly easy to determine the coordinates of a vector with respect to an orthogonal basis. To appreciate the savings in computation. consider how coordinates are found when the basis is not orthogonal. For instance. the set B, = {VI. v2_ \lJ i" a
218
Chapter 3
The Vector Space R"
basis for R;. where
", ~ [-:J
'" = [ -
l.
~
d
"= [ -;]
As can be seen. "j "3 i= O. and so 8 1 is not an onhogonal basis. Nexl. suppose we wi...r.. 10 express some \'ector \ in R3• say \ = [5, -5. -lJT, in term5 of BI , We musl sohe the (3 x 3) system: 0IV! +02v2 +OJ"3 = v, In matrix terms thecoordinatesol. {/2. ana OJ are found by solving the equation
[ (B)
1-I 2] [ a,] I
2
-2
0'
-I
I
I
(13
[ -55] .
=
-2
elimination. the solution is 01 = I. 02 = -2.03 = I.) Bycomf3st. if 82 = {WI. W2. w31 is an onhogonal basis for R3. it is easy to determine and (13 SO that
Gau~sian
°1.°2.
\' =
01 WI
+ 02\\'2 .L. 0J"3.
To find the coordinate (II in Eq. (4). we form the scalar product \\'T I \'
+ (I,\\', . - + 0-",,1 = 0t (wj wI> + G2(" j W2) 03(Wj W3)
= wT(a,., I
~
-l-
= ol(wj WI).
The last equalil} follows because wj W2 = 0 and wj w, = O. Therefore, from abo\e wTv {/I=-'-,
wjwl
Similarly. 02
=
Wry
---t-
and
\\'2 "'2
(Note: Since B2 is a basis. F\ \\\1'11: I
,
",T\\'J
w; w, > O. I ::: i .::; 3.)
~n
w, =[ ] " =
01""
{WI.
[J .
Beginning with the equation
\' =
-
=
Express the "ector \' in teons of the onhogonal basis B =
,,= [ Solution
Wry GJ
+ 02"'2 + 03"'3·
d
"'2.
"'3}.
where
w, = [
-J
3.6 Orthogonal Bases for Subspaces
219
we form scalar products to obtain
wfv=al(w[wI). wf v
= a2(wf "'2).
wf v = Q3(wf "'3).
-.e ~e wish '"lUSt solve • G1. and
Thus GI = 2. (/2 = 3. and 3W2
a)
or 12= 6a 1 or 33
= 1IG2
or 66 = 660 3 .
= I. Therefore, as can be verified directly, v = 2Wl
+ w3·
+ ~
n
In general, let IV be a subspace of R • and let B = {WI. W2 •.... wpl be an orIhogonal basis for IV. [f \' is any vector in IV, then ,. cun be expressed uniquely in the form v = (/1"'1
+ (/2"'2 + ... + Gpw p •
(Sa)
where WTv , -T-' Wi W,
a, -
tennine
I::::·I
:::
p.
(5b)
Constructing an Orthogonal Basis The next theorem gives a procedure that can be used 10 genenue an onhogonal basis from any given basis. This procedure. called the Gram-Schmidt process, is quite practical from a computational standpoim (although some care must be exercised when programming Ihe procedure for the computer). Generaling an onhogonal basis is oflen the firsl step in solving problems in least-squares approximation; so Gram-Schmidt orthogonalization is of more Ihan theorelical iDleresl.
~bo'e
fllrcW.l \\
I
~
Gram-Schmidl Lei W be a p-dimen::.ional subspace of R". and let {WI, "'2.... , wpl be any basis for W. Then Ihe sel of'ectors {u\. U2 ..... up} is an onhogonal basis for W. where UI =WI
~
and
~here.
ujW!
U2
=
"'2 -
-T-U, U, Ul
U)
=
W) -
-T-U1 u 1 UI
uj"'3
Ur",)
-Y---U2. u 2 U2
in general. i-I
Uj
=
W, -
""' L
r
Uk Wi -T-U,. ,1,=1 Ul Uk
2::: i :::
p.
161
•
The proof of Theorem I ~ is somewha' technical. and we defer it 10 the end of this section. In Eq. (6) we have explidt expre,ssions that can be used to generate an orthogonal set of vectors {UI. U2 ••.. , up} from a given set of linearly independent vectors. These
220
Chapter 3
The Vector Space R"
explicit expressions are especially useful if .... e have reason to implement the GramSchmidt process on a compUier. Ho.... e'er, for hand calculations. it is not necessary to memorize formula (6). AU .... e need to remember is the form or the general pattem of tbe Gram-Schmidt process. In particular. the Gram-Schmidt process smrts wilh a basis {WI. w2 ....• w p ) and generates new vectors Ul. U2. U3 ..•• according 10 the following patlern: UI = WI U2 = "'2+aUI U3
= "'3 + bUI + CU2
u~ = "'~ + dUI + eu! + [U3
Ur = Wj + alul + a2U2 _ ... + ar_1U,_1
In this sequence. the scalars can be determined in a step-b)'-step fashion from the orthogonality conditions. For instance. to determine the scalar a in the definition of U2 ..... e use the condition uf U2 = 0: 0= UfU2 = UfW2 +oufu\:
Therefore: a = -(uf w2)/(uf Ul). To determine the IWO scalars band c in the definition of U3. we use the two condilions uf U3 = 0 and U3 = O. In particular.
ur
0 -- u Tj U3
+ b UTl Ul + CUIT U2 uf "'3 + buf Uj (since uf U2
_ T - u j "'3
=
= 0 by Eq.
(7»
= 0 by Eq.
(7»
Therefore: b = -(uf "'3)/(U; ud· Similarly.
0=
ur U3 = uf "'3 + buI UI -r- cuI U2 = uI w, + CuI (since uI U2
UI
Therefore: c = -(uf w3)/(uI U2). The examples that follow illustrate the previous calculations. Finall). to use the Gram-Schmidt orthogonalization process to find an orthogonal basis for \V. .... e need some basis for \V as a staning poim. In man) of the applications that require an orthogonal basis for a subspace lV. it is relathely easy to produce this initial basis-we will gi\c some examples in a later section. Ghen a basis for lV, the Gram-Schmidt process proceeds in a meehanical fashion using Eq. (6). (Note: It was shown in Exercise 30 of Section 3.5 that every nonzero subspace of R" has a basis. Therefore. by Theorem 14. every nonzero subspace of R n has an orthogonal basis.)
3.6 Orthogonal Bases for Subspaces lb< Grnm-
-EXAM!"l E 5
Let W be the subspace of R 3 defined by
6
::'rtll.:e<.",
w,
J generaL.-.....
~ [ :]
\V ::;;
Sp(Wj, W2}, where
,nd
W,
~
221
-n
[
Use the Gram-Schmidt process to construct an orthogonal basis for W.
Solution
We define vectors
111
and
111
of the form 111 =Wl 112
=
Wl
+auj.
where the scalar a is found from the condition 111112 is given by
llf 112 =
O. Now, 111 = [I. 1, 2]T and thus
u[ 112 = 111
uf
112 = O. we need a = I. With a = I. 112 is given by 112 = [I. 3, -2f. In detaiL an orthogonal basis for W is B = {Ut, U2}, where
Therefore. to have
or-
u, ~ [ :]
..ond.ltiOD
,nd
u, ~ [ _:
l
"'2+111
•
For convenience in hand calculations. we can always eliminate fractional components in a set of orthogonal vectors. Specifically, if x and yare onhogonaL then so are ax and)' for any scalar a:
iiUOlh
If xT y = 0,
then (ax}T)' = a(x Ty} = O.
We will make use of this observation in the following example. EX-i-\I-.IPLE (,
Use the Gram-Schmidt onhogonalization process to generate an onhogonal basis for IV = Sp{Wi. W2. W3), where
W, Solution
.;onaJ
lQtiOfu,
llLe this
"'.the II was
~[]
~
W, [ ]
oed
~
w, [
J
First we should check to be sure that {WI. w2. W3} is a linearly independent set. A calculation shows that the vectors are linearly independent. (Exercise 27 illustrates what happens when the Gram-Schmidt algorithm is applied to a linearly dependent set.) To generate an onhogonal basis lUI. U2. U3} from {WI. "'2, "'3}, we first set UI
=Wl
a basis.
U2 ="'2 +aUj
'h.)
U3 ="'3
+ bUI + CU2·
222
Chapter 3
The Vector Space
R~
With UI = LO. I. 2. I V. the orthogonality condition uj u2 = 0 leads to uj W2 +auj Ul = 0, or 8 + 6a = 0, Therefore. a = -4/3 and hence U2 = W2 - (4/3)u] = [0. -1/3. 1/3, _1/3]7. Next, the conditions uj U3 = 0 and uI u, = 0 lead to 0= uj(w, +bUl +CU2) = 3+6b 0= uI (w,
+ bUI + CU2) = 0 + 0/3)c.
Therefore. b = -1/2 and c = O. Having the scalars band c, U3 = W3 - 0/2)uI - (0)U2 = Ll.l/2.0. -1/2f. For convenience, we can eliminate the fractional components in U2 and u, and obtain an orthogonal basis {v), V2, ",}, where
v,
~
[]
v, =
[J
'nd v,
~
[
J1
•
(Note: In Example 6. we could have also eliminated fractional components in the middle
of the Gram-Schmidt process, That is. we could have redefined U2 to be the vector U2 = [0, -I. 1, -If and then calculated u, with this new. redefined multiple of U2.) As a final example. we use MATLAB to construct orthogonal bases. ~
[ '-"\PI r -
Let A be the (3 x 5) matrix
I
12132] A=
4
[I
I
0
6
I
I
2 4
5
Find an orthogonal basis for R(A) and an orthogonal basis for N(A).
Solutio"
The MATLAB command orth (A) gives an orthonormal basis for the range of A. The command null (A) gives an orthonormal basis for the null space of A. The results are shown in Fig. 3.14. Observe that the basis for R(A) has three vectors; that is, the dimension of R(A) is three or. equivalently, A has rank three. The basis for N(A) has two vectors; that is, the dimension of JV(A) is two, or equivalently, A has .' nullity two.
Proof of Theorem 14 (Optional) We first show that the expression given in Eq. (6) is always defined and that the vectors UI. U2 .... , up are all nonzero. To begin, UI is a nonzero vector since UI = WI. Thus uj III > O. and so we can define U2. Furthennore. we observe that U2 has the fonn u2 = w2 - hUl = w2 - blw]; so U2 is nonzero since it is a nomriviallinear combination
3.6 Orthogonal Bases for Subspaces
223
.
au u == A. 1
2
1
o
1
1 1
•
J
2
••
2
1
5
»orth{A)
ana. 0.3841 0.7682 0.5121
ob<.un an
0.5908
·0.9158 0.2466
·0.7983
0.3170
·0.1173
»null (A)
ana_ -0.7528
•
-0.0690 0.1800 -0.9047 -0.OU9 0.3772
-0.2063 -0.1069 0.5736 -0.2243
middl(
'«tor of u~.)
Figure 3.14 The ~IATLAB command arth IA) produces an orthonormal basis for the range of A. The command null (A) gives an orthononnal basis for the null space of A.
ofw) and Wl. Proceeding inductively. suppose thai ul. "1_ .... U,_I have been generated by Eq. (6): and suppose that each Ut
=
Wt -
Ut
has (he form
CIWI -
C1Wl -
... -
Ci_IWt_l_
From this equation, each Uk is nonzero: and it follows thai Eq. (6) is a well-defined expression lsince uf lit > 0 for 1 ~ k ::: (i - OJ. Finally, since each Uk in Eq. (6) is a .ge of A
Th<
l<-
:ors; that basis for ~. A has
•
\ectors I,," . Thus
the fonn
mation
linear combinalion of "'\. \\'2 •.••• "'t. we see thal Ui is a nontrivial linear combination of "'\. w~ ..... w,: and therefore u, is nonzero. All thai remains 10 be proved is Ihat the vectors generated by Eq. (6) are orthogonal. Clearl) ur U2 = O. Proceeding inductivel) again. suppose thai uJ Ut = 0 for any j and k. where j k and I :::: j. k :::: i-I. From (6) we have
t-
T
UjUj
=
;-,
r
Uj
(
Wi -
r
L-r-u, Uk W,
t=1 Uk Ut
) =
r U
J
;-, (r- r - ) L UkUt
r
Uk W,
Wi -
(U j
r (uJ",) r
= UJ Wi -
ud
i_I
-rU j uJ
(U j Uj)
= O.
Thus Ui is orthogonal to u J for I :::: j :::: i - 1. Having this result. we have shown thal {UI. U2 ..... up} is an orthogonal set of p nonzero veClors. So. by the corollary of Theorem 13. the \ectoT'S Ul. U1 ..... up are an orthogonal basis for \V.
224
The Vector Space R n
Chapter 3
3.(,
EXERCISES
In Exercises 1--4. verify that {UI. set for the given vectors.
l.u,~
U1. U3}
is an orthogonal
[:J u,~ [-~ l u,~[!]
l
l ~ [J ~ U l~ ~ ~U l ~ -n
2'U'~[ ~ u,~[ -~ u'~[:J 3.
u,
4.
u, [;] u,
In
u,
Exercise~
(UI.U2.U3) IS
u, [ -;] u, [
5-8. tind values a. b. and c such that an orthogonal set.
l ~ [J ~ ~ l ~ [-J ~
u,
u, [
6
u, [
u,
u, [ : ]
7.
UI
1 ] I
.
U!
=
[
8.
u, ~
1
[ -2 ] ~
I
[J ~ [J a
u,
.
=
[ 4 ] b
u, ~ [
:J
c
in Exercises 9-12. express the given vector v in terms of the orthogonal basis B = (UI. "2. U3). where UI. U2. and U3 are as in Exercise T.
9'~[l]
1O"~[;]
[
n
12'~[!]
In Exercises 13-18. use the Gram-5chmidt process to generate an orthogonal set from the given linearly independent \,ectors.
I3tH;H~] 14. [
iH;H-~ ]
15. [ 1 ] [ 0 ] [1 ] 1.2. I
16. [ ;
~
U3
~
n H;H~~ ]
;. u, [ :
=
~
11.'
17. [ ; ] . [ : ] . [
0
0
I
0
~
;]
1
0
18. [ 1] [0] [0]
~
.: .~
In Exercises [9 and 20. find a basis for the null space and the range of the given matrix. Then use Gram-Schmidt to obtain orthogonal bases.
19. [ 1 -2 2 I I -1
1 -; ] 7 5 2-2
3.7 Linear Transformations from R" to R-
,
]
process to linearly inde-
Idl
20. [
1 -I
3 10 11
2
5
4
2 -I -I
1
;]
21. Argue that any set of four or more nonzero veCIQrs in R) cannOI be an orthogonal set.
2!. Let S = {Ut. "2, U31 bean orthogonal set of nonzero vectors in RJ . Define the (3 x 3) matrix A by A = [Ul. "2. U3]. Show thaI A is nonsingu!ar and ATA = D. where D is a diagonal matrix. Calculate the diagonal matrix D when A is created from the orthogonal vectOrs in Exercise I. B. Let IV be a p-dimensional subspace of Rn . If v is a vector in IV such Ihal \,7 w = 0 for every w in W, show that v = 8. [Hint: Consider w = v.1 2-1. The Callch)'-Schwa~ ineqlloliry. Let x and y be vectors in R~. Prove thai I"TYI .:5 IIx JI: YII. [Hint: Observe that IIx - cyU 2 ~ 0 for any scalar c. If y #- 8.
225
letc = x T~'/yT y and expand (x _cy)T (x -Q') ~ O. Also treal the case y = 8.]
25. The triangle inequality. lei x and y be \ ectors in R~. Provethatllx+yl!::: Jlxll+IIYII. [Him: Expand IIx + )"1[2 and use Exercise 24.) 26. LeI x and y be vectors in R~. Prove that IlIxl lIyUI ::: IIx - Yl!. [Him: For one part consider IIx + (y - x)11 and Exercise 25.] 27. If the hypotheses for Theorem 14 were altered so that {Wi is linearly independent and (Wj is linearly dependent. use Exercise 23 to show that Eq. (6) yields up = 8.
If:/
Ir.1
28. Let B = (UI. U2 •.••• upl be an orthonormal basis for a subspace W. Let \' be any vector in IV. where v = 01111 + 02U2 + ... + opu p . Show that
lIvll 2 = af + oi + ... + a~.
LINEAR TRANSFORMATIONS FROM R n TO R m In lhis section we consider a special class of funclions. called linear transfomwlions. that map veclors to veClors. As we will presenlly obsen e. linear transfonnalions arise naturally as a generalization of matrices. Moreover. linear uansfonnations have important applications in engineering science. the social sciences. and various branches of mathemalics. The notation for linear transformations follows the usual notation for functions. If V is a subspace of R" and \V is a subspace of R"'. then the notation
F:V __ W \loill denote a function, F. whose domain is the subspace V and whose range is contained in W. Furthennore. for \' in V we write
w = F(v) 10 indicate that F maps \' to w. To illustrate. let F: R3 -+ R2 be defined by X, -
x, ]
F(x)= [ xl+X3
IJI 'p3Ceand ~Schmidl
where
x~
[:J
•
226
Chapter 3
The Veclor Space R II
In this case if, for example. v is the vecwr
v~ [~
l
then F(\» = w, where w
~[
-:]
In earlier seclions we have seen that an (m x n) matrix A detemlines a function from R" to R"'. Specifically for x in R". the formula T(x) = Ax
II
defines a function T: W --+ R"'. To illustrate. let A be the (3 x 2) matrix
A=
1-I] [ ~
~
.
In this case Eq. (I) defines a function T: R2 --+ R J • and the formula for T is
T(,) = T ([ :: ])
~ [~ -~] 3
[ :: ]
I
~
[
3Xl
for instance.
T ([ : ])
~ X'2~:V2
[;
]:
+ X2
l
Returning to the general case in which A is an (m x n) matrix. note that the function
T defined by Eq. (I) salisfies the following linearity properties: T(v
+ w)
T(ev)
= A(\'
+ w) =
Av + Aw = T(v)
,
+ T(w)
= A(e\') = eAv = eT(v).
\\here v and w are any vectors in R" and e is an arbitrary scalar. We next define a linear transformation to be a function that satisfies the two linearity properties given in Eq. (2).
DEFI'lTlO'
5
Let \I and \V be subspaces of Wand R"', respectively, and let T be a function from \' to lV, T: V --+ IV. We say that T is a linear transformation if for all u and \' in V and for all scalars Q T(u
+ v) =
T(u)
+ Tv)
and T(au) = aT(u).
(~)
3.7 Linear Transformations from R" to R-
227
II is apparent from Eq. (2) that the funclion T defined in Eq. (I) by matrix muhiplication is a linear transformation. Conversely, if T: R" ... R'" is a linear transformation. then (see Theorem 15 on page 232) there is an (m x 11) matrix A such that T is defined by Eq. (I). Thus linear transformations from R" to R'" are precisely thosc functions that can be defined by matrix multiplication as in Eq. (I). The situation is not so simple for linear transformations on arbitrary vector spaces or even for linear transformations on subspaces of R". Thus the concept of a linear transformation is a convenient and useful generalization to arbitrary subspaces of matrix functions defined as in Eq. (I).
Examples of Linear Transformations
j1e~ a funclion
Most of the familiar functions from the reals to the reals are not linear transformations. For example. none of the functions f(x)=x+l. 8(X)=X 2• h(x) = sin.r. k(.t)=e~
(I
is a linear transformation. Indeed, it will follow from the exercises that a function f: R -l- R is a linear transformation if and only if f is defined by f(x) = ax for some scalara. We now give several examples to illustrate the use of Definition 8 in verifying whether a function is or is not a linear transformation. E\..'I,\\PLL
I
Lei F: R3 _ R2 be the function defined by
x, ]
X, -
where x =
F(x)= [ X2+ X3
[."." ] X2
'
Determine whether F is a linear transformation.
Solution function
"
define a lIe>. gi\en in
C:ll
We must determine whether the Thus let u and \' be in R 3.
u
~
1\\'0
linearity propenies in Eq. (3) are satisfied by F.
[ ::] 143
"~
[ :
] .
(/3
and let c be a scalar. Then
u, - u, ] u.J,.\,=
1.Jlk:t1on )f all u
and
[
U2+l/2 "3 T V3
Therefore. from the rule defining F, F(u
+ v) =
+ u,J - (u, + ",) ] (142 + (/2) + (113 + (/3)
(u, [
~ [ u, - u, =
] + [ u, - u, ]
+ 143 F(u) + F(v). 142
(/2
+ (/3
228
Chapter 3
The "edor Space R"
SimiLarly, F(cu) =
[
cu, -cu, J= c [u, -u, J= cF(u). CII2
+ CU)
+
112
II)
so F is a linear lransfonnation. Note that F can also be defined as F(x) = Ax. "here A is the (2 x 3) matrix
A= E.\.\.\\PLE
2
[
1
-I
0
o
1
1
Define H: R 2 -+ R 2 by H(x)
=
xl-x2+1 [
J.
•
l .h"ex~ l [::
3X2
Determine 'Whether H is a linear transformation.
Solution
Let u and v be in R 2:
[ u'J
u=
112
and v=
["'J L'2
Then H(u~v)=
(ul-t'Il-(1I2+t'l)+1 [
~
3(112
.
J .
L'l)
while H(u)
+ H(",)
=
[ u, - '" + 1 J+ ["' - '" + 1 J 3112
= [
(/ll
3V2
+ L'l) -
+ L'!) + 2 + l'2)
(U2
3(112
J.
Thus v.e see that H(u +v) ¥= H(u) + H(v). Therefore. H is not a linearlransformation. Although. it is nOI necessary. it can also be \erified easily thai if C ¥= I. then H(cu) r cHeu). E.X+\\lPI E
3
•
Let \V be a subspace of R n such Ihatdim{W) = p.and let 5 = (WI. ""2 ..... Wpl bean orthonormal basis for IV. Define T: R~ ....... IV by T(v) = {\ TWd""t
+ (\,T w2)W2
-+- .•• _
(\,T ""p)""p'
Pro\e that T is a linear transformation.
Solution
If u and \' are in T(u
R~.
then
+ vl wiJwr + [(u + V)T w21w2 + ... + [(u + v)T wp]w/! = HUT + \'T)\\'11wl + [CUT + VT)W2JW2 + ... + [CUT + vT)\\'p]W p = (u T WI)""I + (u T w2)\\'2 + + (u T wp)W p +(\,T wI)\\ I + (\,T W2)W2 + -.,. (v TWp)W p = T(u) + T{\).
+ \') =
[(u
{4,
3.7 Linear Transformations from R" 10 R'"
II can be shown similarly that Tku) uansformation.
=::
229
cT(u) for each scalar c, SO T is a linear •
The \'ector T(\'> defined by Eq. (.J) is called the onhogonal projection of v onto tV and will be considered funher in Sections 3.8 and 3.9. As a specific illustration of Example 3. let tV be the subspace of R) consisting of all vectors of the form
ilj maui"
X, ]
•
X=
[
~
Thus tV is the .\)'-plane. and the set tel. e21 is an orthonormal basis for tV. For x in R),
x=
[." ] ::
•
the formula in Eq. (.J) yields T(x)
= (x T edel + (x T C2)C2 = Xle, + X2C2·
Thus.
rex)
~[~
l
This transformation is illustrated geomeuically by Fig. 3.15.
,
.
ormation. H(cu)
(.l"\.X2·.l"))
x
=::
• 14',}bean T(x)
J" ,
E\ \\\PLE 4Solution
y
(.l! • .l"2' 0)
x FigUff 3.15
, ,,, ~
Onhogonal projection onto the .l)·-plane
Let \V be a subspace of R~. and lei a be a scalar. Define T: lV --,. \V by T(w) = aw. Demonstrate that T is a linear transformation. If \' and
14'
are in tV. then
T(,·
+ w) = a(v + 14')
= a\'
+ a 14' =
T(v)
+ T(w).
230
Chapter 3
The Vector Space R-
Like\l,'ise. if e is a scalar. then T(ew) = a(ew) = c(ow) = cT(w).
•
It foUows that T is a linear transfonnation.
The linear transfonnation defined in Example 4 is called a dilation when 0 > I arx. a COlllroction when 0 < (/ < I. These cases are illustrated geometrically in Fig. 3.16.
Figun 3.16
a > I. dilalion
o < a < I. conlfaClion
(,)
(b)
Dilations and conlfaCtions
The mapping I: \V --+ \V defined by I(w) = w is the special case of Example 4 in which a = I. The linear transfonnation I is called the identity trans/onnation. t:\.\ \\1111:>
Solutio"
Let \V be a subspace of R~. and let (J be the zero vector in R"'. Define T: IV --+ R'" by T(w) = 9 for each w in IV. Show that T is a linear transfonnation. Let v and w be vectors in IV. and let c be a scalar. Then T(v
+ w) =
(J
= (J
+9 =
T(v)
+ T(w)
and T(cv) = 9 = c9 = eT(v).
•
so T is a linear transfonnation.
The linear transfonnation T defined in Example 5 is called the :uo trallS/ormatioll. Later in this section we will consider other examples when",e study a particular class of linear tralLSfonnations from R" to R 2. For the present. we tum to further properties of linear transfonnations.
The Matrix of a Transformation Let V and IV be subspaces. and let T: V __ IV be a linear transfonnation. If u and \' are vectors in V and if a and b are scalars. then the linearity properties (3) yield T(au
+ bv) =
T(au)
+ T(b\')
= aT(u)
+ bT(,-).
"
3.7 Linear Transformations from R" to Rill
Inducti\ely we can extend Eq. (5) to any finite subset of V. That is. if VI. \'2 ..... veclors in V and if CI. C! •••.. Cr are scalars. then nCI\'l
•
lena> land Fig. 3.16.
+ ell'! + ... + crvr ) = Cl T(l'l) + clT(l'l) + ... + Cr T(v r ).
Vr
are
,6,
The following example illustrates an application of Eq. (6). E'.\\\PI
rb
Let \V be the subspace of R 3 defined by
W
= Ix: x =
X,+2X,] x!
.
[ x,
X2
and X3 any real numbers}.
Then {WI. w:d is a basis for W. where
w, = Suppose that T: W Ul. "'here
[i]
'n
'lld
w, =
[
~
l
R l is a linearlransformation such Ihat
u, ~ [ :]
ple..J in
'lld
u, = [
_:
T(wd =
ul
and T(wl) =
l
lei the vector W be: gi\ en by R·b~
-
231
w
~
[: ]
Show that" is in 1V. express w as a linear combinalion of WI and determine Trw).
w2.
and use Eq. (6)
10
Solution
Ii folio", s from the description of 1V that
W
is in W. Funhennore. it is easy to see that
W = 3\\"1 - 2"'2.
•
B)
Eq. (6)•
Innation
Trw) = 3T(wd - 2T(W2) = 3u\ - 2U2 = 3 [ : ] - 2 [ _: ].
.vhrcl'b~ >~nie)
Thus. T,w)
~
[a
•
u and,
Example 6 illustrates that Ihe action of a linear transfonnation T on a subspace 1V is complelely determined once the action of T on a basis for W is known. Our next example provides yet another illuSiration of this fact.
~
232
Chapter 3
I\A\\PLf -
The Vector Space Rn
Let T: R 3 ---+ R 2 be a linear transformation such that
TC',)
~[ :
l
~[
TC',)
-:
l
'nd T(,,)
~[
a
For an arbitrary vector x in R 3 • X, ]
x=
[ ::
'
give a formula for T(x).
Solution
The vector x can be written in the form x
=
Xlet
+ X2e2 + X3e3,
so by Eq. (6). T(x) =
Xl
T(ed
+ X2T(e2) + X3T(e3).
(7'
Thus, T(X)=Xl
[ 1] [ -1 ] [2] [x, - xd 2x, ] 2
+X2
I
+x3
=
3
2XI+X2+3x3
.
•
Continuing with the notation of the preceding example, leI A be the (2 x 3) matrix with columns T(el). T(e2), T(e): thus. A = [T(ed. T(e2), T(e))] = [
~
-1
1
a
It is an immediate consequence ofEq. (7) and Theorem 5 of Section 1.5 that T(x) = Ax.
Thus Example 7 illustrates the following theorem.
-i
111 EORf\\
15
Let T: Rn ---+ Rm be a linear transformation, and let el. e2, .... en be the unit vectors in R n. If A is the (m x n) matrix defined by A = [T(el)' T(e2),"" T(e n )],
then T(x) = Ax for all x in Rn.
Proof
[f x is a vector in Rn,
X, ]
x,
x=
.
[
.;n
then x can be expressed in the form
x=
Xlel
+ X2e2 +.. + xnen .
3.7 Linear Transformations from R II to R'"
233
It now follows from Eq. (6) that
rex) =
Xl T(el)
+ X2T(e2) -r- ... + x"T(e,,).
",
if A = [T(ed. T(e!) . .... T(e,,)].lhen by Theorem 5 of Section 1.5. the right-hand side of Eq. (8) is simply Ax. Thus Eq. (8) is equivaJenllo rex) = Ax. • E,\.\\\Pl L~
Let T: R 2 -+ R 3 be the linear transformation defined by the formula
T ([
x,X2 ])
= [ _:'
+ 2x,
]
1+-"2·
2.\'1 -
Xl
Find a matrix A such that T(x) = Ax for each x in R!. Solution
By Theorem 15. A is the (3 x 2) matrix A = [T(el). T(e!)).
II is an easy calculation that
T(e,) = [ -
•
~
]
,nd T(e,l = [
J
Therefore,
A=[-:2 -1:]
...
One can easily verify that
rex) =
Ax for each x in R 2 ,
•
:\'u11 Space and Range Associated with a linear transformation. T. are two imponant and useful subspaces called the null space and the range of T. These are defined as follows.
D-I:::fl" ITIOr-..
t)
Let V and W be subspaces. and let T: V --+ \V be a linear transformation. The null space of T, denoted by N(T). is the subset of V given by ..'\'(T)
= Iy: ,. is in V and TCY} = O}.
The range of T, denoted by 'R(T). is the subset of \V defined by 'R(T) = {w: w is in Wand w = T(y} for some " in V}.
That N(T) and R(T) are subspaces will be proved in (he more general COnlex[ of Chapter 5. If T maps RII into R"'. then by Theorem 15 there exists an (m x n) matri'\
234
Chapter 3
The Vector Space R"
A such that T(x) = A x. In this case il is clear that the null space of T is the null spa,~ of A and the range of T coincides with the range of A. As with matrices. the dimension of the null space of a linear transfonnation T I' called the lIullity of T. and the dimension of the range of T is called the rank of T. I: T is defined by matrix multiplication. T(x) = Ax. then the transformation T and Ill.: matrix A have the same nullity and the same rank. Moreo\"er. if T: Rn __ Riff. then A. is an (m x n) matrix. so it follows from the remark in Seclion 3.5 that rank(T)
+ nullily(T) =
/1.
Formula (9) will be pro\'ed in a more general setting in Chapter 5. The next twO examples illustrate the use of the matrix of T 10 determine the null space and the range of T. I \ \ \\PU: 9 Solution
Let F be the linear transformation gken in Example I. F: R3 -+ R~. Describe the null space and Ihe range of F. and delennine the nullity and the rank of F. It follows from Theorem 15 that F(x) = Ax. where A is the (2 x 3) matri:< A = (F(e,). F(e2). F(e)) =
[
1 0
-1 I
a
Thus the null space and the range of F coincide. respectivel). wilh the null space and the range of A. The null space of A is determined by backsol\'ing the homogeneous system Ax = 8. where x is in R 3: X, ]
x=
[
.~~
.
."
This gives ..\'(F)
=
;\"(A)
= Ix:
Xl
=
-X3
and X2 =
-x,l.
Using the techniques of Section 3.4. we can easily see that the \ector
u
= [
=: ]
is a basis for .\'(Fl, so F has nullilY I. By Eq. (9), rank(F) =
II -
nullity(F) = 3 - 1 = 2.
Thus 'R( F) is a two-dimensional subspace of R~. and hence 'R( F) = R~. Alternatively. note that Ihe system of equations Ax = b has a solution for each b in R~. so R(F) = R(A) = R". • ['"","flU:
10
Let T: R1 -- R3 be Ihe linear transformation ghen in Example 8. Describe the null space and the range of T, and determine the nullity and the rank of T.
Ie
3.7 Linear Transformations from R It to R-
null space
'I'lUtion T i~ m!ofT.1f D T and the R" . then A
SOlllliO/I
235
In E:<.ample 8 it was shown that T(x) = Ax. where A is the (3 x 2) matrix
A~[~: ~]. 2
Ifb is the (3 x I) vector.
b~
the null
-I
[:J
then the augmented matrix (A I bj for the linear system Ax = b is row equivalent to the null [
=
I
0
o o
1
(1/3)b, - (2/3)b, (1/3)b, + (l/3)b,
0 (-lj3)b l
and the ) 'tem
~
(10)
+ (5j3)b 2 + b)
=
Therefore. T(x) Ax b can be solved if and only if 0 The range of T can thus be described as 'R(T)
]
= (-I j3)b l + (5j3)b2 + b3•
'R(A)
= fb: b = [
::
].
b l and b2 any real numbersl.
(1/3)b, - (5/3)b, A basis for 'R(T) is lUI. U2} where
u,
~[ ~
]
and U,
1/3
~
[
~]
-5/3
Thus T has rank 2. and by Eq. (9). nullity(T) = n - rank(T) = 2 - 2 = O. II follows that T has null space {9}. Altemathely. it is clear from matrix (10). \lo'ith b = 9. that the homogeneous system of equations Ax = 9 ha... only the trivial solution. • Therefore...qT) = .A'(A) = 19}.
Orthogonal Transformations on R 2 (Optional)
b ..
• n
" is often infonnati\'e and useful to view linear transfonnations on either R 2 or R 3 from a geometric point of view. To illustrate this general notion. the remainder of this section is de\oted to determining those linear tmnsfonnations T: R 2 -+ R 2 that preserve the length of a vector: that is. we are inte,rested in linear transformations T such that
IIT(v)1I
~
IIvll
(II)
for all v in R 2. Transfonnations that satisfy Eq. (II) are called orthogonal transforma. tiolls. We begin by giving some examples of orthogonal transformations.
236
-i
Chapter 3
-EX·MIPl£ 11
The Vector Space R n
Let 8 be a fixed angle. and let T: R" -+ R" be the linear transformation defined ~ T(v) = Av. where A is the (2 x 2) matrix
_[co,O -'inO] .
A_
cos 8
sin (}
Give a geometric interpretation of T. and show that T is an orthogonallransfonnation.
Solution
Suppose that v and T(v) are given by v
~[
"d T(v)
:]
~[ ~].
Then rc\') = A \', so
-'inO] [a] ~ ["OOSO -b"nO ]
[ ~ ]~ [ CO,O sinO
cos8
{ll
asm8+bcosO
b
We proceed now to show thai T(v) is obtained geometrically by rotating the vector, through the angle 8. To see this, let ¢ be the angle between" and the positive x-axis (see Fig. 3.17), and set r = II vII. Then the coordinates (/ and b can be wrilten as
a=rcos¢.
b=rsin¢.
(J.l
Making the substitution (13) for a and b in (12) yields
c = ,. cos rP cos 8 - r sin ¢ sin 8 = r cos(¢!
+ 8)
nnd d = r cos¢sin8
+ r sin¢cos8 =
r sin(¢
+ 8).
Therefore, c and d are the coordinates of the point obtained by rotating the point (a. b) through the angle O. Clearly then, IIT(v)1I = IIvll. and T is an orthogonal linear transformation. _
y
d
(c. d) ""'"
,
",,
"x
c
Figure 3.17
(a. b)
Rotation through the angle 8
The linear transformation T defined in Example II is called a rotation. Thus if A is a (2 x 2) matrix.
A~ [a -b]. b
"
--
3.7 Linear Transformations from R" to R"
1
=
where a~ + b1 1. then Ihe linear transfonnalion T(v) the angle 8. 0;5 8 < 2JT. wherecos8 = a and sin8 = b.
defined b~
fXi\<\IIJU:
12
237
= Av is the rOlalion through
Define T: R~ _ R1 by T(\') = A". where
Ormation.
A
~1/2 ./3/2 ] . ~ [ -./3/2 -1/2
Give a geometric imerpretation of T.
Solution
Since cos(4iT 13) = - t/2 and sin(4rr 13) = -fl/2. T is the rotation through the angle
4JT/J·
,, \e.,:lor \
Now leI I be a line in the plane Ihat passes through the origin. and leI " be a vector in the plane. If we define T(v) to be the symmetric image of v relntive to J (see Fig. 3.18). lhen clearly T preserves the length of v'. It can be shown Ihat T is mulliplication by the matrix
,,
,, ,, ,
-e :r·a"l
.'
•
,, ,, ,
A~[CO'8
\-f
sin 8
T(v)
Figun3.18
Reflection about a line
"08 ] -cos8
.
where (1/2)8 is the angle between I and the positive x-axis. Any such lran!>formation is called a rejlecriolJ. Note that a reflection T is also an orthogonal linear lransformation.
ab li"""
•
EM\\PI t: 13
Lei T: R~ ...... R~ be defined by T(\') = Av. where A is the (2 x 2) matrix
A~[
1/2
./3/2
./3;2 ]. -1'2
Ghe a geometric imerpretalion of T.
Solution
=
=
Sincecos(iT/J) 1/2 and sin(rr 3) ./3/2. T is the reflection about the line I. where I is the line thai passes through the origin at an angle of 30 degrees. _ The next theorem give, a characterization of orthogonal transformations on R1. A consequence of Ihis theorem will be that every orthogonal tranSformluion is either a rotation or a reflection.
itA
THl:OI~l:\\ Ie:,
Let T: R~ -+ R~ be a linear transformation. Then T is an orthogonal transformation if and only if IIT(edll UT(e2 HI 1 and T(et> is perpendicularto T(e1)·
Proof
If T is an orthogonal tmosformalion. then IIT(v)lI = IIvl: for every vector \' in R 2. In particular. IlT(elHI := I,edl := I. and similarly IIT(e~)1I = I. Set u\ = T(ell. U1 = T(e~). and \. = IV = el + e~. Then
=
=
rl.
2 = ","' ~ IIT('lIl' ~ IIT(o,
+ o,lll' ~ llreo,) + T(o,lll'.
238
Chapter 3
The Vector Space R n
Thus. 2 = lIuI
=
(Uj
+ u211 2 + U2)T (Uj + U2)
= (u; + uiHul + U2) - U' +U' U2 +U1' Ul +U'2 U2 l Uj j
2
= !lUI 11 + 2ur U2 + lIu2112 =2+2U;U2'
uf
It follows that U2 = 0, so Ul is perpendicular 10 U2. The proof of the converse is Exercise 47.
•
We can now use Theorem 16 to give a geometric description for any orthogonal linear transformation. T, on R 2. Firsl, suppose that T(ed Ul and T(e2) U2. If
u,
~ [:
=
l
=
then 1 = lIullI" = a" + b 2 • Since lIu211 = I and U2 is perpendicular 10 Ul. there are two choices for U2 (see Fig. 3.19): either
u,
~[
-:]
m u,
~[
_: ]
In either case. it follows from Theorem 15 thai T is defined by T(v) = Av, where A is the (2 x 2) matrix A = [Ul. u21. Thus if
l -: l
u, ~ [ -: th,n A
~[
:
Mod
(-b,o)
u,
--- --
• " (0. b)
<,
,, ,, , ,,, ,,
,,/
(b. -0)
Choices for
U2
.,T
c>T
y
Figure 3.19
L
,
c
'-'-"
3.7 Linear Transformations from Hit to R m
239
so T is a rotation. If
u'~[_~l th'"
A~[: -~l and T is a reflection. In either case note that ATA = f, so AT = A -I (see Exe,rcise 48). An (II x II) real matrix with the propeny that ATA = I is called an orthogonal matrix. Thus we have shown mal an onhogonal transformation on R 1 is defined by
•
T(x) = Ax.
<
If
where A is an orthogonal matrix.
3
""
EXERCISES
I. Define T: R2 ...... R1 by
T ([ ;; ])
A
~[
Which of the following vectors are in the null space ofT?
2x, - 3x, ]. -Xl
+
X2
Find each of the following. aj T ([
,j
T ([
~]) ~])
oj
d) T ([
,j [ ; ]
-~ ])
_[ 1-1]
A_
I
-3
3
T ([
~])
bj T ([ : ])
~])
dj T ([
3. L=l T: R3
--
R"
.
.f>
~ ])
+ 2r1 + 4-':3 2\"1 + 3X 1 + 5X3 Xl
-a
5. Let T: R" _ R2 be lhe function given in Exercise I. Show lhal for each b in R2, there is an x in R 2 such that T(x) = b. 6. Let T be the linear transformation gi\en in Exercise 2. Find x in R 2 such mat T(x) = b. where
be the linear lransfonnation
([ x:X']) = [
-1/2
b~[
fined by
T
dj [ -:/2 ]
4. lei T: R" -+ R 1 be the fUJ1Clion defined in Exercise I. Find x in R" such thai T(x) = b. where
Find each of the following.
,j
bj [
bj T ([ : ])
2. !kline T: R" -+ R1 by T(x) = Ax. where
aj T ([
[n -n
].
de-
b~[
-a
7. Let T be the linear transformalion given in Exercise 2. Show mal mere is no x in R2 such that
240
The Vector Space R n
Chapter 3
16. F: R2 ...... R defined by
T(x) = b for b
~[
: ]
F ([ :: ])
In Exercises 8-17. detennine whether the function F is a linear transfonnation. 8. F: R 2 ...... R 2 defined by
F ([ :: ])
~[
2x, -
." ]
+ 3x2
Xl
9. F: R 2 -+ R 2 defined by
F([ :: ]) ~ [ :: ] F([::]) ~[x,~x,]
~ Ix,1 + Ix,l
T
18. Let W be the subspace of R 3 defined by
T
F ([ :: ])
~ I" ~ [;:] ~ ~ x,
,~[ ~
II. F: R 2 -+ R 2 defined by
~ x~:, ]
12. F: R 3 -+ R 2 defined by
F ([ :: ])
~
[
-Xl
+
+ 3X2 -
XJ ]
2x J
13. F: R3 -+ R:' defined by
14. F: R:' -+ R J defined by
~
l
-
", ~ U]
ood ",
~ ~ [
]
Find each of the following.
(
b)T([_~])
I
,jT([:]) x, - x, ] [
-XI
+ X2
x,
~ ~ [
20. Let T: R 2 -+ R 2 be a linear transfonnation such that T(vl) = Ut and T(\'2) = U2' where
"~ [~ 1 "~ [-: 1
15. F: R:' -+ R 3 defined by
F ([ :: ])
0).
,j T ([ : ])
F([::]) ~ [:: ] F ([ :: ])
x)
Give a geometric interpretation of IV, v. and T (\'). 19. Let T: R 2 -+ R 3 be a linear transformation such that T(el) = U\ and T(e2) = U2, where
[
X2
.J T
Find an orthononnal basis for W. and use Eq, (.t) of Example 3 to give a formula for the orthogonal projection T: R J -+ W; that is. determine T(v) for arbitrary \' in R3 ;
10. F: R 2 ...... R 2 defined by
Xl -
~ 2x, + lx,
17. F: R 2 ...... R defined by
IV
F ([ :: ])
t"..
]
",
~ [ ~ ].
"d ",
~[
: ].
--
3.7 Linear Transformations from R" to R-
Find each of the roIlO'o\'ing.
~OJ
T ([ - : ] )
b)T([_~])
T([ -i])~[n
~
Tn Exercises 25-30. a linear transformation T is given.
])
In E'l:crcises 21-24. the action of a linear trnnsformation T on a basis for either R l or RJ is gi\en. In each case y,.e Eq. (6) to dcrh'e a formula for T.
'" ,J
:onal
T'\ ior
'I.T([: ])=[ _~] ~d
In each case find a matrix A ~uch that T (x) = Ax. Also de!>Cribe the null space and the range of T and give the nmk and the nullity of T.
25. T: Rl .......
R~ defined by
T ([ x, ]) Xl
T ([ :: ]) T • 11. T ([ : ])
~
~
T ([ -: ])
D T ([
~
])
=[
T ([ -:])
T([ U
T ([
[ :]
[
n ~l
~ [a
-l]) ~[n ~
])
~d
= [ -:]
~[
;" + 3x, ]
+
_1"1
Xl
26. T: R l --. R) defined by
~[ ~]
T ([ _: ])
~u.:h
~ [J
aJ T ([ : ])
oj T ([
241
27. T: R" -
~
[ ::::: ]
R defined by
T ( [ : : ])
~ 3" + 'x,
28. T: R 3 -.. RJ defined by
X']) [X' + x,
TX2=X)
([
29. T: R J
x~
.....,.
]
Xl
R2 defined by
T([::]) =[ :;=::] 30. T: RJ
T ([
......
R ddined b)'
:J)
= 2<, -x,+4x;
31. Let a be a real number. and define. j: R _ R by f(x) = ax for each.1" in R. Show lhal f is a linear
transformation.
242
Chapter 3
The Vector Space RR
32. Let T: R ---+ R be a linear transformation. and suppose that T( I) = a. Show that T (x) = ax for each x in R. 33. Let T: R 2 -"" R" be the function that maps each
defining by [G 0 Fj(u) = G(F(u» for each u in V, is a linear transformation. 40. Let F: R3 -+ R" and G: R" -+ R3 be linear transformations defined by
point in R" to its reflection with respect to the ~-axis. Give a formula for T and show that T is a linear transformation. 34. Let T: R" -+ R" be the function that maps each point in R 2 to its reflection with respect to the line )' = x. Give a formula for T and show that T is a linear transformation. 35. Let V and IV be subspaces, and let F: V -+ IV and G: V -+ IV be linear transformations. Define F G: V -+ IV by [F G](v) = F(v) + G(\·) for each v in V. Pro\'e that F G is a linear transformation.
+
+
+
36, Let F: R 3 -"" R" and G: R 3 -+ R" be defined by
F
(["']) ::
=
[
2xI 4Xt
3_~2
+
+ 2X2 -
X3
5X3
]
and
c:: (["D ~ [-2x, I
_ XI +4X2 +2X3 +]"
+
3XJ
a) Give a formula for the linear transformation
F + G (see Exercise 35). b) Find matrices A, B. and C such that F(x) = Ax, G(x) = Bx. and (F + G)(x) = Cx. c) Verify that C = A + B. 37, Let V and IV be subspaces, and let T: V -+ IV be a linear transformation. If a is a scalar. define aT: V -+ IV by [aTl(v) = a[T(v)] for each v in V. Show that (./ T is a linear transformation. 38. Let T: RJ -+ R2 be the linear transformation defined in Exercise 29. The linear transformation [3 T]: R 3 -+ R 2 is defined in Exercise 37. a) Give a formula for the transformation 3T. b) Find matrices A and B such that = Ax and [3T](x) = Bx. c) Verify thai B = 3A.
F([::J)~[
0 F: R3 -4 R3 is a linear transformation. Give a formula for G 0 F. b) Find matrices A. B. and C such that F(x) = Ax, G(x) = Bx, and [G 0 FJ(x) = Cx. c) Verify that C = BA.
a) By Exercise 39. G
41. Let B be an (m x n) matrix, and let T: R" -4 R'" be defined by T(x) = Bx for each x in R n . If A is the matrix for T given by Theorem 15, show that A = B. 42. Let F: R n -+ RP and G: RP -+ R'" be linear transformations. and assume that Theorem 15 yields matrices A and B. respectively. for F and G. Show that the matrix for the composition G 0 F (sec Exercise 39) is BA. [Him: Show that (G 0 F)(x) = BAx for x in R n and then apply Exercise 41.] 43. Let I: R" -+ Rn be the identity transformation. Determine the matrix A such that / (x) = Ax for each x in R n . 44. Let a be a real number and define T: R" -+ R n by T(x) = ax (see Example 4). Detennine the matrix A such that T (x) = Ax for each x in Rn . Exercises 45-49 are based on the optional materia1. 45. Let T: R 2 -+ R 2 be a rotation through the angle 9. In each of the following cases, exhibit the matrix for T. Also represent v and T(v) geometrically, where
~
• •
c([::]) ~ [~:::~::]
v
~
2XI+5x2+ X3
"d
rex)
39, Let U, V. and IV be subspaces. and let F: V -+ V and G: V -+ IV be linear transformations. Prove that the composition G 0 F: U -+ IV of F and G,
-Xl+ 2x 2- 4x 3 ]
.,
[
: ]
a) 9 = ]I" 12 b) 9 = Jr 13 c) 9 = 27r 13 46. Let T: R'- -+ R2 be the reflection with respect to the line I. In each of the following cases, exhibit
3.8 Least-Squares Solutions to Inconsistent S)·stems. with Applications to Data Fitting e.lC'h u in U. near lrans_
""x,
]
~ linear
F.
, XI =
the malrix for T. Also re,present el. <:2. T{el). and T{e2) geometrically. a) I is the x-axis. b) 1 is the y-axis. c) I is the line with equation.'" = x. d) I is the line with equation y = .J3x. 47. Lei T: R 2 -+ R2 be a linear transformation that satisfies the conditions of Theorem 16, Show that T is orthogonal. (Him: If v = [Il, bV, then \' = ael + be1. Now use Eq. (6).J
243
48. Let T: R 2 -+ R2 be an orthogonal linear transfonnation. and let A be the corresp::mding (2 x 2) malrix. Show that ATA = f. [Him: Use Theorem 16.J 49, Let A = [AI. A1J be a (2 x 2) matrix such that ATA f, and define T: R 2 _ R" by T(x) Ax.
=
=
a) Show that fA I, A11 is an orthononnal Set. b) Use Theorem 16 to show that T is an orthogonal transformation.
LEAST-SQUARES SOLUTIONS TO INCONSISTENT SYSTEMS, WITH APPLICATIONS TO DATA FITTING When faced wilh solving a linear syslem of the form Ax = b. our procedure has been to describe all solutions iflhe system is consistent but merely 10 say "there are no solutions" if the system is inconsistent. We now want 10 go a step further with regard 10 inconsistent systems. If a given linear system Ax = b has no solution. then .... e .....ould like to do the next best thing-find a vector x· such Ihat the residual "ector. r = Ax· - b. is as small as possible. In terms of praclical applications. we shall see that any technique for minimizing Ihe residual "ector can also be used to find best least-squares fits to data. A common source of inconsistent syslems are ol'f!rdf!ttrmined s)'sttms (that is. systems wilh more equations than unkn()\\'ns). The syslem that fo11o....s is an e>;ample of an o\'erdetennined system: XI
+ 4X2 =
-2
0..
XI
+2(1 =
6
'fore~
21"1
+ 3X1 =
I.
OH=rdelermined systems are often inconsistent. and the preceding example is no exception. Given that the above system has no solution. a reasonable goal is to find values for Xl and X2 that come as close as possible 10 satisfying all three equations. ~ethods for achieving that goal are the subjecl of this section.
Least·Squares Solutions to Ax = b Consider the linear system Ax = b where A is (m x 1/). If x is a vector in Rn , then the vector r = Ax - b is called a residual ~·f!ClOr. A \'ector x' in R~ Ihat yields Ihe smallesl possible residual ,ector is called a least-squares solution to Ax = b. More precisely. x' is a least-squares sofutioll to Ax = b if
IIAx· - bll ::: IIAx - bl!. for all x in Rn •
,
(If Ax = b happens to be consistent. then a least~squares solution x' is also a solution in the usual sense since IIAx' - bll = 0.)
244
Chapter 3
The Vector Space R M
The special case of an inconsistent (3 x 2) system Ax = b suggests how we can calculate least-squares solutions. In particular. consider Fig. 3.20 which illustrates a vector b that is not in R(A); that is. Ax = b is inconsistent.
b
,1/
I
~ y .~ y
R(A)
., Figure 3.20
)" is the closest vector in R(A) to b
Let the vector y. in R(A) be the closest vector in R(A) to b: that is II,Y~
- bll ::; IIY - bl!.
for all y in R(A).
Geometry suggests (see Fig. 3.20) that the vector y. - b is orthogonal to every vector in R(A). Since the columns of A form a spanning set for R(A). this orthogonality condition leads to
A;(,Y· - b) =0 ArC,Y- - b) = 0 or. in matrix-vector terms. AT(,Y--b)=(}.
Since,Y° = Ax~ for some x- in R 2 • the preceding equation becomes AT (Ax~ - b) = 8. m
ATAx· = Arb.
Thus, the geometry of the (3 x 2) system. as illustrated in Fig. 3.20, suggests that we can find least-squares solutions by solving the associated system (I):
ATAx = Arb. As the following theorem asserts, this solution procedure is indeed valid. ~ TttEOREM
17
Consider the (m x II) system Ax = b. (a) The associated system ATAx = ATb is always consistent. (b) The least-squares solutious of Ax = bare prel.:isely the solutions of ATAx = ATb. (c) The least-squares solution is unique if and only if A has rank n.
•
3.8 Least·Squares Solutions to Inconsistent S)'stems, with Applications to Data "'ltting
245
We will give the proof of Theorem 17 in the next seelion. For now, we will illustrale Ihe use of Theorem 17 and also gi\c some examples showing Ihe connections between data-filting problems and leasl-squares solutions of inconsi!>lcm systems. (In parts (a) and (b) of Theorem 17. [he associated equalions ATAx :::: Arb are called the normal f'quQlions.)
E\.\\\Pll: 1
Solution
Find the least-squares solutions to the inconsistent system Xl
+4X2::::-2
Xl
+ 2X2 =
6
2fl +3X2 =
I.
By Theorem 17. we can find the leasl.squares solUlions by solving Ihe normal equalions. ATAx
=
Arb..... here
A
~
[:
~]
"d b = [
-~
]
Forming ATA and Arb. we obtain
ATA~[ 12612] "d 29
ATb=[6]. 7
Solving the syslcm ATAx :::: Arb. we find the least.squares solUlion
x'~ [-a =
•
Least-Squares Fits to Data One of the major applications for least-squares solutions is to the problem of detennining best least-squares fits to data. To introduce this imp:mant topic. consider a table of data such as the one displayed next.
Table 3.1
H ~
•
" )'0
"
y,
" )'2
"
.
-,
Suppose. when we plot the data in Table 3.1. that it has a distribution such as the one sho\.\n in Fig. 3.21. When we ex:amine Fig. 3.21. it appe~ that the data (Ktints nearly fall along a line of the form y = ml + c. A logical question is: "What is the best line that \.\e can draw through the data. one mat comes closest to representing the dataT
246
Chapter 3
The Vector Space R fl
y
•
• Figure 3.21
• • • •
•
A nearly linear distribution of data points
In order (0 answer this question. we need a way to quantify the terms best ano cfose.n, There are many different methods we might u!>c to measure best. but one oflhe most useful is the least-squares criterion: Find 11/ and c to minimize
" L {(lilt; + c) ,,,,,
)'1 J1
The panicular linear pol) nomial J = lilt + c that minimizes the sum of squares il' Eq. (2) is called the b~st least·squar~s linearjit to the data in Table 3.1. (We see in tht next section that best least-squares linear fits always exist and are unique.) Similarl). suppose the set of data points from Table I has a distribution such as tM one displayed in Fig. 3.22. In this case. it appears that the data might nearly fall along lhe graph of a quadratic polynomial J = all + bt + C. As in Eq. (2). we can use a least-squares criterion to choose the besr leasr-sqllan's quadraric fit: Find a. b. and c to minimize
" L ,,,,, [(at? + btl + c) -
J,ll.
In a like manner. we can consider fitting data in a least-squares sense with pol)nomiah of an) appropriate degree.
,.
•
• •
Figure 3.11
• •
• •
•
•
A nearly parabolic disuibution of data points
In the next several subsections. we examine the connection between least-squares fits to data and least-squares solutions to Ax = b,
3.8 Least-Squares Solutions (0 Inconsistent S:rstems. with Applications to Data Fining
247
Least-Squares Linear Fits to Data Suppose the laws of physics tell us that two measurable quantities, / and)". are related in a linear fashion:
-',
y=m(+c.
Suppose also that we wish to determine experimentally the values of 1/1 and c. If we know that Eq. (3) models the phenomena exactly and that we have made no experimental error. then we can determine m and c with only two experimental observations, For instance. if y = Yo when / = to and if y = .n when / = (I, we can solve for 1/1 and c from mto+c=,\'o nltl
~Sl ~
o'lb<
+c =
,n
m
[to I][m]~[)'o]. /1
I
)"1
C
Usually we must be resigned to experimental errors or to imperfections in the model given by Eq, (3). In Ihis case. we would probably make a number of experimenlal observations. (tj.)"i) for i = O. I..... k. Using these observed values in Eq. (3) leads to an o\erdetennined system of the form 1/I10-C
= Yo
mIl
=)"1
-l-
c
mli-C=Yt·
In matrix lenns. this o\'erdelennined S)Slem can be expressed as Ax = b. where
A =
[
, to '] . .
t, .
I,
X~[:l ond b~[:::]'
,
~'k
In this contexi. a least-squares solution to Ax minimizes liAx - bl:, where
",...,•
IIAx - b ,-, = L.., [(mt,
b is a vector x· =
+ c) -
1m". c·f that
y;]-. '
Comparing the equation abo\'e with Ihe leasl-squares criterion (2). we see thai the best least-squares linear fil. y = m-' + c·, can be detennined by finding the leasl-squares solution of Ax = b.
E, \ \\PUc:2
Con~ider the experimental obsenations gi\en in the following table:
~ y
II
"2
8 4
II
,
Find the least·squares linear fit to the data.
,
248
Chapter 3
SO/l/Iion
The Vector Space R n
For the function defined by )' =
lilt
+ c, the data lead to the overdetermined system m+c= I 4m+c=2 811l+c=4 11m +c= 5.
In matrix terms, the system is Ax = b, where
A~U il X~[';l "d b~[~] The least-squares solution, x~. is found by solving ATAx = ATb, where ATA = [ 202
24
",d A'b~ [96]
2: ]
12
.
There is a unique solution to A TAx = ATb because A has rank 2, The solution is
X'~[ 12 /29]. 15/29
Thus the least-squares linear fit is defined by 12 ) ' -- 29 t
15 + 29'
•
The data points and the linear fit are sketched in Fig. 3.23,
r 6 4 2
2
Figure 3.23
4
6
8
10
12 r
The least-squares linear fit to the data in Example 2
Using MATLAB to Find Least-Squares Solutions Up to now we have been finding least-squares solutions to inconsistent systems by solving the normal equations ATAx = ATb. This method is fine in theory but (because of roundoff error) it is not reliable for machine ca1culations---especial1y for large systems
2~9
3.8 Least·Squares Solutions to Inconsistent Systems, "'ith Applications to Data fitting
"~'Iem.
Ax = b. r-.IATLAB has several reliable altemalhes for finding least-squares solutions to inconsistent systems: these methods do not depend on solving the nonnal equations. If A is not square. the simple MATLAB command x = A\b produces a least· squares solution to Ax = b using a QR-factorization of A. (In Chapter 7. we give a thorough discussion of how to find least~squares solutions using QR-factorizations and Householder transformations.) If A is ~quare but inconsistent. then the command x = A\b results in a warning but does not return a least-squares solution. If A is not square. a warning is also issued when A does not ha\e full rank. In the next section we will give more details about these matters and about using MATLAB to find least·squares solutions.
L\ \ \\ PI r.3
b
•
-
Lubricating characterislics of oils deteriorate at elevated temperatures. and the amount of bearing wear. y. is nonnally a linear function of the operating temperature. I. That is. .r = nil + b. By weighing bearings before and after operation at various temperatures. the following table was constructed: Opernting tempernture. "e
120
1'8
2l"
175
---
Amount or wear. gmllO.OOO hr
•
3
5
232
160
288
316
3·"
311
6
7.5
8.8
10
ILl
11
5.5
Detennine the least-squares linear fit from these readings and use it 10 delennine an operating temperature that should limit bearing wear 10 7 gm/IO.000 hr of operation. SOllllioll
For the system Ax = b. v.e see Ihal A and b are gi\'en by
A = [
b
~
12~
13.
4.
1.8
175
20-l
232
160
288
316
343
1
1
1
1
I
1
1
1
5. 5.5. 6.
7.5,
8.8.
10.
I I. I.
37:]'
12f'.
The least~squaressolution to Ax = b is found from the MATLAB commands in Fig, 3.24,
»A.. [UO 148 175 204 232 260 288 316 343 371; 1] , ; 1 1 1 1 1 1 1 1 1 »b.. [3 4 5 5.5 6 7.5 8.8 10 11.1 12] , ;
»x.. A\b x. 0.0362 ·1.6151
Figure 3.14
The MATLAB commands for E~ample 3
250
Chapter 3
The Vector Space Rn
From Fig. 3.24 we see that the least-squares linear fit is
y = (0.0362)1 - 1.6151. Setting y = 7 yields t = 237.986. Hence an operating temperature of about 238 c C should limit bearing wear to 7 gm/lO.000 hr. _
General Least-Squares Fits Consider the following table of data:
'0 )'
)0
"
"
.n
.'"2
'. ,.'
When the data points (Ii. Yi) are plotted in the ty·plane. the plot may reveal a trend that is nonlinear (see Fig. 3.25). For a set of data such as that sketched in Fig. 3.25. a linear fil would not be appropriate. However, we might choose a polynomial function. y = p(t). where p(t,):::::.\',.O:::;i:::;m.
In particular, suppose we decide to fit the data with an 11th-degree polynomial: p(t) = lInt n
+ lIn_lt n- 1 + ... + alt + ao.
Ill::: II.
,
I
• •
•
• '0
Figure 3.25
•
•
•
•
•
, "
"
Nonlinear data
As a measure for goodness of fit. we can ask for coefficients minimize the quantity Q(ao, al ..... an)' where
00.
a I..... an that
•
Q(ao·(/j·· ... (/,,) =
L [p(I,) -
)'if
i=O
•
=
L ; ..0
[(00
+al1i
+ ... +a"t7) -
yilt.
"
3.8 Least·Squares Solutions to Inconsistent S~·stems. with Applications to Data Fitting
251
As can be seen b} in:-.pection. minimizing Q(ao. at ..... all) is the same as minimizing lAx - bl:!. "here
t
.:!3!'1 C
•
j A
~
[
10
tJ ...
I,
I'
,
I~,
I" ]
:.:=
; I.
[ G, Go ] :
r"
I'
•
YO]
.
[ :~
and b=
a"
•
(5)
As before. we can minimize I1A:.: - bill = Q(ao. {Jt ....• (J~) by solving ATAx = ATb. The 11th-degree polynomial p. that minimizes Eq. (4) is called the least-squares nthdegree fit.
f, \\\1'1 I
~
Consider the data from the following table: -I
0
I
,
5
3
2
4
1M Iule.u
fil
=Dt.
I
~ J
-2 12
Find the least-squares quadratic fit to the data.
Solutio"
Since we want a quadratic fit. "e are trying to match the fonn y = ao + alt the data. The equations are ao - 2a 1 +-Wl2 = 12
ao -
al
+
al
3 = 2 +4a2 = 4.
ao
+ 02t2
to
a2 = 5 =
ao
+
(10
+ 201
+
02
This overdetermined sy1itcm can be shown to be inconsistent. Therefore. we look for a least-squares solution 10 Ax = b. where A and b are as in system (5). with /I = 2 and III = 4. The matrix A and the vectors x and b are
A ~
a. tbx
•
-2
4
-I
1
I 1
0
0
I •
1
1
2
4
12
X~[::l
5
3
and b =
2 4
The least-squares solution of Ax = b is found by solving ATAx = ATb. where
ATA =
[
5 0 10]
0
10
0
10
0
34
and ATb=
[ 26] -19
71
.
252
Chapter 3
The Vector Space R-
The solution is x· = [87/35. -19/10. 19/14]. and hence the least-squares quadratic fi:
is p(t)
A graph of)' =
19,
19
= -r -
-I 10
87 +-. 35
I-\. p(l) and the data poinls are sketched in Fig.
3.26.
•
y
10
., , -,
-2
Figure 3.26
._.!22_'!! +!!
)-IJ 1
JOt
35
2
Least-squares quadratic fit for the data in Example 4
The same principles apply when we decide to fit data with any linear combination of functions. For example. suppose)' = f(t) is defined by f(t)
=
(/181 (t)
+ (/282(t) + ... + (/ng,l(t).
whele 81.82 ..... 8n are given functions. We can use the method of least squares to determine scalars al. (/2 •.... an thaI will minimize Q(al.a2 .. · .• a,,) =
L• If(t,) - y,i i_I
•
=
,.,
L {Ialg\(t,) + alg2(t;) + ... + all 8" (1/» _ y,}2 .
..,
The ideas associated with minimizing Q(a\. a1 . .... all) are explored in the exercises.
Rank Deficient Matrices In each of Examples I--t the least-squares solution to Ax = b was unique. Indeed. if A is (m x n). then pan (c) of Theorem 17 states that least-squares solutions are unique if and only if the rank of A is equal to n. If the rank. of A is less than 11. then we say that A is rank deficiellt. or A does not havefull rank.
3.8 Least·Squares Solutions to Inconsistent S)'stems, with Applications to Data Filling
" quadralic fil
Therefore. when A is rank deficient. there is an infinite family of least·squares solutions to Ax = b. Such an example is gi\cn next. This example is worked using MATLAB. and we note thai MATLAB produces only a single least-squares solution but does give a warning that A is rank deficient. In Section 3.9 we will discuss this topic in more demil.
• E:\+\,\\Pl
r
j
For A and b as given. Ihe system Ax = b has no solution. Find all the least·squares solutions
A
Solution
bination
~ ~ ~ ~] [
-I
I
-I
-I
2
0
.
b = [ -: ] . 0
-3
The MATLAB calculation is displayed in Fig. 3.27(a). Notice that MATLAB warns us that A is rank deficient. having rank two. In Exercise 18 we ask you to verify that A does indeed have rank two. _
»A_[1 0 210 2 21-1 1 -11-1 2 0]; »b_[3 -30 -31 'I »x_A\b
Warning: Rank deficient, rank _ 2
..quares to
253
..
to1_
2.664.5.-15
o -1. 5000 0.5000
Figure J.27(a)
The MATLAB commands for Example 5
~l-.e~_
Indeed. it unique e..cl)
that
Since A is rank deficient. there are infinitely many least-squares solutions 10 the inconsistent system Ax = b. MATI...AB returned just one of these solutions. namely x = [0. -1.5. O.5f. We can find all the solutions by solving the normal equations ATAx = ATb. Fig. 3.27(b) shows the result of using MATLAB to solve the normal equations for the original system (since A and b have already been defined. in Fig. 3.27(a). MATLAB
254
Chapter 3
The Vector Space R"
»NormBqn_ (A' *A. A' *b] NormBqn _
J -J J
,
, ,
-J
J J
-12 0
J
»rret (NormBqn) llnll
_
1 0 0
0 1 0
2 1 0
Figllre 3.17(b)
1 -l
0
Setting up and
~I\'ing
the normal equations for
ExampleS makes it very easy to define the augmented matrix for ATAx = ATb), The complete I - 2t). Xl I ~ X3. or in vector fonn: solution is Xl
=
=-
x· =
1-
[
2.<'] = [ -~'] +x3 [ -2 ] -:
-lx~X3
As can be seen from the complete solUlion just displayed. the particular MATLAB least· squares solution can be reco\'ered by setting X3 = 0.5.
3.S
EXERCISES
In Exercises 1-6. find all \'ectors x· that minimize lAx-bU. where A and bareasgi\en. Use the procedure suggested by Theorem 17. as illuSlrated in Examples I and 5.
I.A~[-::J
b=[:]
2A~[ -:-:-;] b~[;]
3.A=
[
121]
354. -1
2I
4.A= [
5. A~ [: ]
[I] 3
I -4
0
2 3 -I1 ]
-I -1 -2
3
b=
5
.
b=
[ 0I ]
0
I
0
b~ [ In
~
3.9 Theor)' and Practice or Least Squares
6.A=
'00] [ ~~~.
b=
["] ~
In Exercises 7-10. find the least-squares linear fit to the given data. In each exercise. plot the data points and the linear approximation. 7, ~ -1 Y 0
0
1
2
I
2
4
-2
0
I
2
2
I
0
0
-I
o
t
8,
;:l
Y
;r-:.:; f
9.
10. t y
12. \8/,
-2
81 (1m )
_ f
,
2 2
y
-3
-1
:J
-2
0
I
2
5
I
I
5
y
0
3
15. Consider the following table of data:
~
yin
" )'2
JjJ
"d
81Um)
[
Y. 16. Let 81(r) = ..Ii and 82(t) = COSJU. and consider the data points (t;. J;). I ::: j ::: 4. defined by
y
1
4
9
16
0
2
4
5
As in Eq. (6). (/282(t;) -
lei Q(al. (/2) = L::.I(aI81(t;)= JT, and 82(t,) =
cos Jrt,.
•f _ _ -.=2_.::-:.:'_.::0~~
U,
1
-,",f......here 8t(t,)
_,-0_,-1_.::2_.::3 0 0 I 2
0. -
x~ [::
",
10
o
g,(I,) ] 81(f2)
y, ] .. .
b --
-I
2
y
81~(1) [
3
7
3
, [/(1,) - yd'' ~ IIAx - bll',
where
A=
to the given data. In each exercise. plot the data poinls and the quadratic approximation. 11.
L
g,(I,)
in Exercises II-I·t find the least-squares quadratic fit
IILompl~
",.,•
3
2
0
-2
For given functions 81 and g~. consider a function f defined by f(1) = al81 (I) + (/282(r). Show that
2 2
255
f.
,.
.'
a) Use the resull of Exercise 15 to determine A. x. and b so that Q(al. (2) = IIAx - b 12. b) Find the coefficients for f(l) = (/I..Ii .,.. a2 cos;rt thai minimize Q(al. a~). 11. Consider the ((m - I) x (n - I)] matri-.: A in Eq. (5). \\here m ~ n. Show that A has rank" - 1. (Hint: Suppose that Ax = 8. \\here x = lao. al ..... an)T. What can you say about the polynomial pet) =
ao -l- alt + ... + a.t n?)
18. Find the rank of the matrix A in Example 5.
THEORY AND PRACTICE OF LEAST SQUARES In the previous section. we discussed leasHquares solutions to Ax = b and the closely related idea of best least-squares fits todata. In this section. we have IWO majorobjecti\'es: (a) Develop the theory for the least-squares problem in Rn
,
256
Chapter 3
The Vector Space R"
(b) Usc the theory to explain some of the technical language associated with le~· squares so that we can become comfortable using computational packages su~ as MATLAB for least-squares problems.
The Least-Squares Problem in R" The theory necessary for a complete understanding of least squares is fairly conei;,,; and geometric. To ensure our development is completely unambiguous, we begin b' reviewing some familiar terminology and notation. In particular, let x be a vector in R
x~[::] x"
We define the distallce between two vectors x and y to be the length of the vector x recall thaI the length of x - Yis the number IIx - YII where
~
IIx - YII ~ lex - y)'ex - y) = J(X\ ~ .n)2
+ (X2
- Y2)2
+ ... + (x"
- )',,)2.
The problem we wish to consider is stated next.
The Least-Squares Problem in R" Let IV be a p-dimensional subspace of R". Given a vector y in R", find a vector w' in IV such that 1I"-w'II:::lIv-wll,
for all win IV,
The vector w¥ is called the best least-squares approximation to v.
That is, among all vectors w in IV. we wanl to find the special vector w¥ in IV that is closest to v. Although this problem can be extended to some very complicated and abstract settings, examination of the geometry of a simple special case will exhibit a fundamental principle that extends to all such problems. Consider lhe special case where IV is a two-dimensional subspace of R3 • Geometrically, we can visualize IV as a plane through the origin (see Fig. 3.28). Given a point \ not 011 IV, we wish to find the point in the plane. w', that is closest to Y. The geometI) of this problem seems to insist (see Fig, 3.28) that w' is characterized by the fact that the vector v - w* is perpendicular to the plane W. The next theorem shows that Fig. 3,28 is not misleading. That is, if Y - w' is orthogonal to every vector in IV. then w' is the best least-squares approximation to v, I
TH EOI\ E\\ I ~
Let IV be a p-dimensional subspace of R". and let v be a vector in R". Suppose there is a vector w' in \V such that (y - W*)T W = 0 for every vector w in IV. Then w' is the best least-squares approximation to Y.
3.9 Theor)' and Practice of Least Squares
kd with least "kages such
257
"
'--"ib,_w.
v 'airly concise e begin b~ ector in R".
r
w_ .' )' IV
x w* is the closesl point in the plane IV 10 \'
Figure 3.18
'e..-Ior x -
~
Proof
Let w be any vector in Wand consider the fol1owing calculation for the distance from \. to w: Ilv - wll 2 = n(v - w*) -l- (w* - w)11 2 = (,' - w·)T (\. _ w*) -l-
=
+ 2(\' - w'l (w'
- w) (1)
(w· - w)T (w· - w)
nv - w*p2 + hw· -
w11 2•
(The last equality follows because w· -w is a veclOr in W. and therefore ("_W")T (w·w) = 0.) Since nw" - w0 2 :: O. it follows from Eq. (I) that Ill' - wf 2: nv - w·1I 2 . Therefore. w' is lhe best approximation to ". •
:a "ector
The equality in calcu!ation(1).I,v-w,1 2 = ,'_w·1l 2 + Ow*-wIl 2 . is reminiscent of lhe Pylhagorean theorem. A schematic "iew of this connection is sketched in Fig. 3.29.
w
d
" .•
II'
Figun 3.19
A geometric interpretation of !he ,ector w' in W closest
'0'
, - "...
In a later theorem. \\e will show thatlhere is always one. and only one. ,ector w' in W such that \' - w* is onhogonallO e"ery vector in W. Thus it will be established that the beSt approximation always exists and is always unique. The proof of Ihis fact will be constructi\e. so \\e now concentrate on methods for finding w·.
Finding Best Approximations
•
Theorem 18 suggests a procedure for finding the best approximation W". In particular. we should search for a ,'ector w· in \V that satisfies the following condition: If w is any vector in W, then (v - w*)T w = O.
258
Chapter 3
The Veclor Space R"
The search for w· is simplified if we make the following observation: If v - w· is orthogonal to every vector in IV. then v - w· is also onhogonal to every "ector in a baS! for IV. In facL see Theorem 19. the condition that v - w· be onhogonalto the bas! vectors is both necessary and sufficient for" ~ w· to be onhogonal to every vector in n TtH:OHE:\\ It)
Let IV be a p-dimensional subspace of R n • and lelluJ. U2 •.... up) be a basis for \\ Let " be a vector in W. Then (,' - w·)r w = 0 for all w in IV if and only if
:::: i:::: p.
(V_W·)T U/ =0,
•
The proof of Theorem 19 is left as Exercise 17. As Theorem 19 states. the beSt approximation w· can be found by solving the p equations:
(v -
W·)T UI ::;
0
(v - w·)T U2 = 0
" (v - W·)T Up =0
Suppose we can show that these p equations always have a unique solution. Then. Theorem 18, it will follow that the best approximation exists and is unique.
b~
Existence and Uniqueness of Best Approximations We saw above that w· is a best least-squares appro~imation to \' if the vector v - "" satisfies system (2). We now use this result to prove that best approximations always exist and are always unique. In addition. we will gi"e a formula for the best approximation.
I t I rORr" 20
Let IV be a p-dimensionaJ subspace of R" and let \' be a vector in R". Then there is one and only one best least-squares approximation in IV to \'.
Proof
The proof of existence is based on finding a solution to the system of Eq. (2). KQ\\. system (2) is easiest 10 analyze and sohe if we assume the basis vectors are onhogonaL In panicular.lel {u]. U2 ••••• up) be an onhogonal basis for IV (in Section 3.6 \Ioe observed that every subspace of R" has an onhogonal basis). Let w· be a "eclor in nwhere
w· = a]u] -
U2U2
+ ... +apu p.
•
Using Eq. (3). the equations in system (2) become (v - (aJu]
+ 0202 + ... + UpUp»T Uf
= O.
for i = 1.2..... p.
Then. because the basis veclors are onhogonal. the preceding equations simplify con· siderably: V T u, - a;u,T Uj::: 0 . fori = 1.2..... p. Solving for the coefficienlS (II. we obtain OJ
=
v T u, ~T~·
uf
Uj
3.9 Theory and Practice of Least Squares
259
Note that the preceding expression for Ui is well defined since u, is a basis vectOr. and hence the denominator uT u, cannot be zero. Ha\ ing solved the system (2). we can write down an expression for a vector w' such that (v - w·)T W = 0 for all w in IV. By Theorem 18. this vector w· is a best approximation to v:
\- - w' is or· \.'1.or in a ba~is 10 the basis : vector in W
. " "i P
• ba:.is for \\
vT U
w=L-r-.' Uj •
,i
1=1
•
(4)
U,
Having established the existence of best approximations with formula (4). we turn now to the question of uniqueness, To begin, suppose w is any best approximation to \'. and w· is the best approximation defined by Eq. (4). Since the vector \' - w· was constructed so as to be orthogonal to every vector in IV, we can make a calculation similar to the one in Eq, (I) and conclude the following:
hing the p
11\' - wf = 11\' - w'1l 2 + Iw' - w11 2 .
"
But. if wand w' are both best approximations to v. then it follows from the equation above that Ilw' _"'11 2 = O. This equality implies that w· -w = 8 or w· = w. Therefore. uniqueness of best approximations is established.
Then. b)
The following example
iIIu~trates
how a best approximation can be found from
Eq. (4). Ittor v - w·
[\" \\1'1 l
I
Let IV be the subspace of R 3 defined by
'3) S e,i~t
-unauoo.
IV = {x: x = here is ODe
ortbogolL i)Q 3.6 ~e ~ector In "
[ x,
•
Xl
Solutio"
Our first task is 10 find an orthogonal basis for W. We will use the Gram-Schmidt process to find such a basis. To begin. x is in IV if and onl) if XI = ~.1"2 + 3.1"3. That is. if and onl) if x has the
lonn x= f)
COD-
+.1"2 - 3.(3 = O}.
Let \' be the ,ecIOT\' = (I. ~2. -4f. Use Eq, (4) to find the best least·squares approximation to \-'.
:-:ov.
~
X, ]
X2
[
-X,+3X,] :
=.1"2
[-1] [3] ~
+~
~
.
Therefore. a nalUral basis for W consists of the tWO vectors WI = (-I. I, of and
"'2 = (3.0. If· We now use the Gram-Schmidt process to derive an onhogonal basis {UI. U2} from the natural basis {WI. \Il2}. In panicuiar. (a) Let
Ul
=
\\'1·
(b) Choose a scalar {/ so that
U2
="'2
+ QUI
is orthogonal (0 UI.
260
Chapter 3
The Vector Space Nil
To find the scalar II in (b). consider
+ aUl) , 1 W2 + aU I UJ. O. we need uf \\'2 + auf UI = O. or uf U2 = uf (\\'2
, = u
Thus. to have uf u~ =
Uf W 2
a=-~,~
U1
u,
-3 =~2
= 1.5.
Having found a. we calculate the second vector in the orthogonal basis for \V. findi U2 w~ + l.5uJ 13. O. IV + !.S[ -I. l. OlT = [1.5. 1.5. I]T. Next. let \1· = alUI + a2U2 denote the best approximation. and determine lL coefficients of \I using Eq. H):
=
=
v T UI
~3
al=-,-=~=-1.5
u
a, = -
l
UI
v T U2
~- ~
UfU2
_
-5.5
- - =-1. 5.5
Therefore. the best approximation is gh'en by \1'
=
-1.5uI -
U2
= -1.51-1. J. OJT - [1.5. 1.5. If = 10. -3. -If.
(As a check for the calculations. ""e can form " - w· = (I. I. -3f and verify tha. \ - \I i~ orthogonal to each of the original basis vectors. WI = [-I. I. OJT and W2
=
[3.0. lIT.)
•
Least-Squares Solutions to Inconsistent Systems Ax = b In Section 3.8 we ""ere imerested in a special case of least-squares approximationsfinding least-squares solUlions to inconsistent systems Ax = b. Recall that our methoo for finding least-squares solutions consisted of soh ing the normal equations ATAx = ATb. In tum. the validity of the nonnal equations approach ""as based on Theorem 1\\ hich said: (8) The normal equations are ah\ays consistent. (b) The solutions of the normal equations are precisely the least-squares solution,
of Ax = b. (c)
If A is (m x Ill. then if A has rank 11.
least~squares
solutions of Ax = b are unique if and
onl~
We arc now in a po~ition 10 sketch a proof of Theorem 17. The basic ideas supporting Theorem 17 are very important 10 a complete understanding of least-squares solutions of inconsistent systems. These ideas are easy 10 explain and are illustrated in Fig. 3.30.
3.9 Theory and Practice of Least Squares
261
Rm
R'
=Ax -------. ~
Figure 3.30
n°. findin.:-
A
gro~lric
\·isualization of lbeorem 17
In Fig. 3.30. we think of Ihe (m x n) matrix A as defining a function of the fonn)' = Ax from R" to R"'. The subspace 'R(A) represents the range of A: it is a p-dimensional subspace of Rm . We have drawn the vector b so that it is not in R(A), illustrating the case where the system Ax = b is inconsistent. The vector }'- represents the (unique) best approximalion in 'R(A) to b.
'[ttmine the
ProofofTheorem 17
Because ),.- is in 'RCA). Ihere must be veclOrs x in R" such Ihat Ax = )". In addition. because ya is Ihe c10sesl point in 'R(A) 10 b. we can say: A vector x in 'R" is a best least-squares solution to Ax = b if and only if Ax = y•.
_ ,T
'mtv and 1Ir; =
•
:S)
In order to locate y~ in IV. we note that ).• is characterized by wT(y. - b) = 0 for any veclOr w in R.(A). Then. since Ihe columns of A fonn a spanning set for 'RCA), y' can be characterized by Ihe condilions: AJ()·· - b)
= O.
fori
= I. 2..... /1.
The orthogonality conditions abo\e can be rewritten in matrix/vector terms as AT(y·-b)~6.
(7)
Finally. since ya is in 'R(A). finding )'~ to solve Eq. (7) is the same as finding vectors x in R" that salisfy Ihe nonnal equalions: AT(Ax - b) =(J.
We can nov. complete Ihe proof of Theorem 17 by making the observation that ond
Eq. (6) and Eq. (8) are equivalent in the following sense: A veclor x in R" satisfies Eq. (8) if and only if the vector y. satisfies Eq. (6), where)'- = Ax.
To establish part (a) of Theorem 17. we note that Eq. (6) is consistent. and hence lhe nonnal equations given in Eq. (8) are consistent as well. Part (b) of Theorem 17 follows from rule (5) and (he equivalence of equations (6) and (8). Pan (c) of Theorem 17 follows because Ax = y- has a unique solUlion if and only if the columns of A are linearly independent. _
262
Chapter 3
The Vector Space R"
Uniqueness of Least-Squares Solutions to Ax = b Best leasl-squares approximations are always unique but least-squares solutions to A:\ = b mighl or might not be unique. The preceding statement is somewhat confusing becau the term least-squares is being used in two different contexts. To clarify this widel accepted, but somewhat unfortunate. choice of terms, we can refer to Fig. 3.30. In Fig. 3.30. the best least-squares approximation. y•. is unique (uniqueness y,~ proved in Theorem 10). A best least-squares solution to Ax = b. however. is a vector x such that Ax = y·. and there might or might not be infinitely many solutions to Ax = ~ (The equation Ax = y' is always consistent because y. is in R(A): the equation has unique solution if and only if the columns of A are linearly independent.) Recall from the previous section that an (m x n) matrix A is called raukdejiciellt if: has rank less than 11 (that is. if the columns of A are linearly dependent). When A is ran deficient. there are infinitely many least-squares SolUlions to Ax = b. In this instance we might want to seleci the minimum norm solution as the least-squares solution y,e u...: in our application. To explain. we say x' is the minimum norm least-squares solUliQrl 10 Ax = b if Ux·11 minimizes IIxli among al1leasl-squares solUlions. That is. !lx"lI = minfl.x'i : Ax = yOlo II can be shown that the minimum norm solution always exiSIS and is always unique. The minimum nonn solution is associaled with another least-squares concept. Ih of the pseudoinverse of A. The pseudoin\erse of A is. in a sense. the closest thing (, an inverse (hal a rectangular matrix can have. To explain the idea. "e first introdu<.:e the Frobenius noml for an (m x n) matrix A. The Frobenius nonn, denOled HA [IF, defined by the following:
• • lAI:, ~ I I:L>~· ;=1 j=1
Just as I x measures the size of a \'eclor x. IIA !IF measures lite size of a matrix A. Now. let A be an (m XII) mauix. Thepseudoim'erseof A. denoled A+, is the (n xm matrix thai minimizes lAX - IIIF where I de,notes the (m X m) idemit) matrix. It can be shown that such a minimizing matri~ always e~ists and is alwa):, unique. As can be seen from Ihe definilion of the pseudoinw.rse. it is Ihe closest thing (in a least-square sense) 10 an inverse for a reclangular matrix. In the event that A is square and in\enible. then Ihe pseudoinverse coincides with the usual inverse. A -I. It can be shown Ihat 1M minimum nonn least-squares solution of Ax = b can be found from
x· = A-b, An actual calculation of the pseudoin\oerse is usually made with the aid of anolher type of decomposilion. the singular-value decomposition. A discussion of the singular-value decomposition would lead us too far afield. and so we ask the interested reader to consult a reference. such as Golub and Van Loan. Matrix Compillatiolls.
MATLAB and Least-Squares Solutions As we noted in the previous section. there are seveml ways to sohe least-squares problems using MATLAB.
3.9 Theor)' and Practice of leaSI Squares
263
(a) If A is (m x II) wilh III :F 11. then the MATLAB command A\b retums a least·squares solution to Ax = b. If A happens to be rank deliciem. then MATLAB selects a least-squares solution with no more than /) nonzero emries (where p denotes the rank of A). The least-squares solution is calcul.ated using a QR-factorizatioll for A (see Chapter 7).
m,loA). =
ing becau"" this wickl
:UO. uenes\ \\ j,:\ a '-ector I
(b) If A is square and inconsistent. then the MATLAB command A\b will produce a warning that A is singular or nearly singular. but will not give a least-squares solution. One way to use MATLAB to find a least-squares solution for a square but inconsistent system is to set up and solve the normal equations.
I"
loA). =.\
uation h:..,
(c)
I \.\\\PlC2
Whether A is square or rectangular. the MATLAB command x = piny (A)·b will give the minimum norm least-squares solution: the command piny (A) generates the pseudoim'erse A +.
The following sample values from the function =- = f(x. y) were obtained from experimentalobservations: f(1. 1) ~ -1.1 f(1.21 ~ 0.9
f(2. I)
~
0.2
f(2. 2)
~
2.0
f(3. I)
~
0.9
f(3. 2)
~
3.1
We "auld Iil.e to approximate Ihe surface:: = f(:c. y) by a plane of the form:: = + by + c. Use a least-squares criterion to choose the parameters a. b. and c.
ax Solution
The conditions implied b} the experimental observalions are
a- b+c=-I.I 2a- b+c=
0.2
3a- b,c=
0.9
a - 2b +c =
0.9
2a-2b+c=
2.0
3a-l-2b+c=
3.1.
A least-squares solUlion.
a = 1.05.b = 2.00.c= -4.10. to this o\erdetennined system Ax = b was found using MATLAB. see Fig. 3.31. Since MATLAB did not issue a rank deficient warning. we can assume that A has full rank (rank equal to 3) and therefore thai the least-squares solulion is unique. _
C\-\\\PI r
3
Find a least-squarei> solution 10 the equation Ax = b where
112]
A~ [ ~ ~ :
.
b~
[I] :
.
r
-
~
264
Chapler 3
.
The 'ector Space R"
»A_fl,l,l;2,l,1;3,1,l,1,2,l;2,2,l;3,2,l]; »b_[_l.l, .2, .9, .9,2.,3.1] 'i »x_A\b
x • 1.0500 2.0000 -4.1000
) Figure 3.31
Solutioll
The result:. of Example 2
The re~uhs art shown in Fig. 3.32(a}. Note that ~1ATLAB has issued a rank deficiem warning and concluded that A has rank 2. Because A is not full rank. least-squares solutions to Ax = b are not unique. Since A has rank 2. the MATLAB command A\b ~elecb a solution with no more than 2 nonzero components. namely Xl = [0.0. -0.8, uf. As an altemative. we can use the pseudoinverse to calculate the minimum-nonn least-squares solution (see Fig. 3.32(b)). As can be seen from Fig. 3.32(b). the MATLAB command piny (A) *b has produced the least-squares solution X2 = [0.6. -0.2. OAf A calculation shows that IIxll! = 1.2806. while the minimum norm solution in Fig. 3.32(b) has IIx211 = 0.7483. Finally. to complete this example. we can find all possible least-squares solutions by solving the nonnal equations. We find. using the MATLAB command rref (8 .
»x_A\b
x_piny (A) *b
warning: Rank deficient, rank _ 2
x •
x • 0.6000 -0.2000 0.4000
o -0.8000 1.1000
(x)
Figure 3.31 (aj L.:smg the command A.o 10 find a least-squares r.olution for Example 3. (b) Using the pseudoin\eT'Se to find a least.squares solunon for Exampk 3.
(bl
3.9 Theory and Practice of Least Squares
165
that the augmented malrix B = [AT A I ATb] is rov. equhalent to
1 0I 1I .21]
[ oo
0 0
0
Thus, the set of aillenst-squares solutions are found from x = [I - ,\"3. 0.2 - ,\"3. [I.O.2.0V +X3[-1, -I, IV· I \-\ \\1'1 L ~
I
XJ
f
= •
As a final example 10 illustrate how MATLAB treats inconsistent square systems. find a
least-squares solution
Ax = b where
10
A~[: ~
I
n b~[:J
~ deficient
~uares solu;. bselem
. . uf. mum-norm
.'IATlAS -f>:!. OAf ,Iulion in
~~Iution
Soilltioll
The results are given in Fig. 3.33 \\, here. for clarity. we used Ihe rlllional formal to display the calculations. As can be seen. the MATLAB command A\b results in a \\ aming that A may be ill conditioned and ma~ have a solution vector with "ery large components. Then. a least-squares solution is calculmed using the pseudoinverse. The leaslsquares solution found is x = [2/39. -2/13. 8/39J1. •
'"
/' »A_[~,3,5;l,O,3;3,3,8};
- ::-::-ef E
»b_ [1,1,11'; »x_A\b Warning: Matrix is close to singular or badly scaled. Results may be inaccurate. RCOND _ 6.405133.·18
1.,0,
1
x • -6755399441055744 7S0S9gg3789508~ ~~S1799813685248
»x_pinv(A)*b
x • 2/39 -2/13 8139
Figure J.JJ
The results from Example 4
266
Chapter 3
The Vector Space RR
3.':11 EXERCISES Exercises 1-16 refer to the following subspaces: a) IV =
(x x~
[:J
b) IV ~ R(8).
oj IV
d) IV
~
R(B).
~
(x x~
1. IV given by (a).
x, -
2" +xJ ~ 0)
[:J
[1.2.6f
7. IV given by (c). \' = [2. O. 4V
9. IV given by (d). v = [1. 3. If
3
+ Xl + .'1'3 =
X\ -
Xl -
X3
c
10, IV given by (d), v = [3.4. OJT
-2
1
Xl
5. IV given by (b). v = [3, 3. 3]T
8. IV given by (c). v = [4.0. -IV
2o 4]
[ -I
3. IV giYen by (a). \' = [1. I. If 4, IV given by (b). v = [l. I. 6f 6. IV given by (b). \' = [3. O. 3V
B~[~:] B~
\'=
2. IV given by (a). \' = [3, O. 3V
0
= 0
I
In Exercises 11-16. find an onhogonal basis for the indicated subspace IV. Use Eq. (4) to detennine the beSt approximation w* for the given vector v. II. IV and v as in Exercise 1
12. IV and v as in Exercise 2 13. IV and v as in Exercise 4
14. IV and v as in Exercise 5
In Exercises 1- [0, find a basis for the indicated subspace IV. For the given vector v, solve the nonnal equations (2) and determine the best approximation w·. Verify that v - w· is onhogonalto the basis vectors.
15. IV and \' as in Exercise 7
16. IV and v as in Exercise 8 17. Prove Theorem 19,
•
SUPPLEMENTARY EXERCISES 1. Let
3. Let IV = jx: x =
X, ] [ x2 .
XIX2
= OJ. A =
Verify that IV satisfies propenies (sl) and (s3) of Theorem 2. l11ustrate by example that IV docs not satisfy (s2).
['-I I] 1
4-1
2
2
and
2. Let
IV
= {x: x = [
x, ]. X2
x\
~ 0,
X2
~ OJ.
Verify that IV satisfies prope,nies (51) and (s2) of Theorem 2. lllustrate by example that IV does not satisfy (53).
I
IV = {x: x =
["
X2
XJ
] •
Ax = 3x}.
a) Show that IV is a subspace of R3 • b) Find a basis for Wand determine dim(IV).
~
Supplementary Exercises
~.
Find a basis for the row space of A. and dctennine thc rank and the nullity of A.
If
s~IUH;]1
8. In a)--c). use the givcn infonnation to detennine the nullity of T.
R 2 and the rank of T is 2. 3 b) T: R ...,.. Rl and the rank of T i~ 2, c) T: R 3 _ R 3 and the rank of Tis 3. a) T: R' _
"d
T~I[~lUHnl
then show that Sp(S) = Sp(T), [Him: Obtain an algebraic specification for cach of Sp(S) and Sp(T).} S. Let
A=
[
1-I 25 43]
2 -2 I -I
0
9. In a)--c). use the given information to delennine the rank of T. a) T: R 3 ...,.. R 2 and the nullity of T i~ 2. b) T: R·1 __ R 1 and the nullity ofT is I. c) T: R2 _ Rl and the nullity of T is O. 10. Let B = (XI.X~J be a basis for R 2. and let T: R 2 _ R 2 be a linear transfonnation such thai
7
a) Reduce the matrix A to echelon fonn. and
b) e) d)
e)
T(xd = [ : ]
detennine lhe rank and the nullity of A. Exhibit a basis for thc roy, ~pace of A. Find a basis for the column space of A (that i!>. for R.(A» con... l ... ting of columns of A. Use the answers obtained in pan... bl and c) to exhibit bases for the to\\ ... pace and the column space of AT. Find a basis for.'·(A).
"~ [-:J "~ [J
11. Le,t
b~ [ :
,00
[~
a) Find a subset of S that is a ba...is for Sp( S).
b) Find a basis for SpCS) by setting A = [VI. "2. "3J and reducing AT to echelon fonn. e) Ghe an algebraic specification for Sp(S). and use that specification to obtain a basis for SpeS). 7. Let A be the (m )( n) matrix dcfined by
•-1.
=
[
2,,-1
211-1211+2
3/1-1
ml1:+111II1'+2
(111+1)/1-1 (m
2" 3"
_~
l
l
and suppose that T: R~ - R! is a linear transformation defined by T (x) = Ax. "here A is a (2 )( 3) matrix such that the augmented malrix (A bl reduces to
"~[ J ,,+2
and T( X2) = [
If e\ = XI ~ 2X2 and C1 = 21' + 1'2. y,here CI and C2 are the unit \ectors in R 2 • then find the matrix ofT.
6. leI S = {";. "~. ')}. y,here
,,_I
267
]
+ 1)/1
o
8
I -3
-5a"'t"3b]. 2a -
b
a) Find \ectors Xl and X1 in R' such (hat T(xI) = Cl and T(xJ) = C1_ where CI and C2 are the unit
\-cetoTS in R1 • b) Exhibit a nonzero \eClor inX(T).
X3
in R 1 such that
X3
is
c) Show that B = {XI. Xl. x31 is a basis for R 3. d) Express each of the unit \CCtors el_ C2_ el of R) as a linear combination of the vectors in B. Now calculate T(c,). i = 1.2.3. and determine the matrix A.
268
Chapter 3
The Vedor Space R"
In Exercise~ 12-18, b = [(I.b.c.dJT, T: R6 ....... R4 is a linear transfonnation defined by rex) = Ax. and A is a (4 x 6) malrix such that the augmemed malrix [A bl reduces to
[~
o
2
0-3
-la+ b-2,
I -I
o
o o
0
I -I -2
0
o
2 0
2 0
I~
]
+ 5b -7c
-5a-2h+3c -1&1-7b -r9c
.
+d
12. Exhibit a basis for the row space of A. and deh~nnine the rank and the nullity of A. 13. Dctennine \\hich of the follo\\ing \ector'S are in 'R(T). Explain hO\\ you can tell.
W,
~
W, =
[-]
[-]
Wl
= [;
". = [
l
1]
16. Suppose that A = lAt. Az. A3. A4. As. A61. a) For each \ector w,. ; = I. 2. 3. 4. listed in E..;ercise 13. if w is in the column space of A. the'" express ". as a linear combination of the columns of A. b) Find II subset of (AI. A:!. A3. A..!. As. a basis for the column space of A.
~l
thilt
c) For each column. A). of A thai does nOI appe:tr in the basis obtained in part b). express Aj as a linear combination of the basis \ ectors. d) Letb = (I. -2. I. -7f. Shay, that bisinthe column space of A. and expres~ b as a linear combination of the basis vectors found in part b). e) If.!!: = [2,3.1. -I. I. If.lhenexpress Ax asa linear combination of the ba;,is vectors found in pari b).
17. a) Give an algebraic specification for "R(T). and use that ~pecification to detennine a basis for R(T).
b) Sho", thaI b = (I. 2. 3. 3f is in "R(T). and expre!>s b a.. a linear combination of the basis wctors found in part a).
18. a) Exhibit a basis for X'(T). 14. For each vector WI, i = 1.2.3.4 listed in E:t.::ercise 13. if the system of equations Ax = W; i!> consistent. then exhibit a solution. 15. For each vector w . ; = 1.2.3. -I listed in Exer~ cise 13. ifw, i!> in "R(T). then find a \ectOT x in ~ such thaI rex) = ",.
b) Show that x = [6. I. 1, ~2. 2. _2)T is in. \'(T). and expres~ x as a linear combination of the basis vectors found in pan a).
CONCEPTUAL EXERCISES In Exercises 1-12. answer true or fal ..e. Justify your ans"'er by prO\'iding a counterexample if the stalement is false or an outline of a proof if the statement is true.
{xl •.... x.tl is a ..ubsetof R· andk > II. then S i!> a linearly dependent .set.
·t If S = 5. If S =
{Xl ....• xd is a subset of R· and k < n. then S j;, not a spanning ..et for R".
1. If IV is a subspace of R" and x and ). are \ ectors in R~ such that x + y is in IV. then x is in IV and )' is in IV.
6. If 5 =
2. If IV is II subspace of R" and (IX is in IV. where a is a nonzero scalar and x is in R". then x is in IV.
7. If 51 and S2 are linearly independent subsets of R·. then the sel SI U S~ is also linearly independent.
3. If S = /XI ..... x~l is a subset of R· and k ::: n. then S is a linearly independent .set.
8. If IV is a subspace of R-. then IV has exactly one basis.
xd is a subsel of Wand k :::: n. spanning set for R".
{Xt .....
then 5 is
l\
Conceptual Exercises
Illo ..\"j.
R'. and dimllV) = k. then 1\ comains e"actly I. \'ttlors.
9. If \1' is a subspace of
Xow sel pan c).)
W
=
CIW
+ ...
269
+ C,,~I"II_I and use
,ted in Exo ;:e of A.llIen lillie
10. If B is a ba~i~ for R~ and \1/ is a .subspace of R . Ihcn some subset of B i~ a basis for IV.
19. Let V and W be ~ubspaces of R" such that V n IV {9\ and dim(V) + dim(W) = n.
II. If IV is a IV = Rn •
of W. and dim(lV) = II. thcn
, A~lth:l1l'
a) If\ +\1 = 8. where \. is in V and wisin W. show that \' = (J and II = 8.
~ubspace
~ub~paces of R" with base~ B l and 8~. respectively. Then B j n Hz b a basis for IV1 n IVl .
12. Let IV I and IVz be not 3ppear " ... AJasa
'"bbinthe "" a linear m
,-lox 3.lI a found "
In Exerl'ises 13-23. give a brief answer.
13. Lei IV be a subspace of R~. and set V = Ix: x is in R~ bul x is not in I\' ,. Delcrmine if V is a sub~pace of R~. I·t Explain .... hat is wrong with the following argument: Let \\' be a subspace of R~. and let B = {el ..... c, \ be the basis of R- consisting of lhe um! \ecIOr... Since 8 is hnearl~ indepcndem and since e'-er) \eclOr \I in W can be wrilten as a linear combmation of the \'ectors in 8. it follows Ihal 8 is a basis for It'.
IS. If 8 = {xl.x~.x;1 is a basis for RJ • ~how that 8' = {XI. X + X!. Xl + x~ - x~1 is also a basis for R 1 • 16. Let IV be a subspace of R". and leI S = {WI' .••. \Ill be a linearly indepcndcnlsubsct of \I' such that {"''t ..... \It- II} is Iinearl) dependent for e\'er)' \I in IV. Prove that 5 is a basis for IV.
17. Lei [Uj ••.. , u,,} be a linearly independent .subset of R~.andlctxinRnbesuchlhaturx=... = u:x = O. Sho.... that x = (J.
a nonzero vector in R~. and let IV be the subset of Rn defincd by IV = {x: u r x = OJ.
18. LeI u be
a) Pro\C Ihal IV is a subspace of
R~.
b) Show Ihal dim/IV) = 11 - I. e) If9 = w+cu. wherew is in IV andcisa scalar. show Ihal "'. = 9 and c = O. fHim: Consider u' (w + cu).) d) If I" w,,_.}isabasisforn'.~howlhal ("I \1',,-1' uJ is a basi~ for R". [Him: Supposelhalcl\lI+"'+C__ 1W,,_ +cu=9.
=
b) If 8 1 is a basb for V and 8: is a basis for W. show thaI 8 1 U B~ b a basis for Rn . [Hillt: Use part a) 10 show Ihat 8j U B2 is linearly independent.] e) If x i~ in R". ~how Ihat x can be ""TIllen in the fonn x = \' + w. where \' is in V and \I is in IV. [Him: Fir
T is onto.
b) If the nullny of T is zero. then T is onlo. e) If T is onto. then lhe rank of Tis n and Ihe
nullit) of T is zero. 21. [f T: R" - R'" i~ a linear tramfonnation. then show that Tr9n ) = 9",. where 9" and 9", are the 7.ero \'ectol'S in R" and R"'. respectively. -+ R'" be a linear transfOrmation. and that 5 = (x) ..... xd is a subset of W such Ihal{T(xl) ..... T(Xll} is a linearly independent subset of R"'. Show tha! thc set 5 i~ linearly independent. R'" be a linear lran~fonnalion 23. Let T: R n _ with nullit} zcro. If S = Ix\ ..... XlJ is a linearly independent sub~1 of R". Ihen show that {T(xi I..... T(l:l}} is a linearl~ indepe,ndem subset of R".
22. Let
T;
suppo~e
R"
Answers to Selected Odd-Numbered Exercises
3.x=t.y=4-2/ . .:=1+31
21. R = (-3. -I)
5. The lines are parallel.
Y
7. The lines are nOf parallel.
8
"
,
9.x= 1+31.)'=2+41.:= l-t
~' 4 8
x
-8 -4 w
-,
II. The line intersects the plane at P = (-1.4.1). 13. The line intersects the plane at P = (-8. -13. 36). 15.6x..;..y-.:=16
-8
17.-1.l-y+4.:=5
23 ",
H. ",
33. [
~[:
~
]. ",
[ ] ",
~ [ _~ ]
~ _~ [
-:J
35.3i-j-5k
37. [
39. [
41.
]
19. 2.l - 7)' - 3.: = I
21. n =
2/3] [
1/3
-2/3
23. .1:-1-2.1'-2.::= 17
25.
_l
=4 -t.y = 5+1 . .: = I
CHAPTER 3 Exercises 3.1, p. 166 I.
."
=: ]
x
-u
-1)
[J
(3. 1)
u
(-3. -I)
3.
)
.
43.4..16 square units 45. 3JTT square units
u
47. 24 cubic units
-
x
49. not coplanar Exercises 2.4. p. 157 I.x=2+31.)'=4+2I. z = -3+41
(3. I)
(-9. -3)
AN9
I,~W'" 1 0
I
~] ~
'.W ,,, "'" [
= '" - "+ " [
:: ]
{J~WnUrC~.L':U-e D :[ ~
l""W," ~w
AI"
= AI ' "
] } = A\ 'S'Z
~]
,,,. [
I
(0'[ 'l)
I
'Il
= ,II "
·I=.~-,·\+tr ~ds 0Xj1 JO jlllll mdn ~I UO SlU!od jO
'0
= ::, +.\ .... r
UO!1tm~
j;/S
~ S!
i" 'I, (
!P!.\\. ;Jtrnjd;HJl S! ,n '61
-
"
-t
'LT
O-'V , ,,
"
~~~~., .l-
, ([ 'f)
.l-
n
n "
(l'l) 'L
" , ,,, , ,, (f'r)
.,
'£I
,,\
'S
OI.\rl:V
Answers to Selected Odd·Number«! Exercises
Exercises 3.2, p. 174 I. \V is a subspace. \V is the sct of poillls on the line with equation x = 1.\'.
3. II' is not &subspace. 5, II' is the subspace consi~ting of the points on the y-allis. 7. II' is
110I
a subspace.
9. II' is the subspace consisting of the points on the plane 2f-y-:=0. 11. II' is not a subspace.
13. \V is DOl a subspace. IS. II' is the SUbspace consisting of the points on the line with parametric equations X "" 2/. -" "" -t . .:::::: t. 17. II' is the SUbspace consisting of the points on the x-allis. 19. II' is the sct of points on the plane x 23. II' .f
IS
3~
0 and X.l = 0):
Ix in R.l: Ix in R.l:
x, = 3x[1
.f[ = -7xJ and Xl = 2_fll: + 2X2 + Xl = O}
-4.f[
41. a) The vectors b in i). iii). v). and vi) are in RCA I. b) For i), x = [-I. 1. Olr is one choice: for iii I. x = (-2. 3, O)T is one choice: for \). x = 1-2. I. OJT is one choice: for \i). x = 10. O. Olr is one choice.
:::: O.
= 2t.: = I.
c) For i), b = -A[ T A!; for iii). b = -2A[ T JA; f \'), b = -2AI + A 2 : fO£ \i), b = OA I -OA~ - ,\
47. w[ = (-2. 1. 3JT.
OJ: Spes) isthe line witheqllation
x+)'=O. 3, SpeS) = Ie}: Sp(S) is the poim (0. 0). 5, SpeS) = R 1 7. SpeS) = {x: 3x[ + 2x 2 = OJ: Spes) is the line with equation 3x + 2)' = O. 9, SpeS) = R2
IWf of the sphere
+ 21: 2 =
37. ~\'(A) = {I}: "R(A) = R' 39. a) The vectors b in ii). v). and vi) arc in R{A). b) For ii), x = (1. of is one choice: for v). x = 10. I) is one choice: for vi), x = 10. OJT is one choice c) For ii), b = AI: fonl). b = A~. fon-il, b=OA[+OA2.
25. II' is the set of points on the plane x - : = O. Exercises 3.3, p. 186 1. SpeS) = Ix: x[ -I-Xl =
-1.:: "" 0,
35. Ar(A) = "R(A) =
the line formed by the two intersecting pl:lnes -6/.)'
X[
33. _\'(A) = Ix in R2 : X2 = OJ: "R(A) = Ix in R.l: .12 = lx] and
+2)'+2: = 0 :lndx +3)' = O. The line has p:lrametric
equ&tionsx = -./
+ 2.1' -
31. ..V(A) = Ix in R.l: "R(A) = R2
II. Spes) = {x: Xl +X2 = 0): SpeS) is thc line withequalion x+y=O.
+ X, = 0 and XI = OJ: Sp(S) is the line through (0. O. 0) and (0. -I. J). The parametric equationsforthe line are x = 0.)' = -t.: = I.
13. SpeS) = Ix: X2
49,
WI
=ll.2.2]T.
17. Spes) = Rl
W2
= 10.3. IV
3, [[ 1. l. O. O]T. [-1. O. l. OJ' .13. S.I[-I.1.0.01
T
[0,0.1.01
•
T
,
o. O. liT}
10.0.0. I]TI
7. H2. I. -I. OIT, (-I. 0, O. I]T}
")X~2[t]-[-] b) <)
19. Spes) = {x: X2 + XJ = OJ: Sp(S) is the plane with equation)' + .: O.
= [0. 3. 2J'
W2
Exercises 3.4. p. 200 1, II I. O. I. OV. [-1. I. O. liT}
15. SpeS) = {x: Ix[ - X1 +Xl = OJ: Sp(S) is the plane with equation 2x - )' -I- .:: = o.
x
~
-3 [ -
~
l
d) x
x;,oo"
~ t] 2[
=
21. Thnectors u in b), c), and e) are in SpeS): for b)_ u = for c), u = ": for e). u = ],. - 4x.
23. dande
,I
25. x and Y
Ix in R1: -Xl -lf2 = 01: "R(A) = Ix in R1: lx[ +X2 = O}
27. -"(A) =
29. N(A) = {S): "R(A)
=
Rl
A..'dl
x: Il.a)8=
I
0
o
0 0
0
[
I
I
']
I-I
0
b) A basis for .V(A) is {[- L -1. I. 0]1. (-1.1.0. lIT}. c) {AI. A 2} is a basis for the column space of A; = AI + A 2 and Ai = A[ - A 1.
I
ANI2
Answers to Selected Odd-Numbered Exercises
d) ([LO.!. I). spacc of A.
13. a) B=
to.
17. dim(W) = 2
I. L-I]) is a basis forthc row
19. dim(lV) = I
o1 ° -1 2]
[ oo
I
I-I
0
0
0
0
0
0
WI.
c) (AI. A 2} is a basis for the column space of A: A] = -AI +A 2 andA 4 =2A I -A, . d) Il I. O. -I, 2], [0. 1. 1. -I JJ is a basis forthe row space of A. 15.a)8=
[
=
I:
23. {[-5. -2, If} is a basis for N(A): nullity(A) = I: rank(A) = 2.
b) A basis for N(A) is ([I. -1. 1, of. [-2. 1. 0,
1 2 0 001
f~"
21. (( -2. WI is a basis for ..V (A): nullity(A) rank(A) = 1.
]
25. HI. -I. If. [0. 2. 3)T) is a basis for R(A): rank(A) = 2; nullity(A) = I.
27. a) ([I. 1. -2). [0. -1. If. [0. o. W} is a basis for IV: dim(W) = 3, b) (II, 2. -I. If. [0.1, -1. If. [0.0. -1. 4n is a basis for W: dim(W) = 3.
Exerc~
1..) [
!
d1
3. c) i:;, 9. F is 11. F
i~
O.
13. F
i~
j,
29. dim(W) = 2 (33. a) rank(A)
~
3 and nullity(A)
~
b) A basis for N(A) is ([ -2, I. Of I·
b) rank(A)
~
3 and nullity(A)
~
1.
15. F
c) (A I' A31 is a basis for the column space of A: A 2 = lA I.
c) rank(A)
~
4 and nullity(A)
~
O.
17. F i,
000
d) HI. 2. 0). [0.
17. Ill. 3.
o. III isa basis for the row space of A.
W. [0. -1. -If I is a basis for R(A).
19. al
Exercises 3.6, p. 224 S. ur U3 = 0 requires a - b + c = 0: ur U3 = 0 requires la
19. Ill. 1, 2. Of. [0. I. -2, IjT} is a basis for R(A). 2La) {[1.2jT}: b) (II.WI 23. a) ([1.2.lf.[2,5.0jTj: b) IlL2.ljT.[0.1.-2f)
+ lb -
ur U2 = 0 forces a = 3: then ur
9.
V
27. -2v] - 3V2 + \', = 9. so 5 is linearly dependent. Since \'] = 2v] + 3V2. if" = a) VI + a,l', + aJ"] is in Sp{\'I. v,. '·Jl. then v = (a] + 2a3)VI + (a, + 3aJ)\'l. Therefore ,. is in SP(vl. \·l}.
13. u, =
(VI. v,. V4),
{vl. V). '·4}.
33. 5 is not a basis. 35. 5 is not a basis.
15 u, =
Exercises 3.5, p. 212 L 5 does not span R ' . 3. S is linearly dependent.
5. 5 is linearly dependent and does not span R',
+b =
O.
= 0 requires
21. T (
+ (1/6)U3
= (2/3)u] - (1/2)U2
11." = 3uI
\'J}.
U3
-8 - b + 3c = O. while ui U3 = 0 requires 4 + b + c = 0: therefore b = -5. c = 1.
25. a) {l0.1.0n: b) {[-I. 1.0jT.[0.0. 1jT1 c) ([-l.l.ofl
29. The subsets are {Vj. "2.
4c = 0: therefore c = Oanda
7.
17.u,=
[n
u, = [ ;
l
[J J u, = [
0]~ [
. u,=
[-1] -~
23. T (
-:~:]
u, = [
II. 5 is a basis.
13. Sis nOl a basis. 15. dim(W) = 3
19. For the null space:
'R
u, = [ -;]
. u,=
-3] [ -I
[° I
•
r.m 27. A
1 [-2 3] _:::
29..'"
"" '"
Exerci
7. 5 does not span R3. 9. 5 is linearly dependent.
25. A,
1. x'
-27/11 7/11] -6/11 1
, 3. x·
Answers to Selected Odd-Numbered Exercises
,
[ '] [-11/6] 2.
for the range space:
•
,
I
•
5. X'
8/6
-5/6
• •
I..)
2x'+'6f1 ] .
11. Y = 0.5t
[a -a ,) [-: l
T
1
.
-x,
.11 arbltrary
1.1
9. y = 1.51
+ a.lt
13. y = 0.251:+2.151+0.45
Exercises 3.9, p. 266
b) [
d) [
[
7. )' = 1.3t
Exercises 3.7. p. 239
.'- =
=
1. w'
-: ]
~ IE, ]
,.'. [:]
[
3. c) is not but a). b). :lnd d) nre.
9. F is a linear Iransfonn31ion.
5. w' • [ : ]
11. F is not a linear transformation.
7. w' • [ -: ]
13. F is a linear transformation. 15. F is a linear transformation. 17. F is nOl a linear transformation.
19.•j [ _:
1=
u
~wre~
• 21. T
:] ]
-~ -:
;
I J
-1:3
I: ], II
]
bj [
11. w"" =
([ x,]) [x, + x, ] ~: -;x, -;x, + ;x, =
.T1
13. T ( [
-:
9w'. [ - : ]
J =: J,) [_~ ] -:c, + -.(, + 2 2"
[
J
25. A =
I
']
2~
2
121
[-1/'] 2~5
15.'-.'[ -; ].[ ~]
-;c.
[~ ~]; .\~(n = lSI: 'R.(T) = R
i [~'] +
13.W'.[~]-4[ ~~',]
2
1
]).
;c.
h
XI _
:
rank(T) = 2: nullity(T) = 0
21. A = (3 2]: X'(T) = Ix in R 2 ; 3xI + 2t2 = Ol: 'R(T) = Rl : rank(r) = 1: nullity(T) = I 29. A
=
[ o1-I 0] I-I
: A"CT) = (x in
Rl; Xl
and.1"1 = Xl}: 'RCT) = R2 ; rank(Tl = 2: nullit)(T) = I
=
.1)
CHAPTER 4 Exercises 4.1. p. 279
1.;.=l.x=a
1. x. = [ -5{13 ]
7/13
3.).•
(28/74) - 3xJ [
(27/74) +
x,
-I I
] ,"0
;.=3.x=a[ ~].
Exercises 3.8, p. 254
3. x· =
[
XJ
]. x, ,m;"."
1.'.,[
ANI3
a#O
: ]. a #0:
).3,.,[ -: l ,"0
