Optimization in several variables Dr. Tran Thai-Duong International University of Hochiminh City
Aug 6, 2012
Dr. Tran Thai-Duong (IU HCMC)
Optimization
Aug 6, 2012
1 / 16
Local/Relative extrema Suppose f (x, y ) is defined on a neighborhood D of a point P(a, b) f attains local maximum f (a, b) at (a, b) iff f (a, b) ≥ f (x, y ), ∀(x, y ) ∈ D f attains local minimum f (a, b) at (a, b) iff f (a, b) ≤ f (x, y ), ∀(x, y ) ∈ D
Remark. A neighborhood of (a, b) contains all points that are close enough to (a, b). The word extremum is used for either a maximum or a minimum. Dr. Tran Thai-Duong (IU HCMC)
Optimization
Aug 6, 2012
2 / 16
Critical points Suppose f (x, y ) has a local extremum at (a, b) where fx , fy exist, then fx (a, b) = 0 = fy (a, b) In this case, (a, b) is a critical point of f .
Dr. Tran Thai-Duong (IU HCMC)
Optimization
Aug 6, 2012
3 / 16
Critical points Suppose f (x, y ) has a local extremum at (a, b) where fx , fy exist, then fx (a, b) = 0 = fy (a, b) In this case, (a, b) is a critical point of f . Example. Let C (x, y ) = 2200 + 27x 3 − 72xy + 8y 2 Solve Cx = 81x 2 − 72y = 0 = −72x + 16y = Cy 9 0 = −72x + 16y ⇒ y = x 2 81x 2 − 72y = 81(x 2 − 4x) = 81x(x − 4) = 0 Therefore x = 0 or x = 4 and the critical points (0, 0), (4, 18) are the only points that may have a local extremum of C (x, y ) Dr. Tran Thai-Duong (IU HCMC)
Optimization
Aug 6, 2012
3 / 16
Example
Find the critical points of f (x, y ) = 36xy − x 3 − 8y 3 Solve fx = 36y − 3x 2 = 0 = 36x − 24y 2 2 0 = 36x − 24y 2 ⇒ x = y 2 3 4 4 36y − 3x 2 = 36y − y 4 = y (27 − y 3 ) = 0 3 3 Therefore y = 0 or y = 3 and the critical points are (0, 0), (6, 3)
Dr. Tran Thai-Duong (IU HCMC)
Optimization
Aug 6, 2012
4 / 16
Test for relative extrema
Suppose (a, b) is a critical point of a function f with continuous second-order partial derivatives. Define D = fxx (a, b)fyy (a, b) − (fxy (a, b))2
If D > 0 and fxx (a, b) < 0, then f attains a local maximum at (a, b). If D > 0 and fxx (a, b) > 0, then f attains a local minimum at (a, b). If D < 0 then f does not attain a local extremum at (a, b), and (a, b) is called a saddle point.
Dr. Tran Thai-Duong (IU HCMC)
Optimization
Aug 6, 2012
5 / 16
Example Let f (x, y ) = 6x 2 + 6y 2 + 6xy + 36x − 5 and solve fx (x, y ) = 12x + 6y + 36 = 0 = 12y + 6x = fy (x, y ) These equations gives the only critical point (-4,2) Since fxx (−4, 2) = 12 = fyy (−4, 2) fxy (−4, 2) = fyx (−4, 2) = 6 D = 108 > 0, fxx > 0 f attains the local minimum −77 = f (−4, 2) at (−4, 2)
Dr. Tran Thai-Duong (IU HCMC)
Optimization
Aug 6, 2012
6 / 16
Example Let f (x, y ) = 6x 2 + 6y 2 + 6xy + 36x − 5 and solve fx (x, y ) = 12x + 6y + 36 = 0 = 12y + 6x = fy (x, y ) These equations gives the only critical point (-4,2) Since fxx (−4, 2) = 12 = fyy (−4, 2) fxy (−4, 2) = fyx (−4, 2) = 6 D = 108 > 0, fxx > 0 f attains the local minimum −77 = f (−4, 2) at (−4, 2) Note that f (x, y ) = 6(0.5x + y )2 + 4.5(x + 4)2 − 77 and we really have a global/absolute minimum at (−4, 2) Dr. Tran Thai-Duong (IU HCMC)
Optimization
Aug 6, 2012
6 / 16
Example f (x, y ) = 2200 + 27x 3 − 72xy + 8y 2 From the equations fx = 81x 2 − 72y = 0 = 16y − 72x = fy we get y =
9x 2 ,
and 81x 2 − 324x = 81x(x − 4) = 0
and the critical points are (0, 0) and (4, 18)
Dr. Tran Thai-Duong (IU HCMC)
Optimization
Aug 6, 2012
7 / 16
Example f (x, y ) = 2200 + 27x 3 − 72xy + 8y 2 From the equations fx = 81x 2 − 72y = 0 = 16y − 72x = fy we get y =
9x 2 ,
and 81x 2 − 324x = 81x(x − 4) = 0
and the critical points are (0, 0) and (4, 18) At (4, 18), fxx = 648, fyy = 16, fxy = −72, and D > 0 Hence, f attains a local minimum at (4, 18) which is f (4, 18) = 1336.
Dr. Tran Thai-Duong (IU HCMC)
Optimization
Aug 6, 2012
7 / 16
Example f (x, y ) = 2200 + 27x 3 − 72xy + 8y 2 From the equations fx = 81x 2 − 72y = 0 = 16y − 72x = fy we get y =
9x 2 ,
and 81x 2 − 324x = 81x(x − 4) = 0
and the critical points are (0, 0) and (4, 18) At (4, 18), fxx = 648, fyy = 16, fxy = −72, and D > 0 Hence, f attains a local minimum at (4, 18) which is f (4, 18) = 1336. At (0, 0), fxx = 0, fyy = 16, fxy = −72, and D < 0. Hence, (0,0) is a saddle point Dr. Tran Thai-Duong (IU HCMC)
Optimization
Aug 6, 2012
7 / 16
Remark
f (x, y ) does not attain a minimum at (4,18) because f (−10, −45) < 0
Dr. Tran Thai-Duong (IU HCMC)
Optimization
Aug 6, 2012
8 / 16
Remark
f (x, y ) does not attain a minimum at (4,18) because f (−10, −45) < 0 However, f (x, y ) = 2200 + 2(9x − 2y )2 + 27(x 3 − 6x 2 )
Dr. Tran Thai-Duong (IU HCMC)
Optimization
Aug 6, 2012
8 / 16
Remark
f (x, y ) does not attain a minimum at (4,18) because f (−10, −45) < 0 However, f (x, y ) = 2200 + 2(9x − 2y )2 + 27(x 3 − 6x 2 ) If we restrict that x ≥ 0 then the minimum of x 3 − 6x 2 is -32 when x = 4 and the minimum of f (x, y ) is 2200+27(-32)=1336 when x = 4, y = 18
Dr. Tran Thai-Duong (IU HCMC)
Optimization
Aug 6, 2012
8 / 16
Example
f (x, y ) = (x 2 + y 2 )e −x fx = (−x 2 − y 2 + 2x)e −x = 2ye −x = fy = 0 That implies y = 0 and −x 2 + 2x = x(2 − x) = 0. Hence, the critical points are (0, 0) and (2, 0)
Dr. Tran Thai-Duong (IU HCMC)
Optimization
Aug 6, 2012
9 / 16
Example
f (x, y ) = (x 2 + y 2 )e −x fx = (−x 2 − y 2 + 2x)e −x = 2ye −x = fy = 0 That implies y = 0 and −x 2 + 2x = x(2 − x) = 0. Hence, the critical points are (0, 0) and (2, 0) Then fxx = (x 2 + y 2 − 2x − 2x + 2)e −x , fxy = −2ye −x , fyy = 2e −x D(x, y ) = (4 − 8x + 2x 2 − 2y 2 )e −2x We have a saddle point at (2, 0) because D(2, 0) < 0, and a local minimum at (0, 0) because D(0, 0) > 0 and fxx (0, 0) > 0.
Dr. Tran Thai-Duong (IU HCMC)
Optimization
Aug 6, 2012
9 / 16
Example
f (x, y ) = (x 2 + y 2 )e −x fx = (−x 2 − y 2 + 2x)e −x = 2ye −x = fy = 0 That implies y = 0 and −x 2 + 2x = x(2 − x) = 0. Hence, the critical points are (0, 0) and (2, 0) Then fxx = (x 2 + y 2 − 2x − 2x + 2)e −x , fxy = −2ye −x , fyy = 2e −x D(x, y ) = (4 − 8x + 2x 2 − 2y 2 )e −2x We have a saddle point at (2, 0) because D(2, 0) < 0, and a local minimum at (0, 0) because D(0, 0) > 0 and fxx (0, 0) > 0. Remark. It is obvious that the global minimum is f (0, 0) = 0
Dr. Tran Thai-Duong (IU HCMC)
Optimization
Aug 6, 2012
9 / 16
Absolute extrema An extremum at a boundary point does not have to be a local extremum, therefore boundary points need to be considered separately.
Dr. Tran Thai-Duong (IU HCMC)
Optimization
Aug 6, 2012
10 / 16
Absolute extrema An extremum at a boundary point does not have to be a local extremum, therefore boundary points need to be considered separately. Example. Find the extremum values of f (x, y ) = 2x + y − 3xy , ∀x, y ∈ [0, 1] The only local extremum is f 13 , 23 = 32 .
Dr. Tran Thai-Duong (IU HCMC)
Optimization
Aug 6, 2012
10 / 16
Absolute extrema An extremum at a boundary point does not have to be a local extremum, therefore boundary points need to be considered separately. Example. Find the extremum values of f (x, y ) = 2x + y − 3xy , ∀x, y ∈ [0, 1] The only local extremum is f 13 , 23 = 32 .On the boundary, x =0 y −2y + 2 x = 1 f (x, y ) = 2x y =0 −x + 1 y =1
Dr. Tran Thai-Duong (IU HCMC)
Optimization
Aug 6, 2012
10 / 16
Absolute extrema An extremum at a boundary point does not have to be a local extremum, therefore boundary points need to be considered separately. Example. Find the extremum values of f (x, y ) = 2x + y − 3xy , ∀x, y ∈ [0, 1] The only local extremum is f 13 , 23 = 32 .On the boundary, x =0 y −2y + 2 x = 1 f (x, y ) = 2x y =0 −x + 1 y =1 Hence, the maximum is f (1, 0) = 2 and the minimum f (0, 0) = f (1, 1) = 0. Dr. Tran Thai-Duong (IU HCMC)
Optimization
Aug 6, 2012
10 / 16
Extrema with a constraint Suppose f , g are differentiable functions of variables x, y , z, . . . and f has a local extremum on the constraint set G = {v |g (v ) = 0} at a point P ∈ G such that ∇g (P) 6= ~0. Then ∇f (P) = λ∇g (P) or fx (P) = λgx (P), fy (P) = λgy (P), fz (P) = λgz (P) The number λ is called a Lagrange multiplier
Dr. Tran Thai-Duong (IU HCMC)
Optimization
Aug 6, 2012
11 / 16
Extrema with a constraint Suppose f , g are differentiable functions of variables x, y , z, . . . and f has a local extremum on the constraint set G = {v |g (v ) = 0} at a point P ∈ G such that ∇g (P) 6= ~0. Then ∇f (P) = λ∇g (P) or fx (P) = λgx (P), fy (P) = λgy (P), fz (P) = λgz (P) The number λ is called a Lagrange multiplier Note that fx (P)gy (P) = λgx (P)gy (P) = gx (P)λgy (P) = gx (P)fy (P) The equations fx gy = fy gx , fx gz = fz gx , fy gz = fz gy are the Lagrange equations
Dr. Tran Thai-Duong (IU HCMC)
Optimization
Aug 6, 2012
11 / 16
Example Find the extreme values of f (x, y ) = x + 2y on the ellipse 3x 2 + 4y 2 = 3 Solution. g (x, y ) = 3x 2 + 4y 2 − 3 The Lagrange equation is 8y = fx gy = fy gx = 12x Substitute y =
3x 2
in the constraint, we have 3x 2 + 4
3x 2
2
− 3 = 12x 2 − 3 = 0
Therefore x = ± 12 and y = 32 x = ± 43 Maximum is f 12 . 34 = 2. Minimum is f − 21 . − 43 = −2
Dr. Tran Thai-Duong (IU HCMC)
Optimization
Aug 6, 2012
12 / 16
Example
Find the extreme values of f (x, y ) = x 2 + 2y 2 on the circle x 2 + y 2 = 1 Solution. The Lagrange equation is fx gy = 4xy = fy gx = 8xy which implies x = 0 or y = 0. Therefore the critical points are (0, ±1), (±1, 0) and after comparing the values of f at these points we have max = f (0, ±1) = 2, min = f (±1, 0) = 1
Dr. Tran Thai-Duong (IU HCMC)
Optimization
Aug 6, 2012
13 / 16
Example
Find the extreme values of f (x, y ) = x 2 + 2y 2 on the circle x 2 + y 2 = 1 Solution. The Lagrange equation is fx gy = 4xy = fy gx = 8xy which implies x = 0 or y = 0. Therefore the critical points are (0, ±1), (±1, 0) and after comparing the values of f at these points we have max = f (0, ±1) = 2, min = f (±1, 0) = 1 It is obvious that f (±1, 0) = 1 = x 2 + y 2 ≤ f (x, y ) = x 2 + 2y 2 ≤ 2x 2 + 2y 2 = 2 = f (0, ±1)
Dr. Tran Thai-Duong (IU HCMC)
Optimization
Aug 6, 2012
13 / 16
Example Find the extremum of f (x, y , z) = xyz subject to the constraint xy + yz + zx = 3 Solution. g (x, y , z) = xy + yz + zx − 3, then fx = yz, fy = xz, fz = xy , gx = y + z, gy = x + z, gz = x + y From the constraint, at least two variables must be different from zero. Assume x and y are not zero and Solve the equations xy (x + z) = xz(x + y ) xy (y + z) = yz(x + y ) xy + yz + xz = 3 we then get x = y = z = ±1 max=f(1,1,1)=1, min=f(-1,-1,-1)=-1 Dr. Tran Thai-Duong (IU HCMC)
Optimization
Aug 6, 2012
14 / 16
Cobb-Douglas Production function
Find x, y to get the maximum of f (x, y ) = x 0.4 y 0.6 subject to the constraint g (x, y ) = x + 2y = 200 The Lagrange equation is 0.8x −0.6 y 0.6 = fx gy = fy gx = 0.6x 0.4 y −0.4 Multiply by x 0.6 y 0.4 we have 0.8y = 0.6x or y = constraint equation we get x = 80, y = 60
Dr. Tran Thai-Duong (IU HCMC)
Optimization
2x 3 .
Use this with the
Aug 6, 2012
15 / 16
Cobb-Douglas Production function
Find x, y to get the maximum of f (x, y ) = x 0.4 y 0.6 subject to the constraint g (x, y ) = x + 2y = 200 The Lagrange equation is 0.8x −0.6 y 0.6 = fx gy = fy gx = 0.6x 0.4 y −0.4 Multiply by x 0.6 y 0.4 we have 0.8y = 0.6x or y = constraint equation we get x = 80, y = 60
2x 3 .
Use this with the
Remark. x ∈ [0, 100] and the value of f is minimum iff x = 0, 100. Hence, we get the maximum f (80, 60) = 800.4 600.6 = 20(40.4 30.6 )
Dr. Tran Thai-Duong (IU HCMC)
Optimization
Aug 6, 2012
15 / 16
HOMEWORK #8 1. a. b. c.
Find the critical points of the functions x 3 − y 3 + 6xy xy 2 − 6x 2 − 3y 2 2 2 e −(x +y −6y )
2. Determine the global extreme values of the functions f (x, y ) = x + y , g (x, y ) = x − y where x ∈ [0, 1], y ∈ [3, 5] 3. Find local extremum of a. f (x, y ) = x 3 − xy + y 3 b. g (x, y ) = x 3 + 2xy − 2y 2 − 10x 4. Find the minimum and maximum values of the function f subject to the given constraint a. f (x, y ) = 2x + y , x 2 + y 2 = 4 b. f (x, y ) = 4x 2 + 9y 2 , xy = 4 c. f (x, y ) = x 2 y + x + y , xy = 4 d. f (x, y ) = x 2 y 4 , x 2 + 2y 2 = 6 Dr. Tran Thai-Duong (IU HCMC)
Optimization
Aug 6, 2012
16 / 16
