Name: Akash Sajeev

Roll No: G24AIT092

Sub: Optimization of Data Science
Assignment-1

Q(1). Show that the function f(x, y) = x4 −y4 has neither local minima nor local maxima.
Solution:

Critical Points:

First, we find the critical points by computing the gradient of f:

Setting the gradient to zero:

4x3=0⟹x=0,
−4y3=0⟹y=0.
So, the only critical point is at (0,0).
Second Derivative:
Compute the Hessian matrix HHH at (0,0):

At the critical point (0,0), we substitute x=0 and y=0 into this matrix:

This is the zero matrix.

Analyze the Behavior Near (0,0)

To prove that the function has neither local minima nor local maxima, we need to examine how the function
behaves in a neighborhood around the point (0,0). This can be done by checking whether there are both positive
and negative function values arbitrarily close to (0,0). If we can find such values, it would imply that (0,0) is
neither a local maximum nor a local minimum (which suggests it's a saddle point).

Let's consider different directions approaching (0,0) and check the function values.

Along the xxx-axis (y=0):

When y=0, the function reduces to:

f(x,0) = x4−04 = x4.

Here, f(x,0)≥0 and f(0,0)=0. As x moves away from 0, the function values are non-negative, increasing as x moves
away from 0. This suggests a local minimum along the xxx-axis.

1|Page
Name: Akash Sajeev
Roll No: G24AIT092
Sub: Optimization of Data Science
Assignment-1

Along the y-axis (x=0):

When x=0, the function reduces to:

f(0,y)=04−y4=−y4

Here, f(0,y)≤0 and f(0,0)=0. As y moves away from 0, the function values are non-positive, decreasing as y moves
away from 0. This suggests a local maximum along the y-axis.

Along the line y=x.

Now, consider y=x. The function becomes:

f(x,x)=x4−x4=0.

Along this line, the function is constantly zero, indicating that it remains unchanged along this direction.

Along the line y=−x

Now, consider y=−x.

The function becomes:

f(x,−x)=x4−(−x)4=x4−x4=0.

Similarly, along this line, the function is also constantly zero.

Near the origin with small perturbations:

Consider points close to the origin, such as:

 For (x,y)=(0.1,0)

f(0.1,0)=(0.1)4−04=0.0001.

This is positive.

 For (x,y)=(0,0.1)(x, y) = (0, 0.1)

f(0,0.1)=04−(0.1)4=−0.0001.f(0, 0.1) = 04 - (0.1)4 = -0.0001.

This is negative.

We see that f takes both positive and negative values in the neighborhood of (0,0).

Conclusion:

Since f(x,y) takes both positive and negative values arbitrarily close to (0,0) we conclude that (0,0) is not a local
minimum or local maximum.

2|Page
 Name: Akash Sajeev
Roll No: G24AIT092
Sub: Optimization of Data Science
Assignment-1

Ques(2). Show that the function f(x, y) = x3 − 12xy + 8y3 has no global minima, but it has a local minima at (2, 1).
Compute the partial derivative of f(x, y) = x3 − 12xy + 8y3.
df 2
=3 x −12 y
dx

df 2
=−12 x +24 y
dy

Set the partial derivatives to zero:

3x-−12y=0---(1)
−12x+24y2=0 -----(2).
Solve Equation (1) for y
3x2−12y=0

⟹y=
x2
4
Substitute y into Equation (2):
2 2
x
−12 x+24 ( ) =0.
4
x4
−12 x+24 ( )=0.
16

x4
−12 x+(3 )=0 .
2

Multiply both sides by 2 to eliminate fractions:

−24x+3x4=0

Divide by 3:

−8x+x4=0

Factor the equation:

x(−8+x3) = 0

Set each factor to zero:

x3=8 ⟹
 x=0

 x=2

3|Page
Name: Akash Sajeev
Roll No: G24AIT092
Sub: Optimization of Data Science
Assignment-1

Find corresponding y values:

For x=0
2
0
y= =0 .
4

For x=2
2
2
y= =1.
4

Critical Points:

 (0,0).
 (2,1).

Classifying Critical Points Using the Second Derivative Test

Compute the second-order partial derivatives:

2
∂ f
2
=6 x
∂x
2
∂ f
2
=48 y
∂y
2
∂ f
=−12
∂ x∂ y

At (0,0).

fxx=6×0=0.

fyy=48×0=0.

fxy=−12

Calculate the determinant of the Hessian matrix (DDD):

D=fxxfyy−(fxy)2=(0)(0)−(−12)2=−144

Since D<0 the critical point (0,0) is a saddle point.

At (2,1).

fxx=6×2=12.

fyy=48×1=48

4|Page
 Name: Akash Sajeev
Roll No: G24AIT092
Sub: Optimization of Data Science
Assignment-1

fxy=−12

Calculate the determinant of the Hessian matrix (D)

D=(12)(48)−(−12)2=576−144=432

Since D>0 and fxx>0 the critical point (2,1) is a local minimum.

Showing the Function Has No Global Minima.

To demonstrate that f(x,y) has no global minima, we need to show that the function can attain arbitrarily large
negative values.

Consider the function along the line y=0.

f(x,0)=x3−12x(0)+8(0)3=x3.

As x→−∞

f(x,0)=(−∞)3=−∞

Since f(x,y)f(x, y)f(x,y) can be made arbitrarily large in the negative direction, it does not have a global minimum.

However, due to the positive definiteness of the Hessian at (2,1)(2, 1)(2,1), there is a local minimum at that point.

Ques(3). Use the golden section search to find the value of x that minimizes f(x) = x4−14x3+60x2 − 70x in the
range [0, 2]. Locate this value of x to within a range of 0.3.

Golden Ratio (RRR):= 0.618

Interval: [a,b]=[0,2]

Desired Accuracy: ϵ=0.3.

Iteration 1

x1=a+C(b−a)=0+0.382×(2−0)=0.764

x2=a+R(b−a)=0+0.618×(2−0)=1.236

f(x1)=f(0.764)≈−24.377

f(x2)=f(1.236)≈−18.976

Since f(x1)<f(x2) the minimum lies in [a,x2].

New Interval: [a,b]=[0,1.236].

5|Page
 Name: Akash Sajeev
Roll No: G24AIT092
Sub: Optimization of Data Science
Assignment-1

Iteration 2

x1=a+C(b−a)=0+0.382×(1.236−0)=0.472

x2=a+R(b−a)=0+0.618×(1.236−0)=0.764

f(x1)=f(0.472)≈−21.106

Since f(x2)<f(x1), the minimum lies in [x1,b].

New Interval: [a,b]=[0.472,1.236].

Iteration 3

x1=a+C(b−a)=0.472+0.382×(1.236−0.472)=0.764

x2=a+R(b−a)=0.472+0.618×(1.236−0.472)=0.944

f(x2)=f(0.944)≈−23.601

Since f(x1)<f(x2), the minimum lies in [a,x2].

New Interval: [a,b]=[0.472,0.944].

Iteration 4

x1=a+C(b−a)=0.472+0.382×(0.944−0.472)=0.653

x2=a+R(b−a)=0.472+0.618×(0.944−0.472)=0.764

f(x1)=f(0.653)≈−23.853

Since f(x2)<f(x1), the minimum lies in [x1,b].

New Interval: [a,b]=[0.653,0.944].

Current Interval Length: b−a=0.944−0.653=0.291.

Since 0.291≤0.30.291, we have achieved the desired accuracy .

The minimum lies within the interval [0.653,0.944].

We can take x≈0.764. f(0.764)≈−24.377.

Ques(4). Solve the above problem using the Fibonacci method. f(x) = x4−14x3+60x2 − 70x

b−a
Fn ​≥
l

6|Page
Name: Akash Sajeev
Roll No: G24AIT092
Sub: Optimization of Data Science
Assignment-1

Initial interval: [a,b]=[0,2].

Desired interval length: l=0.3

b−a 2−0
Calculate = =6.6667
l 0.3

Since F6 = 8 ≥ 6.6667, we'll use n=6 iterations.

 a0=a=0

 b0=b=0

We'll perform n−1=5 iterations.

k=n−i+1

F k−1
X 1 =ai + (bi−ai)
F k+ 1

Fk
X 2 =ai + (bi ​−ai ​)
F k +1

Iteration 1 (i = 1, k = 6).
F5 5
X 1 =a0 + ( b 0−a 0 )= 0 ​+ ( 2 )=0.7692
F7 13
F6 8
X 2 =a0 + ( b 0−a 0 ) = 0 ​+ ( 2 )=1.2308
F7 13
Evaluate Function:
f(x1) = (0.7692)4−14(0.7692)3+60(0.7692)2−70(0.7692) =−24.378
f(x2) = (1.2308)4−14(1.2308)3+60(1.2308)2−70(1.2308) =−19.068
Since f(x1) < f(x2) the minimum lies in [a0,x2].
a1=a0=0, b1=x2 = 1.2308

Iteration 2 (i = 2, k = 5)
F4 3
X 1 =a1 + ( b1−a1 )= 0 ​+ ( 1.2308 )=0.4615
F6 8
F6 5
X 2 =a1 + ( b1−a1 )= 0 ​+ ( 1.2308 )=0.7692
F7 8
f(x1) = −20.824
Since f(x2) < f(x1), the minimum lies in [x1, b1]

7|Page
 Name: Akash Sajeev
Roll No: G24AIT092
Sub: Optimization of Data Science
Assignment-1

a2 = x1 = 0.4615, b2 = b1 = 1.2308
Iteration 3 (i = 3, k = 4)
F3 2
X 1 =a2 + ( b2−a 2 )= 0.4615 ​+ ( 0 .7693 )=0.7692
F5 5
F4 3
X 2 =a2 + ( b2 −a2 )= 0.4615 ​+ ( 0 .7693 )=0.9230
F5 5
f(x2) = −23.790
Since f(x1) < f(x2), the minimum lies in [a2, x2]
a3 = a2 = 0.4615, b3 = x2 = 0.9230
Iteration 4 (i = 4, k = 3)
F2 1
X 1 =a3 + ( b3 −a3 ) = 0.4615 ​+ ( 0 .4615 )=0.6154
F4 3
F3 2
X 2 =a3 + ( b 3−a3 ) = 0.4615 ​+ ( 0 .4615 )=0.7692
F4 3
f(x1) = −23.477
Since f(x2) < f(x1), the minimum lies in [x1, b3]
a4 = x1 = 0.6154, b4 = b3 = 0.9230
Iteration 5 (i = 5, k = 2)
F1 1
X 1 =a4 + ( b 4−a4 )= 0.6154 ​+ ( 0 .3076 )=0.7692
F3 2
F2 1
X 2 =a4 + ( b 4−a4 ) = 0.6154 ​+ ( 0 .3076 )=0.7692
F3 2
f(x1) and f(x2) are same as before.
Since f(x2) = f(x1), we can choose either interval. Let's choose [a4,x2]
a5 = a4 = 0.6154, b5 = x2 = 0.7692
Since b5−a5 = 0.1538 ≤ l = 0.3, we have achieved the desired interval length.
The minimum lies within the interval [0.6154,0.7692].
(a ¿ ¿ 5+ b5 )
=0.6154+0.7692 ¿
2
x min= =0.6923
2
f(xmin)=f(0.6923)≈−24.4
Ques(5). Perform two iterations of the DFP’s method to minimize the function f(x, y) =100(y − x2)2 + (1 − x)2 from the
starting point (−1.2, 1.0).

( )( )
x1
y1
=
−1.2
1.0

1 0
B1=
0 1

f(x,y)=100(y−x2)2+(1−x)2

8|Page
Name: Akash Sajeev
Roll No: G24AIT092
Sub: Optimization of Data Science
Assignment-1

First Iteration

Compute the Gradient at x0=(−1.2,1.0)

First, compute u and v
U = y0−xv2 = 1.0−(−1.2)2 = −0.44
v = 1−x0 = 1−(−1.2)=2.2
∂f
=−400 x 0 u−2 v=−400(−1.2)(−0.44)−2(2.2)=−215.6
∂x
∂y
=200u=200(−0.44)=−88
∂f

∇f(x0)= [−256
−88 ]

d1 = −B1∇f(x1) = -I [−256 ] [ 88 ]
256
=
−88

Find α1 Minimizing f(x1+α1d1)

X2 /x(α)= x1+αd1= [−1.2+215.6

1.0+88 α ]
α

At α=0.

x(0) = [−1.2
1.0 ]
, f(x(0))=24.2

x(0.0004)= ¿ ,f(x(0.0004))=8.678

x(0.0008)= [−1.02752
1.0704 ]
, f(x(0.0008))=4.1285

x(0.0012)= [−0.94128
1.1056 ]
,f(x(0.0012))=8.5886

f(x(α)) reaches a minimum at α=0.0008. Therefore α1=0.0008

x2=x1+α1d1= [−1.2+0.0008 ×215.6

1.0+0.0008 ×88 ] [ 1.0704 ]
−1.02752
=

Compute ∇f(x2):
At x2 = (−1.02752,1.0704)
u=1.0704−(−1.02752)2=0.0136
v=1−(−1.02752) = 2.02752

9|Page
Name: Akash Sajeev
Roll No: G24AIT092
Sub: Optimization of Data Science
Assignment-1

df
= −400xu−2v=1.5377
dx
df
= 200u=2.72
dy

∇f(x2) = [ 1.5377
2.72 ]

Update Matrix B2 Using B2=B1+M1+N1

s1=x2−x1 = [ −1.02752−(−1.2)
1.0704−1.0
= ] [
0.17248
0.0704 ]
y1=∇f(x2) −∇f(x1) =
[ 2.72−(−88) ] [ 217.1377
1.5377−(−215.6)
90.72 ]
=

T
s1 s 1
M1 = T
s1 y1

s1 = x2−x1 = [ −1.02752−(−1.2)
1.0704−1.0
= ] [
0.17248
0.0704 ]
y1 = ∇f(x2) − ∇f(x1) =
[ 1.5377−(−215.6)
2.72−(−88)
=
] [
217.1377
90.72 ]
[ ][ ]
T
s1 s 1 1 0.02974 0.01214 0.000679 0.000277
M1 = = =
T
s y1
1
(0.17248)(217.1377)+(0.0704)( 90.72) 0 . 01214 0.00496 0.000277 0.000113

[ ][ ]
T
B1 y 1 y 1 B1 1 47152.727 19699.8 −0.852 −0.355
N1= =- =
T
y B1 y 1
1
55382.0454 19699.8 8229.3184 −0.355 −0.149

B2=B1+M1+N1=I+M1+N1

B2= [ 1+0000277−0.355
0.000679−0.852 0.000277−0.355
1+0.000113−0.149
= ] [
0.148679 −0.354723
−0.354723 0.851113 ]
Second Iteration

∇f(x2) = [ 1.5377
2.72 ]

d2 = −B2∇f(x2) = [−0.354723 ][ ]
0.148679 −0.354723 1.5377
= [ ¿¿1.77072¿ ]
0.851113 2.72

x(α)=x2+αd2 = [−1.02752+0.73608
1.0704−1.77072 α ]
α

10 | P a g e
 Name: Akash Sajeev
Roll No: G24AIT092
Sub: Optimization of Data Science
Assignment-1

at α=0

x(0) = [−1.02752
1.0704 ]
, f(x(0))=4.1285

x(0.1)= [−0.95391
0.89332 ]
, f(x(0.1)) = 3.8465

x(0.15) = [−0.91711
0.8048 ]
, f(x(0.15)) = 3.808

x(0.2) = [−0.8803
0.71624 ]
, f(x(0.2)) = 3.8807

f(x(α)) reaches a minimum at α=0.15. Therefore:

α2=0.15

x3=x2+α2d2 = [−1.02752+0.15
1.0704−0.15× 1.77072 ] [ 0.8048 ]
×0.73608 −0.91711
=

After Two Iterations

x2= [−1.02752
1.0704 ]

x3= [−0.91711
0.8048 ]

[ ]
5 2 2
Ques(6). Find a set of conjugate directions for the matrix A = 2 5 2
2 2 5

[] [] []
1 0 0
v1= 0 , v2 = 1 , v2 = 0
0 0 1

Set p1 = v1

Second Conjugate Direction p2

11 | P a g e
Name: Akash Sajeev
Roll No: G24AIT092
Sub: Optimization of Data Science
Assignment-1

[]
0
[1 0 0 ] . A . 1
T
p1 A v 2 0 2
α21 = = =

[]
T
p A p1
1 1 5
[1 0 0 ] . A . 0
0

[] [][ ]
−2
0 1
2 5
p2 = v2−α21p1 = 1 − 0 =
5 1
0 0
0
Third Conjugate Direction p3
T
p1 A v 2 2
α31 = T = ,
p A p1
1
5
T
p2 A v 3 2
α32 = =
T
p A p2
2
7

[ ][ ]
−2
−2

[] []
0 1 7
2 2 5
p3 = v3−α31p1−α32p2 = 0 − 0 − = −2
5 7 1
1 0 7
0
1

Ensure that
p1⊤Ap2=0
p1⊤Ap3=0
p2⊤Ap3=0

Final Set of Conjugate Directions:

[] []
−2
−2

[]
1 7
5
p 1 = 0 , p2 = , p3 = −2
1
0 7
0
1

12 | P a g e
 Name: Akash Sajeev
Roll No: G24AIT092
Sub: Optimization of Data Science
Assignment-1

Ques(7).Use Newton’s method to find the optimal solution for the function f(x, y) = 2x2 + 4y2 + 5xy − 13x − 18y + 20

df
=4 x +5 y−13
dx
df
=8 y +5 x−18
dy
Hessian Matrix (H)

[ ]
2 2
d f d f

[ 45 58 ]
2
dx dxdy
H= 2 2
=
d f d f
2
dxdy dy
det(H) = (4)(8)−(5)(5) = 32−25 = 7.

H−1 =
1
[
8 −5 1 8 −5
det (H) −5 4
=
7 −5 4
.] [ ]
(x0,y0) = (0,0).

∇f(0,0) =
[ 4 (0)+5(0)−13
8(0)+5 (0)−18
=
−13
−18 ][ ]
Newton update formula

= k −H−1∇f(xk,yk).
[ ][ ]
x k +1
y k+1
x
yk

Compute H−1∇f(0,0) = [ ][ ] [ ] [ ][ ]
1 8 −5 −13 1 (−104 )+ 90 1 −14 −2
= = =
7 −5 4 −18 7 65−72 7 −7 −1

x1=x0−(−2) = 0+2 = 2,
y1=y0−(−1) = 0+1 = 1.

∇f(2,1) =
[ 4 (2)+5 (1)−13
8(1)+5(2)−18
= ][
8+5−13
8+10−18
=
0
0 ] []
Eigenvalues of H=λ, where λ2−12λ+7=0.
12± √ 123−4.1 .7 12± √ 144−28 12± √ 116
λ= = =
2 2 2
The optimal solution is at x=2 and y=1.

13 | P a g e
 Name: Akash Sajeev
Roll No: G24AIT092
Sub: Optimization of Data Science
Assignment-1

Ques(8). Using the least square method, find the best approximate solution to the system of equations
x1 + 2x2 = 1
x1 − x2 = 0
x1 + x2 = 1

Let A be the coefficient matrix, x be the unknown vector, and b be the constants vector.

[ ] [] []
1 2 x1 1
A= 1 −1 x= , b= 0
x2
1 1 1

AT = [ 12 1 1
−1 1 ]
[ ][ ]
2 2 2
1 +1 + 1 1.2+1. (−1 ) +1.1 3 2
A AT = 2 2 2 =
2⋅1+(−1)⋅1+1 ⋅1 2 +−1 + 1 2 6

AT b = [ 1 ⋅1+1 ⋅0+1 ⋅1
2⋅1+(−1)⋅0+ 1⋅1
=
2
3 ] []
AT A x = AT b

[ 32 62][ xx ]=[23 ]
1

3x1+2x2 = 2
2x1+6x2 = 3
2−2 x 2
X1 =
3
Put x1 another equation.

2( 2−23 x )+6 x =3
2
2

2(2−2x2)+18x2 = 9
4−4x2+18x2 = 9
4+14x2 = 9
5
X2 =
14
5
2−2( ) 3
X1 = 14 =
7
3

14 | P a g e
Name: Akash Sajeev
Roll No: G24AIT092
Sub: Optimization of Data Science
Assignment-1

15 | P a g e
Name: Akash Sajeev
Roll No: G24AIT092
Sub: Optimization of Data Science
Assignment-1

16 | P a g e
Name: Akash Sajeev
Roll No: G24AIT092
Sub: Optimization of Data Science
Assignment-1

17 | P a g e
Name: Akash Sajeev
Roll No: G24AIT092
Sub: Optimization of Data Science
Assignment-1

18 | P a g e
Name: Akash Sajeev
Roll No: G24AIT092
Sub: Optimization of Data Science
Assignment-1

19 | P a g e
Name: Akash Sajeev
Roll No: G24AIT092
Sub: Optimization of Data Science
Assignment-1

20 | P a g e
Name: Akash Sajeev
Roll No: G24AIT092
Sub: Optimization of Data Science
Assignment-1

21 | P a g e
Name: Akash Sajeev
Roll No: G24AIT092
Sub: Optimization of Data Science
Assignment-1

22 | P a g e
Name: Akash Sajeev
Roll No: G24AIT092
Sub: Optimization of Data Science
Assignment-1

23 | P a g e
Name: Akash Sajeev
Roll No: G24AIT092
Sub: Optimization of Data Science
Assignment-1

24 | P a g e