Dr. S.

Bourazza Math-473 Jazan University Department of Mathematics

Dr. Said Bourazza

Department of Mathematics

Jazan University

1|P ag e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

Contents:
Chapter 0: Modelization 3

Chapter1: Graphical Methods 7

Chapter2: Simplex method 13

Chapter3: Duality 36

Chapter4: Transportation Problem 39

Chapter5: Integer Linear problem 46

Chapter6: Nonlinear programming problem 50

2|P ag e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

Chapter 0: Modelization
Introduction:

The operation research (OR), known also as Management Science, System Analysis, Decision
Analysis, is the application of the scientific methods, techniques and tools to problem involving
the operations of systems so as to provide those in control of the operations with optimum
solutions to the problems.

During the Second World War, the OR began was to be considered as a separate discipline.

• In England (Operational Research): British government organized teams of experts to

solve perplexing strategic and tactical problems.

• In USA (Operations Research): Applied OR to the deployment of merchant marine

convoys to minimize losses from enemy submarines.

After this war, OR extended into industry like Oil refineries, steel, paper mills and to almost all
industries, services and large social and urban systems.

Mathematical Model:

Generally, we look for the solution of the following kind of problem:

Max or Min z = c1x1 + c2x2 + c3x3 +…+ cnxn (Objective function)

Subject to : a11x1 + a12x2 +…+ a1nxn ≤ b1

a21x1 + a22x2 +…+ a2nxn ≤ b2

… … … … … … (Constraints)

am1x1 + am2x2 +…+ amnxn ≤ bm

all xi ≥ 0 , i = 1, 2, 3, …, n (non-negativity constraints)

Definitions:

• Linear programming problem: is to optimize the linear function Z subject to certain

conditions

3|P ag e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

• Objective functions: Is a function of several variables

• Optimal value: Is a maximum or minimum value of an objective function

Constructing the Linear Programming (LP) Problem for optimizing of the Objective
Function:

Constructing the LP problem requires three steps:

 Step 1: Define the decision variables.

 Step 2: Define the objective function.
 Step 3: Determine the constraints.

Example1: Maximization problem

A manufacturer produces 2 types of toys A and B.

• Each toy of type A requires 4Hours of molding and 2 Hours of polishing.
• Each toy of type B requires 3Hours of molding and 5 Hours of polishing.
• Molder worker works for 80Hours in a week
• Polisher worker works 180 Hours in a week
• Profit on buying a toy A is 3 SAR and on B is 4 SAR
Give the linear programming problem of this problem to maximize the profit per week.

Answer:

This information can be written in tabular form as follows:

Molding Polishing Profit

A 4 2 3

B 3 5 4

Time available 80 180

(Hrs)
Step1: Decision variables:

• x: number of toys of type A per week

• y: number of toys of type B per week
Step 2: Objective function:
• Maximize the profit Z= 3x+4y
Step3: Constraints:
• Molding total time required 4x+3y ≤80
• Polishing total time required 2x+5y≤180
• The number of toys produced is a positive integer: x ≥ 0, y ≥ 0

4|P ag e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

Summary

Maximize Z=3x+4y
Subject to:
4x+3y ≤ 80
2x+5y ≤ 180
x ≥ 0, y ≥ 0 and integers
Example 2: Minimization (Diet Problem)

A dietician mixes two kinds of food X, and Y in such way that the mixture contains at least 6
units of vitamin A, 7 units of vitamin B, 12 units of vitamin C and 9 units of vitamin D.

The vitamin contents of 1kg of food X and 1 kg of food Y are shown in the following table:

Vitamin A Vitamin B Vitamin C Vitamin D

Food X 1 1 1 2

Food Y 2 1 3 1
1Kg of food X costs 5 SAR whereas 1Kg of food Y costs 8 SAR.
Formulate this text as linear programming problem.
Answer:

Step1: Decision variables:

• x: number of kg of food X
• y: number of kg of food Y
Step 2: Objective function:
• Minimize the cost Z= 5x+8y
Step3: Constraints:
The word “at least “ in the inequality will be ≥.
• Vitamin A: x +2y ≥ 6
• Vitamin B: x + y ≥ 7
• Vitamin C: x +3y ≥ 12
• Vitamin D: 2x +y ≥ 9
• Non negativity of the variables: x ≥ 0, y ≥ 0
In summary :

Min Z= 5x + 8y
Subject to:
x + 2y ≥ 6
x+y≥7
x + 3y≥ 12
2x + y ≥ 9
x ≥ 0, y ≥ 0

5|P ag e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

Example 3: Transportation problem

There is a factory located at each of the 2 places P and Q. From these places, a certain
commodity is delivered to each of 3 depots situated at A, B, and C.

The weekly requirements of the depots are respectively 5, 5, and 4 while the production capacity
of factories at P, Q are 8 and 6 Units resp. just sufficient for the requirement of the depots.

The cost is given by the following table:

A B C

P 16 10 15

Q 10 12 10
Formulate the LPP of this transportation problem in order to minimize the transportation
cost.

Answer:

For the Modelization of this problem, we can use the easy way by using six variables or the
following way by using only two variables:

We obtain the following where x is the quantity of commodity transported from factory P to
depot A and y is from P to depot B:

Min Z = x - 7y + 190
Subject to: x+y≤8
x+y≥4
0≤ x≤5
0≤ y≤5

6|P ag e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

Chapter1: Graphical Methods

The graphical method is used only for solving LPP with only 2 variables. There are two
methods:
a) Corner point method
b) Iso-profit or Iso-cost method

Definitions:

• Feasible region: The portion determined by all constraints including non-negative

constraint of a linear P.P.

• Feasible solutions: Points within and on the boundary of the feasible region.

• Optimal solution: Any point in the feasible region gives the optimal value
(maximum or minimum)

Theorem1:

The optimal value of objective function must occur at a corner point (vertex) of the feasible
region.

Theorem 2:

If the feasible region is bounded then the objective function Z has both maximum and
minimum values on corner points of the bounded regions.

To solve an LP, the graphical method includes two major steps.

a) The determination of the solution space that defines the feasible solution. Note that the set of
values of the variable x1, x2, x3,....xn which satisfy all the constraints and also the non-negative
conditions is called the feasible solution of the LP.

b) The determination of the optimal solution from the feasible region.

a) To determine the feasible solution of an LP, we have the following steps.

Step 1: Since the two decision variable x and y are non-negative, consider only the first quadrant
of xy-coordinate plane
Step2: each constraint is the form ax+by ≤c or ax+by ≥c.

7|P ag e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

Draw the line ax + by = c (1)

For each constraint, the line (1) divides the first quadrant in to two regions say R1 and R2, suppose
(x1, 0) is a point in R1. If this point satisfies the in equation ax + by ≤ c or (≥ c), then shade the
region R1. If (x1, 0) does not satisfy the inequality, shade the region R2.
Step 3: Corresponding to each constant, we obtain a shaded region. The intersection of all these
shaded regions is the feasible region or feasible solution of the LP.

Example1: Let us find the feasible solution for the problem of a decorative item dealer whose LPP
is to maximize profit function. Z = 50x + 18y (1)
Subject to the constraints: 2x+y ≤ 100
x + y ≤ 80
x, y ≥ 0
Step 1: Since x ≥ 0, y ≥ 0, we consider only the first quadrant of the xy-plane
Step 2: We draw straight lines for the equation
2x+ y = 100 (2)
x + y = 80
To determine two points on the straight line 2x + y = 100
Put y = 0, 2x = 100 => x=50 => (50, 0) is a point on the line (2)
put x = 0 in (2), y =100 =>(0, 100) is the other point on the line (2)
Plotting these two points on the graph paper draw the line which represent the line 2x + y =100.

This line divides the 1st quadrant into two regions, say R1 and R2. Choose a point say (1, 0) in R1.
(1, 0) satisfy the inequality 2x + y ≤ 100. Therefore R1 is the required region for the constraint
2x + y ≤ 100.
Similarly draw the straight line x + y = 80 by joining the point (0, 80) and (80, 0). Find the
required region say R1', for the constraint x + y ≤ 80.
The intersection of both the region R1 and R1' is the feasible solution of the LPP. Therefore every
point in the shaded region OABC is a feasible solution of the LPP, since this point satisfies all the

8|P ag e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

constraints including the non-negative constraints.

b) There are two techniques to find the optimal solution of an LPP.
Corner Point Method
The optimal solution to a LPP, if it exists, occurs at the corners of the feasible region.
The method includes the following steps
Step 1: Find the feasible region of the LLP.
Step 2: Find the co-ordinates of each vertex of the feasible region.
These co-ordinates can be obtained from the graph or by solving the equation of the lines.
Step 3: At each vertex (corner point) compute the value of the objective function.
Step 4: Identify the corner point at which the value of the objective function is maximum (or
minimum depending on the LP)
The co-ordinates of this vertex is the optimal solution and the value of Z is the optimal value
Corner points Z= 50x + 18y
O(0,0) ZO=0 + 0=0
A(0, 80) ZA=0 + 1440=1440
B(20, 60) ZB= 1000 + 1080=2080
C(50, 0) ZC=2500

Max Z=Zc=2500 for x=50 and y=0.

Since our object is to maximize Z and Z has maximum at (50, 0) the optimal solution is x = 50 and
y = 0. The optimal value is 2500.
If an LPP has many constraints, then it may be long and tedious to find all the corners of the
feasible region. There is another alternate and more general method to find the optimal solution of
an LP, known as 'ISO profit or ISO cost method'
ISO- PROFIT (OR ISO-COST)
This method of optimization involves the following method.
Step 1: Draw the half planes of all the constraints
Step 2: Shade the intersection of all the half planes which is the feasible region.
Step 3: Since the objective function is Z = ax + by, draw a dotted line for the equation ax + by = k,
where k is any constant. Sometimes it is convenient to take k as the Lower Common Multiple of a
and b.
Step 4: To maximize Z draw a line parallel to ax + by = k and farthest from the origin. This line
should contain at least one point of the feasible region. Find the coordinates of this point by
solving the equations of the lines on which it lies.
To minimize Z draw a line parallel to ax + by = k and nearest to the origin. This line should
contain at least one point of the feasible region. Find the co-ordinates of this point by solving the
equation of the line on which it lies.
Step 5: If (x1, y1) is the point found in step 4, then x = x1, y = y1, is the optimal solution of the LPP
and Z = ax1 + by1 is the optimal value.
Example 2: Solve the following LPP graphically using ISO- profit method.
Maximize Z =100 + 100y.
Subject to the constraints: 10x + 5y ≤ 80
6x+6y ≤ 66
4x+ 8y ≥ 24
5x+6y ≤ 90
x, y ≥ 0

9|P ag e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

Answer:
Since x ≥ 0, y ≥ 0, consider only the first quadrant of the plane graph and the following straight
lines on a graph paper.
10x + 5y = 80 or 2x+y =16
6x + 6y = 66 or x +y =11
4x+ 8y = 24 or x+ 2y = 6
5x + 6y = 90
Identify all the half planes of the constraints. The intersection of all these half planes is the
feasible region as shown in the figure.

Give a constant value 600 to Z in the objective function, then we have an equation of the line
120x + 100y = 600 (1)
or 6x + 5y = 30 (Dividing both sides by 20)
P1Q1 is the line corresponding to the equation 6x + 5y = 30. We give a constant 1200 to Z then the
P2Q2 represents the line.
120x + 100y = 1200
6x + 5y = 60
P2Q2 is a line parallel to P1Q1 and has one point 'M' which belongs to feasible region and farthest
from the origin. If we take any line P3Q3 parallel to P2Q2 away from the origin, it does not touch
any point of the feasible region.
The co-ordinates of the point M can be obtained by solving the equation 2x + y = 16
x + y =11 which give x = 5 and y = 6 . The optimal solution for the objective function is x = 5
and y = 6 . The optimal value of Z : 120 (5) + 100 (6) = 600 + 600 = 1200

10 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

Example 3: Solve the following L.P.P. by using Corner point method:

Max Z = 3x + 2y
s.t. x + 2y ≤ 10;
3x + y ≤ 15
x, y ≥ 0
Answer:

• Step 1: The inequalities are converted into equations.

We obtain two lines:

• L1 with the equation x + 2y = 10. Then there are two obvious points belong to it
A(0,5) and B(10, 0).
• L2 with the equation 3x + y = 15. Then there are two obvious points belong to it
D(0, 15) and E(5,0).
• Step2: Draw the lines.

• Step 3: Determine which side verify the constraint

• Step 4: Calculate the values of the objective function in all corners of the feasible
region and compare them to obtain the optimum.

Point X coordinate (X) Y coordinate (Y) Z=3x + 2y

O 0 0 0
A 0 5 10
C 4 3 18
E 5 0 15
From the table, Max Z=18 for x = 4 and y = 3.

11 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

Example4: Solve the following L.P.P. graphically:

Min and Max Z = x + 2y

S.t. x + 2y ≥ 100
2x - y ≤ 0
2x + y ≤ 200
x, y ≥ 0

Answer of example 4:

Point X coordinate (x) Y coordinate (y) Z= x + 2y

A 0 50 100
C 20 40 100
D 50 100 250
E 0 200 400

Max Z= 400 for x = 0 and y=200 and Min Z=100 for x = 20 and y = 40 or x=0 and y=50.

12 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

Chapter2: Simplex method

Introduction

The Simplex method is an iterative algorithm for efficiently solving LP problems using slack
variables, tableaus, and pivot variables as a means to finding the optimal solution of an
optimization problem. A linear program is a method of achieving the best outcome given a
maximum or minimum equation with linear constraints. Most linear programs can be solved
using software, but the Simplex method is a technique for solving linear programs by hand. To
solve a linear programming model using the Simplex method the following steps are necessary:

● Standard form
● Introducing slack variables
● Creating the tableau
● Pivot variables
● Creating a new tableau
● Checking for optimality
● Identify optimal values
LP model in equation form:

The development of the Simplex method computations is facilated by imposing two

requirements on the constraints of the problem:

1. All constraints (with the exception of the nonnegativity of the variables) are equations
with nonnegative rift-hand side.
2. All the variables are nonnegative.

Converting inequalities into equations with nonnegative right-hand side:

To convert a ≤-inequality to an equation, a nonnegative slack S1 variable is added to the left-

hand side of the constraint.

For example, 6x1+ 4x2 ≤ 24 ---- 6x1+ 4x2 +S1= 24 .

To convert a ≥-inequality to an equation, a nonnegative surplus S2 variable is subtracted to

the left-hand side of the constraint.

For example, x1+ x2 ≥ 800 ---- x1+ x2 –S2= 800 .

If the right-hand side of the resulting equation is nonnegative, we multiply the both sides of
the equation by (-1).

For example, -x1+ x2 ≤ -3 ---- -x1+ x2 +S3= -3  (×(-1))  x1- x2 -S3= 3 .

13 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

Dealing with unrestricted variables:

If the variable x1 is unrestricted, we use the following substitution; x1 = x1 - - x1+ with x1 - and
x1+ are nonnegative variables, in all equations.

Simplex method (maximization)

Example1: Solve the following LPP by simplex method:

Maximize Z = 2x1 + 3x2
Subject to: 2x1+ x2 ≤ 4
x1+ 2x2 ≤ 5
x1, x2 ≥ 0
Answer:
This paragraph breaks down the Simplex method into the above steps and follows the example
linear programming model shown below throughout to find the optimal solution.

First, we start by converting all inequalities into equations by adding slack variables:

2x1+ x2 ≤ 4 - 2x1+ x2 +S1 = 4

x1+ 2x2 ≤ 5  x1+ 2x2 +S2 = 5
We add these slack variables into the objective function with coefficient zero:

Z = 2x1 + 3x2 + 0S1 +0S2 - Z - 2x1 - 3x2 - 0S1 -0S2 =0

Second, we construct the initial tableau of Simplex method as follows:

Z x1 x2 S1 S2 Right-Hand side (RHS)

Z-row Z 1 -2 -3 0 0 0
S1-row S1 0 2 1 1 0 4
S2-row S2 0 1 2 0 1 5

From the tableau: The basic variables are S1 = 4, S2 = 5and the Nonbasic variables x1= x2 = 0.
The value of the objective function is 0.

Third, choose the entering variable by using the optimality condition; the most negative
coefficient in Z-row corresponding to (-3). Thus, the entering variable is x2. And we choose the
leaving variable S2 by using the feasibility condition; the most nonnegative ratio of the right-
hand side of the equations by the coefficients under the entering variable x2.

Z x1 x2 S1 S2 RHS Ratio
Z-row Z 1 -2 -3 0 0 0 ------
S1-row S1 0 2 1 1 0 4 4/1=4
S2-row S2 0 1 2 0 1 5 5/2=2.5

14 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

The next tableau is constructed basing on the Gauss-Jordan row operations. It identifies the
entering variable column as the pivot column and the leaving variable row as the pivot row. The
intersection is called the pivot element.

The Gauss-Jordan computations needed to produce the new basic solution include two types.

1) Pivot row:
a) Replace the leaving variable S2 in the basic column with the entering variable x2.
b) New pivot row = Current pivot row ÷ pivot element
x2-row=S2-row ÷ 2  x2-row = (0, 1, 2, 0, 1, 5)/2= (0, ½, 1,0, 1/2, 5/2)
2) All other rows, including Z
New Row = (current row) - (its pivot column coefficient) × (New pivot row)

New Z-row=Z-row –(-3) ×(x2-row) = (1, -2, -3, 0, 0, 0) – (-3) ×(0, ½, 1, 0, 1/2, 5/2)
New Z-row=(1, -1/2, 0, 0, 3/2, 15/2)
New S1-row= S1-row –(1) ×(x2-row) = (0, 2, 1, 1, 0, 4) – (1) ×(0, ½, 1, 0, 1/2, 5/2)
New S1-row= (0, 3/2, 0, 1, -1/2, 3/2)
We obtain the following tableau:
Z x1 x2 S1 S2 RHS
Z-row Z 1 -1/2 0 0 3/2 15/2
S1-row S1 0 3/2 0 1 -1/2 3/2
x2-row x2 0 1/2 1 0 1/2 5/2
We cannot terminate now because we still have a negative in Z-row. The solution obtained at
this step is S1=3/2, x2=5/2,x1=S2=0, and Z=15/2.
Z x1 x2 S1 S2 RHS Ratio
Z-row Z 1 -1/2 0 0 3/2 15/2 -----
S1-row S1 0 3/2 0 1 -1/2 3/2 (3/2)÷ (3/2)=1
x2-row x2 0 1/2 1 0 1/2 5/2 (5/2)÷ (1/2)=5

The entering variable x1 take the place of the leaving variable S1 and the pivot is 3/2.
We do the following computations:
x1-row=S1-row ÷ (3/2)  x1-row = (0, 3/2, 0, 1, -1/2, 3/2)×(2/3)= (0, 1, 0, 2/3, -1/3, 1)
New Z-row=Z-row –(-1/2) ×(x1-row)
= (1, -1/2, 0, 0, 3/2, 15/2) – (-1/2) × (0, 1, 0, 2/3, -1/3, 1)
New Z-row=(1, 0, 0, 1/3, 8/6=4/3, 16/2=8)
New x2-row= x2-row – (1/2) ×(x1-row) = (0, 1/2, 1, 0, 1/2, 5/2) – (1/2) × (0, 1, 0, 2/3, -1/3, 1)
New x2-row= (0, 0, 1, -1/3, 4/6=2/3, 4/2=2)
Z x1 x2 S1 S2 RHS
Z-row Z 1 0 0 1/3 4/3 8
x1-row x1 0 1 0 2/3 -1/3 1
x2-row x2 0 0 1 -1/3 2/3 2
At the end Max Z= 8 where x1 = 1 and x2 = 2.

15 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

Example2: Solve the following LPP by simplex method:

∶ = 8 + 10 +7

+ 3 + ≤ 10
s.t.: − − 5 − ≥ −8
, , ≥0

Answer:

Step 1: Standard Form

Standard form for maximization problem is the baseline format for all linear programs before
solving for the optimal solution and has two requirements:
(1) All linear constraints must be in a less-than-or-equal-to inequality,
(2) All variables are non-negative.
These requirements can always be satisfied by transforming any given linear program using
basic algebra and substitution. Standard form is necessary because it creates an ideal starting
point for solving the Simplex method as efficiently as possible as well as other methods of
solving optimization problems.

Transforming linear constraints from a ≥ inequality to a ≤ inequality can be by multiplying by

-1 on both sides, the inequality can be changed to ≤ .

−1 × (− −5 − ≥ −8)

+5 + ≤8

Step 2: Determine Slack Variables

Slack variables are additional variables that are introduced into the linear constraints of a linear
program to transform them from inequality constraints to equality constraints. If the model is in
standard form, the slack variables will always have a +1 coefficient. Slack variables are needed
in the constraints to transform them into solvable equalities with one definite answer.

+3 + + = 10
+5 + + =8
, , , , ≥0

Step 3: Setting up the Tableau

A Simplex tableau is used to perform row operations on the linear programming model as well as
to check a solution for optimality. The tableau consists of the coefficient corresponding to the
linear constraint variables and the coefficients of the objective function. In the tableau below,

16 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

the bolded top row of the tableau states what each column represents. The first row represents
the objective function variable coefficients multiply by (-1) and the following two rows represent
the linear constraint variable coefficients from the linear programming model.

−8 − 10 − 7 + 0 + 0 =0
+ 3 + + = 10
+5 + + =8

Z x1 x2 x3 S1 S2 Right Hand side (RHS)

Z-row Z 1 -8 -10 -7 0 0 0
S1-row S1 0 1 3 2 1 0 10
S2-row S2 0 1 5 1 0 1 8

Once the tableau has been completed, the model can be checked for an optimal solution as
shown in Step 4.

Step 4: Check Optimality

The optimal solution of a maximization linear programming model is the values assigned to the
variables in the objective function to give the largest Z value. The optimal solution would exist
on the corner points of the graph of the entire model. To check optimality using the tableau, all
values in the Z- row must contain values greater than or equal to zero. If a value is less than zero,
it means that variable has not reached its optimal value. As seen in the previous tableau, three
negative values exist in Z- row indicating that this solution is not optimal. If a tableau is not
optimal, the next step is to identify the pivot variable to base a new tableau on, as described in
Step 5.

Step 5: Identify the entering Variable

The entering variable is used in row operations to identify which variable will become the unit
value and is a key factor in the conversion of the unit value. The entering variable can be
identified by looking at the Z-row of the tableau and the ratio. Assuming that the solution is not
optimal, pick the smallest negative value in the Z- row. One of the values lying in the column of
this value will be the entering variable. To find the ratio, divide the beta values of the linear
constraints by their corresponding values from the column containing the possible entering
variable. The intersection of the row with the smallest non-negative ratio and the smallest
negative value in the Z- row will become the pivot.

In the example shown below, -10 is the smallest negative in the last row. This will designate the
x2 column to contain the entering variable. Solving for the ratio gives us a value of for the first

17 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

constraint, and a value of for the second constraint. Due to being the smallest non-negative
indicator, the pivot value will be in the second row and have a value of 5.

Z x1 x2 x3 S1 S2 RHS Ratio
Z 1 -8 -10 -7 0 0 0 ×
S1 0 1 3 2 1 0 10 10/3≈ 3.33
S2 0 1 5 1 0 1 8 8/5 ≈ 1.6
Now that the sorting variable has been identified (S2), the new tableau can be created in Step 6
to optimize the variable and find the new possible optimal solution.

Step 6: Create the New Tableau

The new tableau will be used to identify a new possible optimal solution. Now that the pivot
variable has been identified in Step 5, row operations can be performed to optimize the pivot
variable while keeping the rest of the tableau equivalent.

I. To optimize the pivot variable, it will need to be transformed into a unit value (value of
1). To transform the value, multiply the row containing the pivot variable by the
reciprocal of the pivot value. In the example below, the pivot variable is originally 5, so
multiply the entire row by .
Z x1 x2 x3 S1 S2 RHS
Z
S1
x2 0 1/5 1 1/5 0 1/5 8/5

II. After the unit value has been determined, the other values in the column containing the
unit value will become zero. This is because the x2 in the second constraint is being
optimized, which requires x2 in the other equations to be zero.
Z x1 x2 x3 S1 S2 RHS
Z 0
S1 0
x2 0 1/5 1 1/5 0 1/5 8/5
III. In order to keep the tableau equivalent, the other variables not contained in the pivot
column or pivot row must be calculated by using the new pivot values. For each new
value, multiply the negative of the value in the old pivot column by the value in the new
pivot row that corresponds to the value being calculated. Then add this to the old value
from the old tableau to produce the new value for the new tableau.
New tableau value = (Negative value in old tableau pivot column) x (value in new tableau
pivot row) + (Old tableau value)

18 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

Old Tableau:

Z x1 x2 x3 S1 S2 RHS
Z-row Z 1 -8 -10 -7 0 0 0
S1-row S1 0 1 3 2 1 0 10
S2-row S2 0 1 5 1 0 1 8

New Tableau:

Z x1 x2 x3 S1 S2 RHS
New Z-row Z 1 -6 0 -5 0 2 16
New S1-row S1 0 2/5 0 7/5 1 -3/5 26/5
x2-row x2 0 1/5 1 1/5 0 1/5 8/5
New Z-row = (10 × x2-row) + Z-row

New S1-row = ((-3) × x2-row) + S1-row

Numerical examples are provided below to help explain this concept a little better.

Numerical examples:

I. To find the s2 value in S1-row:

New tableau value = (Negative value in old tableau pivot column) * (value in new
tableau pivot row) + (Old tableau value)

New tableau value = (-3) * ( ) + 0 = -

II. To find the x1 variable in Z- row :

New tableau value = (Negative value in old tableau pivot column) * (value in new
tableau pivot row) + (Old tableau value)

New value = (10) * ( ) + -8 = -6

Once the new tableau has been completed, the model can be checked for an optimal solution.

Step 7: Check Optimality

As explained in Step 4, the optimal solution of a maximization linear programming model are the
values assigned to the variables in the objective function to give the largest Z value. Optimality
will need to be checked after each new tableau to see if a new entering variable needs to be
identified. A solution is considered optimal if all values in the Z- row are greater than or equal to
zero. If all values are greater than or equal to zero, the solution is considered optimal and Steps
8 through 11 can be ignored. If negative values exist, the solution is still not optimal and a new
pivot point will need to be determined which is demonstrated in Step 8.

19 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

Step 8: Identify New Pivot Variable

If the solution has been identified as not optimal, a new pivot variable will need to be
determined. The pivot variable was introduced in Step 5 and is used in row operations to
identify which variable will become the unit value and is a key factor in the conversion of the
unit value. The pivot variable can be identified by the intersection of the row with the smallest
non-negative indicator and the smallest negative value in the Z- row.

Z x1 x2 x3 S1 S2 RHS Ratio
Z 1 -6 0 -5 0 2 16 -------
S1 0 2/5 0 7/5 1 -3/5 26/5 (26/5)/(2/5)=13
x2 0 1/5 1 1/5 0 1/5 8/5 (8/5)/(1/5)=8

With the new pivot variable identified, the new tableau can be created in Step 9.

Step 9: Create New Tableau

After the new pivot variable has been identified, a new tableau will need to be created.
Introduced in Step 6, the tableau is used to optimize the pivot variable while keeping the rest of
the tableau equivalent.

I. Make the pivot variable 1 by multiplying the row containing the pivot variable by the
reciprocal of the pivot value. In the tableau below, the pivot value was , so everything is
multiplied by 5.
Z x1 x2 x3 S1 S2 RHS
Z
S1
x1 0 1 5 1 0 1 8

II. Next, make the other values in the column of the pivot variable zero. This is done by
taking the negative of the old value in the pivot column and multiplying it by the new
value in the pivot row. That value is then added to the old value that is being replaced.

Z x1 x2 x3 S1 S2 RHS
Z 1 0 30 1 0 8 64
S1 0 0 -2 1 1 -1 2
x1 0 1 5 1 0 1 8

20 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

Step 10: Check Optimality

Using the new tableau, check for optimality. Explained in Step 4, an optimal solution appears
when all values in the Z- row are greater than or equal to zero. If all values are greater than or
equal to zero, skip to Step 11 because optimality has been reached. If negative values still exist,
repeat steps 8 and 9 until an optimal solution is obtained.

Step 11: Identify Optimal Values

Once the tableau is proven optimal the optimal values can be identified. From the first column
and the last one, we obtain that: Z =64, x1=8, S1=2, and the others variables are equals to zero.
Then, the maximum optimal value is 64 and found at (8, 0, 0) of the objective function.

Example3: Solve the following LPP by simplex method:

Maximize Z = 5x1 + 4x2
Subject to: 6x1+ 4x2 ≤ 24
x1 + 2x2 ≤ 6
-x1 + x2 ≤ 1
0 ≤ x2 ≤ 2
x1 ≥ 0
Answer:
LP form equation:
6x1+ 4x2 ≤ 24  6x1+ 4x2 +S1 = 24
x1 + 2x2 ≤ 6  x1 + 2x2 +S2 =6
-x1 + x2 ≤ 1  -x1 + x2 +S3 = 1
x2 ≤ 2  x2 +S4 = 2
The objective function becomes: Z - 5x1 - 4x2 -0 S1 -0S2 - 0S3 -0S4 = 0

The initial tableau of Simplex method will be:

entering
Basic z X1 X2 S1 S2 S3 S4 RHS Ratio
z 1 -5 -4 0 0 0 0 0 -----
Pivot
leaving S1 0 6 (pivot) 4 1 0 0 0 24 4 Row

S2 0 1 2 0 1 0 0 6 6
S3 0 -1 1 0 0 1 0 1 -ve
S4 0 0 1 0 0 0 1 2 ∞
Pivot column

The second tableau is:

entering
Basic z X1 X2 S1 S2 S3 S4 RHS Ratio
z 1 0 -2/3 5/6 0 0 0 20 -----
x1 0 1 2/3 1/6 0 0 0 4 6
Pivot
leaving S2 0 0 4/3 -1/6 1 0 0 2 6/4=1.5 Row

S3 0 0 5/3 1/6 0 1 0 5 3
S4 0 0 1 0 0 0 1 2 2
Pivot column

21 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

We obtain the optimal tableau:

Basic z X1 X2 S1 S2 S3 S4 RHS
z 1 0 0 3/4 1/2 0 0 21
x1 0 1 0 1/4 -1/2 0 0 3
x2 0 0 1 -1/8 3/4 0 0 3/2
S3 0 0 0 3/8 -5/4 1 0 5/2
S4 0 0 0 1/8 -3/4 0 1 1/2

Based on the optimality condition, none of the z-row coefficients associated with the
nonbasic variables, S1 and S2, are negative. Hence the last tableau is optimal.
Thus, max Z = 21 where x2=3, x2=3/2, S3=5/2, S4=1/2, S1=S2=0
The solution also gives the status of the resources. A resource is designated as scarce if
the corresponding slack variable is zero. Otherwise the resource is abundant. The
following table classifies the constraints of the model:
Resource Value of slack Status
Constraint 1 S1=0 scarce
Constraint 2 S2=0 scarce
Constraint 3 S3=5/2 abundant
Constraint 4 S4=1/2 abundant

Summary of the Simplex Method

So far we have dealt with the maximization case. In minimization problems, the
optimality conditions calls for selecting the entering variable as the nonbasic variable
with the most positive objective coefficient in the objective equation. This follows
because Max Z = - Min ( -Z). As for the feasibility condition for selecting the leaving
variable, the rule remains unchanged.
Optimality condition: the entering variable in maximization (minimization) problem is
the nonbasic variable having the most negative (positive) coefficient in the Z-row. Ties
are broken arbitrarily. The optimum is reached at the iteration where all the Z-row
coefficient of the nonbasic variables are nonnegative (nonpositive).
Feasibility condition: for both the maximization and minimization problems, the leaving
variable is the basic variable associated with the smallest nonnegative ratio. Ties are
broken arbitrarily.
Gauss- Jordan row operations:
1) Pivot row:
a. Replace the leaving variable S2 in the basic column with the entering
variable x2.
b. New pivot row = Current pivot row ÷ pivot element
2) All other rows, including Z
New Row = (current row) - (its pivot column coefficient) × (New pivot row)

22 | P a g e
 Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

The steps of the Simplex method are:

Step1. Determine a starting basic feasible solution.

Step2. Select an entering variable using the optimality condition. Stop if there is no entering
variable; the last solution is optimal. Else, go to step3.
Step3. Select a leaving variable using the feasibility condition.
Step4. Determine the new basic solution by using the appropriate Gauss-Jordan computations.
Go to step2.

Artificial starting solution

We said that the LPs is “ill behaved” if the constraints are (=) or (≥). We will use artificial
variables that play the role of slacks at the first iteration. Two methods are introduced here: the
M-method and two-phase method.

M-method (Big M-method):

The artificial variable Ri is added to form a starting solution and they are assigned high penality
in the objective function:

Given M, a sufficiently large positive value.

−M, in maximization problems

Artificial variable objective coefficient =
+M, in minimization problems

Example 4: Solve the following LPP by using M-method:

Min z = +

+ =
+ ≥
s.t.:
+ ≤
, ≥

Answer:

Using x3 as a surplus variable in the second constraint and x4 as a slack in the third constraint,
and we add R in the first equation and R in the second constraint, the equation form of the
problem is given as:

Min z = + Min z = + +0 + Min z = + +0 + +

+ = + = +
+ ≥ + − = + + =
s.t.: s.t.:
+ ≤ + + = + − + =
s.t.:
, ≥ , , , ≥ + + =
, , , , , ≥

23 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

Using M=100, the starting simplex tableau is given as follows (for convenience, the z-column is
eliminated because it does not change in all the iterations):

basic X1 X2 X3 X4 R1 R2 RHS
z -4 -1 0 0 -100 -100 0
R1 3 1 0 0 1 0 3
R2 4 3 -1 0 0 1 6
X4 1 2 0 1 0 0 4
Before proceeding with the Simplex method computations, we need to make the z-row consistent
with the rest of the tableau.

From the tableau, we z =0 for R1=3, R2=6,X4=4,and X1=X2=X3=0.

But, z =100*3+100*6=900 not 0.

We can eliminate this inconsistency by:

New z-row = Old z-row+ (100 × R1-row + 100 × R2-row)

The modified tableau thus becomes:

basic X1 X2 X3 X4 R1 R2 RHS Ratio

z 696 399 -100 0 0 0 900 ×
R1 3 1 0 0 1 0 3 3/3=1
R2 4 3 -1 0 0 1 6 6/4=1.5
X4 1 2 0 1 0 0 4 4/1=4
Because it is a minimization problem we start by choosing the most positive number in z-row to
obtain the entering variable and we obtain the optimum if the entire coefficient in this row is ≤ 0.
We obtain the following tableau:

basic X1 X2 X3 X4 R1 R2 RHS
z 0 167 -100 0 -232 0 204
x1 1 1/3 0 0 1/3 0 1
R2 0 5/3 -1 0 -4/3 1 2
X4 0 5/3 0 1 -1/3 0 3
The last tableau shows that x2 and R2 are the entering and the leaving variables, respectively.
Continuing with the simplex computations, two more iterations are needed to reach the optimum:
x1=2/5, x2=9/5, and z=17/5.

basic X1 X2 X3 X4 R1 R2 RHS
z -3 0 1/5 0 -98 2/5 -100 1/5 3 3/5 Ratio
X1 1 0 1/5 0 3/5 - 1/5 3/5 3
X2 0 1 - 3/5 0 - 4/5 3/5 1 1/5 -2
X4 0 0 1 1 1 -1 1 1

24 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

basic X1 X2 X3 X4 R1 R2 RHS
z -3 0 0 - 1/5 -98 3/5 -100 3 2/5
X1 1 0 0 - 1/5 2/5 0 2/5
X2 0 1 0 3/5 - 1/5 0 1 4/5
X4 0 0 1 1 1 -1 1

Remark:

If we obtain in the last tableau that an artificial variable has a value not zero then the LP does not
have a feasible solution.

Exercise:

Consider the problem

Maximize Z = 2x1+4x2+4x3-3x4
Subject to: x1 + x2 + x3= 4
x1 + 4x2 + x4 = 8
x1, x2, x3, x4 ≥ 0

The problem shows that x3 and x4 can play the role of slacks for two equations. They differ from
slacks in that they have nonzero coefficients in the objective function. We can use x3 and x4 as
starting variable, but as in the case of artificial variables, they must be substituted out in the
objective function before the simplex iterations are carried out. Solve the problem with x3 and x4
as the starting basic variables and without using any artificial variables.
Two-phase method:

As the name suggests, the method solves the LP in two phases: Phase I attempts to find a starting
basic feasible solution by using always minimize the sum of artificial variables as objective
function, and, if one is found the minimum value of the objective function is zero, Phase II is
invoked to solve the original problem.

We will use the same example 4 given in the M-method.

Phase I: Min r = +

+ + =
+ − + =
s.t.:
+ + =
, , , , , ≥

The associated tableau is given as

25 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

Basic X1 X2 X3 X4 R1 R2 RHS
r 0 0 0 0 -1 -1 0
R1 3 1 0 0 1 0 3
R2 4 3 -1 0 0 1 6
X4 1 2 0 1 0 0 4

As in the M-method, r1 and R2 are substituted out in the r-row by using the following
computations:

New r-row = Old r-row + (1 × R1-row+1 × R2-row)

Basic X1 X2 X3 X4 R1 R2 RHS
r 7 4 -1 0 0 0 9
R1 3 1 0 0 1 0 3
R2 4 3 -1 0 0 1 6
X4 1 2 0 1 0 0 4

Basic X1 X2 X3 X4 R1 R2 RHS
r 0 1 2/3 -1 0 -2 1/3 0 2 ratio
X1 1 1/3 0 0 1/3 0 1 3
R2 0 1 2/3 -1 0 -1 1/3 1 2 1.2
X4 0 1 2/3 0 1 - 1/3 0 3 1.8

The optimum tableau of the first Phase is:

Basic X1 X2 X3 X4 R1 R2 RHS
r 0 0 0 0 -1 -1 0
x1 1 0 1/5 0 3/5 -1/5 3/5
x2 0 1 -3/5 0 -4/5 3/5 6/5
X4 0 0 1 1 1 -1 1
Because minimum r=0, Phase I produces the feasible solution x1=3/5, x2=6/5, and x4=1. At this
point we eliminated the R1 and R2 columns from the tableau and move on to phase II.

Phase II

After deleting the artificial columns, we substitute the r-row by the original objective
function. Min z = 4x1 + x2

26 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

Basic X1 X2 X3 X4 RHS
z -4 -1 0 0 0
x1 1 0 1/5 0 3/5
x2 0 1 -3/5 0 6/5
X4 0 0 1 1 1
Because the variables x1 and x2 have nonzero coefficients in the z-row, they must be
substituted out , using the following computations.

New z-row = Old z-row + (4 × x1-row+1 × x2-row)

The initial tableau of the phase II is thus given as

Basic X1 X2 X3 X4 RHS
z 0 0 1/5 0 18/5
x1 1 0 1/5 0 3/5
x2 0 1 -3/5 0 6/5
X4 0 0 1 1 1
Because we are minimizing x3 must enter the solution. Application of the Simplex
method produces the optimum in one iteration.

Basic X1 X2 X3 X4 RHS
Z 0 0 1/5 0 3 3/5 Ratio
X1 1 0 1/5 0 3/5 3
X2 0 1 - 3/5 0 1 1/5 -2
X4 0 0 1 1 1 1

Basic X1 X2 X3 X4 RHS
Z 0 0 0 - 1/5 3 2/5
X1 1 0 0 - 1/5 2/5
X2 0 1 0 3/5 1 4/5
X4 0 0 1 1 1

27 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

Special cases:

This section considers 4 special cases that arise in the use of the Simplex method:

1. Degeneracy

In the application of the feasibility condition , a tie for the minimum ratio occur and can
be broken arbitrary when this happens at least one basic variable will be zero in the next
iteration and the new solution is said to be degenerate. From the practical standpoint, the
condition reveals that the model has at least one redundant constraint.

Example 5 : max z= 3x1+9x2

+ ≤
s.t.: + ≤
, ≥
Given the slack variables x3 and x4 , the following tableaus provide the simplex
iterations of the problem:

Iteration Basic X1 X2 X3 X4 RHS

0 Z -3 -9 0 0 0
X2 enters X3 1 4 1 0 8
X3 leaves X4 1 2 0 1 4
1 Z -3/4 0 9/4 0 18
X1 enters X2 1/4 1 1/4 1/4 2
X4 leaves X4 1/2 0 -1/2 1 0
2 Z 0 0 3/2 3/2 18
optimum X2 0 1 ½ -1/2 2
X1 1 0 -1 2 0
2. Alternative optima
When the objective function is parallel to a constraint that satisfied as an equation at the
optimal solution (non redundant binding constraint), the objective function can assume
the same optimal value at more than one solution point , thus giving rise to alternative
optima.
Example 6: max z= 2x1+4x2
+ ≤
s.t.: + ≤
, ≥

28 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

The iterations of the model are given by the following tableaus.

Iteration Basic X1 X2 X3 X4 RHS
0 Z -2 -4 0 0 0
X2 enters X3 1 2 1 0 5
X3 leaves X4 1 1 0 1 4
1 Z 0 0 2 0 10
X1 enters X2 1/2 1 1/2 0 5/2
X4 leaves X4 1/2 0 -1/2 1 3/2
2 Z 0 0 2 0 10
Alternative X2 0 1 1 -1 1
optimum X1 1 0 -1 2 3
Iteration 1 gives the optimum solution x1=0, x2=5/2 and z=10 . look at the z-row
coefficients of the nonbasic variables in iteration1. The coefficient of x1 is zero
indicating that x1 can enter the basic solution without changing the value of z.
3. Unbounded solutions
In some LP models the value of the variables may be increased indefinitely without
violating the constraint. That means that the solution space is unbounded.
Example 7: max z= 2x1+3x2
− ≤
s.t.: ≤
, ≥
The initial simplex table is:
Basic X1 X2 X3 X4 RHS
Z -2 -3 0 0 0
X3 1 -1 1 0 10
X4 2 0 0 1 40
We cannot choose the leaving variable.
4. Non existing (or infeasible ) solutions
LP models with inconsistent constraints have no feasible solution. This situation can
never occur if all the constraints are of the type ≤ with non-negative right-hand sides. For
others types of constraints, we use artificial variables if one of them in optimum point has
the nonzero value then the problem has infeasible solution.

Example 8: max z= 3x1+2x2

+ ≤
s.t.: + ≥
, ≥
Using the penalty M=100 for the artificial variable R, the following tableau provide the
simplex iteration of the model.

29 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

iteration basic X1 X2 X3 X4 R RHS

0 z -303 -402 0 100 0 -1200
X2 enters X3 2 1 1 0 0 2
X3 leaves R 3 4 0 -1 1 12
1 z 501 0 402 100 0 -396
Pseudo X2 2 1 1 0 0 2
optimum R -5 0 -4 -1 1 4
Optimum iteration 1 show that the artificial variable R is positive (=4), which indicates
that the problem is infeasible.

Sensitivity analysis
1. Sensitivity analysis, which deals with determining the conditions that will keep the
current solution unchanged.
2. Post-optimal analysis, which deals with finding a new optimal solution when the data of
the model are changed.
In sensitivity Analysis two cases will be considered:
1. Sensitivity of the optimum solution to change in the availability of the resources (right-
hand side of the constraints)
2. Sensitivity of the optimum to changes in unit profit or unit cost (coefficients of the
objective function)

Example9:
A factory produces two products on two machines. A unit of product 1 requires 2 hours on
machine 1 and 1 hour on machine 2. For product 2, a unit requires 1 hour on machine 1 and 3
hours on machine 2. The revenues per unit of product 1 and 2 are 30 SAR and 20 SAR,
respectively. The total daily processing time available for each machine is 8 hours.
Letting x1and x2 represent the daily number of units of products 1 and 2 respectively the LP
model is given as
Maximize Z=30x1 +20 x2
Subject to : 2x1+x2 ≤ 8 (machine1)
x1+3x2 ≤ 8 (machine 2)
x1 and x2 ≥ 0

Change in the right-hand side constraint:

If we increase the right-side of the first constraint from 8 hours to 9 hours we will obtain the
following graph:

30 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

Dual or Shadow price= = 14 /ℎ

The change in the optimal objective value per

unit change in the availability of the resource
represents the unit worth of a resource (in
SAR/hr).

This means that a unit increase (decrease) in

machine 1 capacity will increase (decrease)
revenue by 14 SAR.

The dual price of 14 SAR/hr remains valid for

changes in machine 1 on any point of the line
segment BF. Where B(0,2.67) and F(8,0).
Regarding the constraint 1 , we can conclude
that the dual price of 14 remains valid for the
range : 2.67 ≤ Capacity of machine1 ≤ 16hr

Some calculations can be done for the machine 2.

Dual price =2SAR /hr and the range is on the line segment DE where D(4, 0) and E(0, 8). So
the Dual price remains valid for the range is
4 ≤ capacity of machine 2 ≤ 24
Changes in the Objective Coefficients:
The optimum occurs at point C (x1=3.2, x2=1.6, ZC=128). Changes in revenues units will change
the slope of Z. However, as we can see from the figure the optimum solution will remain at the
point C so long as the objective function lies between line BF and DE, the two constraints that
define the optimum point. This means that there is a range for the coefficients of the objective
function that will keep the optimum solution unchanged at C.

31 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

We can write the objective function in the

general format:

Maximize Z= c1x1+c2 x2.

Imagine now that the line Z is pivoted at C and

that it can rotate clockwise and
counterclockwise. The optimum solution
remains at the point C as long as z=c1 x1+c2 x2
lies between the two lines x1+3x2=8 and
2x1 +x2=8. This means that the ratio c1/c2 can
vary between 1/3 and 2/1 which yield the
following condition: 0.333≤ c1/c2 ≤ 2.

The new objective function is maximize Z= 35 x1 +25 x2 .

The solution at C will remain optimal because c1/c2=35/25=1.4 remains within the optimality
range (0.33,2). When the ratio falls outside this range, additional calculations are needed to find
the new optimum.
Notice that although the values of the variables at the optimum point C remain unchanged, the
optimum value of Z changes to 35*3.2+25*1.6=152.
Algebraic sensitivity Analysis – Changes in the right –hand side
Example 10:
Consider the following LPP:
Max Z= 3x1 +2x2 +5x3
Subject to : x1+ 2x2 +x3 ≤ 430 mn (operation 1)
3x1 + 2x3 ≤460 mn (operation2)
x1+ 4x2 ≤ 420 mn (operation3)
x1, x2, x3 ≥0

Using x4, x5, and x6 as the slack variables for the constraints of operations 1, 2,and 3,
respectively, the optimum tableau is
Basic X1 X2 X3 X4 X5 X6 solution
Z 4 0 0 1 2 0 1350
X2 -1/4 1 0 1/2 -1/4 0 100
X3 3/2 0 1 0 1/2 0 230
X6 2 0 0 -2 1 1 20
Max Z= 1350 when x2=100, x3=230, x6=20, and x1=x4=x5=0
1) Give the shadow or dual price for each constraint.
The Z-row in the optimal tableau yields directly the dual prices, as the following table
shows:

32 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

Resource Slack variable Optimal z-equation coefficient Dual price

of slack variables
Operation1 X4 1 1 SAR/min
Operation2 X5 2 2 SAR/min
Operation 3 X6 0 0 SAR/min
The zero dual price for opeartion3 means that there is no economic advantage in
allocating more production time to this operation.
2) Give the feasibility conditions of any change at right-hand sides of the constraints.
Assume that Di is the change in the right–hand side of the ith constraint.
Basic solution D1 D2 D3
Z 1350 1 2 0
X2 100 1/2 -1/4 0
X3 230 0 1/2 0
X6 20 -2 1 1
From the last tableau, we can easily obtain the feasibility conditions:
X2= 100+(1/2)D1+(-1/4)D2 ≥0
X3=230 +(1/2)D2≥ 0
X6=20+(-2)D1 +D2+ D3 ≥0
3) Give the optimality conditions of any change in the coefficients of decision variables
in the objective function.
Assume that di is the change in the coefficient of the variable xi.
d1 d2 d3 0 0 0
Basic X1 X2 X3 X4 X5 X6 solution
1 Z 4 0 0 1 2 0 1350
d2 X2 -1/4 1 0 1/2 -1/4 0 100
d3 X3 3/2 0 1 0 1/2 0 230
0 X6 2 0 0 -2 1 1 20

The optimality conditions are:

1*4+ (-1/4)d2+ (3/2)d3 +2*0 - d1 ≥0
1+(1/2)d2 ≥0
2 –(1/4)d2 +(1/2)d3 ≥0
4) If we change the right-hand side of the constraints to the values 400, 448, and 430
respectively. What will be the optimal solution?
Since D= new value – old value, then D1=400- 430 =-30, D2= 448-460=-12, and
D3=430-420=10. Substituting these values in the feasibility conditions, we obtain:
X2= 100+ (1/2)(-30)+ (-1/4)*(-12)=103- 15=88 >0
X3=230 + (1/2)(-12)=224 > 0
X6=20+ (-2)(-30) +(-12)+ 10 =78 > 0
The new feasible solution is x2=88, x3=224, and x6=78 with Z= (0)+2(88)+5(224)=1296.

33 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

Notice that optimum objective value can also be computed as Z=1350 + 1(-30) + 2(-
12)=1296.
5) Give the optimal solution if Z is substituted by Z’=2x1 + x2 + 6x3 .
The old objective function was Z= 3x1 +2x2 +5x3
Since d = new value – old value. Then d1= 2- 3=-1, d2=1-2=-1, and d3=6-5=1.
Substitution in the optimality conditions yields
1*4+ (-1/4)d2+ (3/2)d3 +2*0 - d1 =6.75 >0
1+(1/2)d2 =0.5 >0
2 –(1/4)d2 +(1/2)d3 =2.75 >0
The results shows that the proposed changes will keep the current solution (x1=0,
x2=100,x3=230) optimal. Hence no further calculations are needed except that the objective
value will change to Z=1350 +100 d2 +230 d3 =1478. If any of the conditions is not satisfied,
anew solution must be determined by using the Simplex method.
Remark:
The computation above show the dual prices are determined from the optimal tableau for ≤
constraints.
For ≥ constraints, the same idea remains applicable except that the dual price will assume the
opposite sign of that associated with ≤ constraint.
In the case of the constraint is an equation, the determination of the dual price from the optimal
simplex table require somewhat “involved calculation”.
If the change doesn’t satisfy the feasibility or optimality conditions.

34 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

Chapter3: Duality
-With every LP maximization problem, there is an associated minimization problem and vice
versa.
-The original problem is called the primal and the other is called the dual.

Properties of duality
1. If the primal is a maximization problem, the dual is a minimization problem, and vice
versa.
2. An optimal solution to the dual exists only when the primal has an optimal solution, and
vice versa.
3. Both the primal and dual problems have the same optimal value of the objective function.
4. The dual of the dual is the primal.
5. The solution of the dual problem can be obtained from the solution of the primal
problem, and vice versa.
6. The dual variables may assume negative values.

Formulation of the dual

Objective
If the primal is a maximization problem, then the dual is a minimization problem, and vice versa.
Decision variables
For each constraint in the primal (not nonnegativity constraints), there is one decision variable in
the dual.
Objective function
The coefficient of each decision variable in the objective function of the dual is equal to the
right-hand side of the corresponding constraint in the primal.
Constraints
-Before structuring the dual constraints, all primal constraints should be transformed to
equalities.
-For each decision variable in the primal, there is a corresponding constraint in the dual.
-The right-hand side of the dual constraints is the same as the corresponding coefficients of the
objective function in the primal.
-In the dual problem, all constraints should be  in maximization and  in minimization.

Example1: Find the dual problem for the following LPP:

max z= 2x1+4x2 + 4x3

+ + ≤
s.t.: − + =
, , ≥

Answer:
First of all, make the primal problem in the equation form without using the artificial
variable just slack and surplus variables.

35 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

Primal Primal in equation form Dual

variables
max z= 2x1+4x2+ 4x3 max z= 2x1+4x2 + 4x3+ 0x4
s.t.: + + + = y1
+ + ≤ s.t.: − + + = y2
− + = , , , ≥
, , ≥

The dual problem is:

Minimize w=10 y1+ 8 y2
+ ≥
⎧ − ≥
⎪
s.t.: + ≥
⎨ + ≥ →( ≥ ,
⎪
⎩ ,

Example2: Find the dual problem for the following LPP:

Min z= 15x1+12x2
+ ≥
s.t.: − ≤
, ≥
Answer:
Primal Primal in equation form Dual
variables
Min z= 15x1+12x2 Min z= 15x1+12x2+ 0x3+ 0x4
+ ≥ + − + = y1
s.t.: − ≤ s.t.: − + + = y2
, ≥ , , , ≥

The dual problem is:

Maximize w=10 y1+ 5 y2
+ ≤
⎧ − ≤
⎪
s.t.: − ≤
⎨ ≤ →( ≥ , ≤ )
⎪
⎩ ,

36 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

Example3: Find the dual problem for the following LPP:

max z= 5x1+6x2
+ =
− + ≥
s.t.:
+ ≤
, ≥

Answer:
Primal Primal in equation form Dual
variables
Max z= 5x1+6x2 Substitute = −
s.t.: Max z= 15 − +12x2+ 0x3+ 0x4
+ = − + + + = y1
⎧
− + ≥ ⎪− + + − + = y2
+ ≤ s.t.: y3
⎨ − + + + =
, ≥ ⎪
⎩ , , , , ≥

The dual problem is:

= + + = + +
− + ≥ − + =
⎧ − + − ⎧
≥−
⎪ ⎪
+ + ≥ + + ≥
. .: . .:
⎨ − ≥ ⎨ ≤
⎪ ≥ ⎪ ≥
⎩ , , , ⎩ ,

37 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

Chapter4: Transportation Problem

Let Si be the supplier or factory, Di destination or depot and Cij is the cost of transportation from
Si to Dj. If the supply and the demand are equal the problem is called balanced (∑ = ∑ ).

D1 D2 D3 D4 D5 Supply
S1 C11 C12 C13 C14 C15 a1
S2 C21 C22 C23 C24 C25 a2
S3 C31 C32 C33 C34 C35 a3
S4 C41 C42 C43 C44 C45 a4
S5 C51 C52 C53 C54 C55 a5
Demand b1 b2 b3 b4 b5
We are focalizing on:

• North-West Corner Method

• Vogel’s approximation method

• Check the optimality

Example1: The following table is the cost of transportation problem with 3 factories and 4
warehouses:

1. Find the basic feasible solution by the North-West Corner Method (NWCM) and its
cost.
2. Find the basic feasible solution by the Vogel’s approximation method (VAM) and its
cost.
3. Check the optimality of the solution obtained by Vogel’s approximation method.
Answer:
1. We copy the table without the cost of transportation Cij and we start at the cell C11 (North –
West of the table). We calculate the min (a1,b1). Put the results on this cell and subtract it
from the a1 and b1 and write the result.

38 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

W1 W2 W3 W4 Supply
F1 Min(11,6)=6 11-6=5
F2 13
F3 19
Demand 6-6=0 10 12 15
We cannot go vertically, but horizontally is possible from 6

W1 W2 W3 W4 Supply
F1 6 Min(5,10)=5 5-5=0
F2 13
F3 19
Demand 0 10-5=5 12 15

W1 W2 W3 W4 Supply
F1 6 5 0
F2 Min(5,13)=5 13-5=8
F3 19
Demand 0 5-5=0 12 15

W1 W2 W3 W4 Supply
F1 6 5 0
F2 5 Min(8,12)=8 8-8=0
F3 19
Demand 0 0 12-8=4 15

W1 W2 W3 W4 Supply
F1 6 5 0
F2 5 8 0
F3 Min(4,19)=4 19-4=15
Demand 0 0 4 -4=0 15

W1 W2 W3 W4 Supply
F1 6 5 0
F2 5 8 0
F3 4 Min(15,15)=15 15
Demand 0 0 0 15

39 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

We can do it in one table and the basic feasible solution obtained by NWCM is:
x11=6,x12=5,x22=5, x23=8, x33=4, x34=15.
The cost of this solution is: 6×21+5×16 +5× 18+8×14 +4× 18+15×41=1095.

2. The VAM method also takes costs into account in allocation. Five steps are involved in
applying this heuristic:
Step 1: Determine the difference between the lowest two cells in all rows and columns,
including dummies.
Step 2: Identify the row or column with the largest difference. Ties may be broken arbitrarily.
Step 3: Allocate as much as possible to the lowest-cost cell in the row or column with the
highest difference. If two or more differences are equal, allocate as much as possible to the
lowest-cost cell in these rows or columns.
Step 4: Stop the process if all row and column requirements are met. If not, go to the next step.
Step 5: Recalculate the differences between the two lowest cells remaining in all rows and
columns. Any row and column with zero supply or demand should not be used in calculating
further differences. Then go to Step 2.
The Vogel's approximation method (VAM) usually produces an optimal or near- optimal starting
solution. One study found that VAM yields an optimum solution in 80 percent of the sample
problems tested.

We use all values of the table here. We add one column, on the right side, containing the
difference between the two smallest value in the each row of Cij. And add one row on the
bottom of the table containing the difference between the two smallest value of Cij in each
column.

W1 W2 W3 W4 Supply Difference
F1 21 16 25 13 11 16-13-3
F2 17 18 14 23 13 17-14=3
F3 32 27 18 41 19 27-18=9
Demand 6 10 12 15
Difference 21-17=4 18-16=2 18-14=4 23-13=10
The big difference is 10 on w4 column. We choose in this column the smallest Cij
corresponding to 13.

W1 W2 W3 W4 Supply
F1 21 16 25 11// 13 0
F2 17 18 14 23 13
F3 32 27 18 41 19
Demand 6 10 12 4

40 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

W1 W2 W3 W4 Supply Difference
F2 17 18 14 4//23 13-4=9 17-14=3
F3 32 27 18 41 19 27-18=9
Demand 6 10 12 4-4=0
Difference 32-17=15 27-18=9 18-14=4 41-23=18

W1 W2 W3 Supply Difference
F2 6//17 18 14 9-6=3 17-14=3
F3 32 27 18 19 27-18=9
Demand 6-6=0 10 12
Difference 32-17=15 27-18=9 18-14=4

W2 W3 Supply Difference
F2 3//18 14 3-3=0 18-14=4
F3 27 18 19 27-18=9
Demand 10-3=7 12
Difference 27-18=9 18-14=4

W2 W3 Supply Difference
F3 7//27 12// 18 19-7=12 27-18=9
Demand 7-7=0 12
Difference 27 18

The solution is: x14=11, x21=6, x22=3, x24=4, x32=7, x33=12.

W1 W2 W3 W4 Supply
F1 21 16 25 11 // 13 11
F2 6 // 17 3 // 18 14 4 // 23 13
F3 32 7 // 27 12 // 18 41 19
Demand 6 10 12 15

The cost is 11×13+6×17+3×18+4×23+7×27+12×18= 796.

3. 1st step: consider only the nonempty boxes and the row or the column having the greatest
number of solutions.

41 | P a g e
 Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

Consider the second row and put r2 = 0 .

r2 +c1=17 => c1=17 , r2 + c2=18 =>c2= 18 , r2 +c4=23 =>c4=23
r3 + c2=27 => r3= 9, r3 + c3=18 => c3= 9 , r1 + c4=13 => r1= -10
2nd step: Consider only the empty boxes and calculate Eij = ri + cj- Cij
E11=r1+c1-21=-10+17-22 ≤ 0
E12=-10+18-16 ≤ 0
E23=0 + 9-14 ≤ 0
E31=9 + 17 -32 ≤ 0
E34 = 9 + 23 -41 ≤ 0
For all empty boxes, Eij is negative , Then the solution obtained is optimal.

Remark:

In process, if we find one positive Eij for empty boxes. We stop and then the solution is not
optimal.

Assignment problem
The assignment problem is a special type of transportation problem, where the objective is to
minimize the cost or time of completing a number of jobs by a number of persons. In other
words, when the problem involves the allocation of n different facilities to n different tasks.
The assignment problem also encompasses an important sub-class of so-called shortest- (or
longest-) route models. The assignment model is useful in solving problems such as, assignment
of machines to jobs, assignment of salesmen to sales territories, travelling salesman problem, etc.
Hungarian Method
1. Identify the minimum element in each row and subtract it from every element of that row.
2. Identify the minimum element in each column and subtract it from every element of that
column.
3. Make the assignments for the reduced matrix obtained from steps 1 and 2 in the following way:
i. For each row or column with a single zero value cell that has not be assigned or
eliminated, box that zero value as an assigned cell.
ii. For every zero that becomes assigned, cross out (X) all other zeros in the same
row and the same column.

42 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

iii. If for a row and a column, there are two or more zeros and one cannot be
chosen by inspection, then you are at liberty to choose the cell arbitrarily for
assignment.
iv. The above process may be continued until every zero cell is either assigned
or crossed (X).
4. An optimal assignment is found, if the number of assigned cells equals the number of rows
(and columns). In case you have chosen a zero cell arbitrarily, there may be alternate optimal
solutions. If no optimal solution is found, go to step 5.
5. Draw the minimum number of vertical and horizontal lines necessary to cover all the zeros in
the reduced matrix obtained from step 3 by adopting the following procedure:
1. Mark all the rows that do not have assignments.
2. Mark all the columns (not already marked) which have zeros in the marked rows.
iii. Mark all the rows (not already marked) that have assignments in marked
columns.
iv. Repeat steps 5 (i) to (iii) until no more rows or columns can be marked.
v. Draw straight lines through all unmarked rows and marked columns.
You can also draw the minimum number of lines by inspection.
5. Select the smallest element from all the uncovered elements. Subtract this smallest element
from all the uncovered elements and add it to the elements, which lie at the intersection of two
lines. Thus, we obtain another reduced matrix for fresh assignment.
6. Go to step 3 and repeat the procedure until you arrive at an optimal assignment.
Example: The Funny Toys Company has four men available for work on four separate jobs.
Only one man can work on any one job. The cost of assigning each man to each job is given in
the following table.

The objective is to assign men to jobs in such a way that the total cost of assignment is
minimum.
Answer:

43 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

The total cost of assignment = A1 + B4 + C2 + D3

Substituting values from original table: 20 + 17 + 17 + 24 = SAR. 78.
Since the number of assignments is equal to the number of rows (& columns), this is the optimal
solution.

44 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

Chapter5: Integer Linear problem

Integer (or discrete) programming problem is a type of problem in which some, or all, of the
variables are allowed to take only integral values. If all the variables are restricted to take only
integral values (i.e., p = n), the model is called a pure integer programming problem.

To the contrary, if some variables are restricted to take only integer values, and the remaining are
free to take any non-negative values, then it is called a mixed integer programming problem.

When the decision variables are required to take value of either zero or one, it is called zero-one
programming.

Example1:(Hillier and Lieberman)

Solve graphically the following LPP:
Maximize Z = x1 + 5x2
Subject to: x1 + 10x2 ≤ 20
x1 ≤ 2
x1, x2 ≥ 0 and integer.

Answer:

The linear programming solution to this problem yields Z=11 at (2, 1.8).
The first instinct is to round x2 to the nearest integer value, i.e. x2=2. Unfortunately (2, 2) is
infeasible by the first constraint.
The natural inclination is then to try rounding x2 in the opposite direction, i.e. x2=1. This yields a
feasible point (2, 1) with Z=7.
However, this is not the optimum point! The integer-feasible optimum point is at (0, 2) where
Z=10.
Figure shows a sketch of this problem. The dots in Figure show points at which both x1 and x2
have integer values simultaneously.

45 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

Notice that the integer optimum point is far away from the LP optimum. Notice also how you
can’t get to the integer optimum (0,2) by rounding the LP optimum (2, 1.8).

Basically, there are two algorithms to determine the optimal solution for an integer programming
problem. One of these is the cutting plane algorithm devised by Gomory and the other is the
branch & bound algorithm developed by Land & Doig.

Cutting Plane Algorithm

Historically, the first method for solving IPP. was the cutting plane method. This method is for
the pure integer programming model.

The procedure is, first, ignore the integer stipulations, and solve the problem as an ordinary LPP.
If the solution satisfies the integer restrictions, then an optimal solution for the original
problem is found.

Otherwise, at each iteration, additional constraints are added to the original problem. These
constraints are added to reduce or cut the solution space in every successive iteration, ruling out
the current fractional solution, while ensuring that no integer solution is excluded in the process.

The method terminates in a finite number of iterations to an integer-valued.

To summarize the approach, a series of steps are stated below.

Steps

1. Use the simplex method to find an optimal solution of the problem, ignoring the integer
condition.
2. Examine the optimal solution. Terminate the iterations if all the basic variables have
integer values. Otherwise, construct a Gomory's fractional cut from the row, which has
the largest fractional part, and add it to the original set of constraints.
3. Gomory's constraint
-  fixj  -fI
-  fixj + SI = -fI
Where:
fI - fractional part
SI - slack variable
4. In case of a tie, you are at liberty to choose any one arbitrarily.
5. Now, find a new linear programming solution. If the solution thus obtained is integral
valued, then this is the required optimal solution of the original I.P.P.; otherwise, return
to step 2.

46 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

Gomory's Cut Recipe:

Step 1: Express each of the coefficients of the constraint as the sum of an integer and a non-
negative fraction. Note that this representation is unique.
Step 2: Drop all the integer terms and change the = to ≥.
Step3: Multiply by -1
Example
Consider the linear equality constraint
0 x1 - 3/4 x2 + 1/3 x3 - 13/4 x4 = 73/7
Step 1: (0 + 0/1)x1 + (-1 + 1/4)x2 + (0 + 1/3) x3 + (-4 + 3/4)x4 = 10 + 3/7
Step 2: 0/1 x1 + 1/4 x2 + 1/3 x3 + 3/4 x4 ≥ 3/7
Step3: -1/4 x2-1/3x3-3/4x4 ≤-3/7
This is the Gomory's Cut of our constraint.

Branch and Bound Algorithm

The branch and bound method can be used to solve problems containing a few integer valued
variables. It can be applied to both mixed & pure integer programming problems. This
method partitions the area of feasible solution into smaller parts until an optimal solution
is obtained.
The steps of branch and bound method are as follows:
1. Solve the given problem as an ordinary LPP.
2. Examine the optimal solution. Terminate the iteration if the optimal solution
satisfies the integer constraints. Otherwise, go to step3.
3. Divide the problem into two parts:
Problem 1: ≤[ ] Problem 2: ≥ [ ]+
Max Z= Cx Max Z= Cx
s.t. ax ≤ b s.t. ax ≤ b
≤[ ] ≥ [ ]+
x≥0 x≥0

where [t] is the largest integer.

4. Now, solve problem 1 & 2 separately.
5. If for any of the sub-problems, optimal integer solution is obtained, then that
problem is not further branched. Otherwise, move to step 3.

Example:
Maximize z = x1 + x2
subject to
3x1 + 2x2  12
x2  2
x1, x2 are integers  0

The complete branch and bound solution is displayed by means of the following tree like
diagram.

47 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

I
= 8/3
=2 x1≥3
x1≤2 Z=14/3
I
III
II
=3
=2 X2≤1 = 3/2
=2 Z=9/3 X2≥2
Z=4 I
I IV V
= 10/3
x1≤3 =1 x1≥4
Z=13/3
I I
VI VII

=3 =4
=1 =0
Z=4 Z=4
I I
There are three popular policies for node selection:
 Best-first or global-best node selection: choose the bud node that has the best value of the
bounding function anywhere on the branch and bound tree. If we are minimizing, this means
choosing the bud node with the smallest value of the bounding function; if maximizing choose
the bud node with the largest value of the bounding function.
 Depth-first: choose only from among the set of bud nodes just created. Choose the bud node with
the best value of the bounding function. Depth-first node selection takes you one step deeper into
the branch and bound tree at each iteration, so it reaches the leaf nodes quickly. This is one way
of achieving an early incumbent solution. If you cannot proceed any deeper into the tree, back up
one level and choose another child node from that level.
 Breadth-first: expand bud nodes in the same order in which they were created.
Similarly, once a bud node has been chosen for expansion, how do we choose the next variable to use
in creating the child nodes of the bud node? The variable selection policy governs this choice. There
are few standard policies for variable selection. The variables are often selected just in their natural
order, though a good variable selection policy can improve efficiency greatly.

48 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

Chapter6: Nonlinear programming problem

Determinants
To each square matrix, we associate a number, called its determinant, defined as follows:

If =( ) , then det (A)= ,

If = , then det ( ) = − .

For a square matrix A of dimensions n × n, the determinant can be obtained as follows. First,
define as the matrix of dimensions (n-1) ×( n-1) obtained from A by deleting row 1 and
column j. Then

det( ) = det( )− det( )+ det( ) − ⋯ (−1)( )

det( )

Note that, in this formula, the signs alternate between + and -.

For example, if = ,

then det A= ( − )− ( − )+ ( − ).

Determinants have several interesting properties. For example, the following statements are
equivalent for a square matrix A:

 det(A) =0,
 A has no inverse, i.e. A is singular,
 the columns of A form a set of linearly dependent vectors,
 the rows of A form a set of linearly dependent vectors.

For our purpose, however, determinants will be needed mainly in our discussion of classical
optimization, in conjunction with the material from the following section.
4 2 3
Exercise: compute the determinant of A = 1 0 1 .
0 1 2

49 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

Positive Definite Matrices

When we study functions of several variables, we will need the following matrix notions.

A square matrix A is positive definite if > 0 for all nonzero column vectors x. It is
negative definite if < 0 for all nonzero x. It is positive semidefinite if ≥ 0 and
negative semidefinite if ≤ 0 for all x. These definitions are hard to check directly and
you might as well forget them for all practical purposes.

More useful in practice are the following properties, which hold when the matrix A is symmetric
(that will be the case of interest to us), and which are easier to check.

The ith principal minor of A is the matrix formed by the first i rows and columns of A. So, the
first principal minor of A is the matrix = ( ), the second principal minor is the
matrix = , and so on.

 The matrix A is positive definite if all its principal minors 1 , 2 ,…, , have strictly
positive determinants.
 If these determinants are nonzero and alternate in signs, starting with det( 1 ) < 0 , then
the matrix A is negative definite.
 If the determinants are all nonnegative, then the matrix is positive semidefinite,
 If the determinant alternate in signs, starting with det( 1 ) ≤ 0 , then the matrix is
negative semidefinite.

To fix ideas, consider a 2×2 symmetric matrix = .

It is positive definite if: (i) det ( ) = >0
(ii) det ( )= − >0
and negative definite if: (i) det ( ) = <0
(ii) det ( )= − >0
It is positive semidefinite if: (i) det ( ) = ≥0
(ii) det ( )= − ≥0
and negative semidefinite if: (i) det ( ) = ≤0
(ii) det ( )= − ≥0
Exercise: Check whether the following matrices are positive definite, negative definite, positive
semidefinite, negative semidefinite or none of the above.

2 1 −2 4 −2 2 2 4
1) = 2) = 3) = 4) =
1 4 4 −8 2 −4 4 3

50 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

Gradient

Given a function f of n variables , , … , , we define the partial derivative relative to

variable , written as , to be the derivative of f with respect to treating all variables except
as constant.

Example : Compute the partial derivatives of ( , ) = ( − 2) + 2( − 1)

The answer is: ( , ) = 2( − 2) , ( , ) = 4( − 1) .

Let x denote the vector ( , ,…, ) . With this notation, f(x)=f( , ,…, ), ( )=
( )
⎛ ⎞
( , ), etc. The gradient of f at x, written ∇ ( ) , is the vector ⎜ ( )⎟
,…, . The
⎜ ⋮ ⎟
⎝ ( )⎠
gradient vector ∇ ( ) gives the direction of steepest ascent of the function f at point x. The
gradient acts like the derivative in that small changes around a given point ∗ can be estimated
using the gradient. ( ∗ + ∆) ≈ ( ∗ ) + ∆∇ ( ∗ )

Where ∆= (∆ , ∆ , … , ∆ ) denotes the vector of changes.

Example: If ( , )= − + , then f(1, 1)= -1. What about f(1.01, 1.01)?

∗
In this case, = (1, 1) and ∆= (0.01, 0.01). Since ( , )=2 −3 and ( , )=
−1
−32 + 2 , we get ∇ (1, 1) = .
−1

So (1.01, 1.01) = ((1,1) + (0.01, 0.01)) ≈ (1, 1) + (0.01, 0.01)∇ (1, 1)

−1
= −1 + (0.01, 0.01) = −1.02.
−1

Hessian matrix

Second partials ( ) are obtained from f(x) by taking the derivative relative to (this

yields the first partial ( ) and then by taking the derivative of ( ) relative to . So we can

51 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

compute ( ), ( ) and so on. These values are arranged into the Hessian matrix

( ) ( ) ⋯ ( )
⎛ ⎞
( )=⎜ ( ) ( ) ⋯ ( )⎟
⎜ ⋮ ⋮ ⋱ ⋮ ⎟

⎝ ( ) ( ) ⋯ ( )⎠

The Hessian matrix is a symmetric matrix, that is ( )= ( ).

Example : Find the Hessian matrix of ( , )=( − 2) + 2( − 1)

2 0
The answer is H(x) =
0 4

Maximum and Minimum

Optima can occur in three places:

1. at the boundary of the domain,

2. at a nondifferentiable point, or
3. at a point ∗ with ∇ ( ∗ ) = 0.

We will identify the first type of point with Kuhn-Tucker conditions. The second type is found
only by ad hoc methods. The third type of point can be found by solving the gradient equations.

In the remainder of this chapter, we discuss the important case where∇ ( ∗ ) = 0. To identify if
a point ∗ with zero gradient is a local maximum or local minimum, check the Hessian.

 If H(x ∗ )is positive definite then x ∗ is a local minimum.

 If H(x ∗ )is negative definite, then x ∗ is a local maximum.

Example: Find the local extrema of ( , )= − +

∗
This function is everywhere differentiable, so extrema can only occur at points such that
∇ ( ∗) = 0 .

−
∇ ( )=
−

52 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

6 −3
This equals 0 iff ( , ) = (0, 0)or (1,1). The Hessian is ( )=
−3 6

(0, 0) = 0 −3
So,
−3 0

Let denote the first principal minor of H(0,0) and let denote its second principal minor.
Then det ( ) = 0 and det ( ) = −9. Therefore H(0,0) is neither positive nor negative
definite.

(1, 1) = 6 −3
−3 6

Its first principal minor has det ( ) = 6 > 0and its second principal minor has
det ( ) = 36 − 9 = 25 > 0. Therefore H(1,1) is positive definite, which implies that (1,1) is a
local minimum.

Global Optima

Finding global maxima and minima is harder. There is one case that is of interest.

We say that a domain is convex if every line drawn between two points in the domain lies within
the domain.

We say that a function f is convex if the line connecting any two points lies above the function.
That is, for all x,y in the domain and 0 <α <1 , we have f(α x+(1- α )y) ≤ α f(x)+(1- α )f(y).

 If a function is convex on a convex domain, then any local minimum is a global minimum.
 If a function is concave on a convex domain, then any local maximum is a global maximum.

To check that a function is convex on a domain, check that its Hessian matrix H(x) is positive
semidefinite for every point x in the domain. To check that a function is concave, check that it’s
Hessian is negative semidefinite for every point in the domain.

Example: Show that the function ( , , )= +( + ) +( + ) is convex

over .

( )=4 + 2( + ) + 2( + ).
( ) = 2( + ).
( ) = 2( + ).

12 +4 2 2
( , , )= 2 2 0
2 0 2

53 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

The determinants of the principal minors are det ( ) = 12 + 4 > 0 , det ( ) = 12 ≥0

and det( ) = 48 ≥ 0 . So H( , , )is positive semidefinite for all ( , , ) in .
This implies that f is convex over .

Equality Constraints (Lagrangians)

Suppose we have a problem:
Maximize 5 − ( − 2) − 2( − 1)
subject to +4 =3

If we ignore the constraint, we get the solution = 2, = 1 , which is too large for the
constraint. Let us penalize ourselves λ for making the constraint too big. We end up with a
function ( , , λ) = 5 − ( − 2) − 2( − 1) − λ( + 4 − 3)

This function is called the Lagrangian of the problem. The main idea is to adjust λ so that we use
exactly the right amount of the resource.

λ = 0 leads to (2,1).
λ =1 leads to (3/2,0) which uses too little of the resource.
λ =2/3 gives (5/3, 1/3) and the constraint is satisfied exactly.

Given a nonlinear program (P) with equality constraints:

Minimize (or maximize) f(x)

( )=
( )=
subject to
⋮
( )=
a solution can be found using the Lagrangian: ( , λ) = f(x) + ∑ λ (b − g (x))
(Note: this can also be written ( , λ) = f(x) − ∑ λ (g (x) − b )).
Each λ gives the price associated with constraint i.

The reason L is of interest is the following:

Assume ∗ = (x ∗ , x ∗ , … , x ∗ ) maximize or minimize f(x) subject to the constraints g (x) = b for

i=1, 2, …, m. then either

i- The vectors ∇ ( ∗ ), ∇ ( ∗ ), … , ∇ ( ∗ ) are linearly dependent, or

ii- There exists a vector λ∗ = (λ∗ , λ∗ , … , λ∗ ) ℎ ℎ ∇ ( ∗ , λ∗ ) = 0

i.e. ( ∗ , λ∗ ) = ( ∗ , λ∗ ) = ⋯ = ( ∗ , λ∗ ) = 0

And ( ∗ , λ∗ ) = ( ∗ , λ∗ ) = ⋯ = ( ∗ , λ∗ ) = 0

54 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

Of course, Case (i) above cannot occur when there is only one constraint. The following example
shows how it might occur.

Example: Minimize + +

=1
s.t.:
+ =1

It is easy to check directly that the minimum is achieved at ( , , ) = (1, 0, 0) . The

associated Lagrangian is ( , , , λ , λ ) = + + + λ (1 − ) + λ (1 − − )

Observe that (1, 0, 0, λ , λ ) = 1 λ ,λ .

And consequently does not vanish at the optimal solution. The reason for this is the
following. Let ( , , ) = and ( , , ) = + denote the left hand sides of
the constraints. Then ∇ (1,0,0) = (1,0, 0) and ∇ (1,0,0) = (2,0, 0)are linearly dependent
vectors. So Case (i) occurs here!

Nevertheless, Case (i) will not concern us in this course. When solving optimization problems
with equality constraints, we will only look for solutions ∗ that satisfy Case (ii).

Note that the equation ( ∗ , λ∗ ) = 0 is nothing more than g ( ∗ ) = b or g ( ∗ ) − b = 0

In other words, taking the partials with respect to λ does nothing more than return the original
constraints.

Once we have found candidate solutions ∗ , it is not always easy to figure out whether they
correspond to a minimum, a maximum or neither. The following situation is one when we can
conclude. If f(x) is concave and all of the g ( ) are linear, then any feasible ∗ with a
corresponding λ∗ making ∇ ( ∗ , λ∗ ) = 0 maximizes f(x) subject to the constraints. Similarly, if
f(x) is convex and each ( ) is linear, then any ∗ with a λ∗ making ∇ ( ∗ , λ∗ ) = 0 minimizes
f(x) subject to the constraints.

Example: Minimize 2 +
s.t.: + =1

Answer:

( , , λ) = 2 + + λ(1 − − )

( ∗, ∗
, λ∗ ) = 4 ∗
− λ∗ = 0

55 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

( ∗, ∗
, λ∗ ) = 2 ∗
− λ∗ = 0
( ∗, ∗
, λ∗ ) = 1 − ∗
− ∗
=0

Now, the first two equations imply 2 ∗ = ∗ . Substituting into the final equation gives the
solution ∗ = , ∗ = , and λ∗ = , with function value 2/3.

Since f( , ) is convex (its Hessian matrix ( , ) = 4 0 is positive definite) and

0 2
g( , ) = + is a linear function, the above solution minimizes f( , ) subject to the
constraint.

Equality and Inequality Constraints

How do we handle both equality and inequality constraints in (P)? Let (P) be:

Maximize f(x)
( )=
⎧
⋮
⎪
( )=
Subject to ℎ ( ) ≤
⎨
⎪ ⋮
⎩ℎ ( ) ≤

If you have a program with ≥ constraints, convert it into ≤ by multiplying by -1. Also convert a
minimization to a maximization.

The Lagrangian is ( , λ, ) = ( ) + ∑ λ( − ( )) + ∑ ( − ℎ ( ))

The fundamental result is the following:

Assume ∗ = (x ∗ , x ∗ , … , x ∗ ) maximize or minimize f(x) subject to the constraints g (x) = b for

i=1, 2, …, m. and h (x) ≤ d for j =1, 2, …, p. then either

-i- The vectors ∇ ( ∗ ), … , ∇ ( ∗ ), ∇ℎ ( ∗ ), … , ∇ℎ ( ∗ ) are linearly dependent, or

-ii- There exists a vector λ∗ = (λ∗ , λ∗ , … , λ∗ ) μ∗ = μ∗ , μ∗ , … , μ∗

ℎ ℎ ∇ ( ∗ , λ∗ , μ∗ ) = ∇ ( ∗ ) − ∑ λ∗ (∇ ( ∗ )) − ∑ μ∗ ∇ℎ ( ∗ ) = 0
( ∗) =
ℎ ( ∗) ≤
μ∗ ℎ ( ∗ ) − =0 ( )
μ∗ ≥ 0

56 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

In this course, we will not concern ourselves with Case (i). We will only look for candidate
solutions ∗ for which we can find λ∗ and μ∗ satisfying the equations in Case (ii) above.

In general, to solve these equations, you begin with complementarity and note that either μ∗ must
be zero or ℎ ( ∗ ) − = 0 . Based on the various possibilities, you come up with one or more
candidate solutions. If there is an optimal solution, then one of your candidates will be it.

The above conditions are called the Kuhn-Tucker (or Karush-Kuhn-Tucker) conditions.

For ∗ optimal, some of the inequalities will be tight and some not. Those not tight can be
ignored (and will have corresponding price μ∗ = 0. Those that are tight can be treated as
equalities which leads to the previous Lagrangian stuff.

So μ∗ ℎ ( ∗ ) − =0 ( ) forces either the price μ∗ to be 0 or the constraint

to be tight.

Example: Solve the following NLPP:

Max x3 - 3x
S.t. x ≤ 2

The Lagrangian is : ( , ) = x − 3x + μ(2 − x). So, we need :

3x – 3 − μ = 0,
x ≤ 2,
μ(2 − x) = 0,
≥ 0.
Typically, at this point, we must break the analysis into cases depending on the complementarily
conditions.
If = 0, then 3x – 3 = 0 so x=1 or x=-1. Both are feasible. f(1)=-2 and f(-1)=2.
If x=2 then = 9 which is feasible. Since f(2)=2, we have two solutions x = -1 and x=2.

Example: Minimize ( − 2) + 2( − 1)

+4 ≤ 3
subject to
≥

Answer:

First we convert to standard form, to get

Maximize −( − 2) − 2( − 1)
+4 ≤3
Subject to
− + ≤0

The Lagrangian is ( , , μ , μ ) = −( − 2) − 2( − 1) + μ (3 − − 4 ) + μ (x − y)

This gives optimality conditions:

57 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

−2( − 2) − μ + μ = 0
−4( − 1) − 4μ − μ = 0
μ (3 − −4 )=0
μ x − y) = 0
(
+4 ≤3
− + ≤0
μ ,μ ≥ 0
Since, there are two complimentarily conditions, there are four cases to check:
μ = 0, μ = 0: gives x=2, y=1 which is not feasible.
μ = 0, x − y = 0: gives x=4/3, y=4/3, μ = − which is not feasible.
μ = 0, 3 − x − 4y = 0: gives x=5/3, y=1/3, μ = which is feasible.
3 − x − 4y = 0, x − y = 0: gives x=3/5, y=3/5, μ = ,μ = − which is not feasible.

Since it is clear that there is an optimal solution x=5/3, y=1/3.

Economic Interpretation

The economic interpretation is essentially the same as the equality case. If the right hand side of
a constraint is changed by a small amount ∆, then the optimal objective changes by μ∗ ∆, where
μ∗ is the optimal Lagrange multiplier corresponding to that constraint. Note that if the constraint
is not tight then the objective does not change (since then μ∗ = 0).

Handling Nonnegativity

A special type of constraint is nonnegativity. If you have a constraint ≥ 0 , you can write this
as − ≤ 0 and use the above result. This constraint would get a Lagrange multiplier of its own,
and would be treated just like every other constraint.

An alternative is to treat nonnegativity implicitly. If must be nonnegative:

1. Change the equality associated with its partial to a less than or equal to zero:

ℎ
( )− λ ( )− μ ( )≤0

2. Add a new complementarity constraint:

ℎ
( )− λ ( )− μ ( ) =0

3. Don't forget that ≥ 0 for x to be feasible.

58 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

Sufficiency of conditions

The Karush-Kuhn-Tucker conditions give us candidate optimal solutions x ∗ . When are these
conditions sufficient for optimality? That is, given x ∗ with λ∗ and μ∗ satisfying the KKT
conditions, when can we be certain that it is an optimal solution?

The most general condition available is:

1. f(x) is concave, and

2. the feasible region forms a convex region.

While it is straightforward to determine if the objective is concave by computing its Hessian

matrix, it is not so easy to tell if the feasible region is convex. A useful condition is as follows:

The feasible region is convex if all of the ( ) are linear and all of the ℎ ( ) are convex.

If this condition is satisfied, then any point that satisfies the KKT conditions gives a point that
maximizes f(x) subject to the constraints.

Review of Optimality Conditions.

The following reviews what we have learned so far:

Single Variable (Unconstrained)
Solve f'(x) = 0 to get candidate x ∗ .
If " ( ∗ ) > 0 then x ∗ is a local min. " ( ∗ ) < 0 then x ∗ is a local max
If f(x) is convex then a local min is a global min.
f(x) is concave then a local max is a global max.

Multiple Variable (Unconstrained)

Solve ∇ ( ) = 0 to get candidate x ∗ .
If ( ∗ )is positive definite then x ∗ is a local min. ( ∗ )is negative definite x ∗ is a local max.
If f(x) is convex then a local min is a global min.
f(x) is concave then a local max is a global max.
Multiple Variable (Equality constrained) Form Lagrangian
( , λ) = f(x) + ∑ λ (b − g (x))
Solve ∇ ( , λ) = 0 to get candidate x ∗ (and λ∗ ).
Best x ∗is optimum if optimum exists.

Multiple Variable (Equality and Inequality constrained)

Put into standard form (maximize and ≤ constraints)

( , λ, ) = ( ) + λ( − ( )) + ( − ℎ ( ))

Form Lagrangian
Solve ∇ ( ∗ , λ∗ , μ∗ ) = ∇ ( ∗ ) − ∑ λ∗ (∇ ( ∗ )) − ∑ μ∗ ∇ℎ ( ∗ ) = 0

59 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

( ∗) =
ℎ ( ∗) ≤
μ∗ ℎ ( ∗ ) − =0 ( )
μ∗ ≥ 0
to get candidates x ∗ (and λ∗ , μ∗ ).
Best x ∗ is optimum if optimum exists.
Any x ∗ is optimum if f(x) concave, ( ) convex, ℎ ( ) linear.

Constrained optimization problem

Optimization problem with one equality constraint:

Lagrangian multiplier method

Max or Min ( , )  Max or Min ( , )

s.t.: ( , ) = s.t.: ( , ) =

Lagrangian objective function is : ( , , λ) = ( , )−λ ( , )

The necessary conditions for maximum or minimum are:

= <=> −λ =

= <=> −λ =

= <=> −ℎ = 0
λ
The condition will be sufficient if f is concave for maximization problem or if f is convex for
minimization problem.

Definition of concave/convex two-variable functions:

Theroem:

" " " " "

f is concave  ≤ , ≤ − ≥ .

" " " " "

f is convex  ≥ , ≥ − ≥ .

Example :
Solve the Non Linear Programming Problem (NLPP):
Max Z = 4x1+6 x2 -2 x12-2x1x2-2x22
s.t. x1+x2=2
x1, x2 ≥ 0

60 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

Answer:
The Lagrangian function will be:
L(x1, x2, λ) = 4x1+6 x2 -2 x12-2x1x2-2x22 - λ(x1+x2 - 2)

The necessary conditions for maximum or minimum are:

= 0 <=> 4 − 4 −2 − λ = 0 <=> −4 −2 − λ = −4

= 0 <=> 6 − 2 −4 − λ = 0 <=> −2 −4 − λ = −6

= 0 <=> −( + − 2) = 0 <=> + =2
λ

Use any method to solve the above linear system.

You will obtain = , = , λ = −1
∗
( , ) is candidate to be maximum of f. for this raison, f must be concave.
" (4 − 4 "
= − 2 ) = −4, = (6 − 2 − 4 ) = −4,

" (4 − 4 "
= − 2 ) = −2 = .

Since " = −4 < 0, " = −4 < 0 , " "

− " = 16 − 4 = 12 > 0 . Then f is
concave. ∗ is a maximum of f and Zmax=f( ∗
)=9/2=4.5.

Optimization problem with one inequality constraint:

Maximization case Minimization case

( , )  ( , ) ( , )  ( , )
s.t. ( , ) ≤ s.t. ℎ( , ) ≤ 0 s.t. ( , ) ≥ s.t. ℎ( , ) ≥ 0

Khun-tucker conditions: Khun-tucker conditions:

(i) − λℎ = 0 (i) − λℎ = 0
(ii) λh(X) = 0 (ii) λh(X) = 0
(iii) h(X) ≤ 0 (iii) h(X) ≥ 0
(iv) λ ≥ 0 (iv) λ ≥ 0

Remark:
For single constraint non-linear programming problem, the Kuhn-Tucker conditions are also
sufficient conditions for
• The maximization problem, when f(X) is concave and h(X) is convex.
• The minimization problem , when both f(X) and h(X) are concave.

61 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

Example: Solve the NLPP: Maximize Z= 4x1 - x13 + 2x2

S.t. x1+x2 ≤ 1
x1, x2 ≥ 0
3
We put f(X)= 4x1 - x1 + 2x2, and h(X)= x1+x2 - 1
Khun-tucker maximization conditions are:
(i) −λ =  4 - 2x12 – λ=0
(i’) −λ =  2– λ=0
(ii) λh(X) = 0  λ(x1+x2 – 1)=0
(iii) h(X) ≤ 0  x1+x2 ≤ 1
(iv) λ ≥ 0
From (i') λ = 2, therefore from (ii) x1+x2 = 1. These results satisfy the conditions (iii) and (iv).

Solution of (i), (i’), and (ii) yields x = ≈ 0.8165 and x = 1 − ≈ 0.1835.

It’s easy to show that f(X) is concave and h(X) is convex

Example: Maximize f(x, y) = x y subject to x2+y2 ≤ 1.

Answer: The Lagrangian is : L =xy –λ(x2+y2 - 1)

The necessary conditions are: Lx=0  y=2λx, Ly = 0  x = 2λy, Complementary λ(x2+y2 - 1) =

0 and λ≥0 and x2+y2 ≤ 1.

Solutions satisfying theses equalities and inequalities are(a) ,x=y=λ=0, (b) λ=1/2, x=y=1/√2, and (c)
λ=1/2, x=y=-1/√2.

In solution (a) the constraint is not binding and the solution is a saddle point not a maximum.

In both solution (b) and (c), the constraint is binding because λ≠0, and so the constraint

62 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

Exercises:

Q1. Consider the following L.P.:

=− + +

− + + ≤
Subject to: + + ≤
, , ≥

The final table of the Simplex method is:

X1 X2 X3 S1 S2 RHS
Z 0 0 2 5 0 100
X2 -1 1 3 1 0 20
S2 16 0 -2 -4 1 10
a. What are the dual prices (shadow prices) for each constraint?
b. Give the feasibility conditions of any change at the right hand sides of the constraints.
c. Give the optimality conditions of any change in the coefficients of decision variables in the objective
function.
d. If we change the RHS of the first constraint from the value 20 to 22 what will be the optimal
solution.
e. Give the optimal solution if Z is substituted by Z’=− + +
Q2. Write the dual of the following LPP

Maximize =3 +5

Subject to constraints 5 − ≤4
3 −6 ≥ −7
− + 7 ≥ −5
−4 + 5 ≤ 12
, ≥0
Q3. We consider the following transportation problem:

Distribution Centers
Sources Supply
6 8 4 14

63 | P a g e
Dr. S. Bourazza Math-473 Jazan University Department of Mathematics

4 9 8 12
1 2 6 5
Demand 6 10 15
a) Determine the basic feasible solution by North- West Corner Method.
b) Determine the basic feasible solution by Vogel’s approximation method.
c) Check the optimality of the solution obtained by Vogel’s approximation method.

Q4. The assignment cost of assigning any one operator to any one machine is given in the following
matrix

Operators
I II III IV
A 10 24 30 15
Machines

B 16 22 28 12
C 12 20 32 10
D 9 26 34 16
Find the optimal solution by Hungarian Method.

Q5. Determine whether the function is concave, convex or neither:

( , )= + + −
Q6. Solve the NLPP by using Lagrangian multiplier method.

Maximize = + −( − )

Subject to: + =

, ≥ 0

64 | P a g e