Simplex Algorithm

Introduction
The Simplex algorithm or Simplex method is a commonly used algorithm to solve Linear
Programming (LP) optimization problems. The Simplex algorithm was first proposed by George
Dantzig.

Overview
The Simplex algorithm operates on linear programs in the standard form:
𝑚𝑎𝑥𝑖𝑚𝑖𝑧𝑒 𝑐 𝑇 𝑥
𝑠𝑢𝑏𝑗𝑒𝑐𝑡 𝑡𝑜 𝐴𝑥 ≤ 𝑏 𝑎𝑛𝑑 𝑥 > 0
With 𝑐 = (𝑐1 , … , 𝑐𝑛 ) being the coefficients of the objective function, and 𝑥 = (𝑥1 , … , 𝑥𝑛 ) being the
variables of the problem, 𝐴 is a 𝑝 × 𝑛 matrix, and 𝑏 = (𝑏1 , … , 𝑏𝑝 ).

Theorem 1: the feasible region defined by all values of 𝑥 such that 𝐴𝑥 ≤ 𝑏 and ∀𝑖, 𝑥𝑖 ≥ 0 is a convex
set. Therefore if a LP has an optimal solution, there must be an extreme point of the feasible region
that is optimal.
Theorem 2: for a LP optimization problem there is only one point of the LP’s feasible region
regarding every feasible solution, and for every extreme point in the feasible region there will be a
minimum of one corresponding feasible solution.

Based on the two theorems above, the geometric

illustration of the LP problem could be depicted.
Each line/edge of the polyhedral is a boundary for one
of the LP constraints, with each vertex being an
extreme point according to the theorems

The Simplex algorithm starts at any extreme point,

and travels along the edges of the feasible area until
the optimal solution is found, or it reaches an
unbounded infinite edge on which the objective
function is unbounded above.
Algorithm
Consider the following expression as the general linear programming problem standard form:
𝑛

max ∑ 𝑐𝑖 𝑥𝑖
𝑖=1

With the constraints:

𝑛

∑ 𝑎𝑖𝑗 𝑥𝑗 ≤ 𝑏𝑖 𝑖 = 1,2, … , 𝑚
𝑗=1
𝑥𝑗 ≥ 0 𝑗 = 1,2, … , 𝑛
The first step is to add slack variables and symbols which represent the objective functions:
𝑛

𝜙 = ∑ 𝑐𝑖 𝑥𝑖
𝑖=1
𝑛

𝑧𝑖 = 𝑏𝑖 − ∑ 𝑎𝑖𝑗 𝑥𝑗 𝑖 = 1,2, … , 𝑚
𝑗=1

The slack variables 𝑧𝑖 are added to end of the list of x-variables:

𝑛

𝜙 = ∑ 𝑐𝑖 𝑥𝑖
𝑖=1
𝑛

𝑥𝑖+𝑛 = 𝑏𝑖 − ∑ 𝑎𝑖𝑗 𝑥𝑗 𝑖 = 1,2, … , 𝑚

𝑗=1

The above functions define the starting dictionary. As the algorithm progresses (moves between the
different vertices), the dictionary will be defined by:
𝑛

𝜙 = 𝜙̅ + ∑ 𝑐𝑖 𝑥𝑖
𝑖=1
𝑛

𝑥𝑖 = 𝑏̅𝑖 − ∑ ̅̅̅̅𝑥
𝑎𝑖𝑗 𝑗 𝑖 = 1,2, … , 𝑛 + 𝑚
𝑗=1

The variables with bar mean that their corresponding values will change accordingly with the
progression of the simplex method.
For these variables one will be selected to be calculated (entering variable), while another will be set
to 0 (leaving variable). The entering variables are selected from the set {1,2, . . , 𝑛}, with the goal of
increasing 𝜙.
Once the entering variables are determined, the corresponding leaving variables will change
accordingly from the equation below:

𝑥𝑖 = 𝑏̅𝑖 − ̅̅̅̅
𝑎𝑖𝑘 𝑥𝑘 𝑖 ∈ {1,2, … , 𝑛 + 𝑚}
Since the entering variables must be non-negative, we define the following inequality

𝑏̅𝑖 − 𝑎̅𝑖 𝑥𝑘 ≥ 0 𝑖 ∈ {1,2, … , 𝑛 + 𝑚}

The minimum 𝑥𝑖 should be 0 to get the minimum value:
̅̅̅̅
𝑏𝑖
𝑥𝑘 =
𝑎𝑖𝑘
̅̅̅̅

the value of 𝑥𝑘 should be raised to the largest of all of those values calculated from above equation:
̅̅̅̅
𝑏𝑖
𝑥𝑘 = min 𝑖 = 1,2, … , 𝑛 + 𝑚
𝑎𝑖𝑘
̅̅̅̅̅ 𝑎𝑖𝑘
̅̅̅̅
Once the leaving and entering variables are chosen, reasonable row operation should be conducted to
switch from the current dictionary to the new dictionary, as this step is called pivot.
The entering variable's coefficient in its row becomes 1 by multiplying the entire row by its
reciprocal. And the other rows are set to 0 using suitable row operations.
The pivot process is repeated till there are no more negative values in the dictionary. The optimal
solution is found.

Example
𝑚𝑎𝑥𝑖𝑚𝑖𝑧𝑒 4𝑥1 + 𝑥2 + 4𝑥3
𝑠𝑢𝑏𝑗𝑒𝑐𝑡 𝑡𝑜
2𝑥1 + 𝑥2 + 𝑥3 ≤ 2
𝑥1 + 2𝑥2 + 3𝑥3 ≤ 4
2𝑥1 + 2𝑥2 + 𝑥3 ≤ 8
𝑥1 , 𝑥2 , 𝑥3 ≥ 0
Adding slack variables
𝑧 − 4𝑥1 + 𝑥2 + 4𝑥3 = 0
2𝑥1 + 𝑥2 + 𝑥3 + 𝑠1 = 2
𝑥1 + 2𝑥2 + 3𝑥3 + 𝑠1 = 4
2𝑥1 + 2𝑥2 + 𝑥3 + 𝑠2 = 8
𝑥1 , 𝑥2 , 𝑥3 , 𝑠1 , 𝑠2 , 𝑠3 ≥ 0
Starting simplex table/dictionary

𝑥1 𝑥2 𝑥3 𝑠1 𝑠2 𝑠3 𝑧 𝑏
2 1 1 1 0 0 0 2
1 2 3 0 1 0 0 4
2 2 1 0 0 1 0 8
−4 −1 −4 0 0 0 1 0
From the last row the column with the smallest value is selected, and pivot process is
conducted
𝑏𝑖
𝑏̅𝑖 =
𝑥1

𝑥1 𝑥2 𝑥3 𝑠1 𝑠2 𝑠3 𝑧 𝑏̅
1 0.5 0.5 0.5 0 0 0 1
1 2 3 0 1 0 0 4
2 2 1 0 0 1 0 8
−4 −1 −4 0 0 0 1 0

Preform row operations

𝑥1 𝑥2 𝑥3 𝑠1 𝑠2 𝑠3 𝑧 𝑏
1 0.5 0.5 0.5 0 0 0 1
0 1.5 2.5 −0.5 1 0 0 3
0 1 0 −1 0 1 0 6
0 1 −2 2 0 0 1 4

Negative value in last row so process is preformed again

𝑏𝑖
𝑏̅𝑖 =
𝑥3

𝑥1 𝑥2 𝑥3 𝑠1 𝑠2 𝑠3 𝑧 𝑏
1 0.5 0.5 0.5 0 0 0 1
0 0.6 1 −0.2 0.4 0 0 1.2
0 1 0 −1 0 1 0 6
0 1 −2 2 0 0 1 4

Performing row operations to make columns 0

𝑥1 𝑥2 𝑥3 𝑠1 𝑠2 𝑠3 𝑧 𝑏
1 0.2 0 0.6 −0.2 0 0 0.4
0 0.6 1 −0.2 0.4 0 0 1.2
0 −0.1 0 0.2 0.6 −1 0 −4.2
0 2.2 0 1.6 0.8 0 1 6.4

No negative values in last row. Optimal solution is found

𝑥1 = 0.4
𝑥2 = 0
𝑥3 = 1.2
Implementation using python
Import numpy as np
Import scipy as sp
From scipy.optimize import linprog
# coefficents
c = [-8,-12,-22]
A = [17, 27, 34], [12, 21, 15]]
b = [91800, 42000]
# bounds
R = (0, None)
T = (0, None)
M = (0, None)
# execution
res = linprog(c, A_ub=A, b_ub=b, bounds=(R, T, M), method="simplex",
options={"disp": True})
# print result
print(res)