Linear Programming (In a Nutshell)

Prof. Yogesh Agarwal

Indian Institute of Management, Lucknow

Solving Linear Equations
A linear program is simply a bunch of linear algebraic equations. In order to solve an LP,
we must brush up our knowledge of solving such equations. Right from our childhood we
are taught to solve two equations with two unknowns, e.g. x + y = 10 and x – y = 4. In
order to solve such equations, we simply manipulate or combine them to form new
equations, which involve only one variable on the left hand side. Then the right hand side
gives the value of that variable. In the present example, if we add the two equations, we
get 2x = 14, i.e. x = 7. Substituting this in either equation yields y=3. The technical name
for this procedure is Gaussian elimination, which we shall formally learn a little later.

Does a given systems of equations always have one and only one solution? Can the
number of solutions be more than one? Does there exist any solution at all? What are the
possibilities? Let us examine all possible cases. Suppose we have m equations in n
unknowns. From linear algebra, we know that there are three possible cases:

Case 1: (m = n)
In this case there is a unique solution. However there is a little catch that we must be
aware of. The equations must be independent of each other. In other words, it should not
be possible to write any one equation as a combination of other equations. For example,
suppose we have two equations: x + y = 10 and 2x + 2y = 20. Can we solve them to get a
unique solution? No, because we really don’t have two independent equations. The
second equation is simply two times the first one. We call such an equation a redundant
equation, because it adds no value.

Although, right from our childhood, we have been most familiar with this case, it is not
of much interest in the context of LP. The reason is obvious. If there is only one solution,
it is meaningless to try to maximize or minimize. The issue of optimization arises only
when there is choice of solutions available.

Case 2: (m > n)
When the number of equations is more than the number of unknowns, first we have to
check if any equation can be written as a combination of other equations. If there is any
such equation, we better remove it, because we know that it does not add any value.
Having done that, if the number of equations still exceeds the number of variables, the
equations have no solution at all.

To see a simple case of three equations in two unknowns, consider: x + y = 10, x – y =

4, and 2x + y = 15. We already know that the first two equations have a unique
solution (x=7, y=3). Since it does not satisfy the third equation, clearly no solution is
possible.

Now suppose the third equation had 13 on the RHS rather than 17. The values x=7,y=3
would indeed satisfy the third equation. But then the third equation is simply a
combination of first two equations (multiply the first equation by two and then subtract
the second equation from it), and is redundant.

2
We can summarize this case as follows. If m > n, either there is a unique solution with
(m-n) equations being redundant, or there is no solution at all. Once again, this case too is
not of much interest for LP.

Case 3: (m < n)
This last case is the only one that is relevant in the context of LP. For such a case, there
can be an infinite number of possible solutions. For example, consider one equation with
two unknowns: x + y = 10. What is the solution? Some of us might jump to the
conclusion and say that it cannot be solved. But that is not correct. What you really mean
to say is that there is no unique solution. In fact, there are an infinite number of solutions.
Pick any value of x, and there is a corresponding value of y that satisfies the equations,
e.g. (x=1, y=9), (x=2, y=8) etc. Even (x=11, y = -1) is valid solution, though we may not
be interested in it if we want only non-negative solutions.

Graphical Interpretation of the Three Cases

In graphical method, we plotted each LP constraints as a straight line of a graph. With
two variables, each equation can be plotted as a straight line on the graph. This gives us a
useful graphical insight into the three cases discussed above. Since, this is a vital insight,
let us dwell on it at some length.

Take the case of m = n, i.e. two equations in two variables. Each equation is a straight
line on the graph. Which point on the graph satisfies both the equations? Clearly, it is the
point at the intersection of two lines. Since two straight lines can intersect only at a single
point, the solution must be unique. But what happens when the equations are not
independent? For example, in the case of equations: x + y = 10 and 2x + 2y = 20. It is
easy to see, that the two lines in this case are parallel to each other, and do not intersect.

To understand the second case of m > n, let us add a third equation to the original two.
The line corresponding to the third equation will either pass through the intersection point
of first two lines, or it will not. In the former case, the intersection point solves all three
equations, but then the third equation is redundant. It is always possible to write it as a
combination of other two equations. On the other hand, in the latter case, i.e. when the
third line does not pass through the intersection point of the first two, there is no single
point that lies at all three lines. Therefore, no solution exists. The equations are said to be
inconsistent in this case.

In the last case of m < n, consider one equation in two variables. We have only a single
line on the graph. Which point on the graph satisfied the equation? The answer is: every
point on the line. Hence, there are an infinite number of solutions in this case.

This last case can be understood more meaningfully in the case of three variables and two
equations. The graph of an equation in three variables cannot be drawn on a piece of
paper. In fact, it is a plane in a three dimensional space; something like the wall or ceiling
of a room. With two equations, we have two such planes. Which point(s) satisfy both the
equations? It is those points that lie at the intersection of the two planes. Two planes

3
always intersect along a straight line, like two walls of a room intersect along a vertical
line at the corner of the room. Each point on this line is a solution.

Note that in the above picture, if we add one more equation, i.e. a third plane, it will
intersect the line at a single point, which is the unique solution to the three equations in
three unknowns. What happens when the third equations is a linear combination of the
first two? In this case, the line of intersection of the two planes is actually contained in
the third plane, and there is no unique solution.

Basic Solution
Let us focus our attention on Case 3, i.e. m < n. If there are an infinite number of
solutions, let’s make a beginning by finding at least one of them. How do we do this?
Recall that a unique solution exists if m=n. But in this case n >m, i.e. we have (n-m)
excess variables than the equations. If we can find a way to make these variables
somehow disappear from the scene, we are left with the case of m=n, which can be
solved by Gaussian elimination to get a unique solution.

Well, an easy way to make these excess variables disappear is to assign them some
specific numerical value, for example zero. Let us illustrate this with the two equations in
three variables given below.

x1 – x2 + x3 = 3
x1 + x2 – 2x3 = 10

Note that in this case m=2, n=3, and n-m=1 is the number of excess variables. So let us
pick any one variable, say x3 and fix its value to zero. This results in following two
equations in two unknowns.

x1 – x2 = 3
x1 + x2 = 10

The solution, as we already know, is x 1 = 7 and x2 = 3. But while reporting this solution,
let us not forget the variable x 3 whose value we have fixed to zero. The complete solution
is: x1=7, x2=3, x3 = 0. Alternatively, we can put this solution in a vector form, i.e. (7, 3,
0), and refer to it as the solution vector. A solution obtained in this manner is called a
basic solution.

For a system of m equations and n variables with m < n, Basic Solution is a solution
obtained by fixing any (n-m) variables to value zero, and solving uniquely for the
remaining variables.

Basic and Non-basic variables

In a basic solution, we have two types of variables. Those we deliberately fixed to value
zero, and those for we solve uniquely. The (n-m) variables fixed to value zero are called
non-basic variables, and m variables whose values we obtain by solving the m equations

4
are called basic variables. In the above example, x 3 is the non-basic variable, while x 1 and
x2 are basic variables.

Note that we can also obtain a solution by fixing the (n-m) variables to any set of values,
not necessarily zero. For example, if we fix x 3=1 in the above example, we get x 1 = 7 and
x2=5 i.e. the solution vector will be: (7,5,1). However, such a solution is NOT a basic
solution. In a basic solution, the values of all non-basic variables must be set to value
zero. We shall see later that for solving the LP problem, we are only interested in the
basic solutions.

Number of Basic Solutions

One question that we have not addressed is: which particular set of (n-m) variables
should be selected to fix to value zero. The answer is that you may select any subset you
like, and each one will give you a potentially different basic solution. A natural question
is: how many such basic solutions are possible? We can select (n-m) variables out of n in
n
Cn-m different ways, which, by the way, is same as nCm. Thus, there could be as many as
n
Cm basic solutions.

We may note that nCm represents an upper limit on the number of basic solution, because
not all combinations may lead to a valid solution. For example, in some cases, after
setting the selected variables to value zero, the resulting m equations may not be
independent of each other, and a unique solution may not exist.

Basic Feasible Solution

After setting the non-basic variables to value zero, when we solve the resulting equations
for the value of basic variables, the values of these variables may turn out to be either
positive or negative (or even zero). Having decided the set of non-basic variables, we
have no control on the values of basic variables. However, in the context of LP, if one or
more values of basic variables turn out to be negative, the solution is not feasible for the
given LP, as it does not satisfy the non-negativity constraints. Such a solution is basic,
but not feasible.

On the other hand, a basic solution in which the values of all variables are non-negative is
clearly feasible for the LP, and is called a Basic Feasible Solution, or bfs.

Corner Points and Basic Feasible Solutions

What is the relevance of basic feasible solution in solving LP? We know from our
experience of graphical method that the optimal solution to an LP always occurs at one of
the corner points of the feasible region. The relevance of basic solutions lies in the fact
that each basic feasible solution corresponds to one of the corner points. Since the
optimal solution always occurs at a corner point, we can find the same by confining our
search to basic feasible solutions, which are finite in number.

It can be mathematically proved that a bfs corresponds to a corner point solution. But
here, rather than providing the mathematical proof, we demonstrate this fact with an
example to help us understand it.

5
Consider the following constraint set of an LP.

-x1 + x2  5 (I)
x1 + x2  10 (II)
x1 – x2  5 (III)

This is an LP with 2 variables and 3 constraints. However, since the constraints are
inequality constraints, we must add three slack variables, say s1, s2 and s3 to constraints
(I), (II) and (III) respectively. This will result in a system of 3 equations with 5 variables,
i.e. m=3 and n=5. A basic solution to this systems equation will have three basic and 3
non-basic variables.

The diagram below shows the graph of the three constraints. Each constraint line is
appropriately labeled, and all the corner points of the feasible regions are labeled from (1)
to (5).

x2 3

II
2

III
4

1 5 6 III
x1

At extreme point 1, the value of x 1 and x2 is zero. But what are the values of s1, s2 and s3?
Recall that the value of a slack variable is zero at the line corresponding to that
constraint, and strictly positive on the feasible side of the line. Since corner point 1 does
not lie on any one of the three constraint lines, clearly, the values of s 1, s2 and s3 are
strictly positive. The solution vectors of this corner point may be written as: (0, 0, x, x,
x), where x represents any strictly positive value. [Note that the variables in the solution
vector are in the following order: (x 1, x2, s1, s2, s3)].

6
Does this solution vector represent a basic solution? Since we have m=3 and n=5, in a
basic solution we should have (n-m), i.e. 2, non-basic variables at value zero, and m=3
variables at positive values. This is indeed the case here. So, clearly this corner point
corresponds to a basic solution.

The solution vectors for the remaining four corner points are of the following form:

Corner point Solution vector Non-basic Variables Basic Variables

2 (0, x, 0, x, x) x1, s1 x2, s2, s3
3 (x, x, 0, 0, x) s1 , s2 x1, x2, s3
4 (x, x, x, 0, 0) s2 , s3 x1, x2, s1
5 (x, 0, x, x, 0) x2, s3 x1, s1, s2

Thus we can see that at each corner point exactly two variables are at value zero, and the
remaining three have positive values. Hence each corner point is represented by a basic
solution. Since we know that the optimal solution must occur at one of the corner points,
we only need to search the basic solutions.

Why does a corner point correspond to a basic solution? Note that a corner point is
characterized by the intersection of two constraint lines. If it is a non-negativity
constraint, for any point lying on that line the value of the corresponding variable is zero.
For example, value of x 1 is zero at every point on the x-axis. If it is a functional
constraint, at each point of the line, the value of the corresponding slack variable is zero.
Hence at the intersection point of two lines, precise two variables must be zero. These
two are the non-basic variables, and the remaining are basic variables.

Are these five points the only intersection points of constraint lines? We can choose two
lines out of five in 5C2 = 10 different ways, so there can be as many as 10 intersection
points? What about the remaining five points? For example, look at the intersection point
of x-axis and constraint II, which is labeled “6” in the figure. It is outside the feasible
region, and hence not feasible. Well, any point at the intersection to two lines, is a basic
solution, but not all of them are feasible. Point 6 is not feasible, because the value of s3
will be negative for this point, because it lies on the infeasible side of constraint III. See if
you can locate the remaining four infeasible intersection points in the diagram. Also
identify the basic and non-basic variables associated with each point.

A Possible Solution Approach

We already know the LP optimal solution must occur at a corner point, and each corner
point corresponds to a basic feasible solution. Since there are a finite number of basic
solution (no more than nCm), one of them has to be the optimal solution. Therefore, a
possible approach for solving LP is to compute all possible basic solutions, select the one
that has the best value of objective function. Of course we must discard the infeasible
ones.

7
On the surface, this approach appears reasonable. However, there is a catch. Even though
the number of basic solution is finite, it can get very large very quickly, as the problem
size increases. Consider the following numbers:
n
n m Cm
10 5 252
20 10 1,84,756
30 15 155,117,520

Most real world LPs involve hundreds or sometimes even thousands of variables and
constraints. The number of basic solutions for such problems can be astronomically large,
and computing all of them may take months and years even with the fastest computers.
Clearly, this brute force approach is doomed to fail. We must find a better way.

Climbing a Hill
Consider a hill with many roads criss-corssing it, some of them leading to the top of the
hill. You are standing at one of intersections, and wish to reach the top. What do you do?
You could survey the entire hill, make a list of all the intersections of roads, measure the
height of each one, and thereby decide which one represents the top.

Or you could do something smarter. Start from any intersection. There are several roads
leading in different directions. Simply select the road that is most steep, and move along
that road until you reach the next intersection. Keep repeating this basic step, until you
reach the top. But how do you know that you have reached the top? Obviously, when all
the roads from an intersection lead downwards, you must be at the top of the hill. It is that
simple.

We are going to do almost the same thing to solve the LP. The analogy should be clear by
now. Each intersection of roads is a corner point, or a basic feasible solution. Height of
an intersection point represents the objective function value of the basic solution.
Intersection at the top of the hill is the optimal solution. But what are the roads? In order
to answer this question, let’s learn about the concept of neighborhood.

Neighbors of a Basic Solution

Two basic solutions are called neighbors if they differ in only one basic variable.
Consider an LP with five variables: x 1 through x5, and three constraints. Following is a
list of a few basic solutions:

Solution Basic Variables

S1 x1, x2, x4
S2 x1, x3, x4
S3 x1, x4, x5
S4 x1, x3, x5

Consider solutions S1 and S2 and their respective sets of basic variables. Variables x1 and
x4 are common to both, i.e. two variables out of three. They differ in only one basic

8
variable. If x2 is replaced by x3 in the set of basic variables, solution S 1 changes to S2.
Thus, S1 and S2 are considered to be neighboring basic solutions. Note that S 3 is also a
neighbor of S1, but S4 is not. Which points are the neighbors of S2? You would notice that
all thee solutions: S1, S3 and S4 are neighbors of S2, as each differs from S2 in exactly one
basic variable.

Consider drawing a picture in which each solution is a node. Now connect each pair of
nodes with a line if they happen to be neighbors of each other (see Figure 2). Indeed,
these lines are the roads in the hill analogy. Each road leads from one intersection to a
neighboring intersection.

S2

S1 S4

S3

Figure 2

The purpose of defining the neighborhood concept should be clear from the hill climbing
analogy. If we have a good mechanism to move from one basic solution to a neighboring
basic solution, ensuring that the objective function gets better with each move, we can
successfully use the hill climbing idea. We shall see later that the algebraically moving
from one basic solution to a neighboring one is quite easy.

Another important consideration in defining the neighborhood is that the number of

neighbors of a point should not be too many. What is the number of neighboring
solutions for any bfs? Note that each solution obtained by exchanging a basic variable
with a non-basic variable is a neighboring solution. Since there are m basic and (n-m)
non-basic variables, we can exchange them in m(n-m) possible ways. Thus, any basic
solution has m(n-m) neighbors, which is a very small number compared to nCm. Keep in
mind that not all neighbors of a solution may be feasible. The number of feasible
neighbors will generally be less than m(n-m).

We can now summarize our approach as follows:

- Start from any basic feasible solution.

- Examine its neighbors, and move to a neighboring solution so that the objective
function gets better.
- Continue this process until you reach a solution where none of its neighbors has a
better value of the objective function. This is the optimal solution.

9
In the next section, we shall learn the necessary algebra to make this idea work. Note that
this approach is extremely fast compared to the brute force approach of examining all
basic solutions. It examines a very tiny fraction of the total number of basic solution in
reaching the optimal solution, just as in climbing the hill we would pass through a very
small number of intersections.

Elementary Row Operations

In order to learn how to algebraically move from one corner point to another, we have to
learn a bit of algebra. Recall how we solved two equations in two variables. We
combined them in a way so that only one variable remains on the left hand side. The
process of combining equations is carried out by applying the so called elementary row
operations. They are called row operations because if you write the equations in a matrix
form, each equation corresponds to a row of the matrix.

There are only two types of row operations.

1. Multiply a given equation by a constant.

For example, multiply the equation 2x 1 + 3x2 = 4 by 3. The equation changes to 6x 1+9x2
= 12.

2. Add a multiple of one equation to another equation.

For example, consider the following equations:

2x1 + 3x2 = 4 I
x1 - x2 = 2 II

Suppose we add 2 times equation II to equation I. The new equation I (or say I) now is

4x1 + x2 = 8 I

By intelligently using these elementary row operations, we can find a solution to a given
systems of equations. This process is called Gaussian elimination, whose details you can
find in any algebra textbook. The important thing to remember is that when we apply row
operations to any given set of equations, the form of equations changes, but the set of
solutions of those equations remains unchanged. Also remember that these row
operations can only be applied to equations, not to inequalities. The inequalities must first
be converted into equations by introducing a slack or surplus variables, before we can
apply row operations to them.

Canonical Form
Consider a set of m equations in n variables (m < n). A basic solution will have m basic
variables, and (n-m) non-basic variables. By using elementary row operations we can
transform the equations into a form where each of the m basic variables appears in only

10
one of the equations with coefficient 1, and is absent from other equations. Let us
understand with the following example of two equations with three variables.

3x1 + 6x2 + 9x3 = 30 I

2x1 + 6x2 + 4x3 = 24 II

Suppose that x1 and x2 are basic variables, and x 3 is non-basic. We want to transform the
equations so that x 1 has coefficient 1 in the first equation, and is absent from the second
equation. Similarly, x 2 should have coefficient 1 in the second equation, and is absent
from the first equation. This can be easily achieved by the following sequence of row
operations:

1. Divide equation I by 3. This turns coefficient of x 1 to 1 in equation I.

x1 + 2x2 + 3x3 = 10 I
2x1 + 6x2 + 4x3 = 24 II

2. Subtract 2 times equation I from equation II. This removes x 1 from equation II.

x1 + 2x2 + 3x3 = 10 I
2x2 - 2x3 = 4 II

3. Divide equation II by 2. This makes coefficient of x 2 as 1 in equation II

x1 + 2x2 + 3x3 = 10 I
x2 - x3 = 2 II

4. Lastly, subtract 2 times equation II from equation I. This eliminates x 2 from
equation I.

x1 + 5x3 = 6 I
x2 - x3 = 2 II

Now the process is complete. This is nothing but the process of Gaussian elimination
mentioned earlier. Notice that x1 appears only in the first equation, and x2 only in the
second equation. Moreover, each has unit coefficient in the equation where it appears.
Such a form is called the canonical form. By using the elementary row operations, any
set of equations can be reduced to canonical form, if we know which variables are basic
variables.

But what is the point of doing this? Recall that in a basic the non-basic variables are set to
value 0, reducing the system to m equations in m variables. The unique solution to these
m equations, gives us the values of basic variables. What is that solution? Note that if the
equations are in canonical form, and non-basic variables are set to value zero, the right
hand side of each equation gives the value of the basic variable in that equation. Just put
x3 = 0 in the above equations (I and II), and immediately you get the solution: x 1 = 6

11
and x2 = 2. There is nothing to solve as each equation has only one variable. The solution
vector then is: (6, 2, 0).

Thus, the canonical form is very convenient way to represent a basic solution. It
immediately reveals which variables are basic and which are non-basic; and what are the
values of basic variables. Moreover, if we wish to move from a basic solution to a
neighboring basic solution, it is very easy to accomplish if the equations are in canonical
form, as we learn next.

Moving to a Neighboring Basic Solution

Just as we used Gaussian elimination to obtain a canonical form which corresponds to a
basic solution, we can also use it to transform a given canonical form into another, so that
it represents a neighboring basic solution. Recall that two neighboring basic solutions
differ in only one basic variable. The process of moving to a neighboring solution can be
viewed as turning one of the basic variables into non-basic, and a non-basic into basic.

Once we have decided the incoming variable (the non-basic which will become basic),
and the outgoing variable (one that turn from basic to non-basic), a few row operations
will accomplish the task. Some textbooks also call these entering and leaving variables.

In the canonical form given below, let us replace x 1 with x3 as a basic variable.

x1 + 5x3 = 6 I
x2 - x3 = 2 II

Recall that each basic variable occupies one row in the canonical form. So the incoming
variable will have to occupy the same row that is being vacated by the outgoing variable.
In this case, it means that coefficient of x 3 must be one in row 1, and zero in all other
rows. We need the following row operations to achieve this.

1. Divide row 1 by 5.
2. Add -1 times row 1 to row 2.

This yields the following canonical form, which represents the solution vector (0, 0.8,
1.2).

0.2x1 + x3 = 1.2 I

-0.2x1 + x2 = 0.8 II

This operation of moving from one basic solution to another is called a pivot operation.
The column corresponding to the incoming variable is called the pivot column, and the
row occupied by the outgoing variable is called the pivot row. The element at the
intersection of pivot row and pivot column is called the pivot element.

Selecting the Incoming Variable

12
Now that we know the mechanics of moving from one corner-point/basic solution to
another, how do we utilize it to solve the linear programming problem? A given basic
solution has many neighbors. Which one should we move to? In other words, which
variable should enter the basis, and which one should leave? We must make this choice
so that a) the objective function improves, and b) the solution stays feasible. This is best
understood with an example.
Consider the following LP.

Maximize Z = 3x1 + 2x2

Subject to: 2x1 + x2  6
x1 + 2x2  6
x1, x2  0
After adding the slack variables s 1 and s2, and treating z as an additional variable, we can
write it in the form of following three equations.

Z -3x1 -2x2 = 0 Equation 0

2x1 + x2 +s1 = 6 Equation 1
x1 +2x2 +s2 = 6 Equation 2

Note that we have converted the objective function into another equation. Since this is a
special equation, we write it on the top, and label it equation 0, while other equations are
numbered from 1 to m. Also note that the system is already in canonical form with z, s 1,
and s2 being the basic variables, and x 1, x2 non-basic. Each value on the rhs represents the
value of the basic variable in that equation. If all the rhs values in equations 1 through m
are positive, the solution satisfied the non-negativity constraints, and is called a feasible
solution. The rhs value of equation 0 gives the value of z, i.e. the objective function.

In order to move to a neighbor, first we must decide the incoming variable, i.e. which
non-basic variable should become basic. Since we wish to maximize Z, let us select the
incoming variable which will lead to maximum possible increase in Z. The coefficients of
equation 0 give us this information. Every unit increase in x 1 will increase Z by 3, and
every unit increase in x 2 will increase Z by 2. Clearly, x 1 is a better choice. This gives us
the following rule for selecting the incoming variable.

Select the variable with largest negative coefficient in equation 0.

Note that for a minimization problem this rule will get reversed, i.e. select the variable
with largest positive coefficient.

The Tableau Format

The equations above can be written more compactly in a tabular form as follows.

z x1 x1 s1 s2 rhs
1 -3 -2 0 0 0
0 2 1 1 0 6

13
 0 1 2 0 1 6

The form is called a tableau, which simply means a table in French. Each equation
corresponds to a row of the table, and each variable to a column. Row 0 is the objective
function row, and the rhs column gives us the values of basic variables in each row.
There is an implied “=” sign before the rhs column.

The Outgoing Variable

Now let us address the issue of outgoing variable. Any basic variable is a candidate for
outgoing variable. However, variable z, being the objective function, cannot be an
outgoing variable and always remains basic. It may be considered a permanent basic
variable.

Suppose we make an arbitrary decision to select s 2 as the outgoing variable. It means that
the incoming variable x 1 will “occupy” row 2, which is currently occupied by s2. In other
words, coefficient of x 1 will be 1 in row 2, and zero in all other rows. We’ll have to
perform the following row operations to accomplish this.

1. Subtract 2 times row 2 from row 1.

2. Add 3 time row 2 to row 0.

The new tableau after performing these operations is:

z x1 x1 s1 s2 rhs
1 0 4 3 0 18
0 0 -3 1 -2 -6
0 1 2 0 1 6

Note that one of the values in the rhs column has turned negative, i.e. s 1 = -6. However,
since all variables, including the slack variables, are required to be non-negative, this is
not a feasible solution. Why did this happen? It happened because we did not select the
outgoing variable with due care. In fact, if we want to ensure the new solution remains
feasible, we have almost no choice in respect of the outgoing variable. We must select the
outgoing variable in such a way so that all the rhs values in the new tableau remain non-
negative. What choice of outgoing variable will ensure this?

In order to explore this issue, let us focus our attention on the column corresponding to
incoming variable, and the rhs column. Let i be the entry in the ith row of incoming
variable, and  i be the rhs of row i (see figure below).

Row (i) Incoming Column RHS column

1 1 1
2 2 2
3 3 3

14
Can we select a row with a negative value of  as the pivot row? Recall that we have to
divide the pivot row by  in order to perform the pivot operation. If  is negative, this
division will make the rhs of pivot row negative, turning the solution infeasible. Hence a
negative value cannot be selected as the pivot element.

The choice of outgoing variable essentially amounts to selecting the pivot row. We have
to examine the effect of choosing a particular pivot row on the rhs values after the pivot
operation. Suppose we choose row i=2 as the pivot row. Then, after the necessary row
operations, the values in the rhs columns will be as shown below:

Row (i) Incoming Column RHS column

1 0 1 - 1.2/2
2 1 2/2
3 0 3 - 3.2/2

If the rhs value of row 1 is to remain positive, we must have

1 - 1.2/2  0
Or 1/1  2/2

Similarly, for row 3 we must have  3/3  2/2. In general, we can say that if i is the
pivot row, then for every other row j, we must have  j/j   i/i. What choice of i will
ensure this condition? Suppose we compute  i/i for all rows, and select the row with
smallest ratio as the pivot row. Clearly, this will ensure that / ratio for every other row
is greater than that for the pivot row, which, in turn, ensures that after the pivot, all rhs
values remain non-negative. This is called the minimum ratio test.

For the current example, if x 1 is the incoming variable, the determination of pivot row by
minimum ratio test is demonstrated below:

Pivot
Column Ratio
  /
z x1 x1 s1 s2 rhs
1 -3 -2 0 0 0 Row 0
0 (2) 1 1 0 6 6/2 = 3.0 Smallest Row 1
0 1 2 0 1 6 6/1 = 6.0 Row 2

The / ratio is smallest for row 1. Hence row 1 is the pivot row, and the basic variable
occupying this row, i.e. s 1 is the outgoing variable. Thus the element “2” enclosed in
parentheses is the pivot element. If the pivot is performed with this choice of outgoing
variable, the result is the following.

15
 z x1 x1 s1 s2 rhs
1 0 -0.5 1.5 0 9
0 1 0.5 0.5 0 3
0 0 1.5 -0.5 1 3

Notice that none of the rhs values has turned negative. The new solution vector is
(3,0,0,3) with Z = 9.

The rule for selecting the pivot row is:

Given the pivot column, select the pivot row so that the pivot element  is positive, and
mimimizes the  / ratio.

Sometimes there may be a tie for the smallest ratio when performing this test, i.e. more
than one rows may have the same / value which is also the smallest. Then we can
break the tie arbitrarily, and pick any one of these rows as the pivot row. This situation
leads to a condition called degeneracy, which we shall discuss later.

Simplex Algorithm
Now we have learnt all the basics to formally describe an algorithm, i.e. a systematic,
step-by-step, iterative procedure for solving the linear programming problem. This
algorithm is called the Simplex Algorithm. The tableau format of representing a basic
solution of a linear program discussed earlier is called a Simplex Tableau. A rather
informal description of this algorithm is given below.

Step 0 : Find an initial basic feasible solution of the LP, and set up the simplex tableau
corresponding to this solution. (The issue of how to find the initial solution will be
discussed later).

Step 1 : Select the variable with the largest negative (positive) coefficient in the objective
function row (Row 0) as the incoming variable. If all variables have non-negative (non-
positive) coefficients in Row 0, the current solution is optimal, STOP. Otherwise, go to
step 2.

Step 2 : Perform the Minimum Ratio Test to determine the pivot row and the outgoing
variable.

Step 3 : Perform the pivot and obtain the tableau corresponding to the new basic feasible
solution. Go back to Step 1.

In Step 0, we are yet to fully answer the issue of finding the initial basic feasible solution,
which we shall discuss in the next chapter. However, it is worth noting that if all
constraints in the LP are of  type, and the rhs of each constraint is non-negative, an
initial basic feasible solution can be easily constructed. After we add the slack variables,
to convert inequalities into equalities, we find that the equations are already in canonical

16
form with each of the slack variables being basic. Since the rhs values are all non-
negative, this is clearly a feasible basic solution.

When some constraints are  or = type, some additional work is needed to construct an
initial feasible solution, which we shall discuss later.

In Step 1, we select the variable with most negative coefficient in Row 0 as the incoming
variable. Note the value of this variable represents the increase in Z for each unit increase
in the value of incoming variable. If there is no variable with a negative coefficient in
Row 0, it means that all the neighboring solutions have a Z-value worse than the current
solution. Hence the current solution is optimal, and we can stop. This is the so called
stopping rule of the algorithm. Every algorithm must have a stopping rule, otherwise,
being an iterative procedure, it will keep running for ever.

Next we take up a slightly bigger LP and solve it using the Simplex Algorithm in order to
consolidate our understanding of the algorithm.

An Example

Consider the following linear program with two variables and three constraints.

Maximize Z = 4x1 + 5x2

Subject to 2x1 + 3x2  24
3x1 + 2x2  24
x1 + x2  9
x1, x2  0
We add slack variables s1, s2 and s3, and write the equation in the tableau format as
follows.

Tableau 1
z x1 x2 s1 s2 s3 rhs
1 -4 -5 0 0 0 0
0 2 (3) 1 0 0 24
0 3 2 0 1 0 24
0 1 1 0 0 1 9

Note that the tableau is in canonical form with s 1, s2 and s3 being basic, and x 1, x2 non-
basic. The solution vector is (0, 0, 24, 24, 9), with Z = 0. We select x 1 as incoming
variable, as it has the most negative coefficient in Row 0. The Minimum Ratio Test is
conducted, and Row 1 is selected at the pivot row. Performing the pivot leads to the
following tableau.

17
Tableau 2
z x1 x2 s1 s2 s3 rhs
1 -0.667 0 1.667 0 0 40
0 0.667 1 0.333 0 0 8
0 1.667 0 -0.667 1 0 8
0 (0.333) 0 0.333 0 1 1

The new solution vector is (0,8,0,8,1), and value of Z has jumped up to 40. Is this
solution optimal? Not yet, because there exists a negative coefficient in Row 0. We select
x1 as the pivot column, and Row 3 selected as the pivot row according to the minimum
ratio test. The pivot element is the highlighted value 0.333 in Row 3. The result of the
pivot is the following tableau.

Tableau 3
z x1 x2 s1 s2 s3 rhs
1 0 0 1 0 2 42
0 0 1 1 0 -2 6
0 0 0 1 1 -5 3
0 1 0 -1 0 3 3

The new solution vector is (3, 6, 0, 3, 0), and Z has further increased to 42. Now there are
no negative values in Row 0. Therefore, this solution is optimal, and we may stop the
algorithm.

Since this is a two variable problem, we can also solve it graphically. The diagram below
shows the feasible region, and the progress of the Simplex algorithm.

Constraint 1

Tableau 2 solution
Constraint 3

Objective Function
Tableau 3
(Optimal Solution)
Constraint 2
Tableau 1 solution
Feasible Region

18
Multiple Optimal Solutions

Sometimes, the LP optimal solution need not be unique. In other words, there may be
more than one solutions which are optimal. How do we know if this is indeed the case? If
there are multiple optimal solutions, we would like to exercise that choice, and choose the
solution that suits us better in some respect that cannot be modeled by LP.

Consider the LP given below.

Maximize 2x1+x2
subject to: -x1 + x2  2
4x1+2x2  12
2x1  5
xi  0  i

The initial and final tableaux for this problem are given below:

Tableau 1 (Initial Tableau)

Z X1 x2 s1 s2 s3 RHS
1 -2 -1 0 0 0 0
0 -1 1 1 0 0 2
0 4 2 0 1 0 12
0 2 0 0 0 1 5

Tableau 2 (Final Tableau)

Z x1 x2 s1 s2 s3 RHS
1 0 0 0 0.5 0 6
0 0 0 1 -0.5 1.5 3.5
0 0 1 0 0.5 -1 1
0 1 0 0 0 0.5 2.5

The solution vector is (2.5, 1, 3.5, 0, 0), and Z=6. The final tableau is optimal because
there are no negative coefficients in the objective function row. However, note that the
coefficient of s3, which is a non-basic variable, is exactly zero. We know that if a non-
basic variable with negative coefficient is made basic, the objective function increases;
and if it is positive, the objective function decreases. But it is zero, what do you expect to
happen if a pivot is performed with s3 as incoming variable? It is quite obvious that the
objective function will remain unchanged.

Let us verify this. If s3 is selected as the incoming variable, as per the min ratio test, the
pivot row will be Row 1. The new tableau after the pivot is:

19
Tableau 3 (Alternate Optimal)
Z x1 x2 s1 s2 s3 RHS
1 0 0 0 0.5 0 6
0 0 0 0.667 -0.333 1 2.333
0 0 1 0.667 0.167 0 3.333
0 1 0 -0.333 0.167 0 1.333

The solution vector for this tableau is (1.333, 3.333, 0, 0, 2.333), which is quite different
from the previous one. However, the value of objective function is still 6. We may use
either one of these solutions, as both are optimal.
Multiple optimal solutions exist whenever the Row 0 coefficient of a non-basic variable
is zero in the optimal tableau.

But are these the only two solutions which are optimal, or are there more? To answer the
question, let us look at the graphical solution of this problem.

Objective Function

Tableau 3 Solution

Constraint 2
B
Tableau 2 Solution

Tableau 1 Solution

Note that two solutions corresponding to Tableaux 2 and 3 are at corner points labeled A
and B, respectively. Since the objective function line is parallel to constraint 2, if it is
drawn so as to pass through A, it will also pass through B. Note that with the objective
function line in this position, the entire feasible region is on the “worse” side of the line.
Hence, any feasible solution that lies on the objective function line is optimal. Therefore,
not only A and B, but any point on the line segment from A to B is optimal.

If x1 and x2, respectively, are the solution vectors corresponding to points A and B, then
any point between A and B can be written in the form x = x1 + (1-)x2, where  is a
number between 0 and 1. When we combine two solution points in this way to obtain a

20
new point, this new point is called a convex combination of the two points. In fact, every
point on the line segment A-B can be written as a convex combination of points A and B.
Note that with =0, the formula yields the point B itself, and with =1, it yields A.

An LP either has a unique optimal solution, or an infinite number of them. If any two
solutions are optimal, their convex combination is also optimal.

Unbounded Solution

A third possibility for the LP is that it may be unbounded, i.e. the objective function can
be improved without any limit without running into any constraint. How do we detect this
condition in a simplex tableau? Let us understand by following example.

Maximize 2x1 + x2
subject to: -x1 + x2  1
x1 - 2x2  2
x1, x2  0

Tableau 1 (Initial Tableau)

z x1 x2 s1 s2 RHS
1 -2 -1 0 0 0
0 -1 1 1 0 1
0 1 -2 0 1 2

Select x1 as incoming variable, and Row 2 as the pivot row (i.e. s 2 is the outgoing
variable). The next tableau after the pivot is:

Tableau 2
z x1 x2 s1 s2 RHS
1 0 -5 0 2 4
0 0 -1 1 1 3
0 1 -2 0 1 2

Note that x2 is the only choice available for the incoming variable. But what about the
pivot row? All the coefficients in this column are negative, and none of the rows can be
selected as the pivot row. We cannot proceed any further, even though we have not
reached the optimality condition yet.

It is precisely this condition in the simplex tableau which indicates that the problem is
unbounded. In order to understand this, let us write each row in the equation form, with
the basic variable on the left hand side, and all other variables on the right hand side, as
shown below.

z = 4 + 5x2 – 2s2 Row 0 (Objective Function)

s1 = 3 + x 2 – s2 Row 2
x1 = 2 + 2x2 – s2 Row 1

21
Variables x2 and s2 are both non-basic, and hence have value zero. If we select x2 as the
incoming variable, it means we intend to increase its value. For each unit increase in x2, z
will increase by 5. So, we would like to increase x 2 as much as possible, i.e. until one of
the basic variables reaches value 0. But, a look at the equations above reveals that
increasing x2 will only increase the values of both s 1 and x1, and neither one of them will
ever become zero. It means that we can go on increasing x2 without any limit, and the
objective function will keep getting better, and the solution will remain feasible. This is
precisely what unboundedness means.

The problem is unbounded if, in a simplex tableau, a non-basic variable is eligible to

enter the basis, but there is no strictly positive coefficient in that column.

Let’s look at the graphical solution of this problem.

Direction of
Initial Solution
B unboundedness
A

Tableau 2 Solution

After reaching point B represented by tableau 2, if we try to make x 2 basic, the solution
must move in a direction indicated by the dotted line arrow. But as we move in this
direction, no other constraint comes in the way, and the solution remains feasible, as the
feasible region extends indefinitely in that direction.

22
23