Solving Linear Programming Problems: The Simplex Method

Simplex method is a general procedure for solving linear programming problems. Developed by George Dantzig in 1947, it has proved to be a remarkably efficient method that is used routinely to solve huge problems on todays computers. Except for its use on tiny problems, this method is always executed on a computer, and sophisticated software packages are widely available. Extensions and variations of the simplex method also are used to perform postoptimality analysis (including sensitivity analysis) on the model. The simplex method is an algebraic procedure. However, its underlying concepts are geometric. Consider the figure on the right. The five constraint boundaries and their points of intersection are highlighted. Here, each constraint boundary is a line that forms the boundary of what is permitted by the corresponding constraint. The points of intersection are the corner-point solutions of the problem. The five that lie on the corners of the feasible region(0, 0), (0, 6), (2, 6), (4, 3), and (4, 0)are the corner-point feasible solutions (CPF solutions). [The other three(0, 9), (4, 6), and (6, 0)are called corner-point infeasible solutions since they are outside the feasible region.] In this example, each corner-point solution lies at the intersection of two constraint boundaries. (For a linear programming problem with n decision variables, each of its corner-point solutions lies at the intersection of n constraint boundaries.)

Optimality test: Consider any linear programming problem that possesses at least one optimal solution. If a CPF solution has no adjacent CPF solutions that are better (as measured by Z), then it must be an optimal solution. Thus, for the example, (2, 6) must be optimal simply because its Z = 36 is larger than Z = 30 for (0, 6) and Z = 27 for (4, 3). This optimality test is the one used by the simplex method for determining when an optimal solution has been reached.

The Key Solution Concepts

Concept 1: The simplex method focuses solely on CPF solutions. For any problem with at least one optimal solution, finding one requires only finding a best CPF solution.

Concept 2: The simplex method is an iterative algorithm (a systematic solution procedure that keeps repeating a fixed series of steps, called an iteration, until a desired result has been obtained) with the following structure. Initialization: Set up to start iterations, including finding an initial CPF solution. Optimality test: Is the current CPF solution optimal? If no, go to Iteration. If yes, Stop. Iteration: Perform an iteration to find a better CPF solution. Concept 3: Whenever possible, the initialization of the simplex method chooses the origin (all decision variables equal to zero) to be the initial CPF solution. When there are too many decision variables to find an initial CPF solution graphically, this choice eliminates the need to use algebraic procedures to find and solve for an initial CPF solution. Concept 4: Given a CPF solution, it is much quicker computationally to gather information about its adjacent CPF solutions than about other CPF solutions. Therefore, each time the simplex method performs an iteration to move from the current CPF solution to a better one, it always chooses a CPF solution that is adjacent to the current one. No other CPF solutions are considered. Consequently, the entire path followed to eventually reach an optimal solution is along the edges of the feasible region. Concept 5: After the current CPF solution is identified, the simplex method examines each of the edges of the feasible region that emanate from this CPF solution. Each of these edges leads to an adjacent CPF solution at the other end, but the simplex method does not even take the time to solve for the adjacent CPF solution. Instead, it simply identifies the rate of improvement in Z that would be obtained by moving along the edge. Among the edges with a positive rate of improvement in Z, it then chooses to move along the one with the largest rate of improvement in Z. The iteration is completed by first solving for the adjacent CPF solution at the other end of this one edge and then relabeling this adjacent CPF solution as the current CPF solution for the optimality test and (if needed) the next iteration. At the first iteration of the example, moving from (0, 0) along the edge on the x1 axis would give a rate of improvement in Z of 3 (Z increases by 3 per unit increase in x1), whereas moving along the edge on the x2 axis would give a rate of improvement in Z of 5 (Z increases by 5 per unit increase in x2), so the decision is made to move along the latter edge. At the second iteration, the only edge emanating from (0, 6) that would yield a positive rate of improvement in Z is the edge leading to (2, 6), so the decision is made to move next along this edge. Concept 6: Solution concept 5 describes how the simplex method examines each of the edges of the feasible region that emanate from the current CPF solution. This examination of an edge leads to quickly identifying the rate of improvement in Z that would be obtained by moving along the edge toward the adjacent CPF solution at the other end. A positive rate of improvement in Z implies that the adjacent CPF solution is better than the current CPF solution, whereas a negative rate of improvement in Z implies that the adjacent CPF solution is worse. Therefore, the optimality test consists simply of checking whether any of the edges give a positive rate of improvement in Z. If none do, then the current CPF solution is optimal. In the example, moving along either edge from (2, 6) decreases Z. Since we want to maximize Z, this fact immediately gives the conclusion that (2, 6) is optimal.

SETTING UP THE SIMPLEX METHOD

The preceding section stressed the geometric concepts that underlie the simplex method. In this section, the algebraic language of the simplex method is introduced and relate it to the concepts of the preceding section. The algebraic procedure is based on solving systems of equations. Therefore, the first step in setting up the simplex method is to convert the functional inequality constraints to equivalent equality constraints. (The nonnegativity constraints are left as inequalities because they are treated separately.) This conversion is accomplished by introducing slack variables. To illustrate, consider the first functional constraint in the example: x1 4. The slack variable for this constraint is defined to be x3 = 4 x1, which is the amount of slack in the left-hand side of the inequality. Thus,

x1 + x3 4. The original constraint x1 4 is entirely equivalent to the pair of constraints x1 + x3 4 and x3 0. Upon the introduction of slack variables for the other functional constraints, the original linear programming model for the example (shown below on the left) can now be replaced by the equivalent model (called the augmented form of the model) shown below on the right:

The slack variables are not shown in the objective function because the coefficients there are 0.

Although both forms of the model represent exactly the same problem, the new form is much more convenient for algebraic manipulation and for identification of CPF solutions. We call this the augmented form of the problem because the original form has been augmented by some supplementary variables needed to apply the simplex method. If a slack variable equals 0 in the current solution, then this solution lies on the constraint boundary for the corresponding functional constraint. A value greater than 0 means that the solution lies on the feasible side of this constraint boundary, whereas a value less than 0 means that the solution lies on the infeasible side of this constraint boundary. An augmented solution is a solution for the original variables (the decision variables) that has been augmented by the corresponding values of the slack variables. For example, augmenting the solution (x1, x2) = (3, 2) in the example yields the augmented solution (x1, x2, x3, x4, x5) = (3, 2, 1, 8, 5) because the corresponding values of the slack variables are x3 = 1, x4 = 8, and x5 = 5. A basic solution is an augmented corner-point solution. To illustrate, consider the corner-point infeasible solution (4, 6) in the figure on page 1. Augmenting it with the resulting values of the slack variables x3 = 0, x4 = 0, and x5 = 6 yields the corresponding basic solution (4, 6, 0, 0, -6). The fact that corner-point solutions (and so basic solutions) can be either feasible or infeasible implies the following definition: A basic feasible (BF) solution is an augmented CPF solution. Thus, the CPF solution (0, 6) in the example is equivalent to the BF solution (0, 6, 4, 0, 6) for the problem in augmented form. The only difference between basic solutions and corner-point solutions (or between BF solutions and CPF solutions) is whether the values of the slack variables are included. For any basic solution, the corresponding cornerpoint solution is obtained simply by deleting the slack variables. Therefore, the geometric and algebraic relationships between these two solutions are very close.

For the augmented form of the example, notice that the system of functional constraints has 5 variables and 3 equations, so number of variables number of equations = 5 3 = 2. This fact gives us 2 degrees of freedom in solving the system, since any two variables can be chosen to be set equal to any arbitrary value in order to solve the three equations in terms of the remaining three variables.( This method of determining the number of degrees of freedom
for a system of equations is valid as long as the system does not include any redundant equations. This condition always holds for the system of equations formed from the functional constraints in the augmented form of a linear programming model.) The

simplex method uses zero for this arbitrary value. Thus, two of the variables (called the nonbasic variables) are set equal to zero, and then the simultaneous solution of the three equations for the other three variables (called the basic variables) is a basic solution. These properties are described in the following general definitions.

A basic solution has the following properties: 1. Each variable is designated as either a nonbasic variable or a basic variable. 2. The number of basic variables equals the number of functional constraints (now equations). Therefore, the number of nonbasic variables equals the total number of variables minus the number of functional constraints. 3. The nonbasic variables are set equal to zero. 4. The values of the basic variables are obtained as the simultaneous solution of the system of equations (functional constraints in augmented form). (The set of basic variables is often referred to as the basis.) 5. If the basic variables satisfy the nonnegativity constraints, the basic solution is a basic feasible solution. When we deal with the problem in augmented form, it is convenient to consider and manipulate the objective function equation at the same time as the new constraint equations. Therefore, before we start the simplex method, the problem needs to be rewritten once again in an equivalent way:

THE SIMPLEX METHOD IN TABULAR FORM

The tabular form of the simplex method uses a simplex tableau to compactly display the system of equations yielding the current BF solution. For this solution, each variable in the leftmost column equals the corresponding number in the rightmost column (and variables not listed equal zero). When the optimality test or an iteration is performed, the only relevant numbers are those to the right of the Z column. The term row refers to just a row of numbers to the right of the Z column (including the right side number), where row i corresponds to Eq. (i). The following table compares the initial system of equations in algebraic form (on the left) and in tabular form (on the right), where the table on the right is called a simplex tableau. The basic variable for each equation is shown in bold type on the left and in the first column of the simplex tableau on the right. [Although only the xj variables are basic or nonbasic, Z plays the role of the basic variable for Eq. (0).] All variables not listed in this basic variable column (x1, x2) automatically are nonbasic variables. After we set x1 = 0, x2 = 0, the right side column gives the resulting solution for the basic variables, so that the initial BF solution is (x1, x2, x3, x4, x5) = (0, 0, 4, 12,18) which yields Z = 0.

The Simplex Method Procedure (and Iteration 1 for the Example) Initialization. Introduce slack variables. Select the decision variables to be the initial nonbasic variables (set equal to zero) and the slack variables to be the initial basic variables. For the Example: This selection yields the initial simplex tableau so the initial BF solution is (0, 0, 4, 12, 18). Optimality Test. The current BF solution is optimal if and only if every coefficient in row 0 is nonnegative ( 0). If it is, stop; otherwise, go to an iteration to obtain the next BF solution, which involves changing one nonbasic variable to a basic variable (step 1) and vice versa (step 2) and then solving for the new solution (step 3). For the Example: Just as Z = 3x1 + 5x2 indicates that increasing either x1 or x2 will increase Z, so the current BF solution is not optimal, the same conclusion is drawn from the equation Z 3x1 5x2 = 0. Iteration. Step 1: Determine the entering basic variable by selecting the variable (automatically a nonbasic variable) with the negative coefficient having the largest absolute value (i.e., the most negative coefficient) in Eq. (0). Put a box around or encircle the column below this coefficient, and call this the pivot column. For the Example: The most negative coefficient is 5 for x2, so x2 is to be changed to a basic variable. (This change is indicated in the table below by the box around the x2 column below 5.) Step 2: Determine the leaving basic variable by applying the minimum ratio test. Minimum Ratio Test 1. Pick out each coefficient in the pivot column that is strictly positive (> 0). 2. Divide each of these coefficients into the right side entry for the same row. 3. Identify the row that has the smallest of these ratios. 4. The basic variable for that row is the leaving basic variable, so replace that variable by the entering basic variable in the basic variable column of the next simplex tableau. Put a box around this row and call it the pivot row. Also call the number that is in both boxes the pivot number.

For the Example: The calculations for the minimum ratio test are shown to the right of the table above. Thus, row 2 is the pivot row (see the box around this row in the first simplex tableau below), and x4 is the leaving basic variable. In the next simplex tableau, x2 replaces x4 as the basic variable for row 2.

Step 3: Solve for the new BF solution by using elementary row operations (multiply or divide a row by a nonzero constant; add or subtract a multiple of one row to another row) to construct a new simplex tableau in proper form from Gaussian elimination below the current one, and then return to the optimality test. The specific elementary row operations that need to be performed are listed below. 1. Divide the pivot row by the pivot number. Use this new pivot row in steps 2 and 3. 2. For each other row (including row 0) that has a negative coefficient in the pivot column, add to this row the product of the absolute value of this coefficient and the new pivot row. 3. For each other row that has a positive coefficient in the pivot column, subtract from this row the product of this coefficient and the new pivot row. For the Example: Since x2 is replacing x4 as a basic variable, we need to reproduce the first tableaus pattern of coefficients in the column of x4 (0, 0, 1, 0) in the second tableaus column of x2. To start, divide the pivot row (row 2) by the pivot number (2), which gives the new row 2 shown in the table abouve. Next, we add to row 0 the product, 5 times the new row 2. Then we subtract from row 3 the product, 2 times the new row 2 (or equivalently, subtract from row 3 the old row 2). These calculations yield the new tableau shown below for iteration 1. Thus, the new BF solution is (0, 6, 4, 0, 6), with Z = 30. We next return to the optimality test to check if the new BF solution is optimal. Since the new row 0 still has a negative coefficient (- 3 for x1), the solution is not optimal, and so at least one more iteration is needed.

The second iteration starts anew from the second tableau above to find the next BF solution. Following the instructions for steps 1 and 2, we find x1 as the entering basic variable and x5 as the leaving basic variable, as shown below.

For step 3, we start by dividing the pivot row (row 3) by the pivot number (3). Next, we add to row 0 the product, 3 times the new row 3. Then we subtract the new row 3 from row 1. We now have the set of tableau shown below. Therefore, the new BF solution is (2, 6, 2, 0, 0), with Z = 36. Going to the optimality test, we find that this solution is optimal because none of the coefficients in row 0 is negative, so the algorithm is finished.

Consequently, the optimal solution for the problem (before slack variables are introduced) is x1* = 2, x2* = 6 with Z* = 36.

TIE BREAKING IN THE SIMPLEX METHOD

Tie for the Entering Basic Variable Step 1 of each iteration chooses the nonbasic variable having the most negative coefficient in the current Eq. (0) as the entering basic variable. Now suppose that two or more nonbasic variables are tied for having the largest negative coefficient (in absolute terms). For example, this would occur in the first iteration for our example if its objective function were changed to Z = 3x1+ 3x2, so that the initial Eq. (0) became Z 3x1 3x2 = 0. How should this tie be broken? The answer is that the selection between these contenders may be made arbitrarily. The optimal solution will be reached eventually, regardless of the tied variable chosen, and there is no convenient method for predicting in advance which choice will lead there sooner. In this example, the simplex method happens to reach the optimal solution (2, 6) in three iterations with x1 as the initial entering basic variable, versus two iterations if x2 is chosen. Tie for the Leaving Basic VariableDegeneracy Now suppose that two or more basic variables tie for being the leaving basic variable in step 2 of an iteration. Does it matter which one is chosen? Theoretically it does, and in a very critical way, because of the following sequence of events that could occur. First, all the tied basic variables reach zero simultaneously as the entering basic variable is increased. Therefore, the one or ones not chosen to be the leaving basic variable also will have a value of zero in the new BF solution. (Note that basic variables with a value of zero are called degenerate, and the same term is applied to the corresponding BF solution.) Second, if one of these degenerate basic variables retains its value of zero until it is chosen at a subsequent iteration to be a leaving basic variable, the corresponding entering basic variable also must remain zero (since it cannot be increased without making the leaving basic variable negative), so the value of Z must remain unchanged. Third, if Z may remain the same rather than increase at each iteration, the simplex method may then go around in a loop, repeating the same sequence of solutions periodically rather than eventually increasing Z toward an optimal solution. In fact, examples have been artificially constructed so that they do become entrapped in just such a perpetual loop. Fortunately, although a perpetual loop is theoretically possible, it has rarely been known to occur in practical problems. If a loop were to occur, one could always get out of it by changing the choice of the leaving basic variable. Furthermore, special rules have been constructed for breaking ties so that such loops are always avoided. However, these rules frequently are ignored in actual application, and they will not be repeated here. For your purposes, just break this kind of tie arbitrarily and proceed without worrying about the degenerate basic variables that result. No Leaving Basic VariableUnbounded Z In step 2 of an iteration, there is one other possible outcome that we have not yet discussed, namely, that no variable qualifies to be the leaving basic variable. This outcome would occur if the entering basic variable could be increased indefinitely without giving negative values to any of the current basic variables. In tabular form, this means that every coefficient in the pivot column (excluding row 0) is either negative or zero. Multiple Optimal Solutions Under the definition of optimal solution a problem can have more than one optimal solution. This fact is illustrated in the figure on page 1 by changing the objective function in the to Z = 3x1 + 2x2, so that every point on the line segment between (2, 6) and (4, 3) is optimal.

The simplex method automatically stops after one optimal BF solution is found. However, for many applications of linear programming, there are intangible factors not incorporated into the model that can be used to make meaningful choices between alternative optimal solutions. In such cases, these other optimal solutions should be identified as well. After the simplex method finds one optimal BF solution, you can detect if there are any others and, if so, find them as follows: Whenever a problem has more than one optimal BF solution, at least one of the nonbasic variables has a coefficient of zero in the final row 0, so increasing any such variable will not change the value of Z. Therefore, these other optimal BF solutions can be identified (if desired) by performing additional iterations of the simplex method, each time choosing a nonbasic variable with a zero coefficient as the entering basic variable. (If such an iteration has no leaving basic variable, this indicates that the feasible region is unbounded and the entering basic variable can be increased indefinitely without changing the value of Z.) To illustrate, consider again the case just mentioned, where the objective function is changed to Z = 3x1 + 2x2. The simplex method obtains the first three tableaux shown below and stops with an optimal BF solution. However, because a nonbasic variable (x3) then has a zero coefficient in row 0, we perform one more iteration in to identify the other optimal BF solution. Thus, the two optimal BF solutions are (4, 3, 0, 6, 0) and (2, 6, 2, 0, 0), each yielding Z = 18. Notice that the last tableau also has a nonbasic variable (x4) with a zero coefficient in row 0. This situation is inevitable because the extra iteration does not change row 0, so this leaving basic variable necessarily retains its zero coefficient. Making x4 an entering basic variable now would only lead back to the third tableau. (Check this.)

Summary of the Simplex Method So far we have dealt with the maximization case. In minimization problems, the optimality condition calls for selecting the entering variable as the nonbasic variable with the most positive objective coefficient in the objective equation, the exact opposite rule of the maximization case. This follows because max Z is equivalent to min ( Z ). As for the feasibility condition for selecting the leaving variable, the rule remains unchanged. Optimality condition. The entering variable in a maximization (minimization) problem is the nonbasic variable having the most negative (positive) coefficient in the Z-row. Ties are broken arbitrarily. The optimum is reached at the iteration where all the Z-row coefficients of the nonbasic variables are nonnegative (nonpositive). Feasibility condition. For both the maximization and the minimization problems, the leaving variable is the basic variable associated with the smallest nonnegative ratio (with strictly positive denominator). Ties are broken arbitrarily. Gauss-Jordan row operations 1. Pivot row a. Replace the leaving variable in the Basic column with the entering variable. b. New pivot row = Current pivot row Pivot element 2. All other rows, including Z New row = (Current row) - (pivot column coefficient) (New pivot row) The steps of the simplex method Step 1.Determine a starting basic feasible solution. Step 2. Select an entering variable using the optimality condition. Stop if there is no entering variable; the last solution is optimal. Else, go to step 3. Step 3. Select a leaving variable using the feasibility condition. Step 4. Determine the new basic solution by using the appropriate Gauss-Jordan computations. Go to step 2.

Sources: Hillier, F.S. and G.J. Lieberman. 2001. Introduction to Operations Research. 7th ed. Taha, Hamdy A. 2007.Operations Research: An Introduction. 8th ed.