Linear Programming
Example: maximize x1 + x2, subject to the following constraints:
4x1 - x2 ≤ 8 2x1 + x2 ≤ 10 5x1 - 2x2 ≥ -2 x1 ≥ 0 x2 ≥ 0Feasible solution: values of x1, x2 that satisfy the constraints. Feasible region and objective function of the problem:
Many problems, such as single-source shortest path and maximum flow, can be converted into LP problems.
maximize cTx (cT is the transpose of c) subject to Ax ≤ b, x ≥ 0Consequently, a LP problem in standard form is fully specified by (A, b, c). Any linear program can be converted into the standard form by coefficient negation (minimizing becomes maximizing, and "≤" exchanges with "≥"), variable substitution (every variable can be replaced by the difference of two non-negative variables), and equality-to-inequality-pair replacement (see textbook).
A slack variable defined as s = b - Ax can turn the inequality into an equation, plus the equality s ≥ 0. As a result, the linear program changes from standard form to slack form:
z = v + cTx s = b - Ax x ≥ 0 s ≥ 0where z is the variable to be maximized, and v is an optional constant. We can also append s into x, while using two index sets to distinguish them: B for s ("basic variables") and N for the original x ("nonbasic variables"). Consequently, a LP problem in slack form is fully specified by (N, B, A, b, c, v).
The simplex algorithm is described in Section 29.3, and in the following it will be explained using a concrete example.
The following LP problem is in standard form:
maximize 3x1 + x2 + 2x3
subject to x1 + x2 + 3x3 ≤ 30
2x1 + 2x2 + 5x3 ≤ 24
4x1 + x2 + 2x3 ≤ 36
x1, x2, x3 ≥ 0
convert the problem into slack form:
z = 3x1 + x2 + 2x3 x4 = 30 - x1 - x2 - 3x3 x5 = 24 - 2x1 - 2x2 - 5x3 x6 = 36 - 4x1 - x2 - 2x3A solution (x1, ..., x6) is feasible if and only if all of the 6 values are nonnegative. The basic solution (0, 0, 0, 30, 24, 36) can be obtained by setting all nonbasic variables to be 0. This solution is feasible, and the value of the objective function for this solution is 0.
In each iteration, the simplex algorithm does a "pivoting", i.e., reformulating the problem to get a better basic solution by selecting a nonbasic variable whose coefficient in the objective function is positive, and turning it into a basic variable. This is the "entering" variable, and the "leaving" (basic) variable is the first one to become negative when the entering variable increases.
When x1 is selected as the entering variable, the leaving variable is x6. The last equation above can be reformulated into
x1 = 9 - x2/4 - x3/2 - x6/4Using it to substitute the other x1 on the right hand side:
z = 27 + x2/4 + x3/2 - 3x6/4 x1 = 9 - x2/4 - x3/2 - x6/4 x4 = 21 - 3x2/4 - 5x3/2 + x6/4 x5 = 6 - 3x2/2 - 4x3 + x6/2Now the basic solution is (9, 0, 0, 21, 6, 0), and the objective function value is 27.
Do a pivot again to let x3 enter and x5 leave:
z = 111/4 + x2/16 - x5/8 - 11x6/16 x1 = 33/4 - x2/16 + x5/8 - 5x6/16 x3 = 3/2 - 3x2/8 - x5/4 + x6/8 x4 = 69/4 + 13x2/16 + 5x5/8 - x6/16Now the basic solution is (33/4, 0, 3/2, 69/4, 0, 0), and the objective function value is 111/4.
Do a pivot again to let x2 enter and x3 leave:
z = 28 - x3/6 - x5/6 - 2x6/3 x1 = 8 + x3/6 + x5/6 - x6/3 x2 = 4 - 8x3/3 - 2x5/3 + x6/3 x4 = 18 - x3/2 + x5/2Now all coefficients in the objective function are negative, so the optimal value is z = 28, which corresponds to solution (8, 4, 0, 18, 0, 0), or x = (8, 4, 0)T in the original form of the problem.
If in a LP problem the feasible solutions corresponds to a distortion of an n-dimensional cube, the worst-case complexity of the simplex algorithm may reach Θ(2n). Nevertheless, it is usually remarkably efficient in practice.