Linear Programming - Simplex, Big-M and Duality

Ritajit Majumdar
April 23, 2017

Linear programming is the problem of minimizing or maximizing a linear cost function, subject
to linear equality and inequality constraints.

• First studied by Dantzig on around 1947 (end of World War).

• Koopmaus coined the term “Linear Programming” in 1948.

• Simplex method was published by Dantzig in 1949.

1 Basic definitions and preliminary mathematics

Definition 1. Let a be a non-zero vector in Rn and b be a scalar

i. The set {x ∈ Rn |a′ x = b} is called a hyperplane

ii. The set {x ∈ Rn |a′ x ≥ b} is called a half-space

The intersection of many half-spaces is called a polyhedron.

Definition 2. A polyhedron is a set that can be described in the form {x ∈ Rn |Ax ≥ b}, where A
is an m × n matrix and b is a vector in Rm

A polyhedron can either extend to infinity or can be confined in a finite region.

Definition 3. A set S ⊂ Rn is bounded if there exists a constant k such that the absolute value of
every component of every element of S is less than or equal to k.

Definition 4. A set S ⊂ Rn is convex is for any x, y ∈ S and any λ ∈ [0, 1], we have λx+(1−λ)y ∈ S.

If λ ∈ [0, 1], then λx + (1 − λ)y is a weighted average of the vectors x, y and therefore belongs to
the line segment joining x and y. Thus a set is convex if the segment joining any two of its elements
is contained in the set.

Theorem 1. The intersection of convex sets is convex and every polyhedron is a convex set.

Definition 5. Let P be a polyhedron. A vector x ∈ P is an extreme point if it is not possible to

find two vectors y, z ∈ P , both different from x and a scalar λ ∈ [0, 1] such that x = λy + (1 − λ)z.

Theorem 2. Let P be a non-empty polyhedron and let x∗ ∈ P . Then the following are equivalent

i. x∗ is a vertex

ii. x∗ is an extreme point

iii. x∗ is a Basic Feasible Solution

1
2 Showing that a given set is convex
(usually for 3 marks)
Problem 1. Prove that the set S = {u ∈ Rn |u.v ≤ 9} is a convex set.

Ans. Let u, w ∈ S and λ ∈ [0, 1]. By definition of convex set, the point λu + (1 − λ)w should
belong to S. So we have

(λu + (1 − λ)w).v = λ.u.v + (1 − λ).w.v

≤ λ.9 + (1 − λ).9 = 9

since u, v, w ∈ S and by problem u.v ≤ 9 and w.v ≤ 9. Thus

(λu + (1 − λ)w).v ≤ 9

Hence S is convex.

Problem 2. Show that X = {x : |x| ≤ 2} is a convex set. [WBUT 2011]

Ans. Let |u|, |v| ∈ X and λ ∈ [0, 1]. By definition of convex set, the point λ|u| + (1 − λ)|v| should
also belong to X. So we have

λ|u| + (1 − λ)|v| ≤ λ.2 + (1 − λ)2 = 2

since |u|, |v| ∈ X and by problem |u| ≤ 2 and |v| ≤ 2. Thus

λ|u| + (1 − λ)|v| ≤ 2

Hence X is convex.

Problem 3. Show that the set S = {(x1 , x2 ) : x21 + x22 ≤ 4} is a convex set. [WBUT 2013]

Ans. Let xi , xj ∈ S and λ ∈ [0, 1]. By definition of the problem and the definition of convex set,
the point λ(x2i + x2j ) + (1 − λ)(x2i + x2j ) should belong to S. So we have

λ(x2i + x2j ) + (1 − λ)(x2i + x2j ) ≤ λ.4 + (1 − λ).4 = 4

since xi , xj ∈ S and by the problem x2i + x2j ≤ 4. Thus

λ(x2i + x2j ) + (1 − λ)(x2i + x2j ) ≤ 4

Hence S is convex.

2
3 Simplex method
3.1 Graphical method to simplex solution
Maximize Z = x1 + x2
Subject to −x1 + x2 ≤ 1
x1 ≤3
x2 ≤ 2
x 1 , x2 ≥ 0

It requires some effort to draw the graph in LATEX, hence not trying to do it. Follow the class for
the graph. The table for this problem is as follows -
Extreme points Coordinates (x1 , x2 ) Z = x1 + x2
A (0,0) 0
B (0,1) 1
C (1,2) 3
D (3,2) 5
E (3,0) 3

When the number of variables increase, it is not possible to draw graph for it. Furthermore,
it is known that the optimal solution is always at the vertex. The number of vertices increases
exponentially with the dimension. Hence, storing the value for each vertex requires exponential
amount of space. Simplex method is an algebraic process to find the optimal solution. The idea of
the algorithm can be stated in four simple points :

i. An optimal solution is located at a vertex of the feasible region.

ii. A vertex is a Basic Feasible Region (BFS).

iii. One can move from one BFS to a neighbouring BFS.

iv. One can detect that a solution is optimal.

The standard form of an LPP is

Max Z = cT x
Subject to Ax ≤ b
x≥0

where A is the matrix containing the coefficients

  of the decision
 variables
 in the constraints. In
    1 −1 1
1 x1
the above problem, c = ,x= , b = 3 and A =  1 0. Consider an LP problem
1 x2
2 0 1
with n number of decision variables and m number of constraints (m < n, except for degenerate cases)

Max Z = c1 x1 + c2 x2 + · · · + cn xn
Subject to a11 x1 + a12 x2 + · · · + a1m xm ≤ b1
.. ..
. .
am1 x1 + am2 x2 + · · · + amm xm ≤ bm

Consider that the constraints are equations. Using Gaussian elimination method, keep m of the
decision variables to the left and express them in terms of other decision variables.

3
 Pn
x1 = b1 − i=m+1 a1i xi
..
P.n
x m = bm − i=m+1 a1i xi

The variables on the right hand side are called non-basic variables and they are all assigned to 0.
Thus the variables on the left hand side take the corresponding b-values (i.e. xi = bi ) and are called
basic variables. So the assigned values to the set of decision variables can be denoted as
S
{xi = bi |1 ≤ i ≤ m} {xi = 0|m + 1 ≤ i ≤ n}
If a basic variable also satisfies the non-negativity constraints, i.e., xi = bi and xi ≥ 0, then the
solution is called Basic Feasible Solution.

Slack variables
Simplex method allows only equalities in constraints. For inequality constraints, we can add some
non-negative variable to change each inequality into equality. Such variables are called slack vari-
ables. For ≥ type inequalities, we need to subtract some non-negative variable. Such a variable is
called surplus. We shall see more about it in the section on Big-M method.

In the problem considered at the beginning of this section, we can add some slack variables to
get equations -

−x1 + x2 + x3 =1
x1 +x4 =3
x2 +x5 = 2
 
−1 1 1 0 0
where x3 , x4 , x5 are slack variables. So the matrix A is modified as A =
′  1 0 0 1 0 .
0 1 0 0 1
Note that the last 3 columns of the matrix A forms a 3 × 3 identity matrix. Each
′
column of an
identity matrix are independent.
Definition 6. X and Y are considered to be linearly independent if to satisfy the equation αX +
βY = 0, where α, β are scalar, it is necessary to have α = β = 0.
   
1 0
Putting X = and Y = , the equation is satisfied iff α = β = 0. Hence, here X and
0 1
Y are linearly independent. Can X and Y be linearly independent if (i) both are real numbers, (ii)
both are complex numbers, (iii) one is real and the other one is complex?
Definition 7. Dimension of a space is the minimum number of linearly independent variables re-
quired to uniquely identify a point in that space.
Definition 8. Rank of matrix A, R(A) = min{dimension of the column-space of A, dimension of
the row-space of A}
Column-space consists of the number of elements in each column of the matrix. Similar is the
definition of row-space. Hence, rank of matrix A′ , rank(A′ ) = min{dim(col-space(A)), dim(row-
space(A))} = min{3, 5} = 3.

In a general LP problem, if there are n decision variables and rank of matrix A (coefficients of
variables in the constraints) is m, define B := m-element subset of {1,2, . . . , n} and AB :=sub-matrix
of A indexed by B.

4
First Simplex Tableau
Consider again the problem stated at the beginning of the section. In the modified matrix A′ , the
last 3 columns are definitely linearly independent since they form the 3 × 3 identity matrix. So
choose B = {3, 4, 5}.

– Variables corresponding to set B are the basic variables.

– Set all the non-basic variables to 0.

– in the tableau, keep the basic variables to the left and the non-basic variables to the right (this
holds for all tableau).

x3 = 1 − x2 + x1
x4 = 3 − x1
x5 = 2 − x2

Z = x1 + x2 = 0 (x1 = x2 = 0 since they non-basic variables)

B.F.S. : x1 = 0, x2 = 0, x3 = 1, x4 = 3, x5 = 2.
(x1 = 0, x2 = 0 corresponds to point A in the table)

Second Simplex Tableau

To increase the value of Z, we need to increase the value of one of the non-basic variables. We can
increase either x1 or x2 . Let us choose to increase x2 .

– While increasing x2 , care should be taken to satisfy the non-negativity constraints for each of
the variables.

– From the 3rd equation in the previous tableau, we can increase x2 upto 2 such that the condition
x5 ≥ 0 is satisfied.

– However, in the first equation, x1 = 0. So x2 can be at most 1 to satisfy non-negativity

constraint for x3 .

– It is said that the value of x2 is bounded by x3 . Replace 3 by 2 in the set B, i.e. B = {2, 4, 5}.

x2 = 1 + x1 − x3
x4 = 3 − x1
x5 = 1 − x1 + x3

Z = 1 + 2x1 − x3 = 1 (x1 = 0, x3 = 0. It is useful to write Z in terms of non-basic vari-

ables, as we shall see in 3rd tableau)

B.F.S. : x1 = 0, x2 = 1, x3 = 0, x4 = 3, x5 = 1.
(x1 = 0, x2 = 1 corresponds to point B in the table)

Third Simplex Tableau

We shall again try to increase the value of one of the non-basic variables. However, check that in the
previous tableau, x3 has a negative sign in the equation for Z. So if we increase x3 , Z will decrease.
So x1 will be increased. Here x1 is bounded by x5 and we can increase x1 to be 1. Hence B = {1, 2, 4}.

5
 x1 = 1 + x3 − x5
x2 = 2 − x5
x4 = 2 − x3 + x5

Z = 3 + x3 − 2x5 = 3 (x3 = 0, x5 = 0 since they are non-basic variables)

B.F.S. : x1 = 1, x2 = 2, x3 = 0, x4 = 2, x5 = 0.
(x1 = 1, x2 = 2 corresponds to point C in the table)

Fourth Simplex Tableau

Now we choose to increase x3 since increasing x5 will decrease the value of Z. The value of x3 is
bounded by x4 and x3 can be increased up to 2. Hence B = {1, 2, 3}.

x1 = 3 − x4
x2 = 2 − x5
x3 = 2 − x4 + x5

Z = 5 − x4 − x5 = 5 (x4 = 0, x5 = 0 since they are non-basic variables)

B.F.S. : x1 = 3, x2 = 2, x3 = 2, x4 = 0, x5 = 0.
(x1 = 3, x2 = 2 corresponds to point D in the table)

Note here, in the equation of Z, both x4 and x5 have negative sign. Hence no further increase of
these variables are possible. Increasing any one of the variables will decrease the value of Z. This is
the stopping criteria of the Simplex Algorithm.

Hence, the solution is - Max Z = 5 and x1 = 3, x2 = 2.

3.2 Sensitivity Analysis : Shadow Price

Dealing with sensitivity analysis is beyond the scope of syllabus. However, we shall mention what
shadow price is for future reference. Consider in the previous problem, the RHS of the second con-
straint equation (x1 + x4 = 3) changes from 3 to 4, i.e. 1 unit increase. How does this 1 unit increase
affect the optimal solution of the objective function?

Note that the equation x1 + x4 = 4 (RHS increased by 1 unit) is equivalent to x1 + (x4 − 1) = 3,

i.e., increasing the RHS by 1 unit is same as decreasing the slack variable by 1.

In the final tableau, the value x4 was 0. So increasing the RHS by 1 is equivalent to allowing x4
to take the value -1. Then the objective function Z = 5 − x4 − x5 increases by 1 and takes value 6.
So if x4 = −1, then the objective function increases by 1, which is called the shadow price.
Definition 9. The shadow price associated with a particular constraint is the change in the optimal
value of the objective function per unit increase in the right hand side value for that constraint, all
other problem data remaining unchanged.

– Shadow prices are associated with the constraints of the problem and not the variables.
– The shadow price for a particular constraint is the negative of the coefficient of the appropriate
slack (or artificial) variable in the objective function of the final tableau.
– In the problem considered, the shadow price for the first constraint is 0. For the second and
third constraints, shadow price is 1.
6
3.3 Solutions to some simplex problems
(Usually 6-9 marks)
Problem 4. Solve the following LPP using Simplex method [WBUT 2011]
Maximize Z = x1 − x2
Subject to x1 + 2x2 ≤ 4
6x1 + 2x2 ≤ 9
x 1 , x2 ≥ 0

Ans. First we need to formulate the LP problem in its standard form, i.e. the constraints will be
equalities only. The standard form of the problem is

Maximize Z = x1 − x2 + 0.s1 + 0.s2

Subject to x1 + 2x2 + s1 =4
6x1 + 2x2 +s2 = 9
x1 , x2 , s1 , s2 ≥ 0

where s1 and s2 are slack variables. Like before, since the coefficients of the slack variables form
an Identity matrix, whose columns are always linearly independent, we shall start with them. In
the following table, column B (in some book it is written as xB ) contains the basic variables (in the
first table s1 , s2 ), cB contains the coefficients of the basic variables in the objective function (in the
first table coefficients of both s1 , s2 are 0) and b contains the RHS values of the constraint equations.
The row cj contains the coefficients of the variables (both decision variables and slack variables) in
the objective function.

cj 1 −1 0 0
cB B b x1 x2 s1 s2 Min ratio
0 s1 4 1 2 1 0 4
3
0 s2 9 6 2 0 1 2
→
Z=0 zj 0 0 0 0
cj − z j 1↑ -1 0 0

zj for each column is the sum of the product of each element of that column with the corresponding
cB value in that row, i.e., for the column x1 , zj = 1 × 0 + 6 × 0. After finding out the values of zj
for each column, we find out the values of cj − zj for each column.

– The value of the objective function is the summation of the product of the values of decision
variables and its coefficient.
– The variables in the column B are the decision variables and are assigned the value of the
corresponding element of the column b, while all other decision variables are non-basic and
assigned the value 0.
P
– Here the value of Z is cB .b = 4 × 0 + 9 × 0 = 0.
– Consider the largest non-negative value of cj − zj . Give an upward arrow for that value and
call it the entering column. Here 1 is the only positive value for cj − zj , hence x1 is the entering
column.
– Note that some books follow the convention of taking the smallest negative value of zj − cj .
– The entering column is also called the pivot column. Here x1 is the pivot column.
– Calculate the value of b/(element of pivot column) for each row and put the value in the
column Min ratio. In this table, the values are 14 = 4 and 96 = 32 .
7
 – Take the minimum non-negative value in the Min ratio column. Give an arrow mark to it and
call it the outgoing row. So in this table, x1 is the entering column and s2 is the outgoing row.

– The element at the intersection of entering column and outgoing row is called the pivot element
and is marked with a box.

– In the second table, the rows will be as follows -

– The column B will contain s1 and x1 , since x1 replaced s2 . The column cB will contain
the coefficients of these variables in the objective function i.e. 0 and 1.
– The column Min ratio is not filled up now, since its calculation will be done after the
table is complete.
– In the row containing the pivot element, the new values of all the remaining columns will
be simply its old value divided by the value of the pivot element, i.e.
R2 (new) = R2 (old) ÷ 6(pivot)
– In all the remaining rows, consider the element in the column which contains the pivot
element, i.e. here the column x1 contains the pivot element. So for the first row, we
consider the value of the element directly above (or below) the pivot element, here, 1
(marked with a circle).
– Multiply the new row (i.e. the row containing pivot element in the previous table) with
this element (here 1.R2 (new)) and subtract this value from the old row, i.e.
R1 (new) = R1 (old) − 1.R2 (new)
– Note again that the value 1 is because it is the value in the first row above the pivot
element. The value here will change according to the value of the element above or below
the pivot element.

The second simplex table is as follows -

cj 1 −1 0 0
cB B b x1 x2 s1 s2 Min ratio
5 5
0 s1 2
0 3
1 − 61
3 1 1
1 x1 2
1 3
0 6
3 1 1
Z= 2
zj 1 3
0 6
cj − z j 0 - 34 0 - 61

The value of Z is, as before cB .b = 0 × 25 + 1 × 23 = 32 . Here all the values of cj − zj ≤ 0. Thus it

is not possible to choose a non-negative value for entering column. This is the stopping criteria
of the Simplex algorithm. Note that, if you follow the convention of zj − cj , then the algorithm
terminates when all values of zj − cj ≥ 0.

The variables under the column B are basic while all other variables are non-basic. The basic
variables are assigned the corresponding b value while the non-basic variables are assigned the value
0. So the optimal solution is
3
Z= 2
and the decision variables x1 = 32 , x2 = 0

Problem 5. Solve the following LP problem by Simplex method [WBUT 2012]

8
 Maximize Z = x1 + x2 + 3x3
Subject to 3x1 + 2x2 + x3 ≤ 3
2x1 + x2 + 2x3 ≤ 2
x 1 , x2 , x3 ≥ 0

Ans. The LP problem in its standard form is as follows -

Maximize Z = x1 + x2 + 3x3 + 0.s1 + 0.s2

Subject to 3x1 + 2x2 + x3 + s1 ≤3
2x1 + x2 + 2x3 + s2 ≤ 2
x1 , x2 , x3 , s1 , s2 ≥ 0

where s1 , s2 are the slack variables. The first simplex tableau can be written as follows -

cj 1 1 3 0 0
cB B b x1 x2 x3 s1 s2 Min ratio
0 s1 3 3 2 1 1 0 3
0 s2 2 2 1 2 0 1 1→
Z=0 zj 0 0 0 0 0
cj − z j 1 1 3↑ 0 0

– We start with the slack variables s1 , s2 under column B and cB contains their coefficients. All
the coefficients of the variables are written in the row cj .
P
– As before Z = cB .b = 3 × 0 + 2 × 0.

– Recall, we need to choose the greatest non-negative value of cj − zj , which is equal to 3 here.
So 3 marks the entering column.

– We need to choose the minimum non-negative value of the ratio of the elements of b divided by
the elements of x3 (entering column). Here 1 is the minimum value. Hence s2 is the outgoing
row.

– The pivot element is the element at the intersection of the incoming column and outgoing row.
The pivot element, 2, is marked by a box.

– In the second tableau, the elements of the row corresponding to the pivot element will be

R2 (new) = R2 (old) ÷ 2 (pivot element)

– For row 1, we consider the value of the element directly above the pivot element, here 1 (marked
with a circle). The elements of the first row will be

R1 (new) = R1 (old) − 1.R2 (new)

The second simplex tableau can be written as follows -

cj 1 1 3 0 0
cB B b x1 x2 x3 s1 s2 Min ratio
3
0 s1 2 2 2
0 1 - 21
1 1
3 x3 1 1 2
1 0 2
3 3
Z=3 zj 3 2
3 0 2
cj − z j -2 - 12 0 0 - 23
9
 Note, here all the values of cj − zj ≤ 0. Hence this simplex algorithm terminates here. The
optimal solution is

Z = 3 and the decision variables x1 = 0 , x2 = 0 and x3 = 1

By looking at the constraints, can you argue that this is the best assignment of values for the
decision variables? Can you convince yourself that decreasing x3 and giving the other two decision
variables positive values will not be a better assignment?

Problem 6. Solve the following LP problem by Simplex method [WBUT 2013, 2015]

Minimize Z = −2x2 − x3
Subject to x1 + x2 − 2x3 ≤ 7
−3x1 + x2 + 2x3 ≤ 3
x 1 , x 2 , x3 ≥ 0

Ans. This problem is interesting in two ways - (i) It’s constraints contain three variables, x1 , x2 , x3
whereas the objective function contains only 2, (ii) the objective is to minimize the function, but
simplex works only for maximize type of objective.

If there is a value p which we need to decrease, it is same as increasing the value of −p. For
example, crudely speaking, going up a hill is same as going along the -ve direction of down the hill.
So instead of minimizing Z, we can maximize −Z. Then this problem is solvable by the simplex
method. Note here, for simplex algorithm, we need maximization type objective function and “≤”
type constraints. If a constraint is “≥” type, then either use Big-M method or multiply both sides
of the constraint by -1 to make it “≤” type.

First, we change the problem type to maximization as follows -

Maximize −Z = 2x2 + x3
Subject to x1 + x2 − 2x3 ≤ 7
−3x1 + x2 + 2x3 ≤ 3
x 1 , x2 , x3 ≥ 0

In the standard form, the objective function should contain all the decision variables as well as
slack variables. So the LP problem in its standard form is -

Maximize −Z = 0.x1 + 2x2 + x3 + 0.s1 + 0.s2

Subject to x1 + x2 − 2x3 + s1 =7
−3x1 + x2 + 2x3 + s2 ≤ 3
x1 , x2 , x3 , s1 , s2 ≥ 0

The first simplex tableau can be written as follows -

cj 0 2 1 0 0
cB B b x1 x2 x3 s1 s2 Min ratio
0 s1 7 1 1 -2 1 0 7
0 s2 3 -3 1 2 0 1 3→
Z=0 zj 0 0 0 0 0
cj − z j 0 2↑ 1 0 0

– Here 2 is the maximum non-negative value of cj − zj . So x2 is the entering column.

10
 – Dividing the elements of b by the elements of x2 (entering column) and store the values in the
Min ratio column.

– The minimum of the values is 3, corresponding to the row s2 . So it is the outgoing row. The
pivot element is 1, marked by a box.

– New row R2 is given as

R2 (new) = R2 (old) ÷ 1 (pivot element)

– The new row R1 is given by

R1 (new) = R1 (old) - 1.R2 (new)

this “1” is marked by a circle, the element above the pivot element in column x2 .

The second simplex tableau is as follows -

cj 0 2 1 0 0
cB B b x1 x2 x3 s1 s2 Min ratio
0 s1 4 4 0 -4 1 -1 1→
2 x2 3 -3 1 2 0 1 -1
Z=6 zj -6 2 4 0 2
cj − z j 6↑ 0 -3 0 -2

– cj − zj has only one positive entry 6. Hence x1 is the entering row.

– Recall that we choose the minimum non-negative value of Min ratio. So we choose 1 over -1
and hence s1 is the outgoing column.

– Here the pivot element is in the first row. New row R1 is

R1 (new) = R1 (old) ÷ 4 (pivot element)

– For the second row, we choose the element of x1 below the pivot element, here -3 (marked with
circle). So new row R2 is

R2 (new) = R2 (old) - (-3).R1 (new)

The third simplex tableau is as follows -

cj 0 2 1 0 0
cB B b x1 x2 x3 s1 s2 Min ratio
1
0 x1 1 1 0 -1 4
- 41 -1
3 1
2 x2 6 0 1 -5 4 4
- 56
3 1
Z = 12 zj 0 2 -10 2 2
cj − z j 0 0 11 ↑ - 32 - 21

Note here that the result is not optimal, since all the cj − zj values are not ≤ 0. However,
both the Min ratios are negative, whereas we stated previously that we shall consider the minimum
non-negative value for the min ratio, so it is not possible to choose an outgoing row. This condition
shows that the solution is unbounded.
11
Problem 7. Write the LPP in its standard form [WBUT 2009]

Maximize Z = x1 − 3x2 + 5x3

Subject to x1 + x2 + x3 ≤ 7
x1 − x2 + x3 ≥ 2
3x1 − x2 + 2x3 = −5
x1 , x2 ≥ 0 and x3 unrestricted in sign

Ans. The general structure of simplex problem is

Max Z = cT x
Subject to Ax ≤ b
x≥0

Here all the constraints are not of ≤ type. For equality, if a = b, then we can write that a ≤ b
and a ≥ b. If a is both less than equal to and greater than equal to b, then a and b must be equal.
So we can write the third constraint as two inequalities. Hence the problem becomes

Maximize Z = x1 − 3x2 + 5x3

Subject to x1 + x2 + x3 ≤ 7
x1 − x2 + x3 ≥ 2
3x1 − x2 + 2x3 ≤ −5
3x1 − x2 + 2x3 ≥ −5
x1 , x2 ≥ 0 and x3 unrestricted in sign

Now, for the constraints with ≥ sign, multiply both sides by -1. Then the sign will flip. So the
problem becomes

Maximize Z = x1 − 3x2 + 5x3

Subject to x1 + x2 + x3 ≤ 7
−x1 + x2 − x3 ≤ 2
3x1 − x2 + 2x3 ≤ −5
−3x1 + x2 − 2x3 ≤ 5
x1 , x2 ≥ 0 and x3 unrestricted in sign

Non-negativity constraint is an essential component of LP problems. But here x3 is unrestricted

in sign and hence can take up either positive or negative value. Hence, we need to change the prob-
lem accordingly such that the decision variables are non-negative.
′′ ′′ ′′ ′′
Let us write x3 = x′3 − x3 where x′3 , x3 ≥ 0. If x3 ≥ 0 then we can choose x′3 ≥ x3 or x′3 < x3 if
′′
x3 < 0. This allows to have x′3 , x3 ≥ 0. So the problem can be re-written as -
′′
Maximize Z = x1 − 3x2 + 5x′3 − 5x3
′′
Subject to x1 + x2 + x′3 − x3 ≤ 7
′′
−x1 + x2 − x′3 + x3 ≤ 2
′′
3x1 − x2 + 2x′3 − 2x3 ≤ −5
′′
−3x1 + x2 − 2x′3 + 2x3 ≤ 5
′′
x1 , x2 , x′3 , x3 ≥ 0

Now the problem can be written in standard form by introducing slack variables

12
 ′′
Maximize Z = x1 − 3x2 + 5x′3 − 5x3 + 0.s1 + 0.s2 + 0.s3 + 0.s4
′′
Subject to x1 + x2 + x′3 − x3 + s1 =7
′′
−x1 + x2 − x3 + x3
′
+ s2 =2
′′
3x1 − x2 + 2x′3 − 2x3 + s3 = −5
′′
−3x1 + x2 − 2x′3 + 2x3 + s4 = 5
′′
x1 , x2 , x′3 , x3 , s1 , s2 , s3 , s4 ≥ 0

Degeneracy
Problem 8. Solve the following LP problem using Simplex method
Maximize Z = 2x1 + x2
Subject to 4x1 + 3x2 ≤ 12
4x1 + x2 ≤ 8
4x1 + 2x2 ≤ 8
x 1 , x2 ≥ 0

Ans. The problem in its standard form can be written as follows -

Maximize Z = 2x1 + x2 + 0.s1 + 0.s2 + 0.s3

Subject to 4x1 + 3x2 + s1 = 12
4x1 + x2 + s2 =8
4x1 + 2x2 + s3 = 8
x1 , x2 , s1 , s2 , s3 ≥ 0

We write the Simplex tableau as follows -

cj 2 1 0 0 0
cB B b x1 x2 s1 s2 s3 Min Ratio
0 s1 12 4 3 1 0 0 3
0 s2 8 4 1 0 1 0 2
0 s3 8 4 2 0 0 1 2
Z=0 zj 0 0 0 0 0
cj − z j 2↑ 1 0 0 0

Check that there is a tie between s2 and s3 to be the outgoing row. This is called degeneracy.
We can have a better understanding of degeneracy by plotting the constraint equations.

13
 The equations 4x1 + x2 = 8, 4x1 + 2x2 = 8 define the corner point (2, 0). However, if we choose
x2 = 0 and 4x1 + x2 = 8, then these two automatically satisfy 4x1 + 2x2 = 8. Therefore, the slack
variable s3 associated with 4x1 + 2x2 = 8 must be 0 (since it is already satisfied by the other two
equations). This accounts for the appearance of the degenerate basic variable s3 in the simplex table.
Similarly, we can find other degeneracies. Hence

– Graphically, degeneracy is a manifestation of redundancy in information. While only x2 = 0

and 4x1 + x2 = 8 were sufficient to define the corner point (2, 0), the equation 4x1 + 2x2 = 8
also defines the same point.

– Algebraically, degeneracy occurs if a basic variable takes value 0 in any simplex tableau. We
argued earlier that if we consider the equations x2 = 0 and 4x1 + x2 = 8, then it automatically
satisfies 4x1 + 2x2 = 8 and hence the slack variable s3 , associated with the later equation, must
be 0. But it is a basic variable since it is in the B column of the simplex tableau.

To approach such a situation, we can choose to consider any of the two rows (s2 , s3 ) as the
outgoing row. However, there is a risk that the algorithm will become slow. We have stated in
previous sections that the simplex algorithm jumps from one corner point to the next corner point
of the feasible region. However, here 4x1 + x2 = 8, 4x1 + 2x2 = 8 and x2 = 0 define a single corner
point. So there are three combinations of constraints which define the same corner point. This will
cause the simplex problem to remain in the same point for three consecutive steps, thus making the
algorithm slow. So we consider the following steps -

– Rearrange the simplex tableau so that the identity matrix comes first (Recall that the coeffi-
cients of the slack variables form an identity matrix).

– Calculate min{(elements of the first column)/(corresponding elements of the entering column)}.

Consider only those rows which are candidates for being outgoing row, not the others.

– The row having the minimum value becomes the outgoing row.

cj 0 0 0 2 1
s1 s2
cB B b s1 s2 s3 x1 x2 Min ratio x1
Min ratio x1
0 s1 12 1 0 0 4 3 - -
1
0 s2 8 0 1 0 4 1 0 4
0 s3 8 0 0 1 4 2 0 0→

We do not consider the Min ratio for the first row since it is not a potential candidate for be-
coming outgoing row. In the first Min ratio column xs11 , there is a tie once more. So we consider the
second Min ratio column xs21 . Now, since s3 has the minimum value, it becomes the outgoing row. If
there were a tie here too, then we would have considered Min ratio xs31 .

The second simplex tableau is as follows -

cj 2 1 0 0 0
cB B b x1 x2 s1 s2 s3 Min Ratio
0 s1 4 0 1 1 0 -1
0 s2 0 0 -1 0 1 -1
1 1
1 x1 2 1 2
0 0 4
1
Z=4 zj 2 1 0 0 2
cj − z j 0 0 0 0 - 21

Since all cj − zj ≤ 0, the algorithm terminates here. The optimal solution is Z = 4, the decision
variables x1 = 2, x2 = 0 and x3 = 0.
14
4 Duality in Linear Programming
We shall try to demonstrate the idea of duality in linear programming with an example. Consider
the following LP problem -

Maximize Z = 2x1 + 3x2

Subject to 4x1 + 8x2 ≤ 12
2x1 + x2 ≤ 3
3x1 + 2x2 ≤ 4
x 1 , x2 ≥ 0

The original problem is called the primal. Let Z ∗ be the optimal solution of this problem. We
want to find a lower bound on this solution i.e. can we answer the question - Is Z ∗ at least α? If we
can find an α such that Z ∗ ≤ α, then α is an upper bound on the optimal value. Mathematically
stating, out objective is to derive an inequality of the form

d1 x1 + d2 x2 ≤ α, d1 ≥ 2, d2 ≥ 3

with α as small as possible. α is a good bound if we can claim

2x1 + 3x2 ≤ d1 x1 + d2 x2 ≤ α

The values of the decision variables x1 and x2 are bounded by the constraint inequalities. Multi-
plying both sides of the first, second and third constraints by y1 , y2 and y3 respectively where each
yi ∈ Rn and adding up the constraints, we get

(4y1 + 2y2 + 3y3 )x1 + (8y1 + y2 + 2y3 )x2 ≤ 12y1 + 3y2 + 4y3

This is of the form d1 x1 + d2 x2 ≤ α where d1 = 4y1 + 2y2 + 3y3 , d2 = 8y1 + y2 + 2y3 and
α = 12y1 + 3y2 + 4y3 . We wish to minimize α given d1 ≥ 2, d2 ≥ 3. Writing this in mathematical
form, we get an LP problem as follows -

Minimize Z ′ = 12y1 + 3y2 + 4y3

Subject to 4y1 + 2y2 + 3y3 ≥ 2
8y1 + y2 + 2y3 ≥ 3
y 1 , y2 , y3 ≥ 0

This is called the dual of the original problem. Note that if we represent the original problem
(primal ) as
Max Z = cT x
Subject to Ax ≥ b
x≥0

where c, b, x and A have the same meaning as explained in Section 3, then its dual can be repre-
sented as
Max Z ′ = bT y
Subject to Ay ≥ c
y≥0
 
y1
 y2 
where y =  .. 
 
.
yn

15
 Primal Dual
1. Objective: Minimize Objective: Maximize
2. Objective: Maximize Objective: Minimize
3. j number of variables j number of constraints
4. i number of constraints i number of variables
5. Variable xj is unrestricted in sign Constraint j is of “=” type
6. Constraint i is of “=” type Variable yi is unrestricted in sign

4.1 Solutions to some duality problems

(Usually for 5 marks)
To find dual of an LP problem, if the objective of the primal is of Maximization type, then all the
constraints should have “≤” sign and if the primal is of Minimization type, then all the constraints
should have “≥” sign.

Problem 9. Find the dual of the following LP problem

Minimize Z = 3x1 − 2x2 + 4x3
Subject to 3x1 + 5x2 + 4x3 ≥ 7
6x1 + x2 + 3x3 ≥ 4
7x1 − 2x2 − x3 ≤ 10
x1 − 2x2 + 5x3 ≥ 3
4x1 + 7x2 − 2x3 ≥ 2
x 1 , x2 , x 3 ≥ 0

Ans. Note, here the objective is minimization, but all the constraints are not of “≥” type. Mul-
tiplying the third constraint by −1 on both sides will change the sign of inequality. The primal
problem in standard form is as follows -

Minimize Z = 3x1 − 2x2 + 4x3

Subject to 3x1 + 5x2 + 4x3 ≥ 7
6x1 + x2 + 3x3 ≥ 4
−7x1 + 2x2 + x3 ≥ −10
x1 − 2x2 + 5x3 ≥ 3
4x1 + 7x2 − 2x3 ≥ 2
x 1 , x 2 , x3 ≥ 0

There are 5 constraints, hence there will be 5 decision variables in the dual problem. Let
y1 , y2 , y3 , y4 , y5 be the dual variables corresponding to the five primal constraints. Hence dual of
the primal LP problem is

Maximize Z ′ = 7y1 + 4y2 − 10y3 + 3y4 + 2y5

Subject to 3y1 + 6y2 − 7y3 + y4 + 4y5 ≤ 3
5y1 + y2 + 2y3 − 2y4 + 7y5 ≤ −2
4y1 + 3y2 + y3 + 5y4 − 2y5 ≤ 4
y 1 , y 2 , y 3 , y 4 , y5 ≥ 0

Problem 10. Find the dual of the following LP problem [WBUT 2015]
Minimize Z = x1 + x2 + x3
Subject to x1 − 3x2 + 4x3 = 5
x1 − 2x2 ≤3
16
 2x2 − x3 ≥ 4
x1 , x2 ≥ 0, x3 is unrestricted in sign

Ans. Here the first constraint is of “=” type. If two numbers a and b are equal, i.e. a = b, then
we can write it as a ≤ b and a ≥ b. Since two numbers cannot be lesser and greater than each other
at the same time, they must be equal. Furthermore, since x3 is unrestricted in sign, we can represent
′′ ′′
it as x3 = x′3 − x3 where x′3 , x3 ≥ 0. So we can write the problem as
′′
Minimize Z = x1 + x2 + x′3 − x3
′′
Subject to x1 − 3x2 + 4x′3 − 4x3 ≤ 5
′′
x1 − 3x2 + 4x′3 − 4x3 ≥ 5
x1 − 2x2 ≤3
′′
2x2 − x3 + x3 ≥ 4
′
′′
x1 , x2 , x′3 , x3 ≥ 0

Since this is a minimization type problem, all the constraints should be “≥” type. We can change
the constraints accordingly by multiplying both sides with -1. The resulting set of constraints are
′′
−x1 + 3x2 − 4x′3 + 4x3 ≥ −5
′′
x1 − 3x2 + 4x′3 − 4x3 ≥ 5
−x1 + 2x2 ≥ −3
′′
2x2 − x′3 + x3 ≥ 4
′′
x1 , x2 , x′3 , x3 ≥ 0

There are 4 constraints. So there will be 4 decision variables in the dual. Let y1 , y2 , y3 , y4 be the
decision variables in the dual problem. Hence the dual will be

Maximize Z ′ = −5y1 + 5y2 − 3y3 + 4y4

Subject to −y1 + y2 − y3 ≤1
3y1 − 3y2 + 2y3 + 2y4 ≤ 1
−4y1 + 4y2 − y4 ≤ 1
4y1 − 4y2 + y4 ≤ −1
y 1 , y2 , y3 , y4 ≥ 0

Now, in the original problem, there were 3 constraints, so there should be 3 decision variables in
the dual. Let y = y2 − y1 . So the problem can be written as

Maximize Z ′ = 5y − 3y3 + 4y4

Subject to y − y3 ≤1
−3y + 2y3 + 2y4 ≤ 1
4y − y4 ≤ 1
−4y + y4 ≤ −1
y3 , y4 ≥ 0, y is unrestricted in sign.

Furthermore, the last two constraints can be rewritten as

4y − y4 ≤ 1
4y − y4 ≥ 1

If a ≥ b and a ≤ b, then a must be equal to b. So the complete dual formulation is

Maximize Z ′ = 5y − 3y3 + 4y4

17
 Subject to y − y3 ≤1
−3y + 2y3 + 2y4 ≤ 1
4y − y4 = 1
y3 , y4 ≥ 0, y is unrestricted in sign.

Problem 11. Find the dual of the following LPP [WBUT 2014]

Minimize Z = 2x1 + 7x2 + 5x3

Subject to 2x1 + 5x2 + 7x3 ≤ 17
3x1 + 2x2 + 5x3 = 13
5x1 + 3x2 + x3 ≤ 9
x1 , x3 ≥ 0, x2 is unrestricted in sign

Ans. Here the second constraint is “=” type. So we can write the constraints as

2x1 + 5x2 + 7x3 ≤ 17

3x1 + 2x2 + 5x3 ≤ 13
3x1 + 2x2 + 5x3 ≥ 13
5x1 + 3x2 + x3 ≤ 9
x1 , x3 ≥ 0, x2 is unrestricted in sign

Since, this is a minimization type problem, all the constraints should be “≥” type. So multiplying
the “≤” type constraints with -1 on both sides

−2x1 − 5x2 − 7x3 ≥ −17

−3x1 − 2x2 − 5x3 ≥ −13
3x1 + 2x2 + 5x3 ≥ 13
−5x1 − 3x2 − x3 ≥ −9
x1 , x3 ≥ 0, x2 is unrestricted in sign
′′ ′′
Since x2 is unrestricted in sign, we can replace x2 = x′2 − x2 where x′2 , x2 ≥ 0. Rewriting the
problem, we have
′′
Minimize Z = 2x1 + 7x′2 − 7x2 + 5x3
′′
Subject to −2x1 − 5x′2 + 5x2 − 7x3 ≥ −17
′′
−3x1 − 2x′2 + 2x2 − 5x3 ≥ −13
′′
3x1 + 2x′2 − 2x2 + 5x3 ≥ 13
′′
−5x1 − 3x′2 + 3x2 − x3 ≥ −9
′′
x1 , x′2 , x2 , x3 ≥ 0

Since there are 4 constraints in the primal, the dual will consist of 4 decision variables. Let the
decision variables be y1 , y2 , y3 , y4 . Hence the dual will be

Maximize Z ′ = −17y1 − 13y2 + 13y3 − 9y4

Subject to −2y1 − 3y2 + 3y3 − 5y4 ≤ 2
−5y1 − 2y2 + 2y3 − 3y4 ≤ 7
5y1 + 2y2 − 2y3 + 3y4 ≤ −7
−7y1 − 5y2 + 5y3 − y4 ≤ 5
y 1 , y2 , y 3 , y 4 ≥ 0

There were 3 constraints in the original primal problem. So the dual problem should have 3 deci-
sion variables. Since the variables y2 , y3 were obtained by replacing the 2nd equality type constraint
18
in two inequality type constraints, we can recombine these two variables. Let y = y3 − y2 . Then the
dual problem becomes

Maximize Z ′ = −17y1 + 13y − 9y4

Subject to −2y1 + 3y − 5y4 ≤ 2
−5y1 + 2y − 3y4 ≤ 7
5y1 − 2y + 3y4 ≤ −7
−7y1 + 5y − y4 ≤ 5
y1 , y4 ≥ 0, y is unrestricted in sign.

Furthermore, the variable x2 was unrestricted in sign in the primal problem. So the 2nd constraint
in the dual, corresponding to this variable should be of “=” type. Multiplying the 3rd constraint
the the dual by -1, we get

Maximize Z ′ = −17y1 + 13y − 9y4

Subject to −2y1 + 3y − 5y4 ≤ 2
−5y1 + 2y − 3y4 ≤ 7
−5y1 + 2y − 3y4 ≥ 7
−7y1 + 5y − y4 ≤ 5
y1 , y4 ≥ 0, y is unrestricted in sign.

Here, the second and third constraints are of type a ≥ b and a ≤ b. This is possible only when
a = b. So we can replace these two inequalities with an equality type constraint. The final dual
problem is

Maximize Z ′ = −17y1 + 13y − 9y4

Subject to −2y1 + 3y − 5y4 ≤ 2
−5y1 + 2y − 3y4 = 7
−7y1 + 5y − y4 ≤ 5
y1 , y4 ≥ 0, y is unrestricted in sign.

Reference
1. Talk by Dr. Arijit Bishnu at the workshop “Introductory Lectures on Optimization” held at
Indian Statistical Institute, Kolkata

2. Introduction to Linear Optimization by Dimitris Bertsimas and John N. Tsitsiklis

3. Online course on “Discrete Optimization” offered by the University of Melbourne at www.coursera.org

19