1

Linear programming

Examples

(1) Maximize Z= x1+3x2

Subject To
-x1 + x2 ≤ 1 ………….(1)
x 1 + x2 ≤ 2 …………..(2)
x1, x2 ≥ 0
-x1 + x2 = 1

Solution: x1= 1/2

x2 = 3/2 4

x1 + x2 =2
(2) Minimize Z

Solution: x1=0 x2=0 2 1

C=5
C=0 2
C=3
(3) Removing the constraint : x1 + x2 ≤ 2

x2
unbounded

x1

Max Z is unbounded

3
Objective function

Z=2x1+2x2
Max occurs at (x1=1/2, x2=3/2)

& (x1=2, x2=0) & all linear combinations

Four possibilities:

(1) No solution
(2) Unbounded solution
(3) Finite optimal solutions
(4) Infinite no. of optimal solutions

4
The Simplex method will deal with all solutions. Why Simplex ?

Beyond two dimensions, the graphical solution can’t be used.

Standard form of a linear programming problem

Find x= (x1,…………….,xn) to minimize

n
Z = cjxj (3)
j=1

Subject to
n
∑ aij x j =bi ….(4) i=1,2,……..m
j=1

xj≥0 …...(5) j=1,………..n

5
In matrix form

Min cx
Subject To
Ax=b
x≥0

A is a m x n matrix

Basic theorems of Linear Programming

Generalization of the idea from 2 to n dimensions

Definition 1:

A feasible solution to the linear programming problem is a vector

x=(x1,x2,……xn) which satisfies the equations (4) & the non-negativity
constraints (5)

6
Definition 2:

A basis matrix is an m x m non-singular matrix formed from some m columns

of the constraint matrix A

Definition 3:

A basic solution to a linear programming problem is the unique vector

determined by choosing a basis matrix , setting the (n-m) variables associated
with columns of A not in the basis matrix equal to zero & solving the resultant
square non-singular system of equations for the remaining m variables

7
Definition 4:

A basic feasible solution is a basic solution in which all variables have non
negative values
(Note: By definition 3, at most m variables can be positive)

Definition 5:

A non degenerate basic feasible solution is a basic feasible solution with exactly
m positive .xi

Definition 6:

An optimal solution is a basic feasible solution which also minimizes Z

in (3)

8
Example:

Consider fuel mix problem:

Converted into standard from the constraints look like

-x1+x2+x3 =1
x1+x2 +x4=2
x i≥ 0 i=1,…..4

x3, x4 slack variables

A= -1 1 1 0
1 1 0 1

b= 1
2 The matrix B = 1 0
0 1

9
Formed from column 3&4 is a basic matrix
The corresponding basic solution

Bx=b

Will give x1=0, x2=0, x3=1, x4=2

This is a non degenerate basic feasible solution

B= -1 0 from column 1& 4

1 1
Is also a basic matrix
The corresponding basic solution is obtained by setting x2= x3= 0 & solving
-x1 =1
x1 +x4=2

Yielding x1= -1 x4=3

This basic solution is not feasible
10
Theorem 1:

The objective function Z assumes its min at an extreme point of the constraint
set. If it assumes its min at more than one extreme point, then it takes the same
value at every point of the line segment joining any two optimal extreme
points.

Theorem 2:

A vector x=(x1,x2,………xn) is an extreme point of the constraint set of an LP

problem if x is a basic feasible solution of the constraint set

n variable, m constraint problem

How many extreme points?

=  n 
 m
11
The simplex method usually takes between m & 2m step to arrive at the solution
It is an iterative procedure

Consider the system of equation:

5x1+3x2=3 ……………(1)
10x1+7x2=20 ……………… (2) ……..I

To solve we multiply equation (1) by 2

10x2+6x2=6
10x1+7x2=20 …………………II
& subtract given from equation (2)
X2=14
10x1+7x2=20 ………………………III
& finally we get
X2= 14
X1= -7.8 ……………………………IV

(I), (II), (III), (IV) are equivalent

12
Two sets of equation are said to be equivalent if they have the same solution
set. Hence the following operations transform a given linear system into an
equivalent system

(1) Multiply an equation Et by a constant k≠0

(2) Replacing any equation Et by the equation Et + K Ei

where Ei is any other equation in the system

13
 INTUITIVE INTRODUCTION TO L.P.

Example: Consider the formulated problem

Min Z = x1+6x2-7x3+x4+5x5
s.t. 5x1-4x2+13x3-2x4+5x5=20
x1-x2+5x3-x4+x5=8
-------------------------------------------------------------------------------------
Augment it with x1+6x2-7x3+x4+5x5-Z =0
Objective function
xj ≥ 0 j=1,2,………5

Use x5, x1, & -Z as the basic variables

Pivoting we get
x5 -1/4x2 +3x3 -3/4x4 =5
x1 -3/4x2 +2x3- 1/4x4 =3
----------------------------------------------------------------------------------------
-Z +11x2 -25x3 -1/4x4 =-28
14
A basic solution is
x5=5 x1=3 x2=x3=x4=0 & Z=28

This is a basic solution & is an extreme point

NOTE: If x1& x2 & -Z had been chosen

x1=-12 x2=-20 x3=x4= x5= 0 & Z= -132

Basic solution not feasible

C3 = -24
Z = 28-24x3
Hence if x3 ↑ can improve solution

x5 = 5-3x3
x1 = 3-2x3
x3 = 3/2

15
New B.F.S pivoting on (2x3)

x5 =1/2 x3 =3/2 x2=x3=x4=0 & Z = -8

x5 -3/2x1 +7/8x2 -3/8x4 =1/2

x3 +1/2x1 -3/8x2- 1/8x4 =3/2
-Z +12x1 -x2 +2x4 =8

x5 =1/2-7/8x2
x3 = 3/2 + 3/8x2
Z= -8 –x2
X2 -12/7 x1 – 3/7x4 + 8/7x5 = 4/7
X3 -1/7x1 - 2/7x4 + 8/7x5 = 12/7
-Z +72/7 x1 + 11/7x4 +8/7x5= 60/7

x2 =4/7 x3 =12/7 x1=x4=x5 =0 Z=-60/7

16
Problem 1:

Max x1+x2 =Z
s.t. x1+3x2≤1
3x1+x2≤1
x1, x2≥0
(0,1)
(a) Show the feasible region
(b) Find all basic solutions
(c) Find all B.F.S.
(d) Find the optimal solution by:
(i) graphical method
(ii) using the B.F.S (0,1/3)
(iii) simplex method (1/4,1/4)
(e) Are any solutions Degenerate?
(1,0)
(0,0)
(1/3,0)
17
Solution using Simplex method

Iteration 1:

x1 x2 x3 x4 -Z t
1 3 1 0 0 1

3 1 0 1 0 1
1 1 0 0 1 0

18
Iteration 2:

x1 x2 x3 x4 -Z t
0 8/3 1 -1/3 0 2/3
1 1/3 0 1/3 0 1/3
0 2/3 0 -1/3 1 -1/3

19
Iteration 3:

x1 x2 x3 x4 -Z t
0 1 3/8 -1/8 0 1/4
1 0 -1/8 3/8 0 1/4
0 0 -1/4 -1/4 1 -1/2
This is the optimal solution

20
Canonical system:

Assume that the first m equations of the linear system of equations from a
basic matrix B i.e.

nX
mA =b
n
m[ B, B ] X = b

−1
Then pre-multiplying both side by B yields
−1 n −1
m [ I,B B ] x=B b

In this transformed but equivalent system the coefficients of the variables

(x1…,xm) are an identity matrix.

21
Such a system is called canonical & has the form shown below

Canonical system with basic variables (x1… xm)

Dependent (basic) Independent (non basic) variables

variables

x1 + ā1,m+1xm+1 + ā1,m+2xm+1 + ……………+ ā1,nxn = Ь1

x2 + ā2,m+1xm+1 + ā2,m+2xm+1 + ……………+ ā2,nxn = Ь2
.
.
.
.
xm +ām,m+1xm+1 + ām,m+2xm+1 + ………..…+ ām,nxn = Ьm

22
All Ь’s are always ≥ 0

(x1, x2……xm) are dependent because by fixing values of other x’s their value
gets fixed

In particular if xm+1…….xn =0

We get

x 1 = Ь1 x2 = Ь2 ………….xm= Ьm
& this is a B.F.S

23
Def.

If one or more of the Ьi's are zero, the solution is degenerate.

To start the simplex method we must first get the system of equations into their
canonical form (in other words we must get the B.F.S.- an extreme point)
This extreme point is used to begin the Simplex method

Step by step procedure for Simplex algorithm:

(1) Convert system of equations into the std. form

(2) Augment the constraint equation set with the objective function put in the
equation form i.e.
-Z + c1x1 + c2x2 + …………….cnxn = 0

24
(3) the simplex algorithm can begin with this augmented system is in canonical
form i.e.

The B.F.S corresponding to this is

Z = Z, x1 = b1...........x m = b m , x m+1 = x m+2 = ...........x n = 0

25
Test for optimality


Theorem 3: A B.F.S. is an optimal solution with total cost Z if all constants

c j are ≥ 0 j=m+1,……..n
c j are called the relative cost factor
(4) Improve solution by choosing

out of rows corresponding to the sth column which have positive coefficients
corresponding to xs choose that row ‘r’ such that

If all a’s in that column have –ve

coefficients,
we get an unbounded solution

These two checks give us the sth column and rth row which enable us to locate
the pivot term.

(5) Pivot on the pivot term & reduce to new B.F.S and repeat 26
 REVISED SIMPLEX METHOD

Much information of simplex tableau is not used. Only the following are
needed
(1) Relative cost factorcj

We compute

(2) Assuming c s < 0 we require elements of updating column

Ps = {a1s ,......................a ms }' & the value of the basic variables
xB = {bs ,.......................bm }'
Using these the quantity

is calculated & a pivot operation is performed on a rs

Only one column (non-basic) p s is required
27
LP problem Columns >> Rows

More efficient procedure – first generate c j & then p s

Revised simplex method does just that

LP Problem in column form

Min Z = c1x1+ ………..+cnxn
S.t. P1x1+P2x2+………+Pnxn = b ……………(1)
xj ≥ 0 i = 1, 2,……..

Consider B = [Pj1 Pj2……….Pjm]

xB = (xj1 xj2……… xjm)’
cB= (cj1 cj2………..cjm)
Then BxB = b

Whose solution xB = B 1b = b

Assume feasible basic B
i.e. xB ≥ 0
28
Consider augmented system
^
P j = {a1 j ,........amj ,c j }' j = 1............n
^
P n+1 = {0,0......................1}'
^
b = {b1 ..............bm ,0}'
n ^ ^ ^

P x +P
j=1
j j n+1 ( z) = b

^ ^ ^ ^
consider B = [ P j1 ,............ P jm , P n+1 ] =  B 0
 
^ 1 c B 1 
B obviously =  B 1 0
 

  c B
B 1
1
 29
Def. The row vector

π =(π1……………. πm) = cBB-1

is called vector of simplex multipliers with the basis B

Multiply system (1) by π1, π2, …….. πm & sum and subtract from the
Z equation

Coefficient of xi becomes cj- πPj

Set cj- πPj = 0 j basic

We get πB = cB

Whose solution is the simplex multiplier vector cBB-1

−1
Multiplying system by B gives

30
 x j1 = b1




x jm +  Pj x j = bm
non basic
Z+  c j x j = Z 0
non basic

p   B1 0   p j 
 j =    
c   π 1  c j 

 j 
    
Update column is p j = B 1 p j

cj = cj π pj 31
Relative cost factor is
 Only B1 and π are needed to perform the simplex iteration

How to bring in Ps – remove Pjr

^ ^ ^
P j1 .......... ..... P jm P n + 1  u1.......... u m + 1  a rs
a rs
am

righ

    

32
 −1
Multiply by B , we get
u1 .......... ..... u r .......... .......... u m+1  B 1  a rs

a rs

a ms

cs

righ

[] 

     
Pivot on ārs yield
 ^ 1 
u1........ u r 1 , α,u r+1 .......... u m+1  B new | u r 
 

33
 Summary of Revised Simplex Method
^ 1
At any iteration assume current basis matrix is
B
associate solution is xB = B-1b
& the data of the original problem A, b & c are available.

The new cycle proceeds as follows

^ 1
(1) Row(m+1) of B has the form
^ 1m+1
B = (-π1,-π2……….-πm,1)

where πi are simplex multipliers

For each non-basic variables from the relative cost factor c j using the equation
m
c j = c j   πi aij = c j  π p j
i=1

 
or cB = cB  cB B 1.B
The pricing out of column Pj
34
(2) Std column selective rule

(3) If cs ≥ 0 stop → the current basis is optimal

(4) cs < 0 compute transformed column

 Ps  = B 1  Ps 
   
  c 
 cs   s
(5) Let
Ps = ( a1s , a 2s ................ams )'
If all a is  0 stop
→ the optimum is unbounded
35
(6) Otherwise compute

(7) Construct the augmented matrix

 ^ 1 Ps 
B 
 
 cs 
& transform it by pivoting on the first m+1th column of the result containing
the inverse of the new basis
Update basis solution by
xB i  xB i  Θais i  r
xB i Θ
& return to step 1

36
Max Z = x1+3x2
S.t. -x1+x2≤1
x1+x2≤2
x1≥0 , x2 ≥ 0
^ ^
Solution: A =  1 +1 +1 0 0  b = 1 
   
+1 +1 0 +1 0   2
 1  3 0 0 +1 0

^ ^ 1
B= I = B

37
pivoting element are circled

Step :- 1

1 0 0 | 1 pivoting 1 0 0 |1
0 1 0 1 1 1 0 0
0 0 1 3 +3 0 1 0
1
 New B

Non basic variable are x1 & x2

Hence the relative cost factor c  c (new) →the last row of new  matrix
1 3
would be
The last row of the new multiplied by the P1 & P3
(the first & third column of the old Â)

38
 c1 = + 3 0 1  1 = 4
 
+1
 1
c3 = + 3 0 1 1  = +3
 
0 
0

Solve for final answer

39
Symmetric primal dual pair

Primal Dual

Min ćx Max πb
St. Ax ≥ b St. A’π’ ≤ c
x≥0 π≥0

Primal in standard form

Min ćx Max πb
St. Ax = b St. A’π’ ≤ c
x≥0 π unconstrained in sign

40
 Primal Quantity Corresponding dual Quantity
Objective c’x→ min Objective πb→ max
Variable xj ≥ 0 Constraint π pj ≤ cj (inequality)
Variable xj unconstrained in sign Constraint π pj = cj (equality)
Constraint Aix = bi (equality) Variable πj unconstrained in sign
Constraint Aix ≥ bi (inequality) Variable πj ≥ 0
Coefficient matrix A Coefficient matrix A’
Right hand side b Right hand side c
Cost coefficient c Cost coefficient b

As you may have noticed there is a direct connection between the variable of
any one of the two problems with the constraints of the other

41
 The most general Primal Primal dual formulation
n n
Min Z= ∑ c i x i Max v= ∑ π i bi
S .t. i=1 i=1
S .t.
EU E = 1,2,.....n m
a ij x j = bi i  E π a i ij = cj j  p
a ij x j  bi i  E i=1
m
  
π a i ij  cj j  p
PU P = 1,2,....n
i=1

πi  0 i  E
xi  0 i  P
πi unconstrained in
xi unconstrained
sign i 
in sign i  p
   
42
 Or in matrix form

Min Z = c' x
Max v = πb
' '  
  s.t . A π   c
s.t . Ax  b  
 
π  0 iE
x  b i p i
i

43
The Dual constraints have the structure

a1,1π1 + a2,1π2 + …………….a20,1 π20 ≤ c1

a1,2π1 + a2,2π2 + …………….a20,2 π20 ≤ c2

Here we need only a 2 x 2 basis matrix.

Theorem 1:
If x & π are feasible primal & dual solution, then

Z = c' x > πb = v
Proof : By feasibility hypothesis
x  0, Ax = b, A' π'  c
Then c' x  πb,
∴ Z v 44
Theorem 2:
If both primal & dual have feasible solution & both have optimum
solutions, then
min Z = max v

Proof: As primal objective is bounded below & dual objective above

(theorem 1), both have optimum solutions

To show equality of objective values,

let xo solve the primal with optimal basis Bo & vector of basic variables xB0
Thus, B0 xB0 = b, xB0 ≥ 0

The Simplex multipliers associated with the problem:

π0 = cB(B0)-1
Since x0 is optimal

c j (relative cost factor) = c j π o p j  0

for all j 45
Or in matrix form
A’(π0)’ ≤ c
Thus π0 satisfies dual const. & has objective value
1
v 0 = π 0 b = cB (B 0 ) b = cB xB0 = Z 0

since this is an upper band for v, π0 solve the dual & the theorem is prove

z Z

Min Z = Max v

v v

46
Theorem 3:
If either primal or dual prob. has an unbounded solution then the other prob. is
infeasible.
Proof:
Assume primal unbounded
Then by theorem
π b ≤ - ∞ for all dual feasible π
Existence of a solution to the dual constraints would imply a finite value of
dual objective contradicting the above.
Hence dual is feasible

Dual Feasible Infeasible

Primal Max c’x=min πb c’x → -∞

feas.
Infeas. πb → +∞ possible

47
Theorem 4:
A pair (x0, π0) with x0 & π0 primal and dual feasible respectively, are optimal if
and only if:
(c’ – π0 A) x0 = 0………(1)
Proof: For any primal feasible x ,
A x = b .....................( 2 )
c' x = z .....................( 3 )
Multiply (2) by some dual feasible & subtract the result from (3) to get,
c' x  π A x = z  πb
or, setting πb = v
(c' π A)x = z  v
If (x0, π0)
are primal dual optimal,
Then z0 – v0 = 0 = (c’ – π0A)x0
Conversely if (x0, π0) satisfy
(c’ – π0 A)x0 = z0 - v0 = 0
Then by theorem 2, z attains lower bounds at x0 & v upper bound at π0.
So x0, & π0 are optimal

(c’ – π0A) x0 = 0 is called the complementary slackness condition.

48
 DUAL SIMPLEX METHOD

It was shown that

(c’ – π0A)x0 = z0 –v0 = 0
(The complementary slackness condition)

For optimality

→ Positive variable in primal solution implies the corresponding constraint holds with
equality while a slack constraint implies zero level for the corresponding variables.

There are three optimality conditions:

(1) primal feasible

(2) dual feasible
(3) complementary slackness

Most algorithms enforce two of these conditions throughout while relaxing the third,
enforcing it as the iteration progresses:
49
 Algorithm Primal feasibility Dual feasibility Complementary
Slackness

Simplex method Enforced Relaxed Enforced

Dual Simplex X √ √

Primal Dual X √ √

The later two methods make use of different means to reduce primal
infeasibility. The dual simplex method increases the value of dual objective at
each step while the primal dual algorithm minimizes the infeasibility form.

50
 DUAL SIMPLEX METHOD

Two cases where it is very useful

Example:
Min Z
s.t. x1 + 3x3 + x4 = 1
x2 – x3 + 5x4 = 2 …………(1)
(-Z) + 2x3 + x4= -3
Already optimal
Introduce additional constraints
x1 + x2 ≤ 2
Or x 1 + x2 + s = 2

Subtracting first two equations of (1) from above, we get

s - 2x3 – 6x4 = -1
The basic solution is infeasible
Since s = -1

But the relative cost factor are all non-negative → Dual feasible
(This is necessary condition to start dual simplex method)
51
Case 1:
Constraints added to a problem whose solution is known

x1 x2 s x3 x4 (-Z) b

1 0 1/6 8/3 0 0 5/6

0 1 5/6 -8/3 0 0 7/6

0 0 -1/6 1/3 1 0 1/6

0 0 1/6 5/3 0 1 -19/6

52
Case 2:
A linear programming is given whose solution for a number of different right
hand side vector bj is desired.
It would be insufficient to apply the two phases for each bj .Desired solution
for b’ may be close to that for b2
Procedure- Solve for b’ obtaining an optimal basis B if this is feasible for all
other right hand sides i.e. if
B-1 bj ≥ 0 all j
Then it is optimal for these as well
If not so that
B-1 br < 0 for some r
Then π = cB B-1
is dual feasible since
c j =c j − πp j ≥ 0
is independent of RHS
only primal feasible is not there.

53
 DUAL SIMPLEX METHOD

It is precisely the simplex method applied to the dual but constructed so as to

operate with in the standard primal simplex tableau.

Step wise description.

Initial problem in canonical form

All c j  0 (dual feasibilit y)

basic c j = 0 (complementary slackness)
not all b j  0 (primal feasibilit y relaxed)

54
(1) Choose row ‘r’ as pivot row where

br = min bi < 0
(2) Choose column ‘S’ as pivod column, where

cj
Choosethe min of
 a rj
If all a rj > 0  primal is in feasible
(3) Pivot on ars

(4) If bi ≥ 0 all i
Stop → optimal
Otherwise return to 1
55