Indian Institute of Information Technology Design and Manufacturing,

Instructor
Kancheepuram, Chennai 600 127, India
N.Sadagopan
An Autonomous Institute under MHRD, Govt of India
Scribe:
http://www.iiitdm.ac.in
Dhanalakshmi S
COM 209T Design and Analysis of Algorithms -Lecture Notes

Greedy Algorithms
Objective: This module focuses on yet another paradigm, namely, greedy algorithms.

Greedy algorithms solve optimization problems by making the best choice (local optimum) at each step.
We shall look at the knapsack problem in various perspectives and we solve them using greedy technique.
Note that a greedy algorithm do not always yield optimal solutions, but a feasible solution. For example, if
the given optimization problem is a minimization problem, then the solution obtained by greedy is always
greater than or equal to the optimal solution and if it is a maximization problem, then the solution obtained
by greedy is always less than or equal to the optimal solution. This calls for a proof of correctness if greedy
indeed works. In general,
Heuristics + Proof of Correctness = Greedy Algorithms

1 0/1 Knapsack
Given a set of n objects, say x1 , x2 , . . . , xn , along with its weight w1 , w2P
, . . . , wn and profits p1 , p2 , . . . , pn .
The objective is to find S ⊆ {x1 , x2 , . . . , xn } such that P rof it(S) = pi is maximum subject to the
P xi ∈S
constraint wi ≤ W , where W is the capacity of the knapsack. This problem is called as 0/1 knapsack
xi ∈S
because for all xi , either xi = 1 (i.e., the object xi is included in the knapsack) or xi = 0 (i.e., the object
xi is not included in the knapsack). Recall that this problem can be solved using the dynamic programming
paradigm in pseudo-polynomial time (O(nW )). Note that if W = 2n , then the algorithm runs in exponential
in n. We shall now try a greedy approach for the given problem and typically greedy algorithms run in
polynomial time.
Greedy with respect to weight:

Step 1: Sort the weights in increasing order, say w1 ≤ w2 ≤ . . . ≤ wn . This step takes O(n log n)
time.
Step 2: Pick the minimum weight (local optimum), which is w1 in this case. Check w1 ≤ W . If so, add x1
to S.
Step 3: Check w1 + w2 ≤ W . If so, add x1 and x2 to S.
Step 4: Proceed this process until we get the least j such that w1 + w2 + . . . + wj > W . Now return
S = {x1 , . . . , xj−1 }.
The Steps 2-4 take O(n). Overall, the above approach takes O(n log n) time.P
The solution obtained by this greedy approach is a feasible solution, since, wi ≤ W .
xi ∈S
Query: Is the solution obtained by greedy w.r.t weight is optimum ?
Solution: No. Here is a counter example:
The sorted weights and the corresponding profits are as follows
Given the weight of the knapsack is 5, our algorithm picks the first two objects as an output (Since,

Objects x1 x2 x3 x4
Weights 1 2 4 5
Profits 2 2 3 5

w1 + w2 + w3 = 1 + 2 + 4 = 7 > W and w1 + w2 = 1 + 2 = 3 ≤ W ) and the corresponding profit is 2 + 2 = 4.

However, the optimum solution is {x4 } and the profit is 5. Thus, the greedy with respect to weight does not
yield optimum always.
The greedy with respect to weight fails because in some cases the objects chosen by the algorithm gives less
profit than any optimum solution. So, let us try another greedy approach which is greedy with respect to
profit.
Greedy with respect to profit:

Step 1: Sort the weights in decreasing order, say p1 ≥ p2 ≥ . . . ≥ pn . Let w1 , w2 , . . . , wn be the cor-
responding weights for p1 ≥ p2 ≥ . . . ≥ pn .
Step 2: Pick the maximum profit (local optimum), which is p1 in this case. Check w1 ≤ W . If so, add x1
to S.
Step 3: Check w1 + w2 ≤ W . If so, add x1 and x2 to S.
Step 4: Proceed this process until we get the least j such that w1 + w2 + . . . + wj > W . Now return
S = {x1 , . . . , xj−1 }.
P
The solution obtained by this greedy approach is a feasible solution, since, wi ≤ W .
xi ∈S
Query: Is the solution obtained by greedy w.r.t profit is optimum ?
Solution: No. Here is a counter example:
The sorted profits and the corresponding weights are as follows

Objects x1 x2 x3 x4
Profits 4 3 2 1
Weights 5 1 4 5

Given the weight of the knapsack is 5, our algorithm outputs S = {x1 } (Since, w1 + w2 = 5 + 1 = 6 > W
and w1 = 5 ≤ W ) and the corresponding profit is 4. The optimum solution is {x2 , x3 } and the profit is 5.
Thus, greedy with respect to profit does not give optimum solution always.
Both the greedy with respect to weight and the greedy with respect to profit fail, because in some cases
the objects chosen by the algorithm gives less profit than any optimum. So, now let us try yet another
greedy approach with respect to profit per unit weight(profit/weight).

Greedy with respect to profit/weight:

Step 1: Sort the weights in decreasing order, say p1 /w1 ≥ p2 /w2 ≥ . . . ≥ pn /wn .
Step 2: Pick the maximum pi /wi (local optimum), which is p1 /w1 in this case. Check the corresponding
w1 ≤ W . If so, add x1 to S.
Step 3: Check w1 + w2 ≤ W . If so, add x1 and x2 to S.
Step 4: Proceed this process until we get the least j such that w1 + w2 + . . . + wj > W . Now return
S = {x1 , . . . , xj−1 }.
P
The solution obtained by this greedy approach is a feasible solution, since, wi ≤ W .
xi ∈S
Query: Is the solution obtained by greedy w.r.t profit/weight is optimum ?
Solution: No. Here is a counter example:
The sorted weights and the corresponding profits are as follows

Objects x1 x2 x3
Profits 7 9 6
Weights 3 5 4
Profit/weight 2.33 1.8 1.5
Given the weight of the knapsack is 5. Our algorithm picks the first object as an output (Since, w1 + w2 =

2
3 + 5 = 8 > W and w1 = 3 ≤ W ) and the corresponding profit is 7. The optimum solution is {x2 } and the
profit is 5. Thus, the above greedy with respect to profit/weight does not give optimum always.
Fine tuning profit/weight:
Now, let us try to change our strategy slightly by picking the objects as follows. Instead of terminating the
algorithm when w1 + w2 + . . . + wj > w for least j, neglect wj and proceed with wj+1 , and check whether
the constraint is satisfied or not, terminate this process once all the objects are scanned.
Query: Is the solution obtained by greedy approach 2 w.r.t profit/weight is optimum ?
Solution: No. Here is a counter example:
The sorted profit/weight and the corresponding weights and profits are as follows

Objects x1 x2 x3 x4
Profits 7 9 6 1
Weights 3 5 4 1
Profit/weight 2.33 1.8 1.5 1
Given the weight of the knapsack is 5. Our algorithm picks the object x1 and x4 as an output (Since,
w1 + w2 = 3 + 5 = 8 > W , neglect w2 and proceed from w3 , w1 + w3 = 3 + 4 = 7 > W , neglect w3 and
proceed from w4 , w1 + w4 = 3 + 1 = 4 < W ) and the corresponding profit is 8. The optimum solution is
{x2 } and the profit is 9. Thus, greedy with respect to profit/weight is not optimum.

2 Variant Knapsack
Given a set of n objects, say x1 , x2 , . . . , xn , along with its weight w1 , w2 , . . . , wn and profits p1 , p2 ,P
. . . , pn . The
objective is to find S ⊆ {x1 , x2 , . . . , xn } such that | S | is maximum subject to the constraint wi ≤ W ,
xi ∈S
where W is the capacity of the knapsack.

Greedy with respect to weight

Step 1: Sort the weights in increasing order, say w1 ≤ w2 ≤ . . . ≤ wn .
Step 2: Pick the minimum weight (local optimum), which is w1 in this case. Check w1 ≤ W . If so, add x1
to S.
Step 3: Check w1 + w2 ≤ W . If so, add x1 and x2 to S.
Step 4: Proceed this process until we get the least j such that w1 + w2 + . . . + wj > W . Now return
S = {x1 , . . . , xj−1 }.
P
The solution obtained by this greedy approach is a feasible solution, since, wi ≤ W .
xi ∈S
Example: The sorted weights and the corresponding profits are as follows

Objects x1 x2 x3 x4 x5
Weights 1 2 4 5 6
Profits 3 5 6 1 8
Given the weight of the knapsack is 10. Our algorithm picks the objects {x1 , x2 , x3 } as an output (Since,
w1 + w2 + w3 + w4 = 1 + 2 + 4 + 5 = 12 > W and w1 + w2 + w3 = 1 + 2 + 4 = 7 < W ) and | S |= 3.
Optimum Solutions: {x1 , x2 , x5 }, {x1 , x3 , x4 }, {x1 , x2 , x4 } and {x1 , x2 , x3 }. Thus, any optimum solution will
pick exactly three objects from {x1 , x2 , x3 , x4 , x5 }. Interestingly, this greedy strategy gives optimum always
which we shall prove next.

Claim: Greedy variant knapsack indeed returns an optimum solution (OPT).

Proof: Let x1 , x2 , . . . , xn be the set of objects with weights w1 , w2 , . . . , wn and let the weight of the knapsack
be W . Sort w1 , . . . , wn and rename the corresponding objects as a1 , . . . , an . Thus, a1 ≤ a2 ≤ . . . ≤ an .

3
  
a1 a2 . . . ak−1 ak . . . an
Let A = be the output of the algorithm. i.e., S = {a1 , . . . , ak−1 } is
1 1 ... 1 0 ... 0
the solution
 obtained by the algorithm. 
b1 b2 . . . bk bk+1 . . . bn
Let B = be any feasible solution such that b1 = a1 , b2 = a2 , . . . , bn =
1 1 ... 0 1 ... 0
an .

Remark: We work with the binary vector (solution vector) of OPT and the algorithm to show that the
number of 1’s in both are same. It is important to note that the behavior of OPT is unknown to the algo-
rithm, i.e., why did OPT set bk = 0 and bk+1 = 1, and what is only known to the algorithm is the solution
vector output by OPT. P P
We
P shallPshow next that bi = ai . Our proof consists of two parts. In sub claim-1, we show that
bi ≥ ai . Further, P
in sub P
claim-2, we present a stronger claim: for any feasible solution B (which
includes OPT as well) , ai ≥ bi .

Observation 1: There exists a k ∈ {1, . . . , n} such that ai = 1, 1 ≤ i ≤ k − 1 and aj = 0, k ≤ j ≤ n.

P P
Thus, to show that the algorithm is OP T , we need to establish ai ≥ bi . Towards this end, we prove
the following sub claim.
P P
Sub Claim 1: If B is OP T , then bi ≥ ai .
Since, the algorithm outputs a feasible solution and since, the number of 10 s in OP T ≥ the number of 10 s
in any feasible solution, the above claim is true.
P P
Sub Claim 2: If B is any feasible solution, then ai ≥ bi .
Proof of sub claim 2: Proof is by mathematical induction on m, where m is the number of bits in which
A and B differ. P P
Base Case: m = 0. Clearly, ai ≥ bi .
Hypothesis: Assume that the claim is true if they differ in less than m-places, m ≥ 1.
Induction Step: Let A and B differ in (m + 1)-places, m ≥ 0.

Observation 2: Since, A and B differ in (m + 1)-places there exists j such that aj 6= bj . Clearly, it
can not be the case that such a j ∈ {k, . . . , n}. If so, then B is not a feasible solution. This is true because,
if B matches with A in the first (k − 1) places and differ in j ∈ {k, . . . , n}, then clearly B has exceeded the
capacity of the knapsack and hence B is infeasible.
Therefore, there exists j, 1 ≤ j ≤ k − 1 such that aj = 1 and bj = 0.
Find the least j ∈ {1, . . . , k − 1} such that aj = 1 and bj = 0.
Now, set bj = 1 (For example, if A = (1, 1, 1, 1, 0, 0, . . . , 0) and B = (1, 1, 0, 0, 0, 1, . . . , 0), then the modified B
is C = (1, 1, 1, 0, 0, 1, . . . , 0)). By doing so, the modified solution vector of B, say C, may become infeasible.
Case 1: C is feasible P P
The
P numberP of positionsP in Pwhich A and C differs is m. By the hypothesis, ai ≥ ci . Note that,
ci ≥ bi . Thus, ai ≥ bi . Hence, the sub claim is true if C is feasible.
Case 2: C is not feasible
Since C is not feasible, there exists a j ∈ {k, . . . , n} such that aj = 0 and cj = 1. Now, set cj = 0.
This
P implies,
P A and the modified P C,P say D, differsP in (mP − 1)-places and by the induction hypothesis,
ai ≥ di . Observe that di = bi . Thus, ai ≥ bi .
P P
From all the above sub claims, we can conclude that ai = bi . Hence, the greedy variant knapsack is
OP T .

4
3 Fractional Knapsack
Given a set of n objects, say x1 , x2 , . . . , xn , along with its weight w1 , w2 , . . . , wn and
P profits p1 , p2 , . . . , pn .
The objective is to find a subset S ⊆ {f x1 , f x2 , . . . , f xn }, such that P rof it(S) = f xi · pi is maximum
P xi ∈S
subject to the constraint wi ≤ W , where f xi is the fraction of xi and 0 ≤ f xi ≤ 1 (so far, we are allowed
xi ∈S
to include xi or to exclude xi , now, we can choose a fraction of xi which varies between 0 and 1), and W is
the capacity of the knapsack. Now let us try to establish a greedy approach for the given problem as follows
and we will also check whether the solution obtained by the greedy is optimum or not.

Greedy Strategy 1: with respect to weight:

Step 1: Sort the weights in increasing order, say w1 ≤ w2 ≤ . . . ≤ wn .

Step 2: Pick the minimum weight (local optimum), which is w1 in this case. Check w1 ≤ W . If so, add x1
to S.
Step 3: Check w1 + w2 ≤ W . If so, add x1 and x2 to S.
Step 4: Proceed this process until we get the least j such that w1 + w2 + . . . + wj > W . With respect to
j−1
P
W− wi
i=1
wj , calculate f xj = wj . Now return S = {x1 , . . . , xj−1 , f xj }.
P
The solution obtained by this greedy approach is a feasible solution, since, (f xi · wi ) ≤ W .
xi ∈S

Example: The sorted weights and the corresponding profits are as follows

Objects x1 x2 x3
Weights 1 2 5
Profits 1 3 5

Given the weight of the knapsack is 5. Our algorithm first chooses {x1 , x2 } (Since, w1 +w2 +w3 = 2+3+5 =
10 > W and w1 + w2 = 2 + 3 = 5 ≤ W ) and then it chooses f x3 amount of x3 , where, f x3 = 5−(1+2) 5 = 52 .
2 2
Thus, the algorithm outputs (x1 , x2 , x3 ) = (1, 1, 5 ) and the corresponding profit is 1 + 3 + 5 × 5 = 6, which
is same as the optimum.

Query: Is the solution obtained by fractional greedy w.r.t weight is optimum ?

Solution: No. Here is a counter example:
The sorted weights and the corresponding profits are as follows

Objects x1 x2 x3
Weights 2 3 5
Profits 5 8 15

Given the weight of the knapsack is 5. Our algorithm picks the first two objects as an output (Since,
w1 + w2 + w3 = 2 + 3 + 5 = 10 > W and w1 + w2 = 2 + 3 = 5 ≤ W ) and the corresponding profit is 5 + 8 = 13.
The optimum solution is {x3 } and the profit is 15. Thus, greedy with respect to weight is not optimum.

Greedy Strategy 2: with respect to profit:

5
Step 1: Sort profits in decreasing order, say p1 ≥ p2 ≥ . . . ≥ pn . Let w1 , w2 , . . . , wn be the corresponding
weights for p1 ≥ p2 ≥ . . . ≥ pn .
Step 2: Pick the maximum profit (local optimum), which is p1 in this case. Check w1 ≤ W . If so, add x1
to S.
Step 3: Check w1 + w2 ≤ W . If so, add x1 and x2 to S.
Step 4: Proceed this process until we get the least j such that w1 + w2 + . . . + wj > W . With respect to
j−1
P
W− wi
i=1
wj , calculate f xj = wj . Now return S = {x1 , . . . , xj−1 , f xj }.
P
The solution obtained by this greedy approach is a feasible solution, since, (f xi · wi ) ≤ W .
xi ∈S

Example: The sorted profits and the corresponding weights are as follows

Objects x1 x2 x3
Profits 5 1 1
Weights 4 2 1

Given the weight of the knapsack is 5. Our algorithm first chooses {x1 } (Since, w1 + w2 = 4 + 2 = 6 > W
and w1 = 4 ≤ W ) and then it chooses f x2 amount of x2 , where, f x2 = 5−(4) 2 = 12 = 0.5. Thus, the algorithm
1
outputs (x1 , x2 , x3 ) = (1, 1/2, 0) and the corresponding profit is 5 + 2 × 1 = 5.5, where as the OP T chooses
{x1 , x3 } and the corresponding profit is 6.

Query: Is the solution obtained by fractional greedy w.r.t weight is optimum ?

Solution: No. The above example is a counter example:

Greedy Strategy 3: with respect to profit/weight:

Step 1: Calculate the profit/weight for all objects and sort it in decreasing order, say p1 /w1 ≥ p2 /w2 ≥
. . . ≥ pn /wn .
Step 2: Pick the maximum profit/weight (local optimum), which is p1 /w1 in this case. Check w1 ≤ W . If
so, add x1 to S.
Step 3: Check w1 + w2 ≤ W . If so, add x1 and x2 to S.
Step 4: Proceed this process until we get the least j such that w1 + w2 + . . . + wj > W . With respect to
j−1
P
W− wi
i=1
wj , calculate f xj = wj . Now return S = {x1 , . . . , xj−1 , f xj }.
P
The solution obtained by this greedy approach is a feasible solution, since, (f xi · wi ) ≤ W .
xi ∈S

Example: The sorted order of profit/weight, and the corresponding weights and profits are as follows

Objects x1 x2 x3 x4
Profits 7 9 6 1
Weights 3 5 4 1
Profit/weight 2.33 1.8 1.5 1

Given the weight of the knapsack is 5. Our algorithm first picks the object x1 (Since, w1 +w2 = 3+5 = 8 > W
and w1 < W ) and then it picks f x2 , where f x2 = 5−3 2
5 = 5 . Thus the algorithm outputs (x1 , x2 , x3 , x4 ) =
(1, 2/5, 0, 0) and the corresponding profit is 7 + 52 × 9 = 10.6, which is same as optimum solution.

6
Claim: Greedy fractional knapsack indeed returns an optimum solution (OPT).
Proof: Let x1 , x2 , . . . , xn be the set of objects with weights w1 , w2 , . . . , wn and profits p1 , p2 , . . . , pn , respec-
tively. Let the weight of the knapsack be W . Sort p1 /w1 , . . . , pn /wn and without loss of generality assume
that, p1 /w1 ≥ . . . ≥ pn /wn .
Let A = be the output of the algorithm and Let B be the solution output by any OPT algorithm.

Observation 1: (with respect to A) There exist a k ∈ {1, . . . , n} such that aj = 1, 1 ≤ j ≤ k − 1,

ak , 0 ≤ ak ≤ 1 and al = 0, k + 1 ≤ l ≤ n.

The objective is to show that profit earned by the algorithm is equal to the profit earned by OP T . Compare
A and B. If they are equal, then there is nothing to prove. If A 6= B:
• Find least r such that ar 6= br , such r exists in the range {1, . . . , k}, if not, B is not feasible. For
example, A = (1, 1, 1, 2/3, 0, . . . , 0) and B = (0, 1, 1/4, 1/7, 1, . . . , 0).

Observation 2: ar > br .
• Increase br to ar and accordingly decrease the weights among (br+1 , . . . , bn ). Let C be the modified
configuration of B.
• Due to increase in the value of rth bit (br is increased to ar ), there will be an increase in profit at rth
bit. Since, the increase at rth bit is compensated appropriately by decreasing the values at (r + 1)th
bit, (r + 2)nd bit, and so on, there will be a decrease in profit at these bits. We shall now calculate the
modified profit as follows.
bi · pi + (cr − br ) × wr × wprr
P P
• Profit gained by C = ci · pi =
− (br+1 − cr+1 )wr+1 × wpr+1 r+1
− (br+2 − cr+2 )wr+2 × wpr+2
r+2
− . . . − (bn − cn )wn × wpnn ,
where, (cr − br )wr × wprr is the increase in profit w.r.t the rth bit and −(br+1 − cr+1 )wr+1 × wpr+1
r+1
denotes
the decrease in profit at (r + 1)th bit.

Since p1 /w1 ≥ . . . ≥ pn /wn , wprr ≥ wpr+1 r+1

≥ wpr+2
r+2
. . .. Using this we shall simplify the above ex-
pression as,
∴, ci · pi ≥ bi · pi + (cr − br ) × wr × wprr
P P
− (br+1 − cr+1 )wr+1 × wprr − (br+2 − cr+2 )wr+2 × wprr − . . . − (bn − cn )wn × wprr
 n

bi · pi + wprr (cr − br )wr −
P P P
Thus, ci · pi ≥ (bi − ci ) · wi
 n
 i=r+1
(bi − ci ) · wi = 0 as increase at rth bit is compensated with bits in
P
Note that (cr − br )wr −
i=r+1
(r + 1), . . . , n.
P P P P P P
Therefore,
P ci · pi ≥ bi · pi ⇒ we now have either
P ci · pi > Pbi · pi or Pci · pi = bi · pi
If ci · pi > bi · pi , then it contradicts the optimality of B. Thus, ci · pi = bi · pi .
• Now, check whether A = C or not. If A = C, then the proof is done. If A 6= C, then repeat the above
process till A = C. Note that at the end of the first iteration of the above proof, if A 6= C, then they
will match at first r places and may differ at any of the places in the range r + 1, . . . , n. By repeating
the above argument (at max n iterations), we eventually see that the behavior of the greedy algorithm
and optimum are same.
• Thus, we conclude the output of greedy is indeed optimum.

7
 Acknowledgements: Lecture contents presented in this module and subsequent modules are based
on the following text books and most importantly, author has greatly learnt from lectures by algorithm
exponents affiliated to IIT Madras/IMSc; Prof C. Pandu Rangan, Prof N.S.Narayanaswamy, Prof Venkatesh
Raman, and Prof Anurag Mittal. Author sincerely acknowledges all of them. Special thanks to Teaching As-
sistants Mr.Renjith.P and Ms.Dhanalakshmi.S for their sincere and dedicated effort and making this scribe
possible. Author has benefited a lot by teaching this course to senior undergraduate students and junior
undergraduate students who have also contributed to this scribe in many ways. Author sincerely thank all
of them.

References:
1. E.Horowitz, S.Sahni, S.Rajasekaran, Fundamentals of Computer Algorithms, Galgotia Publications.
2. T.H. Cormen, C.E. Leiserson, R.L.Rivest, C.Stein, Introduction to Algorithms, PHI.
3. Sara Baase, A.V.Gelder, Computer Algorithms, Pearson.