Lec 2
Lec 2
Lec 2
1
Algorithm
• An algorithm is a set of instructions to be followed to
solve a problem.
– There can be more than one solution (more than one
algorithm) to solve a given problem.
– An algorithm can be implemented using different
programming languages on different platforms.
• An algorithm must be correct. It should correctly solve
the problem.
– e.g. For sorting, this means even if (1) the input is already
sorted, or (2) it contains repeated elements.
• Once we have a correct algorithm for a problem, we
have to determine the efficiency of that algorithm.
2
Algorithmic Performance
There are two aspects of algorithmic performance:
• Time
• Instructions take time.
• How fast does the algorithm perform?
• What affects its runtime?
• Space
• Data structures take space
• What kind of data structures can be used?
• How does choice of data structure affect the runtime?
We will focus on time:
– How to estimate the time required for an algorithm
– How to reduce the time required
3
Analysis of Algorithms
• Analysis of Algorithms is the area of computer science that
provides tools to analyze the efficiency of different methods of
solutions.
• How do we compare the time efficiency of two algorithms that
solve the same problem?
Naïve Approach: implement these algorithms in a programming
language (C++), and run them to compare their time
requirements. Comparing the programs (instead of algorithms)
has difficulties.
– How are the algorithms coded?
• Comparing running times means comparing the implementations.
• We should not compare implementations, because they are sensitive to programming
style that may cloud the issue of which algorithm is inherently more efficient.
– What computer should we use?
• We should compare the efficiency of the algorithms independently of a particular
computer.
– What data should the program use?
• Any analysis must be independent of specific data.
4
Analysis of Algorithms
• When we analyze algorithms, we should employ
mathematical techniques that analyze algorithms
independently of specific implementations,
computers, or data.
• To analyze algorithms:
– First, we start to count the number of significant
operations in a particular solution to assess its
efficiency.
– Then, we will express the efficiency of algorithms using
growth functions.
5
General Rules for Estimation
• Loops: The running time of a loop is at most the running time
of the statements inside of that loop times the number of
iterations.
• Nested Loops: Running time of a nested loop containing a
statement in the inner most loop is the running time of
statement multiplied by the product of the sized of all loops.
• Consecutive Statements: Just add the running times of those
consecutive statements.
• If/Else: Never more than the running time of the test plus the
larger of running times of S1 and S2.
6
Asymptotic Notation
7
Big Oh Notation
8
Big-O Visualization
9
Big Omega Notation
10
Big- Visualization
11
Big Theta Notation
12
Big- Visualization
13
Algorithm Growth Rates
• We measure an algorithm’s time requirement
as a function of the problem size.
• An algorithm’s proportional time requirement
is known as growth rate.
• We can compare the efficiency of two
algorithms by comparing their growth rates.
14
Growth-Rate Functions
O(1) Time requirement is constant, and it is independent of the problem’s size.
O(log2n) Time requirement for a logarithmic algorithm increases slowly
as the problem size increases.
O(n) Time requirement for a linear algorithm increases directly with the size
of the problem.
O(n*log2n) Time requirement for a n*log2n algorithm increases more rapidly than
a linear algorithm.
O(n2) Time requirement for a quadratic algorithm increases rapidly with the
size of the problem.
O(n3) Time requirement for a cubic algorithm increases more rapidly with the
size of the problem than the time requirement for a quadratic algorithm.
O(2n) As the size of the problem increases, the time requirement for an
exponential algorithm increases too rapidly to be practical.
15
A Comparison of Growth-Rate Functions
(cont.)
16
Some Mathematical Facts
• Some mathematical equalities are:
n
n * (n 1) n2
i 1
i 1 2 ... n
2
2
3
n
n * ( n 1) * ( 2 n 1) n
i 1
i 2
1 4 ... n 2
6
3
n 1
2 i
i 0
0 1 2 ... 2 n 1
2 n
1
17
The Execution Time of Algorithms
A sequence of operations:
Total Cost = c1 + c2
18
The Execution Time of Algorithms (cont.)
Example1:
Cost Times
if (n < 0) c1 1
absval = -n c2 1
else
absval = n; c3 1
19
The Execution Time of Algorithms (cont.)
Example2:
Cost Times
i = 1; c1 1
sum = 0; c2 1
while (i <= n) { c3 n+1
i = i + 1; c4 n
sum = sum + i; c5 n
}
Example3:
Cost Times
i=1; c1 1
sum = 0; c2 1
while (i <= n) { c3 n+1
j=1; c4 n
while (j <= n) { c5 n*(n+1)
sum = sum + i; c6 n*n
j = j + 1; c7 n*n
}
i = i +1; c8 n
}
T(n) = c1 + c2 + (n+1)*c3 + n*c4 + n*(n+1)*c5+n*n*c6+n*n*c7+n*c8
= (c5+c6+c7)*n2 + (c3+c4+c5+c8)*n + (c1+c2+c3)
= a*n2 + b*n + c
Example4:
CostTimes
for (i=1; i<=n; i++) c1 n+1
x=x+1; c4 n3
23
Coding exercise #2
for ( i=0 ; i<n ; i++ )
for( j=0 ; j<n ; j++ )
sum[i] += entry[i][j];
24
Coding exercise #3
for ( i=0 ; i<n ; i++ )
for( j=0 ; j<i ; j++ )
m += j;
25
Coding exercise #4
for ( i=0 ; i<n ; i++ )
for( j=0 ; j<n ; j++ )
for( k=0 ; k<n ; k++ )
sum[i][j] += entry[i][j][k];
26
Coding exercise #5
for ( i=0 ; i<n ; i++ )
for( j=0 ; j<n ; j++ )
sum[i] += entry[i][j][0];
for ( i=0 ; i<n ; i++ )
for( k=0 ; k<n ; k++ )
sum[i] += entry[i][0][k];
27