Lec 02 - Intro Algorithms PDF

Advanced Data Structures and Algorithms
CS 361 - Fall 2013 Lec. #02: Introduction to Algorithms

Tamer Nadeem
Dept. of Computer Science
Class Objective/Overview
Main Objective: Start thinking about designing and analyzing algorithms. Understand simple sorting and searching techniques that will be used as examples. Understand running-time complexity/analysis of an algorithm using Big-O notation Apply Big-O analysis on simple sorting/searching algorithms Understand function template syntax Understand and use of recursion.
Page 2
Fall 2013
CS 361 - Advanced Data Structures and Algorithms
Simple Sorting & Searching Techniques
Page 3
Fall 2013
The Problem of Sorting

Input: Output: Sequence a1, a2, , an of numbers. Permutation a'1, a'2, , a'n such that a'1 a'2 a'n .
Example: Input: 8 2 4 9 3 6 Output: 2 3 4 6 8 9
Page 4
Fall 2013
Insertion Sort
Consider A[j]=9. Not in the correct place. Need to make room for 9. We shift all elements right, starting from 10.
Page 5
Fall 2013
Example of Insertion Sort
Page 6
Fall 2013
Page 7
Fall 2013
Page 8
Fall 2013
Page 9
Fall 2013
Page 10
Fall 2013
Page 11
Fall 2013
Page 12
Fall 2013
Page 13
Fall 2013
Page 14
Fall 2013
Page 15
Fall 2013
Page 16
Fall 2013
Algorithm of Insertion Sort
Page 17
Fall 2013
outer loop
inner loop
Page 18 Fall 2013 CS 361 - Advanced Data Structures and Algorithms
Code of Insertion Sort

void insertionSort (int *a , int size) { int i , j , n = size; int key; // place a[i] into the sublist // a[0] . . . a[i-1], 1 <= i < n, // so it is in the correct position for (int j=1; j<n; ++j) // outer loop { key = a[j]; i=j-1; while ((i > 0) && (key < a[i] )) // inner loop { a[i+1] = a[i]; i = i - 1; } a[i+1] = key; } }
Comments on Insertion Sort

If the key value is greater than all the sorted elements in the sublist, the algorithm does 0 iterations of the inner loop. If the initial array is sorted:
Then each new key will be greater than all the ones already inserted into the array. For each outer loop iteration, the algorithm does 0 iterations of the inner loop.
This special case is very common. Many practical problems require the construction of a sorted sequence of elements from a set of data that is already sorted or nearly so, with only a few items out of place. Note the "work from the back is very efficient for such inputs.
Page 20
Fall 2013
Analysis of Algorithms
Page 21
Fall 2013
Algorithm Complexity
A code of an algorithm is judged by its correctness, its ease of use, and its efficiency. This course focus on the computational complexity (time efficiency) of algorithms that apply to container objects (data structures) that hold a large collection of data. We will learn how to develop measures of efficiency (criteria) that depends on n, the number of data items in the container. The criteria of computational complexity often include the number of comparison tests and the number of assignment statements used by the algorithm.
Page 22
Fall 2013
Running Time
The term running time is often used to refer to the computational complexity (how fast the algorithm). The running time depends on the input (e.g., an already sorted sequence is easier to sort).
Parameterize the running time by the size of the input n Seek upper bounds on the running time T(n) for the input size n, because everybody likes a guarantee. T(n) function counts the frequency of the key operations in terms of n.
Page 23
Fall 2013
Brute Force Timing Analysis

1: for ( i = 0; i < N; ++ i ) { 2: a [ i ] = 0; 3: for ( j = 0; j < N; ++ j ) 4: a[i]=a[i]+i*j; 5: }
The total run time, T(N), for this algorithm is:

T(N) = (2N2 + 3N + 1) * tasst + (N2 + 2N + 1) * tcomp + (4N2 + 2N) * tadd + (3N2 + N) * tmult = c1N2 + c2N + c3
Page 24 Fall 2013
c1 = 2tasst + tcomp + 4tadd + 3tmult c2 = 3tasst + 2tcomp + 2tadd + tmult c3 = tasst + tcomp
Analysis Rules for Program Statements

Individual Statements
1. Expressions and Assignments
The complexity of an expression is the sum of the complexity of all of the operations within it. Assignment statements have a complexity equal to the sum of the complexities of the expressions to either side of the = operator, plus the complexity of the actual copy.
2. Function Calls
Function/Procedure calls are counted as the complexity of their bodies.
3. Loops
Compute the run time by adding up the time required for all the iterations.

3. Loops
Compute the run time by adding up the time required for all the iterations. The running time of a loop is at most:
where tinit is the time required to do the loop initialization, tcondition is the time to evaluate the loop condition, with tfinal condition being the time to evaluate the loop condition the final time (when we exit the loop), and tbody is the time required to do the loop body.
Example: A simple loop for ( int j = 0; j < n; ++ j ) a [ i +10* j ] = i + j ;
Page 26 Fall 2013
tloop = O(cinit + (ccondition + cincrement + cbody) + ccondition) = O(c0 + n * c1) = O(n)


3. Conditional Statement
The worst case time for the if is the slower of the two possibilities. The running time of an if-then-else is at most:
where tcondition is the time to evaluate the if condition, with tthen is the time to do the then body, and telse is the time required to do the else body.
A missing else clause (or, for that matter, any empty statement list) is O (1).
Page 27
Fall 2013

Combining Statements 1. Sequences of Statements
The time for consecutive (straight-line) statements is the sum of the individual statements.
2. Nested Statements
In general, nested statements are evaluated by applying the other rules from the inside out.
Example: A simple nested loop for ( int i = 0; i < n; ++ i ) { for ( int j = i ; j < n; ++ j ) { a[i][j]=a[j][i]; } }

Combining Statements 2. Nested Statements (contd)
In general, nested statements are evaluated by applying the other rules from the inside out.
Example: A simple nested loop for ( int i = 0; i < n; ++ i ) { for ( int j = i ; j < n; ++ j ) { a[i][j]=a[j][i]; } }
Page 29
Fall 2013
Machine-independent time
What is insertion sorts worst-case time?
It depends on the speed of our computer:
relative speed (on the same machine), absolute speed (on different machines).
BIG IDEA:
Ignore machine-dependent constants. Look at growth of T(n) as n
Asymptotic Analysis
Page 30
Fall 2013
Worst Case Analysis

Definition:
We say that an algorithm requires time proportional to f(n) if there are constants c and n0 such that the algorithm requires no more than c * f(n) time units to process an input set of size n whenever n n0. The f(n) here describes the rate at which the time required by this algorithm goes up as you change the size of the input for particular program or algorithm. The multiplier c is used so that we can talk about the algorithm requiring no more than c*f(n) time units. n0 is used to place a lower limit on how small the inputs are we really worried about.
Page 31
Fall 2013
O-notation
Math:
We say that T(n)= O(g(n)) iff there exists positive constants c1, and n0 such that 0 T(n) c1 g(n) for all n n0 Usually T(n) is running time, and n is size of input
Engineering:
Drop low-order terms; ignore leading constants. Example: 3n3 + 90n2 5n + 6046 = O(n3)
-notation
Math:
We say that T(n)= (g(n)) iff there exists positive constants c2, and n0 such that 0 c2 g(n) T(n) for all n n0
Engineering:
Drop low-order terms; ignore leading constants. Example: 3n3 + 90n2 5n + 6046 = (n3)
Page 33
Fall 2013
O-notation (contd)
So if T(n)= O(n2) then we are also sure that T(n)= O(n3) and that T(n)= O(n3.5) and T(n)= O(2n) But it might or might not be true that T(n)= O(n1.5). However, if T(n)= (n2 ) then it is not true that T(n)= O( n1.5 )
Page 34
Fall 2013
-notation
Math:
We say that T(n)= (g(n)) iff there exist positive constants c1, c2, and n0 such that 0 c1 g(n) T (n) c2 g(n) for all n n0
In other words
T(n)= (g(n)) iff T(n)= O(g(n)) and T(n)=(g(n))
Engineering:
Drop low-order terms; ignore leading constants. Example: 3n3 + 90n2 5n + 6046 = (n3)
Asymptotic Performance
When n gets large enough, a (n2) algorithm always beats a (n3) algorithm.
W e s h o u l d n t i g n o r e asymptotically slower algorithms, however. Real-world design situations often call for a careful balancing of engineering objectives. Asymptotic analysis is a useful tool to help to structure our thinking.
Detailed Timing Example Revisited

1. The total run time, T(N) = c1N2 + c2N + c3 2. For n0 = 1, It is clear that c 1 N 2 + c 2 N + c3 c 1 N 2 + c 2 N 2 + c 3 N 2 3. Then, T(N) c1N2 + c2N2 + c3N2 4. Let c = c1 + c2 + c3, then T(N) cN2 5. This mean that T(N) is in order of O(N2)
Page 37
Fall 2013
Special Cases
Constant Time Algorithms: An algorithm is O(1) when its running time is independent of the number of data items. The algorithm runs in constant time. Direct Insert at Rear
e.g., direct insert at rear of array involves a simple assignment statement and thus has efficiency O(1)
front rear
Linear Time Algorithms: An algorithm is O(n) when its running time is proportional to the size of the list.
e.g., sequential search. When the number of elements doubles, the number of operations doubles.
1 Sequential Search for the Minimum Element in an Array
32
46
12
3
n=5
2 3 4 5 minimum element found in the list after n comparisons

Page 38
Fall 2013
Special Cases
Exponential Algorithms: Algorithms with running time O(n2) are quadratic.
practical only for relatively small values of n. Whenever n doubles, the running time of the algorithm increases by a factor of 4.
Algorithms with running time O(n3) are cubic.

efficiency is generally poor; doubling the size of n increases the running time eight-fold.
Logarithmic Time Algorithms: The logarithm of n, base 2, is commonly used when analyzing computer algorithms
Ex. log2(2) = 1, log2(75) = 6.2288 When compared to the functions n and n2, the function log2 n grows very slowly
n2
log2n
Page 39
Fall 2013
Comparison of different O()

n 2 4 8 16 32 128 1024 65536 log2n 1 2 3 4 5 7 10 16 n log2n 2 8 24 64 160 896 10240 1048576 n2 4 16 64 256 1024 16384 1048576 4294967296 n3 8 64 512 4096 32768 2097152 1073741824 2.8 x 1014 2n 4 16 256 65536 4294967296 3.4 x 1038 1.8 x 10308 Forget it!
Page 40
Fall 2013
The Algebra of Big-O

Algebraic Rule 1: f(n) + g(n) O(f(n) + g(n)) Algebraic Rule 2: f(n) * g(n) O(f(n) * g(n)) Algebraic Rule 3: O(c * f(n)) = O(f(n))
Page 41
Fall 2013

Proof of Algebraic Rule 3: O(c * f(n)) = O(f(n)) 1. Given running time T(n) = O(c * f(n)) 2. By O() definition, c1,n0|n>n0 T(n) <= c1(c * f(n)) 3. 5. 6. c1,n0|n>n0 T(n) <= (c1 * c) f(n)) T(n) = O(f(n)) O(c * f(n)) = O(f(n)) 4. Let c2 = c1 * c c1,n0|n>n0 T(n) <= c2 * f(n))
Similary, O(c1 * f(n) + c2 * g(n)) = O(f(n) + g(n))
Page 42
Fall 2013

Algebraic Rule 4: Larger Terms Dominate a Sum
ifn0|n>n0, f(n)>=g(n),
then O(f(n) + g(n)) = O(f(n))
Page 43
Fall 2013

Proof of Algebraic Rule 4:
n>n0, f(n)>=g(n),
then O(f(n) + g(n)) = O(f(n))
1. Given running time T(n) = O(f(n) + g(n)) 2. By O() definition, c,n1|n>n1 T(n) <= c(f(n) + g(n)) 3. Assume n>0, f(n)>=g(n) then n > max(n0,n1) T(n) <= c (f(n) + g(n)) n > max(n0,n1) f(n) > g(n) 4. Using (2) to replacing g(n) by f(n) in (1) then n > max(n0,n1) T(n) <= c (f(n) + f(n)) n > max(n0,n1) T(n) <= 2c * f(n) 5.
Page 44
(1) (2) (3) (4)
Fall 2013
T(n) = O(f(n))

Algebraic Rule 5: Logarithms are Fast
k 0, O(logk(n)) O(n)
Page 45
Fall 2013
Analysis of Sorting/Searching Algorithms
Page 46
Fall 2013
Algorithm of Insertion Sort
Page 47
Fall 2013
Insertion sort analysis

So if T(n) = O(n2) then we are also sure that T(n) = O(n3) and that T(n) = O(n3.5) and T(n) = O(2n) But it might or might not be true that T(n) = O(n1.5). However, if T(n) = (n2) then it is not true that T(n) = O(n1.5)
Page 48
Fall 2013
Sequential Search
Search algorithm start with a target value and employ sequential visit to the elements looking for a match.
If target is found, the index of the matching element becomes the return value.
Array arr 0 1 first 2 3 4 5 last 6 7 8
int seqSearch (const int arr[ ], int first, int last, int target) // Look for target within a sorted array arr [first . . last -1]. // Return the position where found, or last if not found . { int i = first; while (i < last && arr[i] != target ) index = seqSearch(arr, 0, 8, 3); i++; target = 3 return i ; Index 0 1 2 3 4 5 6 } 3 10 6 4 2 9 5
7 7
match at index = 5 return index 5

Sequential Search for a Sorted Array

Previous code for any general array (i.e., non sorted). When arrays have been sorted, they could be searched using a slightly faster variant of the sequential sort. If array is sorted, then encountering a value greater than the target would imply that the target value is not in the array.
Replacing the != operator by a < causes the algorithm to exit the loop as soon as we see a value that is equal to or greater than the target .
int seqSearch (const int arr[ ], int first, int last, int target) // Look for target within a sorted array arr [first . . last -1]. // Return the position where found, or last if not found . { int i = first; while (i < last && arr[i] < target ) i++; return i ; }
Binary Search
An alternative method of searching sorted arrays is the binary search 1 49
Example: Search for number 85 in the 63-entry list
2 5 6 8 12 16 17 18 21 23 24 30 32 33 35 38 40 44 45 47 54 57 58 59 64 66 67 69 70 71 74 75 77 80 81 83 85 87 88 90
First 1 33 33 33 37 37
Last 63 63 47 39 39 37
Middle Entry Outcome 32 71 > 48 102 < 40 87 < 36 80 > 38 83 > 37 85 =
4 5 3
Six probes are needed with a 63-entry list, in the worst case
Page 51 Fall 2013
91 93 99 100 101 102 105 107 110 111 116 117 118 120 122 125 126 128 130 131 133
Code of Binary Search

int binSearch (const int arr[ ], int first, int last, int target) // Look for target within a sorted array arr [first . . last -1]. // Return the position where found, or , or last if not found { int mid; // index of the midpoint int midValue ; // object that is assigned arr[mid] int origLast = last ; // save original value of last // repeatedly reduce the area of search // until it is just one element while (first < last) { // test for nonempty sublist mid = (first + last) / 2; midValue = arr[mid] ; if (target == midValue) return mid; else if (target < midValue) last = mid; // search lower sublist , reset last else first = mid+1; // search upper sublist , reset first } return origLast; }
Binary Search Tree

Example 1: Find 47
35 71
< 71 > 35
> 71
Example 2: Find 112

102
> 102
17 57 87 120
< 57
6 24 45 66 80 93
< 120
111 128
> 45
2 12 21 32 40 49 59 69 75 83 90 100 107
> 111
117 125 131
< 49
1 5 8 16 18 23 30 33 38 44 47 54 58 64 67 70 74 77 81 85 88 91 99 101
< 117
105 116 122 130 133 110 118 126
1 2 5 6 8 12 16 17 18 21 23 24 30 32 33 35 38 40 44 45 47
49 54 57 58 59 64 66 67 69 70 71 74 75 77 80 81 83 85 87 88 90
91 93 99 100 101 102 105 107 110 111 116 117 118 120 122 125 126 128 130 131 133
Found
Page 53 Fall 2013
< 116 Not found
63-item list
Making Algorithms Generic
Page 54
Fall 2013
Generalize the Element Type

Lets go back to some of the array manipulation functions we developed earlier. Our binSearch routine operates on arrays of int. What to do if we wan to have a binary search over arrays of double, or of string ?
int binSearch (const int arr[ ], int first, int last, int target) // Look for target within a sorted array arr [first . . last -1]. // Return the position where found, or , or last if not found { int mid; // index of the midpoint int midValue ; // object that is assigned arr[mid] int origLast = last ; // save original value of last ... ;. . . return origLast; }
int binSearch (const T arr[ ], T first, T last, T target) // Look for target within a sorted array arr [first . . last -1]. // Return the position where found, or , or last if not found { int mid; // index of the midpoint T midValue ; // object that is assigned arr[mid] int origLast = last ; // save original value of last ... ;. . . return origLast; } Page 55 Fall 2013
Mian.cc
typedef int T; #include "arrayops .h" / / T == int importantNumbers [ 100 ] zn; .. orderedInsert ( importantNumbers , 0 , n, 42) ;

One day, discover that we need to manipulate an array of int and an array of some other type (e.g., int)
Mian.cc
typedef int T; #include "arrayops .h" / / T == int typedef std : : string T; #include "arrayops .h" / / T == s td : : s t r ing int importantNumbers [ 1 00 ] ; std : : s t r ing favoriteNames [ 1 00 ] ; int n1 , n2 ; ... sequent ial Inser t ( importantNumbers , n1 , 42) ; sequent ial Inser t ( favoriteNames , n2 , "Arthur Dent" ) ;
We will get compilation errors, this time at the second typedef, because we cant define T to mean two different things at the same time.
Page 56 Fall 2013

we could simply make a distinct copy of binSearch for each kind of array, using an ordinary text editor to replace T by a real type name. Doing this in a large program does get to be a bit of a project management headache. Since it is a common situation, and it does involve a fair amount of work, some programming language designers (e.g., C++) eventually found a way to do the same thing automatically; use of Templates.
Page 57
Fall 2013
Templates in C++
Templates describe common patterns for similar classes and functions that differ only in a few names. Templates come in two varieties: Class templates, and Function templates Class templates are patterns for similar classes. Function templates are patterns for similar function A C++ template is a kind of pattern for code in which certain names that, like T in the earlier example, we intend to replace by something real when we use the pattern. These names are called template parameters. The compiler instantiates (creates an instance of) a template by filling in the appropriate replacement for these template parameter names, thereby generating the actual code for a function or class, and compiling that instantiated code.
Writing Function Templates

We define a template header for each function to tell the compiler which names in the pattern are to be replaced. The template header indicates that this is a pattern for a function in which certain type names and constants are left to be filled in later. The header begins with the key word template. After that, inside the < >, is a list of all type (class) names to be replaced when we instantiate the template. Bodies of function templates are defined in header files (.h)
Page 59
Fall 2013
Writing Function Templates

Inside the header file (binsearch.h file):
/ * * * * * binsearch .h * * * * * / #ifndef BINSEARCH_H #define BINSEARCH_H template <typename T> //or template <class T> int binSearch (const T arr[ ], int first, int last, T target) // Look for target within a sorted array arr [first . . last -1]. // Return the position where found, or , or last if not found { int mid; // index of the midpoint T midValue ; // object that is assigned arr[mid] int origLast = last ; // save original value of last // repeatedly reduce the area of search // until it is just one element while (first < last) { // test for nonempty sublist ..... ..... } return origLast; } #endif
Using Function Templates

We instantiate a function template when we try to use it: The compiler deduces from the calls that it must use the binSearch template to produce the functions array elements).
#include binsearch.h int importantNumbers [100] ; std::string favoriteNames [100] ; int n1, n2 ; int found; ..... ..... n1 = binSearch(importantNumbers, 0, 100, 42) ; n2 = binSearch(favoriteNames , 0, 100, "Arthur Den"); .....
int binSearch (int arr[], int first, int last, int target); and int binSearch (std::string arr[], int first, int last, std::string target);
It does this simply by replacing T in the template by int and std::string, respectively. We can do this for any T that supports assignment and < operators.
Function Templates and the C++ std Library

The C++ std library has a number of useful templates for small, common programming idioms. relops Function swap Function
namespace relops { template <typename T> inline void swap(T& a , T& b) { T tmp = a; a = b; b = tmp; } template <typename T> inline bool operator!= (const T& a, const T& b){ return !(a == b); } template <typename T> inline bool operator> (const T& a, const T& b){ return b < a; } template <typename T> inline bool operator<= (const T& a, const T& b){ return !(b < a); } template <typename T> inline bool operator>= (const T& a, const T& b){ return !(a < b) ; } }
min & max Functions

template <typename T> inline const T& min(const T& a, const T& b) { return b < a ? b : a ; } template <typename T> inline const T& max(const T& a, const T& b) { return b < a ? a : b; }
Page 62 Fall 2013
Recurssion
Page 63
Fall 2013
The Concept of Recursion

Lets consider the problem of evaluating the power xn where x is real number and n is a nonnegative integer.
Iterative Approach
Double power(double x, int n){ double product = 1; int i;
for (i = 1; i <= n; i++) Another approach, since xn = xm product *= x; //xn = x**x (n times) (n-m) *x , we can split the problem return product; to smaller problems. For example, 215 could be computed } as 25 * 210 = 32 * 1024. Recursion, in simple, is solving a problem by solving smaller problems of same form and using their results to compute the final result.
Recursion is the process a function goes through when one of the steps of the function involves invoking the function itself. A procedure that goes through recursion is said to be 'recursive'.
Page 64
Fall 2013
The Concept of Recursion

A function exhibits recursive behavior when it can be defined by two properties: A simple base case (or cases) (stopping condition) A set of rules that reduce all other cases toward the base case Example: Computing xn Base case: x0 = 1 Rule: xn = xn-1 * x Example: Computing x! Base case: 0! = 1 Rule: x! = x * (x-1)!
Recursive Approach
Double power(double x, int n){ if (n == 0) return 1; else return x * power(x, n-1); }
Page 65 Fall 2013
Recursive Approach
Double factorial(int x){ if (x == 0) return 1; else return x * factorial(x-1); }
Fibonacci Numbers
Fibonacci numbers are the sequence of integers 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55, Example: Computing fib(n) //Fibonacci element with index n Base case: fib(0) = 0 fib(1) = 1 Iterative Approach + Rule: fib(n) = fib(n-1) fib(n-2)
int fibiter(int n){ int oneback = 1, twoback = 0, current, i; if (n <= 1) return n; else for (i=2; i <= n; i++){ current = oneback + twoback; twoback = oneback; oneback = current; } return current; }
Recursive Approach
int fib(int n){ if (n <= 1) return n; else return fib(n-1) + fib(n-2); }
Page 66
Fall 2013
Evaluating Fibonacci Numbers

#include <iostream> #include d_timer.h int fibiter(int n){ ..... } int fib(int n){ ..... } int main(){ timer t1, t2; t1.start(); fibiter(45); t1.stop(); t2.start(); fib(45); t2.stop(); cout << Time required by Iterative version is << t1.time() << sec << endl; cout << Time required by Recursive version is << t2.time() << sec << endl; return 0;
Page 67
//from previous slide
//from previous slide
Fall 2013
Tower of Hanoi
........
........
Needle A
Needle B
Needle C
Needle A
Needle B
Needle C
Page 68
Fall 2013
Tower of Hanoi
1
Needle A Needle B Needle C
2
Needle A Needle B Needle C
3
Needle A Needle B Needle C Needle A Needle B Needle C
Needle A
Needle B
Needle C
Needle A
Needle B
Needle C
6 Needle A Needle A Needle B Needle C
Needle B
Needle C
Needle A
Needle B
Needle C
Needle A
Needle B
Needle C
Page 69
Fall 2013
Questions?
Page 70
Fall 2013

Lec 02 - Intro Algorithms PDF

Uploaded by

Lec 02 - Intro Algorithms PDF

Uploaded by

Advanced Data Structures and Algorithms

CS 361 - Fall 2013 Lec. #02: Introduction to Algorithms

Dept. of Computer Science

CS 361 - Advanced Data Structures and Algorithms

Simple Sorting & Searching Techniques

CS 361 - Advanced Data Structures and Algorithms

The Problem of Sorting

CS 361 - Advanced Data Structures and Algorithms

CS 361 - Advanced Data Structures and Algorithms

Example of Insertion Sort

CS 361 - Advanced Data Structures and Algorithms

Example of Insertion Sort

CS 361 - Advanced Data Structures and Algorithms

Example of Insertion Sort

CS 361 - Advanced Data Structures and Algorithms

Example of Insertion Sort

CS 361 - Advanced Data Structures and Algorithms

Example of Insertion Sort

CS 361 - Advanced Data Structures and Algorithms

Example of Insertion Sort

CS 361 - Advanced Data Structures and Algorithms

Example of Insertion Sort

CS 361 - Advanced Data Structures and Algorithms

Example of Insertion Sort

CS 361 - Advanced Data Structures and Algorithms

Example of Insertion Sort

CS 361 - Advanced Data Structures and Algorithms

Example of Insertion Sort

CS 361 - Advanced Data Structures and Algorithms

Example of Insertion Sort

CS 361 - Advanced Data Structures and Algorithms

Algorithm of Insertion Sort

CS 361 - Advanced Data Structures and Algorithms

Example of Insertion Sort

Code of Insertion Sort

Comments on Insertion Sort

CS 361 - Advanced Data Structures and Algorithms

CS 361 - Advanced Data Structures and Algorithms

CS 361 - Advanced Data Structures and Algorithms

CS 361 - Advanced Data Structures and Algorithms

Brute Force Timing Analysis

The total run time, T(N), for this algorithm is:

Analysis Rules for Program Statements

Analysis Rules for Program Statements

tloop = O(cinit + (ccondition + cincrement + cbody) + ccondition) = O(c0 + n * c1) = O(n)

Analysis Rules for Program Statements

CS 361 - Advanced Data Structures and Algorithms

Analysis Rules for Program Statements

Analysis Rules for Program Statements

CS 361 - Advanced Data Structures and Algorithms

Worst Case Analysis

CS 361 - Advanced Data Structures and Algorithms

CS 361 - Advanced Data Structures and Algorithms

CS 361 - Advanced Data Structures and Algorithms

Detailed Timing Example Revisited

CS 361 - Advanced Data Structures and Algorithms

2 3 4 5 minimum element found in the list after n comparisons

Algorithms with running time O(n3) are cubic.

CS 361 - Advanced Data Structures and Algorithms

Comparison of different O()

CS 361 - Advanced Data Structures and Algorithms

The Algebra of Big-O

CS 361 - Advanced Data Structures and Algorithms

The Algebra of Big-O

Similary, O(c1 * f(n) + c2 * g(n)) = O(f(n) + g(n))

CS 361 - Advanced Data Structures and Algorithms

The Algebra of Big-O

CS 361 - Advanced Data Structures and Algorithms

The Algebra of Big-O

(1) (2) (3) (4)

The Algebra of Big-O

CS 361 - Advanced Data Structures and Algorithms

Analysis of Sorting/Searching Algorithms

CS 361 - Advanced Data Structures and Algorithms

Algorithm of Insertion Sort

CS 361 - Advanced Data Structures and Algorithms