Correctness, Exhaustive Search CMPSC 463 Design and Analysis of Algorithms Dr. Truong X. Tran (Ph.D.) Topics • Reasoning about Algorithms • Time and Space Complexity • Exhaustive Search What is an Algorithm • Definition: An algorithm is a finite set of precise instructions for performing a computation or for solving a problem. • Take some value as input and produces some values as output. • Input: An algorithm has input values from a specified set. • Output: From the input values, the algorithm produces the output values from a specified set. The output values are the solution. • Correctness: An algorithm should produce the correct output values for each set of input values. • Finiteness: An algorithm should produce the output after a finite number of steps for any input. • Effectiveness: It must be possible to perform each step of the algorithm correctly and in a finite amount of time. • Generality: The algorithm should work for all problems of the desired form. Designing and Analyzing Algorithms • Designing: Multiple techniques to design an algorithm: • Increment (induction) • Divide-and-Conquer (recursion) • Analyzing: resources that the algorithm requires • Properties of algorithms • Time and space complexities. Data Structures: a Review All running times are worst case (WCRT), unless otherwise specified Linked List • Prepend: O(1) • Search: O(n) • Access the kth element of the list: O(k) Array • Search: O(n) • Access the kth element of the array: O(1) Sorted Array • Search: O(log n) Dynamic array: where we double the size of the array every time it fills up • Append: O(n) WCRT, O(1) amortized running time Data Structures: a Review Binary Search Tree • Search: O(n) WCRT, O(log n) average case running time (assuming that items are inserted in a random order) • Insert: same as search • Delete: same as search Balanced Search Tree (e.g. 2-3 tree) • Search, Add, Delete: O(log n) Min-Heap • Min: O(1) • Add: O(log n), Delete: O(log n) • MakeHeap (Create a heap from an array of n elements): O(n) Hash Table • Search: O(1) average case running time, if the load factor is bounded above by a constant, O(n) WCRT • Add: O(1), Delete: same as search Space and Time Complexity Performance measure of algorithms • Space: Memory requirement • Time: Running Time This course focus mainly on: Running time complexity • Worse case running time • Average case running time • Best case running time All running times are worst case (WCRT), unless otherwise specified Analyze time complexity: • Determine the number of operations with respected to the input’s size • Number of the operations as a function, f(n), of the size,n, of the input • How fast f(n) grows as the size of input grows • To compare the efficiency of two algorithms when solving the same problem. • The faster grows of f(n), the slower the algorithm can solve a problem Growth of Functions: O-notation
8 Growth of Functions: ꭥ-notation
9 Growth of Functions: Θ-notation
10 Display of Growth of Functions
Note the difference in behavior of functions as n gets
larger Useful Big-O Estimates Involving Logarithms, Powers, and Exponents If d > c > 1, then nc is O(nd), but nd is not O(nc). If b > 1 and c and d are positive, then (logb n)c is O(nd), but nd is not O((logb n)c). If b > 1 and d is positive, then nd is O(bn), but bn is not O(nd). If c > b > 1, then bn is O(cn), but cn is not O(bn). Exhaustive Search • A brute-force algorithm • Examining all possible paths for a valid solution. • Suitable for small and well-defined search space. Problem: Traveling salesman • Given n cities with known distances between each pair, find the shortest tour that passes through all the cities exactly once before returning to the starting city. • Example: Exhaustive Search Model • Choosing • We generally iterate over decisions. What are we iterating over here? The iteration will be done by recursion. • What are the choices for each decision? Do we need a for loop? • Exploring • How can we represent that choice? How should we modify the parameters? • Do we need to use a wrapper due to extra parameters? • Un-Choosing • How do we un-modify the parameters from step 3? Do we need to explicitly un- modify, or are they copied? Are they un-modified at the same level as they were modified? • Base Case • What should we do in the base case when we're out of decisions? Problem: print all binary • Input: A positive integer n. • Return: print all binary number of n bits • Exhaustive Search: Examining all binary of n bits void printBinary(int n) { helper(n, " "); } void helper(int n, string bits) { if (n == 0) { cout << bits<< endl; } else { helper(digits - 1, bits + "0"); helper(digits - 1, bits + "1"); } } printBinary(2) Problem: Subset Sum • Input: An array with n integers and k. • Return: whether there is a subset of integers that adds up to k • Exhaustive Search: Examining all Subsets static boolean subsetSum(int[] values, int k) { return helper(values, 0, k); }
static boolean helper(int[] values, int nextNdx, int sum) {
if (nextNdx == values.length) { return (sum == 0); } return helper(values, nextNdx + 1, sum - values[nextNdx]) || helper(values, nextNdx + 1, sum); } Subset sum example Output all subsets with sum k static void subsetSum(int[] values, int k) { boolean[] included = new boolean[values.length]; helper(values, 0, subset, k);} static void helper(int[] values, int nextNdx, boolean[] included, int sum) { if (nextNdx == values.length) { if (sum == 0) { for (int i = 0; i < values.length; i++) { if (included[i]) System.out.print(" " + values[i]); } System.out.println(); } return; } included[nextNdx] = true; helper(values, nextNdx + 1, included, sum - values[nextNdx]); included[nextNdx] = false; helper(values, nextNdx + 1, included, sum);} Problem: Generating all Strings • Input: A string containing an alphabet, and an integer n • Return: A dynamic array with all of the strings of length n using symbols from the alphabet static ArrayList<String> generateStrings(String alphabet, int n) { ArrayList<String> result = new ArrayList<>(); helper(result, alphabet, "", n); return result;} static void helper(ArrayList<String> strings, String alphabet, String prefix, int charsRemaining) { if (charsRemaining == 0) { strings.add(prefix); return; } for (int i = 0; i < alphabet.length(); i++) { helper(strings, alphabet, prefix + alphabet.charAt(i), charsRemaining - 1); // charsRemaining--? } } Computational Intractability Problems that cannot be solved in polynomial time I.e. not constant k for which the running time of the solution is in O(n^k) • Can only solve small instances of the problem • Seen exhaustive search approaches for this • ONE OF CENTRAL IDEAS IS HOW TO IMPROVE ON EXHAUSTIVE SEARCH… • Find polynomial time algorithms • If not, squeeze performance out of super-polynomial algorithms • Accept “good enough” solutions to problems
