Lecture #16 Wed 20 Feb 2002 NEC 0. Insertion Sort (wrap up) 1. Merge Sort 2. Comparison of various algorithms 4. Upper, Lower, Tight Bounds Homework #3 due today Homework #4 comes out tonight!!!!, due Wednesday, or in my box Friday special deal -- No class on the Friday before spring break. Reading Assignment--some or all of Chapter 7, check web page 0. Insertion Sort wrap up. review the summation give geometric example. 1. Merge Sort (Don't forget to use the cards!) OK, so we've seen a sorting algorithm that runs in polynomial time (or quadratic, to be more specific). Can we do better than this? What if we apply a method similar to what we did in binary search, a "binary sort" of sorts. We'll examine a function called MERGE SORT, described on pages 83-85 of Harel. We can write this using two functions: "mergesort" and "merge" MERGESORT takes in an unsorted list of numbers, and produces the same list of numbers, but sorted in ascending order. MERGE takes in two sorted lists of numbers, and produces a single sorted list, containing all the elments in both of the lists. i.e. (merge (list 1 3 5) (list 2 4 6)) = (list 1 2 3 4 5 6) Hmm, how do we go about writing/using these functions? Well, here's the magic of recursion in action!! Mergesort will take in a list of N elements and halve it, then halve those halves, and halve those halves, and so on, until you have N lists of one element each. A list of one element is by definition sorted, right? Then it will merge each of these little lists back together using the function merge, until it ends up with one complete sorted list of N elements. Let's take a look at sorting a list of 8 elements . . . (list 5 4 2 6 7 3 8 1) First, we split it into two halves (5 4 2 6) (7 3 8 1) Then we keep splitting those into halves, until we only have lists of 1 or 0 elements (5 4) (2 6) (7 3) (8 1) (5) (4) (2) (6) (7) (3) (8) (1) Now we begin the merge, we start merging them back together, remember merge, merges ordered lists in order, it's pretty easy to do, and the most compartsons will ever have is the sum of the length of both lists. Merge only takes two lists as input, so we can only do two at a time. We merge (5) and (4) to get, (4 5), and so on . . . The first set of merges gives us: (4 5) (2 6) (3 7) (1 8) Then we merge those to get: (2 4 5 6) (1 3 7 8) And one more merge: (1 2 3 4 5 6 7 8) We did it!!!! Great, but now we need to analyze this algorithm to see how fast it was. Well, we began with 8 elements, then halved that to 4, then halved those to 2, etc. So this kind of forms a tree. 8 / \ 4 4 / \ / \ 2 2 2 2 /\ /\ /\ /\ 1 1 1 1 1 1 1 1 This would be a computation tree, not a decision tree, right? So, assuming that the only real work done that will matter in our final analysis is the number of comparisons (it turns out that this is the case), how many comparisons do I make? Well, we're just splitting to get down to the 1's, right? There's no comparisons there. What about merging? That's where the comparisons are, in the merge function. Recall we said that the maximum number of comparisons needed in merge is the sum of the lengths of the two lists being merged. So Keeping the tree in mind, how many comparisons are there at each level? 8 / \ (4 + 4) = 8 4 4 / \ / \ (2 + 2) + (2 + 2) = 8 2 2 2 2 /\ /\ /\ /\ (1 + 1) + (1 + 1) + (1 + 1) + (1 + 1) = 8 1 1 1 1 1 1 1 1 So, how many comparisons do we have, worst case, at each level? 8 And how many levels do we have? Recall we said that the height of a full binary tree is log2(size+1)-1 ? Well, as it turns out, the size of a full binary tree is always (2*leaves)-1, where "leaves" is the number of leaves. This means that the height of a full binary tree is always log2(leaves). Cool, huh? Add that to your list of tree properties. This means that since we have log2(8) levels, or 3 levels. So, as you can see, using this logic, we end up with 8 * log2(8) comparisons in this case. How do we generalize this with the number of elements. Well, this will only vary depending on our number of elements N, which in this case is 8, so this means that the RUNNING TIME for Merge Sort is O(N*logN). This is much faster than the O(N^2) time we had for insertion sort. Yay!!! A faster algorithm! 2. Show Chart, Comparison of These algorithms. Show chart on p. 166-67, also p. 142 of Harel 3. Upper, Lower, and "Tight" Bounds Computer Scientists are constantly trying to find better algorithms to do a particular thing. We have examined worst-case analyses of the algorithms we've discussed thus far. Since we have shown an algorithm that solves these worst cases in a given time, this is an UPPER BOUND on the running time of that algorithm. That means that we know we can at least do it within a certain amount of time, but there might be something better out there. (i.e., we know we can sort a list of numbers in O(N*logN) time, as we saw with merge sort, but can we do better?) To find out how good we actually can do, we can seek LOWER BOUNDS for the algorithms, by proving that some given task will take at least so many steps to do. This would be the best possible amount of time we would have to spend doing the problem, given arbitrary input. When computer scientists perform their proofs and analyses and so on and find that their UPPER BOUND is the in the same order as the LOWER BOUND for a particular algorithm, (that is, the lower bound MEETS the upper bound,) then they know they have found an optimal algorithm, for something. More or less. We've seen how to calculate an upper bound for a running time, but how do we compute a lower bound? Well, there's many ways, but most involve a lot of math. We won't discuss these so much, but here's an example proof that relates to stuff we've seen: Remembering DECISION TREES with COMPARISONS: The number of leaves is the number of results, and the height of any full binary tree is log2(leaves), so we know we must have at least as many leaves as the number of results, in a comparison tree, and then log2(results) comparisons, no matter what. This would be a lower bound. PROOF: if we have x possible results, then it's gonna requre us at least log2(x) comparisons to get there, based on our knowledge of binary trees. Aside: In case you're confused, recall that, with COMPUTATION TREES, every node counts as part of the computation, and with DECISION TREES, only one path counts . . . i.e. the height of the tree.