Qualitative Analysis of Algorithms

Simple overview of the concepts of algorithm analysis

Transcript

1. A Qualitative Sense of Algorithms Thursday, July 26, 12
2. An algorithm is a repeatable process for determining a solution to a problem. Thursday, July 26, 12
3. For example: 1.Let sum be equal to 0 2.Let count be equal to the size of the list 3.For each element in the list: 1.Add its value to count 4.Divide count by sum Find the mean of a list of numbers: Thursday, July 26, 12
4. In contrast, an heuristic is a loose rule or set of rules that assists in discovering a solution via trial and error. Thursday, July 26, 12
5. Algorithmic analysis is the determination of the characteristic behavior of an algorithm with respect to time and/or space. Thursday, July 26, 12
6. How long does it take to run? How much memory will it consume? Thursday, July 26, 12
7. How long does it take to run? How much memory will it consume? Thursday, July 26, 12
8. Find an element: Locate the value 22 in a list of 8 randomized elements. How long will it take? Thursday, July 26, 12
9. Find an element: Locate the value 22 in a list of 8 randomized elements. How long will it take? Thursday, July 26, 12
10. Find an element: Locate the value 22. 17 0 14 8 22 4 3 23 0 steps Thursday, July 26, 12
11. Find an element: Locate the value 22. 17 0 14 8 22 4 3 23 1 step Thursday, July 26, 12
12. Find an element: Locate the value 22. 17 0 14 8 22 4 3 23 2 steps Thursday, July 26, 12
13. Find an element: Locate the value 22. 17 0 14 8 22 4 3 23 3 steps Thursday, July 26, 12
14. Find an element: Locate the value 22. 17 0 14 8 22 4 3 23 4 steps Thursday, July 26, 12
15. Find an element: Locate the value 22. 17 0 14 8 22 4 3 23 5 steps Thursday, July 26, 12
16. Find an element: Locate the value 17. 17 0 14 8 22 4 3 23 0 steps Thursday, July 26, 12
17. Find an element: Locate the value 17. 17 0 14 8 22 4 3 23 1 step Thursday, July 26, 12
18. Find an element: Locate the value 23. 17 0 14 8 22 4 3 23 0 steps Thursday, July 26, 12
19. Find an element: Locate the value 23. 17 0 14 8 22 4 3 23 1 step Thursday, July 26, 12
20. Find an element: Locate the value 23. 17 0 14 8 22 4 3 23 2 steps Thursday, July 26, 12
21. Find an element: Locate the value 23. 17 0 14 8 22 4 3 23 3 steps Thursday, July 26, 12
22. Find an element: Locate the value 23. 17 0 14 8 22 4 3 23 4 steps Thursday, July 26, 12
23. Find an element: Locate the value 23. 17 0 14 8 22 4 3 23 5 steps Thursday, July 26, 12
24. Find an element: Locate the value 23. 17 0 14 8 22 4 3 23 6 steps Thursday, July 26, 12
25. Find an element: Locate the value 23. 17 0 14 8 22 4 3 23 7 steps Thursday, July 26, 12
26. Find an element: Locate the value 23. 17 0 14 8 22 4 3 23 8 steps Thursday, July 26, 12
27. Find an element: 17 0 14 8 22 4 3 23 How many steps, on average, will be needed? Thursday, July 26, 12
28. Find an element: 17 0 14 8 22 4 3 23 Assuming randomness, approximately 8 / 2 steps. Thursday, July 26, 12
29. Find an element: 17 0 14 8 22 4 3 23 Generalize to n elements, about n / 2. Thursday, July 26, 12
30. Find an element: 17 0 14 8 22 4 3 23 So, the time performance is a linear function of n. Thursday, July 26, 12
31. Find an element: 17 0 14 8 22 4 3 23 So, the time performance is a linear function of n. (times some constant) Thursday, July 26, 12
32. Find an element: 17 0 14 8 22 4 3 23 0 1 2 3 4 5 6 7 Thursday, July 26, 12
33. Find an element: What is the value at position 3? 17 0 14 8 22 4 3 23 0 1 2 3 4 5 6 7 Thursday, July 26, 12
34. Find an element: 17 0 14 8 22 4 3 23 What is the value at position 3? 0 1 2 3 4 5 6 7 The value is 8. Thursday, July 26, 12
35. Find an element: What is the value at position 5? 17 0 14 8 22 4 3 23 0 1 2 3 4 5 6 7 Thursday, July 26, 12
36. Find an element: What is the value at position 5? 17 0 14 8 22 4 3 23 0 1 2 3 4 5 6 7 The value is 4. Thursday, July 26, 12
37. Find an element: 17 0 14 8 22 4 3 23 How many steps, on average, will be needed? Thursday, July 26, 12
38. Find an element: 17 0 14 8 22 4 3 23 Only 1 step! Thursday, July 26, 12
39. Find an element: 17 0 14 8 22 4 3 23 So, the time performance is simply the cost of an access. Thursday, July 26, 12
40. Find an element: 17 0 14 8 22 4 3 23 So, the time performance is simply the cost of an access. (meaning, a constant) Thursday, July 26, 12
41. In general, the time performance of an algorithm can be described by an algebraic function times some constant. Thursday, July 26, 12
42. This is described by something referred to as Omicron notation, or simply O- notation. Thursday, July 26, 12
43. Linear search is O(n). Array access is O(C). Thursday, July 26, 12
44. Thursday, July 26, 12
45. Linear search is O(n). Array access is O(C). Thursday, July 26, 12
46. Linear search is O(n). Array access is O(C). Sorting is generally O(n log(n)) to O(n2). Thursday, July 26, 12
47. Thursday, July 26, 12
48. Sorting There is much more to algorithms than sorting, but sorting is foundational to many algorithms. In general, you should defer to the standard library sorting implementations for your language. But understanding and thinking about sort is valuable. Thursday, July 26, 12
49. Sorting Code examples of Mergesort and Quicksort... Thursday, July 26, 12
50. Bubble sort procedure bubbleSort( A : list of sortable items ) repeat swapped = false for i = 1 to length(A) - 1 inclusive do: /* if this pair is out of order */ if A[i-1] > A[i] then /* swap them and remember something changed */ swap( A[i-1], A[i] ) swapped = true end if end for until not swapped end procedure http://en.wikipedia.org/wiki/Bubble_sort Thursday, July 26, 12
51. Bubble sort procedure bubbleSort( A : list of sortable items ) repeat swapped = false for i = 1 to length(A) - 1 inclusive do: /* if this pair is out of order */ if A[i-1] > A[i] then /* swap them and remember something changed */ swap( A[i-1], A[i] ) swapped = true end if end for until not swapped end procedure http://en.wikipedia.org/wiki/Bubble_sort Bubble sort is O(?) Thursday, July 26, 12
52. Bubble sort procedure bubbleSort( A : list of sortable items ) repeat swapped = false for i = 1 to length(A) - 1 inclusive do: /* if this pair is out of order */ if A[i-1] > A[i] then /* swap them and remember something changed */ swap( A[i-1], A[i] ) swapped = true end if end for until not swapped end procedure http://en.wikipedia.org/wiki/Bubble_sort Bubble sort is O(n2) Thursday, July 26, 12
53. http://www.cs.columbia.edu/~hgs/teaching/isp/hw/qsort.c Real-world quick sort: Thursday, July 26, 12
54. Find an element via Binary Search: Locate the value 22 in a sorted list. 0 3 4 8 14 17 22 23 0 steps Thursday, July 26, 12
55. Find an element via Binary Search: Locate the value 22 in a sorted list. 0 3 4 8 14 17 22 23 0 steps Thursday, July 26, 12
56. Find an element via Binary Search: Locate the value 22 in a sorted list. 0 3 4 8 14 17 22 23 1 steps Thursday, July 26, 12
57. Find an element via Binary Search: Locate the value 22 in a sorted list. 0 3 4 8 14 17 22 23 1 steps Thursday, July 26, 12
58. Find an element via Binary Search: Locate the value 22 in a sorted list. 0 3 4 8 14 17 22 23 2 steps Thursday, July 26, 12
59. Find an element via Binary Search: Locate the value 22 in a sorted list. 0 3 4 8 14 17 22 23 2 steps Thursday, July 26, 12
60. Find an element via Binary Search: Locate the value 22 in a sorted list. 0 3 4 8 14 17 22 23 3 steps Thursday, July 26, 12
61. Find an element via Binary Search: Locate the value 22 in a sorted list. 0 3 4 8 14 17 22 23 3 steps = log2(8) Thursday, July 26, 12
62. Find an element via Binary Search: Locate the value 22 in a sorted list. 0 3 4 8 14 17 22 23 Binary Search is O(log2(n)) Thursday, July 26, 12
63. Thursday, July 26, 12
64. fin Thursday, July 26, 12