130

# Computational Complexity

June 07, 2020

## Transcript

3. ### Why is this relevant? • Allows to predict future performance

• Help us identify the limits of our computations • Minimize resources
4. ### Empirical methods •Are not platform-independent •Depends on the implementation •We

only know what we measure
5. ### Search Algorithm comparison Given a sorted list of size ,

ﬁnd the location of an arbitrary entry. N
6. ### Search Algorithm comparison Linear search: 1. Read the elements in

the list from left to right keeping track of the index 2. If the entry is found, return the index
7. ### Search Algorithm comparison Binary search: 1. Position in the middle

of the list (keeping track of the index) 2. If the entry is in the middle, return the index 3. If the value in the middle is greater than the entry, perform binary search on the left half of the list. 4. Else, perform binary search in the right half of the list.
8. ### Search Algorithm comparison Benchmarking: • A state-of-the-art computer will run

linear search. • A slower computer will run binary search. A B Source: https://en.wikipedia.org/wiki/Analysis_of_algorithms

11. ### Analysis of algorithms Assumptions • Lets deﬁne a model of

computation on which: • We have a single processing unit • Instructions are executed in order, one after the other. • Each instruction have its own cost. • Instructions are primitive, no magic procedures exists in the processing unit.

14. ### Input size • Depends on the problem being studied. •

Can involve more than one number.
15. ### Sample problem Find if sum is possible Given a list

of numbers and a number return whether any two numbers from the list add up to . L k k
16. ### Sample problem Find if sum is possible Given a list

of numbers and a number return whether any two numbers from the list add up to . L k k Input: [10, 15, 3, 7] 17 L = k = Output: YES! Explanation: 10 + 7 = 17 = k
17. ### Find if sum is possible Sol #1 SUM-EXISTS(L, k): 1.

for up to : 2. for up to : 3. if and : 4. return YES 5. return NO i = 0 L . length − 1 j = 0 L . length − 1 i ≠ j L[i] + L[j] = k
18. ### Find if sum is possible Sol #1 SUM-EXISTS(L, k): 1.

for up to : 2. for up to : 3. if and : 4. return YES 5. return NO i = 0 L . length − 1 j = 0 L . length − 1 i ≠ j L[i] + L[j] = k cost c1 c2 c3 c4 c5 times
19. ### Find if sum is possible Sol #1 SUM-EXISTS(L, k): 1.

for up to : 2. for up to : 3. if and : 4. return YES 5. return NO i = 0 L . length − 1 j = 0 L . length − 1 i ≠ j L[i] + L[j] = k times 1 1 n n ⋅ n = n2 n ⋅ n = n2 cost c1 c2 c3 c4 c5
20. ### Find if sum is possible Sol #1 SUM-EXISTS(L, k): 1.

for up to : 2. for up to : 3. if and : 4. return YES 5. return NO i = 0 L . length − 1 j = 0 L . length − 1 i ≠ j L[i] + L[j] = k times 1 1 n n ⋅ n = n2 n ⋅ n = n2 T(n) = c1 n + c2 n2 + c3 n2 + c4 + c5 cost c1 c2 c3 c4 c5
21. ### Find if sum is possible Sol #1 T(n) = c1

n + c2 n2 + c3 n2 + c4 + c5 T(n) = n2(c2 + c3 ) + nc1 + (c4 + c5) T(n) = an2 + bn + c
22. ### Find if sum is possible Sol #2 SUM-EXISTS_2(L, k): 1.

sort( ) 2. for in : 3. 4. if find( , ): 5. return YES 6. return NO L x L d ← k − x L d
23. ### Find if sum is possible Sol #2 SUM-EXISTS_2(L, k): 1.

sort( ) 2. for in : 3. 4. if find( , ): 5. return YES 6. return NO L x L d ← k − x L d cost n ⋅ log(n) c2 c3 log(n) c5 c6
24. ### Find if sum is possible Sol #2 SUM-EXISTS_2(L, k): 1.

sort( ) 2. for in : 3. 4. if find( , ): 5. return YES 6. return NO L x L d ← k − x L d cost n ⋅ log(n) c2 c3 log(n) c5 c6 times 1 n n n 1 1
25. ### Find if sum is possible Sol #2 SUM-EXISTS_2(L, k): 1.

sort( ) 2. for in : 3. 4. if find( , ): 5. return YES 6. return NO L x L d ← k − x L d cost n ⋅ log(n) c2 c3 log(n) c5 c6 times 1 n n n 1 1 T(n) = n ⋅ log(n) + c2 n + c3 n + n ⋅ log(n) + c5 + c6
26. ### Find if sum is possible Sol #2 T(n) = n

⋅ log(n) + c2 n + c3 n + n ⋅ log(n) + c5 + c6 T(n) = 2 ⋅ n log(n) + n(c2 + c3 n) + (c5 + c6 ) T(n) = a ⋅ n log(n) + bn + c

28. ### Find if sum is possible Sol #1 Sol #2 T(n)

= an2 + bn + c T(n) = a ⋅ n log(n) + bn + c

30. ### Find if sum is possible Sol #3 SUM-EXISTS_3(L, k): 1.

map[int]bool{} 2. for in : 3. if .contains( ) 4. return YES 5. .add( ) 6. return NO store ← x L store x store k − x
31. ### Find if sum is possible Sol #3 SUM-EXISTS_3(L, k): 1.

map[int]bool{} 2. for in : 3. if .contains( ) 4. return YES 5. .add( ) 6. return NO store ← x L store x store k − x Total Cost c1 c2 n c3 n c4 c5 n c6
32. ### Find if sum is possible Sol #3 SUM-EXISTS_3(L, k): 1.

map[int]bool{} 2. for in : 3. if .contains( ) 4. return YES 5. .add( ) 6. return NO store ← x L store x store k − x Total Cost c1 c2 n c3 n c4 c5 n c6 T(n) = an + b
33. ### Find if sum is possible Sol #1 T(n) = an2

+ bn + c T(n) = a ⋅ n log(n) + bn + c Sol #2 Sol #3 T(n) = an + b
34. ### Order of growth 13n2 + 1.2n + 9 Cost function

Order of growth 1 100 n3 + 1010n2 + 1 30n − 49 2n + n10 + n34 + n2 + 1
35. ### Order of growth 13n2 + 1.2n + 9 Cost function

Order of growth n2 1 100 n3 + 1010n2 + 1 n3 30n − 49 n 2n + n10 + n34 + n2 + 1 2n

37. ### Asymptotic notation Best (lower bound) Ω Θ Range (lower &

upper bound) Worst (upper bound) O
38. ### Asymptotic notation Big-O notation For a given function we deﬁne

as the set of functions: g(n) O(g(n)) there exist positive constants and such that O(g(n)) = {f(n) : c n0 0 ≤ f(n) ≤ cg(n)∀n ≥ n0 }
39. ### Asymptotic notation Big-O notation there exist positive constants and such

that O(g(n)) = {f(n) : c n0 0 ≤ f(n) ≤ cg(n)∀n ≥ n0 } Source: Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein. 2009. Introduction to Algorithms, Third Edition (3rd. ed.). The MIT Press.
40. ### Asymptotic notation Big-O notation Source: Thomas H. Cormen, Charles E.

Leiserson, Ronald L. Rivest, and Clifford Stein. 2009. Introduction to Algorithms, Third Edition (3rd. ed.). The MIT Press. Ω Θ O
41. ### Asymptotic notation Conventions • Even though is true, we use

the closest bound. • is abused and expressed as . • Constants are ignored, i.e: becomes • If is constant, we express it as . O(n) ⊂ O(n2) ⊂ O(n3) ⊂ O(cn) ⊂ . . . f(n) ∈ O(g(n)) f(n) = O(g(n)) O(3n2) O(n2) f(n) O(1)