Computational Complexity

Computational Complexity

3fafd51aa8dad766208e11eaaa1d534c?s=128

Pablo Ángel Trinidad

June 07, 2020
Tweet

Transcript

  1. Computational Complexity Analyzing algorithms Pablo Trinidad | @_pablotrinidad_

  2. Efficient?

  3. Why is this relevant? • Allows to predict future performance

    • Help us identify the limits of our computations • Minimize resources
  4. Empirical methods •Are not platform-independent •Depends on the implementation •We

    only know what we measure
  5. Search Algorithm comparison Given a sorted list of size ,

    find the location of an arbitrary entry. N
  6. Search Algorithm comparison Linear search: 1. Read the elements in

    the list from left to right keeping track of the index 2. If the entry is found, return the index
  7. Search Algorithm comparison Binary search: 1. Position in the middle

    of the list (keeping track of the index) 2. If the entry is in the middle, return the index 3. If the value in the middle is greater than the entry, perform binary search on the left half of the list. 4. Else, perform binary search in the right half of the list.
  8. Search Algorithm comparison Benchmarking: • A state-of-the-art computer will run

    linear search. • A slower computer will run binary search. A B Source: https://en.wikipedia.org/wiki/Analysis_of_algorithms
  9. Search Algorithm comparison Source: https://en.wikipedia.org/wiki/Analysis_of_algorithms

  10. Search Algorithm comparison Source: https://en.wikipedia.org/wiki/Analysis_of_algorithms

  11. Analysis of algorithms Assumptions • Lets define a model of

    computation on which: • We have a single processing unit • Instructions are executed in order, one after the other. • Each instruction have its own cost. • Instructions are primitive, no magic procedures exists in the processing unit.
  12. Running time
 
 Number of primitive operations executed

  13. Running time T(n)

  14. Input size • Depends on the problem being studied. •

    Can involve more than one number.
  15. Sample problem Find if sum is possible Given a list

    of numbers and a number return whether any two numbers from the list add up to . L k k
  16. Sample problem Find if sum is possible Given a list

    of numbers and a number return whether any two numbers from the list add up to . L k k Input: [10, 15, 3, 7] 17 L = k = Output: YES! Explanation: 10 + 7 = 17 = k
  17. Find if sum is possible Sol #1 SUM-EXISTS(L, k): 1.

    for up to : 2. for up to : 3. if and : 4. return YES 5. return NO i = 0 L . length − 1 j = 0 L . length − 1 i ≠ j L[i] + L[j] = k
  18. Find if sum is possible Sol #1 SUM-EXISTS(L, k): 1.

    for up to : 2. for up to : 3. if and : 4. return YES 5. return NO i = 0 L . length − 1 j = 0 L . length − 1 i ≠ j L[i] + L[j] = k cost c1 c2 c3 c4 c5 times
  19. Find if sum is possible Sol #1 SUM-EXISTS(L, k): 1.

    for up to : 2. for up to : 3. if and : 4. return YES 5. return NO i = 0 L . length − 1 j = 0 L . length − 1 i ≠ j L[i] + L[j] = k times 1 1 n n ⋅ n = n2 n ⋅ n = n2 cost c1 c2 c3 c4 c5
  20. Find if sum is possible Sol #1 SUM-EXISTS(L, k): 1.

    for up to : 2. for up to : 3. if and : 4. return YES 5. return NO i = 0 L . length − 1 j = 0 L . length − 1 i ≠ j L[i] + L[j] = k times 1 1 n n ⋅ n = n2 n ⋅ n = n2 T(n) = c1 n + c2 n2 + c3 n2 + c4 + c5 cost c1 c2 c3 c4 c5
  21. Find if sum is possible Sol #1 T(n) = c1

    n + c2 n2 + c3 n2 + c4 + c5 T(n) = n2(c2 + c3 ) + nc1 + (c4 + c5) T(n) = an2 + bn + c
  22. Find if sum is possible Sol #2 SUM-EXISTS_2(L, k): 1.

    sort( ) 2. for in : 3. 4. if find( , ): 5. return YES 6. return NO L x L d ← k − x L d
  23. Find if sum is possible Sol #2 SUM-EXISTS_2(L, k): 1.

    sort( ) 2. for in : 3. 4. if find( , ): 5. return YES 6. return NO L x L d ← k − x L d cost n ⋅ log(n) c2 c3 log(n) c5 c6
  24. Find if sum is possible Sol #2 SUM-EXISTS_2(L, k): 1.

    sort( ) 2. for in : 3. 4. if find( , ): 5. return YES 6. return NO L x L d ← k − x L d cost n ⋅ log(n) c2 c3 log(n) c5 c6 times 1 n n n 1 1
  25. Find if sum is possible Sol #2 SUM-EXISTS_2(L, k): 1.

    sort( ) 2. for in : 3. 4. if find( , ): 5. return YES 6. return NO L x L d ← k − x L d cost n ⋅ log(n) c2 c3 log(n) c5 c6 times 1 n n n 1 1 T(n) = n ⋅ log(n) + c2 n + c3 n + n ⋅ log(n) + c5 + c6
  26. Find if sum is possible Sol #2 T(n) = n

    ⋅ log(n) + c2 n + c3 n + n ⋅ log(n) + c5 + c6 T(n) = 2 ⋅ n log(n) + n(c2 + c3 n) + (c5 + c6 ) T(n) = a ⋅ n log(n) + bn + c
  27. Find if sum is possible Sol #1 Sol #2

  28. Find if sum is possible Sol #1 Sol #2 T(n)

    = an2 + bn + c T(n) = a ⋅ n log(n) + bn + c
  29. Find if sum is possible

  30. Find if sum is possible Sol #3 SUM-EXISTS_3(L, k): 1.

    map[int]bool{} 2. for in : 3. if .contains( ) 4. return YES 5. .add( ) 6. return NO store ← x L store x store k − x
  31. Find if sum is possible Sol #3 SUM-EXISTS_3(L, k): 1.

    map[int]bool{} 2. for in : 3. if .contains( ) 4. return YES 5. .add( ) 6. return NO store ← x L store x store k − x Total Cost c1 c2 n c3 n c4 c5 n c6
  32. Find if sum is possible Sol #3 SUM-EXISTS_3(L, k): 1.

    map[int]bool{} 2. for in : 3. if .contains( ) 4. return YES 5. .add( ) 6. return NO store ← x L store x store k − x Total Cost c1 c2 n c3 n c4 c5 n c6 T(n) = an + b
  33. Find if sum is possible Sol #1 T(n) = an2

    + bn + c T(n) = a ⋅ n log(n) + bn + c Sol #2 Sol #3 T(n) = an + b
  34. Order of growth 13n2 + 1.2n + 9 Cost function

    Order of growth 1 100 n3 + 1010n2 + 1 30n − 49 2n + n10 + n34 + n2 + 1
  35. Order of growth 13n2 + 1.2n + 9 Cost function

    Order of growth n2 1 100 n3 + 1010n2 + 1 n3 30n − 49 n 2n + n10 + n34 + n2 + 1 2n
  36. Asymptotic notation

  37. Asymptotic notation Best (lower bound) Ω Θ Range (lower &

    upper bound) Worst (upper bound) O
  38. Asymptotic notation Big-O notation For a given function we define

    as the set of functions: g(n) O(g(n)) there exist positive constants and such that O(g(n)) = {f(n) : c n0 0 ≤ f(n) ≤ cg(n)∀n ≥ n0 }
  39. Asymptotic notation Big-O notation there exist positive constants and such

    that O(g(n)) = {f(n) : c n0 0 ≤ f(n) ≤ cg(n)∀n ≥ n0 } Source: Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, and Clifford Stein. 2009. Introduction to Algorithms, Third Edition (3rd. ed.). The MIT Press.
  40. Asymptotic notation Big-O notation Source: Thomas H. Cormen, Charles E.

    Leiserson, Ronald L. Rivest, and Clifford Stein. 2009. Introduction to Algorithms, Third Edition (3rd. ed.). The MIT Press. Ω Θ O
  41. Asymptotic notation Conventions • Even though is true, we use

    the closest bound. • is abused and expressed as . • Constants are ignored, i.e: becomes • If is constant, we express it as . O(n) ⊂ O(n2) ⊂ O(n3) ⊂ O(cn) ⊂ . . . f(n) ∈ O(g(n)) f(n) = O(g(n)) O(3n2) O(n2) f(n) O(1)
  42. Big-O comparison Complexity/n A LOT!

  43. Big-O comparison Source: https://www.bigocheatsheet.com/

  44. Big-O comparison

  45. We want to find efficient procedures to solve large-scale problems.

  46. bit.ly/pm-comp-x2 Computational Complexity