Analysis of Algorithms
Asymptotics / Complexity Theory
Slide 9
Slide 9 text
No content
Slide 10
Slide 10 text
Applied Maths → Computer Science
Slide 11
Slide 11 text
Complexity Analysis
We need a way
to define the runtime
of an algorithm
regardless of the machine
it’s currently running on.
Slide 12
Slide 12 text
Linear Search
Slide 13
Slide 13 text
Binary Search
Slide 14
Slide 14 text
Big O
Big O is a mathematical notation that describes the
limiting behaviour of a function when
the argument tends towards
a particular value or infinity.
Slide 15
Slide 15 text
The letter O is used because the growth rate of a function
is also referred to as the order of the function.
[...] an upper bound on
the growth rate of the function.
Big O
Slide 16
Slide 16 text
Big O
Slide 17
Slide 17 text
Big O
Length Iteration worst case
1 1
10 10
100 100
1000 1000
10000 10000
… ...
Slide 18
Slide 18 text
Big O
Slide 19
Slide 19 text
No content
Slide 20
Slide 20 text
Big O
Slide 21
Slide 21 text
How do we profile these algorithms?
Slide 22
Slide 22 text
Just count the loops, then?
Slide 23
Slide 23 text
Merge Sort
Slide 24
Slide 24 text
Merge Sort
Slide 25
Slide 25 text
Merge Sort
Slide 26
Slide 26 text
Merge Sort
O(n)
Slide 27
Slide 27 text
Merge Sort
Slide 28
Slide 28 text
Merge Sort
O(n)
Slide 29
Slide 29 text
Merge Sort
O(n)
O(n)
Slide 30
Slide 30 text
Merge Sort
O(n)
O(n)
?
Slide 31
Slide 31 text
Merge Sort
Recurrence relation
T(n) = c if n == 1
= 2T(n/2) + n
Slide 32
Slide 32 text
Merge Sort
Recurrence relation
T(n) = c if n == 1
= 2T(n/2) + n
T(n) = 2T(n/2) + n
Slide 33
Slide 33 text
Merge Sort
Recurrence relation
T(n) = c if n == 1
= 2T(n/2) + n
T(n) = 2T(n/2) + n
= 2 [ 2T(n/4) + n/2 ] + n
Slide 34
Slide 34 text
Merge Sort
Recurrence relation
T(n) = c if n == 1
= 2T(n/2) + n
T(n) = 2T(n/2) + n
= 2 [ 2T(n/4) + n/2 ] + n
= 4T(n/4) + 2n
= 4 [ 2T(n/8) n/4 ] + n
= 8T(n/8) + 3n
= 16T(n/16) + 4n
= ...
= (2^k)T(n/(2^k)) + kn
Slide 35
Slide 35 text
Recurrence relation
T(n) = c if n == 1
= 2T(n/2) + n
T(n) = (2^k)T(n/(2^k)) + kn
T(1) = c
Merge Sort
Slide 36
Slide 36 text
Recurrence relation
T(n) = c if n == 1
= 2T(n/2) + n
T(n) = (2^k)T(n/(2^k)) + kn
T(1) = c
n/(2^k) = 1
Merge Sort
Slide 37
Slide 37 text
Recurrence relation
T(n) = c if n == 1
= 2T(n/2) + n
T(n) = (2^k)T(n/(2^k)) + kn
T(1) = c
n/(2^k) = 1
2^k = n
Merge Sort
Slide 38
Slide 38 text
Recurrence relation
T(n) = c if n == 1
= 2T(n/2) + n
T(n) = (2^k)T(n/(2^k)) + kn
T(1) = c
n/(2^k) = 1
2^k = n
k = lg n
Merge Sort
Slide 39
Slide 39 text
Recurrence relation
T(n) = c if n == 1
= 2T(n/2) + n
T(n) = (2^k)T(n/(2^k)) + kn
k = lg n
T(n) =
Merge Sort
Slide 40
Slide 40 text
Recurrence relation
T(n) = c if n == 1
= 2T(n/2) + n
T(n) = (2^k)T(n/(2^k)) + kn
k = lg n
T(n) = (2^lg n)T(1) + n lg n
Merge Sort
Slide 41
Slide 41 text
Merge Sort
Recurrence relation
T(n) = c if n == 1
= 2T(n/2) + n
T(n) = (2^k)T(n/(2^k)) + kn
k = lg n
T(n) = (2^lg n)T(1) + n lg n
= cn + n lg n
Slide 42
Slide 42 text
Merge Sort
Recurrence relation
T(n) = c if n == 1
= 2T(n/2) + n
T(n) = (2^k)T(n/(2^k)) + kn
k = lg n
T(n) = (2^lg n)T(1) + n lg n
= cn + n lg n
O(n log n)
Slide 43
Slide 43 text
Does it matter?
Slide 44
Slide 44 text
Data Structures
Slide 45
Slide 45 text
Arrays vs Linked Lists
Slide 46
Slide 46 text
Binary Search Tree
Slide 47
Slide 47 text
B+ Trees
Slide 48
Slide 48 text
Back to Fibonacci… Why was it bad?
Slide 49
Slide 49 text
Fibonacci
Slide 50
Slide 50 text
Fibonacci
Slide 51
Slide 51 text
Fibonacci
Slide 52
Slide 52 text
Fibonacci
Slide 53
Slide 53 text
Fast Fibonacci
Slide 54
Slide 54 text
Paradigms / Techniques
Slide 55
Slide 55 text
Divide and Conquer
Dynamic Programming
Greedy
Paradigms / Techniques
Slide 56
Slide 56 text
The Classics
Slide 57
Slide 57 text
Fast Fourier Transform
Page Rank
Dijkstra
Miller-Rabin
The Classics