BIG O NOTATION How well do algorithms scale as the data size increases? 

Big O Notation ■ Big O notation is used in Computer Science to describe the performance or complexity of an algorithm. ■ It specifically gives an indication of how well an algorithm scales, it is not a measure of efficiency. ■ Big O specifically describes the worst-case scenario, and can be used to describe the execution time required or the space used (e.g. in memory or on disk) by an algorithm 

Big O Notation ■ Special notation is given to describe the performance as shown in the graph. 

Example Before we look at some specific examples let’s take a look at the following equation to get an idea of how algorithms can grow as the data set increases: 45n3 + 20n2 + 19 = ? What is the answer when n = 1? 

Example What is the answer when n = 2? 45n3 + 20n2 + 19 = ? 

Example What is the answer when n = 10? 45n3 + 20n2 + 19 = ? 

Example ■ Where n = 1 45n3 + 20n2 + 19 = 84 ■ Where n = 2 45n3 + 20n2 + 19 = 459 When n increases from 1 to just 2, 19 becomes much less significant. 

Example ■ Where n = 10 45n3 + 20n2 + 19 = 47,019 When n increases from 2 to 10, both the 19 and the 20n2 become less significant. ■ The real contributory factor is 45n3 = 45,000 ■ Algorithm is said to be order n3 or O (n3) 

Big O Notation ■ Here, in order of best performance to worst are some typical notations: – O(1) – O(log n) – O(n) – O(n log n) – O(n2) ■ Let’s look at some examples in python to observe the difference in time taken to complete algorithms. 

O(1) 
 ■ Executes in the same amount of time no matter how big the data set. ■ Irrelevant how many items are in a list, adding one more takes the same time regardless. 

A note on Logarithms 
 ■ In computer science log n means, the exponent you would need to raise the number 2 to to get n. ■ So imagine, if n = 16. Our exponent would be much much smaller than the actual n value. ■ It would be 4. 

O(log n)
 
 ■ A divide and conquer algorithm, after each step you effectively half your problem. If you double the problem size n, your algorithm needs only a constant number of steps more. ■ O(log n) code 

O(n)
 
 ■ O(n) describes an algorithm whose performance will grow linearly and in direct proportion to the size of the input data set. ■ Big O notation will always assume the upper limit where the algorithm will perform the maximum number of iterations. ■ O(n) code 

O(n log n)
 
 
 ■ Time to complete grows in direct proportion to the amount of data, best example of an algorithm of O(n log n) is merge sort. ■ O(n log n) code 

O(n 2 )
 
 
 ■ Represents an algorithm whose performance is directly proportional to the square of the size of the input data set. ■ This is common with algorithms that involve nested iterations over the data set. Deeper nested iterations will result in O(n3), O(n4) etc. ■ O(n2) code 

Calculating Big O – Rule of thumb
 
 ■ Simple programs can be analysed by counting the nested loops of the program. ■ A single loop over n items yields f(n) = n ■ A loop within a loop yields f(n) = n2 ■ A loop within a loop within a loop yields f(n) = n3 

Calculating Big O – Nested Loops
 
 
 ■ If you have nested loops, and the outer loop iterates i times and the inner loop iterates j times, the statements inside the inner loop will execute a total of i x j times. ■ This is because the inner loop will iterate j times for each of the i iterations of the outer loop. ■ This means that if both the outer and inner loop are dependent on the p roblem size n, the statements in the inner loop will be executed O(n2) times.