Algorithm
Complexity (BigO)
Algorithmic Complexity
1
2
Agenda
●Why use BigO Notation?
●How do we use BigO Notation?
●Examples
○The Basics
○Common Pitfalls
Algorithmic Complexity
3
Properties of Sorting Algorithms
●Time Complexity (Big-O)
i. “How does the amount of work performed scale with input size?”
Sorting
4
A multitude of factors can affect the pure
“speed” of a program at runtime…
- computer type
- RAM
- are you running other programs at the
same time?
Big O Notation CSCI Tutors | CSCI 1933 | Midterm 1 Review by Sebastian Helgeson | https://z.umn.edu/CSCI1933BigO
5
Since hardware and runtime state varies, we
need to quantify the relative speed of an
algorithm without using
hours/minutes/seconds/etc.
That’s where big-O notation is useful.
Big O Notation CSCI Tutors | CSCI 1933 | Midterm 1 Review by Sebastian Helgeson | https://z.umn.edu/CSCI1933BigO
6
In big-O notation, we’re mainly concerned with how much work an
algorithm takes in relation to the size of a problem
time (# of operations)
size of the problem (n)
Big O Notation
Graphic from CSCI Tutors | CSCI 1933 | Midterm 1 Review by Sebastian Helgeson |
7
As problem size increases, the # of operations an algorithm performs
also increases.
time (# of operations)
size of the problem (n)
Big O Notation
Graphic from CSCI Tutors | CSCI 1933 | Midterm 1 Review by Sebastian Helgeson |
8
sorting a list of 10 should take a lot less time then sorting a list of 100
items, sorting 200 items would take even longer
time (# of operations)
size of the problem (n)
Big O Notation
n = 10
n = 100
n = 200
Graphic from CSCI Tutors | CSCI 1933 | Midterm 1 Review by Sebastian Helgeson |
9
The overall shape of this graph shapes the complexity
time (# of operations)
size of the problem (n)
Big O Notation
n = 10
n = 100
n = 200
Graphic from CSCI Tutors | CSCI 1933 | Midterm 1 Review by Sebastian Helgeson |
10
Is it linear?
time (# of operations)
size of the problem (n)
Big O Notation
Graphic from CSCI Tutors | CSCI 1933 | Midterm 1 Review by Sebastian Helgeson |
11
Is it linear? Is it logarithmic?
time (# of operations)
size of the problem (n)
Big O Notation
Graphic from CSCI Tutors | CSCI 1933 | Midterm 1 Review by Sebastian Helgeson |
12
Is it linear? Is it logarithmic? Is it quadratic?
time (# of operations)
size of the problem (n)
Big O Notation
Graphic from CSCI Tutors | CSCI 1933 | Midterm 1 Review by Sebastian Helgeson |
13
Is it linear? Is it logarithmic? Is it quadratic?
time (# of operations)
size of the problem (n)
Big O Notation
O(n2)O(n)
O(log(n))
Graphic from CSCI Tutors | CSCI 1933 | Midterm 1 Review by Sebastian Helgeson |
14
Let’s talk about constants
time (# of operations)
size of the problem (n)
Big O Notation
O(n2)O(n)
O(log(n))
Graphic from CSCI Tutors | CSCI 1933 | Midterm 1 Review by Sebastian Helgeson |
15
Let’s talk about constants. O(2n) is objectively worse than O(n)...
yet, it’s still not nearly as bad as O(n2)
time (# of operations)
size of the problem (n)
Big O Notation
O(n2)O(n)
O(log(n))
O(2n)
Graphic from CSCI Tutors | CSCI 1933 | Midterm 1 Review by Sebastian Helgeson |
16
And O(n) is objectively better than O(2n)... yet not nearly as good
as O(log(n))
time (# of operations)
size of the problem (n)
Big O Notation
O(n2)O(n)
O(log(n))
O(2n)
Graphic from CSCI Tutors | CSCI 1933 | Midterm 1 Review by Sebastian Helgeson |
17
Let’s consider added constants with O(n + 50)
time (# of operations)
size of the problem (n)
Big O Notation
O(n2)O(n)
O(log(n))
Graphic from CSCI Tutors | CSCI 1933 | Midterm 1 Review by Sebastian Helgeson |
O(n+50)
18
O(n+50) begins performing more work than O(n2) and anything without
constants added.
time (# of operations)
size of the problem (n)
Big O Notation
O(n2)O(n)
O(log(n))
Graphic from CSCI Tutors | CSCI 1933 | Midterm 1 Review by Sebastian Helgeson |
O(n+50)
19
As the problem size grows, the effect of the constant is lessened and we
reach a point beyond which when O(n2) is once again greater than
O(n+50).
time (# of operations)
size of the problem (n)
Big O Notation
O(n2)O(n)
O(log(n))
Graphic from CSCI Tutors | CSCI 1933 | Midterm 1 Review by Sebastian Helgeson |
O(n+50)
20
These constants matter, but they play much less of a role than the
actual functions themselves, because of this, we don’t include any
constants when we talk about big-O notation.
time (# of operations)
size of the problem (n)
Big O Notation
O(n2)O(n)
O(log(n))
O(2n)
Graphic from CSCI Tutors | CSCI 1933 | Midterm 1 Review by Sebastian Helgeson |
21
BigO notation is considered “worst case”
runtime, so we evaluate with respect to what
the upper bound is.
As problem size grows toward infinity,
constants get infinitely small so we drop them.
Algorithmic Complexity
22
O(n+50) →O(n)
time (# of operations)
size of the problem (n)
Big O Notation
O(n2)O(n)
O(log(n))
Graphic from CSCI Tutors | CSCI 1933 | Midterm 1 Review by Sebastian Helgeson |
O(n+50)
23
O(2n) →O(n)
time (# of operations)
size of the problem (n)
Big O Notation
O(n2)O(n)
O(log(n))
O(2n)
Graphic from CSCI Tutors | CSCI 1933 | Midterm 1 Review by Sebastian Helgeson |
24
Consider something stranger:
O(n + logn) →?????
time (# of operations)
size of the problem (n)
Big O Notation
O(n2)O(n)
O(log(n))
Graphic from CSCI Tutors | CSCI 1933 | Midterm 1 Review by Sebastian Helgeson |
25
Consider something stranger:
O(n + logn) →?????
time (# of operations)
size of the problem (n)
Big O Notation
O(n2)O(n)
O(log(n))
Graphic from CSCI Tutors | CSCI 1933 | Midterm 1 Review by Sebastian Helgeson |
O(n+log(n))
26
The larger term dominates as we approach infinity. (n > logn)
O(n + logn) →?????
time (# of operations)
size of the problem (n)
Big O Notation
O(n2)O(n)
O(log(n))
Graphic from CSCI Tutors | CSCI 1933 | Midterm 1 Review by Sebastian Helgeson |
O(n+log(n))
27
The larger term dominates as we approach infinity. (n > logn)
O(n + logn) →O(n)
time (# of operations)
size of the problem (n)
Big O Notation
O(n2)O(n)
O(log(n))
Graphic from CSCI Tutors | CSCI 1933 | Midterm 1 Review by Sebastian Helgeson |
O(n+log(n))
28
Special Case: Anything that runs in constant time… O(1)
time (# of operations)
size of the problem (n)
Big O Notation
O(n2)O(n)
O(log(n))
O(2n)
O(1)
Graphic from CSCI Tutors | CSCI 1933 | Midterm 1 Review by Sebastian Helgeson |
29
How is Big O calculated when looking at
code?
Algorithmic Complexity
30
How is Big O calculated when looking at
code?
○CS1 (1133) Indicator - “Count the loops”
■Good shortcut, but it excludes some nuance we
expect.
Algorithmic Complexity
31
How is Big O calculated when looking at
code?
○CS1 (1133) Indicator - “Count the loops”
■Good shortcut, but it excludes some nuance we
expect.
○Full Review
■Quantify the cost at each line, then evaluate the
total.
Algorithmic Complexity
32
Big O Notation CSCI Tutors | CSCI 1933 | Midterm 1 Review by Sebastian Helgeson | https://z.umn.edu/CSCI1933BigO
public static int method1(int n){
int total = 0;
System.out.println(“Hello World”);
for(int i = 0; i < n; i++){
total += n;
}
for(int j = 0; j < 500; j++){
System.out.println(“Goodbye.”);
}
return total;
}
To calculate BigO:
1. Count how many times each line is going to be run in relation to the size of the
input (the parameter of the function).
2. Choose largest term (this will dominate).
33
Big O Notation CSCI Tutors | CSCI 1933 | Midterm 1 Review by Sebastian Helgeson | https://z.umn.edu/CSCI1933BigO
public static int method1(int n){
int total = 0; 1
System.out.println(“Hello World”); 1
for(int i = 0; i < n; i++){
total += n;
}
for(int j = 0; j < 500; j++){
System.out.println(“Goodbye.”);
}
return total;
}
So how do we actually calculate an algorithm’s big O notation?
You count how many times each line is going to be run in relation to the size of the
input (the parameter of the function). The largest power is your big O.
34
Big O Notation CSCI Tutors | CSCI 1933 | Midterm 1 Review by Sebastian Helgeson | https://z.umn.edu/CSCI1933BigO
public static int method1(int n){
int total = 0; 1
System.out.println(“Hello World”); 1
for(int i = 0; i < n; i++){ n
total += n; n
}
for(int j = 0; j < 500; j++){
System.out.println(“Goodbye.”);
}
return total;
}
So how do we actually calculate an algorithm’s big O notation?
You count how many times each line is going to be run in relation to the size of the
input (the parameter of the function). The largest power is your big O.
35
Big O Notation CSCI Tutors | CSCI 1933 | Midterm 1 Review by Sebastian Helgeson | https://z.umn.edu/CSCI1933BigO
public static int method1(int n){
int total = 0; 1
System.out.println(“Hello World”); 1
for(int i = 0; i < n; i++){ n
total += n; n
}
for(int j = 0; j < 500; j++){ 500
System.out.println(“Goodbye.”); 500
}
return total; 1
}
So how do we actually calculate an algorithm’s big O notation?
You count how many times each line is going to be run in relation to the size of the
input (the parameter of the function). The largest power is your big O.
36
Big O Notation CSCI Tutors | CSCI 1933 | Midterm 1 Review by Sebastian Helgeson | https://z.umn.edu/CSCI1933BigO
public static int method1(int n){
int total = 0; 1
System.out.println(“Hello World”); 1
for(int i = 0; i < n; i++){ n
total += n; n
}
for(int j = 0; j < 500; j++){ 500
System.out.println(“Goodbye.”); 500
}
return total; 1
}
So how do we actually calculate an algorithm’s big O notation?
You count how many times each line is going to be run in relation to the size of the
input (the parameter of the function). The largest power is your big O.
37
Big O Notation CSCI Tutors | CSCI 1933 | Midterm 1 Review by Sebastian Helgeson | https://z.umn.edu/CSCI1933BigO
public static int method1(int n){
int total = 0; 1
System.out.println(“Hello World”); 1
for(int i = 0; i < n; i++){ n
total += n; n
}
for(int j = 0; j < 500; j++){ 1 - constant cost
System.out.println(“Goodbye.”); 1 - constant cost
}
return total; 1
}
Step 1: Drop Constants (500 →1)
(if we just care about BigO, overall speed these still matter)
38
Big O Notation CSCI Tutors | CSCI 1933 | Midterm 1 Review by Sebastian Helgeson | https://z.umn.edu/CSCI1933BigO
public static int method1(int n){
int total = 0; 1
System.out.println(“Hello World”); 1
for(int i = 0; i < n; i++){ n
total += n; n
}
for(int j = 0; j < 500; j++){ 1
System.out.println(“Goodbye.”); 1
}
return total; 1
}
Step 2: Find largest n term
39
Algorithmic Complexity
public static int method1(int n){
int total = 0; 1
System.out.println(“Hello World”); 1
for(int i = 0; i < n; i++){ n
total += n; n
}
for(int j = 0; j < 500; j++){ 1
System.out.println(“Goodbye.”); 1
}
return total; 1
}
Step 3: Done
O(n)
40
In More Depth (Preview):
1. Constant cost outside of loops (3)
2. Cost of loops
a. 4n + 2
b. 2(500) + 2 + 500
3. Add everything together: (4n+2) + 1502 + 3
4. Drop constants: O(4n+1507) →(n)
Algorithmic Complexity
41
Algorithmic Complexity
public static int method1(int n){
int total = 0;
System.out.println(“Hello World”);
for(int i = 0; i < n; i++){ ( ; ; ; )
total += n;
}
for(int j = 0; j < 500; j++){
System.out.println(“Goodbye.”);
}
return total;
}
True Loop Calculation
42
Algorithmic Complexity
public static int method1(int n){
int total = 0;
System.out.println(“Hello World”);
for(int i = 0; i < n; i++){ ( 1 ; ; )
total += n;
}
for(int j = 0; j < 500; j++){
System.out.println(“Goodbye.”);
}
return total;
}
Check all components
-int i = 0
43
Algorithmic Complexity
public static int method1(int n){
int total = 0;
System.out.println(“Hello World”);
for(int i = 0; i < n; i++){ ( 1 ; ; )
total += n;
}
for(int j = 0; j < 500; j++){
System.out.println(“Goodbye.”);
}
return total;
}
Check all components
-int i = 0 (this happens once)
-i < n (checked at start of each loop, account for stop)
44
Algorithmic Complexity
public static int method1(int n){
int total = 0;
System.out.println(“Hello World”);
for(int i = 0; i < n; i++){ ( 1 ; n+1; )
total += n;
}
for(int j = 0; j < 500; j++){
System.out.println(“Goodbye.”);
}
return total;
}
Check all components
-int i = 0 (this happens once)
-i < n (checked at start of each loop, account for stop)
45
Algorithmic Complexity
public static int method1(int n){
int total = 0;
System.out.println(“Hello World”);
for(int i = 0; i < n; i++){ ( 1 ; n+1; )
total += n;
}
for(int j = 0; j < 500; j++){
System.out.println(“Goodbye.”);
}
return total;
}
Check all components
-int i = 0 (this happens once)
-i < n (checked at start of each loop, account for stop)
-i++ (once per loop at end)
46
Algorithmic Complexity
public static int method1(int n){
int total = 0;
System.out.println(“Hello World”);
for(int i = 0; i < n; i++){ ( 1 ; n+1; n)
total += n;
}
for(int j = 0; j < 500; j++){
System.out.println(“Goodbye.”);
}
return total;
}
Check all components
-int i = 0 (this happens once)
-i < n (checked at start of each loop, account for stop)
-i++ (once per loop at end)
47
Algorithmic Complexity
public static int method1(int n){
int total = 0;
System.out.println(“Hello World”);
for(int i = 0; i < n; i++){ ( 1 ; n+1; n)
total += n;
}
for(int j = 0; j < 500; j++){
System.out.println(“Goodbye.”);
}
return total;
}
Check all components
-int i = 0 (this happens once)
-i < n (checked at start of each loop, account for stop)
-i ++ (once per loop at end)
-total += n (both add and assign, cost of 2 done n times)
48
Algorithmic Complexity
public static int method1(int n){
int total = 0;
System.out.println(“Hello World”);
for(int i = 0; i < n; i++){ ( 1 ; n+1; n)
total += n; 2 * n
}
for(int j = 0; j < 500; j++){
System.out.println(“Goodbye.”);
}
return total;
}
Check all components
-int i = 0 (this happens once)
-i < n (checked at start of each loop, account for stop)
-i ++ (once per loop at end)
-total += n (both add and assign, cost of 2 done n times)
49
Algorithmic Complexity
public static int method1(int n){
int total = 0;
System.out.println(“Hello World”);
for(int i = 0; i < n; i++){ ( 1 ; n+1; n)
total += n; 2 * n
}
for(int j = 0; j < 500; j++){
System.out.println(“Goodbye.”);
}
return total;
}
Check all components
-int i = 0 (this happens once)
-i < n (checked at start of each loop, account for stop)
-i ++ (once per loop at end)
-total += n (both add and assign, cost of 2 done n times)
Loop Cost:
1 + (n+1) + n + (2n)
50
Algorithmic Complexity
public static int method1(int n){
int total = 0;
System.out.println(“Hello World”);
for(int i = 0; i < n; i++){ ( 1 ; n+1; n)
total += n; 2 * n
}
for(int j = 0; j < 500; j++){
System.out.println(“Goodbye.”);
}
return total;
}
Check all components
-int i = 0 (this happens once)
-i < n (checked at start of each loop, account for stop)
-i ++ (once per loop at end)
-total += n (both add and assign, cost of 2 done n times)
Loop Cost:
4n + 2
51
Big O Notation CSCI Tutors | CSCI 1933 | Midterm 1 Review by Sebastian Helgeson
public static int method2(int n){
int total = 0;
System.out.println(“Hello World”);
for(int i = 0; i < n; i++){
total += n;
for(int j = 0; j < 200; j++){
System.out.println(“cool”);
total += j;
}
for(int k = 0; k < n-3; k++){
total += k;
}
}
return total;
}
Let’s do another example
52
Big O Notation CSCI Tutors | CSCI 1933 | Midterm 1 Review by Sebastian Helgeson | https://z.umn.edu/CSCI1933BigO
public static int method2(int n){
int total = 0;
System.out.println(“Hello World”);
for(int i = 0; i < n; i++){
total += n;
for(int j = 0; j < 200; j++){
System.out.println(“cool”);
total += j;
}
for(int k = 0; k < n-3; k++){
total += k;
}
}
return total;
}
Let’s do another example
1
1
53
Big O Notation CSCI Tutors | CSCI 1933 | Midterm 1 Review by Sebastian Helgeson | https://z.umn.edu/CSCI1933BigO
public static int method2(int n){
int total = 0;
System.out.println(“Hello World”);
for(int i = 0; i < n; i++){
total += n;
for(int j = 0; j < 200; j++){
System.out.println(“cool”);
total += j;
}
for(int k = 0; k < n-3; k++){
total += k;
}
}
return total;
}
Let’s do another example
1
1
n
54
Big O Notation CSCI Tutors | CSCI 1933 | Midterm 1 Review by Sebastian Helgeson | https://z.umn.edu/CSCI1933BigO
public static int method2(int n){
int total = 0;
System.out.println(“Hello World”);
for(int i = 0; i < n; i++){
total += n;
for(int j = 0; j < 200; j++){
System.out.println(“cool”);
total += j;
}
for(int k = 0; k < n-3; k++){
total += k;
}
}
return total;
}
Let’s do another example
1
1
n
n
n
n
n
n
n
n
55
Big O Notation CSCI Tutors | CSCI 1933 | Midterm 1 Review by Sebastian Helgeson | https://z.umn.edu/CSCI1933BigO
public static int method2(int n){
int total = 0;
System.out.println(“Hello World”);
for(int i = 0; i < n; i++){
total += n;
for(int j = 0; j < 200; j++){
System.out.println(“cool”);
total += j;
}
for(int k = 0; k < n-3; k++){
total += k;
}
}
return total;
}
Let’s do another example
1
1
n
n
n
n
n
n
n
n
200
56
Big O Notation CSCI Tutors | CSCI 1933 | Midterm 1 Review by Sebastian Helgeson | https://z.umn.edu/CSCI1933BigO
public static int method2(int n){
int total = 0;
System.out.println(“Hello World”);
for(int i = 0; i < n; i++){
total += n;
for(int j = 0; j < 200; j++){
System.out.println(“cool”);
total += j;
}
for(int k = 0; k < n-3; k++){
total += k;
}
}
return total;
}
Let’s do another example
1
1
n
n
n
n * 200
n * 200
n * 200
n
n
200
57
Big O Notation CSCI Tutors | CSCI 1933 | Midterm 1 Review by Sebastian Helgeson | https://z.umn.edu/CSCI1933BigO
public static int method2(int n){
int total = 0;
System.out.println(“Hello World”);
for(int i = 0; i < n; i++){
total += n;
for(int j = 0; j < 200; j++){
System.out.println(“cool”);
total += j;
}
for(int k = 0; k < n-3; k++){
total += k;
}
}
return total;
}
Let’s do another example
1
1
n
n
n
n * 200
n * 200
n * 200
n
n-3 n
200
58
Big O Notation CSCI Tutors | CSCI 1933 | Midterm 1 Review by Sebastian Helgeson | https://z.umn.edu/CSCI1933BigO
public static int method2(int n){
int total = 0;
System.out.println(“Hello World”);
for(int i = 0; i < n; i++){
total += n;
for(int j = 0; j < 200; j++){
System.out.println(“cool”);
total += j;
}
for(int k = 0; k < n-3; k++){
total += k;
}
}
return total;
}
Let’s do another example
1
1
n
n
n
n * 200
n * 200
n * 200
n * (n-3)
n-3 n * (n-3)
200
1
59
Big O Notation CSCI Tutors | CSCI 1933 | Midterm 1 Review by Sebastian Helgeson | https://z.umn.edu/CSCI1933BigO
public static int method2(int n){
int total = 0;
System.out.println(“Hello World”);
for(int i = 0; i < n; i++){
total += n;
for(int j = 0; j < 200; j++){
System.out.println(“cool”);
total += j;
}
for(int k = 0; k < n-3; k++){
total += k;
}
}
return total;
}
Drop all the constants
1
1
n
n
n
n * 200
n * 200
n * 200
n * (n-3)
n-3 n * (n-3)
200
1
60
Big O Notation CSCI Tutors | CSCI 1933 | Midterm 1 Review by Sebastian Helgeson | https://z.umn.edu/CSCI1933BigO
public static int method2(int n){
int total = 0;
System.out.println(“Hello World”);
for(int i = 0; i < n; i++){
total += n;
for(int j = 0; j < 200; j++){
System.out.println(“cool”);
total += j;
}
for(int k = 0; k < n-3; k++){
total += k;
}
}
return total;
}
Drop all the constants
1
1
n
n
n
n
n
n
n * (n)
n-3 n * (n)
200
1
61
Big O Notation CSCI Tutors | CSCI 1933 | Midterm 1 Review by Sebastian Helgeson | https://z.umn.edu/CSCI1933BigO
public static int method2(int n){
int total = 0;
System.out.println(“Hello World”);
for(int i = 0; i < n; i++){
total += n;
for(int j = 0; j < 200; j++){
System.out.println(“cool”);
total += j;
}
for(int k = 0; k < n-3; k++){
total += k;
}
}
return total;
}
Simplify
1
1
n
n
n
n
n
n
n2
n-3 n2
200
1
62
Big O Notation CSCI Tutors | CSCI 1933 | Midterm 1 Review by Sebastian Helgeson | https://z.umn.edu/CSCI1933BigO
public static int method2(int n){
int total = 0;
System.out.println(“Hello World”);
for(int i = 0; i < n; i++){
total += n;
for(int j = 0; j < 200; j++){
System.out.println(“cool”);
total += j;
}
for(int k = 0; k < n-3; k++){
total += k;
}
}
return total;
}
Simplify
1
1
n
n
n
n
n
n
n2
n-3 n2
200
1
O(n2)
63
Shortcuts
○CS1 (1133) Indicator - “Count the loops”
■Excludes some nuance we expect.
Algorithmic Complexity
64
Big O Notation
public static int method1(int n){
int total = 0;
System.out.println(“Hello World”);
for(int i = n; i > 0; i/=2){
total += n;
}
for(int j = 0; j < 500; j++){
System.out.println(“Goodbye.”);
}
return total;
}
What is the time complexity of modified method1?
65
Big O Notation
public static int method1(int n){
int total = 0;
System.out.println(“Hello World”);
for(int i = n; i > 0; i/=2){
total += n;
}
for(int j = 0; j < 500; j++){
System.out.println(“Goodbye.”);
}
return total;
}
What is the time complexity of modified method1?
Temptation: 2 loops, not nested →n+500 →O(n)
66
Big O Notation
public static int method1(int n){
int total = 0;
System.out.println(“Hello World”);
for(int i = n; i > 0; i/=2){
total += n;
}
for(int j = 0; j < 500; j++){
System.out.println(“Goodbye.”);
}
return total;
}
What is the time complexity of modified method1?
When looking at loops, pay attention to the increment as well as any nesting.
67
Big O Notation
public static int method1(int n){
int total = 0;
System.out.println(“Hello World”);
for(int i = n; i > 0; i/=2){
total += n;
}
for(int j = 0; j < 500; j++){
System.out.println(“Goodbye.”);
}
return total;
}
What is the time complexity of modified method1?
i/=2 is halving the problem each step in the loop, so this method is…
68
Big O Notation
public static int method1(int n){
int total = 0;
System.out.println(“Hello World”);
for(int i = n; i > 0; i/=2){
total += n;
}
for(int j = 0; j < 500; j++){
System.out.println(“Goodbye.”);
}
return total;
}
What is the time complexity of modified method1?
O(logn)
