Recursion

CSCI 1913 – Introduction to Algorithms, Data Structures, and Program Development
Adriana Picoral

Recursion is Magic

• To write a recursive function, you need to believe that it already works (but only for a smaller version of the same problem).
• Sometimes that step is called the “recursive leap of faith”

What is recursion?

• In general? Something defined in terms of itself
• In Computer Science? A recursive function that calls itself
• An alternative to loops

Recursion inside recursion inside recursion inside recursion inside…

Recursion vs. Loops

• Loops are easier to understand conceptually
• Loops are generally faster
• In many problems, loops will be fewer lines of code
• Anything you can do with recursion, you can do with loops (and vice versa)
• So why use recursion?

Recursively Defined Problems

• There are some problems where the recursive solution is significantly less code, easier to understand, and requires fewer constructs.
• These are generally problems where the problem structure is tree-like in nature
  • For example, visiting all nodes in a binary tree in-order, you recursively visit the left subtree, process the current node, then recursively visit the right subtree.

Function Stacks

• Some programming languages use what is called a function call stack
• A function call stack is a data structure that holds a function name, arguments, and a return point for functions
• A new function adds to the stack building up
• The implementation we are all likely using of Python (CPython) is implemented using function call stacks

Function Stacks

What will this print?

def fun_a():
  print("A")
  fun_b()
  print("Another A")
  
def fun_b():
  print("B")
  
def fun_c():
  print("C")
  
if __name__ == "__main__":
  fun_a()
  fun_b()
  fun_c()

Function Stacks – recursion

Repetition through recursion – function calls itself repeatedly until a specific end condition (base case) is met.

What will my_recursive_function(10, 5) return?

def my_recursive_function(a, b):
  if b == 0:
    return 0
  return a + my_recursive_function(a, b-1)

Function Stacks – recursion

Repetition through recursion – function calls itself repeatedly until a specific end condition (base case) is met.

What will my_recursive_function(10, 5) return?

def my_recursive_function(a, b):
  if b == 0:
    return 0
  return a + my_recursive_function(a, b-1)

The my_recursive_function method above calculates \(a × b\) (multiplication through repeated addition)

Function Stacks – recursion

What will my_recursive_function(10, 5) return?

def my_recursive_function(a, b):
  if b == 0:
    return 0
  return a + my_recursive_function(a, b-1)

my_recursive_function(10, 5) → my_recursive_function(10, 4)

my_recursive_function(10, 4) → my_recursive_function(10, 3)

my_recursive_function(10, 3) → my_recursive_function(10, 2)

my_recursive_function(10, 2) → my_recursive_function(10, 1)

my_recursive_function(10, 1) → my_recursive_function(10, 0)

my_recursive_function(10, 0) returns 0

my_recursive_function(10, 1) returns 10 + 0 (10)

my_recursive_function(10, 2) returns 10 + 10 (20)

my_recursive_function(10, 3) returns 10 + 20 (30)

my_recursive_function(10, 4) returns 10 + 30 (40)

my_recursive_function(10, 5) returns 10 + 40 (50)

Recursive Algorithm Basics

• Reduction formula – making the problem size smaller – this needs to get closer to the base case
• Base Case – the trivial (easiest) case, check for it first! It the solution you already know.

Key: find what is the smaller/simpler version of a problem

RecursionError: maximum recursion depth exceeded

RecursionError Recursion

RecursionError: maximum recursion depth exceeded

• Happens when your recursive steps never reach a base case
• Each time you call a function, memory is allocated to remember things like what it was called with, what line it’s currently at, variables, etc. (Function stack in Cpython)
• Python limits how deep the stack can get to avoid memory problems (maximum recursion depth)

Recursion

• a recursive function calls itself
• a recursive function must have at least one base case
• recursive case(s) must always move towards a base case

Remember factorial?

What’s the runtime?

def factorial(n):
  product = 1
  while n > 0:
    product *= n
    n -= 1
  return product

if __name__ == "__main__":
  assert factorial(1) == 1
  assert factorial(5) == 120
  assert factorial(7) == 5040

Remember factorial?

What’s the runtime? \(O(n)\)

def factorial(n):
  product = 1
  while n > 0:
    product *= n
    n -= 1
  return product

if __name__ == "__main__":
  assert factorial(1) == 1
  assert factorial(5) == 120
  assert factorial(7) == 5040

Recursive factorial

“a recursive function calls itself”

def factorial(n):
  factorial(n)

You must have at least one Base Case

The First Law of Recursion

Recursive factorial

“a recursive function must have at least one base case”

What’s our base case for our recursive factorial function?

Recursive factorial

“a recursive function needs a base case”

def factorial(n):
  if n == 1:
    return 1
  factorial(n)

Base Case

• Directly return the solution for that specific case
• Do not have a recursive call
• Are generally a “trivial solution”

Your Recursive Case(s) must always move towards a Base Case

The Second Law of Recursion

Recursive factorial

“recursive case(s) must always move towards a base case”

What’s our recursive case for our factorial function?

Recursive factorial

“recursive case(s) must always mode towards a base case”

We need to get to 1

def factorial(n):
  if n == 1:
    return 1
  factorial(n-1)

Recursive factorial

1. a recursive function needs at lease one base case
2. recursive case(s) must always mode towards a base case
3. it needs to provide the correct answer

\(5! = 5 * 4 * 3 * 2 * 1\)

For various math reasons, \(0!\) is defined to be \(1\)

Recursive factorial

def factorial(n):
  if n == 1 or n == 0:
    return 1
  return n*factorial(n-1)

if __name__ == "__main__":
  assert factorial(1) == 1
  assert factorial(5) == 120

Recurrence relations

The runtime complexity T(N) of a recursive function will have function T on both sides of the equation

1. single recursive call \(T(N-1)\)
2. one multiplication of the result \(O(1)\)

def factorial(n):
  if n == 1 or n == 0:
    return 1
  return n*factorial(n-1)

Using O-notation to express runtime complexity of a recursive function requires solving the recurrence relation.

Recurrence relations

1. single recursive call \(T(n-1)\)
2. one multiplication of the result \(O(1)\)

def factorial(n):
  if n == 1 or n == 0:
    return 1
  return n*factorial(n-1)

\(T(n) = T(n-1) + 1\)

Recurrence relations

\(T(n) = T(n-1) + 1\)

def factorial(n):
  if n == 1:
    return 1
  return n*factorial(n-1)

1. \(T(1) = 1\) base case
2. \(T(2) = T(1) + 1 = 1 + 1 = 2\)
3. \(T(3) = T(2) + 1 = 2 + 1 = 3\)
4. \(T(4) = T(3) + 1 = 3 + 1 = 4\)
5. \(T(N) = N\)

Recurrence relations – Recursive Factorial

• \(T(N) = T(N-1) + 1\)
• \(T(N-1) = T(N-2) + 1\)
• \(T(N) = T(N-2) + 1 + 1\)
• \(T(N) = T(N-3) + 1 + 1 + 1\)
• \(T(N) = T(N-4) + 1 + 1 + 1 + 1\) …
• \(T(N) = T(0) + N\)
• \(T(N) = 1 + N\)
• \(T(N) = N\)

How to Write a Recursive Function

• Construct at least one base case (more may be needed if there are inputs that won’t eventually reach that base case).
• Find a recurrence relation: a solution expressed in terms of the solution to a smaller instance of the same problem. (a function that gets closer to the base case)
• Assume that your function works for the smaller instance of the problem, use this to construct the recursive case

Exercise – Binary Search

Here’s our original Binary Search implementation:

def binary_search(arr, v):
  low = 0
  high = len(arr)-1
  while low <= high:
    mid = (low+high) // 2
    if v > arr[mid]: # update low
      low = mid + 1
    elif v < arr[mid]: # update high
      high = mid - 1
    else:
      return mid
  return -1

if __name__ == "__main__":
  numbers = [0, 5, 10, 23, 41, 43, 44, 60, 99, 120, 343]
  assert binary_search(numbers, 99) == 8
  assert binary_search(numbers, 9) == -1

Exercise – Binary Search

Write binary search as a recursive function.

1. What are the base cases?
2. What’s the recursive case?

Remember:

1. a recursive function needs at lease one base case
2. recursive case(s) must always mode towards a base case
3. it needs to provide the correct answer

Exercise – Binary Search

Write binary search as a recursive function.

1. What are the base case? low > high
2. What’s the recursive case? n/2 – divide list by two or change either low or high

Call your function binary_search and your file binary_search.py to submit to gradescope

Exercise – Binary Search

def recursive_binary_search(lst, target, low, high):
  mid = (low+high)//2
  
  if lst[mid] == target:
    return mid
  
  if low > high:
    return -1
  
  if target < lst[mid]:
    return recursive_binary_search(lst, target, low, mid-1)
  
  if target > lst[mid]:
    return recursive_binary_search(lst, target, mid+1, high)
  
def binary_search(arr, target):
  return recursive_binary_search(arr, target, 0, len(arr)-1)
  
if __name__ == "__main__":
  numbers = [0, 5, 10, 23, 41, 43, 44, 60, 99, 120, 343]
  assert binary_search(numbers, 99) == 8
  assert binary_search(numbers, 9) == -1
  print("Passed tests")

Passed tests

Recurrence relation – Binary Search

What’s the runtime for recursive binary search?

def recursive_binary_search(lst, target, low, high):
  mid = (low+high)//2
  
  if lst[mid] == target:
    return mid
  
  if low > high:
    return -1
  
  if target < lst[mid]:
    return recursive_binary_search(lst, target, low, mid-1)
  
  if target > lst[mid]:
    return recursive_binary_search(lst, target, mid+1, high)

Recurrence relation – Binary Search

What’s the runtime for recursive binary search?

The runtime complexity T(N) of a recursive function will have function T on both sides of the equation

Using O-notation to express runtime complexity of a recursive function requires solving the recurrence relation.

Recurrence relation – Binary Search

What’s the runtime for recursive binary search?

The runtime complexity T(N) of a recursive function will have function T on both sides of the equation

1. one recursive call \(T(N/2)\)
2. one return of the result \(O(1)\)

\(T(N) = T(N/2) + 1\)

Using O-notation to express runtime complexity of a recursive function requires solving the recurrence relation.

Recurrence relation – Binary Search

What’s the runtime for recursive binary search?

\(T(N) = T(N/2) + 1\) N is size of list

Using O-notation to express runtime complexity of a recursive function requires solving the recurrence relation.

• \(T(1) = 1\) – two options either the list value matches target (return index) or it doesn’t (return -1)
• \(T(2) = T(2/2) + 1 = T(1) + 1 = 1 + 1\)
• \(T(4) = T(4/2) + 1 = T(2) + 1 = 2 + 1\)
• \(T(8) = T(8/2) + 1 = T(4) + 1 = 3 + 1\)
• \(T(16) = T(16/2) + 1 = T(8) + 1 = 4 + 1\)

Recurrence relation – Binary Search

What’s the runtime for recursive binary search?

• \(T(2) = T(2/2) + 1 = T(1) + 1 = 1 + 1\)
• \(T(4) = T(4/2) + 1 = T(2) + 1 = 2 + 1\)
• \(T(8) = T(8/2) + 1 = T(4) + 1 = 3 + 1\)
• \(T(16) = T(16/2) + 1 = T(8) + 1 = 4 + 1\)
• \(T(N) = T(N/2) + 1 = 2^k + 1\)

What’s the value of \(k\)?

\(k = \log_2 N\)

\(O(\log N)\)

Recurrence relation – Binary Search

What’s the runtime for recursive binary search?

\(T(N) = T(N/2) + 1\) N is size of list

Using O-notation to express runtime complexity of a recursive function requires solving the recurrence relation.

• \(T(N) = T(N/2) + 1\)
• \(T(N/2) = T(N/4) + 1\)
• \(T(N) = [T(N/4) + 1] + 1 = T(N/4) + 2\)
• \(T(N) = [[T(N/8) + 1] + 1] + 1 = T(N/8) + 3\)
• \(T(N) = [[[T(N/16) + 1] + 1] + 1] + 1 = T(N/16) + 4\)

…

• \(T(N) = T(1) + 2^k\)

\(O(\log N)\)

Exercise – Power function

Power Function: Computing \(x^n\) can be defined as \(x^n = x × x^{(n-1)}\), with base case \(x^0 = 1\). You can assume the exponent is zero or greater.

if __name__ == "__main__":
  assert power(2, 2) == 4
  assert power(3, 2) == 9
  assert power(2, 4) == 16

Submit your power.py file to Gradescope

Solution – Power function

def power(base, exp):
  if exp == 0:
    return 1
  
  return base * power(base, exp-1)

if __name__ == "__main__":
  assert power(2, 2) == 4
  assert power(3, 2) == 9
  assert power(2, 4) == 16

Runtime – Power function

What’s the runtime?

def power(base, exp):
  if exp == 0:
    return 1
  
  return base * power(base, exp-1)

Runtime – Power function

What’s the runtime? \(T(N) = T(N-1) + 1\)

def power(base, exp):
  if exp == 0:
    return 1
  
  return base * power(base, exp-1)

\(O(N)\)

Exercise – Two to X

Two to X: Computing \(2^x\) can be defined as \(2^x = 2^1 * 2^{x-1}\) or \(2^x = 2^{x-1} + 2^{x-1}\), with base case \(2^0 = 1\). You can assume the exponent x is zero or greater.

if __name__ == "__main__":
    assert two_to(0) == 1
    assert two_to(1) == 2
    assert two_to(2) == 4
    assert two_to(3) == 8
    assert two_to(4) == 16
    print("Passed")

Two to X – solution 1

def two_to(exp):
    if exp == 0:
        return 1
    
    return 2 * two_to(exp-1)

Two to X – solution 2

def two_to(exp):
    if exp == 0:
        return 1
    
    return two_to(exp-1) + two_to(exp-1)

Runtime – Two to X

What’s the difference in runtime between the two solutions?

For solution 1, the call stack is linear: two_to(4) → two_to(3) → two_to(2) → two_to(1) → two_to(0)

For solution 2, each call branches out twice:

                     two_to(4)
                   /          \
              two_to(3)      two_to(3)
              /      \        /      \
         two_to(2) two_to(2) two_to(2) two_to(2)
         /    \      /  \      /  \      /  \
  two_to(1)  ...  ...  ...  ...  ...  ...  ...
    /   \    / \  / \  / \  / \  / \  / \  / \
two_to(0)  ...

Runtime – Two to X

While the two solutions produce the same result, Solution 1 is far more efficient. Solution 2’s repeated computation of the same subproblems makes it extremely wasteful

Recurrence Relation of Solution 1

• \(T(0) = O(1)\) — base case
• \(T(N) = T(N-1) + O(1)\) — one recursive call plus constant work (multiplication)

Solving the recurrence:

• \(T(N) = T(N-1) + 1\)
• \(T(N) = T(N-2) + 1 + 1 = T(N-2) + 2\)
• \(T(N) = T(N-3) + 3\) …
• \(T(N) = T(0) + N = O(N)\)

Big-O: O(N) where N = exp

Recurrence Relation of Solution 2

• \(T(0) = O(1)\) — base case

• \(T(N) = 2 · T(N-1) + O(1)\) — two recursive calls plus constant work (addition)

• \(T(N) = 2 · T(N-1) + 1\)

• \(T(N) = 2 · (2 · T(N-2) + 1) + 1 = 4 · T(N-2) + 2 + 1\)

• \(T(N) = 4 · (2 · T(N-3) + 1) + 3 = 8 · T(N-3) + 7\)

• \(T(N) = 2^k · T(N-k) + (2^k - 1)\)

When \(k = N\): \(T(N) = 2^N · T(0) + (2^N - 1) = O(2^N)\)

Exercise – Fibonacci Sequence

A fibonacci sequence is such that each number is the sum of the two preceding ones: \(F(n) = F(n-1) + F(n-2)\), with base cases \(F(0) = 0\) and \(F(1) = 1\).

This generates the sequence 0, 1, 1, 2, 3, 5, 8, 13, 21, 32, …

Test cases:

assert fib(0) == 0
assert fib(1) == 1
assert fib(2) == 1
assert fib(3) == 2
assert fib(11) == 89

Submit your fibonacci.py file with your fib(n) function to gradescope

Exercise – Fibonacci Sequence

def fib(n):
  if n == 0:
    return 0
  
  if n == 1:
    return 1
  
  return fib(n-1) + fib(n-2)

Runtime – Fibonacci Sequence

Recurrence Relation:

• T(0) = O(1) — base case
• T(1) = O(1) — base case
• T(n) = T(n-1) + T(n-2) + O(1) — two recursive calls plus constant work

\(O(2^n)\)

Exercise – Fibonacci Sequence

What’s the runtime of this solution?

def fib(n):
    sequence = [0, 1]
    for i in range(2, n + 1):
        sequence.append(sequence[i-1] + sequence[i-2])
    return sequence[n]

Exercise – Fibonacci Sequence

What’s the runtime of this solution?

def fib(n):
    a, b = 0, 1
    for _ in range(n):
        a, b = b, a + b
    return a

Tail Recursion (Iterative Recursion)

Return statement needs to calculate something, but it has to wait for recursive call to return a value first

def sum_recur(a, b):
  if a > b:
    return 0
  return a + sum_recur(a + 1, b)

vs. The accumulator builds up the result as it recurses

def sum_recur(a, b, total):
  if a > b:
    return total
  return sum_recur(a + 1, b, total + a)

Tail Recursion sum_recur(a, b, total)

• The addition happens before the recursive call (accumulated in total)
• The recursive call is the last operation (nothing happens after it returns)
• No pending operations need to be remembered

Call stack example for sum_recur(a, b, total):

sum_recur(1, 3, 0) → sum_recur(2, 3, 1)
sum_recur(2, 3, 1) → sum_recur(3, 3, 3)
sum_recur(3, 3, 3) → sum_recur(4, 3, 6)
sum_recur(4, 3, 6) → returns 6

Tail Recursion (Iterative Recursion)

• The recursive call is in the “tail position” - it’s the very last thing the function does
• In languages with tail call optimization, the compiler can replace the recursion with a loop, making it as efficient as iteration
• It doesn’t build up the call stack since there’s nothing to remember after each call

Tail Recursion (Iterative Recursion)

Python does not support native Tail Call Optimization (TCO).

While you can write tail-recursive functions in Python, the interpreter (CPython) does not optimize the stack frames.