CS352 : Lecture - 3 : Analysis of Fibonacci series

CS352: Algorithms and Data Structures
Analysis of Fibonacci Series

unsigned int fib(unsigned int n)
{
(1)         if (n <= 1)
(2)             return 1
            else
(3)             return ( fib(n-1) + fib(n-2) );
}

Let T(n) be running time for fib(n)

T(0) = T(1) = 1 since line(1) takes constant time.
when n>2 work consists of line 1 + line 3 (2 units time)
function calls: T(n-1) units for first call; T(n-2) units for second call

Running time of fib(n) is T(n) = T(n-1) + T(n-2) + 2

since it can be shown that fib(n) > (3/2ⁿ), it proves that fib(n) runs in exponential time

Example 2: Analyze the code fragment

(1)     sum = 0;
(2)     for (j=1; j<=n; j++)
(3)         for (i=1; i<=j; i++)
(4)             sum++;
(5)     for (k=1; k<=n; k++)
(6)         A[k] = k-1;

(1) assignment statement takes constant time: c₁
(5) for loop like ex.1 takes c₂n = Q(n) time
line (4) takes constant time: c3
inner loop (3) executes j times --> cost c₃j
outer loop (2) executes n times, but cost of each inner loop is different with each j --> c₃ * summation of j from 1 to n cost = c₁ + c₂n + c₃ * (n(n+1) / 2)
= O(c₁+c₂n+c₃n²) = Q(n²)

Example 3. Compare asymptotic analysis for following:

(1)   sum1 = 0;
        for (i=1; i<=n; i++)
            for (j=1; j<=n; j++)                   sum1++;

(2)   sum2 = 0;
        for (i=1; i<=n; i++)
            for (j=1; j<=i; j++)
                sum2++;

In (1) the inner loop executes n times for each of n executions made by outer loop --> sum1++ executes Q(n²) times

In (2) the inner loop executes with cost (summation from 1 to n of j), which is about n²/2.

Hence, both loops cost Q(n²), but the second requires about half the time of the first

Example 4. A for-loop that does not run in Q (n²) time

sum=0;
    for (k=1; k<=n; k*=2)
        for (j=1; j<=n; j++)
            sum++

Outer loop executes log n times since k is multiplied by 2 on each iteration up to n
The inner loop executes n times

total cost --> S_i=1..log
n n = n log n

MAXIMUM SUBSEQUENCE SUM PROBLEM Given (possibly negative) integers a1,a2,...,an, find the maximum value of S a_k. Ex. for input -2, 11, -4, 13, -5, -2, answer is 20 (a₂ through a₄)

ALGORITHM : 1

    Line 1 is O(1); Loop line 2 is size n; Loop line 3 could be small or could be size n as well (worst case); Loop line 5 is size n;
    Lines 7 to 10 take O(n²)
    Total Running time is O(n³). More precisely, Q(n³)

ALGORITHM : 2

Third for-loop removed.
Running time is O(n²)

ALGORITHM : 3

Recursive n log n running time
Array is passed as VAR, not as copy; otherwise, algorithm degenerates to O(n²) since copy of array must be made with each recursive call
Analysis:

T(n) = time to solve problem of size n
if n=1, T(n) = constant time = 1
two loops run in O(n) time
assignments run in constant time
recursive calls: each solve 2 subsequence problems of size n/2 (if n is even)

T(n/2) for each call => 2T(n/2) time

ALGORITHM : 3
Total time for algorithm:
T(1) = 1
T(n) = 2T(n/2) + O(n) = 2T(n/2) + n

T(1) = 1
T(2) = 4 = 2¹ . 2
T(4) = 12 = 2² . 3
T(8) = 32 = 2³ . 4
T(16) = 80 = 2⁴ . 5
...
T(2k) = 2^k (k+1)

Now, if n=2^k, T(n) = n(k+1) (substitution)
if n = 2^k , then log n = k

=> T(n) = n(log n + 1) = n log n + n = O(n log n)

Some notes about this analysis:

n is assumed to be even such that n is a power of 2 (n=6 doesn't work)
Big O result is same if this is not the case, but analysis is more complex(have to show if true for power of 2, then also true for other powers)
analysis not valid if array is passed by value--copies of array made on each recursive call; then running time is O(n²)
Algo. 4 is the best algorithm

- O(n) running time

- makes one pass through data ----------------------------> 5

- array can be stored on secondary medium -2

- at any time, algorithm can give correct answer to 3 subseq. problem for data it has already used 6

- known as on-line algorithm: requires only constant -7

space and runs in linear time 6

Other Logarithmic Running Time Algorithms

An algorithm is O(log n) if it takes constant (O(1)) time to cut the problem size by a fraction (usually 1/2). If constant time is needed to reduce the problem by a constant amount (i.e. make smaller by 1), then algorithm is O(n).

Binary Search

Given a sorted list, at each step, 1/2 the elements are discarded (the half not containing search key)

begin looking in the middle, compare search key to mid, then BSearch left or right

int binary search(input_type a[], input_type x, unsigned int n)
{
      int low, mid, high;
1     low = 0; high = n-1;
2     while (low <= high) {
3         mid = (low + high) / 2;
4         if (a[mid] > x)
5             low = mid + 1;
           else
6         if (a[mid] < x)
7             high = mid - 1;
8         else return mid; } /* found */
9     return NOT_FOUND;
}

Each step of loop takes constant time, so time of search depends on number of steps(recursive calls), which correspond to number of times the remaining elements are halved.

This clearly takes at most log n steps, or Q(log n) search time

Euclid's Algorithm

Greatest Common Divisor of two integers is the largest integer that divides both integers.

e.g. gcd(24,36) = 12

Algorithm computes remainders recursively until 0 reached

e.g. 36 mod 24= 12

24 mod 12 = 0 => gcd = 12

Running time depends on sequence of remainders

Can be proved that after two iterations, remainder is at most half its original value(upper bound - see thm. p.32)

- This is worst-case performance; average-case requires sophisticated mathematical analysis which yields average number iterations = (12 ln 2 lnn )/p² + 1.47

Analysis vs Empirical Results

To verify analysis, code the algorithm and compare analysis to actual running times

Compute values T(n)/f(n) for a range of n (spaced by factors of 2) where T(n) is empirically observed running time
If computed values converge to positive constant, then f(n) is good answer
If values converge to 0, the f(n) is over-estimate
If values diverge, f(n) is an under-estimate
Example: algorithm to estimate prob. that two numbers are relatively prime (p.49)

-table of fig. 2.13 verifies n²log n running time

Practical Complexities

Time complexity function can be used to compare two programs P and Q that perform the same task.

e.g. assume program P is Q(n) and program Q is Q(n²)

Assertion: program P is faster than program Q for sufficiently large n

if program P actually runs in 10⁶ n milliseconds, and program Q runs in n² milliseconds, and if we always have n < 10⁶, then, other factors being equal, we should use program Q.

Function growth for various values of n…

log n	n	n log n	n²	n³	2ⁿ
0	1	0	1	1	2
1	2	2	4	8	4
2	4	8	16	64	16
3	8	24	64	512	256
4	16	64	256	4096	65,536
5	32	160	1024	32,768	4,294,967,296