I learned that recursive Fibonacci is $O(2^N)$.
However, when I implement it and print out the recursive calls that were made, I only get 15 calls for N=5. What I am missing? Should it not be 32 or near abouts? It's like less then half!
$COUNT = 0
def fibonacci( n )
$COUNT +=1
return n if ( 0..1 ).include? n # Base Case returns 0 or 1
fibonacci( n - 1 ) + fibonacci( n - 2 ) # Recursive case
end
You ask,
I have an $O(2^n)$ runtime, why do I not observe $2^n$ recursive calls for $n=15$?
There are many things wrong in the implied conclusion.
Since you use $O$ as though it was $\Omega$, note that a (lower) bound on runtime does not necessarily translate into a (lower) bound on recursive calls, each of which may take $\omega(1)$ time.
Similarly, a tight upper runtime bound (which you don't have) may not be a tight upper bound on the number of recursive calls for the same reasons.
You can analyse exactly how many recursive calls you need. Just solve the recurrence
$\qquad\displaystyle \begin{align*} f(0) &= 1,\\ f(1) &= 1,\\ f(n) &= f(n-1) + f(n-2) + 1, \quad n \geq 3, \end{align*}$
which adds up $1$ for every recursive call your program makes. See here for more detail on how you get to such recurrences.
See here how to do that, or use computer algebra. The result is
$\qquad f(n) = 2F_{n+1} - 1$;
insert $n=5$ and you will get $f(n) = 15$. It's not that surprising that the Fibonacci numbers $F_n$ should show up. Inserting their known closed form, we get
$\qquad\displaystyle f(n) = \frac{2}{\sqrt{5}} \cdot \Bigl( \varphi^{n+1} - (1-\varphi)^{n+1} \Bigr) - 1$
with $\varphi = \frac{1 + \sqrt{5}}{2} \approx 1.62$ the golden ratio. This closed form of $f(n)$ implies
$\qquad\displaystyle f(n) \sim \frac{2}{\sqrt{5}} \cdot \varphi^{n+1} \approx 1.45 \cdot 1.62^n$
since $0< 1 - \varphi < 1 < \varphi$. We see that the $O(2^n)$-bound is rather loose -- it has exponential absolute and relative error.
Double-recursion ends when n<=1, so it's more like 2^(n-1). As @pjs says, this doesn't change anything.
A function that take 0.1234 * (2^(n-3) + n^400) + 5000 cycles, or operations, or whatever, is still of the O(2^n) complexity class.
In addition to the 3 reasons given by Raphael in his answer, regarding why the aymptotic cost upperbound $O(2^n)$ does not give you $2^n$ recursive calls for $n=15$, I think you should add a fourth one:
There is a reason for that: the cost unit is undefined. This makes analysis easier and somewhat support independent. As long as all primitive operations have a cost within a bounded interval, the relative cost of the various elementary steps of the algorithm can be ignored.
The price to be paid of course, is that the cost thus evaluated has no absolute meaning, only a relative one.
If we were to assume that:
$O(2^n)$ is not an upperbound (as indicated by big $O$) but an exact cost function (point 1 of Raphael);
$O(2^n)$ is not asymptotic but exact cost for all values of $n$ (point 2 of Raphael);
each recursive call takes constant time (point 3 of Raphael);
and that furthermore there are no cost other than the constant cost of each recursive call (not including sub-calls), which seems to be your implicit assumption,
we would still be unable to assert that for $n=15$ you should have $2^{15}$ calls, from the simple knowledge of $O(2^n)$ (ignoring how the formula was obtained, of course). The reason is the arbitrary constant factor that does not show in the formula.
At best you would know that there is a factor $k$ such that for each value of $n$ you should have $k2^n$ call. Thus, if you know that for $n=15$ you had $p$ recursive calls, that would imply that $p=k2^{15}$, which would allow you to compute the value of $k$, and then predict the number of calls for other values of $n$.
But this comes only after a lot of other assumptions, none of which is really true. With apologies for awkwardly attemtping to separate issues that really have meaning only together.
