Median of medians: bound on pivot position

Question

If I understand correctly (from reading Wikipedia), median-of-medians pivot selection makes quickselect $O(n)$ because the pivot is guaranteed to be in between the 30th and 70th percentiles and so at least 30% of elements will be removed in each selection pass.

But I have tried to prove this percentage inductively but failed. In fact, I have found what I thought is a counterexample (where $\vec{v} \times n$ repeats $\vec{v}$'s elements $n$ times and $+$ is concatenation, as in Python lists):

\begin{align} \begin{bmatrix} 1 & 2 & 3 & \infty &\infty \end{bmatrix} \times 3 + \begin{bmatrix}\infty &\infty&\infty &\infty&\infty \end{bmatrix}\times2 \\+ \begin{bmatrix} 4 & 5 & 6 & \infty &\infty \end{bmatrix} \times 3 + \begin{bmatrix}\infty &\infty&\infty &\infty&\infty \end{bmatrix}\times2\\ + \begin{bmatrix} 7 & 8 & 9 & \infty &\infty \end{bmatrix} \times 3 + \begin{bmatrix}\infty &\infty&\infty &\infty&\infty \end{bmatrix}\times2\\ + \begin{bmatrix} 10 & 11 & 12 & \infty &\infty \end{bmatrix} \times 3 + \begin{bmatrix}\infty &\infty&\infty &\infty&\infty \end{bmatrix}\times2\\ + \begin{bmatrix} 13 & 14 & 15 & \infty &\infty \end{bmatrix} \times 3 + \begin{bmatrix}\infty &\infty&\infty &\infty&\infty \end{bmatrix}\times2 \end{align}

In place of $\infty$ we can just put any large number.

After the first step, we have: \begin{bmatrix}3&3&3&\infty&\infty&6&\cdots&9&\cdots&12&\cdots &15&15&15&\infty&\infty\end{bmatrix} Second step: \begin{bmatrix}3&6&9&12&15\end{bmatrix} Finally we get the median-of-medians $9$. However in the initial array only $3\cdot 3\times 3 = 27$ of $5 \cdot (2+3) \cdot 5 = 125$ elements ($21.6\%$) are $\le 9$. Which seems to violate the 30% bound.

I doubt the algorithm is incorrect so what am I getting wrong? A proof would be much appreciated.

Answer 1

You missed the recursive step of this recursive algorithm.

For the sake of discussion, assume you want to find the 99th smallest element of the initial 125 elements. Here 99 is just an arbitrary number between 1 and 125.

After you have got the first batch of medians below, $$[3\,\,3\,\,3\,\,\infty\,\,\infty,\,\,6\,\,6\,\,6\,\,\infty\,\,\infty,\,\,9\,\,9\,\,9\,\,\infty\,\,\infty,\,\,12\,\,12\,\,12\,\,\infty\,\,\infty,\,\,15\,\,15\,\,15\,\,\infty\,\,\infty]$$ you are supposed to apply the same algorithm to find its median, $m_1$, its actual real median instead of an approximate median.

When you get the second batch of medians, $$\begin{bmatrix}3&6&9&12&15\end{bmatrix}$$ you are supposed to apply the same algorithm to find its median, $m_2$. In fact, we have reached the base case of the recursion. So we just use some simple routine to get its median, $m_2=9$.

Using 9 as the pivot, we will find in the second batch of medians, two elements that are smaller than 9, 3 and 6. Going back to the first batch of medians, we will then skip three 3's and three 6's and two 9's, 3+3+2=8 elements in total. That is where the 30% comes from: $8 \ge 25 * 30\%$. We also find $12\gt9$ and $15\gt9$. Going back to the first batch of medians again, we will skip $[\infty,\infty]$, $[12,\infty,\infty]$ and $[15,\infty,\infty]$, another 8 element in total. That is where the 70% comes from. Now it is time to run the same algorithm again to find the median $m_1$ as the median of the remaining $25-8-8=9$ elements, $$[\infty\,\,\infty\,\,\infty\,\,\infty\,\,9\,\,12\,\,12\,\,15\,\,15]$$

$\cdots, \cdots, \cdots$ (rounds of recursions omitted)

Finally, we find that $m_1=15$ (the first 15). Using 15 as the pivot, we will find in the first batch of medians, $3<15$, $6<15$, $9<15$ and $12 < 15$. Going back to the initial 125 elements, we will skip three [1,2,3], three [4,5,6], three [7,8,9], three [10,11,12] and one [15, 15], 3*3*4+2=38 elements in total. That is again where the 30% comes from: $38 \ge 125 * 30\%$. We will apply the same algorithm to find the ultimately wanted element as the $99-38=61$-th smallest element of the remaining $125-38=87$ elements. Why can we skip those 38 elements? Because each of one of them is not greater than $m_1$, which is no greater than the 38 elements on the big side. Together with the fact $38 + 99 \ge 125$, we know the 99-th smallest element must be one of the remaining elements.

Hopefully, I have given enough explanation to show where you have gone wrong. Hopefully, you will appreciate the ingenuity and subtlety of this algorithm of median of medians more.

For more analysis, you may want to check the beautiful visualization page of the algorithm. You may want to read wikipedia. You can also read CLRS, edition 4, section 9.3, "Selection in worst-case linear time" and its accompanying exercises, whose analysis on the pivot and the recurrence relation is more rigorous and comprehensive.