My first thought was using the third case of the master theorem, but I am not sure if I can use $\epsilon \rightarrow 0$, so $f(n) \in \Omega(n^{log_2^4+\epsilon})$.
Otherwise, I tried solving the equation by iteration where I got $T(n) \in \Theta(n^2log_2^3n)$ instead of $T(n) \in \Theta(n^2log_2n)$.
Which solution is right and why? Thanks.
When you applied the master theorem, f(n) is n^2 right. This is anyways asymptotically lesser than O(n) which is the n^2log(n). Hence at each node of the recursion tree, the O(n) would dominate computation hence T(n) is O(n). The reason why this is true is because when you apply the recursion tree the number of processes at each node if you calculate would be 2O(n/2)+4O(n/4)...... whose sum turns out to be lesser than O(n).
