The proof appears convincing at first glance, but it is bogus because of a flaw in the reasoning.
The correct proof of AM-GM should start with the fact that $ (a - b)^2 \geq 0$ and then deduce $ \frac{a+b}{2} \geq \sqrt{ab} $.
Instead, this proof assumes the inequality to be true (by attempting to "prove" it via algebraic manipulations) and then derives a true statement.
This is circular reasoning—it does not actually prove the original inequality but rather confirms that if the inequality holds, then a true statement follows.
A valid proof of AM-GM follows from starting with $ (a - b)^2 \geq 0 $:
$$
a^2 - 2ab + b^2 \geq 0
$$
Rearrange:
$$
a^2 + b^2 \geq 2ab
$$
Adding $2ab $ to both sides:
$$
a^2 + 2ab + b^2 \geq 4ab
$$
Taking the square root:
$$
a + b \geq 2\sqrt{ab}
$$
Dividing by 2:
$$
\frac{a+b}{2} \geq \sqrt{ab}
$$
This approach proves the inequality without assuming it.
Second proof
To prove the AM-GM inequality using contradiction, assume that the inequality is false. That is, suppose there exist nonnegative real numbers $a$ and $b$ such that:
$$
\frac{a+b}{2} < \sqrt{ab}.
$$
Multiplying both sides by 2 gives:
$$
a + b < 2\sqrt{ab}.
$$
Squaring both sides (which preserves the inequality because both sides are nonnegative) results in:
$$
(a + b)^2 < 4ab.
$$
Expanding the left-hand side:
$$
a^2 + 2ab + b^2 < 4ab.
$$
Subtracting $2ab$ from both sides:
$$
a^2 - 2ab + b^2 < 0.
$$
But the left-hand side is $ (a - b)^2$, so we get:
$$
(a - b)^2 < 0.
$$
This is a contradiction because the square of any real number is always nonnegative, i.e., $(a - b)^2 \geq 0$.
Thus, our assumption that $ \frac{a+b}{2} < \sqrt{ab}$ must be false.
Therefore, we conclude that:
$$
\frac{a+b}{2} \geq \sqrt{ab}.
$$
and we are done. This completes the proof by contradiction.
Below I provide a relevant example from Liebeck's A Concise Introduction to Pure Mathematics which make help you understand better.
