Find all functions $f:\mathbb{R}\to\mathbb{R}$ such that for all $x,y\in\mathbb{R}, f(f(x))+f(f(y))+2f(xy)=f(x^2+y^2)$.

Question

Find all functions $f:\mathbb{R}\to\mathbb{R}$ such that for all $x,y\in\mathbb{R}$,$$f(f(x))+f(f(y))+2f(xy)=f(x^2+y^2).$$

Current progress: We want to prove that the only solutions are $f\equiv0$ and $f(x)=x^2$. Let $P(x,y)$ denote the proposition above.

From $P(0,0)$: $$2f(f(0))+f(0)=0\tag{1}.$$

From $P(x,0)$: $$f(f(x))+f(f(0))+2f(0)=f(f(x))+\frac{3f(0)}{2}=f(x^2)\\\implies f(f(x))=f(x^2)-\frac{3f(0)}{2}\tag{2}$$ by substituting $(1)$.

Then, $P(x,y)$ now implies: $$f(x^2)+f(y^2)+2f(xy)-3f(0)=f(x^2+y^2)\\\implies \left[f(x^2)-f(0)\right]+\left[f(y^2)-f(0)\right]+2\left[f(xy)-f(0)\right]=\left[f(x^2+y^2)-f(0)\right]\tag{3}$$ by substituting $(2)$. Let $g(x)=f(x)-f(0)$. Then, $(3)$ becomes: $$g(x^2)+g(y^2)+2g(xy)=g(x^2+y^2).$$ Let this new proposition be $Q(x,y)$.

From $Q(-x,y)$: $$g(x^2)+g(y^2)+2g(-xy)=g(x^2+y^2).$$ Comparing this to $Q(x,y)$, we conclude that $g$ is even.

Then, for all $x,y,z\geq0$,

$$\begin{align} &g(x)+g(y)+g(z)+2g(\sqrt{xy+xz})+2g(\sqrt{yz}) \\ =&g(x)+g(y+z)+2g(\sqrt{xy+xz}) \\ =&g(x+y+z) \\ =&g(y)+g(x+z)+2g(\sqrt{xy+yz}) \\ =&g(x)+g(y)+g(z)+2g(\sqrt{xz})+2g(\sqrt{xy+yz}) \end{align}$$ $$ \implies \forall x,y,z\geq0, g(\sqrt{x+y})+g(\sqrt{z})=g(\sqrt{x})+g(\sqrt{y+z}) \\ \implies \forall x,y\geq0, g(\sqrt{x})+g(\sqrt{y}) = g(\sqrt{x+y}) $$

For $x\geq0$, let $h(x)=g(\sqrt{x})$. Then, $h$ is additive. Substituting back, we find that $\forall x, f(x)=h(x^2)+c$. $c$ can be easily determined to be $0$, so $f(x)=h(x^2)$. Substituting back, our only equation is that $h(h(x^2)^2)=h(x^4)$. I’m curious to know that if it is possible to uniquely determine $h$ based only on this condition (and the fact that $h$ is additive).

Answer 1

Here is a self-contained, full solution.

Step 1. Let $\mathsf{FE}_1(x, y)$ denote the original functional equation

$$ \color{blue}{\mathsf{FE}_1(x,y)} : \quad f(f(x)) + f(f(y)) + 2f(xy) = f(x^2 + y^2). \tag{1} $$

Now we make a series of observations:

• $\mathsf{FE}_1(0, 0)$ implies that $f(\alpha) = -\frac{1}{2}\alpha$, where $\alpha = f(0)$.

• $\mathsf{FE}_1(x, 0)$ reduces to $$ f(f(x)) = f(x^2) - \tfrac{3}{2}\alpha. \tag{2} $$

• Let $g(x) = f(x) - \alpha$. Plugging $\text{(2)}$ to $\mathsf{FE}_1(x, y)$, we obtain $$ \color{blue}{\mathsf{FE}_2(x, y)} : \quad g(x^2) + g(y^2) + 2g(xy) = g(x^2 + y^2) \tag{3} $$ for all $x, y \in \mathbb{R}$.

• $\mathsf{FE}_2(x, 1)$ reduces to $g(x) = \frac{1}{2}[g(x^2 + 1) - g(x^2) - g(1)]$. In particular, $g(x)$ is an even function.

---

Step 2. Now, for $x, y, z \in \mathbb{R} $ we define

\begin{align*} R(x, y, z) = g(x^2 + y^2 + z^2) - \sum_{\text{cyc}} g(x^2) - 2\sum_{\text{cyc}} g(xy). \end{align*}

Note that $R(x, y, z)$ is a symmetric function of $x, y, z$. Also, expanding $R(x, y, z)$ by using $\mathsf{FE}_2$ twice, we get

\begin{align*} R(x, y, z) &= g(x^2+y^2) + 2g(\sqrt{x^2z^2+y^2z^2}) - g(x^2) - g(y^2) - 2\sum_{\text{cyc}} g(xy) \\ &= 2[g(\sqrt{x^2z^2+y^2z^2}) - g(xz) - g(yz)]. \end{align*}

Hence, simplifying $R(x, y, z) = R(x, z, y)$ yields

$$ g(\sqrt{y^2z^2 + x^2z^2}) - g(xz) = g(\sqrt{y^2z^2 + x^2y^2}) - g(xy) $$

Now let $a, b, c \in \mathbb{R}_{>0}$. Then plugging $(x, y, z) = (\sqrt[4]{bc/a}, \sqrt[4]{ca/b}, \sqrt[4]{ab/c})$ to the above equality yields the following functional identity

$$ \color{blue}{\mathsf{FE}_3(a, b, c)} : \quad g(\sqrt{a+b}) - g(\sqrt{b}) = g(\sqrt{a+c}) - g(\sqrt{c}) \tag{4} $$

for all $a, b, c > 0$. In fact, $\mathsf{FE}_3(a, b, c)$ continues to hold even when $a = 0$, since both sides of $\mathsf{FE}_3(0, b, c)$ are identically zero.

---

Step 3. We employ @Sil's argument as described in the comment. Indeed, let's define

$$ h_0(x) = g(\sqrt{x+a})-g(\sqrt{a}),\qquad x\geq 0$$

where $a > 0$ is an arbitrary constant. Although the definition of $h_0$ involves the unspecified constant $a$, $\mathsf{FE}_3$ tells that $h_0$ is well-defined and is independent of the choice of $a$. Moreover, for any $x,y \geq 0$ and for any $a > 0$,

\begin{align*} h_0(x+y) &= g(\sqrt{x+y+a}) - g(\sqrt{a}) \\ &= g(\sqrt{x+y+a}) - g(\sqrt{y+a}) + g(\sqrt{y+a}) - g(\sqrt{a}) \\ &= h_0(x) + h_0(y). \end{align*}

Now, define $h(x)$ by

$$ h(x) = \begin{cases} h_0(x), & x \geq 0, \\ -h_0(-x), & x \leq 0. \end{cases} $$

Since $h_0(0) = 0$, the function $h$ is well-defined and satisfies $h(-x) = -h(x)$. Moreover, an easy but tedious computation shows that $h$ is also additive. (You may check the following four cases separately: (1) $x, y \geq 0$, (2) $x, y \leq 0$, (3) $0 \leq -x \leq y$, (4) $0 \leq y \leq -x$.)

Then for any $y \geq x > 0$,

\begin{align*} g(y) - g(x) &= g(\sqrt{y^2 - x^2 + x^2}) - g(\sqrt{x^2}) \\ &= h(y^2 - x^2) \\ &= h(y^2) - h(x^2). \end{align*}

By using the fact that $h$ is an odd function, it is easy to check that this relation extends to any $x, y > 0$. In particular,

$$ g(x) = h(x^2) + \beta $$

with $\beta = g(1) - h(1)$. Plugging this into $\mathsf{FE}_2$, we can determine that $\beta = 0$. From this and noting that $g(0) = 0 = h(0)$, we conclude:

$$ g(x) = h(x^2), \quad \forall x \in \mathbb{R}. \tag{5} $$

---

Step 4. Plugging $\text{(4)}$ into the definition of $g$, we know that

$$ f(x) = h(x^2) + \alpha \tag{6} $$

for an additive function $h$. We claim:

Claim. $h(xy) = h(h(x)h(y))$ for all $x, y \in \mathbb{R}$.

Proof. Let $x \geq 0$. Then by $\text{(2)}$, we have

$$ f(x) - f(f(\sqrt{x})) = \tfrac{3}{2}\alpha = f(0) - f(f(0)). $$

Plugging $\text{(6)}$ into the above identity and simplifying a bit, we obtain $h(\alpha^2) = -\frac{3}{2}\alpha$ and

$$ \color{blue}{\mathsf{FE}_4(x)}: \quad h(h(x)^2 - x^2) + 2h(h(x)\alpha) = 0. \tag{7} $$

To make use of this identity, we invoke the polarization trick. Indeed, let $x, y \geq 0$ and consider $\mathsf{FE}_4(x+y)$, $\mathsf{FE}_4(x)$, and $\mathsf{FE}_4(y)$:

\begin{align*} \mathsf{FE}_4(x+y) : \quad 0 &= h( (h(x) + h(y))^2 - (x + y)^2) + 2h((h(x) + h(y))\alpha) \\ \mathsf{FE}_4(x) : \quad 0 &= h(h(x)^2 - x^2) + 2h(h(x)\alpha) \\ \mathsf{FE}_4(y) : \quad 0 &= h(h(y)^2 - y^2) + 2h(h(y)\alpha) \end{align*}

Subtracting the below two identities from the first one, it follows that

$$ 2h(h(x)h(y) - xy) = 0. $$

Using the additivity of $h$, it is easy to check that this identity continues to hold for any $x, y \in \mathbb{R}$. $\square$

---

Step 5. Returning to the original problem, we consider two possibilities:

Case 1. Assume first that $h(x) \neq 0$ for all $x \neq 0$. Equivalently, let's assume that $h(x) = 0$ implies $x = 0$. Then the above claim implies that

$$ h(x)h(y) = xy $$

for all $x, y \in \mathbb{R}$. Plugging $x = y = 1$ into this identity, we get $h(1) = \pm 1$. This then implies that both

$$ f(x) = x^2 \qquad \text{and} \qquad f(x) = -x^2 $$

are solutions of the original functional equation.

Case 2. If $h(a) = 0$ for some $a \neq 0$, then for any $x \in \mathbb{R}$,

$$ h(x) = h((x/a)a) = h(h(x/a)h(a)) = 0. $$

Therefore, $f$ is identically zero in this case.

---

Conclusion. The only solutions of the functional equation $\mathsf{FE}_1$ are

$$ f(x) = x^2, \qquad f(x) = -x^2, \qquad f(x) = 0. $$