Let $f:\mathbb R \to \mathbb R $ be a function such that $f \circ f=f'$ and $f(0)=0$ , I proved that $f$ is the null function.
Can we prove that the same result holds if we change $f \circ f=f'$ by $f \circ f \circ f \circ f=f'$?
Thank you for your help.
For $n\in\mathbb N$, let $f^{\circ n}$ denote the $n$th iterate of $f$, i.e. $f^{\circ1}=f$ and recursively $f^{\circ(n+1)}=f\circ f^{\circ n}$.
Proposition. Let $m\ge 2$ and assume $f\colon \mathbb R\to\mathbb R$ is a differentiable function with $f'=f^{\circ m}$ and $f(0)=0$. Then $f(x)=0$ for all $x\in\mathbb R$.
Proof. Let $$I=\{\,r\in(0,\infty)\mid \forall x\in[-r,r]\colon f(x)=0\,\}. $$ We want to show that $I$ is unbounded.
To start, we shall show that $I$ is nonempty. By what is given, $f$ is continuous and then from $f(0)=0$ we see that there exists $r_0$ with $0<r_0<1$ such that $|f(x)|\le 1$ for all $x$ with $|x|\le r_0$. Let $$ S=\{\,n\in\mathbb N_0\mid \forall x\in[-r_0,r_0]\colon |f(x)|\le |x|^n\,\}.$$ Then we have just seen that $S$ is non-empty because $1\in S$. Assume that $S$ is finite and let $M$ be its maximal element. Consider $x$ with $0<|x|\le r_0$. Then by the Intermediate Value Theorem $$ f(x)=xf'(\xi)$$ for some intermediate $\xi$; especially, $|\xi|\le r_0$.
By induction, we have $|f^{\circ n}(\xi)|\le |\xi|^{M^n}$ for all $n\in\mathbb N$. Indeed, the base case $|f^{\circ 1}(\xi)|=|f(\xi)|\le |\xi|^M$ follows from $M\in S$ and $\xi\in[-r_0,r_0]$. As $ |\xi|\le r_0< 1$, we also have $|\xi^M|\le r_0$ so that the induction step works: $$ |f^{\circ(n+1)}(\xi)|=|f^{\circ n}(f(\xi))|\le |f(\xi)|^{M^n}\le (|\xi|^M)^{M^n}=|\xi|^{M^{n+1}}.$$ We conclude that $$|f(x)|=|x|\cdot|f'(\xi)|\le |x|\cdot|\xi|^{M^m}\le |x|^{M^m+1}. $$ This shows $M^m+1\in S$, contradicting the maximality of $M$. We conclude that $S$ is unbounded. This implies that $f(x)=0$ for all $x$ with $|x|\le r_0$. In other words, $r_0\in I$.
Assume $I$ is bounded and let $R=\sup I\in(0,\infty)$. By continuity of $f$, there exists $s>R$ such that $|f(x)|<R$ for all $x$ with $|x|<s$. Then for $x\in(-s,s)$, we have $f(f(x))=0$ and by induction $f^{\circ n}(x)=0$ for all $n\ge 2$. We conclude that $f'(x)=f^{\circ m}(x)=0$ for all $x\in(-s,s)$, hence $f(x)=f(0)=0$ for all $x\in(-s,s)$. But then $s\in I$, which contradicts $s>\sup I$. We conclude that $I$ is unbounded. $_\square$
Remark: The proposition is also true for $m=1$ as $f'=f$ implies $f(x)=ce^x$ for some constant $c$, which must be zero here.
