On finding polynomials that approximate a function and its derivative (extensions of Stone-Weierstrass?)

Question

The Stone-Weierstrass Theorem tells us that we can approximate any continuous $f:\mathbb{R}^n\to\mathbb{R}$ arbitrary well on a compact subset of $\mathbb{R}^n$ by some polynomial. Suppose that $f$ is continuously differentiable, under what conditions can we guarantee we can find a polynomial $p$ such that $p$ approximates arbitrarily well $f$ and the partial derivatives approximate of $p$ arbitrarily well those of $f$ (on a compact subset of $\mathbb{R}^n$)? References are welcome. See below for my (probably naive) approach to the question.

---

It is straightforward to find conditions if $n=1$: Let $K:=[a,b]\subseteq\mathbb{R}$ and let $K_+$ denote some open cover of $K$. Suppose that $f:K_+\to\mathbb{R}$ is continuously differentiable and let $f'$ denote its derivative. Then, by Weierstrass's approximation Theorem we have that there exists a polynomial $q$ such that

$$||f'(x)-q(x)||\leq \varepsilon$$

for all $x\in K$. Choose some $y\in K$. For any $x\in K$ define

$$p(x):=f(y)+\int_y^xq(t)dt.\quad\quad(*)$$

By the Fundamental Theorem of calculus we have that $p'=q$. So for any $x\in K$

$$||f(x)-p(x)||=\left|\left|\int_y^x (f'(t)-q(t))dt\right|\right|\leq \int_y^x ||f'(t)-q(t)||dt\leq \varepsilon||x-y||\leq \varepsilon (a-b).$$

However, the above argument doesn't work in higher dimensions because the analogue of $(*)$ does not imply that $\nabla p = q$ (since not every vector of polynomials spans a gradient field). Can anyone suggest a way around this? That is, a way to to show that for continuously differentiable $f:\mathbb{R}^n\to\mathbb{R}$ and for any $\varepsilon>0$, we can always find a polynomial $p$ such that

$$||f(x)-p(x)||+||\nabla f(x)-\nabla p(x)||\leq \varepsilon$$

for all $x$ in some compact set $K$? Feel free to add extra conditions on $K$, if it makes life easier.

Answer 1

If we assume a little more smoothness, we can use a trick similar to the one in this question.

Let me take $n=2$ for concreteness. Suppose $f$ is $C^2$ on a neighborhood $U$ of $K$. With an appropriate cutoff function we can find a $C^2$, compactly supported $g$ which agrees with $f$ on a (possibly smaller) neighborhood of $K$. We can also find a large enough square $Q = (-M,M)$ which contains the support of $g$, so that $g$ and all its derivatives vanish on the boundary $\partial Q$.

Fix $\epsilon > 0$. The second partial $\partial_x \partial_y g$ is continuous, so we can find a polynomial $p_0$ with $|p_0 - \partial_x \partial_y g| < \epsilon$ on $\bar{Q}$. Set $$p(x,y) = \int_{-M}^x \int_{-M}^y p_0(s,t)\,dt\,ds.$$ Then by the fundamental theorem of calculus and Fubini's theorem we have, for any $(x,y) \in Q$, $$\begin{align*}|p(x,y) - g(x,y)| &= \left|\int_{-M}^x \int_{-M}^y p_0(s,t) - \partial_x \partial_y g(s,t)\,ds\,dt\right|\\ & \le \int_{-M}^x \int_{-M}^y |p_0(s,t) - \partial_x \partial_y g(s,t)|\,ds\,dt \\&\le (2M)^2 \epsilon \end{align*}$$ and $$\begin{align*}|\partial_x p(x,y) - \partial_x g(x,y)| &= \left| \int_{-M}^y p_0(s,t) - \partial_x \partial_y g(s,t)\,dt\right|\\ & \le \int_{-M}^y |p_0(s,t) - \partial_x \partial_y g(s,t)|\,dt \\&\le 2M \epsilon. \end{align*}$$ A similar argument works for $\partial_y$. Since $f=g$ on a neighborhood of $K$, we have that $p$ and $f$, as well as their derivatives, are uniformly close on a neighborhood of $K$.

In $\mathbb{R}^n$, this approach requires us to assume that $f$ is $C^n$.

Answer 2

By using a partition of unity, we may assume that $f$ has compact support. Let $\phi_m$ be the density of the normal vector with mean $\bf 0$ and covariance matrix $I/m$. Let $q_m$ be a Maclaurin polynomial approximating $\phi_m$ of sufficiently high degree that $\lim_{m \to \infty} \|\phi_m - q_m\|_{C(K)} = 0$, where $K$ is the Minkowski difference of where you want the approximation to hold and the support of $f$. Now let $p_m := q_m * f$. These $p_m$ do the job.

Remark: Nachbin has a generalization of the Stone-Weierstrass theorem to include approximating derivatives, even on manifolds. See MR0030590: Nachbin, Leopoldo, Sur les algèbres denses de fonctions différentiables sur une variété. C. R. Acad. Sci. Paris 228, (1949), 1549–1551.

Answer 3

I published a constructive proof of this back in 2008 for the $n$-variate case. Continuously differentiable is all that is required. See Theorem 5 in

Peet, Matthew M., Exponentially stable nonlinear systems have polynomial Lyapunov functions on bounded regions, IEEE Trans. Autom. Control 54, No. 5, 979-987 (2009). ZBL1367.93442.

Arxiv link here.

The following theorem combines Lemmas 3 and 4 with the Weierstrass approximation theorem. It says that the polynomials are dense in $\mathcal{C}_{\infty}^{1}$ with respect to the Sobolev norm for $W^{1, \infty},$ among others.

Theorem 5. Suppose $v \in \mathcal{C}_{\infty}^{1}(B) .$ Then for any $\epsilon>0,$ there exists a polynomial $p,$ such that $$ \max _{\alpha \in Z^{n}}\left\|D^{\alpha} p-D^{\alpha} v\right\|_{\infty} \leq \epsilon. $$

Here, $\mathcal{C}_{\infty}^{i}(\Omega):=\left\{f: D^{\alpha} f \in \mathcal{C}(\Omega) \text{ for any } \alpha \in \mathbb{N}^{n} \text{ such that } \|\alpha\|_{\infty}=\max _{j} \alpha_{j} \leq i \right\}$.

Summary of the construction from the paper using the multinomial notation:

The difficulty of this problem is a consequence of the fact that, for a given function, not all the partial derivatives of that function are independent. For example, we have the identities \begin{align*} &D^{(1,1,0)}f(x,y,z)=\\ &D^{(1,1,0)}\left(f(x,y,0) + \int_{0}^z D^{(0,0,1)}f(x,y,s)\,ds\right)\\ &=D^{(1,0,0)}\left(D^{(0,1,0)}f(x,y,z)\right), \end{align*} among others. Therefore, given approximations to the partial derivatives of a function, these approximations will, in general, not be the partial derivatives of any function. The problem becomes, for each partial derivative approximation, how to extract the information which is unique to that partial derivative in order to form an approximation to the original function. The following construction shows how this can be done.

In the paper, there then is a rather general formula for the construction, K, which maps the essential information from all the partial derivatives to the original function. However, this can be better understood using examples.

Basically, to approximate the original function, p, approximate each partial derivative, $q_\alpha = D^\alpha p$ with a polynomial, $\tilde q_\alpha$. Then construct the polynomial as

Example: If $p=K(\{q_\alpha\}_{\alpha \in Z^2})$, then \begin{align*} p(x_1,x_2)&=\int_0^{x_1} \int_0^{x_2} q_{(1,1)}(s_1,s_2)\,ds_2\, ds_1\\ &+\int_0^{x_1} q_{(1,0)}(s_1,0)\,ds_1\\ &+\int_0^{x_2} q_{(0,1)}(0,s_2)\,ds_2\\ &+q_{(0,0)}(0,0). \end{align*}

If $n=3$, then \begin{align*} p(x_1,x_2,x_3)&=\int_0^{x_1} \int_0^{x_2} \int_0^{x_3} q_{(1,1,1)}(s_1,s_2,s_3)\,ds_3\, ds_2\, d s_1\\ &+\int_0^{x_1} \int_0^{x_2} q_{(1,1,0)}(s_1,s_2,0)\,ds_2\, ds_1\\ &+\int_0^{x_1} \int_0^{x_3} q_{(1,0,1)}(s_1,0,s_3)\,ds_3\, d s_1\\ &+ \int_0^{x_2} \int_0^{x_3} q_{(0,1,1)}(0,s_2,s_3)\, ds_3\, d s_2\\ &+\int_0^{x_1} q_{(1,0,0)}(s_1,0,0)\,ds_1\\ &+\int_0^{x_2} q_{(0,1,0)}(0,s_2,0)\,ds_2\\ &+\int_0^{x_3}q_{(0,0,1)}(0,0,s_3)\,d s_3\\ &+q_{(0,0,0)}(0,0,0). \end{align*}

Notice that this structure gives a natural way of approximating the function $p$ and all of its partial derivatives. For example, if $p$ is defined as above for $n=2$, then \begin{equation} \frac{\partial }{\partial x_1}p(x_1,x_2)=\int_0^{x_2} q_{(1,1)}(x_1,s_2)\, ds_2 + q_{(1,0)}(x_1,0). \end{equation} Now, if one approximates $q_\alpha$ by $\tilde q_\alpha$ and uses the construction to create $\tilde p$, then \begin{equation} \frac{\partial }{\partial x_1}\tilde p(x_1,x_2)=\int_0^{x_2} \tilde q_{(1,1)}(x_1,s_2)\, ds_2 + \tilde q_{(1,0)}(x_1,0). \end{equation}

Since the integral is a bounded operator, and since $q_\alpha \approx \tilde q_\alpha$, then $\frac{\partial }{\partial x_1}\tilde p \approx \frac{\partial }{\partial x_1}p$.

Answer 4

For completeness sake, let me summarize the answer of Alessandra Faggionato, namely Section 1.6.2, Page 33, "The Weierstrass Approximation Theorem" of the book by R. Narashimhan "Analysis on Real and Complex manifolds", 1985, North-Holland. (Let me add that on the Amazon web site at the time this answer was written, one could obtain a free sample of this book that included these pages.)

He starts by noting that $f$ is equal to a function $g$ on the compact set $K$, where $g \in C^k(\mathbb R^n)$, and $g$ has compact support. Then he considers the molification of $g$: $$ g_\lambda(x) = c_\lambda \int_{\mathbb R^n} g(y) \exp(-\lambda |x-y|^2) \, dy, $$ where $c_\lambda$ is such that $$ c_\lambda \int_{\mathbb R^n} \exp(-\lambda |x|^2) \, dx = 1 .$$ He shows that $g_\lambda$ converges to $g$ in $C^k(\mathbb R^n)$ as $\lambda \to \infty$. Finally he approximates $\exp(-\lambda |-x|^2)$ by its Taylor approximation.