I want to solve a high-dimensional PDE $F(\mathbf{x}, v ,\triangledown v , \triangledown^2 v )$
$$ -\dot{V_{t}}(\mathbf{x}) - A (\triangledown v_t (\mathbf{x})) = f(\mathbf{x}) , \quad \mathbf{x} \in \Omega \subset \mathbb{R}^{p} $$
where $\Omega$ is compact and $v : \mathbb{R}^{p} \mapsto \mathbb{R}$.
Standard Approximation Scheme (FD)
A standard approach to solving this problem is to use a finite difference scheme $ F^h(h,\mathbf{x_{k}},v(\mathbf{x_{k}}),\left [ v(.) \right ]_k^h) $ over a grid $\mathcal{G}_h = \times_{k=0}^K \left\{ x_{0k}, \ldots , x_{N k} \right\} $ with step parameter $h$, which under some conditions, converge to the correct solution.
This method uses $N^p$ points, hence you quickly face the curse of dimensionality problem, even with $p$ equal to 4 or 5.
If I use a time-marching algorithm to approach the solution (what I'm doing for now as $f$ is highly non-linear), the goal becomes finding a fixed point $v^*$ of the recursion below:
$$ \mathcal{A} \mathbf{v}_{t+\Delta t} = b \left ( \mathbf{v}_t \right ), \quad \mathbf{v} \in \mathbb{R}^{N^p} $$
i.e. the discretized algorithm effectively solves a linear system at every iteration and I have already established convergence (here)
Approximation using a Function Interpolant
In order to reduce the number of grid points, I want to solve for a function interpolant that is defined using a small set of nodes $\mathcal{S}_m = \left\{ y^{ \left ( i \right )} \in \mathbb{R}^p , 1 \leq i \leq M \right\}$ where the $y^{ \left ( i \right )}$'s are the grid points of a sparse grid. Typically, $M << N^p$.
Consider $M$ basis functions $\left\{ \phi_1(.), \cdots , \phi_m(.) \right\}$ mapping $\Omega$ into $\mathbb{R}$ (I chose local polynomials in my case). $\phi_i(.) $ is the basis function associated with sparse grid node $y^{ \left ( i \right )}$. Let $\tilde{V}$ be the space spanned by my basis functions.
$$ \tilde{V} = \left\{ \tilde{v} \in \mathbb{R} : \tilde{v} = \sum_{m=1}^{M} \phi_m\left ( \mathbf{x} \right ) w_m = \Phi \left ( \mathbf{x} \right ) \cdot \mathbf{w}, \mathbf{w} \in \mathbb{R}^m \text{ and } \mathbf{x} \in \Omega \right\} $$
Now, the problem rewrites
$$ \mathcal{A} \Phi \mathbf{w}_{t+\Delta t} = b \left ( \mathbf{w}_t \right ), \quad \mathbf{w} \in \mathbb{R}^{M} $$
i.e. I want to find a fixed-point $\mathbf{w}^* \in \mathbb{R}^M$ of the above equation.
What is important is that if I want to use the same dataset of $N^p$ values of $v$ defined at grid points $\mathbf{x}_i $, then the problem becomes overdetermined because the matrix $\Phi$ is rectangular and every iteration solves a least squares regression.
$$ \Phi = \Biggl( \begin{matrix} \phi_1 (x_{1}) & \cdots & \phi_M (x_{1}) \\ \vdots & \ddots & \vdots \\ \phi_1 (x_{N^p}) & \cdots & \phi_M (x_{N^p}) \\ \end{matrix}\Biggl) \quad \in \mathbb{R}^{N^p \times M} \quad \quad b : \mathbb{R}^{ M} \mapsto \mathbb{R}^{N^p} $$
$$ \mathbf{w}_{t+\Delta t} = \underset{\mathbf{w} \in \mathbb{R}^{M} }{\min} \left\| \mathcal{A} \Phi \mathbf{w} - b \left ( \mathbf{w}_t \right ) \right\|_2 $$
With this scheme, I have the possibility to actually choose the points $\left \{ \mathbf{x}^{ \left ( i \right ) } , 1 \leq i \leq \tilde{N} \right \}$. They may or may not come from the regular grid $\mathcal{G}_h$ of the FD scheme. I just need to ensure that $M \leq \tilde{N}$ and that $\Phi$ has full column rank (no multi-collinearity).
Questions
- There is a trade-off between performance (number of points) and approximation error over $\Omega$. Is there a theory that could guide the choice of the regression points ?
- The accuracy of the estimation of the $\mathbf{w}$ coefficients depends on the condition number of the matrix $\Phi$. Is there a way of optimally placing the points $\mathbf{x}^{ \left ( i \right )}$ over $\Omega$ so as to achieve best accuracy with minimal amount of points ?
- For a given function approximation scheme $\phi$ and $M$ points, can I find an upper bound or order of magnitude of the approximation error of $v$ I am making ? something like $\sup \int_{\Omega} \left\| \left ( v^*(x)-\Phi(x)\cdot w^*\right ) \right \| \mathrm{d} x \leq T(M)$
- Is using an over-determined system a good idea to start with?
