$ \let\oldcdot\cdot \renewcommand{\cdot}{\!\oldcdot\!} \newcommand{\e}{\varepsilon} \renewcommand{\p}{\varphi} \renewcommand{\p}{\varphi} \renewcommand{\vp}{\vec{\boldsymbol\p}(x)} \newcommand{\P}{\boldsymbol \varPhi} \newcommand{\O}{\boldsymbol \varOmega} \newcommand{\x}{\vec{\boldsymbol{x}}} \newcommand{\y}{\vec{\boldsymbol{y}}} \newcommand{\f}{\vec{\boldsymbol{f}}} \newcommand{\a}{\vec{\boldsymbol{\alpha}}} $ Given two set of points $\,\x = \left\{x _i \right\}_{i=1}^{n},\ \; \y = \left\{y _i \right\}_{i=1}^{n}\,$ in domain $\,\O\in \mathbb R^d$ and a target function $\,f:\O\to\mathbb R\,$ I need to approximate $\,f\,$ by Radial Basis Functions with reasonable accuracy.
Description of approximation process:
Choose generating function $\,\p:\mathbb R \to \mathbb R\,$ and define $\,\p_i(x) = \p\big(\e\left\| x_i - x\right\|\big)$, so we can approximate $\,f(x) \approx \sum_{i=1}^{n} \alpha_i \p_i(x).$ Denote vector of coefficients $\,\a = \{ \alpha_i\}_{i=1}^{n},\,$ interpolation matrix $\, \P = \{ \p_i(\,y_j\,)\}_{i,j = 1}^{n},\,$ estimation vector $\,\vp = \{ \p_i\left(x\right)\}_{i = 1}^{n},\,$ and vector of values of $\,f\,$ at $\,\x\,$ as $\,\f = f(\,\x\,).\,$ Then we can write the RBF expansion of $\,f\,$ in the matrix form: $\qquad \qquad \quad \f = \P \cdot \a \implies \a = \P^{-1}\cdot\f \implies f(x) \approx \vp \cdot \a = \vp \cdot \P^{-1}\cdot\f .$
I have troubles choosing optimal shape parameter $\e$, optimal distribution of RBF centers and optimal type of RBF. To make things worse, the domain $\,\O\,$ is supposed to be general manifold.
Starting from the simple case of $1\mathrm{D}$ domain I cannot lower the accuracy of the approximation as the interpolation matrix $\P$ becomes increasingly ill-conditioned. Ultimately I need to be able to reconstruct not just function, but also its derivatives and more sophisticated differential operators, but I cannot even reach $10^{-3}$ order of error even for straightforward function reconstruction on a nice $1\mathrm{D}$ manifold like semicircle or straight interval.
I would be very happy if someone could advise me techniques which could help increase accuracy of RBF approximation and/or prevent condition number of $\P$ from decreasing too rapidly.
PS This is a follow-up on my previous question about RBF approximation.
