Gaussian relaxation

As Gaussian kernels are often used in density estimation and also in function approximation (e.g. for radial basis functions [191]) we consider the example

$${\bf A} = \sum_{k=0}^\infty \frac{1}{k!} \left( \frac{{\bf M}^2 }{2\tilde\sigma^2}\right)^k =e^{\frac{{\bf M}^2}{2\tilde\sigma^2}} : {\rm Gaussian}$$ (654)

with positive semi-definite ${\bf M}^2$. The contribution for $k=0$ corresponds to a mass term so for positive semi-definite ${\bf M}$ this ${\bf A}$ is positive definite and therefore invertible with inverse

$${\bf A}^{-1} = e^{-\frac{{\bf M}^2}{2\tilde\sigma^2}},$$ (655)

which is diagonal and Gaussian in ${\bf M}$-representation. In the limit $\tilde \sigma\rightarrow \infty$ or for zero modes of ${\bf M}$ the Gaussian ${\bf A}^{-1}$ becomes the identity ${\bf I}$, corresponding to the gradient algorithm. Consider

$${\bf M}^2 (x^\prime , y^\prime ; x, y) = - \delta (x-x^\prime )\delta (y-y^\prime ) \Delta$$ (656)

where the $\delta$-functions are usually skipped from the notation, and

$$\Delta = \frac{\partial^2}{\partial x^2} + \frac{\partial^2}{\partial y^2},$$

denotes the Laplacian. The kernel of the inverse is diagonal in Fourier representation

$$A(k_x^\prime,k_y^\prime;,k_x,k_y) = \delta (k_x-k_x^\prime )... ...ta (k_y-k_y^\prime ) e^{-\frac{k_x^2+k_y^2}{2\tilde\sigma^2}}$$ (657)

and non-diagonal, but also Gaussian in $(x,y)$-representation

$${\bf A}^{-1} (x^\prime , y^\prime ; x,y ) = e^{-\frac{\Delta... ...}{2 \tilde\sigma^2} +i k_x (x-x^\prime ) +i k_y (y-y^\prime )}$$ (658)

$$=\left(\frac{\tilde\sigma}{ \sqrt{2 \pi}}\right)^d e^{-\tild... ..., \, e^{-\frac{(x-x^\prime )^2 + (y-y^\prime )^2}{2\sigma^2}},$$ (659)

with $\sigma = 1/{\tilde\sigma}$ and $d=d_X+d_Y$, $d_X$ = dim($X$), $d_Y$ = dim($Y$).