Example: Square root of $P$

We already discussed the cases $\phi = \ln P$ with $P^\prime = P = e^L$, $P/P^\prime = 1$ and $\phi = P$ with $P^\prime = 1$, $P/P^\prime = P$. The choice $\phi = \sqrt{P}$ yields the common $L_2$-normalization condition over $y$

$$1 =\int \!dy\, \phi^2(x,y), \quad \forall x\in X,$$ (195)

which is quadratic in $\phi$, and $P=\phi^2$, $P^\prime = 2 \phi$, $P/P^\prime = \phi/2$. For real $\phi$ the non-negativity condition $P\ge 0$ is automatically satisfied [82,211].

For $\phi$ = $\sqrt{P}$ and a negative Laplacian inverse covariance ${{\bf K}}$ = $-\Delta$, one can relate the corresponding Gaussian prior to the Fisher information [38,211,207]. Consider, for example, a problem with fixed $x$ (so $x$ can be skipped from the notation and one can write $P(y)$) and a $d_y$-dimensional $y$. Then one has, assuming the necessary differentiability conditions and vanishing boundary terms,

$$(\,\phi \,,\, {{\bf K}} \, \phi \,) =-(\,\phi \,,\, \Delta \... ...y} \left\vert \frac{\partial \phi}{\partial y_k}\right\vert^2$$ (196)

$$= \sum_k^{d_y} \int \frac{dy }{ 4 P(y) } \left( \frac{ \par... ...artial y_k } \right)^2 = \frac{1}{4} \sum_k^{d_y} I^F_k (0) ,$$ (197)

where $I^F_k (0)$ is the Fisher information, defined as

$$I^F_k (y_0) = \int \! dy \frac{ \left\vert \frac{\parti... ...ial \ln P(y - y^0) }{\partial y^0_k}\right\vert^2 P(y-y^0_k),$$ (198)

for the family $P(\cdot-y^0)$ with location parameter vector $y^0$.

A connection to quantum mechanics can be found considering the case without training data

$$E_{\phi} = \frac{1}{2}(\,\phi,\,\,{{\bf K}}\,\phi\,) + (\Lambda_X,\,\phi) ,$$ (199)

having the homogeneous stationarity equation

$${{\bf K}} \, \phi = -2 \Phi \Lambda_X .$$ (200)

For $x$-independent $\Lambda_X$ this is an eigenvalue equation. Examples include the quantum mechanical Schrödinger equation where ${{\bf K}}$ corresponds to the system Hamiltonian and

$$-2 \Lambda_X = \frac{( \phi ,\, {{\bf K}} \, \phi )}{(\phi ,\,\phi)},$$ (201)

to its ground state energy. In quantum mechanics Eq. (201) is the basis for variational methods (see Section 4) to obtain approximate solutions for ground state energies [55,197,27].

Similarly, one can take $\phi = \sqrt{-(L-L_{\rm max})}$ for $L$ bounded from above by $L_{\rm max}$ with the normalization

$$1 =\int \!dy\, e^{-\phi^2(x,y)+L_{\rm max}}, \quad \forall x\in X,$$ (202)

and $P = e^{-\phi^2 + L_{\rm max}}$, $P^\prime = - 2 \phi P$, $P/P^\prime$ = $-1/(2\phi )$.