Linear trial spaces

Solving a density estimation problem numerically, the function $\phi$ has to be discretized. This is done by expanding $\phi$ in a basis $B_l$ (not necessarily orthonormal) and, choosing some $l_{\rm max}$, truncating the sum to terms with $l\le l_{\rm max}$,

$$\phi = \sum_{l=1}^\infty c_l B_l \rightarrow \phi = \sum_{l=1}^{l_{\rm max}} c_l B_l .$$ (380)

This, also called Ritz's method, corresponds to a finite linear trial space and is equivalent to solving a projected stationarity equation. Using a discretization (380) the functional (187) becomes

$$E_{\rm Ritz} = -(\,\ln P(\phi),\,N\,) +\frac{1}{2}\sum_{kl} c_k c_l (\,B_k,\,{{\bf K}}\,B_l\,) + (\,P(\phi),\,\Lambda_X\,).$$ (381)

Solving for the coefficients $c_l$, $l\le l_{\rm max}$ to minimize the error results according to Eq.[355) and

$$\Phi^\prime (l;x,y) = B_l(x,y),$$ (382)

in

$$0 = (\, B_l ,\, {\bf P}^\prime {\bf P}^{-1}\,N\,) - \sum_k ... ...,\, {\bf P}^\prime \, \Lambda_X\,) , \forall l\le l_{\rm max},$$ (383)

corresponding to the $l_{\rm max}$-dimensional equation

$${{\bf K}}_B c = N_B (c) - \Lambda_B (c),$$ (384)

with

$$c(l) = c_l,$$ (385)

$${{\bf K}}_B (l,k) = (\,B_l,\, {{\bf K}}\,B_k\,),$$ (386)

$$N_B (c)(l) = (\,B_l,\,{\bf P}^\prime(\phi(c))\,{\bf P}^{-1}(\phi(c))\,N\,),$$ (387)

$$\Lambda_B (c)(l) = (\, B_l,\, {\bf P}^\prime(\phi (c)) \,\Lambda_X\,).$$ (388)

Thus, for an orthonormal basis $B_l$ Eq. (384) corresponds to Eq. (189) projected into the trial space by the projector $\sum_l B_l\,B_l^T$.

The so called linear models are obtained by the (very restrictive) choice

$$\phi(z) = \sum_{l=0}^{1} c_l B_l = c_0 + \sum_l c_l z_l$$ (389)

with $z=(x,y)$ and $B_0$ = 1 and $B_l$ = $z_l$. Interactions, i.e., terms proportional to products of $z$-components like $c_{mn}z_mz_n$ can be included. Including all possible interaction would correspond to a multidimensional Taylor expansion of the function $\phi (z)$.

If the functions $B_l(z)$ are also parameterized this leads to mixture models for $\phi$. (See Section 4.4.)