Invariant determinants

In this section we discuss parameterizations of the inverse covariance of a Gaussian specific prior which leave the determinant invariant. In that case no $\theta$-dependent normalization factors have to be included which are usually very difficult to calculate. We have to keep in mind, however, that in general a large freedom for ${{\bf K}}(\theta)$ effectively diminishes the influence of the parameterized prior term.

A determinant is, for example, invariant under general similarity transformations, i.e., $\det \tilde {{\bf K}}$ = $\det {{\bf K}}$ for ${{\bf K}} \rightarrow \tilde {{\bf K}}$ = ${\bf O}{{\bf K}}{\bf O}^{-1}$ where ${\bf O}$ could be any element of the general linear group. Similarity transformations do not change the eigenvalues, because from ${{\bf K}}\psi$ = $\lambda \psi$ follows ${\bf O}{{\bf K}}{\bf O}^{-1}{\bf O}\psi$ = $\lambda {\bf O}\psi$. Thus, if ${{\bf K}}$ is positive definite also $\tilde {{\bf K}}$ is. The additional constraint that $\tilde {{\bf K}}$ has to be real symmetric,

$$\tilde {{\bf K}} = \tilde {{\bf K}}^T = \tilde {{\bf K}}^\dagger ,$$ (469)

requires ${\bf O}$ to be real and orthogonal

$${\bf O}^{-1} = {\bf O}^T = {\bf O}^\dagger .$$ (470)

Furthermore, as an overall factor of ${\bf O}$ does not change $\tilde {{\bf K}}$ one can restrict ${\bf O}$ to a special orthogonal group $SO(N)$ with $\det {\bf O}=1$. If ${{\bf K}}$ has degenerate eigenvalues there exist orthogonal transformations with ${{\bf K}}$ = $\tilde {{\bf K}}$.

While in one dimension only the identity remains as transformation, the condition of an invariant determinant becomes less restrictive in higher dimensions. Thus, especially for large dimension $d$ of ${{\bf K}}$ (infinite for continuous $x$) there is a great freedom to adapt inverse covariances without the need to calculate normalization factors, for example to adapt the sensible directions of a multivariate Gaussian.

A positive definite ${{\bf K}}$ can be diagonalized by an orthogonal matrix ${\bf O}$ with $\det {\bf O}$ = $1$, i.e., ${{\bf K}} = {\bf O}{\bf D}{\bf O}^{T}$. Parameterizing ${\bf O}$ the specific prior term becomes

$$\frac{1}{2} \Big(\phi-t ,\,{{\bf K}} (\theta)\,(\phi-t)\Big)... ...,\,{\bf O} (\theta){\bf D}{\bf O}^{T}(\theta)\,(\phi-t)\Big) ,$$ (471)

so the stationarity Eq. (459) reads

$$\Big(\phi-t ,\, \frac{\partial {\bf O}}{\partial \theta}{\bf D}{\bf O}^{T} \,(\phi-t)\Big) = -E_\theta^\prime .$$ (472)

Matrices ${\bf O}$ from $SO(N)$ include rotations and inversion. For a Gaussian specific prior with nondegenerate eigenvalues Eq. (472) allows therefore to adapt the `sensible' directions of the Gaussian.

There are also transformations which can change eigenvalues, but leave eigenvectors invariant. As example, consider a diagonal matrix ${\bf D}$ with diagonal elements (and eigenvalues) $\lambda_i\ne 0$, i.e., $\det {\bf D}$ = $\prod_i \lambda_i$. Clearly, any permutation of the eigenvalues $\lambda_i$ leaves the determinant invariant and transforms a positive definite matrix into a positive definite matrix. Furthermore, one may introduce continuous parameters $\theta_{ij}>0$ with $i<j$ and transform ${\bf D} \rightarrow \tilde{\bf D}$ according to

$$\lambda_i \rightarrow \tilde \lambda_i = \lambda_i \theta_{... ...rightarrow \tilde \lambda_j = \frac{\lambda_j}{\theta_{ij}} ,$$ (473)

which leaves the product $\lambda_i\lambda_j$ = $\tilde\lambda_i\tilde\lambda_j$ and therefore also the determinant invariant and transforms a positive definite matrix into a positive definite matrix. This can be done with every pair of eigenvalues defining a set of continuous parameters $\theta_{ij}$ with $i<j$ ($\theta_{ij}$ can be completed to a symmetric matrix) leading to

$$\lambda_i \rightarrow \tilde \lambda_i = \lambda_i \frac{\prod_{j>i} \theta_{ij}}{\prod_{j<i} \theta_{ji}} ,$$ (474)

which also leaves the determinant invariant

$$\det \tilde {\bf D} = \prod_i \tilde \lambda_i = \prod_i \le... ...\prod_{j<i} \theta_{ji}} = \prod_i \lambda_i = \det {\bf D} .$$ (475)

A more general transformation with unique parameterization by $\theta_i>0$, $i\ne i^*$, still leaving the eigenvectors unchanged, would be

$$\tilde \lambda_i = \lambda_i \theta_i,\; i\ne i^* ;\quad \ti... ...\lambda_{i^*} = \lambda_{i^*} \prod_{i\ne i^*} \theta_i^{-1} .$$ (476)

This techniques can be applied to a general positive definite ${{\bf K}}$ after diagonalizing

$${{\bf K}} = {\bf O}{\bf D}{\bf O}^{T} \rightarrow \tilde {{\... ...bf O}^{T} \Rightarrow \det {{\bf K}} = \det \tilde{{\bf K}} .$$ (477)

As example consider the transformations (474, 476) for which the specific prior term becomes

$$\frac{1}{2} \Big(\phi-t ,\,{{\bf K}} (\theta)\,(\phi-t)\Big)... ...g(\phi-t ,\,{\bf O}{\bf D}(\theta){\bf O}^{T}\,(\phi-t)\Big) ,$$ (478)

and stationarity Eq. (459)

$$\frac{1}{2} \Big(\phi-t ,\, {\bf O}\frac{\partial {\bf D}}{\partial \theta}{\bf O}^{T} \,(\phi-t)\Big) = -E_\theta^\prime ,$$ (479)

and for (474), with $k<l$,

$$\frac{\partial {\bf D}(i,j)} {\partial \theta_{kl}} = \delt... ...prod_{n>l} \theta_{ln}}{\prod_{k\ne n<l} \theta_{nl}} \Bigg) ,$$ (480)

or, for (476), with $k\ne i^*$,

$$\frac{\partial {\bf D}(i,j)} {\partial \theta_{k}} = \delta... ...i^*} \frac{1}{\theta_k \prod_{l\ne i^*} \theta_{l}} \Bigg) .$$ (481)

If, for example, ${{\bf K}}$ is a translationally invariant operator it is diagonalized in a basis of plane waves. Then also $\tilde {{\bf K}}$ is translationally invariant, but its sensitivity to certain frequencies has changed. The optimal sensitivity pattern is determined by the given stationarity equations.