High and low temperature limits

Low and high temperature limits are extremely useful because in both cases the stationarity Eq.(21) becomes linear, corresponding thus to classical quadratic regularisation approaches.

In the high temperature limit $\beta \rightarrow 0$ the exponential factors $a_j$ become $h$-independent

$$a_j \stackrel{\beta\rightarrow0}{\longrightarrow} a^0_j = \... ...\sum_k^me^{-E_{\theta,\beta,k}+\frac{1}{2}\ln\det{\bf K}_k}} ,$$ (34)

(for $b_j^{*}\rightarrow b_j^{0,*}$ replace ${\bf K}_j$ by $\widetilde{\bf K}_j$). The solution $h = \bar t$ is a (generalised) `complete template average'

$$\bar t = {\bf K}_{a^0}^{-1} \left( {\bf K_T} t_T + \sum_l^m a^0_j \,{\bf K}_j t_j \right) ,$$ (35)

with

$${\bf K}_{a^0} = {\bf K}_T+\sum_j a_j^0 \,{\bf K}_j .$$ (36)

This high temperature solution corresponds to the minimum of the quadratic functional $E_{T} +\sum_j^m a_j^0 E_{h,j}$,

In the low temperature limit $\beta\rightarrow \infty$ only the maximal component contributes, i.e.,

$$a_j \stackrel{\beta\rightarrow\infty}{\longrightarrow} a^\in... ...,j} \\ 0 & j \ne {\rm argmin}_j E_{h,j} \end{array} \right. ,$$ (37)

(for $b_j^*$ replace $E_{h,j}$ by $\widetilde E_j$) assuming $E_{\beta,\theta,j}$ = $E_{\beta}$ + $E_{\theta,j}$ or $E_{\beta,\theta,j}$ = $E_{\beta}$ + $E_{j}$ + $\beta E_{\theta}$. Hence, low temperature solutions $h = \bar t_{j}$, are all (generalised) `component averages' $\bar t_j$ provided they fulfil the stability condition

$$E_{h,j} (h=\bar t_j) <E_{h,j^\prime} (h=\bar t_j) , \quad \forall j^\prime \ne j ,$$ (38)

or, after performing a (generalised) `bias-variance' decomposition, $2 V_{j}<{B}_{j^\prime} (j,j) + 2 V_{j^\prime}$, with $m\times m$ matrices

$${B}_j (k,l) = \Big(\bar t_k-\bar t_j,\,\left( {\bf K}_D + {\bf K}_j\right) \,(\bar t_l-\bar t_j)\Big)$$ (39)

and (generalised) `template variances'

$$V_j \!\! = \frac{1}{2} \Bigg( \Big(t_T,\,{\bf K}_T\,t_T\Big) +\Big(t_j,\,{\bf K}_j\,t_j\Big) \quad$$

$$- \Big(\bar t_j,({\bf K}_T+{\bf K}_j) \,\bar t_j\Big) \Bigg) = \widetilde E_{0,j}.$$ (40)

That means single component averages $\bar t_j$ (which minimise $E_{h,j}$ and thus $-\beta E_j+c_j$) become solutions at zero temperature $1/\beta$ in case their (generalised) variance $V_j$ measuring the discrepancy between data and prior term is small enough.