Conclusions

Many nonparametric real world learning problems are generically non-Gaussian. Examples include general density estimation or the reconstruction of forces for quantum systems from observational data. Increasing computational resources allow to solve at least some of such low-dimensional non-Gaussian problems directly by discretization.

Figure 1: (a): True conditional likelihood $P_{\rm true}(x,y)$ = $p(y\vert x,\phi_{\rm true})$. (b): Corresponding true field, here chosen as (conditional) log-likelihood $L$ = $\ln P$, i.e., $\phi_{\rm true}(x,y)$ = $\ln P_{\rm true}(x,y)$ = $L_{\rm true}(x,y)$.

Figure 2: (a): Conditional empirical density $P_{\rm emp}(x,y)$ = $\sum _i\delta (x-x_i)\sum _i\delta (y-y_i)\sum _i/\sum _i \delta (x-x_i)$, as obtained from 50 data points sampled from the true likelihood $P_{\rm true}$ of Fig. 1a under uniform $p(x)$. (b): Conditional likelihood $P$ reconstructed from the conditional empirical density shown on the l.h.s. using, besides the normalization constraint, an additional Gaussian prior term on $\phi$ (see text).

Figure 3: (a): Regression function for $P_{\rm true}$ of Fig. 1a. (b): Regression function for the reconstructed $P$ of Fig. 2b. (The regression function for a field $\phi$ is defined as $h(x)$ = $\int\!dy\, y\, p(y\vert x,\phi)$. The dashed lines indicate the range of one standard deviation above and below the regression function.)