Next: Product ansatz
Up: Parameterizing likelihoods: Variational methods
Previous: Mixture models
  Contents
Additive models
Trial functions may be chosen as sum
of simpler functions each depending only
on part of the and variables.
More precisely, we consider functions
depending on projections =
of the vector =
of all and components.
denotes an projector in the vector space of
(and not in the space of functions ).
Hence, becomes of the form
|
(391) |
so only one-dimensional functions
have to be determined.
Restricting the functions
to a
parameterized function space
yields
a ``parameterized additive model''
|
(392) |
which has to be solved for the parameters .
The model can also be generalized
to a model ``additive in parameters ''
|
(393) |
where the functions
are not restricted to one-dimensional functions
depending only on projections on the coordinate axes.
If the parameters determine the component functions
completely, this yields just the mixture models
of Section 4.4.
Another example is projection pursuit,
discussed in Section 4.8),
where a parameter vector
corresponds to a projections .
In that case even for given still a one-dimensional function
has to be determined.
An ansatz like (391) is made more flexible
by including also interactions
|
(394) |
The functions
can be chosen to depend on
product terms
like
,
or
,
where denotes one-dimensional
sub-variables of .
In additive models in the narrower sense
[218,92,93,94]
is a subset of , components, i.e.,
,
denoting the dimension of ,
the dimension of .
In regression, for example, one takes usually the one-element subsets
= for
.
In more general schemes the projections of
do not have to be restricted to projections on the coordinates axes.
In particular, the projections can be optimized too.
For example, one-dimensional projections
= with
(where denotes a scalar product
in the space of variables)
are used by ridge approximation schemes.
They include for regression problems
one-layer (and similarly multilayer)
feedforward neural networks
(see Section 4.9)
projection pursuit regression
(see Section 4.8),
ridge regression [151,152],
and hinge functions [31].
For a detailed discussion of the regression case see
[76].
The stationarity equation for
becomes
for the ansatz (391)
|
(395) |
with
|
(396) |
Considering a density being also decomposed into
components determined
by the components
|
(397) |
the derivative (396) becomes
|
(398) |
so that specifying an additive prior
|
(399) |
the stationary conditions are coupled equations
for the component functions
which, because is diagonal,
only contain integrations over -variables
|
(400) |
For the parameterized approach (392)
one finds
|
(401) |
with
|
(402) |
For the ansatz (393)
would be restricted to a subset of .
Next: Product ansatz
Up: Parameterizing likelihoods: Variational methods
Previous: Mixture models
  Contents
Joerg_Lemm
2001-01-21