Learning from data with generalization capability by neural networks
and kernel methods
speaker: Dr. Vera Kurkova
Academy of Sciences
Institute of Computer Science Prague
Czech Republic
vera@cs.cas.cz
The goal of supervised learning is to adjust parameters of a neural
network so that it approximates with a desired accuracy a functional
relationship between inputs and outputs by learning from a set of examples
in such a way that the network has a generalization capability, i.e.,
it can be used for processing new data that were not used for learning.
To guarantee generalization, one needs some global knowledge of the
desired input/output functional relationship, such as smoothness and
lack of high frequency oscillations.
The lecture will present various approaches to modelling of learning
with generalization capability based on regularization methods. Learning
as a regularized optimization problem will be studied for a special
class of function spaces called reproducing kernel Hilbert spaces, in
which many types of oscillations and smoothness conditions can be formally
described. It will be shown how methods developed for treating inverse
problem related to differential equations from physics can be used as
tools in mathematical theory of learning.
There will be described properties and relationships of important types
of regularization techniques (Ivanov's regularization based on a restriction
of the space of input/output functions, Tychonov's one adding to an
empirical error enforcing fitting to empirical data a term penalizing
undesired properties of input/output function and Miller's and Philips'
combining Ivanov's and Tychonov's method). Various versions of the Representer
Theorem describing the form of the unique solution of the learning problem
will be derived. Algorithms based on such theorems will be discussed
and compared with typical neural network algorithms designed for networks
with limited model complexity.
