Christian Forssen
53 lines
1.9 KiB
Plaintext
53 lines
1.9 KiB
Plaintext
TITLE: Data Analysis and Machine Learning: Machine learning with Gaussian Processes
|
|
AUTHOR: Christian Forssén at Department of Physics, Chalmers University of Technology, Sweden
|
|
AUTHOR: Morten Hjorth-Jensen {copyright, 1999-present|CC BY-NC} at Department of Physics, University of Oslo & Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University
|
|
DATE: today
|
|
|
|
!split
|
|
===== What is a Gaussian Process? =====
|
|
|
|
* We have considered splines and kernel regression methods. These
|
|
require choice of somewhat arbitrary set of knots.
|
|
|
|
* Antoher possibility is to setup a prior distribution for the
|
|
regression function using a *Gaussian Process*.
|
|
|
|
* This is a very flexible class of models that has distinct computational
|
|
and theoretical advantages. It can be viewed as a potentially
|
|
infinite-dimensional generalization of Gaussian distributions.
|
|
|
|
* See the excellent (and free) book "Gaussian Processes for Machine
|
|
Learning": "http://www.gaussianprocess.org/gpml/" by Carl Edward
|
|
Rasmussen and Christopher K. I. Williams.
|
|
|
|
!split
|
|
===== Gaussian process regression =====
|
|
|
|
* Realizations from a Gaussian process correspond to random functions
|
|
|
|
* Let us first consider an unknown regression function $\mu(x)$ that
|
|
depends on a single, continuous variable $x$.
|
|
|
|
* The Gaussian process is written as $\mu \sim \mathrm{GP}(m,k)$, and
|
|
is parametrized in terms of a mean function $m(x)$ and a covariance
|
|
function $k(x,x')$.
|
|
|
|
* The GP prior on $\mu$ describes it as a random function for which
|
|
the values at any set of $N$ prespecified points $\{x_i\}_{i=1}^N$
|
|
are a draw from a $N$-dimensional normal distribution
|
|
|
|
!bt
|
|
$$
|
|
\mu(x_1), \ldots \mu(x_N) \sim \mathrm{N}\left( \left( m(x_1),
|
|
\ldots, m(x_N) \right), K(x_1, \ldots, x_N) \right),
|
|
$$
|
|
!et
|
|
with mean $m$ and covariance $K$.
|
|
|
|
!split
|
|
===== Topics =====
|
|
* More matematical details
|
|
* The role of the covariance function (different kernels)
|
|
* multidimensional case
|
|
* examples.
|