Christian Forssen
165 lines
4.7 KiB
HTML
165 lines
4.7 KiB
HTML
<!--
|
|
Automatically generated HTML file from DocOnce source
|
|
(https://github.com/hplgit/doconce/)
|
|
-->
|
|
<html>
|
|
<head>
|
|
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
|
|
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
|
|
<meta name="description" content="Data Analysis and Machine Learning: Machine learning with Gaussian Processes">
|
|
|
|
<title>Data Analysis and Machine Learning: Machine learning with Gaussian Processes</title>
|
|
|
|
|
|
<style type="text/css">
|
|
/* bloodish style */
|
|
|
|
body {
|
|
font-family: Helvetica, Verdana, Arial, Sans-serif;
|
|
color: #404040;
|
|
background: #ffffff;
|
|
}
|
|
h1 { font-size: 1.8em; color: #8A0808; }
|
|
h2 { font-size: 1.6em; color: #8A0808; }
|
|
h3 { font-size: 1.4em; color: #8A0808; }
|
|
h4 { color: #8A0808; }
|
|
a { color: #8A0808; text-decoration:none; }
|
|
tt { font-family: "Courier New", Courier; }
|
|
/* pre style removed because it will interfer with pygments */
|
|
p { text-indent: 0px; }
|
|
hr { border: 0; width: 80%; border-bottom: 1px solid #aaa}
|
|
p.caption { width: 80%; font-style: normal; text-align: left; }
|
|
hr.figure { border: 0; width: 80%; border-bottom: 1px solid #aaa}
|
|
|
|
div { text-align: justify; text-justify: inter-word; }
|
|
</style>
|
|
|
|
|
|
</head>
|
|
|
|
<!-- tocinfo
|
|
{'highest level': 2,
|
|
'sections': [('What is a Gaussian Process?', 2, None, '___sec0'),
|
|
('Gaussian process regression', 2, None, '___sec1'),
|
|
('Topics', 2, None, '___sec2')]}
|
|
end of tocinfo -->
|
|
|
|
<body>
|
|
|
|
|
|
|
|
<script type="text/x-mathjax-config">
|
|
MathJax.Hub.Config({
|
|
TeX: {
|
|
equationNumbers: { autoNumber: "AMS" },
|
|
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
|
|
}
|
|
});
|
|
</script>
|
|
<script type="text/javascript" async
|
|
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
|
|
</script>
|
|
|
|
|
|
|
|
|
|
<!-- ------------------- main content ---------------------- -->
|
|
|
|
|
|
|
|
<center><h1>Data Analysis and Machine Learning: Machine learning with Gaussian Processes</h1></center> <!-- document title -->
|
|
|
|
<p>
|
|
<!-- author(s): Christian Forssén, and Morten Hjorth-Jensen -->
|
|
|
|
<center>
|
|
<b>Christian Forssén</b> [1]
|
|
</center>
|
|
|
|
<center>
|
|
<b>Morten Hjorth-Jensen</b> [2, 3]
|
|
</center>
|
|
|
|
<p>
|
|
<!-- institution(s) -->
|
|
|
|
<center>[1] <b>Department of Physics, Chalmers University of Technology, Sweden</b></center>
|
|
<center>[2] <b>Department of Physics, University of Oslo</b></center>
|
|
<center>[3] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
|
|
<br>
|
|
<p>
|
|
<center><h4>Mar 19, 2018</h4></center> <!-- date -->
|
|
<br>
|
|
<p>
|
|
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
|
|
|
|
<h2 id="___sec0">What is a Gaussian Process? </h2>
|
|
|
|
<ul>
|
|
<li> We have considered splines and kernel regression methods. These</li>
|
|
</ul>
|
|
|
|
require choice of somewhat arbitrary set of knots.
|
|
|
|
<ul>
|
|
<li> Antoher possibility is to setup a prior distribution for the
|
|
regression function using a <em>Gaussian Process</em>.</li>
|
|
<li> This is a very flexible class of models that has distinct computational
|
|
and theoretical advantages. It can be viewed as a potentially
|
|
infinite-dimensional generalization of Gaussian distributions.</li>
|
|
<li> See the excellent (and free) book <a href="http://www.gaussianprocess.org/gpml/" target="_blank">Gaussian Processes for Machine
|
|
Learning</a> by Carl Edward
|
|
Rasmussen and Christopher K. I. Williams.</li>
|
|
</ul>
|
|
|
|
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
|
|
|
|
<h2 id="___sec1">Gaussian process regression </h2>
|
|
|
|
<ul>
|
|
<li> Realizations from a Gaussian process correspond to random functions</li>
|
|
<li> Let us first consider an unknown regression function \( \mu(x) \) that
|
|
depends on a single, continuous variable \( x \).</li>
|
|
<li> The Gaussian process is written as \( \mu \sim \mathrm{GP}(m,k) \), and
|
|
is parametrized in terms of a mean function \( m(x) \) and a covariance
|
|
function \( k(x,x') \).</li>
|
|
<li> The GP prior on \( \mu \) describes it as a random function for which
|
|
the values at any set of \( N \) prespecified points \( \{x_i\}_{i=1}^N \)
|
|
are a draw from a \( N \)-dimensional normal distribution</li>
|
|
</ul>
|
|
|
|
$$
|
|
$$
|
|
\mu(x_1), \ldots \mu(x_N) \sim \mathrm{N}\left( \left( m(x_1),
|
|
\ldots, m(x_N) \right), K(x_1, \ldots, x_N) \right),
|
|
$$
|
|
$$
|
|
|
|
with mean \( m \) and covariance \( K \).
|
|
|
|
<p>
|
|
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
|
|
|
|
<h2 id="___sec2">Topics </h2>
|
|
|
|
<ul>
|
|
<li> More matematical details</li>
|
|
<li> The role of the covariance function (different kernels)</li>
|
|
<li> multidimensional case</li>
|
|
<li> examples.</li>
|
|
</ul>
|
|
|
|
|
|
<!-- ------------------- end of main content --------------- -->
|
|
|
|
|
|
<center style="font-size:80%">
|
|
<!-- copyright --> © 1999-2018, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
|
|
</center>
|
|
|
|
|
|
</body>
|
|
</html>
|
|
|
|
|