lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Adding motivation to BNN

Browse Source
This commit is contained in:
mhjensen
2019-06-08 20:03:19 -04:00
parent 26610911ad
commit 00d4e6e22f
12 changed files with 129 additions and 71 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/pub/Bayesian/html/._Bayesian-bs000.html
+3 -3
View File
@@ -41,7 +41,7 @@ Automatically generated HTML file from DocOnce source
<!-- tocinfo
{'highest level': 2,
'sections': [('What is Bayesian Statistics', 2, None, '___sec0'),
'sections': [('Why Bayesian Statistics?', 2, None, '___sec0'),
('Inference', 2, None, '___sec1'),
('Statistical Inference', 2, None, '___sec2'),
('Some history', 2, None, '___sec3'),
@@ -115,7 +115,7 @@ MathJax.Hub.Config({
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._Bayesian-bs001.html#___sec0" style="font-size: 80%;"><b>What is Bayesian Statistics</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs001.html#___sec0" style="font-size: 80%;"><b>Why Bayesian Statistics?</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs002.html#___sec1" style="font-size: 80%;"><b>Inference</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs003.html#___sec2" style="font-size: 80%;"><b>Statistical Inference</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs004.html#___sec3" style="font-size: 80%;"><b>Some history</b></a></li>
@@ -175,7 +175,7 @@ MathJax.Hub.Config({
<center>[3] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>Jun 7, 2019</h4></center> <!-- date -->
<center><h4>Jun 8, 2019</h4></center> <!-- date -->
<br>
<p>
doc/pub/Bayesian/html/._Bayesian-bs001.html
+21 -11
View File
@@ -41,7 +41,7 @@ Automatically generated HTML file from DocOnce source
<!-- tocinfo
{'highest level': 2,
'sections': [('What is Bayesian Statistics', 2, None, '___sec0'),
'sections': [('Why Bayesian Statistics?', 2, None, '___sec0'),
('Inference', 2, None, '___sec1'),
('Statistical Inference', 2, None, '___sec2'),
('Some history', 2, None, '___sec3'),
@@ -115,7 +115,7 @@ MathJax.Hub.Config({
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="#___sec0" style="font-size: 80%;"><b>What is Bayesian Statistics</b></a></li>
<!-- navigation toc: --> <li><a href="#___sec0" style="font-size: 80%;"><b>Why Bayesian Statistics?</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs002.html#___sec1" style="font-size: 80%;"><b>Inference</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs003.html#___sec2" style="font-size: 80%;"><b>Statistical Inference</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs004.html#___sec3" style="font-size: 80%;"><b>Some history</b></a></li>
@@ -151,19 +151,29 @@ MathJax.Hub.Config({
<a name="part0001"></a>
<!-- !split -->
<h2 id="___sec0" class="anchor">What is Bayesian Statistics </h2>
<h2 id="___sec0" class="anchor">Why Bayesian Statistics? </h2>
<div class="panel panel-default">
<div class="panel-body">
<p> <!-- subsequent paragraphs come in larger fonts, so start with a paragraph -->
Morten's original plan: Reminder about probabilities from the statistics section
<ol>
<li> Product rule</li>
<li> Binomial distribution</li>
<li> Gaussian PDF</li>
<li> other PDFs</li>
<li> Bayesian regression analysis</li>
</ol>
<p>
We have already made ourselves familiar with elements of a statistical
data analysis via quantities like the bias-variance tradeoff as well
as some central distribution functions such as the Normal
distribution, the binomial distribution and other probability
distribution functions.
<p>
In essentially all the Machine Learning
algorithms we have studied, our focus has been on a so-called
<b>frequentist approach</b>, where knowledge of an underlying likelihood
function has not been emphasized. Our data, whether we had a classification or a regression problem, have been our central points of departure.
<p>
Here we wish to merge this approach with the derivation of a likelihood function which can be used to make prediction on how our system under study evolves.
We will venture into the realm of what is called Bayesian Neural Networks. To get an overarching view on what this entails, the following figure conveys the essential differences between a standard Neural network that we have met earlier and a Bayesian Neural Network. In order to get there, we need to present some of the basic elements of Bayesian statistics, starting with the product rule and Bayes' theorem.
<p>
</div>
</div>
doc/pub/Bayesian/html/Bayesian-bs.html
+3 -3
View File
@@ -41,7 +41,7 @@ Automatically generated HTML file from DocOnce source
<!-- tocinfo
{'highest level': 2,
'sections': [('What is Bayesian Statistics', 2, None, '___sec0'),
'sections': [('Why Bayesian Statistics?', 2, None, '___sec0'),
('Inference', 2, None, '___sec1'),
('Statistical Inference', 2, None, '___sec2'),
('Some history', 2, None, '___sec3'),
@@ -115,7 +115,7 @@ MathJax.Hub.Config({
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._Bayesian-bs001.html#___sec0" style="font-size: 80%;"><b>What is Bayesian Statistics</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs001.html#___sec0" style="font-size: 80%;"><b>Why Bayesian Statistics?</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs002.html#___sec1" style="font-size: 80%;"><b>Inference</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs003.html#___sec2" style="font-size: 80%;"><b>Statistical Inference</b></a></li>
<!-- navigation toc: --> <li><a href="._Bayesian-bs004.html#___sec3" style="font-size: 80%;"><b>Some history</b></a></li>
@@ -175,7 +175,7 @@ MathJax.Hub.Config({
<center>[3] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>Jun 7, 2019</h4></center> <!-- date -->
<center><h4>Jun 8, 2019</h4></center> <!-- date -->
<br>
<p>
doc/pub/Bayesian/html/Bayesian-reveal.html
+18 -10
View File
@@ -153,7 +153,7 @@ MathJax.Hub.Config({
<center>[3] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>&nbsp;<br>
<center><h4>Jun 7, 2019</h4></center> <!-- date -->
<center><h4>Jun 8, 2019</h4></center> <!-- date -->
<br>
<p>
@@ -164,19 +164,27 @@ MathJax.Hub.Config({
<section>
<h2 id="___sec0">What is Bayesian Statistics </h2>
<h2 id="___sec0">Why Bayesian Statistics? </h2>
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<p>
Morten's original plan: Reminder about probabilities from the statistics section
We have already made ourselves familiar with elements of a statistical
data analysis via quantities like the bias-variance tradeoff as well
as some central distribution functions such as the Normal
distribution, the binomial distribution and other probability
distribution functions.
<p>
In essentially all the Machine Learning
algorithms we have studied, our focus has been on a so-called
<b>frequentist approach</b>, where knowledge of an underlying likelihood
function has not been emphasized. Our data, whether we had a classification or a regression problem, have been our central points of departure.
<p>
Here we wish to merge this approach with the derivation of a likelihood function which can be used to make prediction on how our system under study evolves.
We will venture into the realm of what is called Bayesian Neural Networks. To get an overarching view on what this entails, the following figure conveys the essential differences between a standard Neural network that we have met earlier and a Bayesian Neural Network. In order to get there, we need to present some of the basic elements of Bayesian statistics, starting with the product rule and Bayes' theorem.
<ol>
<p><li> Product rule</li>
<p><li> Binomial distribution</li>
<p><li> Gaussian PDF</li>
<p><li> other PDFs</li>
<p><li> Bayesian regression analysis</li>
</ol>
</div>
</section>
doc/pub/Bayesian/html/Bayesian-solarized.html
+21 -11
View File
@@ -61,7 +61,7 @@ div { text-align: justify; text-justify: inter-word; }
<!-- tocinfo
{'highest level': 2,
'sections': [('What is Bayesian Statistics', 2, None, '___sec0'),
'sections': [('Why Bayesian Statistics?', 2, None, '___sec0'),
('Inference', 2, None, '___sec1'),
('Statistical Inference', 2, None, '___sec2'),
('Some history', 2, None, '___sec3'),
@@ -144,24 +144,34 @@ MathJax.Hub.Config({
<center>[3] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>Jun 7, 2019</h4></center> <!-- date -->
<center><h4>Jun 8, 2019</h4></center> <!-- date -->
<br>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec0">What is Bayesian Statistics </h2>
<h2 id="___sec0">Why Bayesian Statistics? </h2>
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<p>
Morten's original plan: Reminder about probabilities from the statistics section
<ol>
<li> Product rule</li>
<li> Binomial distribution</li>
<li> Gaussian PDF</li>
<li> other PDFs</li>
<li> Bayesian regression analysis</li>
</ol>
<p>
We have already made ourselves familiar with elements of a statistical
data analysis via quantities like the bias-variance tradeoff as well
as some central distribution functions such as the Normal
distribution, the binomial distribution and other probability
distribution functions.
<p>
In essentially all the Machine Learning
algorithms we have studied, our focus has been on a so-called
<b>frequentist approach</b>, where knowledge of an underlying likelihood
function has not been emphasized. Our data, whether we had a classification or a regression problem, have been our central points of departure.
<p>
Here we wish to merge this approach with the derivation of a likelihood function which can be used to make prediction on how our system under study evolves.
We will venture into the realm of what is called Bayesian Neural Networks. To get an overarching view on what this entails, the following figure conveys the essential differences between a standard Neural network that we have met earlier and a Bayesian Neural Network. In order to get there, we need to present some of the basic elements of Bayesian statistics, starting with the product rule and Bayes' theorem.
</div>
doc/pub/Bayesian/html/Bayesian.html
+21 -11
View File
@@ -66,7 +66,7 @@ div { text-align: justify; text-justify: inter-word; }
<!-- tocinfo
{'highest level': 2,
'sections': [('What is Bayesian Statistics', 2, None, '___sec0'),
'sections': [('Why Bayesian Statistics?', 2, None, '___sec0'),
('Inference', 2, None, '___sec1'),
('Statistical Inference', 2, None, '___sec2'),
('Some history', 2, None, '___sec3'),
@@ -149,24 +149,34 @@ MathJax.Hub.Config({
<center>[3] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>Jun 7, 2019</h4></center> <!-- date -->
<center><h4>Jun 8, 2019</h4></center> <!-- date -->
<br>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec0">What is Bayesian Statistics </h2>
<h2 id="___sec0">Why Bayesian Statistics? </h2>
<div class="alert alert-block alert-block alert-text-normal">
<b></b>
<p>
Morten's original plan: Reminder about probabilities from the statistics section
<ol>
<li> Product rule</li>
<li> Binomial distribution</li>
<li> Gaussian PDF</li>
<li> other PDFs</li>
<li> Bayesian regression analysis</li>
</ol>
<p>
We have already made ourselves familiar with elements of a statistical
data analysis via quantities like the bias-variance tradeoff as well
as some central distribution functions such as the Normal
distribution, the binomial distribution and other probability
distribution functions.
<p>
In essentially all the Machine Learning
algorithms we have studied, our focus has been on a so-called
<b>frequentist approach</b>, where knowledge of an underlying likelihood
function has not been emphasized. Our data, whether we had a classification or a regression problem, have been our central points of departure.
<p>
Here we wish to merge this approach with the derivation of a likelihood function which can be used to make prediction on how our system under study evolves.
We will venture into the realm of what is called Bayesian Neural Networks. To get an overarching view on what this entails, the following figure conveys the essential differences between a standard Neural network that we have met earlier and a Bayesian Neural Network. In order to get there, we need to present some of the basic elements of Bayesian statistics, starting with the product rule and Bayes' theorem.
</div>
doc/pub/Bayesian/ipynb/Bayesian.ipynb
+13 -9
View File
@@ -12,24 +12,28 @@
"<!-- dom:AUTHOR: Morten Hjorth-Jensen at Department of Physics, University of Oslo & Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University -->\n",
"<!-- Author: --> **Morten Hjorth-Jensen**, Department of Physics, University of Oslo and Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University\n",
"\n",
"Date: **Jun 7, 2019**\n",
"Date: **Jun 8, 2019**\n",
"\n",
"Copyright 1999-2019, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license\n",
"\n",
"\n",
"\n",
"\n",
"## What is Bayesian Statistics\n",
"Morten's original plan: Reminder about probabilities from the statistics section\n",
"1. Product rule\n",
"## Why Bayesian Statistics?\n",
"\n",
"2. Binomial distribution\n",
"We have already made ourselves familiar with elements of a statistical\n",
"data analysis via quantities like the bias-variance tradeoff as well\n",
"as some central distribution functions such as the Normal\n",
"distribution, the binomial distribution and other probability\n",
"distribution functions. \n",
"\n",
"3. Gaussian PDF\n",
"In essentially all the Machine Learning\n",
"algorithms we have studied, our focus has been on a so-called\n",
"**frequentist approach**, where knowledge of an underlying likelihood\n",
"function has not been emphasized. Our data, whether we had a classification or a regression problem, have been our central points of departure. \n",
"\n",
"4. other PDFs\n",
"\n",
"5. Bayesian regression analysis\n",
"Here we wish to merge this approach with the derivation of a likelihood function which can be used to make prediction on how our system under study evolves. \n",
"We will venture into the realm of what is called Bayesian Neural Networks. To get an overarching view on what this entails, the following figure conveys the essential differences between a standard Neural network that we have met earlier and a Bayesian Neural Network. In order to get there, we need to present some of the basic elements of Bayesian statistics, starting with the product rule and Bayes' theorem.\n",
"\n",
"\n",
"\n",
doc/pub/Bayesian/ipynb/ipynb-Bayesian-src.tar.gz
BIN
View File
Binary file not shown.
doc/pub/Bayesian/pdf/Bayesian-minted.pdf
BIN
View File
Binary file not shown.
doc/src/Bayesian/Bayesian.do.txt
+16 -7
View File
@@ -5,14 +5,23 @@ DATE: today
!split
===== What is Bayesian Statistics =====
===== Why Bayesian Statistics? =====
!bblock
Morten's original plan: Reminder about probabilities from the statistics section
o Product rule
o Binomial distribution
o Gaussian PDF
o other PDFs
o Bayesian regression analysis
We have already made ourselves familiar with elements of a statistical
data analysis via quantities like the bias-variance tradeoff as well
as some central distribution functions such as the Normal
distribution, the binomial distribution and other probability
distribution functions.
In essentially all the Machine Learning
algorithms we have studied, our focus has been on a so-called
_frequentist approach_, where knowledge of an underlying likelihood
function has not been emphasized. Our data, whether we had a classification or a regression problem, have been our central points of departure.
Here we wish to merge this approach with the derivation of a likelihood function which can be used to make prediction on how our system under study evolves.
We will venture into the realm of what is called Bayesian Neural Networks. To get an overarching view on what this entails, the following figure conveys the essential differences between a standard Neural network that we have met earlier and a Bayesian Neural Network. In order to get there, we need to present some of the basic elements of Bayesian statistics, starting with the product rule and Bayes' theorem.
!eblock
!split
doc/web/course.do.txt
+2 -2
View File
@@ -18,7 +18,7 @@ chapters = {
'Linalg': 'Review of central linear algebra elements',
'Statistics': 'Monte Carlo methods and elements of probability theory',
'Regression': 'Regression Methods',
'Splines': 'Gradient methods',
'Splines': 'Gradient methods and Minimization Algorithms',
'LogReg': 'Logistic Regression',
'NeuralNet': 'Neural Networks',
'DimRed': 'Reduction of dimensionality',
@@ -29,7 +29,7 @@ chapters = {
'Autoencoders': 'Autoencoders',
'Reinforce': 'Reinforcement Learning',
'odenn': 'Solving ordinary and partial differential equations with Neural Networks',
'Bayesian': 'Elements of Bayesian theory',
'Bayesian': 'Elements of Bayesian theory and Bayesian Neural Networks',
'summary': 'Summary',
}
%>
doc/web/course.html
+11 -4
View File
@@ -6,6 +6,7 @@ Automatically generated HTML file from DocOnce source
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="description" content="Overview of course material: Data Analysis and Machine Learning">
<title>Overview of course material: Data Analysis and Machine Learning</title>
@@ -83,7 +84,10 @@ div { text-align: justify; text-justify: inter-word; }
None,
'___sec4'),
('Regression Methods', 2, None, '___sec5'),
('Gradient methods', 2, None, '___sec6'),
('Gradient methods and Minimization Algorithms',
2,
None,
'___sec6'),
('Logistic Regression', 2, None, '___sec7'),
('Neural Networks', 2, None, '___sec8'),
('Reduction of dimensionality', 2, None, '___sec9'),
@@ -104,7 +108,10 @@ div { text-align: justify; text-justify: inter-word; }
2,
None,
'___sec16'),
('Elements of Bayesian theory', 2, None, '___sec17'),
('Elements of Bayesian theory and Bayesian Neural Networks',
2,
None,
'___sec17'),
('Summary', 2, None, '___sec18'),
('Python and Scikit Learn, a short guide', 2, None, '___sec19'),
('Teach yourself C++', 2, None, '___sec20'),
@@ -366,7 +373,7 @@ formulas in HTML or ipython notebook files.
</ul>
<h2 id="___sec6">Gradient methods </h2>
<h2 id="___sec6">Gradient methods and Minimization Algorithms </h2>
<ul>
<li> LaTeX PDF:</li>
@@ -685,7 +692,7 @@ formulas in HTML or ipython notebook files.
</ul>
<h2 id="___sec17">Elements of Bayesian theory </h2>
<h2 id="___sec17">Elements of Bayesian theory and Bayesian Neural Networks </h2>
<ul>
<li> LaTeX PDF:</li>