added exercises week 39
This commit is contained in:
Morten Hjorth-Jensen
@@ -0,0 +1,59 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "f35930ac",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"<!-- HTML file automatically generated from DocOnce source (https://github.com/doconce/doconce/)\n",
|
||||
"doconce format html exercisesweek39.do.txt -->\n",
|
||||
"<!-- dom:TITLE: Exercises week 39 -->"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "8cb567a0",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"# Exercises week 39\n",
|
||||
"**September 25-29, 2023**\n",
|
||||
"\n",
|
||||
"Date: **Deadline is Sunday October 1 at midnight**"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "934324b3",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"## Overarching aims of the exercises this week\n",
|
||||
"\n",
|
||||
"The aim of the exercises this week is to aid you in getting started\n",
|
||||
"with writing the report. This will be discussed during the lab\n",
|
||||
"sessions as well. One of the lab sessions will be recorded.\n",
|
||||
"\n",
|
||||
"A general guideline can be found at <https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/EvaluationGrading/EvaluationForm.md>.\n",
|
||||
"\n",
|
||||
"Similarly, an example of an earlier project can be found at <https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ReportExample/ReportSample.pdf>\n",
|
||||
"\n",
|
||||
"Your task this week is to\n",
|
||||
"1. Write an abstract for your project\n",
|
||||
"\n",
|
||||
"2. Write an introduction\n",
|
||||
"\n",
|
||||
"3. Include references\n",
|
||||
"\n",
|
||||
"Ashort feedback to the this exercise will be available after the deadline. And you can reuse these elements in your final report."
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
@@ -1,77 +1,24 @@
|
||||
TITLE: Exercises week 38
|
||||
AUTHOR: September 18-22, 2023
|
||||
DATE: Deadline is Sunday September 24 at midnight
|
||||
TITLE: Exercises week 39
|
||||
AUTHOR: September 25-29, 2023
|
||||
DATE: Deadline is Sunday October 1 at midnight
|
||||
|
||||
|
||||
===== Overarching aims of the exercises this week =====
|
||||
|
||||
The aim of the exercises this week is to derive the equations for the bias-variance tradeoff to be used in project 1 as well as testing this for a simpler function using the bootstrap method. The exercises here can be reused in project 1 as well.
|
||||
The aim of the exercises this week is to aid you in getting started
|
||||
with writing the report. This will be discussed during the lab
|
||||
sessions as well. One of the lab sessions will be recorded.
|
||||
|
||||
Consider a
|
||||
dataset $\mathcal{L}$ consisting of the data
|
||||
$\mathbf{X}_\mathcal{L}=\{(y_j, \boldsymbol{x}_j), j=0\ldots n-1\}$.
|
||||
A general guideline can be found at URL:"https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/EvaluationGrading/EvaluationForm.md".
|
||||
|
||||
We assume that the true data is generated from a noisy model
|
||||
|
||||
!bt
|
||||
\[
|
||||
\bm{y}=f(\boldsymbol{x}) + \bm{\epsilon}.
|
||||
\]
|
||||
!et
|
||||
|
||||
Here $\epsilon$ is normally distributed with mean zero and standard
|
||||
deviation $\sigma^2$.
|
||||
|
||||
In our derivation of the ordinary least squares method we defined
|
||||
an approximation to the function $f$ in terms of the parameters
|
||||
$\bm{\beta}$ and the design matrix $\bm{X}$ which embody our model,
|
||||
that is $\bm{\tilde{y}}=\bm{X}\bm{\beta}$.
|
||||
|
||||
The parameters $\bm{\beta}$ are in turn found by optimizing the mean
|
||||
squared error via the so-called cost function
|
||||
|
||||
!bt
|
||||
\[
|
||||
C(\bm{X},\bm{\beta}) =\frac{1}{n}\sum_{i=0}^{n-1}(y_i-\tilde{y}_i)^2=\mathbb{E}\left[(\bm{y}-\bm{\tilde{y}})^2\right].
|
||||
\]
|
||||
!et
|
||||
Here the expected value $\mathbb{E}$ is the sample value.
|
||||
|
||||
Show that you can rewrite this in terms of a term which contains the variance of the model itself (the so-called variance term), a
|
||||
term which measures the deviation from the true data and the mean value of the model (the bias term) and finally the variance of the noise.
|
||||
That is, show that
|
||||
!bt
|
||||
\[
|
||||
\mathbb{E}\left[(\bm{y}-\bm{\tilde{y}})^2\right]=\mathrm{Bias}[\tilde{y}]+\mathrm{var}[\tilde{y}]+\sigma^2,
|
||||
\]
|
||||
!et
|
||||
with
|
||||
!bt
|
||||
\[
|
||||
\mathrm{Bias}[\tilde{y}]=\mathbb{E}\left[\left(\bm{y}-\mathbb{E}\left[\bm{\tilde{y}}\right]\right)^2\right],
|
||||
\]
|
||||
!et
|
||||
and
|
||||
!bt
|
||||
\[
|
||||
\mathrm{var}[\tilde{y}]=\mathbb{E}\left[\left(\tilde{\bm{y}}-\mathbb{E}\left[\bm{\tilde{y}}\right]\right)^2\right]=\frac{1}{n}\sum_i(\tilde{y}_i-\mathbb{E}\left[\bm{\tilde{y}}\right])^2.
|
||||
\]
|
||||
!et
|
||||
Similarly, an example of an earlier project can be found at URL:"https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ReportExample/ReportSample.pdf"
|
||||
|
||||
|
||||
Your task this week is to
|
||||
o Write an abstract for your project
|
||||
o Write an introduction
|
||||
o Include references
|
||||
|
||||
Explain what the terms mean and discuss their interpretations.
|
||||
|
||||
Perform then a bias-variance analysis of a simple one-dimensional (or other models of your choice) function by
|
||||
studying the MSE value as function of the complexity of your model. Use ordinary least squares only.
|
||||
|
||||
Discuss the bias and variance trade-off as function
|
||||
of your model complexity (the degree of the polynomial) and the number
|
||||
of data points, and possibly also your training and test data using the _bootstrap_ resampling method.
|
||||
You can follow the code example in the jupyter-book at URL:"https://compphysics.github.io/MachineLearning/doc/LectureNotes/_build/html/chapter3.html#the-bias-variance-tradeoff".
|
||||
|
||||
|
||||
See also the whiteboard notes from week 37 at URL:"https://github.com/CompPhysics/MachineLearning/blob/master/doc/HandWrittenNotes/2023/NotesSep14.pdf"
|
||||
|
||||
Ashort feedback to the this exercise will be available after the deadline. And you can reuse these elements in your final report.
|
||||
|
||||
|
||||
|
||||
@@ -0,0 +1,59 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "f35930ac",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"<!-- HTML file automatically generated from DocOnce source (https://github.com/doconce/doconce/)\n",
|
||||
"doconce format html exercisesweek39.do.txt -->\n",
|
||||
"<!-- dom:TITLE: Exercises week 39 -->"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "8cb567a0",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"# Exercises week 39\n",
|
||||
"**September 25-29, 2023**\n",
|
||||
"\n",
|
||||
"Date: **Deadline is Sunday October 1 at midnight**"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "markdown",
|
||||
"id": "934324b3",
|
||||
"metadata": {
|
||||
"editable": true
|
||||
},
|
||||
"source": [
|
||||
"## Overarching aims of the exercises this week\n",
|
||||
"\n",
|
||||
"The aim of the exercises this week is to aid you in getting started\n",
|
||||
"with writing the report. This will be discussed during the lab\n",
|
||||
"sessions as well. One of the lab sessions will be recorded.\n",
|
||||
"\n",
|
||||
"A general guideline can be found at <https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/EvaluationGrading/EvaluationForm.md>.\n",
|
||||
"\n",
|
||||
"Similarly, an example of an earlier project can be found at <https://github.com/CompPhysics/MachineLearning/blob/master/doc/Projects/ReportExample/ReportSample.pdf>\n",
|
||||
"\n",
|
||||
"Your task this week is to\n",
|
||||
"1. Write an abstract for your project\n",
|
||||
"\n",
|
||||
"2. Write an introduction\n",
|
||||
"\n",
|
||||
"3. Include references\n",
|
||||
"\n",
|
||||
"Ashort feedback to the this exercise will be available after the deadline. And you can reuse these elements in your final report."
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 5
|
||||
}
|
||||
Reference in New Issue
Block a user