Revised first two weeks from feedback
This commit is contained in:
KarlHenrik
File diff suppressed because one or more lines are too long
+17
-35
@@ -224,12 +224,16 @@
|
||||
"source": [
|
||||
"With the expression for $\\boldsymbol{\\hat{\\beta}_{OLS}}$, you now have what you need to implement OLS regression with your input data and target data $\\boldsymbol{y}$. But before you can do that, you need to set up you input data as a feature matrix $\\boldsymbol{X}$.\n",
|
||||
"\n",
|
||||
"In a feature matrix, each row is a datapoint and each column is a feature of that data. If you want to predict someones spending based on their income and number of children, for instance, you would create a row for each person in your dataset, and in each column put a 1 for the intercept, the montly income and the number of children."
|
||||
"In a feature matrix, each row is a datapoint and each column is a feature of that data. If you want to predict someones spending based on their income and number of children, for instance, you would create a row for each person in your dataset, with the montly income and the number of children as columns.\n",
|
||||
"\n",
|
||||
"We typically also include an intercept in our models. The intercept is a value that is added to our prediction regardless of the value of the other features. The intercept tries to account for constant effects in our data that are not dependant on anything else. In our current example, the intercept could account for living expenses which are typical regardless of income or childcare expenses.\n",
|
||||
"\n",
|
||||
"We calculate the optimal intercept by including a feature with the constant value of 1 in our model, which is then multplied by some parameter $\\beta_0$ from the OLS method into the optimal intercept value (which will be $\\beta_0$). In practice, we include the intercept in our model by adding a column of ones to the start of our feature matrix."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 59,
|
||||
"execution_count": null,
|
||||
"id": "e5ff2a69",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
@@ -239,7 +243,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 60,
|
||||
"execution_count": null,
|
||||
"id": "a3cf2792",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
@@ -260,7 +264,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 65,
|
||||
"execution_count": null,
|
||||
"id": "5ad87a65",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
@@ -281,7 +285,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 37,
|
||||
"execution_count": null,
|
||||
"id": "8f3f68aa",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
@@ -312,7 +316,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 38,
|
||||
"execution_count": null,
|
||||
"id": "d7476c84",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
@@ -332,7 +336,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 66,
|
||||
"execution_count": null,
|
||||
"id": "91496e40",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
@@ -358,7 +362,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 67,
|
||||
"execution_count": null,
|
||||
"id": "034f502c",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
@@ -376,7 +380,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 68,
|
||||
"execution_count": null,
|
||||
"id": "29171358",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
@@ -396,21 +400,10 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 42,
|
||||
"execution_count": null,
|
||||
"id": "1e346f4c",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"Ellipsis"
|
||||
]
|
||||
},
|
||||
"execution_count": 42,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"..."
|
||||
]
|
||||
@@ -425,21 +418,10 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 43,
|
||||
"execution_count": null,
|
||||
"id": "ceb57457",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"Ellipsis"
|
||||
]
|
||||
},
|
||||
"execution_count": 43,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"..."
|
||||
]
|
||||
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
@@ -446,7 +446,7 @@ document.write(`
|
||||
<h2>Exercise 3 - Setting up a Python virtual environment<a class="headerlink" href="#exercise-3-setting-up-a-python-virtual-environment" title="Link to this heading">#</a></h2>
|
||||
<p>Following the theme of the previous exercises, another way of improving the reproducibility of your results and shareability of your code is having a good handle on which python packages you are using.</p>
|
||||
<p>There are many ways to manage your packages in Python, and you are free to use any approach you want, but in this course we encourage you to use something called a virtual environment. A virtual environemnt is a folder in your project which contains a Python runtime executable as well as all the packages you are using in the current project. In this way, each of your projects has its required set of packages installed in the same folder, so that if anything goes wrong while managing your packages it only affects the one project, and if multiple projects require different versions of the same package, you don’t need to worry about messing up old projects. Also, it’s easy to just delete the folder and start over if anything goes wrong.</p>
|
||||
<p>Virtual environments are typically created, activated, managed and updated using terminal commands, but for now we recommend that you let VS Code handle it for you to make the coding experience much easier.</p>
|
||||
<p>Virtual environments are typically created, activated, managed and updated using terminal commands, but for now we recommend that you let VS Code handle it for you to make the coding experience much easier. If you are familiar with another approach for virtual environments that works for you, feel free to keep doing it that way.</p>
|
||||
<p><strong>a)</strong> Open this notebook in VS Code (<a class="reference external" href="https://code.visualstudio.com/Download">https://code.visualstudio.com/Download</a>). Download the Python and Jupyter extensions.</p>
|
||||
<p><strong>b)</strong> Press ´Cmd + Shift + P´, then search and run ´Python: Create Environment…´</p>
|
||||
<p><strong>c)</strong> Select ´Venv´</p>
|
||||
@@ -483,10 +483,12 @@ document.write(`
|
||||
|
||||
<span class="n">line_model</span> <span class="o">=</span> <span class="n">LinearRegression</span><span class="p">()</span><span class="o">.</span><span class="n">fit</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">)</span>
|
||||
<span class="n">line_predict</span> <span class="o">=</span> <span class="n">line_model</span><span class="o">.</span><span class="n">predict</span><span class="p">(</span><span class="n">x</span><span class="p">)</span>
|
||||
<span class="c1">#line_mse = ...</span>
|
||||
|
||||
<span class="c1">#poly_features = ...</span>
|
||||
<span class="c1">#poly_model = LinearRegression().fit(..., y)</span>
|
||||
<span class="c1">#poly_predict = ...</span>
|
||||
<span class="c1">#poly_mse = ...</span>
|
||||
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">scatter</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">y</span><span class="p">,</span> <span class="n">label</span> <span class="o">=</span> <span class="s2">"Data"</span><span class="p">)</span>
|
||||
<span class="n">plt</span><span class="o">.</span><span class="n">scatter</span><span class="p">(</span><span class="n">x</span><span class="p">,</span> <span class="n">line_predict</span><span class="p">,</span> <span class="n">label</span> <span class="o">=</span> <span class="s2">"Line model"</span><span class="p">)</span>
|
||||
@@ -496,7 +498,7 @@ document.write(`
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<img alt="_images/fbd74da3fcd21613c4c425622594644bbfe45b6a7fd50ac62da4e3136a737128.png" src="_images/fbd74da3fcd21613c4c425622594644bbfe45b6a7fd50ac62da4e3136a737128.png" />
|
||||
<img alt="_images/c62786c19b580c6638248aa3cfe7dea30bdbc00a922e7e45a45a5a7a053bdb38.png" src="_images/c62786c19b580c6638248aa3cfe7dea30bdbc00a922e7e45a45a5a7a053bdb38.png" />
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
@@ -497,7 +497,9 @@ f_i =\sum_{j=0}^{n-1}a_{ij}x_j,
|
||||
<section id="exercise-3-creating-feature-matrix-and-implementing-ols-using-the-analytical-expression">
|
||||
<h2>Exercise 3 - Creating feature matrix and implementing OLS using the analytical expression<a class="headerlink" href="#exercise-3-creating-feature-matrix-and-implementing-ols-using-the-analytical-expression" title="Link to this heading">#</a></h2>
|
||||
<p>With the expression for <span class="math notranslate nohighlight">\(\boldsymbol{\hat{\beta}_{OLS}}\)</span>, you now have what you need to implement OLS regression with your input data and target data <span class="math notranslate nohighlight">\(\boldsymbol{y}\)</span>. But before you can do that, you need to set up you input data as a feature matrix <span class="math notranslate nohighlight">\(\boldsymbol{X}\)</span>.</p>
|
||||
<p>In a feature matrix, each row is a datapoint and each column is a feature of that data. If you want to predict someones spending based on their income and number of children, for instance, you would create a row for each person in your dataset, and in each column put a 1 for the intercept, the montly income and the number of children.</p>
|
||||
<p>In a feature matrix, each row is a datapoint and each column is a feature of that data. If you want to predict someones spending based on their income and number of children, for instance, you would create a row for each person in your dataset, with the montly income and the number of children as columns.</p>
|
||||
<p>We typically also include an intercept in our models. The intercept is a value that is added to our prediction regardless of the value of the other features. The intercept tries to account for constant effects in our data that are not dependant on anything else. In our current example, the intercept could account for living expenses which are typical regardless of income or childcare expenses.</p>
|
||||
<p>We calculate the optimal intercept by including a feature with the constant value of 1 in our model, which is then multplied by some parameter <span class="math notranslate nohighlight">\(\beta_0\)</span> from the OLS method into the optimal intercept value (which will be <span class="math notranslate nohighlight">\(\beta_0\)</span>). In practice, we include the intercept in our model by adding a column of ones to the start of our feature matrix.</p>
|
||||
<div class="cell docutils container">
|
||||
<div class="cell_input docutils container">
|
||||
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span><span class="w"> </span><span class="nn">numpy</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">np</span>
|
||||
@@ -592,11 +594,6 @@ f_i =\sum_{j=0}^{n-1}a_{ij}x_j,
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Ellipsis
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<p><strong>e)</strong> Do the same for each polynomial degree from 2 to 10, and plot the MSE on both the training and test data as a function of polynomial degree. The aim is to reproduce Figure 2.11 of <a class="reference external" href="https://github.com/CompPhysics/MLErasmus/blob/master/doc/Textbooks/elementsstat.pdf">Hastie et al</a>. Feel free to read the discussions leading to figure 2.11 of Hastie et al.</p>
|
||||
<div class="cell docutils container">
|
||||
@@ -605,11 +602,6 @@ f_i =\sum_{j=0}^{n-1}a_{ij}x_j,
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Ellipsis
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
<p><strong>f)</strong> Interpret the graph. Why do the lines move as they do? What does it tell us about model performance and generalizability?</p>
|
||||
</section>
|
||||
|
||||
BIN
Binary file not shown.
|
After Width: | Height: | Size: 18 KiB |
BIN
Binary file not shown.
|
Before Width: | Height: | Size: 18 KiB |
File diff suppressed because one or more lines are too long
@@ -224,12 +224,16 @@
|
||||
"source": [
|
||||
"With the expression for $\\boldsymbol{\\hat{\\beta}_{OLS}}$, you now have what you need to implement OLS regression with your input data and target data $\\boldsymbol{y}$. But before you can do that, you need to set up you input data as a feature matrix $\\boldsymbol{X}$.\n",
|
||||
"\n",
|
||||
"In a feature matrix, each row is a datapoint and each column is a feature of that data. If you want to predict someones spending based on their income and number of children, for instance, you would create a row for each person in your dataset, and in each column put a 1 for the intercept, the montly income and the number of children."
|
||||
"In a feature matrix, each row is a datapoint and each column is a feature of that data. If you want to predict someones spending based on their income and number of children, for instance, you would create a row for each person in your dataset, with the montly income and the number of children as columns.\n",
|
||||
"\n",
|
||||
"We typically also include an intercept in our models. The intercept is a value that is added to our prediction regardless of the value of the other features. The intercept tries to account for constant effects in our data that are not dependant on anything else. In our current example, the intercept could account for living expenses which are typical regardless of income or childcare expenses.\n",
|
||||
"\n",
|
||||
"We calculate the optimal intercept by including a feature with the constant value of 1 in our model, which is then multplied by some parameter $\\beta_0$ from the OLS method into the optimal intercept value (which will be $\\beta_0$). In practice, we include the intercept in our model by adding a column of ones to the start of our feature matrix."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 59,
|
||||
"execution_count": null,
|
||||
"id": "e5ff2a69",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
@@ -239,7 +243,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 60,
|
||||
"execution_count": null,
|
||||
"id": "a3cf2792",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
@@ -260,7 +264,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 65,
|
||||
"execution_count": null,
|
||||
"id": "5ad87a65",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
@@ -281,7 +285,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 37,
|
||||
"execution_count": null,
|
||||
"id": "8f3f68aa",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
@@ -312,7 +316,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 38,
|
||||
"execution_count": null,
|
||||
"id": "d7476c84",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
@@ -332,7 +336,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 66,
|
||||
"execution_count": null,
|
||||
"id": "91496e40",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
@@ -358,7 +362,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 67,
|
||||
"execution_count": null,
|
||||
"id": "034f502c",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
@@ -376,7 +380,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 68,
|
||||
"execution_count": null,
|
||||
"id": "29171358",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
@@ -396,21 +400,10 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 42,
|
||||
"execution_count": null,
|
||||
"id": "1e346f4c",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"Ellipsis"
|
||||
]
|
||||
},
|
||||
"execution_count": 42,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"..."
|
||||
]
|
||||
@@ -425,21 +418,10 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 43,
|
||||
"execution_count": null,
|
||||
"id": "ceb57457",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"Ellipsis"
|
||||
]
|
||||
},
|
||||
"execution_count": 43,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"..."
|
||||
]
|
||||
|
||||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
@@ -224,12 +224,16 @@
|
||||
"source": [
|
||||
"With the expression for $\\boldsymbol{\\hat{\\beta}_{OLS}}$, you now have what you need to implement OLS regression with your input data and target data $\\boldsymbol{y}$. But before you can do that, you need to set up you input data as a feature matrix $\\boldsymbol{X}$.\n",
|
||||
"\n",
|
||||
"In a feature matrix, each row is a datapoint and each column is a feature of that data. If you want to predict someones spending based on their income and number of children, for instance, you would create a row for each person in your dataset, and in each column put a 1 for the intercept, the montly income and the number of children."
|
||||
"In a feature matrix, each row is a datapoint and each column is a feature of that data. If you want to predict someones spending based on their income and number of children, for instance, you would create a row for each person in your dataset, with the montly income and the number of children as columns.\n",
|
||||
"\n",
|
||||
"We typically also include an intercept in our models. The intercept is a value that is added to our prediction regardless of the value of the other features. The intercept tries to account for constant effects in our data that are not dependant on anything else. In our current example, the intercept could account for living expenses which are typical regardless of income or childcare expenses.\n",
|
||||
"\n",
|
||||
"We calculate the optimal intercept by including a feature with the constant value of 1 in our model, which is then multplied by some parameter $\\beta_0$ from the OLS method into the optimal intercept value (which will be $\\beta_0$). In practice, we include the intercept in our model by adding a column of ones to the start of our feature matrix."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 59,
|
||||
"execution_count": null,
|
||||
"id": "e5ff2a69",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
@@ -239,7 +243,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 60,
|
||||
"execution_count": null,
|
||||
"id": "a3cf2792",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
@@ -260,7 +264,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 65,
|
||||
"execution_count": null,
|
||||
"id": "5ad87a65",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
@@ -281,7 +285,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 37,
|
||||
"execution_count": null,
|
||||
"id": "8f3f68aa",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
@@ -312,7 +316,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 38,
|
||||
"execution_count": null,
|
||||
"id": "d7476c84",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
@@ -332,7 +336,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 66,
|
||||
"execution_count": null,
|
||||
"id": "91496e40",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
@@ -358,7 +362,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 67,
|
||||
"execution_count": null,
|
||||
"id": "034f502c",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
@@ -376,7 +380,7 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 68,
|
||||
"execution_count": null,
|
||||
"id": "29171358",
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
@@ -396,21 +400,10 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 42,
|
||||
"execution_count": null,
|
||||
"id": "1e346f4c",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"Ellipsis"
|
||||
]
|
||||
},
|
||||
"execution_count": 42,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"..."
|
||||
]
|
||||
@@ -425,21 +418,10 @@
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 43,
|
||||
"execution_count": null,
|
||||
"id": "ceb57457",
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"Ellipsis"
|
||||
]
|
||||
},
|
||||
"execution_count": 43,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"..."
|
||||
]
|
||||
|
||||
BIN
Binary file not shown.
|
After Width: | Height: | Size: 18 KiB |
BIN
Binary file not shown.
|
Before Width: | Height: | Size: 18 KiB |
Reference in New Issue
Block a user