Revised first two weeks from feedback

2025-08-13 12:54:12 +02:00
parent 3425395c86
commit f88eefcf7a
16 changed files with 89 additions and 140 deletions
@@ -497,7 +497,9 @@ f_i =\sum_{j=0}^{n-1}a_{ij}x_j,
 <section id="exercise-3-creating-feature-matrix-and-implementing-ols-using-the-analytical-expression">
 <h2>Exercise 3 - Creating feature matrix and implementing OLS using the analytical expression<a class="headerlink" href="#exercise-3-creating-feature-matrix-and-implementing-ols-using-the-analytical-expression" title="Link to this heading">#</a></h2>
 <p>With the expression for <span class="math notranslate nohighlight">\(\boldsymbol{\hat{\beta}_{OLS}}\)</span>, you now have what you need to implement OLS regression with your input data and target data <span class="math notranslate nohighlight">\(\boldsymbol{y}\)</span>. But before you can do that, you need to set up you input data as a feature matrix <span class="math notranslate nohighlight">\(\boldsymbol{X}\)</span>.</p>
-<p>In a feature matrix, each row is a datapoint and each column is a feature of that data. If you want to predict someones spending based on their income and number of children, for instance, you would create a row for each person in your dataset, and in each column put a 1 for the intercept, the montly income and the number of children.</p>
+<p>In a feature matrix, each row is a datapoint and each column is a feature of that data. If you want to predict someones spending based on their income and number of children, for instance, you would create a row for each person in your dataset, with the montly income and the number of children as columns.</p>
+<p>We typically also include an intercept in our models. The intercept is a value that is added to our prediction regardless of the value of the other features. The intercept tries to account for constant effects in our data that are not dependant on anything else. In our current example, the intercept could account for living expenses which are typical regardless of income or childcare expenses.</p>
+<p>We calculate the optimal intercept by including a feature with the constant value of 1 in our model, which is then multplied by some parameter <span class="math notranslate nohighlight">\(\beta_0\)</span> from the OLS method into the optimal intercept value (which will be <span class="math notranslate nohighlight">\(\beta_0\)</span>). In practice, we include the intercept in our model by adding a column of ones to the start of our feature matrix.</p>
 <div class="cell docutils container">
 <div class="cell_input docutils container">
 <div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span><span class="w"> </span><span class="nn">numpy</span><span class="w"> </span><span class="k">as</span><span class="w"> </span><span class="nn">np</span>
@@ -592,11 +594,6 @@ f_i =\sum_{j=0}^{n-1}a_{ij}x_j,
 </pre></div>
 </div>
 </div>
-<div class="cell_output docutils container">
-<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Ellipsis
-</pre></div>
-</div>
-</div>
 </div>
 <p><strong>e)</strong> Do the same for each polynomial degree from 2 to 10, and plot the MSE on both the training and test data as a function of polynomial degree. The aim is to reproduce Figure 2.11 of <a class="reference external" href="https://github.com/CompPhysics/MLErasmus/blob/master/doc/Textbooks/elementsstat.pdf">Hastie et al</a>. Feel free to read the discussions leading to figure 2.11 of Hastie et al.</p>
 <div class="cell docutils container">
@@ -605,11 +602,6 @@ f_i =\sum_{j=0}^{n-1}a_{ij}x_j,
 </pre></div>
 </div>
 </div>
-<div class="cell_output docutils container">
-<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Ellipsis
-</pre></div>
-</div>
-</div>
 </div>
 <p><strong>f)</strong> Interpret the graph. Why do the lines move as they do? What does it tell us about model performance and generalizability?</p>
 </section>