update book chapters
This commit is contained in:
Morten Hjorth-Jensen
@@ -645,10 +645,10 @@ number <span class="math notranslate nohighlight">\(i\)</span> is left out. Usin
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Runtime: 0.135976 sec
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Runtime: 0.138685 sec
|
||||
Jackknife Statistics :
|
||||
original bias std. error
|
||||
99.655 99.645 0.148675
|
||||
99.8911 99.8811 0.149732
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -867,7 +867,7 @@ theorem.</p>
|
||||
<div class="cell_output docutils container">
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Bootstrap Statistics :
|
||||
original bias std. error
|
||||
100.098 15.248 100.098 0.153966
|
||||
99.751 15.2137 99.7503 0.153129
|
||||
</pre></div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1092,19 +1092,22 @@ Error: 0.06844519414009442
|
||||
Bias^2: 0.06453579006728317
|
||||
Var: 0.003909404072811237
|
||||
0.06844519414009442 >= 0.06453579006728317 + 0.003909404072811237 = 0.06844519414009441
|
||||
Polynomial degree: 5
|
||||
Polynomial degree:
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 5
|
||||
Error: 0.05227921801205707
|
||||
Bias^2: 0.048187277304303125
|
||||
Var: 0.004091940707753964
|
||||
0.05227921801205707 >= 0.048187277304303125 + 0.004091940707753964 = 0.05227921801205709
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Polynomial degree: 6
|
||||
Polynomial degree: 6
|
||||
Error: 0.03781367141738898
|
||||
Bias^2: 0.03365768507152761
|
||||
Var: 0.004155986345861379
|
||||
0.03781367141738898 >= 0.03365768507152761 + 0.004155986345861379 = 0.03781367141738899
|
||||
Polynomial degree: 7
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Polynomial degree: 7
|
||||
Error: 0.027609773491022498
|
||||
Bias^2: 0.02299949826036597
|
||||
Var: 0.004610275230656537
|
||||
@@ -1119,14 +1122,14 @@ Error: 0.026605727637189085
|
||||
Bias^2: 0.010018312644140933
|
||||
Var: 0.016587414993048166
|
||||
0.026605727637189085 >= 0.010018312644140933 + 0.016587414993048166 = 0.0266057276371891
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Polynomial degree: 10
|
||||
Polynomial degree: 10
|
||||
Error: 0.021592704588043153
|
||||
Bias^2: 0.010516485576652981
|
||||
Var: 0.011076219011390184
|
||||
0.021592704588043153 >= 0.010516485576652981 + 0.011076219011390184 = 0.021592704588043167
|
||||
Polynomial degree: 11
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Polynomial degree: 11
|
||||
Error: 0.07160048164228314
|
||||
Bias^2: 0.01443680008897583
|
||||
Var: 0.0571636815533073
|
||||
@@ -1143,7 +1146,7 @@ Var: 0.20867052175003387
|
||||
0.22842468702166951 >= 0.01975416527163567 + 0.20867052175003387 = 0.22842468702166954
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/chapter3_62_4.png" src="_images/chapter3_62_4.png" />
|
||||
<img alt="_images/chapter3_62_5.png" src="_images/chapter3_62_5.png" />
|
||||
</div>
|
||||
</div>
|
||||
<p>The bias-variance tradeoff summarizes the fundamental tension in
|
||||
@@ -1378,12 +1381,12 @@ Mean squared error on test data: 8.14812206
|
||||
Degree of polynomial: 7
|
||||
Mean squared error on training data: 0.47075725
|
||||
Mean squared error on test data: 2.00607783
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 8
|
||||
Degree of polynomial: 8
|
||||
Mean squared error on training data: 0.04912436
|
||||
Mean squared error on test data: 0.21596432
|
||||
Degree of polynomial: 9
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 9
|
||||
Mean squared error on training data: 0.02522069
|
||||
Mean squared error on test data: 0.08576932
|
||||
Degree of polynomial: 10
|
||||
@@ -1392,56 +1395,54 @@ Mean squared error on test data: 1.20015436
|
||||
Degree of polynomial: 11
|
||||
Mean squared error on training data: 0.01640891
|
||||
Mean squared error on test data: 1.35533774
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 12
|
||||
Degree of polynomial: 12
|
||||
Mean squared error on training data: 0.00813803
|
||||
Mean squared error on test data: 0.17446471
|
||||
Degree of polynomial: 13
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 13
|
||||
Mean squared error on training data: 0.00759119
|
||||
Mean squared error on test data: 1.08131001
|
||||
Degree of polynomial: 14
|
||||
Mean squared error on training data: 0.00472199
|
||||
Mean squared error on test data: 0.81333802
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 15
|
||||
Degree of polynomial: 15
|
||||
Mean squared error on training data: 0.00410478
|
||||
Mean squared error on test data: 92.09160813
|
||||
Degree of polynomial: 16
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 16
|
||||
Mean squared error on training data: 0.00315593
|
||||
Mean squared error on test data: 234.40530431
|
||||
Degree of polynomial: 17
|
||||
Mean squared error on training data: 0.00242999
|
||||
Mean squared error on test data: 1270.94936405
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 18
|
||||
Degree of polynomial: 18
|
||||
Mean squared error on training data: 0.00228741
|
||||
Mean squared error on test data: 108.11945731
|
||||
Degree of polynomial: 19
|
||||
Mean squared error on training data: 0.00156372
|
||||
Mean squared error on test data: 1376.61081005
|
||||
Degree of polynomial: 20
|
||||
Mean squared error on training data: 0.00137945
|
||||
Mean squared error on test data: 1931.97211078
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 21
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 20
|
||||
Mean squared error on training data: 0.00137945
|
||||
Mean squared error on test data: 1931.97211078
|
||||
Degree of polynomial: 21
|
||||
Mean squared error on training data: 0.00118678
|
||||
Mean squared error on test data: 14496.70992192
|
||||
Degree of polynomial: 22
|
||||
Mean squared error on training data: 0.00092686
|
||||
Mean squared error on test data: 873.95463048
|
||||
Degree of polynomial: 23
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 23
|
||||
Mean squared error on training data: 0.00085890
|
||||
Mean squared error on test data: 5535.20053452
|
||||
Degree of polynomial: 24
|
||||
Mean squared error on training data: 0.00084714
|
||||
Mean squared error on test data: 1289.22422186
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 25
|
||||
Degree of polynomial: 25
|
||||
Mean squared error on training data: 0.00079022
|
||||
Mean squared error on test data: 136582.88824397
|
||||
Degree of polynomial: 26
|
||||
@@ -1466,7 +1467,7 @@ Mean squared error on test data: 3204.07047448
|
||||
plt.plot(polynomial, np.log10(testerror), label='Test Error')
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/chapter3_65_10.png" src="_images/chapter3_65_10.png" />
|
||||
<img alt="_images/chapter3_65_9.png" src="_images/chapter3_65_9.png" />
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1729,7 +1730,7 @@ column from the corresponding column and dividing the column with its
|
||||
standard deviation. Most machine learning libraries do this as a default. This means that if you compare your code with the results from a given library,
|
||||
the results may differ.</p>
|
||||
<p>The
|
||||
<a class="reference external" href="https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.StandardScaler.html">Standadscaler</a>
|
||||
<a class="reference external" href="https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.StandardScaler.html">Standardscaler</a>
|
||||
function in <strong>Scikit-Learn</strong> does this for us. For the data sets we
|
||||
have been studying in our various examples, the data are in many cases
|
||||
already scaled and there is no need to scale them. You as a user of different machine learning algorithms, should always perform a
|
||||
@@ -1781,11 +1782,13 @@ simplicity, we will focus on ordinary regression, as done in the above example.
|
||||
\[
|
||||
C(\beta_0, \beta_1, ... , \beta_{p-1}) = \frac{1}{n}\sum_{i=0}^{n} \left(y_i - \beta_0 - \sum_{j=1}^{p-1} X_{ij}\beta_j\right)^2,.
|
||||
\]</div>
|
||||
<p>Recall also that we use the squared value since this leads to an increase of the penalty for higher differences between predicted and output/target values.</p>
|
||||
<p>What we have done is to single out the <span class="math notranslate nohighlight">\(\beta_0\)</span> term in the definition of the mean squared error (MSE).
|
||||
The design matrix
|
||||
<span class="math notranslate nohighlight">\(X\)</span> does in this case not contain any intercept column.
|
||||
When we take the derivative with respect to <span class="math notranslate nohighlight">\(\beta_0\)</span>, we want the derivative to obey</p>
|
||||
<p>Recall also that we use the squared value. This expression can lead to an
|
||||
increased penalty for higher differences between predicted and
|
||||
output/target values.</p>
|
||||
<p>What we have done is to single out the <span class="math notranslate nohighlight">\(\beta_0\)</span> term in the
|
||||
definition of the mean squared error (MSE). The design matrix <span class="math notranslate nohighlight">\(X\)</span>
|
||||
does in this case not contain any intercept column. When we take the
|
||||
derivative with respect to <span class="math notranslate nohighlight">\(\beta_0\)</span>, we want the derivative to obey</p>
|
||||
<div class="math notranslate nohighlight">
|
||||
\[
|
||||
\frac{\partial C}{\partial \beta_j} = 0,
|
||||
@@ -1800,7 +1803,7 @@ When we take the derivative with respect to <span class="math notranslate nohigh
|
||||
\[
|
||||
\sum_{i=0}^{n-1} \beta_0 = \sum_{i=0}^{n-1}y_i - \sum_{i=0}^{n-1} \sum_{j=1}^{p-1} X_{ij} \beta_j.
|
||||
\]</div>
|
||||
<p>Let us special first to the case where we have only two parameters <span class="math notranslate nohighlight">\(\beta_0\)</span> and <span class="math notranslate nohighlight">\(\beta_1\)</span>.
|
||||
<p>Let us specialize first to the case where we have only two parameters <span class="math notranslate nohighlight">\(\beta_0\)</span> and <span class="math notranslate nohighlight">\(\beta_1\)</span>.
|
||||
Our result for <span class="math notranslate nohighlight">\(\beta_0\)</span> simplifies then to</p>
|
||||
<div class="math notranslate nohighlight">
|
||||
\[
|
||||
@@ -1814,9 +1817,9 @@ n\beta_0 = \sum_{i=0}^{n-1}y_i - \sum_{i=0}^{n-1} X_{i1} \beta_1.
|
||||
<p>If we define</p>
|
||||
<div class="math notranslate nohighlight">
|
||||
\[
|
||||
\mu_1=\frac{1}{n}\sum_{i=0}^{n-1} (X_{i1},
|
||||
\mu_{\boldsymbol{x}_1}=\frac{1}{n}\sum_{i=0}^{n-1} X_{i1},
|
||||
\]</div>
|
||||
<p>and if we define the mean value of the outputs as</p>
|
||||
<p>and the mean value of the outputs as</p>
|
||||
<div class="math notranslate nohighlight">
|
||||
\[
|
||||
\mu_y=\frac{1}{n}\sum_{i=0}^{n-1}y_i,
|
||||
@@ -1824,13 +1827,24 @@ n\beta_0 = \sum_{i=0}^{n-1}y_i - \sum_{i=0}^{n-1} X_{i1} \beta_1.
|
||||
<p>we have</p>
|
||||
<div class="math notranslate nohighlight">
|
||||
\[
|
||||
\beta_0 = \mu_y - \beta_1\mu_{1}.
|
||||
\beta_0 = \mu_y - \beta_1\mu_{\boldsymbol{x}_1}.
|
||||
\]</div>
|
||||
<p>In the general case withmore parameters than <span class="math notranslate nohighlight">\(\beta_0\)</span> and <span class="math notranslate nohighlight">\(\beta_1\)</span>, we have</p>
|
||||
<p>In the general case with more parameters than <span class="math notranslate nohighlight">\(\beta_0\)</span> and <span class="math notranslate nohighlight">\(\beta_1\)</span>, we have</p>
|
||||
<div class="math notranslate nohighlight">
|
||||
\[
|
||||
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1}y_i - \frac{1}{n}\sum_{i=0}^{n-1}\sum_{j=1}^{p-1} X_{ij}\beta_j.
|
||||
\]</div>
|
||||
<p>We can rewrite the latter equation as</p>
|
||||
<div class="math notranslate nohighlight">
|
||||
\[
|
||||
\beta_0 = \frac{1}{n}\sum_{i=0}^{n-1}y_i - \sum_{j=1}^{p-1} \mu_{\boldsymbol{x}_j}\beta_j,
|
||||
\]</div>
|
||||
<p>where we have defined</p>
|
||||
<div class="math notranslate nohighlight">
|
||||
\[
|
||||
\mu_{\boldsymbol{x}_j}=\frac{1}{n}\sum_{i=0}^{n-1} X_{ij},
|
||||
\]</div>
|
||||
<p>the mean value for all elements of the column vector <span class="math notranslate nohighlight">\(\boldsymbol{x}_j\)</span>.</p>
|
||||
<p>Replacing <span class="math notranslate nohighlight">\(y_i\)</span> with <span class="math notranslate nohighlight">\(y_i - y_i - \overline{\boldsymbol{y}}\)</span> and centering also our design matrix results in a cost function (in vector-matrix disguise)</p>
|
||||
<div class="math notranslate nohighlight">
|
||||
\[
|
||||
@@ -1960,7 +1974,7 @@ MSE with Sklearn intercept
|
||||
0.004113634617443135
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/chapter3_103_1.png" src="_images/chapter3_103_1.png" />
|
||||
<img alt="_images/chapter3_107_1.png" src="_images/chapter3_107_1.png" />
|
||||
</div>
|
||||
</div>
|
||||
<p>The intercept is the value of our output/target variable
|
||||
@@ -2155,7 +2169,7 @@ MSE values for Scikit-Learn Ridge implementation
|
||||
0.26409315307910036
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/chapter3_111_1.png" src="_images/chapter3_111_1.png" />
|
||||
<img alt="_images/chapter3_115_1.png" src="_images/chapter3_115_1.png" />
|
||||
</div>
|
||||
</div>
|
||||
<p>The results here agree when we force <strong>Scikit-Learn</strong>’s Ridge function to include the first column in our design matrix.
|
||||
@@ -2358,7 +2372,7 @@ MSE values for Scikit-Learn Ridge implementation
|
||||
0.002381316302584886
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/chapter3_113_1.png" src="_images/chapter3_113_1.png" />
|
||||
<img alt="_images/chapter3_117_1.png" src="_images/chapter3_117_1.png" />
|
||||
</div>
|
||||
</div>
|
||||
<p>We see here, when compared to the code which includes explicitely the
|
||||
@@ -2576,7 +2590,7 @@ linear system as an equation would reduce this down to
|
||||
cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/chapter3_144_1.png" src="_images/chapter3_144_1.png" />
|
||||
<img alt="_images/chapter3_148_1.png" src="_images/chapter3_148_1.png" />
|
||||
</div>
|
||||
</div>
|
||||
<p>It is interesting to note that OLS
|
||||
@@ -2720,7 +2734,7 @@ with the form utilized in linear regression, viz.</p>
|
||||
cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/chapter3_162_1.png" src="_images/chapter3_162_1.png" />
|
||||
<img alt="_images/chapter3_166_1.png" src="_images/chapter3_166_1.png" />
|
||||
</div>
|
||||
</div>
|
||||
<p>The results agree perfectly with our previous discussion where we used our own code.</p>
|
||||
@@ -2729,8 +2743,8 @@ regression. In ridge regression we include a <strong>regularizer</strong>. This
|
||||
involves a new cost function which leads to a new estimate for the
|
||||
weights <span class="math notranslate nohighlight">\(\boldsymbol{\beta}\)</span>. This results in a penalized regression problem. The
|
||||
cost function is given by</p>
|
||||
<p>5
|
||||
8</p>
|
||||
<p>6
|
||||
0</p>
|
||||
<p><
|
||||
<
|
||||
<
|
||||
@@ -2768,7 +2782,7 @@ K</p>
|
||||
cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/chapter3_165_1.png" src="_images/chapter3_165_1.png" />
|
||||
<img alt="_images/chapter3_169_1.png" src="_images/chapter3_169_1.png" />
|
||||
</div>
|
||||
</div>
|
||||
<p>In the <strong>Least Absolute Shrinkage and Selection Operator</strong> (LASSO)-method we get a third cost function.</p>
|
||||
@@ -2803,7 +2817,7 @@ K</p>
|
||||
cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/chapter3_169_1.png" src="_images/chapter3_169_1.png" />
|
||||
<img alt="_images/chapter3_173_1.png" src="_images/chapter3_173_1.png" />
|
||||
</div>
|
||||
</div>
|
||||
<p>It is quite striking how LASSO breaks the symmetry of the coupling
|
||||
@@ -2861,40 +2875,40 @@ constant as opposed to ridge and OLS. We get a sparse solution with
|
||||
10%|█ | 1/10 [00:00<00:04, 2.02it/s]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 20%|██ | 2/10 [00:00<00:03, 2.25it/s]
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 20%|██ | 2/10 [00:00<00:03, 2.36it/s]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 30%|███ | 3/10 [00:00<00:02, 2.82it/s]
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 30%|███ | 3/10 [00:00<00:02, 2.91it/s]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 40%|████ | 4/10 [00:01<00:01, 3.33it/s]
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 40%|████ | 4/10 [00:01<00:01, 3.60it/s]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 50%|█████ | 5/10 [00:01<00:01, 3.94it/s]
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 50%|█████ | 5/10 [00:01<00:01, 4.36it/s]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 60%|██████ | 6/10 [00:01<00:00, 4.61it/s]
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 60%|██████ | 6/10 [00:01<00:00, 4.87it/s]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 70%|███████ | 7/10 [00:01<00:00, 5.17it/s]
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 70%|███████ | 7/10 [00:01<00:00, 5.64it/s]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 80%|████████ | 8/10 [00:01<00:00, 5.73it/s]
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 80%|████████ | 8/10 [00:01<00:00, 6.31it/s]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 90%|█████████ | 9/10 [00:01<00:00, 6.46it/s]
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 90%|█████████ | 9/10 [00:01<00:00, 6.88it/s]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>100%|██████████| 10/10 [00:01<00:00, 6.87it/s]
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>100%|██████████| 10/10 [00:01<00:00, 7.10it/s]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>100%|██████████| 10/10 [00:01<00:00, 5.22it/s]
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>100%|██████████| 10/10 [00:01<00:00, 5.64it/s]
|
||||
</pre></div>
|
||||
</div>
|
||||
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>
|
||||
</pre></div>
|
||||
</div>
|
||||
<img alt="_images/chapter3_171_13.png" src="_images/chapter3_171_13.png" />
|
||||
<img alt="_images/chapter3_175_13.png" src="_images/chapter3_175_13.png" />
|
||||
</div>
|
||||
</div>
|
||||
<p>We see that LASSO reaches a good solution for low
|
||||
@@ -2943,7 +2957,7 @@ testing set that is close to the accuracy of the training set.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<img alt="_images/chapter3_173_0.png" src="_images/chapter3_173_0.png" />
|
||||
<img alt="_images/chapter3_177_0.png" src="_images/chapter3_177_0.png" />
|
||||
</div>
|
||||
</div>
|
||||
<p>From the above figure we can see that LASSO with <span class="math notranslate nohighlight">\(\lambda = 10^{-2}\)</span>
|
||||
@@ -3035,7 +3049,7 @@ which polynomial fits the data best.</p>
|
||||
</div>
|
||||
</div>
|
||||
<div class="cell_output docutils container">
|
||||
<img alt="_images/chapter3_177_0.png" src="_images/chapter3_177_0.png" />
|
||||
<img alt="_images/chapter3_181_0.png" src="_images/chapter3_181_0.png" />
|
||||
</div>
|
||||
</div>
|
||||
<div class="section" id="exercise-ordinary-least-square-ols-on-the-franke-function">
|
||||
|
||||
Reference in New Issue
Block a user