lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

update

Browse Source
This commit is contained in:
Morten Hjorth-Jensen
2023-08-31 06:19:44 +02:00
parent e9bac09427
commit d05f791ff6
98 changed files with 3861 additions and 2496 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/LectureNotes/_build/html/chapter3.html
+71 -44
View File
@@ -242,6 +242,33 @@ const thebe_selector_output = ".output, .cell_output"
</a>
</li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
Weekly material, notes and exercises
</span>
</p>
<ul class="nav bd-sidenav">
<li class="toctree-l1">
<a class="reference internal" href="exercisesweek34.html">
Exercises week 34
</a>
</li>
<li class="toctree-l1">
<a class="reference internal" href="week34.html">
Week 34: Introduction to the course, Logistics and Practicalities
</a>
</li>
<li class="toctree-l1">
<a class="reference internal" href="exercisesweek35.html">
Exercises week 35
</a>
</li>
<li class="toctree-l1">
<a class="reference internal" href="week35.html">
Week 35: From Ordinary Linear Regression to Ridge and Lasso Regression
</a>
</li>
</ul>
</div>
</nav> <!-- To handle the deprecated key -->
@@ -760,10 +787,10 @@ number <span class="math notranslate nohighlight">\(i\)</span> is left out. Usin
</div>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Runtime: 0.0893679 sec
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Runtime: 0.0907831 sec
Jackknife Statistics :
original bias std. error
99.9524 99.9424 0.148854
100.142 100.132 0.149864
</pre></div>
</div>
</div>
@@ -982,7 +1009,7 @@ theorem.</p>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Bootstrap Statistics :
original bias std. error
100.188 15.1133 100.19 0.149655
100.033 14.9292 100.032 0.149452
</pre></div>
</div>
</div>
@@ -1216,9 +1243,7 @@ Error: 0.03781367141738902
Bias^2: 0.03365768507152769
Var: 0.0041559863458613296
0.03781367141738902 &gt;= 0.03365768507152769 + 0.0041559863458613296 = 0.03781367141738902
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Polynomial degree: 7
Polynomial degree: 7
Error: 0.027609773491022394
Bias^2: 0.022999498260366198
Var: 0.004610275230656182
@@ -1238,7 +1263,9 @@ Error: 0.021592704588021178
Bias^2: 0.010516485576646504
Var: 0.01107621901137467
0.021592704588021178 &gt;= 0.010516485576646504 + 0.01107621901137467 = 0.021592704588021174
Polynomial degree: 11
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Polynomial degree: 11
Error: 0.07160048164232538
Bias^2: 0.014436800088896381
Var: 0.05716368155342902
@@ -1508,12 +1535,12 @@ Mean squared error on test data: 0.17446471
Degree of polynomial: 13
Mean squared error on training data: 0.00759119
Mean squared error on test data: 1.08131003
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 14
Degree of polynomial: 14
Mean squared error on training data: 0.00472199
Mean squared error on test data: 0.81333804
Degree of polynomial: 15
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 15
Mean squared error on training data: 0.00410478
Mean squared error on test data: 92.09172409
Degree of polynomial: 16
@@ -1528,12 +1555,12 @@ Mean squared error on test data: 108.27092910
Degree of polynomial: 19
Mean squared error on training data: 0.00156376
Mean squared error on test data: 1371.99051150
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 20
Degree of polynomial: 20
Mean squared error on training data: 0.00137818
Mean squared error on test data: 1887.86252988
Degree of polynomial: 21
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 21
Mean squared error on training data: 0.00118508
Mean squared error on test data: 14859.69908626
Degree of polynomial: 22
@@ -1548,12 +1575,12 @@ Mean squared error on test data: 1277.61702282
Degree of polynomial: 25
Mean squared error on training data: 0.00079129
Mean squared error on test data: 128664.31650694
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 26
Degree of polynomial: 26
Mean squared error on training data: 0.00076905
Mean squared error on test data: 19003.94822514
Degree of polynomial: 27
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 27
Mean squared error on training data: 0.00068946
Mean squared error on test data: 2379.66219404
Degree of polynomial: 28
@@ -1564,9 +1591,9 @@ Mean squared error on training data: 0.00060705
Mean squared error on test data: 3250.17647619
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_41811/626635268.py:73: RuntimeWarning: divide by zero encountered in log10
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20614/626635268.py:73: RuntimeWarning: divide by zero encountered in log10
plt.plot(polynomial, np.log10(trainingerror), label=&#39;Training Error&#39;)
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_41811/626635268.py:74: RuntimeWarning: divide by zero encountered in log10
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20614/626635268.py:74: RuntimeWarning: divide by zero encountered in log10
plt.plot(polynomial, np.log10(testerror), label=&#39;Test Error&#39;)
</pre></div>
</div>
@@ -1800,7 +1827,7 @@ cross-validation (LOOCV).</p>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_41811/3817475779.py:63: RuntimeWarning: divide by zero encountered in log10
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20614/3817475779.py:63: RuntimeWarning: divide by zero encountered in log10
plt.plot(polynomial, np.log10(estimated_mse_sklearn), label=&#39;Test Error&#39;)
</pre></div>
</div>
@@ -2689,9 +2716,9 @@ linear system as an equation would reduce this down to
</div>
</div>
<div class="cell_output docutils container">
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_41811/4162706317.py:6: MatplotlibDeprecationWarning: Auto-removal of grids by pcolor() and pcolormesh() is deprecated since 3.5 and will be removed two minor releases later; please call grid(False) first.
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20614/4162706317.py:6: MatplotlibDeprecationWarning: Auto-removal of grids by pcolor() and pcolormesh() is deprecated since 3.5 and will be removed two minor releases later; please call grid(False) first.
cb = fig.colorbar(im)
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_41811/4162706317.py:7: UserWarning: FixedFormatter should only be used together with FixedLocator
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20614/4162706317.py:7: UserWarning: FixedFormatter should only be used together with FixedLocator
cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)
</pre></div>
</div>
@@ -2835,9 +2862,9 @@ with the form utilized in linear regression, viz.</p>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_41811/3777801602.py:6: MatplotlibDeprecationWarning: Auto-removal of grids by pcolor() and pcolormesh() is deprecated since 3.5 and will be removed two minor releases later; please call grid(False) first.
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20614/3777801602.py:6: MatplotlibDeprecationWarning: Auto-removal of grids by pcolor() and pcolormesh() is deprecated since 3.5 and will be removed two minor releases later; please call grid(False) first.
cb = fig.colorbar(im)
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_41811/3777801602.py:7: UserWarning: FixedFormatter should only be used together with FixedLocator
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20614/3777801602.py:7: UserWarning: FixedFormatter should only be used together with FixedLocator
cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)
</pre></div>
</div>
@@ -2877,9 +2904,9 @@ cost function is given by</p>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_41811/438060758.py:9: MatplotlibDeprecationWarning: Auto-removal of grids by pcolor() and pcolormesh() is deprecated since 3.5 and will be removed two minor releases later; please call grid(False) first.
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20614/438060758.py:9: MatplotlibDeprecationWarning: Auto-removal of grids by pcolor() and pcolormesh() is deprecated since 3.5 and will be removed two minor releases later; please call grid(False) first.
cb = fig.colorbar(im)
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_41811/438060758.py:10: UserWarning: FixedFormatter should only be used together with FixedLocator
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20614/438060758.py:10: UserWarning: FixedFormatter should only be used together with FixedLocator
cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)
</pre></div>
</div>
@@ -2914,9 +2941,9 @@ cost function is given by</p>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_41811/3544313922.py:8: MatplotlibDeprecationWarning: Auto-removal of grids by pcolor() and pcolormesh() is deprecated since 3.5 and will be removed two minor releases later; please call grid(False) first.
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20614/3544313922.py:8: MatplotlibDeprecationWarning: Auto-removal of grids by pcolor() and pcolormesh() is deprecated since 3.5 and will be removed two minor releases later; please call grid(False) first.
cb = fig.colorbar(im)
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_41811/3544313922.py:9: UserWarning: FixedFormatter should only be used together with FixedLocator
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20614/3544313922.py:9: UserWarning: FixedFormatter should only be used together with FixedLocator
cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)
</pre></div>
</div>
@@ -2969,43 +2996,43 @@ constant as opposed to ridge and OLS. We get a sparse solution with
</div>
</div>
<div class="cell_output docutils container">
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 0%| | 0/10 [00:00&lt;?, ?it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 0%| | 0/10 [00:00&lt;?, ?it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/linear_model/_coordinate_descent.py:647: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations, check the scale of the features or consider increasing regularisation. Duality gap: 3.924e+00, tolerance: 1.797e+00
model = cd_fast.enet_coordinate_descent(
10%|███████████▉ | 1/10 [00:00&lt;00:06, 1.50it/s]
10%|██████████████▋ | 1/10 [00:00&lt;00:05, 1.53it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 20%|███████████████████████▊ | 2/10 [00:01&lt;00:04, 1.83it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 20%|█████████████████████████████▍ | 2/10 [00:00&lt;00:03, 2.15it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 30%|███████████████████████████████████▋ | 3/10 [00:01&lt;00:02, 2.70it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 30%|████████████████████████████████████████████ | 3/10 [00:01&lt;00:02, 3.06it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 40%|███████████████████████████████████████████████▌ | 4/10 [00:01&lt;00:01, 3.15it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 40%|██████████████████████████████████████████████████████████▊ | 4/10 [00:01&lt;00:01, 3.41it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 50%|███████████████████████████████████████████████████████████▌ | 5/10 [00:01&lt;00:01, 3.60it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 50%|█████████████████████████████████████████████████████████████████████████▌ | 5/10 [00:01&lt;00:01, 4.28it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 60%|███████████████████████████████████████████████████████████████████████▍ | 6/10 [00:01&lt;00:01, 3.91it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 60%|████████████████████████████████████████████████████████████████████████████████████████▏ | 6/10 [00:01&lt;00:00, 5.00it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 70%|███████████████████████████████████████████████████████████████████████████████████▎ | 7/10 [00:02&lt;00:00, 4.65it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 70%|██████████████████████████████████████████████████████████████████████████████████████████████████████▉ | 7/10 [00:01&lt;00:00, 5.70it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 80%|███████████████████████████████████████████████████████████████████████████████████████████████▏ | 8/10 [00:02&lt;00:00, 5.13it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 80%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▌ | 8/10 [00:01&lt;00:00, 6.29it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 90%|███████████████████████████████████████████████████████████████████████████████████████████████████████████ | 9/10 [00:02&lt;00:00, 5.60it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 90%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▎ | 9/10 [00:02&lt;00:00, 6.85it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:02&lt;00:00, 5.96it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:02&lt;00:00, 7.30it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:02&lt;00:00, 3.97it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:02&lt;00:00, 4.68it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>
@@ -3152,9 +3179,9 @@ which polynomial fits the data best.</p>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_41811/3980313467.py:9: MatplotlibDeprecationWarning: Calling gca() with keyword arguments was deprecated in Matplotlib 3.4. Starting two minor releases later, gca() will take no keyword arguments. The gca() function should only be used to get the current axes, or if no axes exist, create new axes with default keyword arguments. To create a new axes with non-default arguments, use plt.axes() or plt.subplot().
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20614/3980313467.py:9: MatplotlibDeprecationWarning: Calling gca() with keyword arguments was deprecated in Matplotlib 3.4. Starting two minor releases later, gca() will take no keyword arguments. The gca() function should only be used to get the current axes, or if no axes exist, create new axes with default keyword arguments. To create a new axes with non-default arguments, use plt.axes() or plt.subplot().
ax = fig.gca(projection=&#39;3d&#39;)
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_41811/3980313467.py:37: MatplotlibDeprecationWarning: Auto-removal of grids by pcolor() and pcolormesh() is deprecated since 3.5 and will be removed two minor releases later; please call grid(False) first.
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20614/3980313467.py:37: MatplotlibDeprecationWarning: Auto-removal of grids by pcolor() and pcolormesh() is deprecated since 3.5 and will be removed two minor releases later; please call grid(False) first.
fig.colorbar(surf, shrink=0.5, aspect=5)
</pre></div>
</div>