lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

update jupyter-book

Browse Source
This commit is contained in:
Morten Hjorth-Jensen
2024-11-03 14:50:52 +01:00
parent e9d6ea5784
commit 620d191340
137 changed files with 22442 additions and 2035 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/LectureNotes/_build/html/chapter3.html
+70 -56
View File
@@ -353,6 +353,11 @@ const thebe_selector_output = ".output, .cell_output"
Week 44, Convolutional Neural Networks (CNN)
</a>
</li>
<li class="toctree-l1">
<a class="reference internal" href="week45.html">
Week 45, Convolutional Neural Networks (CCNs) and Recurrent Neural Networks (RNNs)
</a>
</li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
@@ -889,10 +894,10 @@ number <span class="math notranslate nohighlight">\(i\)</span> is left out. Usin
</div>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Runtime: 0.156654 sec
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Runtime: 0.159617 sec
Jackknife Statistics :
original bias std. error
100.193 100.183 0.150667
99.9045 99.8945 0.150698
</pre></div>
</div>
</div>
@@ -1111,7 +1116,7 @@ theorem.</p>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Bootstrap Statistics :
original bias std. error
100.159 15.0085 100.157 0.150625
99.9457 15.0851 99.9458 0.149903
</pre></div>
</div>
</div>
@@ -1318,7 +1323,9 @@ Error: 0.08426840630693411
Bias^2: 0.0796891867672603
Var: 0.004579219539673834
0.08426840630693411 &gt;= 0.0796891867672603 + 0.004579219539673834 = 0.08426840630693413
Polynomial degree: 2
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Polynomial degree: 2
Error: 0.10398646080125035
Bias^2: 0.1007711427354898
Var: 0.0032153180657605116
@@ -1340,7 +1347,10 @@ Var: 0.004091940707753939
0.05227921801205686 &gt;= 0.0481872773043029 + 0.004091940707753939 = 0.052279218012056844
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Polynomial degree: 6
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Polynomial degree:
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 6
Error: 0.037813671417389005
Bias^2: 0.033657685071527665
Var: 0.00415598634586135
@@ -1360,14 +1370,14 @@ Error: 0.02660572763718093
Bias^2: 0.010018312644137363
Var: 0.016587414993043573
0.02660572763718093 &gt;= 0.010018312644137363 + 0.016587414993043573 = 0.026605727637180936
Polynomial degree: 10
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Polynomial degree: 10
Error: 0.021592704588025025
Bias^2: 0.010516485576645508
Var: 0.011076219011379514
0.021592704588025025 &gt;= 0.010516485576645508 + 0.011076219011379514 = 0.021592704588025022
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Polynomial degree: 11
Polynomial degree: 11
Error: 0.07160048164233104
Bias^2: 0.014436800088904942
Var: 0.05716368155342608
@@ -1386,7 +1396,7 @@ Var: 0.20867052175034223
0.22842468702219465 &gt;= 0.01975416527185249 + 0.20867052175034223 = 0.2284246870221947
</pre></div>
</div>
<img alt="_images/chapter3_66_4.png" src="_images/chapter3_66_4.png" />
<img alt="_images/chapter3_66_6.png" src="_images/chapter3_66_6.png" />
</div>
</div>
<p>The bias-variance tradeoff summarizes the fundamental tension in
@@ -1607,62 +1617,64 @@ Mean squared error on test data: 123711.53703498
Degree of polynomial: 3
Mean squared error on training data: 9011.85263220
Mean squared error on test data: 10913.84780262
Degree of polynomial: 4
Mean squared error on training data: 303.47610036
Mean squared error on test data: 426.30787294
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 5
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 4
Mean squared error on training data: 303.47610036
Mean squared error on test data: 426.30787294
Degree of polynomial: 5
Mean squared error on training data: 3.80354994
Mean squared error on test data: 5.98822371
Degree of polynomial: 6
Mean squared error on training data: 3.66204648
Mean squared error on test data: 8.14812206
Degree of polynomial: 7
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 7
Mean squared error on training data: 0.47075725
Mean squared error on test data: 2.00607783
Degree of polynomial: 8
Mean squared error on training data: 0.04912436
Mean squared error on test data: 0.21596432
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 9
Degree of polynomial: 9
Mean squared error on training data: 0.02522069
Mean squared error on test data: 0.08576932
Degree of polynomial: 10
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 10
Mean squared error on training data: 0.02511518
Mean squared error on test data: 1.20015436
Degree of polynomial: 11
Mean squared error on training data: 0.01640891
Mean squared error on test data: 1.35533773
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 12
Degree of polynomial: 12
Mean squared error on training data: 0.00813803
Mean squared error on test data: 0.17446471
Degree of polynomial: 13
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 13
Mean squared error on training data: 0.00759119
Mean squared error on test data: 1.08131003
Degree of polynomial: 14
Mean squared error on training data: 0.00472199
Mean squared error on test data: 0.81333808
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 15
Degree of polynomial: 15
Mean squared error on training data: 0.00410478
Mean squared error on test data: 92.09163947
Degree of polynomial: 16
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 16
Mean squared error on training data: 0.00315593
Mean squared error on test data: 234.38827994
Degree of polynomial: 17
Mean squared error on training data: 0.00242999
Mean squared error on test data: 1271.34367970
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 18
Degree of polynomial: 18
Mean squared error on training data: 0.00228740
Mean squared error on test data: 108.21093775
Degree of polynomial: 19
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 19
Mean squared error on training data: 0.00156374
Mean squared error on test data: 1385.79778008
Degree of polynomial: 20
@@ -1682,18 +1694,20 @@ Mean squared error on test data: 5567.04664255
Degree of polynomial: 24
Mean squared error on training data: 0.00084707
Mean squared error on test data: 1325.26124692
Degree of polynomial: 25
Mean squared error on training data: 0.00079125
Mean squared error on test data: 129012.83870189
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 26
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 25
Mean squared error on training data: 0.00079125
Mean squared error on test data: 129012.83870189
Degree of polynomial: 26
Mean squared error on training data: 0.00076908
Mean squared error on test data: 18388.59354079
Degree of polynomial: 27
Mean squared error on training data: 0.00069123
Mean squared error on test data: 2351.97979891
Degree of polynomial: 28
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 28
Mean squared error on training data: 0.00062592
Mean squared error on test data: 3983.63037846
Degree of polynomial: 29
@@ -1701,13 +1715,13 @@ Mean squared error on training data: 0.00060704
Mean squared error on test data: 3262.26814548
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20789/626635268.py:73: RuntimeWarning: divide by zero encountered in log10
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_76898/626635268.py:73: RuntimeWarning: divide by zero encountered in log10
plt.plot(polynomial, np.log10(trainingerror), label=&#39;Training Error&#39;)
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20789/626635268.py:74: RuntimeWarning: divide by zero encountered in log10
/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_76898/626635268.py:74: RuntimeWarning: divide by zero encountered in log10
plt.plot(polynomial, np.log10(testerror), label=&#39;Test Error&#39;)
</pre></div>
</div>
<img alt="_images/chapter3_69_9.png" src="_images/chapter3_69_9.png" />
<img alt="_images/chapter3_69_11.png" src="_images/chapter3_69_11.png" />
</div>
</div>
</div>
@@ -1937,7 +1951,7 @@ cross-validation (LOOCV).</p>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20789/3817475779.py:63: RuntimeWarning: divide by zero encountered in log10
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_76898/3817475779.py:63: RuntimeWarning: divide by zero encountered in log10
plt.plot(polynomial, np.log10(estimated_mse_sklearn), label=&#39;Test Error&#39;)
</pre></div>
</div>
@@ -2826,7 +2840,7 @@ linear system as an equation would reduce this down to
</div>
</div>
<div class="cell_output docutils container">
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20789/4162706317.py:7: UserWarning: set_ticklabels() should only be used with a fixed number of ticks, i.e. after set_ticks() or using a FixedLocator.
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_76898/4162706317.py:7: UserWarning: set_ticklabels() should only be used with a fixed number of ticks, i.e. after set_ticks() or using a FixedLocator.
cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)
</pre></div>
</div>
@@ -2970,7 +2984,7 @@ with the form utilized in linear regression, viz.</p>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20789/3777801602.py:7: UserWarning: set_ticklabels() should only be used with a fixed number of ticks, i.e. after set_ticks() or using a FixedLocator.
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_76898/3777801602.py:7: UserWarning: set_ticklabels() should only be used with a fixed number of ticks, i.e. after set_ticks() or using a FixedLocator.
cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)
</pre></div>
</div>
@@ -3010,7 +3024,7 @@ cost function is given by</p>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20789/438060758.py:10: UserWarning: set_ticklabels() should only be used with a fixed number of ticks, i.e. after set_ticks() or using a FixedLocator.
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_76898/438060758.py:10: UserWarning: set_ticklabels() should only be used with a fixed number of ticks, i.e. after set_ticks() or using a FixedLocator.
cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)
</pre></div>
</div>
@@ -3045,7 +3059,7 @@ cost function is given by</p>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_20789/3544313922.py:9: UserWarning: set_ticklabels() should only be used with a fixed number of ticks, i.e. after set_ticks() or using a FixedLocator.
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/var/folders/td/3yk470mj5p931p9dtkk0y6jw0000gn/T/ipykernel_76898/3544313922.py:9: UserWarning: set_ticklabels() should only be used with a fixed number of ticks, i.e. after set_ticks() or using a FixedLocator.
cb.ax.set_yticklabels(cb.ax.get_yticklabels(), fontsize=18)
</pre></div>
</div>
@@ -3098,43 +3112,43 @@ constant as opposed to ridge and OLS. We get a sparse solution with
</div>
</div>
<div class="cell_output docutils container">
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 0%| | 0/10 [00:00&lt;?, ?it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 0%| | 0/10 [00:00&lt;?, ?it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/miniforge3/envs/myenv/lib/python3.9/site-packages/sklearn/linear_model/_coordinate_descent.py:628: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations, check the scale of the features or consider increasing regularisation. Duality gap: 3.924e+00, tolerance: 1.797e+00
model = cd_fast.enet_coordinate_descent(
10%|██████████ | 1/10 [00:00&lt;00:06, 1.30it/s]
10%|█████████████▍ | 1/10 [00:01&lt;00:10, 1.18s/it]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 20%|████████████████████ | 2/10 [00:01&lt;00:06, 1.25it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 20%|██████████████████████████▊ | 2/10 [00:02&lt;00:09, 1.13s/it]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 30%|██████████████████████████████ | 3/10 [00:02&lt;00:04, 1.44it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 30%|████████████████████████████████████████▏ | 3/10 [00:03&lt;00:07, 1.00s/it]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 40%|████████████████████████████████████████ | 4/10 [00:02&lt;00:03, 1.52it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 40%|█████████████████████████████████████████████████████▌ | 4/10 [00:03&lt;00:05, 1.07it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 50%|██████████████████████████████████████████████████ | 5/10 [00:03&lt;00:03, 1.53it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 50%|███████████████████████████████████████████████████████████████████ | 5/10 [00:04&lt;00:04, 1.15it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 60%|████████████████████████████████████████████████████████████ | 6/10 [00:04&lt;00:02, 1.54it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 60%|████████████████████████████████████████████████████████████████████████████████▍ | 6/10 [00:05&lt;00:03, 1.19it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 70%|██████████████████████████████████████████████████████████████████████ | 7/10 [00:04&lt;00:02, 1.46it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 70%|█████████████████████████████████████████████████████████████████████████████████████████████▊ | 7/10 [00:06&lt;00:02, 1.25it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 80%|████████████████████████████████████████████████████████████████████████████████ | 8/10 [00:05&lt;00:01, 1.49it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 80%|███████████████████████████████████████████████████████████████████████████████████████████████████████████▏ | 8/10 [00:07&lt;00:01, 1.24it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 90%|██████████████████████████████████████████████████████████████████████████████████████████ | 9/10 [00:06&lt;00:00, 1.50it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 90%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████▌ | 9/10 [00:07&lt;00:00, 1.18it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>100%|███████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:06&lt;00:00, 1.55it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:08&lt;00:00, 1.19it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>100%|███████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:06&lt;00:00, 1.49it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 10/10 [00:08&lt;00:00, 1.14it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>