lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

updated book

Browse Source
This commit is contained in:
Morten Hjorth-Jensen
2021-10-19 08:42:04 +02:00
parent a29107e743
commit a0360cf460
102 changed files with 15480 additions and 1360 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/LectureNotes/_build/html/chapter3.html
+42 -40
View File
@@ -218,6 +218,11 @@
14. Building a Feed Forward Neural Network
</a>
</li>
<li class="toctree-l1">
<a class="reference internal" href="chapter11.html">
15. Solving Differential Equations with Deep Learning
</a>
</li>
</ul>
</div>
@@ -640,10 +645,10 @@ number <span class="math notranslate nohighlight">\(i\)</span> is left out. Usin
</div>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Runtime: 0.14109 sec
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Runtime: 0.135707 sec
Jackknife Statistics :
original bias std. error
100.203 100.193 0.149917
100.099 100.089 0.150795
</pre></div>
</div>
</div>
@@ -862,7 +867,7 @@ theorem.</p>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Bootstrap Statistics :
original bias std. error
99.9348 15.1379 99.9341 0.151076
100.186 15.0063 100.185 0.148455
</pre></div>
</div>
</div>
@@ -1064,10 +1069,9 @@ Error: 0.32149601703519126
Bias^2: 0.3123314713548606
Var: 0.009164545680330616
0.32149601703519126 &gt;= 0.3123314713548606 + 0.009164545680330616 = 0.3214960170351912
Polynomial degree:
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 1
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Polynomial degree: 1
Error: 0.08426840630693411
Bias^2: 0.07968918676726028
Var: 0.004579219539673833
@@ -1099,9 +1103,7 @@ Error: 0.03781367141738898
Bias^2: 0.03365768507152761
Var: 0.004155986345861379
0.03781367141738898 &gt;= 0.03365768507152761 + 0.004155986345861379 = 0.03781367141738899
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Polynomial degree: 7
Polynomial degree: 7
Error: 0.027609773491022498
Bias^2: 0.02299949826036597
Var: 0.004610275230656537
@@ -1111,7 +1113,9 @@ Error: 0.017355848195591973
Bias^2: 0.010331721306655588
Var: 0.007024126888936384
0.017355848195591973 &gt;= 0.010331721306655588 + 0.007024126888936384 = 0.017355848195591973
Polynomial degree: 9
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Polynomial degree: 9
Error: 0.026605727637189085
Bias^2: 0.010018312644140933
Var: 0.016587414993048166
@@ -1121,9 +1125,7 @@ Error: 0.021592704588043153
Bias^2: 0.010516485576652981
Var: 0.011076219011390184
0.021592704588043153 &gt;= 0.010516485576652981 + 0.011076219011390184 = 0.021592704588043167
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Polynomial degree: 11
Polynomial degree: 11
Error: 0.07160048164228314
Bias^2: 0.01443680008897583
Var: 0.0571636815533073
@@ -1142,7 +1144,7 @@ Var: 0.20867052175003387
0.22842468702166951 &gt;= 0.01975416527163567 + 0.20867052175003387 = 0.22842468702166954
</pre></div>
</div>
<img alt="_images/chapter3_62_6.png" src="_images/chapter3_62_6.png" />
<img alt="_images/chapter3_62_5.png" src="_images/chapter3_62_5.png" />
</div>
</div>
<p>The bias-variance tradeoff summarizes the fundamental tension in
@@ -1388,12 +1390,12 @@ Mean squared error on test data: 0.08576932
Degree of polynomial: 10
Mean squared error on training data: 0.02511518
Mean squared error on test data: 1.20015436
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 11
Degree of polynomial: 11
Mean squared error on training data: 0.01640891
Mean squared error on test data: 1.35533774
Degree of polynomial: 12
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 12
Mean squared error on training data: 0.00813803
Mean squared error on test data: 0.17446471
Degree of polynomial: 13
@@ -1424,34 +1426,34 @@ Mean squared error on test data: 1376.61081005
Degree of polynomial: 20
Mean squared error on training data: 0.00137945
Mean squared error on test data: 1931.97211078
Degree of polynomial: 21
Mean squared error on training data: 0.00118678
Mean squared error on test data: 14496.70992192
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 22
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 21
Mean squared error on training data: 0.00118678
Mean squared error on test data: 14496.70992192
Degree of polynomial: 22
Mean squared error on training data: 0.00092686
Mean squared error on test data: 873.95463048
Degree of polynomial: 23
Mean squared error on training data: 0.00085890
Mean squared error on test data: 5535.20053452
Degree of polynomial: 24
Mean squared error on training data: 0.00084714
Mean squared error on test data: 1289.22422186
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 25
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 24
Mean squared error on training data: 0.00084714
Mean squared error on test data: 1289.22422186
Degree of polynomial: 25
Mean squared error on training data: 0.00079022
Mean squared error on test data: 136582.88824397
Degree of polynomial: 26
Mean squared error on training data: 0.00076923
Mean squared error on test data: 18194.23521766
Degree of polynomial: 27
Mean squared error on training data: 0.00069302
Mean squared error on test data: 2579.13493762
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 28
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Degree of polynomial: 27
Mean squared error on training data: 0.00069302
Mean squared error on test data: 2579.13493762
Degree of polynomial: 28
Mean squared error on training data: 0.00062728
Mean squared error on test data: 3984.82493809
Degree of polynomial: 29
@@ -2880,37 +2882,37 @@ constant as opposed to ridge and OLS. We get a sparse solution with
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>/Users/mhjensen/opt/anaconda3/lib/python3.8/site-packages/sklearn/linear_model/_coordinate_descent.py:529: ConvergenceWarning: Objective did not converge. You might want to increase the number of iterations. Duality gap: 3.924197515789051, tolerance: 1.796796
model = cd_fast.enet_coordinate_descent(
10%|█ | 1/10 [00:00&lt;00:04, 2.01it/s]
10%|█ | 1/10 [00:00&lt;00:04, 2.02it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 20%|██ | 2/10 [00:00&lt;00:03, 2.26it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 20%|██ | 2/10 [00:00&lt;00:03, 2.38it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 30%|███ | 3/10 [00:00&lt;00:02, 2.75it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 30%|███ | 3/10 [00:00&lt;00:02, 2.99it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 40%|████ | 4/10 [00:01&lt;00:01, 3.39it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 40%|████ | 4/10 [00:00&lt;00:01, 3.72it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 50%|█████ | 5/10 [00:01&lt;00:01, 4.00it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 50%|█████ | 5/10 [00:01&lt;00:01, 4.40it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 60%|██████ | 6/10 [00:01&lt;00:00, 4.67it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 60%|██████ | 6/10 [00:01&lt;00:00, 5.16it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 70%|███████ | 7/10 [00:01&lt;00:00, 5.36it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 70%|███████ | 7/10 [00:01&lt;00:00, 5.95it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 80%|████████ | 8/10 [00:01&lt;00:00, 6.06it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 80%|████████ | 8/10 [00:01&lt;00:00, 6.55it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 90%|█████████ | 9/10 [00:01&lt;00:00, 6.05it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> 90%|█████████ | 9/10 [00:01&lt;00:00, 6.99it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>100%|██████████| 10/10 [00:02&lt;00:00, 4.64it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>100%|██████████| 10/10 [00:01&lt;00:00, 7.35it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>100%|██████████| 10/10 [00:02&lt;00:00, 4.68it/s]
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>100%|██████████| 10/10 [00:01&lt;00:00, 5.87it/s]
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>