lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

update

Browse Source
This commit is contained in:
Morten Hjorth-Jensen
2024-10-21 13:45:48 +02:00
parent 0a03ba583c
commit de6a6d8bcb
144 changed files with 7186 additions and 4274 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/LectureNotes/_build/html/week40.html
+125 -122
View File
@@ -343,6 +343,11 @@ const thebe_selector_output = ".output, .cell_output"
Week 43: Deep Learning: Constructing a Neural Network code and solving differential equations
</a>
</li>
<li class="toctree-l1">
<a class="reference internal" href="exercisesweek43.html">
Exercises week 43
</a>
</li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
@@ -1324,17 +1329,17 @@ We summarize some of these here for the methods we hvae studied in project one,
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Parameters for OLS using gradient descent
[[3.99232771]
[2.9722424 ]
[5.02157768]]
[[3.79341574]
[3.45532899]
[4.79869224]]
Parameters for Ridge using gradient descent
[[3.48717555]
[4.38506982]
[4.33519245]]
[[3.66467908]
[3.67744448]
[4.69668411]]
Parameters for Lasso using gradient descent
[[3.82677051]
[3.44384079]
[4.7949388 ]]
[[3.8936154 ]
[3.05785843]
[5.01267038]]
</pre></div>
</div>
</div>
@@ -1386,11 +1391,11 @@ Parameters for Lasso using gradient descent
[[4.]
[3.]
[5.]]
0 [-22.8745559] [-27.28341218]
1 [9.27968813e-14] [1.11928633e-13]
2 [-7.10542736e-17] [-9.75573622e-17]
3 [-7.10542736e-17] [-9.75573622e-17]
4 [-7.10542736e-17] [-9.75573622e-17]
0 [-28.98027789] [-37.42420721]
1 [1.31983313e-14] [3.4924028e-14]
2 [7.99360578e-16] [9.55743376e-16]
3 [-1.42108547e-16] [-2.57209333e-16]
4 [-1.59872116e-16] [6.89684207e-17]
beta from own Newton code
[[4.]
[3.]
@@ -1732,15 +1737,15 @@ function.</p>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
[[3.60753606]
[3.27788872]]
Eigenvalues of Hessian Matrix:[0.25915746 4.55929651]
[[3.8125468]
[3.3410149]]
Eigenvalues of Hessian Matrix:[0.30966921 4.24520733]
theta from own gd
[[3.60753606]
[3.27788872]]
[[3.8125468]
[3.3410149]]
theta from own sdg
[[3.62299126]
[3.28377068]]
[[3.76103892]
[3.31388878]]
</pre></div>
</div>
<img alt="_images/week40_34_1.png" src="_images/week40_34_1.png" />
@@ -2454,12 +2459,12 @@ first example shows results with ordinary leats squares.</p>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
[[4.23770713]
[2.82891669]]
Eigenvalues of Hessian Matrix:[0.26781531 4.61565184]
[[4.14040757]
[2.84482599]]
Eigenvalues of Hessian Matrix:[0.29095968 3.92422684]
theta from own gd
[[4.23770713]
[2.82891669]]
[[4.14040757]
[2.84482599]]
</pre></div>
</div>
<img alt="_images/week40_100_1.png" src="_images/week40_100_1.png" />
@@ -2530,73 +2535,73 @@ theta from own gd
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
[[4.]
[3.]]
Eigenvalues of Hessian Matrix:[0.34031551 4.01826645]
0 [-16.43418708] [-17.66734487]
1 [-0.20743662] [0.18810873]
2 [-0.18986837] [0.1721774]
3 [-0.17378802] [0.15759533]
4 [-0.15906954] [0.14424825]
5 [-0.14559761] [0.13203156]
6 [-0.13326664] [0.12084953]
7 [-0.12198] [0.11061452]
8 [-0.11164926] [0.10124634]
9 [-0.10219344] [0.09267158]
10 [-0.09353846] [0.08482302]
11 [-0.08561649] [0.07763918]
12 [-0.07836545] [0.07106376]
13 [-0.07172852] [0.06504522]
14 [-0.06565368] [0.0595364]
15 [-0.06009333] [0.05449413]
16 [-0.05500389] [0.04987891]
17 [-0.0503455] [0.04565456]
18 [-0.04608163] [0.04178798]
19 [-0.04217888] [0.03824887]
20 [-0.03860666] [0.03500949]
21 [-0.03533698] [0.03204446]
22 [-0.03234422] [0.02933055]
23 [-0.02960492] [0.02684648]
24 [-0.02709761] [0.0245728]
25 [-0.02480266] [0.02249167]
26 [-0.02270207] [0.02058681]
27 [-0.02077938] [0.01884327]
28 [-0.01901953] [0.01724739]
29 [-0.01740873] [0.01578667]
Eigenvalues of Hessian Matrix:[0.28379561 4.4050219 ]
0 [-10.19786399] [-11.56308108]
1 [-0.23478113] [0.19832989]
2 [-0.21965525] [0.1855524]
3 [-0.20550386] [0.1735981]
4 [-0.19226417] [0.16241396]
5 [-0.17987746] [0.15195036]
6 [-0.16828877] [0.14216089]
7 [-0.15744669] [0.13300211]
8 [-0.14730311] [0.12443338]
9 [-0.13781304] [0.1164167]
10 [-0.12893437] [0.1089165]
11 [-0.12062771] [0.10189951]
12 [-0.11285621] [0.09533458]
13 [-0.1055854] [0.08919261]
14 [-0.09878301] [0.08344633]
15 [-0.09241887] [0.07807026]
16 [-0.08646474] [0.07304055]
17 [-0.08089421] [0.06833488]
18 [-0.07568256] [0.06393237]
19 [-0.07080668] [0.0598135]
20 [-0.06624492] [0.05595998]
21 [-0.06197706] [0.05235474]
22 [-0.05798416] [0.04898176]
23 [-0.0542485] [0.04582608]
24 [-0.05075352] [0.04287372]
25 [-0.0474837] [0.04011156]
26 [-0.04442454] [0.03752735]
27 [-0.04156247] [0.03510963]
28 [-0.03888479] [0.03284768]
29 [-0.03637961] [0.03073145]
theta from own gd
[[3.95317772]
[3.04245961]]
0 [-0.01593435] [0.01444966]
1 [-0.01458483] [0.01322589]
2 [-0.01294476] [0.01173863]
3 [-0.01135642] [0.01029828]
4 [-0.00991812] [0.008994]
5 [-0.00864664] [0.00784099]
6 [-0.00753289] [0.00683102]
7 [-0.00656079] [0.00594949]
8 [-0.00571352] [0.00518116]
9 [-0.00497544] [0.00451186]
10 [-0.00433264] [0.00392895]
11 [-0.00377286] [0.00342132]
12 [-0.00328539] [0.00297928]
13 [-0.00286091] [0.00259434]
14 [-0.00249126] [0.00225914]
15 [-0.00216938] [0.00196725]
16 [-0.00188909] [0.00171307]
17 [-0.00164501] [0.00149173]
18 [-0.00143246] [0.00129899]
19 [-0.00124738] [0.00113116]
20 [-0.00108622] [0.00098501]
21 [-0.00094587] [0.00085774]
22 [-0.00082366] [0.00074692]
23 [-0.00071724] [0.00065041]
24 [-0.00062457] [0.00056637]
25 [-0.00054387] [0.0004932]
26 [-0.0004736] [0.00042947]
27 [-0.00041241] [0.00037398]
28 [-0.00035912] [0.00032566]
29 [-0.00031272] [0.00028359]
[[3.88006918]
[3.10131081]]
0 [-0.03403584] [0.02875156]
1 [-0.03184307] [0.02689923]
2 [-0.02913373] [0.02461054]
3 [-0.02644397] [0.02233838]
4 [-0.02393338] [0.02021757]
5 [-0.02163828] [0.01827881]
6 [-0.0195557] [0.01651955]
7 [-0.01767104] [0.0149275]
8 [-0.01596717] [0.01348817]
9 [-0.01442732] [0.01218739]
10 [-0.01303588] [0.01101198]
11 [-0.0117786] [0.0099499]
12 [-0.01064258] [0.00899025]
13 [-0.00961612] [0.00812316]
14 [-0.00868866] [0.00733969]
15 [-0.00785065] [0.00663179]
16 [-0.00709346] [0.00599216]
17 [-0.00640931] [0.00541422]
18 [-0.00579114] [0.00489203]
19 [-0.00523259] [0.0044202]
20 [-0.00472792] [0.00399388]
21 [-0.00427191] [0.00360867]
22 [-0.00385989] [0.00326062]
23 [-0.00348761] [0.00294614]
24 [-0.00315124] [0.00266199]
25 [-0.0028473] [0.00240524]
26 [-0.00257269] [0.00217326]
27 [-0.00232455] [0.00196365]
28 [-0.00210035] [0.00177426]
29 [-0.00189778] [0.00160314]
theta from own gd wth momentum
[[3.99919981]
[3.00072563]]
[[3.99395784]
[3.00510408]]
</pre></div>
</div>
</div>
@@ -2685,20 +2690,18 @@ theta from own gd wth momentum
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
[[4.13706862]
[2.74803743]]
Eigenvalues of Hessian Matrix:[0.2974512 4.33165194]
[[3.97904015]
[3.11938084]]
Eigenvalues of Hessian Matrix:[0.29437712 4.50172351]
theta from own gd
[[3.97904015]
[3.11938084]]
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own gd
[[4.13706862]
[2.74803743]]
</pre></div>
</div>
<img alt="_images/week40_104_2.png" src="_images/week40_104_2.png" />
<img alt="_images/week40_104_1.png" src="_images/week40_104_1.png" />
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own sdg
[[4.09649893]
[2.76751116]]
[[3.97069113]
[3.13904707]]
</pre></div>
</div>
</div>
@@ -2780,17 +2783,17 @@ Eigenvalues of Hessian Matrix:[0.2974512 4.33165194]
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
[[4.16147054]
[2.8496945 ]]
Eigenvalues of Hessian Matrix:[0.31764285 4.29777705]
[[4.20902858]
[2.82421714]]
Eigenvalues of Hessian Matrix:[0.29463222 4.67204559]
theta from own gd
[[4.20862308]
[2.82454109]]
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own gd
[[4.16119967]
[2.84992628]]
theta from own sdg with momentum
[[4.18864495]
[2.78810857]]
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own sdg with momentum
[[4.13343872]
[2.81165023]]
</pre></div>
</div>
</div>
@@ -2859,9 +2862,9 @@ theta from own sdg with momentum
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own AdaGrad
[[2.00028632]
[2.99833653]
[4.00168427]]
[[1.90103664]
[3.54492296]
[3.47989639]]
</pre></div>
</div>
</div>
@@ -2937,9 +2940,9 @@ theta from own sdg with momentum
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own RMSprop
[[2.00002192]
[2.9999923 ]
[4.00001976]]
[[1.99858474]
[3.00521037]
[3.99718155]]
</pre></div>
</div>
</div>
@@ -3019,9 +3022,9 @@ theta from own sdg with momentum
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own ADAM
[[2.00000262]
[2.99995454]
[4.00004164]]
[[2.00002505]
[2.99981314]
[4.00017937]]
</pre></div>
</div>
</div>
@@ -3142,7 +3145,7 @@ It provides composable transformations of Python+NumPy programs: differentiate,
return asarray(x, dtype=self.dtype)
</pre></div>
</div>
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[&lt;matplotlib.lines.Line2D at 0x11baaec10&gt;]
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[&lt;matplotlib.lines.Line2D at 0x117269df0&gt;]
</pre></div>
</div>
<img alt="_images/week40_120_2.png" src="_images/week40_120_2.png" />
@@ -3177,7 +3180,7 @@ It provides composable transformations of Python+NumPy programs: differentiate,
</div>
</div>
<div class="cell_output docutils container">
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;matplotlib.collections.PathCollection at 0x11ba4a5b0&gt;
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;matplotlib.collections.PathCollection at 0x117195940&gt;
</pre></div>
</div>
<img alt="_images/week40_122_1.png" src="_images/week40_122_1.png" />