lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

update

Browse Source
This commit is contained in:
Morten Hjorth-Jensen
2023-11-21 06:16:54 +01:00
parent a854a3e9c0
commit 06b09eb681
173 changed files with 13894 additions and 3718 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/LectureNotes/_build/html/exercisesweek41.html
+128 -113
View File
@@ -353,6 +353,16 @@ const thebe_selector_output = ".output, .cell_output"
Week 45, Recurrent Neural Networks
</a>
</li>
<li class="toctree-l1">
<a class="reference internal" href="week46.html">
Week 46: Decision Trees, Ensemble methods and Random Forests
</a>
</li>
<li class="toctree-l1">
<a class="reference internal" href="week47.html">
Week 47: From Decision Trees to Ensemble Methods, Random Forests and Boosting Methods and Summary of Course
</a>
</li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
@@ -367,7 +377,12 @@ const thebe_selector_output = ".output, .cell_output"
</li>
<li class="toctree-l1">
<a class="reference internal" href="project2.html">
Project 2 on Machine Learning, deadline November 13 (Midnight)
Project 2 on Machine Learning, deadline November 17 (Midnight)
</a>
</li>
<li class="toctree-l1">
<a class="reference internal" href="project3.html">
Project 3 on Machine Learning, deadline December 18 (midnight), 2023
</a>
</li>
</ul>
@@ -824,15 +839,15 @@ regression.</p>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
[[3.69887085]
[3.34977681]]
Eigenvalues of Hessian Matrix:[0.25923926 4.67193435]
[[4.16422434]
[2.90414392]]
Eigenvalues of Hessian Matrix:[0.29405153 4.56291744]
theta from own gd
[[3.69887085]
[3.34977681]]
[[4.16422434]
[2.90414392]]
theta from own sdg
[[3.63685221]
[3.37369014]]
[[4.13747168]
[2.89670222]]
</pre></div>
</div>
<img alt="_images/exercisesweek41_5_1.png" src="_images/exercisesweek41_5_1.png" />
@@ -954,14 +969,14 @@ first example shows results with ordinary leats squares.</p>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
[[3.64342603]
[3.30485583]]
Eigenvalues of Hessian Matrix:[0.29030069 4.6608358 ]
[[3.6398684 ]
[3.27790702]]
Eigenvalues of Hessian Matrix:[0.35845332 4.35403494]
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own gd
[[3.64342603]
[3.30485583]]
[[3.6398684 ]
[3.27790702]]
</pre></div>
</div>
<img alt="_images/exercisesweek41_16_2.png" src="_images/exercisesweek41_16_2.png" />
@@ -1032,73 +1047,73 @@ Eigenvalues of Hessian Matrix:[0.29030069 4.6608358 ]
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
[[4.]
[3.]]
Eigenvalues of Hessian Matrix:[0.27335131 4.46000649]
0 [-13.27099835] [-15.86570776]
1 [0.01163425] [-0.00974702]
2 [0.01092119] [-0.00914964]
3 [0.01025184] [-0.00858886]
4 [0.00962351] [-0.00806245]
5 [0.00903369] [-0.00756831]
6 [0.00848002] [-0.00710445]
7 [0.00796028] [-0.00666902]
8 [0.0074724] [-0.00626028]
9 [0.00701442] [-0.00587659]
10 [0.00658451] [-0.00551642]
11 [0.00618095] [-0.00517832]
12 [0.00580212] [-0.00486095]
13 [0.00544651] [-0.00456302]
14 [0.0051127] [-0.00428336]
15 [0.00479935] [-0.00402083]
16 [0.0045052] [-0.0037744]
17 [0.00422908] [-0.00354307]
18 [0.00396988] [-0.00332591]
19 [0.00372657] [-0.00312207]
20 [0.00349817] [-0.00293072]
21 [0.00328377] [-0.0027511]
22 [0.00308251] [-0.00258249]
23 [0.00289358] [-0.00242421]
24 [0.00271624] [-0.00227563]
25 [0.00254976] [-0.00213616]
26 [0.00239349] [-0.00200523]
27 [0.00224679] [-0.00188233]
28 [0.00210909] [-0.00176697]
29 [0.00197982] [-0.00165867]
Eigenvalues of Hessian Matrix:[0.3431178 4.16725475]
0 [-14.28194794] [-15.81268342]
1 [-0.23712617] [0.2073339]
2 [-0.21760199] [0.19026272]
3 [-0.19968537] [0.17459712]
4 [-0.18324395] [0.16022138]
5 [-0.16815626] [0.14702929]
6 [-0.15431083] [0.13492339]
7 [-0.1416054] [0.12381425]
8 [-0.12994608] [0.1136198]
9 [-0.11924676] [0.10426473]
10 [-0.10942838] [0.09567992]
11 [-0.10041841] [0.08780195]
12 [-0.0921503] [0.08057264]
13 [-0.08456295] [0.07393855]
14 [-0.07760032] [0.0678507]
15 [-0.07121097] [0.0622641]
16 [-0.0653477] [0.05713748]
17 [-0.05996719] [0.05243298]
18 [-0.05502969] [0.04811582]
19 [-0.05049873] [0.04415412]
20 [-0.04634083] [0.04051862]
21 [-0.04252528] [0.03718245]
22 [-0.03902389] [0.03412098]
23 [-0.0358108] [0.03131157]
24 [-0.03286226] [0.02873348]
25 [-0.03015649] [0.02636766]
26 [-0.0276735] [0.02419664]
27 [-0.02539496] [0.02220437]
28 [-0.02330402] [0.02037613]
29 [-0.02138525] [0.01869843]
theta from own gd
[[4.00679887]
[2.994304 ]]
0 [0.00185848] [-0.00155701]
1 [0.00174457] [-0.00146158]
2 [0.00160348] [-0.00134337]
3 [0.00146287] [-0.00122558]
4 [0.00133103] [-0.00111512]
5 [0.0012099] [-0.00101364]
6 [0.00109941] [-0.00092107]
7 [0.00099888] [-0.00083685]
8 [0.0009075] [-0.00076029]
9 [0.00082447] [-0.00069073]
10 [0.00074902] [-0.00062752]
11 [0.00068048] [-0.0005701]
12 [0.00061822] [-0.00051793]
13 [0.00056165] [-0.00047054]
14 [0.00051025] [-0.00042748]
15 [0.00046356] [-0.00038836]
16 [0.00042114] [-0.00035283]
17 [0.0003826] [-0.00032054]
18 [0.00034759] [-0.00029121]
19 [0.00031579] [-0.00026456]
20 [0.00028689] [-0.00024035]
21 [0.00026064] [-0.00021836]
22 [0.00023679] [-0.00019838]
23 [0.00021512] [-0.00018022]
24 [0.00019544] [-0.00016373]
25 [0.00017755] [-0.00014875]
26 [0.0001613] [-0.00013514]
27 [0.00014654] [-0.00012277]
28 [0.00013313] [-0.00011154]
29 [0.00012095] [-0.00010133]
[[3.94280548]
[3.05000867]]
0 [-0.01962446] [0.01715886]
1 [-0.01800865] [0.01574606]
2 [-0.01604113] [0.01402574]
3 [-0.0141301] [0.01235481]
4 [-0.01239337] [0.01083628]
5 [-0.01085192] [0.0094885]
6 [-0.00949598] [0.00830291]
7 [-0.00830733] [0.0072636]
8 [-0.00726673] [0.00635375]
9 [-0.00635624] [0.00555765]
10 [-0.00555974] [0.00486122]
11 [-0.00486302] [0.00425203]
12 [-0.0042536] [0.00371918]
13 [-0.00372054] [0.0032531]
14 [-0.00325429] [0.00284542]
15 [-0.00284647] [0.00248884]
16 [-0.00248975] [0.00217694]
17 [-0.00217774] [0.00190413]
18 [-0.00190483] [0.00166551]
19 [-0.00166612] [0.00145679]
20 [-0.00145732] [0.00127423]
21 [-0.00127469] [0.00111454]
22 [-0.00111495] [0.00097487]
23 [-0.00097522] [0.0008527]
24 [-0.00085301] [0.00074584]
25 [-0.00074611] [0.00065237]
26 [-0.00065261] [0.00057062]
27 [-0.00057083] [0.00049911]
28 [-0.00049929] [0.00043656]
29 [-0.00043672] [0.00038185]
theta from own gd wth momentum
[[4.00040199]
[2.99966322]]
[[3.9988867 ]
[3.00097342]]
</pre></div>
</div>
</div>
@@ -1151,17 +1166,17 @@ theta from own gd wth momentum
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
[[3.71369789]
[3.2314999 ]]
Eigenvalues of Hessian Matrix:[0.30237154 4.4642383 ]
0 [-17.75091492] [-21.33108943]
1 [-4.60742555e-15] [5.64228618e-16]
2 [-5.34294831e-16] [-5.79981535e-16]
3 [-5.34294831e-16] [-5.79981535e-16]
4 [-5.34294831e-16] [-5.79981535e-16]
[[3.87043345]
[3.22475203]]
Eigenvalues of Hessian Matrix:[0.38540656 3.68548206]
0 [-12.40634991] [-11.15578306]
1 [-5.99520433e-15] [-8.10731958e-15]
2 [-2.87964097e-16] [-5.20946226e-16]
3 [4.12864187e-16] [4.15804451e-16]
4 [4.12864187e-16] [4.15804451e-16]
beta from own Newton code
[[3.71369789]
[3.2314999 ]]
[[3.87043345]
[3.22475203]]
</pre></div>
</div>
</div>
@@ -1250,20 +1265,20 @@ beta from own Newton code
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
[[4.39917327]
[2.69542733]]
Eigenvalues of Hessian Matrix:[0.29765192 4.0375827 ]
[[3.98225665]
[3.11495144]]
Eigenvalues of Hessian Matrix:[0.29378469 4.27347181]
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own gd
[[4.39917327]
[2.69542733]]
[[3.98225665]
[3.11495144]]
</pre></div>
</div>
<img alt="_images/exercisesweek41_22_2.png" src="_images/exercisesweek41_22_2.png" />
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own sdg
[[4.32234998]
[2.64530585]]
[[3.89184354]
[3.13010854]]
</pre></div>
</div>
</div>
@@ -1345,17 +1360,17 @@ Eigenvalues of Hessian Matrix:[0.29765192 4.0375827 ]
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
[[3.7635689 ]
[3.10080981]]
Eigenvalues of Hessian Matrix:[0.31633433 3.9824638 ]
[[4.11385399]
[2.90239437]]
Eigenvalues of Hessian Matrix:[0.2870363 4.27221341]
theta from own gd
[[3.76366462]
[3.1007216 ]]
[[4.11265847]
[2.90343239]]
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own sdg with momentum
[[3.75707243]
[3.12422141]]
[[4.07560398]
[2.82894312]]
</pre></div>
</div>
</div>
@@ -1430,9 +1445,9 @@ theta from own gd
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own AdaGrad
[[1.99999956]
[3.00000215]
[3.99999797]]
[[2.00036163]
[2.99784051]
[4.00208264]]
</pre></div>
</div>
</div>
@@ -1514,9 +1529,9 @@ theta from own gd
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own RMSprop
[[1.99907985]
[2.99897733]
[3.99754609]]
[[1.99976935]
[2.99966842]
[4.00437973]]
</pre></div>
</div>
</div>
@@ -1602,9 +1617,9 @@ theta from own gd
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own ADAM
[[2.00003617]
[2.99986253]
[4.00012569]]
[[1.99998899]
[3.00009903]
[3.99988697]]
</pre></div>
</div>
</div>
@@ -1677,7 +1692,7 @@ It provides composable transformations of Python+NumPy programs: differentiate,
return asarray(x, dtype=self.dtype)
</pre></div>
</div>
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[&lt;matplotlib.lines.Line2D at 0x11cb23a60&gt;]
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[&lt;matplotlib.lines.Line2D at 0x126f4fa00&gt;]
</pre></div>
</div>
<img alt="_images/exercisesweek41_39_2.png" src="_images/exercisesweek41_39_2.png" />
@@ -1716,7 +1731,7 @@ It provides composable transformations of Python+NumPy programs: differentiate,
return asarray(x, dtype=self.dtype)
</pre></div>
</div>
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;matplotlib.collections.PathCollection at 0x11cb23fd0&gt;
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;matplotlib.collections.PathCollection at 0x126f922b0&gt;
</pre></div>
</div>
<img alt="_images/exercisesweek41_41_2.png" src="_images/exercisesweek41_41_2.png" />