lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

update

Browse Source
This commit is contained in:
Morten Hjorth-Jensen
2024-11-25 08:12:27 +01:00
parent d63cb194d5
commit c8f2aa3dc1
181 changed files with 13447 additions and 2592 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/LectureNotes/_build/html/exercisesweek41.html
+118 -115
View File
@@ -257,6 +257,9 @@
<li class="toctree-l1"><a class="reference internal" href="week47.html">Week 47: From Decision Trees to Ensemble Methods, Random Forests and Boosting Methods</a></li>
<li class="toctree-l1"><a class="reference internal" href="exercisesweek47.html">Exercise week 47</a></li>
<li class="toctree-l1"><a class="reference internal" href="week48.html">Week 48: Gradient boosting and summary of course</a></li>
<li class="toctree-l1"><a class="reference internal" href="exercisesweek48.html">Exercises week 48</a></li>
</ul>
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Projects</span></p>
<ul class="nav bd-sidenav">
@@ -599,18 +602,18 @@ regression.</p>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
[[4.0010275 ]
[3.10085547]]
Eigenvalues of Hessian Matrix:[0.32462943 4.22813513]
[[4.41622083]
[2.74692745]]
Eigenvalues of Hessian Matrix:[0.26594878 4.45190996]
theta from own gd
[[4.0010275 ]
[3.10085547]]
[[4.41622083]
[2.74692745]]
theta from own sdg
[[4.08051993]
[3.13522576]]
[[4.38222303]
[2.74411787]]
</pre></div>
</div>
<img alt="_images/28af7ec857f6b0c73278974d8a0f02963d4300b7ede7129fc2e95b8d68a8492b.png" src="_images/28af7ec857f6b0c73278974d8a0f02963d4300b7ede7129fc2e95b8d68a8492b.png" />
<img alt="_images/74585d2c5d042ce0364bf21b5a8e9bc5b5504011eaa4a30c445a4415a4c1df98.png" src="_images/74585d2c5d042ce0364bf21b5a8e9bc5b5504011eaa4a30c445a4415a4c1df98.png" />
</div>
</div>
<p>In the above code, we have use replacement in setting up the
@@ -729,17 +732,17 @@ first example shows results with ordinary leats squares.</p>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
[[3.8679219 ]
[3.12497024]]
Eigenvalues of Hessian Matrix:[0.31887646 4.68712364]
[[4.32147704]
[2.78166226]]
Eigenvalues of Hessian Matrix:[0.32306518 4.38577204]
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own gd
[[3.8679219 ]
[3.12497024]]
[[4.32147704]
[2.78166226]]
</pre></div>
</div>
<img alt="_images/634d0baa356c54adef6974350a5d0f31fb981ef2c1199e2bf8abc3dbbd76d38b.png" src="_images/634d0baa356c54adef6974350a5d0f31fb981ef2c1199e2bf8abc3dbbd76d38b.png" />
<img alt="_images/c61bca31ee6787e0e5d1512e4bf365143d789096f02da0a858ff7e7c3834630c.png" src="_images/c61bca31ee6787e0e5d1512e4bf365143d789096f02da0a858ff7e7c3834630c.png" />
</div>
</div>
</section>
@@ -807,73 +810,73 @@ Eigenvalues of Hessian Matrix:[0.31887646 4.68712364]
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
[[4.]
[3.]]
Eigenvalues of Hessian Matrix:[0.3011187 4.49781862]
0 [-10.84111032] [-12.09572749]
1 [-0.48072858] [0.39646149]
2 [-0.4485449] [0.3699193]
3 [-0.41851584] [0.34515405]
4 [-0.39049716] [0.32204677]
5 [-0.36435426] [0.30048647]
6 [-0.33996158] [0.28036959]
7 [-0.31720192] [0.26159948]
8 [-0.29596598] [0.24408599]
9 [-0.27615173] [0.22774499]
10 [-0.257664] [0.21249799]
11 [-0.24041398] [0.19827173]
12 [-0.22431882] [0.18499789]
13 [-0.20930118] [0.1726127]
14 [-0.19528895] [0.16105668]
15 [-0.1822148] [0.1502743]
16 [-0.17001593] [0.14021378]
17 [-0.15863375] [0.13082678]
18 [-0.14801358] [0.12206823]
19 [-0.13810441] [0.11389604]
20 [-0.12885864] [0.10627096]
21 [-0.12023184] [0.09915636]
22 [-0.1121826] [0.09251807]
23 [-0.10467223] [0.08632419]
24 [-0.09766466] [0.08054499]
25 [-0.09112623] [0.07515268]
26 [-0.08502554] [0.07012138]
27 [-0.07933327] [0.06542692]
28 [-0.07402209] [0.06104673]
29 [-0.06906648] [0.05695979]
Eigenvalues of Hessian Matrix:[0.26136301 4.38259367]
0 [-12.42926102] [-13.7797646]
1 [-0.35774662] [0.30560116]
2 [-0.33641182] [0.28737615]
3 [-0.31634936] [0.27023801]
4 [-0.29748336] [0.25412193]
5 [-0.27974246] [0.23896696]
6 [-0.26305957] [0.22471578]
7 [-0.24737159] [0.2113145]
8 [-0.23261919] [0.19871242]
9 [-0.21874657] [0.18686188]
10 [-0.20570127] [0.17571808]
11 [-0.19343394] [0.16523885]
12 [-0.1818982] [0.15538456]
13 [-0.17105041] [0.14611795]
14 [-0.16084954] [0.13740398]
15 [-0.15125702] [0.12920967]
16 [-0.14223657] [0.12150404]
17 [-0.13375406] [0.11425795]
18 [-0.12577742] [0.107444]
19 [-0.11827649] [0.1010364]
20 [-0.11122288] [0.09501094]
21 [-0.10458992] [0.08934481]
22 [-0.09835253] [0.08401658]
23 [-0.09248712] [0.07900612]
24 [-0.08697151] [0.07429446]
25 [-0.08178482] [0.06986379]
26 [-0.07690745] [0.06569735]
27 [-0.07232095] [0.06177939]
28 [-0.06800798] [0.05809507]
29 [-0.06395221] [0.05463048]
theta from own gd
[[3.78598925]
[3.17649673]]
0 [-0.06444264] [0.05314647]
1 [-0.06012835] [0.04958843]
2 [-0.05480861] [0.04520119]
3 [-0.04954337] [0.0408589]
4 [-0.04464699] [0.0368208]
5 [-0.04018906] [0.0331443]
6 [-0.03616111] [0.02982242]
7 [-0.03253183] [0.02682931]
8 [-0.02926511] [0.02413522]
9 [-0.02632586] [0.02171119]
10 [-0.02368163] [0.01953047]
11 [-0.02130293] [0.01756873]
12 [-0.01916314] [0.01580402]
13 [-0.01723827] [0.01421657]
14 [-0.01550675] [0.01278856]
15 [-0.01394915] [0.011504]
16 [-0.01254801] [0.01034846]
17 [-0.0112876] [0.009309]
18 [-0.0101538] [0.00837394]
19 [-0.00913389] [0.00753281]
20 [-0.00821642] [0.00677616]
21 [-0.00739111] [0.00609552]
22 [-0.0066487] [0.00548325]
23 [-0.00598086] [0.00493247]
24 [-0.0053801] [0.00443702]
25 [-0.00483969] [0.00399134]
26 [-0.00435356] [0.00359042]
27 [-0.00391626] [0.00322978]
28 [-0.00352289] [0.00290536]
29 [-0.00316902] [0.00261353]
[[3.76990501]
[3.19655615]]
0 [-0.06013832] [0.05137251]
1 [-0.05655187] [0.04830882]
2 [-0.05210338] [0.04450874]
3 [-0.04766156] [0.04071437]
4 [-0.04348664] [0.03714799]
5 [-0.03964077] [0.0338627]
6 [-0.03612297] [0.03085765]
7 [-0.03291338] [0.02811589]
8 [-0.02998766] [0.02561663]
9 [-0.02732158] [0.02333916]
10 [-0.02489239] [0.02126405]
11 [-0.02267913] [0.0193734]
12 [-0.02066265] [0.01765085]
13 [-0.01882546] [0.01608144]
14 [-0.01715161] [0.01465158]
15 [-0.01562659] [0.01334885]
16 [-0.01423717] [0.01216195]
17 [-0.01297129] [0.01108058]
18 [-0.01181796] [0.01009536]
19 [-0.01076718] [0.00919775]
20 [-0.00980983] [0.00837994]
21 [-0.0089376] [0.00763484]
22 [-0.00814292] [0.006956]
23 [-0.0074189] [0.00633751]
24 [-0.00675926] [0.00577402]
25 [-0.00615826] [0.00526063]
26 [-0.00561071] [0.00479289]
27 [-0.00511184] [0.00436673]
28 [-0.00465732] [0.00397847]
29 [-0.00424322] [0.00362473]
theta from own gd wth momentum
[[3.99053294]
[3.00780757]]
[[3.98520854]
[3.01263545]]
</pre></div>
</div>
</div>
@@ -926,17 +929,17 @@ theta from own gd wth momentum
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
[[4.1739262 ]
[2.85239235]]
Eigenvalues of Hessian Matrix:[0.27968902 4.49424386]
0 [-13.16403338] [-14.97007223]
1 [7.61543606e-16] [2.78435018e-15]
2 [-4.23272528e-16] [-4.70203152e-16]
3 [-4.23272528e-16] [-4.70203152e-16]
4 [-4.23272528e-16] [-4.70203152e-16]
[[3.96856022]
[2.98116672]]
Eigenvalues of Hessian Matrix:[0.3407072 4.46918198]
0 [-14.47784776] [-17.95282048]
1 [2.49106291e-15] [-6.96409595e-15]
2 [8.32667268e-17] [8.72456212e-17]
3 [8.32667268e-17] [8.72456212e-17]
4 [8.32667268e-17] [8.72456212e-17]
beta from own Newton code
[[4.1739262 ]
[2.85239235]]
[[3.96856022]
[2.98116672]]
</pre></div>
</div>
</div>
@@ -1025,18 +1028,18 @@ beta from own Newton code
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
[[4.50455587]
[2.69978846]]
Eigenvalues of Hessian Matrix:[0.35172802 4.35427615]
[[4.20603792]
[2.82345148]]
Eigenvalues of Hessian Matrix:[0.26589296 4.18756348]
theta from own gd
[[4.50455587]
[2.69978846]]
[[4.20603792]
[2.82345148]]
</pre></div>
</div>
<img alt="_images/c8d1a6036144492783b21bafb5df5462fba5688ed1014961e0a3ecd4909a9caf.png" src="_images/c8d1a6036144492783b21bafb5df5462fba5688ed1014961e0a3ecd4909a9caf.png" />
<img alt="_images/5cfc76ca7e820abfb8e15070adf2fa59d6dcfb6d702ef48441625e67b326e24c.png" src="_images/5cfc76ca7e820abfb8e15070adf2fa59d6dcfb6d702ef48441625e67b326e24c.png" />
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own sdg
[[4.51550231]
[2.6728048 ]]
[[4.20773884]
[2.81706996]]
</pre></div>
</div>
</div>
@@ -1118,15 +1121,15 @@ theta from own gd
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
[[3.8301107 ]
[3.31610195]]
Eigenvalues of Hessian Matrix:[0.31047897 4.32701817]
[[3.79928328]
[3.09772988]]
Eigenvalues of Hessian Matrix:[0.33500183 3.88706629]
theta from own gd
[[3.82958153]
[3.31655284]]
[[3.79937084]
[3.09764763]]
theta from own sdg with momentum
[[3.82977678]
[3.25067525]]
[[3.77947186]
[3.08314421]]
</pre></div>
</div>
</div>
@@ -1201,9 +1204,9 @@ theta from own sdg with momentum
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own AdaGrad
[[1.99978312]
[3.00126504]
[3.99880408]]
[[2.00000205]
[2.99998905]
[4.00001059]]
</pre></div>
</div>
</div>
@@ -1285,9 +1288,9 @@ theta from own sdg with momentum
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own RMSprop
[[1.99827523]
[2.99976195]
[3.99585312]]
[[1.99957289]
[3.00371103]
[3.99595959]]
</pre></div>
</div>
</div>
@@ -1373,9 +1376,9 @@ theta from own sdg with momentum
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own ADAM
[[2.00000058]
[2.99998203]
[4.00001412]]
[[2.00001509]
[2.99992841]
[4.00006811]]
</pre></div>
</div>
</div>
@@ -1448,7 +1451,7 @@ It provides composable transformations of Python+NumPy programs: differentiate,
return asarray(x, dtype=self.dtype)
</pre></div>
</div>
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[&lt;matplotlib.lines.Line2D at 0x11d60e820&gt;]
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[&lt;matplotlib.lines.Line2D at 0x139592670&gt;]
</pre></div>
</div>
<img alt="_images/02e94753795fba95a52acc4488a9ab82172ba372cbb7c1cc79a3e88c3ba03d69.png" src="_images/02e94753795fba95a52acc4488a9ab82172ba372cbb7c1cc79a3e88c3ba03d69.png" />
@@ -1483,7 +1486,7 @@ It provides composable transformations of Python+NumPy programs: differentiate,
</div>
</div>
<div class="cell_output docutils container">
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;matplotlib.collections.PathCollection at 0x107a83760&gt;
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;matplotlib.collections.PathCollection at 0x1394faca0&gt;
</pre></div>
</div>
<img alt="_images/60f34a975702165b9275820e0dda6259f453a92c5ad587b37191bb47a9c67ae9.png" src="_images/60f34a975702165b9275820e0dda6259f453a92c5ad587b37191bb47a9c67ae9.png" />