lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

update

Browse Source
This commit is contained in:
Morten Hjorth-Jensen
2024-11-25 08:12:27 +01:00
parent d63cb194d5
commit c8f2aa3dc1
181 changed files with 13447 additions and 2592 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/LectureNotes/_build/html/week40.html
+126 -127
View File
@@ -257,6 +257,9 @@
<li class="toctree-l1"><a class="reference internal" href="week47.html">Week 47: From Decision Trees to Ensemble Methods, Random Forests and Boosting Methods</a></li>
<li class="toctree-l1"><a class="reference internal" href="exercisesweek47.html">Exercise week 47</a></li>
<li class="toctree-l1"><a class="reference internal" href="week48.html">Week 48: Gradient boosting and summary of course</a></li>
<li class="toctree-l1"><a class="reference internal" href="exercisesweek48.html">Exercises week 48</a></li>
</ul>
<p aria-level="2" class="caption" role="heading"><span class="caption-text">Projects</span></p>
<ul class="nav bd-sidenav">
@@ -631,17 +634,17 @@ We summarize some of these here for the methods we hvae studied in project one,
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Parameters for OLS using gradient descent
[[3.633198 ]
[4.1182509 ]
[4.44504789]]
[[4.18426686]
[2.51331698]
[5.22403374]]
Parameters for Ridge using gradient descent
[[3.81542861]
[3.39782637]
[4.80980326]]
[[4.13677307]
[2.53440516]
[5.21680493]]
Parameters for Lasso using gradient descent
[[3.47633954]
[4.56277628]
[4.22765626]]
[[3.40725426]
[4.31893412]
[4.43233965]]
</pre></div>
</div>
</div>
@@ -693,11 +696,11 @@ Parameters for Lasso using gradient descent
[[4.]
[3.]
[5.]]
0 [-25.50604212] [-33.67380616]
1 [-1.07718279e-13] [-1.67721558e-13]
2 [-9.23705556e-16] [-1.2228689e-15]
3 [8.70414851e-16] [1.13386647e-15]
4 [-1.86517468e-15] [-2.57274984e-15]
0 [-21.31398212] [-28.05552428]
1 [-3.03046477e-14] [-1.33496431e-14]
2 [-7.99360578e-16] [-9.75256689e-16]
3 [1.77635684e-16] [2.91496906e-16]
4 [1.77635684e-16] [2.91496906e-16]
beta from own Newton code
[[4.]
[3.]
@@ -1039,18 +1042,18 @@ function.</p>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
[[4.06629035]
[2.90434687]]
Eigenvalues of Hessian Matrix:[0.28385837 4.07166128]
[[3.93518583]
[3.05586004]]
Eigenvalues of Hessian Matrix:[0.31020049 4.44279756]
theta from own gd
[[4.06629035]
[2.90434687]]
[[3.93518583]
[3.05586004]]
theta from own sdg
[[4.04098389]
[2.9248133 ]]
[[3.92681541]
[3.07007326]]
</pre></div>
</div>
<img alt="_images/2de542effe405ced9195c07c5d2061f8911ca88584548f85cb6d28df1f83d693.png" src="_images/2de542effe405ced9195c07c5d2061f8911ca88584548f85cb6d28df1f83d693.png" />
<img alt="_images/81f5882581a53e21f2c20be0ef5d63c90956ddfe5d34db4fb8f83012ab7b8c4a.png" src="_images/81f5882581a53e21f2c20be0ef5d63c90956ddfe5d34db4fb8f83012ab7b8c4a.png" />
</div>
</div>
</section>
@@ -1761,15 +1764,17 @@ first example shows results with ordinary leats squares.</p>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
[[4.05658312]
[2.97504928]]
Eigenvalues of Hessian Matrix:[0.31549215 4.38384294]
theta from own gd
[[4.05658312]
[2.97504928]]
</pre></div>
</div>
<img alt="_images/1bf8fa67622cbeaf3f9c469a206d89f5936b2fade360d11439ef4a3e43cf6f31.png" src="_images/1bf8fa67622cbeaf3f9c469a206d89f5936b2fade360d11439ef4a3e43cf6f31.png" />
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[3.95215017]
[2.98138287]]
Eigenvalues of Hessian Matrix:[0.28889272 3.80178359]
theta from own gd
[[3.95215017]
[2.98138287]]
</pre></div>
</div>
<img alt="_images/5a1776c87da1ba203336178287b47b42ec33ec260ba18e3e2bb9d9f8ed8c8134.png" src="_images/5a1776c87da1ba203336178287b47b42ec33ec260ba18e3e2bb9d9f8ed8c8134.png" />
</div>
</div>
</section>
@@ -1837,73 +1842,73 @@ theta from own gd
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
[[4.]
[3.]]
Eigenvalues of Hessian Matrix:[0.32421852 4.40457549]
0 [-8.91626016] [-10.12979569]
1 [-0.25100049] [0.20953861]
2 [-0.23252447] [0.19411458]
3 [-0.21540847] [0.17982591]
4 [-0.19955236] [0.16658901]
5 [-0.18486342] [0.15432648]
6 [-0.17125572] [0.14296659]
7 [-0.15864967] [0.13244289]
8 [-0.14697155] [0.12269384]
9 [-0.13615305] [0.11366241]
10 [-0.1261309] [0.10529578]
11 [-0.11684647] [0.09754501]
12 [-0.10824546] [0.09036477]
13 [-0.10027757] [0.08371307]
14 [-0.09289619] [0.07755099]
15 [-0.08605815] [0.07184251]
16 [-0.07972345] [0.06655422]
17 [-0.07385505] [0.06165519]
18 [-0.06841862] [0.05711679]
19 [-0.06338236] [0.05291245]
20 [-0.05871681] [0.04901759]
21 [-0.0543947] [0.04540943]
22 [-0.05039073] [0.04206687]
23 [-0.0466815] [0.03897035]
24 [-0.0432453] [0.03610176]
25 [-0.04006203] [0.03344433]
26 [-0.03711309] [0.03098251]
27 [-0.03438121] [0.0287019]
28 [-0.03185043] [0.02658917]
29 [-0.02950594] [0.02463195]
Eigenvalues of Hessian Matrix:[0.29823315 4.72747492]
0 [-17.30620124] [-21.78309726]
1 [-0.05759628] [0.04549505]
2 [-0.05396282] [0.04262499]
3 [-0.05055857] [0.03993599]
4 [-0.04736908] [0.03741662]
5 [-0.0443808] [0.03505619]
6 [-0.04158103] [0.03284467]
7 [-0.03895789] [0.03077266]
8 [-0.03650022] [0.02883136]
9 [-0.03419761] [0.02701253]
10 [-0.03204025] [0.02530845]
11 [-0.03001898] [0.02371186]
12 [-0.02812523] [0.022216]
13 [-0.02635095] [0.0208145]
14 [-0.0246886] [0.01950141]
15 [-0.02313112] [0.01827117]
16 [-0.02167189] [0.01711853]
17 [-0.02030472] [0.0160386]
18 [-0.01902379] [0.01502681]
19 [-0.01782367] [0.01407884]
20 [-0.01669927] [0.01319068]
21 [-0.01564579] [0.01235854]
22 [-0.01465878] [0.0115789]
23 [-0.01373403] [0.01084844]
24 [-0.01286761] [0.01016407]
25 [-0.01205586] [0.00952287]
26 [-0.01129531] [0.00892212]
27 [-0.01058275] [0.00835927]
28 [-0.00991513] [0.00783192]
29 [-0.00928964] [0.00733784]
theta from own gd
[[3.9156926 ]
[3.07038096]]
0 [-0.02733402] [0.02281881]
1 [-0.02532198] [0.02113913]
2 [-0.02285443] [0.01907918]
3 [-0.02043186] [0.01705679]
4 [-0.01820111] [0.01519453]
5 [-0.01619211] [0.01351739]
6 [-0.01439752] [0.01201924]
7 [-0.01279935] [0.01068507]
8 [-0.01137774] [0.00949829]
9 [-0.01011375] [0.0084431]
10 [-0.00899009] [0.00750504]
11 [-0.00799123] [0.00667119]
12 [-0.00710334] [0.00592997]
13 [-0.0063141] [0.0052711]
14 [-0.00561255] [0.00468544]
15 [-0.00498895] [0.00416484]
16 [-0.00443464] [0.0037021]
17 [-0.00394191] [0.00329076]
18 [-0.00350393] [0.00292513]
19 [-0.00311462] [0.00260012]
20 [-0.00276855] [0.00231123]
21 [-0.00246094] [0.00205443]
22 [-0.00218751] [0.00182617]
23 [-0.00194446] [0.00162326]
24 [-0.00172842] [0.0014429]
25 [-0.00153637] [0.00128259]
26 [-0.00136567] [0.00114008]
27 [-0.00121393] [0.00101341]
28 [-0.00107905] [0.00090081]
29 [-0.00095916] [0.00080072]
[[3.97081612]
[3.02305221]]
0 [-0.0087036] [0.00687493]
1 [-0.00815453] [0.00644123]
2 [-0.00747538] [0.00590477]
3 [-0.00680005] [0.00537133]
4 [-0.00616847] [0.00487245]
5 [-0.00558986] [0.0044154]
6 [-0.00506364] [0.00399975]
7 [-0.00458633] [0.00362272]
8 [-0.00415381] [0.00328108]
9 [-0.00376201] [0.0029716]
10 [-0.00340714] [0.00269129]
11 [-0.00308574] [0.00243742]
12 [-0.00279466] [0.00220749]
13 [-0.00253103] [0.00199925]
14 [-0.00229228] [0.00181066]
15 [-0.00207604] [0.00163985]
16 [-0.0018802] [0.00148516]
17 [-0.00170284] [0.00134506]
18 [-0.0015422] [0.00121818]
19 [-0.00139672] [0.00110327]
20 [-0.00126497] [0.00099919]
21 [-0.00114564] [0.00090494]
22 [-0.00103757] [0.00081957]
23 [-0.00093969] [0.00074226]
24 [-0.00085105] [0.00067224]
25 [-0.00077077] [0.00060883]
26 [-0.00069806] [0.00055139]
27 [-0.00063221] [0.00049938]
28 [-0.00057257] [0.00045227]
29 [-0.00051856] [0.00040961]
theta from own gd wth momentum
[[3.99737032]
[3.00219529]]
[[3.99842525]
[3.00124389]]
</pre></div>
</div>
</div>
@@ -1992,22 +1997,20 @@ theta from own gd wth momentum
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
[[3.80284827]
[3.14558948]]
Eigenvalues of Hessian Matrix:[0.33477228 4.32758606]
[[3.93505095]
[3.03255202]]
Eigenvalues of Hessian Matrix:[0.35715186 4.00152151]
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own gd
[[3.93505095]
[3.03255202]]
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[3.80284827]
[3.14558948]]
</pre></div>
</div>
<img alt="_images/0fe401714ec88e67e8834af30a476bb19ae5e8a9677eb38e4161007bae099f84.png" src="_images/0fe401714ec88e67e8834af30a476bb19ae5e8a9677eb38e4161007bae099f84.png" />
<img alt="_images/327366ea5051007f61df82fc528a36028c298264f7cb3d1565bb97cd4c20255c.png" src="_images/327366ea5051007f61df82fc528a36028c298264f7cb3d1565bb97cd4c20255c.png" />
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own sdg
[[3.8286293 ]
[3.19901555]]
[[3.93535203]
[3.02339886]]
</pre></div>
</div>
</div>
@@ -2089,19 +2092,15 @@ Eigenvalues of Hessian Matrix:[0.33477228 4.32758606]
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
[[4.14809298]
[2.84983419]]
Eigenvalues of Hessian Matrix:[0.35213219 4.34352956]
[[4.00794132]
[3.02667095]]
Eigenvalues of Hessian Matrix:[0.29709966 4.68284888]
theta from own gd
[[4.14764332]
[2.85021125]]
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own sdg with momentum
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[4.15364228]
[2.8846002 ]]
[[4.0068889 ]
[3.02750941]]
theta from own sdg with momentum
[[4.01544241]
[3.0176573 ]]
</pre></div>
</div>
</div>
@@ -2170,9 +2169,9 @@ theta from own gd
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own AdaGrad
[[2.00000539]
[2.99996887]
[4.0000263 ]]
[[1.99856653]
[3.01053812]
[3.99013096]]
</pre></div>
</div>
</div>
@@ -2248,9 +2247,9 @@ theta from own gd
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own RMSprop
[[2.00051557]
[2.99775568]
[4.00224621]]
[[2.00002452]
[2.99973034]
[4.00026406]]
</pre></div>
</div>
</div>
@@ -2330,9 +2329,9 @@ theta from own gd
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own ADAM
[[1.99997454]
[3.00018485]
[3.99980356]]
[[1.99997633]
[3.00007035]
[3.99994513]]
</pre></div>
</div>
</div>
@@ -2453,7 +2452,7 @@ It provides composable transformations of Python+NumPy programs: differentiate,
return asarray(x, dtype=self.dtype)
</pre></div>
</div>
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[&lt;matplotlib.lines.Line2D at 0x1109cc4f0&gt;]
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[&lt;matplotlib.lines.Line2D at 0x1323c4070&gt;]
</pre></div>
</div>
<img alt="_images/02e94753795fba95a52acc4488a9ab82172ba372cbb7c1cc79a3e88c3ba03d69.png" src="_images/02e94753795fba95a52acc4488a9ab82172ba372cbb7c1cc79a3e88c3ba03d69.png" />
@@ -2488,7 +2487,7 @@ It provides composable transformations of Python+NumPy programs: differentiate,
</div>
</div>
<div class="cell_output docutils container">
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;matplotlib.collections.PathCollection at 0x110957c40&gt;
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;matplotlib.collections.PathCollection at 0x133fa12e0&gt;
</pre></div>
</div>
<img alt="_images/60f34a975702165b9275820e0dda6259f453a92c5ad587b37191bb47a9c67ae9.png" src="_images/60f34a975702165b9275820e0dda6259f453a92c5ad587b37191bb47a9c67ae9.png" />