lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

update lecture notes

Browse Source
This commit is contained in:
Morten Hjorth-Jensen
2024-10-28 06:15:26 +01:00
parent d58daeae80
commit d6bb9335d0
158 changed files with 29822 additions and 2310 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/LectureNotes/_build/html/week40.html
+127 -119
View File
@@ -348,6 +348,11 @@ const thebe_selector_output = ".output, .cell_output"
Exercises week 43
</a>
</li>
<li class="toctree-l1">
<a class="reference internal" href="week44.html">
Week 44, Convolutional Neural Networks (CNN)
</a>
</li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
@@ -1329,17 +1334,17 @@ We summarize some of these here for the methods we hvae studied in project one,
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Parameters for OLS using gradient descent
[[3.79341574]
[3.45532899]
[4.79869224]]
[[3.52649344]
[4.28384131]
[4.39301381]]
Parameters for Ridge using gradient descent
[[3.66467908]
[3.67744448]
[4.69668411]]
[[3.85580661]
[3.22264403]
[4.90636879]]
Parameters for Lasso using gradient descent
[[3.8936154 ]
[3.05785843]
[5.01267038]]
[[3.48209228]
[4.41003194]
[4.32994479]]
</pre></div>
</div>
</div>
@@ -1391,11 +1396,11 @@ Parameters for Lasso using gradient descent
[[4.]
[3.]
[5.]]
0 [-28.98027789] [-37.42420721]
1 [1.31983313e-14] [3.4924028e-14]
2 [7.99360578e-16] [9.55743376e-16]
3 [-1.42108547e-16] [-2.57209333e-16]
4 [-1.59872116e-16] [6.89684207e-17]
0 [-27.32010544] [-38.1343232]
1 [-8.47855119e-14] [-7.94623681e-14]
2 [-8.17124146e-16] [-1.01009372e-15]
3 [-7.28306304e-16] [-1.47279537e-15]
4 [-8.17124146e-16] [-1.01009372e-15]
beta from own Newton code
[[4.]
[3.]
@@ -1737,15 +1742,15 @@ function.</p>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
[[3.8125468]
[3.3410149]]
Eigenvalues of Hessian Matrix:[0.30966921 4.24520733]
[[3.97648396]
[3.02497282]]
Eigenvalues of Hessian Matrix:[0.33604933 4.1740886 ]
theta from own gd
[[3.8125468]
[3.3410149]]
[[3.97648396]
[3.02497282]]
theta from own sdg
[[3.76103892]
[3.31388878]]
[[3.93398716]
[3.06433206]]
</pre></div>
</div>
<img alt="_images/week40_34_1.png" src="_images/week40_34_1.png" />
@@ -2459,12 +2464,12 @@ first example shows results with ordinary leats squares.</p>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
[[4.14040757]
[2.84482599]]
Eigenvalues of Hessian Matrix:[0.29095968 3.92422684]
[[4.12726334]
[2.9658822 ]]
Eigenvalues of Hessian Matrix:[0.31252573 4.38126738]
theta from own gd
[[4.14040757]
[2.84482599]]
[[4.12726334]
[2.9658822 ]]
</pre></div>
</div>
<img alt="_images/week40_100_1.png" src="_images/week40_100_1.png" />
@@ -2535,73 +2540,76 @@ theta from own gd
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
[[4.]
[3.]]
Eigenvalues of Hessian Matrix:[0.28379561 4.4050219 ]
0 [-10.19786399] [-11.56308108]
1 [-0.23478113] [0.19832989]
2 [-0.21965525] [0.1855524]
3 [-0.20550386] [0.1735981]
4 [-0.19226417] [0.16241396]
5 [-0.17987746] [0.15195036]
6 [-0.16828877] [0.14216089]
7 [-0.15744669] [0.13300211]
8 [-0.14730311] [0.12443338]
9 [-0.13781304] [0.1164167]
10 [-0.12893437] [0.1089165]
11 [-0.12062771] [0.10189951]
12 [-0.11285621] [0.09533458]
13 [-0.1055854] [0.08919261]
14 [-0.09878301] [0.08344633]
15 [-0.09241887] [0.07807026]
16 [-0.08646474] [0.07304055]
17 [-0.08089421] [0.06833488]
18 [-0.07568256] [0.06393237]
19 [-0.07080668] [0.0598135]
20 [-0.06624492] [0.05595998]
21 [-0.06197706] [0.05235474]
22 [-0.05798416] [0.04898176]
23 [-0.0542485] [0.04582608]
24 [-0.05075352] [0.04287372]
25 [-0.0474837] [0.04011156]
26 [-0.04442454] [0.03752735]
27 [-0.04156247] [0.03510963]
28 [-0.03888479] [0.03284768]
29 [-0.03637961] [0.03073145]
Eigenvalues of Hessian Matrix:[0.28925252 4.30058147]
0 [-11.3166934] [-13.25217117]
1 [0.05941571] [-0.05123603]
2 [0.05541947] [-0.04778995]
3 [0.05169202] [-0.04457564]
4 [0.04821527] [-0.04157753]
5 [0.04497236] [-0.03878107]
6 [0.04194757] [-0.0361727]
7 [0.03912622] [-0.03373976]
8 [0.03649463] [-0.03147046]
9 [0.03404004] [-0.02935379]
10 [0.03175054] [-0.02737949]
11 [0.02961504] [-0.02553797]
12 [0.02762316] [-0.02382032]
13 [0.02576526] [-0.02221819]
14 [0.02403231] [-0.02072382]
15 [0.02241593] [-0.01932995]
16 [0.02090825] [-0.01802984]
17 [0.01950199] [-0.01681717]
18 [0.0181903] [-0.01568607]
19 [0.01696684] [-0.01463104]
20 [0.01582567] [-0.01364697]
21 [0.01476125] [-0.01272909]
22 [0.01376843] [-0.01187295]
23 [0.01284238] [-0.01107438]
24 [0.01197861] [-0.01032953]
25 [0.01117294] [-0.00963478]
26 [0.01042146] [-0.00898675]
27 [0.00972053] [-0.00838232]
28 [0.00906674] [-0.00781853]
29 [0.00845692] [-0.00729266]
theta from own gd
[[3.88006918]
[3.10131081]]
0 [-0.03403584] [0.02875156]
1 [-0.03184307] [0.02689923]
2 [-0.02913373] [0.02461054]
3 [-0.02644397] [0.02233838]
4 [-0.02393338] [0.02021757]
5 [-0.02163828] [0.01827881]
6 [-0.0195557] [0.01651955]
7 [-0.01767104] [0.0149275]
8 [-0.01596717] [0.01348817]
9 [-0.01442732] [0.01218739]
10 [-0.01303588] [0.01101198]
11 [-0.0117786] [0.0099499]
12 [-0.01064258] [0.00899025]
13 [-0.00961612] [0.00812316]
14 [-0.00868866] [0.00733969]
15 [-0.00785065] [0.00663179]
16 [-0.00709346] [0.00599216]
17 [-0.00640931] [0.00541422]
18 [-0.00579114] [0.00489203]
19 [-0.00523259] [0.0044202]
20 [-0.00472792] [0.00399388]
21 [-0.00427191] [0.00360867]
22 [-0.00385989] [0.00326062]
23 [-0.00348761] [0.00294614]
24 [-0.00315124] [0.00266199]
25 [-0.0028473] [0.00240524]
26 [-0.00257269] [0.00217326]
27 [-0.00232455] [0.00196365]
28 [-0.00210035] [0.00177426]
29 [-0.00189778] [0.00160314]
[[4.02727068]
[2.97648364]]
0 [0.00788811] [-0.00680217]
1 [0.00735757] [-0.00634466]
2 [0.00670354] [-0.00578067]
3 [0.00605646] [-0.00522268]
4 [0.00545499] [-0.004704]
5 [0.00490765] [-0.00423202]
6 [0.00441336] [-0.00380578]
7 [0.00396824] [-0.00342194]
8 [0.0035678] [-0.00307663]
9 [0.0032077] [-0.0027661]
10 [0.00288393] [-0.0024869]
11 [0.00259283]
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[-0.00223587]
12 [0.0023311] [-0.00201018]
13 [0.0020958] [-0.00180727]
14 [0.00188425] [-0.00162485]
15 [0.00169405] [-0.00146083]
16 [0.00152305] [-0.00131337]
17 [0.00136931] [-0.0011808]
18 [0.00123109] [-0.00106161]
19 [0.00110682] [-0.00095445]
20 [0.0009951] [-0.00085811]
21 [0.00089465] [-0.00077149]
22 [0.00080435] [-0.00069361]
23 [0.00072315] [-0.0006236]
24 [0.00065016] [-0.00056065]
25 [0.00058453] [-0.00050406]
26 [0.00052553] [-0.00045318]
27 [0.00047248] [-0.00040743]
28 [0.00042479] [-0.00036631]
29 [0.00038191] [-0.00032933]
theta from own gd wth momentum
[[3.99395784]
[3.00510408]]
[[4.00118705]
[2.99897637]]
</pre></div>
</div>
</div>
@@ -2690,18 +2698,20 @@ theta from own gd wth momentum
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
[[3.97904015]
[3.11938084]]
Eigenvalues of Hessian Matrix:[0.29437712 4.50172351]
[[4.0673337 ]
[3.09000242]]
Eigenvalues of Hessian Matrix:[0.26130347 4.78927162]
theta from own gd
[[3.97904015]
[3.11938084]]
[[4.0673337 ]
[3.09000242]]
</pre></div>
</div>
<img alt="_images/week40_104_1.png" src="_images/week40_104_1.png" />
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own sdg
[[3.97069113]
[3.13904707]]
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[4.05489878]
[3.10152451]]
</pre></div>
</div>
</div>
@@ -2783,17 +2793,15 @@ theta from own gd
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
[[4.20902858]
[2.82421714]]
Eigenvalues of Hessian Matrix:[0.29463222 4.67204559]
[[4.03339089]
[2.92919287]]
Eigenvalues of Hessian Matrix:[0.26140984 4.96218268]
theta from own gd
[[4.20862308]
[2.82454109]]
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own sdg with momentum
[[4.13343872]
[2.81165023]]
[[4.02898102]
[2.93257133]]
theta from own sdg with momentum
[[3.97015949]
[2.98377605]]
</pre></div>
</div>
</div>
@@ -2862,9 +2870,9 @@ theta from own gd
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own AdaGrad
[[1.90103664]
[3.54492296]
[3.47989639]]
[[2.00049449]
[2.99756956]
[4.00250108]]
</pre></div>
</div>
</div>
@@ -2940,9 +2948,9 @@ theta from own gd
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own RMSprop
[[1.99858474]
[3.00521037]
[3.99718155]]
[[1.99984033]
[3.00099032]
[3.99898545]]
</pre></div>
</div>
</div>
@@ -3022,9 +3030,9 @@ theta from own gd
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own ADAM
[[2.00002505]
[2.99981314]
[4.00017937]]
[[1.99995089]
[3.0002889 ]
[3.99971485]]
</pre></div>
</div>
</div>
@@ -3145,7 +3153,7 @@ It provides composable transformations of Python+NumPy programs: differentiate,
return asarray(x, dtype=self.dtype)
</pre></div>
</div>
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[&lt;matplotlib.lines.Line2D at 0x117269df0&gt;]
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[&lt;matplotlib.lines.Line2D at 0x1181696a0&gt;]
</pre></div>
</div>
<img alt="_images/week40_120_2.png" src="_images/week40_120_2.png" />
@@ -3180,7 +3188,7 @@ It provides composable transformations of Python+NumPy programs: differentiate,
</div>
</div>
<div class="cell_output docutils container">
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;matplotlib.collections.PathCollection at 0x117195940&gt;
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;matplotlib.collections.PathCollection at 0x1180defa0&gt;
</pre></div>
</div>
<img alt="_images/week40_122_1.png" src="_images/week40_122_1.png" />