lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

update

Browse Source
This commit is contained in:
Morten Hjorth-Jensen
2024-10-14 09:13:10 +02:00
parent f1f5b33d5e
commit a7012b5326
140 changed files with 3712 additions and 3488 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/LectureNotes/_build/html/week40.html
+132 -124
View File
@@ -323,6 +323,16 @@ const thebe_selector_output = ".output, .cell_output"
Week 41 Neural networks and constructing a neural network code
</a>
</li>
<li class="toctree-l1">
<a class="reference internal" href="exercisesweek42.html">
Exercises week 42
</a>
</li>
<li class="toctree-l1">
<a class="reference internal" href="week42.html">
Week 42 Constructing a Neural Network code with examples
</a>
</li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
@@ -1304,17 +1314,17 @@ We summarize some of these here for the methods we hvae studied in project one,
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Parameters for OLS using gradient descent
[[4.26604611]
[2.29234681]
[5.33329216]]
[[3.8184887 ]
[3.47851966]
[4.77551387]]
Parameters for Ridge using gradient descent
[[3.6161104 ]
[3.78762558]
[4.65410649]]
[[3.92021197]
[3.11388017]
[4.9458396 ]]
Parameters for Lasso using gradient descent
[[4.24335376]
[2.36219301]
[5.29669411]]
[[3.87323528]
[3.3008836 ]
[4.86284277]]
</pre></div>
</div>
</div>
@@ -1366,11 +1376,11 @@ Parameters for Lasso using gradient descent
[[4.]
[3.]
[5.]]
0 [-26.28886314] [-34.64721597]
1 [-1.83231208e-13] [-1.80848093e-13]
2 [6.92779167e-16] [1.22835717e-15]
3 [-1.3500312e-15] [-1.8110093e-15]
4 [7.46069873e-16] [9.30442002e-16]
0 [-31.91417133] [-44.48017159]
1 [3.48805429e-13] [4.67477123e-13]
2 [6.75015599e-16] [1.30675215e-15]
3 [-1.17239551e-15] [-1.78477759e-15]
4 [6.75015599e-16] [1.30675215e-15]
beta from own Newton code
[[4.]
[3.]
@@ -1712,15 +1722,15 @@ function.</p>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
[[4.36014743]
[2.76030841]]
Eigenvalues of Hessian Matrix:[0.36278226 3.73204357]
[[3.90300704]
[3.16913489]]
Eigenvalues of Hessian Matrix:[0.2964378 4.12443871]
theta from own gd
[[4.36014743]
[2.76030841]]
[[3.90300704]
[3.16913489]]
theta from own sdg
[[4.34582863]
[2.81684805]]
[[3.93272428]
[3.16328315]]
</pre></div>
</div>
<img alt="_images/week40_34_1.png" src="_images/week40_34_1.png" />
@@ -2434,12 +2444,12 @@ first example shows results with ordinary leats squares.</p>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
[[3.85068028]
[3.09808149]]
Eigenvalues of Hessian Matrix:[0.31897935 3.95770298]
[[4.13791264]
[2.92552059]]
Eigenvalues of Hessian Matrix:[0.27874136 4.16226023]
theta from own gd
[[3.85068028]
[3.09808149]]
[[4.13791264]
[2.92552059]]
</pre></div>
</div>
<img alt="_images/week40_100_1.png" src="_images/week40_100_1.png" />
@@ -2508,77 +2518,75 @@ theta from own gd
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[[4.]
[[4.]
[3.]]
Eigenvalues of Hessian Matrix:[0.35713539 3.88632765]
0 [-9.01615836] [-9.6932681]
1 [0.01454469] [-0.01357366]
2 [0.0132081] [-0.0123263]
3 [0.01199434] [-0.01119357]
4 [0.01089212] [-0.01016493]
5 [0.00989118] [-0.00923082]
6 [0.00898223] [-0.00838255]
7 [0.0081568] [-0.00761224]
8 [0.00740723] [-0.00691271]
9 [0.00672654] [-0.00627746]
10 [0.0061084] [-0.00570059]
11 [0.00554707] [-0.00517673]
12 [0.00503732] [-0.00470102]
13 [0.00457441] [-0.00426902]
14 [0.00415405] [-0.00387671]
15 [0.00377231] [-0.00352046]
16 [0.00342565] [-0.00319695]
17 [0.00311085] [-0.00290316]
18 [0.00282498] [-0.00263638]
19 [0.00256537] [-0.0023941]
20 [0.00232963] [-0.0021741]
21 [0.00211555] [-0.00197431]
22 [0.00192114] [-0.00179288]
23 [0.00174459] [-0.00162812]
24 [0.00158427] [-0.0014785]
25 [0.00143869] [-0.00134264]
26 [0.00130648] [-0.00121925]
27 [0.00118642] [-0.00110721]
28 [0.00107739] [-0.00100546]
29 [0.00097839] [-0.00091307]
Eigenvalues of Hessian Matrix:[0.32606365 3.80499859]
0 [-10.98955596] [-10.8332972]
1 [-0.26421616] [0.25444295]
2 [-0.24157456] [0.23263884]
3 [-0.22087319] [0.2127032]
4 [-0.20194579] [0.19447592]
5 [-0.18464035] [0.1778106]
6 [-0.16881787] [0.16257339]
7 [-0.15435127] [0.1486419]
8 [-0.14112437] [0.13590426]
9 [-0.12903093] [0.12425815]
10 [-0.11797382] [0.11361003]
11 [-0.10786423] [0.10387439]
12 [-0.09862097] [0.09497303]
13 [-0.09016979] [0.08683446]
14 [-0.08244283] [0.07939331]
15 [-0.07537801] [0.07258982]
16 [-0.06891861] [0.06636934]
17 [-0.06301273] [0.06068192]
18 [-0.05761295] [0.05548187]
19 [-0.05267589] [0.05072744]
20 [-0.04816191] [0.04638043]
21 [-0.04403475] [0.04240593]
22 [-0.04026126] [0.03877201]
23 [-0.03681113] [0.0354495]
24 [-0.03365665] [0.03241171]
25 [-0.0307725] [0.02963424]
26 [-0.02813549] [0.02709478]
27 [-0.02572446] [0.02477293]
28 [-0.02352005] [0.02265005]
29 [-0.02150453] [0.02070909]
theta from own gd
[[4.00248779]
[2.9976783 ]]
0 [0.00088848] [-0.00082916]
1 [0.00080683] [-0.00075296]
2 [0.00070819] [-0.00066091]
3 [0.00061352] [-0.00057256]
4 [0.00052874] [-0.00049344]
5 [0.00045472] [-0.00042436]
6 [0.00039072] [-0.00036464]
7 [0.00033562] [-0.00031321]
8 [0.00028825] [-0.000269]
9 [0.00024755] [-0.00023102]
10 [0.00021259] [-0.0001984]
11 [0.00018256] [-0.00017038]
12 [0.00015678] [-0.00014631]
13 [0.00013464] [-0.00012565]
14 [0.00011562] [-0.0001079]
15 [9.92929225e-05] [-9.26639089e-05]
16 [8.52694246e-05] [-7.95766504e-05]
17 [7.32265127e-05] [-6.83377498e-05]
18 [6.28844641e-05] [-5.86861591e-05]
19 [5.40030606e-05] [-5.03976976e-05]
20 [4.63760101e-05] [-4.32798457e-05]
21 [3.98261559e-05] [-3.71672742e-05]
22 [3.42013616e-05] [-3.19180036e-05]
23 [2.93709777e-05] [-2.74101067e-05]
24 [2.52228066e-05] [-2.35388766e-05]
25 [2.1660497e-05] [-2.02143946e-05]
26 [1.86013055e-05] [-1.73594414e-05]
27 [1.59741748e-05] [-1.49077038e-05]
28 [1.37180834e-05] [-1.2802234e-05]
29 [1.17806281e-05] [-1.09941275e-05]
[[3.93969971]
[3.05806981]]
0 [-0.01966173] [0.01893445]
1 [-0.01797685] [0.0173119]
2 [-0.01593089] [0.01534161]
3 [-0.01395192] [0.01343585]
4 [-0.01216264] [0.01171276]
5 [-0.0105836] [0.01019212]
6 [-0.00920294] [0.00886253]
7 [-0.00800011] [0.00770419]
8 [-0.00695371] [0.00669649]
9 [-0.0060439] [0.00582034]
10 [-0.00525303] [0.00505872]
11 [-0.00456562] [0.00439674]
12 [-0.00396815] [0.00382137]
13 [-0.00344887] [0.0033213]
14 [-0.00299754] [0.00288666]
15 [-0.00260527] [0.0025089]
16 [-0.00226433] [0.00218058]
17 [-0.00196801] [0.00189522]
18 [-0.00171047] [0.0016472]
19 [-0.00148663] [0.00143164]
20 [-0.00129209] [0.00124429]
21 [-0.001123] [0.00108146]
22 [-0.00097604] [0.00093994]
23 [-0.00084831] [0.00081693]
24 [-0.0007373] [0.00071003]
25 [-0.00064081] [0.00061711]
26 [-0.00055695] [0.00053635]
27 [-0.00048407] [0.00046616]
28 [-0.00042072] [0.00040516]
29 [-0.00036566] [0.00035214]
theta from own gd wth momentum
[[4.00002833]
[2.99997356]]
[[3.99902531]
[3.00093864]]
</pre></div>
</div>
</div>
@@ -2667,20 +2675,18 @@ theta from own gd wth momentum
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
[[3.91650453]
[2.94495682]]
Eigenvalues of Hessian Matrix:[0.34862407 4.10453899]
[[4.11762444]
[3.04098313]]
Eigenvalues of Hessian Matrix:[0.29738252 4.51279273]
theta from own gd
[[4.11762444]
[3.04098313]]
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own gd
[[3.91650453]
[2.94495682]]
</pre></div>
</div>
<img alt="_images/week40_104_2.png" src="_images/week40_104_2.png" />
<img alt="_images/week40_104_1.png" src="_images/week40_104_1.png" />
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own sdg
[[3.8901199 ]
[2.92458892]]
[[4.07058967]
[3.024004 ]]
</pre></div>
</div>
</div>
@@ -2762,15 +2768,17 @@ Eigenvalues of Hessian Matrix:[0.34862407 4.10453899]
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
[[4.05785974]
[2.95842106]]
Eigenvalues of Hessian Matrix:[0.29678339 4.37215356]
[[4.4252885 ]
[2.70944365]]
Eigenvalues of Hessian Matrix:[0.28973035 4.32089655]
theta from own gd
[[4.05738629]
[2.95882223]]
theta from own sdg with momentum
[[4.07489511]
[2.90281987]]
[[4.42394588]
[2.71059619]]
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own sdg with momentum
[[4.44845593]
[2.72577807]]
</pre></div>
</div>
</div>
@@ -2839,9 +2847,9 @@ theta from own sdg with momentum
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own AdaGrad
[[1.99994537]
[3.00034209]
[3.99966798]]
[[2.00036797]
[2.99817613]
[4.00177485]]
</pre></div>
</div>
</div>
@@ -2917,9 +2925,9 @@ theta from own sdg with momentum
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own RMSprop
[[1.99975636]
[3.00348281]
[3.99607299]]
[[2.00119865]
[3.01346635]
[3.99284588]]
</pre></div>
</div>
</div>
@@ -2999,9 +3007,9 @@ theta from own sdg with momentum
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own ADAM
[[2.00002276]
[2.99985884]
[4.00009004]]
[[1.99997244]
[3.00018876]
[3.99983332]]
</pre></div>
</div>
</div>
@@ -3122,7 +3130,7 @@ It provides composable transformations of Python+NumPy programs: differentiate,
return asarray(x, dtype=self.dtype)
</pre></div>
</div>
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[&lt;matplotlib.lines.Line2D at 0x11bcee130&gt;]
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[&lt;matplotlib.lines.Line2D at 0x10cc52cd0&gt;]
</pre></div>
</div>
<img alt="_images/week40_120_2.png" src="_images/week40_120_2.png" />
@@ -3157,7 +3165,7 @@ It provides composable transformations of Python+NumPy programs: differentiate,
</div>
</div>
<div class="cell_output docutils container">
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;matplotlib.collections.PathCollection at 0x11bdd4640&gt;
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;matplotlib.collections.PathCollection at 0x10fcfaeb0&gt;
</pre></div>
</div>
<img alt="_images/week40_122_1.png" src="_images/week40_122_1.png" />