lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

added video

Browse Source
This commit is contained in:
Morten Hjorth-Jensen
2024-09-30 13:11:27 +02:00
parent 31d04e7e60
commit e8b71b6012
14 changed files with 2561 additions and 2079 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/LectureNotes/_build/html/week40.html
+119 -118
View File
@@ -1197,9 +1197,10 @@ doconce format html week40.do.txt --no_mako -->
<ol class="simple">
<li><p>Stochastic Gradient descent with examples and automatic differentiation</p></li>
<li><p>If we get time, we start with the basics of Neural Networks, setting up the basic steps, from the simple perceptron model to the multi-layer perceptron model</p></li>
<li><p><a class="reference external" href="https://youtu.be/jdJoOrCIdII">Video of lecture</a></p></li>
<li><p>Whiteboard notes at <a class="reference external" href="https://github.com/CompPhysics/MachineLearning/blob/master/doc/HandWrittenNotes/2024/NotesSeptember30.pdf">https://github.com/CompPhysics/MachineLearning/blob/master/doc/HandWrittenNotes/2024/NotesSeptember30.pdf</a></p></li>
</ol>
<!-- * [Video of lecture](https://youtu.be/75pr3hKY20U) -->
<!-- * "Whiteboard notes at <https://github.com/CompPhysics/MachineLearning/blob/master/doc/HandWrittenNotes/2023/NotesOct5.pdf> --></div>
</div>
<div class="section" id="suggested-readings-and-videos">
<h2>Suggested readings and videos<a class="headerlink" href="#suggested-readings-and-videos" title="Permalink to this headline"></a></h2>
<p><strong>Readings and Videos:</strong></p>
@@ -1288,17 +1289,17 @@ We summarize some of these here for the methods we hvae studied in project one,
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Parameters for OLS using gradient descent
[[3.870337 ]
[3.34871778]
[4.83363025]]
[[3.53732988]
[4.06277313]
[4.51728978]]
Parameters for Ridge using gradient descent
[[3.66951679]
[3.76942272]
[4.63546236]]
[[3.62671667]
[3.79849321]
[4.63207408]]
Parameters for Lasso using gradient descent
[[3.65759901]
[3.87732748]
[4.58735893]]
[[4.1547556 ]
[2.58013829]
[5.20288581]]
</pre></div>
</div>
</div>
@@ -1350,11 +1351,11 @@ Parameters for Lasso using gradient descent
[[4.]
[3.]
[5.]]
0 [-23.78254537] [-30.51512235]
1 [1.30917499e-14] [-1.01327148e-14]
2 [9.76996262e-16] [1.28164443e-15]
3 [-7.28306304e-16] [-8.17359583e-16]
4 [-7.63833441e-16] [-1.32422234e-15]
0 [-27.10587277] [-37.81035098]
1 [1.16209264e-13] [2.08824304e-13]
2 [8.8817842e-17] [2.62242138e-16]
3 [-1.77635684e-17] [-1.50600514e-16]
4 [-3.37507799e-16] [-2.280464e-16]
beta from own Newton code
[[4.]
[3.]
@@ -1696,15 +1697,15 @@ function.</p>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
[[4.12220276]
[2.95115528]]
Eigenvalues of Hessian Matrix:[0.31601423 3.93074023]
[[4.30637636]
[2.56947078]]
Eigenvalues of Hessian Matrix:[0.30000613 4.12600095]
theta from own gd
[[4.12220276]
[2.95115528]]
[[4.30637636]
[2.56947078]]
theta from own sdg
[[4.12206013]
[2.92370959]]
[[4.34759233]
[2.63142731]]
</pre></div>
</div>
<img alt="_images/week40_34_1.png" src="_images/week40_34_1.png" />
@@ -2418,12 +2419,12 @@ first example shows results with ordinary leats squares.</p>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
[[3.62286539]
[3.36754671]]
Eigenvalues of Hessian Matrix:[0.30134885 4.08702578]
[[3.88064505]
[2.99072374]]
Eigenvalues of Hessian Matrix:[0.30141906 4.73853838]
theta from own gd
[[3.62286539]
[3.36754671]]
[[3.88064505]
[2.99072374]]
</pre></div>
</div>
<img alt="_images/week40_100_1.png" src="_images/week40_100_1.png" />
@@ -2494,73 +2495,73 @@ theta from own gd
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
[[4.]
[3.]]
Eigenvalues of Hessian Matrix:[0.27124938 4.56804783]
0 [-8.64009263] [-10.94092702]
1 [0.18925625] [-0.15527976]
2 [0.17801827] [-0.14605929]
3 [0.16744759] [-0.13738633]
4 [0.1575046] [-0.12922837]
5 [0.14815202] [-0.12155483]
6 [0.1393548] [-0.11433693]
7 [0.13107995] [-0.10754764]
8 [0.12329646] [-0.10116149]
9 [0.11597515] [-0.09515455]
10 [0.10908858] [-0.0895043]
11 [0.10261093] [-0.08418956]
12 [0.09651792] [-0.07919041]
13 [0.09078672] [-0.0744881]
14 [0.08539583] [-0.07006502]
15 [0.08032505] [-0.06590458]
16 [0.07555537] [-0.06199119]
17 [0.07106891] [-0.05831017]
18 [0.06684886] [-0.05484773]
19 [0.06287939] [-0.05159088]
20 [0.05914563] [-0.04852743]
21 [0.05563358] [-0.04564589]
22 [0.05233008] [-0.04293545]
23 [0.04922273] [-0.04038595]
24 [0.0462999] [-0.03798785]
25 [0.04355062] [-0.03573214]
26 [0.0409646] [-0.03361037]
27 [0.03853213] [-0.0316146]
28 [0.03624411] [-0.02973733]
29 [0.03409194] [-0.02797154]
Eigenvalues of Hessian Matrix:[0.31702609 3.84351715]
0 [-13.26083712] [-12.67834752]
1 [-0.55020405] [0.5257011]
2 [-0.50482139] [0.48233952]
3 [-0.46318204] [0.44255455]
4 [-0.42497724] [0.40605118]
5 [-0.38992371] [0.37255873]
6 [-0.3577615] [0.34182884]
7 [-0.32825214] [0.31363366]
8 [-0.30117681] [0.28776411]
9 [-0.27633474] [0.26402837]
10 [-0.25354173] [0.24225043]
11 [-0.23262877] [0.22226881]
12 [-0.21344077] [0.20393534]
13 [-0.19583547] [0.18711407]
14 [-0.17968231] [0.17168028]
15 [-0.16486151] [0.15751952]
16 [-0.15126318] [0.14452678]
17 [-0.13878649] [0.13260573]
18 [-0.12733892] [0.12166797]
19 [-0.11683558] [0.11163239]
20 [-0.10719859] [0.10242458]
21 [-0.0983565] [0.09397626]
22 [-0.09024373] [0.08622478]
23 [-0.08280012] [0.07911268]
24 [-0.07597049] [0.0725872]
25 [-0.06970419] [0.06659997]
26 [-0.06395476] [0.06110658]
27 [-0.05867956] [0.0560663]
28 [-0.05383947] [0.05144177]
29 [-0.04939861] [0.04719868]
theta from own gd
[[4.11822173]
[2.90300219]]
0 [0.03206757] [-0.0263106]
1 [0.03016341] [-0.02474828]
2 [0.02780107] [-0.02281004]
3 [0.02544154] [-0.02087411]
4 [0.02322297] [-0.01905384]
5 [0.02117843] [-0.01737634]
6 [0.0193075] [-0.01584129]
7 [0.01759974] [-0.01444013]
8 [0.01604235] [-0.01316233]
9 [0.01462254] [-0.01199741]
10 [0.01332832] [-0.01093553]
11 [0.01214862] [-0.00996762]
12 [0.01107333] [-0.00908537]
13 [0.01009321] [-0.00828121]
14 [0.00919984] [-0.00754823]
15 [0.00838555] [-0.00688012]
16 [0.00764333] [-0.00627115]
17 [0.0069668] [-0.00571608]
18 [0.00635016] [-0.00521014]
19 [0.00578809] [-0.00474898]
20 [0.00527578] [-0.00432864]
21 [0.00480881] [-0.0039455]
22 [0.00438318] [-0.00359628]
23 [0.00399521] [-0.00327797]
24 [0.00364159] [-0.00298783]
25 [0.00331927] [-0.00272337]
26 [0.00302547] [-0.00248232]
27 [0.00275768] [-0.0022626]
28 [0.00251359] [-0.00206234]
29 [0.00229111] [-0.0018798]
[[3.85703369]
[3.13659941]]
0 [-0.04532405] [0.04330558]
1 [-0.04158557] [0.03973359]
2 [-0.03703391] [0.03538463]
3 [-0.03261373] [0.0311613]
4 [-0.02859759] [0.02732402]
5 [-0.02503392] [0.02391906]
6 [-0.02189994] [0.02092464]
7 [-0.01915337] [0.01830039]
8 [-0.01674956] [0.01600363]
9 [-0.01464686] [0.01399457]
10 [-0.01280793] [0.01223754]
11 [-0.01119981] [0.01070103]
12 [-0.00979357] [0.00935742]
13 [-0.0085639] [0.00818251]
14 [-0.00748862] [0.00715512]
15 [-0.00654835] [0.00625672]
16 [-0.00572613] [0.00547113]
17 [-0.00500716] [0.00478417]
18 [-0.00437846] [0.00418347]
19 [-0.0038287] [0.00365819]
20 [-0.00334797] [0.00319887]
21 [-0.0029276] [0.00279722]
22 [-0.00256001] [0.002446]
23 [-0.00223857] [0.00213888]
24 [-0.0019575] [0.00187032]
25 [-0.00171171] [0.00163548]
26 [-0.00149679] [0.00143013]
27 [-0.00130885] [0.00125057]
28 [-0.00114451] [0.00109354]
29 [-0.00100081] [0.00095624]
theta from own gd wth momentum
[[4.0076989 ]
[2.99368326]]
[[3.99723951]
[3.00263755]]
</pre></div>
</div>
</div>
@@ -2649,18 +2650,18 @@ theta from own gd wth momentum
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
[[3.4870934 ]
[3.55042779]]
Eigenvalues of Hessian Matrix:[0.28793787 4.56453869]
[[3.95935439]
[3.09360113]]
Eigenvalues of Hessian Matrix:[0.32606139 4.1033455 ]
theta from own gd
[[3.4870934 ]
[3.55042779]]
[[3.95935439]
[3.09360113]]
</pre></div>
</div>
<img alt="_images/week40_104_1.png" src="_images/week40_104_1.png" />
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own sdg
[[3.48900413]
[3.56763673]]
[[3.96916206]
[3.10276112]]
</pre></div>
</div>
</div>
@@ -2742,15 +2743,15 @@ theta from own gd
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
[[4.11571507]
[2.86067818]]
Eigenvalues of Hessian Matrix:[0.25849623 4.84605507]
[[4.16532977]
[2.86943859]]
Eigenvalues of Hessian Matrix:[0.29633608 4.22128142]
theta from own gd
[[4.10888197]
[2.86602331]]
[[4.16445858]
[2.87020155]]
theta from own sdg with momentum
[[4.15361856]
[2.92376655]]
[[4.149433 ]
[2.89654756]]
</pre></div>
</div>
</div>
@@ -2819,9 +2820,9 @@ theta from own sdg with momentum
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own AdaGrad
[[1.99968451]
[3.001795 ]
[3.99848215]]
[[1.99415296]
[3.03274273]
[3.96850191]]
</pre></div>
</div>
</div>
@@ -2897,9 +2898,9 @@ theta from own sdg with momentum
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own RMSprop
[[1.99887767]
[3.01261048]
[3.99294056]]
[[2.00022969]
[2.99954278]
[4.00046631]]
</pre></div>
</div>
</div>
@@ -2979,9 +2980,9 @@ theta from own sdg with momentum
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own ADAM
[[1.99996755]
[3.00016535]
[3.99977375]]
[[1.99991245]
[3.00042721]
[3.99954434]]
</pre></div>
</div>
</div>
@@ -3102,7 +3103,7 @@ It provides composable transformations of Python+NumPy programs: differentiate,
return asarray(x, dtype=self.dtype)
</pre></div>
</div>
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[&lt;matplotlib.lines.Line2D at 0x117c17be0&gt;]
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[&lt;matplotlib.lines.Line2D at 0x11975e550&gt;]
</pre></div>
</div>
<img alt="_images/week40_120_2.png" src="_images/week40_120_2.png" />
@@ -3137,7 +3138,7 @@ It provides composable transformations of Python+NumPy programs: differentiate,
</div>
</div>
<div class="cell_output docutils container">
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;matplotlib.collections.PathCollection at 0x117893340&gt;
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;matplotlib.collections.PathCollection at 0x11b160a60&gt;
</pre></div>
</div>
<img alt="_images/week40_122_1.png" src="_images/week40_122_1.png" />