lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

update

Browse Source
This commit is contained in:
Morten Hjorth-Jensen
2023-11-09 17:54:42 +01:00
parent 77d85c179c
commit beb104cebf
7 changed files with 259 additions and 305 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/LectureNotes/_build/.doctrees/environment.pickle
BIN
View File
Binary file not shown.
doc/LectureNotes/_build/.doctrees/week45.doctree
BIN
View File
Binary file not shown.
doc/LectureNotes/_build/html/_sources/week45.ipynb
+74 -70
View File
@@ -2,7 +2,7 @@
"cells": [
{
"cell_type": "markdown",
"id": "7e9869f2",
"id": "967cdaca",
"metadata": {
"editable": true
},
@@ -14,7 +14,7 @@
},
{
"cell_type": "markdown",
"id": "fca263d7",
"id": "7f44e4d0",
"metadata": {
"editable": true
},
@@ -27,7 +27,7 @@
},
{
"cell_type": "markdown",
"id": "8c3d9fea",
"id": "3094316d",
"metadata": {
"editable": true
},
@@ -58,6 +58,10 @@
"\n",
" * These lecture notes\n",
"\n",
" * [Video of lecture](https://youtu.be/z0x-vgyAZUk)\n",
"\n",
" * [Whiteboard notes](https://github.com/CompPhysics/MachineLearning/blob/master/doc/HandWrittenNotes/2023/NotesNov9.pdf)\n",
"\n",
" * For a more in depth discussion on neural networks we recommend Goodfellow et al chapter 10. See also chapter 11 and 12 on practicalities and applications \n",
"\n",
" * Reading suggestions for implementation of RNNs: [Aurelien Geron's chapter 14](https://github.com/CompPhysics/MachineLearning/blob/master/doc/Textbooks/TensorflowML.pdf).\n",
@@ -69,7 +73,7 @@
},
{
"cell_type": "markdown",
"id": "d0107e8f",
"id": "46b1e9db",
"metadata": {
"editable": true
},
@@ -79,7 +83,7 @@
},
{
"cell_type": "markdown",
"id": "44ed2bb7",
"id": "19315bd5",
"metadata": {
"editable": true
},
@@ -98,7 +102,7 @@
{
"cell_type": "code",
"execution_count": 1,
"id": "7d202a1c",
"id": "d03a6f44",
"metadata": {
"collapsed": false,
"editable": true
@@ -155,7 +159,7 @@
},
{
"cell_type": "markdown",
"id": "0fcc974f",
"id": "e6cbd4ca",
"metadata": {
"editable": true
},
@@ -167,7 +171,7 @@
},
{
"cell_type": "markdown",
"id": "693475c8",
"id": "1ea723fc",
"metadata": {
"editable": true
},
@@ -182,7 +186,7 @@
{
"cell_type": "code",
"execution_count": 2,
"id": "cdd8cf73",
"id": "0ebb62df",
"metadata": {
"collapsed": false,
"editable": true
@@ -235,7 +239,7 @@
},
{
"cell_type": "markdown",
"id": "51ef3976",
"id": "0fb161e1",
"metadata": {
"editable": true
},
@@ -250,7 +254,7 @@
},
{
"cell_type": "markdown",
"id": "033bda9d",
"id": "8bdb137e",
"metadata": {
"editable": true
},
@@ -270,7 +274,7 @@
{
"cell_type": "code",
"execution_count": 3,
"id": "0a320c52",
"id": "af61779f",
"metadata": {
"collapsed": false,
"editable": true
@@ -324,7 +328,7 @@
},
{
"cell_type": "markdown",
"id": "bcd8ea70",
"id": "89f07674",
"metadata": {
"editable": true
},
@@ -339,7 +343,7 @@
{
"cell_type": "code",
"execution_count": 4,
"id": "6fe70002",
"id": "37c8ca05",
"metadata": {
"collapsed": false,
"editable": true
@@ -366,7 +370,7 @@
},
{
"cell_type": "markdown",
"id": "c4440f93",
"id": "509bf7a9",
"metadata": {
"editable": true
},
@@ -380,7 +384,7 @@
{
"cell_type": "code",
"execution_count": 5,
"id": "96a74e80",
"id": "e9d06a0c",
"metadata": {
"collapsed": false,
"editable": true
@@ -425,7 +429,7 @@
},
{
"cell_type": "markdown",
"id": "98525677",
"id": "ab1f9810",
"metadata": {
"editable": true
},
@@ -450,7 +454,7 @@
{
"cell_type": "code",
"execution_count": 6,
"id": "d00d3339",
"id": "d8a5dabe",
"metadata": {
"collapsed": false,
"editable": true
@@ -462,7 +466,7 @@
},
{
"cell_type": "markdown",
"id": "39d74647",
"id": "69fd6511",
"metadata": {
"editable": true
},
@@ -473,7 +477,7 @@
{
"cell_type": "code",
"execution_count": 7,
"id": "382459a0",
"id": "b74de9e5",
"metadata": {
"collapsed": false,
"editable": true
@@ -485,7 +489,7 @@
},
{
"cell_type": "markdown",
"id": "06da1caf",
"id": "24e267e4",
"metadata": {
"editable": true
},
@@ -498,7 +502,7 @@
},
{
"cell_type": "markdown",
"id": "2d846128",
"id": "77a9c527",
"metadata": {
"editable": true
},
@@ -531,7 +535,7 @@
},
{
"cell_type": "markdown",
"id": "23991e3b",
"id": "5f6ea3f0",
"metadata": {
"editable": true
},
@@ -545,7 +549,7 @@
},
{
"cell_type": "markdown",
"id": "4581b004",
"id": "b0d324b8",
"metadata": {
"editable": true
},
@@ -557,7 +561,7 @@
},
{
"cell_type": "markdown",
"id": "855fc1d5",
"id": "d5593af9",
"metadata": {
"editable": true
},
@@ -569,7 +573,7 @@
},
{
"cell_type": "markdown",
"id": "7b676376",
"id": "e857f89e",
"metadata": {
"editable": true
},
@@ -581,7 +585,7 @@
},
{
"cell_type": "markdown",
"id": "64e4f58f",
"id": "13dc6da3",
"metadata": {
"editable": true
},
@@ -591,7 +595,7 @@
},
{
"cell_type": "markdown",
"id": "6bef46ee",
"id": "0f77aa9a",
"metadata": {
"editable": true
},
@@ -603,7 +607,7 @@
},
{
"cell_type": "markdown",
"id": "ed7cfd1e",
"id": "3c560ddf",
"metadata": {
"editable": true
},
@@ -613,7 +617,7 @@
},
{
"cell_type": "markdown",
"id": "307589a6",
"id": "b46686df",
"metadata": {
"editable": true
},
@@ -625,7 +629,7 @@
},
{
"cell_type": "markdown",
"id": "24f17f38",
"id": "24aec375",
"metadata": {
"editable": true
},
@@ -636,7 +640,7 @@
},
{
"cell_type": "markdown",
"id": "e825c026",
"id": "8d288b9d",
"metadata": {
"editable": true
},
@@ -655,7 +659,7 @@
},
{
"cell_type": "markdown",
"id": "6dcb34a4",
"id": "e1663205",
"metadata": {
"editable": true
},
@@ -667,7 +671,7 @@
},
{
"cell_type": "markdown",
"id": "a975cae8",
"id": "b34d890c",
"metadata": {
"editable": true
},
@@ -677,7 +681,7 @@
},
{
"cell_type": "markdown",
"id": "d4971baf",
"id": "14e7f314",
"metadata": {
"editable": true
},
@@ -689,7 +693,7 @@
},
{
"cell_type": "markdown",
"id": "a4bd001d",
"id": "879bac6b",
"metadata": {
"editable": true
},
@@ -701,7 +705,7 @@
},
{
"cell_type": "markdown",
"id": "a8b1f114",
"id": "31d2b3b7",
"metadata": {
"editable": true
},
@@ -713,7 +717,7 @@
},
{
"cell_type": "markdown",
"id": "959cdbff",
"id": "34fd7537",
"metadata": {
"editable": true
},
@@ -723,7 +727,7 @@
},
{
"cell_type": "markdown",
"id": "59ee7f47",
"id": "4bebbe84",
"metadata": {
"editable": true
},
@@ -735,7 +739,7 @@
},
{
"cell_type": "markdown",
"id": "5cdba722",
"id": "690c8d76",
"metadata": {
"editable": true
},
@@ -763,7 +767,7 @@
},
{
"cell_type": "markdown",
"id": "35390ed4",
"id": "12d8a75e",
"metadata": {
"editable": true
},
@@ -775,7 +779,7 @@
},
{
"cell_type": "markdown",
"id": "b1651ea3",
"id": "1df29154",
"metadata": {
"editable": true
},
@@ -800,7 +804,7 @@
},
{
"cell_type": "markdown",
"id": "1bfac852",
"id": "9083178e",
"metadata": {
"editable": true
},
@@ -818,7 +822,7 @@
},
{
"cell_type": "markdown",
"id": "f655bb24",
"id": "e7719466",
"metadata": {
"editable": true
},
@@ -829,7 +833,7 @@
{
"cell_type": "code",
"execution_count": 8,
"id": "31f4ee21",
"id": "bcddf060",
"metadata": {
"collapsed": false,
"editable": true
@@ -872,7 +876,7 @@
},
{
"cell_type": "markdown",
"id": "098bfaec",
"id": "6ec55e94",
"metadata": {
"editable": true
},
@@ -882,7 +886,7 @@
},
{
"cell_type": "markdown",
"id": "55cc0e20",
"id": "690b1a08",
"metadata": {
"editable": true
},
@@ -909,7 +913,7 @@
},
{
"cell_type": "markdown",
"id": "8860da9c",
"id": "825bc136",
"metadata": {
"editable": true
},
@@ -920,7 +924,7 @@
{
"cell_type": "code",
"execution_count": 9,
"id": "7252c0b8",
"id": "ce956b33",
"metadata": {
"collapsed": false,
"editable": true
@@ -993,7 +997,7 @@
},
{
"cell_type": "markdown",
"id": "fcbca2f7",
"id": "002c4d99",
"metadata": {
"editable": true
},
@@ -1013,7 +1017,7 @@
},
{
"cell_type": "markdown",
"id": "1c5b70a2",
"id": "6572c07f",
"metadata": {
"editable": true
},
@@ -1029,7 +1033,7 @@
},
{
"cell_type": "markdown",
"id": "29c129fe",
"id": "fbd8e269",
"metadata": {
"editable": true
},
@@ -1051,7 +1055,7 @@
},
{
"cell_type": "markdown",
"id": "c2bc1aa3",
"id": "919bb09e",
"metadata": {
"editable": true
},
@@ -1069,7 +1073,7 @@
},
{
"cell_type": "markdown",
"id": "01ef7e17",
"id": "6e3360db",
"metadata": {
"editable": true
},
@@ -1115,7 +1119,7 @@
},
{
"cell_type": "markdown",
"id": "8dd26bf6",
"id": "0f284cbd",
"metadata": {
"editable": true
},
@@ -1136,7 +1140,7 @@
},
{
"cell_type": "markdown",
"id": "3b42d5ef",
"id": "b5e9785e",
"metadata": {
"editable": true
},
@@ -1197,7 +1201,7 @@
},
{
"cell_type": "markdown",
"id": "2f8de715",
"id": "50a43c30",
"metadata": {
"editable": true
},
@@ -1222,7 +1226,7 @@
},
{
"cell_type": "markdown",
"id": "9998cb2c",
"id": "f3e0d31b",
"metadata": {
"editable": true
},
@@ -1241,7 +1245,7 @@
},
{
"cell_type": "markdown",
"id": "45dfbeff",
"id": "b1571231",
"metadata": {
"editable": true
},
@@ -1265,7 +1269,7 @@
},
{
"cell_type": "markdown",
"id": "141b8f50",
"id": "e7886dd3",
"metadata": {
"editable": true
},
@@ -1347,7 +1351,7 @@
},
{
"cell_type": "markdown",
"id": "fbadc1e0",
"id": "f878b195",
"metadata": {
"editable": true
},
@@ -1363,7 +1367,7 @@
{
"cell_type": "code",
"execution_count": 10,
"id": "9365f92f",
"id": "1889da48",
"metadata": {
"collapsed": false,
"editable": true
@@ -1402,7 +1406,7 @@
},
{
"cell_type": "markdown",
"id": "55c2aa23",
"id": "5bf3bea4",
"metadata": {
"editable": true
},
@@ -1446,7 +1450,7 @@
{
"cell_type": "code",
"execution_count": 11,
"id": "b57a3099",
"id": "6fc9b3dd",
"metadata": {
"collapsed": false,
"editable": true
@@ -1529,7 +1533,7 @@
},
{
"cell_type": "markdown",
"id": "0b519f6c",
"id": "6b02bff4",
"metadata": {
"editable": true
},
@@ -1540,7 +1544,7 @@
{
"cell_type": "code",
"execution_count": 12,
"id": "82db8594",
"id": "7030f585",
"metadata": {
"collapsed": false,
"editable": true
@@ -1642,7 +1646,7 @@
},
{
"cell_type": "markdown",
"id": "4711b0d4",
"id": "53dc1510",
"metadata": {
"editable": true
},
@@ -1664,7 +1668,7 @@
{
"cell_type": "code",
"execution_count": 13,
"id": "22123a0f",
"id": "62aa2c1c",
"metadata": {
"collapsed": false,
"editable": true
@@ -1761,7 +1765,7 @@
},
{
"cell_type": "markdown",
"id": "50cd4d20",
"id": "56fb2d92",
"metadata": {
"editable": true
},
@@ -1787,7 +1791,7 @@
{
"cell_type": "code",
"execution_count": 14,
"id": "5c3d5a53",
"id": "145ed525",
"metadata": {
"collapsed": false,
"editable": true
doc/LectureNotes/_build/html/searchindex.js
+1 -1
View File
File diff suppressed because one or more lines are too long
doc/LectureNotes/_build/html/week45.html
+54 -70
View File
@@ -867,6 +867,8 @@ doconce format html week45.do.txt --no_mako -->
<li><p>Readings and Videos:</p>
<ul>
<li><p>These lecture notes</p></li>
<li><p><a class="reference external" href="https://youtu.be/z0x-vgyAZUk">Video of lecture</a></p></li>
<li><p><a class="reference external" href="https://github.com/CompPhysics/MachineLearning/blob/master/doc/HandWrittenNotes/2023/NotesNov9.pdf">Whiteboard notes</a></p></li>
<li><p>For a more in depth discussion on neural networks we recommend Goodfellow et al chapter 10. See also chapter 11 and 12 on practicalities and applications</p></li>
<li><p>Reading suggestions for implementation of RNNs: <a class="reference external" href="https://github.com/CompPhysics/MachineLearning/blob/master/doc/Textbooks/TensorflowML.pdf">Aurelien Geron’s chapter 14</a>.</p></li>
<li><p><a class="reference external" href="https://www.youtube.com/watch?v=SEnXr6v2ifU&amp;ab_channel=AlexanderAmini">Video on Recurrent Neural Networks from MIT</a></p></li>
@@ -1076,7 +1078,7 @@ case under study.</p>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>RandomizedSearchCV(estimator=Ridge(), n_iter=100,
param_distributions={&#39;alpha&#39;: &lt;scipy.stats._distn_infrastructure.rv_frozen object at 0x107a08b50&gt;})
param_distributions={&#39;alpha&#39;: &lt;scipy.stats._distn_infrastructure.rv_frozen object at 0x168435640&gt;})
Best estimated lambda-value: 0.9849967686928113
MSE score: 1.0853136633465326
R2 score: -0.0002382102844775691
@@ -1620,327 +1622,309 @@ systems such as automatic translation and speech-to-text.</p>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 1/100
</pre></div>
</div>
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>2023-11-08 15:31:22.964588: W tensorflow/core/platform/profile_utils/cpu_utils.cc:128] Failed to get CPU frequency: 0 Hz
<div class="output stderr highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>2023-11-09 17:53:10.939507: W tensorflow/core/platform/profile_utils/cpu_utils.cc:128] Failed to get CPU frequency: 0 Hz
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 3s - loss: 1.3549 - 3s/epoch - 66ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 3s - loss: 0.5078 - 3s/epoch - 65ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 2/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.4772 - 473ms/epoch - 9ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.4096 - 450ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 3/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.4055 - 471ms/epoch - 9ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3998 - 450ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 4/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3987 - 467ms/epoch - 9ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3960 - 449ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 5/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3954 - 475ms/epoch - 9ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3937 - 462ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 6/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3923 - 468ms/epoch - 9ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3922 - 467ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 7/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3914 - 468ms/epoch - 9ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3894 - 462ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 8/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3887 - 457ms/epoch - 9ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3860 - 458ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 9/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3893 - 468ms/epoch - 9ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3877 - 462ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 10/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3882 - 460ms/epoch - 9ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3861 - 463ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 11/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3868 - 463ms/epoch - 9ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3855 - 469ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 12/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3862 - 466ms/epoch - 9ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3860 - 466ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 13/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3862 - 484ms/epoch - 10ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3844 - 470ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 14/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3854 - 464ms/epoch - 9ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3842 - 466ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 15/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3847 - 476ms/epoch - 10ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3857 - 467ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 16/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3839 - 473ms/epoch - 9ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3821 - 472ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 17/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3843 - 474ms/epoch - 9ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3848 - 470ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 18/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3837 - 456ms/epoch - 9ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3823 - 468ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 19/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3829 - 461ms/epoch - 9ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3842 - 470ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 20/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3819 - 494ms/epoch - 10ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3822 - 468ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 21/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3812 - 480ms/epoch - 10ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3818 - 471ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 22/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3816 - 491ms/epoch - 10ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3794 - 470ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 23/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3818 - 459ms/epoch - 9ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3820 - 466ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 24/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3812 - 470ms/epoch - 9ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3803 - 467ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 25/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3811 - 452ms/epoch - 9ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3802 - 466ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 26/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3809 - 455ms/epoch - 9ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3797 - 468ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 27/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3798 - 453ms/epoch - 9ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3789 - 472ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 28/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3809 - 454ms/epoch - 9ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3764 - 472ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 29/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3802 - 457ms/epoch - 9ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3767 - 470ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 30/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3798 - 463ms/epoch - 9ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3745 - 467ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 31/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3789 - 456ms/epoch - 9ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3765 - 469ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 32/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3781 - 457ms/epoch - 9ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3791 - 469ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 33/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3790 - 456ms/epoch - 9ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3770 - 466ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 34/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3775 - 455ms/epoch - 9ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3780 - 468ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 35/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3785 - 459ms/epoch - 9ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3770 - 466ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 36/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3777 - 454ms/epoch - 9ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3748 - 470ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 37/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3782 - 453ms/epoch - 9ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3773 - 467ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 38/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3780 - 460ms/epoch - 9ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3760 - 470ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 39/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3764 - 479ms/epoch - 10ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3759 - 473ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 40/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3773 - 498ms/epoch - 10ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3735 - 470ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 41/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 1s - loss: 0.3749 - 505ms/epoch - 10ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3736 - 470ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 42/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3769 - 489ms/epoch - 10ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3733 - 470ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 43/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3766 - 462ms/epoch - 9ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3727 - 471ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 44/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3752 - 459ms/epoch - 9ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3749 - 469ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 45/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3758 - 469ms/epoch - 9ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3740 - 467ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 46/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3734 - 477ms/epoch - 10ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3705 - 466ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 47/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3753 - 475ms/epoch - 10ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3732 - 469ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 48/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3756 - 469ms/epoch - 9ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3717 - 467ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 49/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3748 - 487ms/epoch - 10ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3714 - 470ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 50/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3741 - 478ms/epoch - 10ms/step
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3705 - 471ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 51/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3744 - 478ms/epoch - 10ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 52/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3756 - 458ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 53/100
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>50/50 - 0s - loss: 0.3725 - 464ms/epoch - 9ms/step
</pre></div>
</div>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Epoch 54/100
</pre></div>
</div>
<div class="output traceback highlight-ipythontb notranslate"><div class="highlight"><pre><span></span><span class="gt">---------------------------------------------------------------------------</span>
<span class="ne">KeyboardInterrupt</span><span class="g g-Whitespace"> </span>Traceback (most recent call last)
<span class="nn">Input In [9],</span> in <span class="ni">&lt;cell line: 53&gt;</span><span class="nt">()</span>
doc/LectureNotes/_build/jupyter_execute/week45.ipynb
+126 -164
View File
File diff suppressed because it is too large Load Diff
doc/LectureNotes/_build/jupyter_execute/week45.py
+4
View File
@@ -36,6 +36,10 @@
#
# * These lecture notes
#
# * [Video of lecture](https://youtu.be/z0x-vgyAZUk)
#
# * [Whiteboard notes](https://github.com/CompPhysics/MachineLearning/blob/master/doc/HandWrittenNotes/2023/NotesNov9.pdf)
#
# * For a more in depth discussion on neural networks we recommend Goodfellow et al chapter 10. See also chapter 11 and 12 on practicalities and applications
#
# * Reading suggestions for implementation of RNNs: [Aurelien Geron's chapter 14](https://github.com/CompPhysics/MachineLearning/blob/master/doc/Textbooks/TensorflowML.pdf).