updated book

doc/LectureNotes/_build/html/genindex.html

@@ -309,16 +309,14 @@ const thebe_selector_output = ".output, .cell_output"
    Week 40: Gradient descent methods (continued) and start Neural networks
   </a>
  </li>
-</ul>
-<p aria-level="2" class="caption" role="heading">
- <span class="caption-text">
-  Projects
- </span>
-</p>
-<ul class="nav bd-sidenav">
  <li class="toctree-l1">
-  <a class="reference internal" href="project1.html">
-   Project 1 on Machine Learning, deadline October 7 (midnight), 2024
+  <a class="reference internal" href="exercisesweek41.html">
+   Exercises week 41
+  </a>
+ </li>
+ <li class="toctree-l1">
+  <a class="reference internal" href="week41.html">
+   Week 41 Neural networks and constructing a neural network code
   </a>
  </li>
 </ul>

doc/LectureNotes/_build/html/intro.html

@@ -310,16 +310,14 @@ const thebe_selector_output = ".output, .cell_output"
    Week 40: Gradient descent methods (continued) and start Neural networks
   </a>
  </li>
-</ul>
-<p aria-level="2" class="caption" role="heading">
- <span class="caption-text">
-  Projects
- </span>
-</p>
-<ul class="nav bd-sidenav">
  <li class="toctree-l1">
-  <a class="reference internal" href="project1.html">
-   Project 1 on Machine Learning, deadline October 7 (midnight), 2024
+  <a class="reference internal" href="exercisesweek41.html">
+   Exercises week 41
+  </a>
+ </li>
+ <li class="toctree-l1">
+  <a class="reference internal" href="week41.html">
+   Week 41 Neural networks and constructing a neural network code
   </a>
  </li>
 </ul>
@@ -676,8 +674,6 @@ It provides composable transformations of Python+NumPy programs: differentiate,
 </div>
 <div class="toctree-wrapper compound">
 </div>
-<div class="toctree-wrapper compound">
-</div>
 </div>
 
     <script type="text/x-thebe-config">

doc/LectureNotes/_build/html/search.html

@@ -315,16 +315,14 @@ const thebe_selector_output = ".output, .cell_output"
    Week 40: Gradient descent methods (continued) and start Neural networks
   </a>
  </li>
-</ul>
-<p aria-level="2" class="caption" role="heading">
- <span class="caption-text">
-  Projects
- </span>
-</p>
-<ul class="nav bd-sidenav">
  <li class="toctree-l1">
-  <a class="reference internal" href="project1.html">
-   Project 1 on Machine Learning, deadline October 7 (midnight), 2024
+  <a class="reference internal" href="exercisesweek41.html">
+   Exercises week 41
+  </a>
+ </li>
+ <li class="toctree-l1">
+  <a class="reference internal" href="week41.html">
+   Week 41 Neural networks and constructing a neural network code
   </a>
  </li>
 </ul>

doc/LectureNotes/_build/html/week40.html

@@ -55,7 +55,6 @@ const thebe_selector_output = ".output, .cell_output"
     <script defer="defer" src="https://cdn.jsdelivr.net/npm/mathjax@3/es5/tex-mml-chtml.js"></script>
     <link rel="index" title="Index" href="genindex.html" />
     <link rel="search" title="Search" href="search.html" />
-    <link rel="next" title="Project 1 on Machine Learning, deadline October 7 (midnight), 2024" href="project1.html" />
     <link rel="prev" title="Week 39: Optimization and Gradient Methods" href="week39.html" />
     <meta name="viewport" content="width=device-width, initial-scale=1" />
     <meta name="docsearch:language" content="None">
@@ -314,18 +313,6 @@ const thebe_selector_output = ".output, .cell_output"
   </a>
  </li>
 </ul>
-<p aria-level="2" class="caption" role="heading">
- <span class="caption-text">
-  Projects
- </span>
-</p>
-<ul class="nav bd-sidenav">
- <li class="toctree-l1">
-  <a class="reference internal" href="project1.html">
-   Project 1 on Machine Learning, deadline October 7 (midnight), 2024
-  </a>
- </li>
-</ul>
 
     </div>
 </nav> <!-- To handle the deprecated key -->
@@ -1289,17 +1276,17 @@ We summarize some of these here for the methods we hvae studied in project one,
 </div>
 <div class="cell_output docutils container">
 <div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Parameters for OLS using gradient descent
-[[3.53732988]
- [4.06277313]
- [4.51728978]]
+[[4.14721582]
+ [2.54760497]
+ [5.2212296 ]]
 Parameters for Ridge using gradient descent
-[[3.62671667]
- [3.79849321]
- [4.63207408]]
+[[3.75424998]
+ [3.49608088]
+ [4.78010668]]
 Parameters for Lasso using gradient descent
-[[4.1547556 ]
- [2.58013829]
- [5.20288581]]
+[[3.62716271]
+ [3.82146046]
+ [4.64516328]]
 </pre></div>
 </div>
 </div>
@@ -1351,11 +1338,11 @@ Parameters for Lasso using gradient descent
 [[4.]
  [3.]
  [5.]]
-0 [-27.10587277] [-37.81035098]
-1 [1.16209264e-13] [2.08824304e-13]
-2 [8.8817842e-17] [2.62242138e-16]
-3 [-1.77635684e-17] [-1.50600514e-16]
-4 [-3.37507799e-16] [-2.280464e-16]
+0 [-25.77106886] [-35.02189606]
+1 [3.49285045e-13] [4.71505132e-13]
+2 [1.24344979e-16] [3.86639832e-16]
+3 [9.05941988e-16] [1.48786826e-15]
+4 [-7.99360578e-16] [-1.35823372e-15]
 beta from own Newton code
 [[4.]
  [3.]
@@ -1616,6 +1603,9 @@ function.</p>
 <div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>gamma_j after 500 epochs: 9.97108e-05
 </pre></div>
 </div>
+<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>
+</pre></div>
+</div>
 </div>
 </div>
 </div>
@@ -1697,15 +1687,15 @@ function.</p>
 </div>
 <div class="cell_output docutils container">
 <div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
-[[4.30637636]
- [2.56947078]]
-Eigenvalues of Hessian Matrix:[0.30000613 4.12600095]
+[[4.16180248]
+ [2.8250103 ]]
+Eigenvalues of Hessian Matrix:[0.26646852 4.68110474]
 theta from own gd
-[[4.30637636]
- [2.56947078]]
+[[4.16180248]
+ [2.8250103 ]]
 theta from own sdg
-[[4.34759233]
- [2.63142731]]
+[[4.16257872]
+ [2.80576215]]
 </pre></div>
 </div>
 <img alt="_images/week40_34_1.png" src="_images/week40_34_1.png" />
@@ -2419,12 +2409,12 @@ first example shows results with ordinary leats squares.</p>
 </div>
 <div class="cell_output docutils container">
 <div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
-[[3.88064505]
- [2.99072374]]
-Eigenvalues of Hessian Matrix:[0.30141906 4.73853838]
+[[3.98246764]
+ [3.04018043]]
+Eigenvalues of Hessian Matrix:[0.31541884 4.4735968 ]
 theta from own gd
-[[3.88064505]
- [2.99072374]]
+[[3.98246764]
+ [3.04018043]]
 </pre></div>
 </div>
 <img alt="_images/week40_100_1.png" src="_images/week40_100_1.png" />
@@ -2495,73 +2485,73 @@ theta from own gd
 <div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
 [[4.]
  [3.]]
-Eigenvalues of Hessian Matrix:[0.31702609 3.84351715]
-0 [-13.26083712] [-12.67834752]
-1 [-0.55020405] [0.5257011]
-2 [-0.50482139] [0.48233952]
-3 [-0.46318204] [0.44255455]
-4 [-0.42497724] [0.40605118]
-5 [-0.38992371] [0.37255873]
-6 [-0.3577615] [0.34182884]
-7 [-0.32825214] [0.31363366]
-8 [-0.30117681] [0.28776411]
-9 [-0.27633474] [0.26402837]
-10 [-0.25354173] [0.24225043]
-11 [-0.23262877] [0.22226881]
-12 [-0.21344077] [0.20393534]
-13 [-0.19583547] [0.18711407]
-14 [-0.17968231] [0.17168028]
-15 [-0.16486151] [0.15751952]
-16 [-0.15126318] [0.14452678]
-17 [-0.13878649] [0.13260573]
-18 [-0.12733892] [0.12166797]
-19 [-0.11683558] [0.11163239]
-20 [-0.10719859] [0.10242458]
-21 [-0.0983565] [0.09397626]
-22 [-0.09024373] [0.08622478]
-23 [-0.08280012] [0.07911268]
-24 [-0.07597049] [0.0725872]
-25 [-0.06970419] [0.06659997]
-26 [-0.06395476] [0.06110658]
-27 [-0.05867956] [0.0560663]
-28 [-0.05383947] [0.05144177]
-29 [-0.04939861] [0.04719868]
+Eigenvalues of Hessian Matrix:[0.32799141 4.41229845]
+0 [-16.27698328] [-18.5653087]
+1 [-0.44867801] [0.37354121]
+2 [-0.41532521] [0.34577375]
+3 [-0.38445171] [0.32007041]
+4 [-0.35587321] [0.29627774]
+5 [-0.32941912] [0.27425372]
+6 [-0.30493151] [0.25386687]
+7 [-0.2822642] [0.23499549]
+8 [-0.26128189] [0.21752693]
+9 [-0.24185931] [0.20135691]
+10 [-0.22388052] [0.1863889]
+11 [-0.2072382] [0.17253354]
+12 [-0.191833] [0.15970814]
+13 [-0.17757295] [0.14783612]
+14 [-0.16437294] [0.13684661]
+15 [-0.15215415] [0.12667402]
+16 [-0.14084366] [0.11725761]
+17 [-0.13037395] [0.10854118]
+18 [-0.12068251] [0.1004727]
+19 [-0.11171148] [0.09300398]
+20 [-0.10340733] [0.08609047]
+21 [-0.09572047] [0.07969087]
+22 [-0.08860502] [0.07376699]
+23 [-0.0820185] [0.06828347]
+24 [-0.0759216] [0.06320757]
+25 [-0.07027791] [0.05850899]
+26 [-0.06505375] [0.05415968]
+27 [-0.06021793] [0.05013368]
+28 [-0.05574159] [0.04640695]
+29 [-0.051598] [0.04295726]
 theta from own gd
-[[3.85703369]
- [3.13659941]]
-0 [-0.04532405] [0.04330558]
-1 [-0.04158557] [0.03973359]
-2 [-0.03703391] [0.03538463]
-3 [-0.03261373] [0.0311613]
-4 [-0.02859759] [0.02732402]
-5 [-0.02503392] [0.02391906]
-6 [-0.02189994] [0.02092464]
-7 [-0.01915337] [0.01830039]
-8 [-0.01674956] [0.01600363]
-9 [-0.01464686] [0.01399457]
-10 [-0.01280793] [0.01223754]
-11 [-0.01119981] [0.01070103]
-12 [-0.00979357] [0.00935742]
-13 [-0.0085639] [0.00818251]
-14 [-0.00748862] [0.00715512]
-15 [-0.00654835] [0.00625672]
-16 [-0.00572613] [0.00547113]
-17 [-0.00500716] [0.00478417]
-18 [-0.00437846] [0.00418347]
-19 [-0.0038287] [0.00365819]
-20 [-0.00334797] [0.00319887]
-21 [-0.0029276] [0.00279722]
-22 [-0.00256001] [0.002446]
-23 [-0.00223857] [0.00213888]
-24 [-0.0019575] [0.00187032]
-25 [-0.00171171] [0.00163548]
-26 [-0.00149679] [0.00143013]
-27 [-0.00130885] [0.00125057]
-28 [-0.00114451] [0.00109354]
-29 [-0.00100081] [0.00095624]
+[[3.85437905]
+ [3.12123488]]
+0 [-0.04776242] [0.039764]
+1 [-0.04421197] [0.03680811]
+2 [-0.0398603] [0.03318519]
+3 [-0.03559176] [0.02963147]
+4 [-0.03166546] [0.02636268]
+5 [-0.02813369] [0.02342235]
+6 [-0.02498282] [0.02079913]
+7 [-0.02218045] [0.01846605]
+8 [-0.01969093] [0.01639344]
+9 [-0.01748034] [0.01455304]
+10 [-0.01551775] [0.0129191]
+11 [-0.01377545] [0.01146857]
+12 [-0.01222875] [0.01018089]
+13 [-0.01085571] [0.00903778]
+14 [-0.00963683] [0.00802302]
+15 [-0.0085548] [0.00712219]
+16 [-0.00759427] [0.00632251]
+17 [-0.00674158] [0.00561262]
+18 [-0.00598464] [0.00498243]
+19 [-0.00531268] [0.004423]
+20 [-0.00471617] [0.00392639]
+21 [-0.00418664] [0.00348553]
+22 [-0.00371656] [0.00309418]
+23 [-0.00329927] [0.00274676]
+24 [-0.00292882] [0.00243835]
+25 [-0.00259997] [0.00216458]
+26 [-0.00230805] [0.00192154]
+27 [-0.0020489] [0.00170579]
+28 [-0.00181885] [0.00151426]
+29 [-0.00161463] [0.00134424]
 theta from own gd wth momentum
-[[3.99723951]
- [3.00263755]]
+[[3.99562995]
+ [3.00363823]]
 </pre></div>
 </div>
 </div>
@@ -2650,18 +2640,18 @@ theta from own gd wth momentum
 </div>
 <div class="cell_output docutils container">
 <div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
-[[3.95935439]
- [3.09360113]]
-Eigenvalues of Hessian Matrix:[0.32606139 4.1033455 ]
+[[4.10426556]
+ [2.93942872]]
+Eigenvalues of Hessian Matrix:[0.31367041 4.07517385]
 theta from own gd
-[[3.95935439]
- [3.09360113]]
+[[4.10426556]
+ [2.93942872]]
 </pre></div>
 </div>
 <img alt="_images/week40_104_1.png" src="_images/week40_104_1.png" />
 <div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own sdg
-[[3.96916206]
- [3.10276112]]
+[[4.08587209]
+ [2.96301764]]
 </pre></div>
 </div>
 </div>
@@ -2743,15 +2733,17 @@ theta from own gd
 </div>
 <div class="cell_output docutils container">
 <div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>Own inversion
-[[4.16532977]
- [2.86943859]]
-Eigenvalues of Hessian Matrix:[0.29633608 4.22128142]
+[[4.33528448]
+ [2.81889188]]
+Eigenvalues of Hessian Matrix:[0.30916402 4.51611732]
 theta from own gd
-[[4.16445858]
- [2.87020155]]
-theta from own sdg with momentum
-[[4.149433  ]
- [2.89654756]]
+[[4.33477019]
+ [2.81931348]]
+</pre></div>
+</div>
+<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own sdg with momentum
+[[4.40800396]
+ [2.78459458]]
 </pre></div>
 </div>
 </div>
@@ -2820,9 +2812,9 @@ theta from own sdg with momentum
 </pre></div>
 </div>
 <div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own AdaGrad
-[[1.99415296]
- [3.03274273]
- [3.96850191]]
+[[2.00003828]
+ [2.99979896]
+ [4.00019258]]
 </pre></div>
 </div>
 </div>
@@ -2898,9 +2890,9 @@ theta from own sdg with momentum
 </pre></div>
 </div>
 <div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own RMSprop
-[[2.00022969]
- [2.99954278]
- [4.00046631]]
+[[2.01129731]
+ [3.01249445]
+ [4.00885858]]
 </pre></div>
 </div>
 </div>
@@ -2980,9 +2972,9 @@ theta from own sdg with momentum
 </pre></div>
 </div>
 <div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>theta from own ADAM
-[[1.99991245]
- [3.00042721]
- [3.99954434]]
+[[2.00001716]
+ [2.99992107]
+ [4.00007706]]
 </pre></div>
 </div>
 </div>
@@ -3103,7 +3095,7 @@ It provides composable transformations of Python+NumPy programs: differentiate,
   return asarray(x, dtype=self.dtype)
 </pre></div>
 </div>
-<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[&lt;matplotlib.lines.Line2D at 0x11975e550&gt;]
+<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[&lt;matplotlib.lines.Line2D at 0x11753f700&gt;]
 </pre></div>
 </div>
 <img alt="_images/week40_120_2.png" src="_images/week40_120_2.png" />
@@ -3138,7 +3130,7 @@ It provides composable transformations of Python+NumPy programs: differentiate,
 </div>
 </div>
 <div class="cell_output docutils container">
-<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;matplotlib.collections.PathCollection at 0x11b160a60&gt;
+<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>&lt;matplotlib.collections.PathCollection at 0x11bc3a1f0&gt;
 </pre></div>
 </div>
 <img alt="_images/week40_122_1.png" src="_images/week40_122_1.png" />
@@ -3818,13 +3810,6 @@ become the most popular for <em>deep neural networks</em></p>
             <p class="prev-next-title">Week 39: Optimization and  Gradient Methods</p>
         </div>
     </a>
-    <a class='right-next' id="next-link" href="project1.html" title="next page">
-    <div class="prev-next-info">
-        <p class="prev-next-subtitle">next</p>
-        <p class="prev-next-title">Project 1 on Machine Learning, deadline October 7 (midnight), 2024</p>
-    </div>
-    <i class="fas fa-angle-right"></i>
-    </a>
 </div>
             
         </div>