corrected some small errors

doc/LectureNotes/_build/html/week35.html

@@ -1611,7 +1611,7 @@ Since we are not using <strong>Scikit-Learn</strong> here we can define our own
 </div>
 </div>
 <div class="cell_output docutils container">
-<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.9959898232423614
+<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.9963202597920238
 </pre></div>
 </div>
 </div>
@@ -1628,7 +1628,7 @@ Since we are not using <strong>Scikit-Learn</strong> here we can define our own
 </div>
 </div>
 <div class="cell_output docutils container">
-<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.008278885543361304
+<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0.008550594684081035
 </pre></div>
 </div>
 </div>
@@ -1643,23 +1643,31 @@ Since we are not using <strong>Scikit-Learn</strong> here we can define our own
 </div>
 </div>
 <div class="cell_output docutils container">
-<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[0.02970592 0.01381723 0.01572164 0.02401344 0.06008452 0.0084603
- 0.00506501 0.08734015 0.00145838 0.00779538 0.00046714 0.02971158
- 0.02263868 0.02541796 0.02724253 0.04120281 0.00578776 0.01574375
- 0.08923158 0.03510756 0.00373019 0.03206303 0.02008059 0.03533092
- 0.01558461 0.00733497 0.01770163 0.03026672 0.05698425 0.02766981
- 0.02421521 0.00943261 0.00207627 0.03352725 0.00139198 0.00770648
- 0.03078597 0.00355175 0.00432302 0.04554454 0.05892262 0.0097018
- 0.04100738 0.03232013 0.02747522 0.04208001 0.00252344 0.03080664
- 0.00437363 0.01614046 0.00158723 0.01471878 0.03235599 0.01285424
- 0.01494533 0.01006893 0.01248731 0.0317159  0.02226654 0.0189082
- 0.01923991 0.01816821 0.02900131 0.0123679  0.08684675 0.00934728
- 0.04331431 0.01574091 0.00017471 0.0164286  0.00118368 0.00839839
- 0.00548686 0.02596838 0.02058701 0.00661104 0.02708158 0.03187054
- 0.0171347  0.04573057 0.04735806 0.00131939 0.00704468 0.02422503
- 0.02818883 0.003738   0.02212893 0.00732273 0.02212776 0.0202731
- 0.0214539  0.02712723 0.02200044 0.01588614 0.03610331 0.02808201
- 0.00801078 0.01885351 0.00458671 0.01133171]
+<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[1.09846636e-02 2.39074222e-02 4.94861749e-03 4.55640387e-02
+ 1.31193395e-02 1.62470815e-03 2.46575843e-02 7.80638926e-02
+ 2.19677483e-02 3.99901131e-02 2.68626148e-03 3.97731700e-02
+ 4.79285683e-02 6.69363975e-02 1.02951554e-02 3.69826476e-02
+ 6.29115152e-03 8.12289498e-02 3.11909685e-03 7.77144798e-04
+ 8.15320771e-03 2.45720968e-02 5.12803365e-02 2.34378387e-02
+ 2.33645986e-02 1.17796101e-02 2.80344720e-02 8.92995253e-02
+ 1.24844778e-02 4.21582995e-02 1.89591553e-02 2.75357959e-02
+ 2.44906236e-02 1.84031027e-02 2.09739330e-02 5.70421972e-03
+ 2.55914617e-02 4.10427270e-02 6.55907387e-04 4.63734811e-03
+ 8.42882114e-03 1.00851638e-04 5.19264104e-02 1.94917559e-05
+ 6.96542994e-03 4.08924325e-03 4.08743287e-03 9.75853556e-03
+ 4.04075841e-02 2.70473799e-02 1.72199579e-02 4.24501973e-02
+ 7.08373167e-04 4.25201286e-02 5.69487891e-03 3.18217743e-03
+ 1.85979474e-02 4.82250097e-02 8.96735081e-02 5.53952664e-02
+ 5.51900727e-02 4.98842948e-03 1.30232456e-02 3.45220770e-02
+ 4.12394241e-02 1.46400080e-02 2.27932015e-02 1.16054190e-03
+ 1.53971260e-02 3.77533038e-02 8.53502096e-03 1.82404007e-02
+ 1.86014003e-02 2.93792215e-02 3.66950317e-02 2.15456415e-02
+ 2.21576737e-02 8.38730917e-03 2.98005862e-02 2.03932541e-03
+ 4.60083846e-03 7.02160095e-03 5.76598941e-03 9.56807589e-03
+ 3.49730046e-02 1.61899355e-02 1.47158029e-02 6.09404851e-04
+ 5.14724135e-03 3.92916438e-02 2.88352503e-02 5.61520194e-02
+ 4.33646268e-02 3.26506981e-03 1.51400374e-02 1.10235180e-02
+ 7.63332882e-03 2.93229640e-02 6.85071429e-03 2.29148545e-03]
 </pre></div>
 </div>
 </div>
@@ -1728,15 +1736,15 @@ but now splitting the data into a training set and a test set.</p>
 </div>
 </div>
 <div class="cell_output docutils container">
-<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[ 2.00025531  0.8850944  -0.44894073 10.27580794 -5.9383327 ]
+<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[ 1.90813514  0.65746064  4.09759873 -0.08945041  0.46023985]
 Training R2
-0.9960039497353536
+0.9965048209369255
 Training MSE
-0.007839875253055148
+0.008089688732934681
 Test R2
-0.9962074631757628
+0.9963525271588923
 Test MSE
-0.006275173922218319
+0.0072559952738542694
 </pre></div>
 </div>
 </div>
@@ -3448,7 +3456,7 @@ decomposition of the design matrix.</p>
 \[
 \tilde{y}_{\mathrm{OLS}}=\boldsymbol{U}\boldsymbol{\Sigma}\boldsymbol{V}^T\left(\boldsymbol{V}\tilde{\boldsymbol{\Sigma}}^{2}(\boldsymbol{V}^T\right)^{-1}\boldsymbol{V}\boldsymbol{\Sigma}^T\boldsymbol{U}^T\boldsymbol{y},
 \]</div>
-<p>which gives us, using the orthogonality of the matrices <span class="math notranslate nohighlight">\(\boldsymbol{U}\)</span> and <span class="math notranslate nohighlight">\(\boldsymbol{V}\)</span>,</p>
+<p>which gives us, using the orthogonality of the matrix <span class="math notranslate nohighlight">\(\boldsymbol{V}\)</span>,</p>
 <div class="math notranslate nohighlight">
 \[
 \tilde{y}_{\mathrm{OLS}}=\boldsymbol{U}\boldsymbol{U}^T\boldsymbol{y}=\sum_{i=0}^{p-1}\boldsymbol{u}_i\boldsymbol{u}^T_i\boldsymbol{y},
@@ -3456,8 +3464,10 @@ decomposition of the design matrix.</p>
 <p>It means that the ordinary least square model (with the optimal
 parameters) <span class="math notranslate nohighlight">\(\boldsymbol{\tilde{y}}\)</span>, corresponds to an orthogonal
 transformation of the output (or target) vector <span class="math notranslate nohighlight">\(\boldsymbol{y}\)</span> by the
-vectors of the matrix <span class="math notranslate nohighlight">\(\boldsymbol{U}\)</span>. Note that the summation ends at <span class="math notranslate nohighlight">\(p-1\)</span>,
-that is <span class="math notranslate nohighlight">\(\boldsymbol{\tilde{y}}\ne \boldsymbol{y}\)</span>.</p>
+vectors of the matrix <span class="math notranslate nohighlight">\(\boldsymbol{U}\)</span>. <strong>Note that the summation ends at</strong>
+<span class="math notranslate nohighlight">\(p-1\)</span>, that is <span class="math notranslate nohighlight">\(\boldsymbol{\tilde{y}}\ne \boldsymbol{y}\)</span>. We can thus not use the
+orthogonality relation for the matrix <span class="math notranslate nohighlight">\(\boldsymbol{U}\)</span>. This can already be
+when we multiply the matrices <span class="math notranslate nohighlight">\(\boldsymbol{\Sigma}^T\boldsymbol{U}^T\)</span>.</p>
 </div>
 <div class="section" id="further-properties-important-for-our-analyses-later">
 <h2>Further properties (important for our analyses later)<a class="headerlink" href="#further-properties-important-for-our-analyses-later" title="Permalink to this headline">¶</a></h2>

doc/LectureNotes/_build/jupyter_execute/week35.py

@@ -1919,7 +1919,7 @@ print(C-X)
 # \tilde{y}_{\mathrm{OLS}}=\boldsymbol{U}\boldsymbol{\Sigma}\boldsymbol{V}^T\left(\boldsymbol{V}\tilde{\boldsymbol{\Sigma}}^{2}(\boldsymbol{V}^T\right)^{-1}\boldsymbol{V}\boldsymbol{\Sigma}^T\boldsymbol{U}^T\boldsymbol{y},
 # $$
 
-# which gives us, using the orthogonality of the matrices $\boldsymbol{U}$ and $\boldsymbol{V}$,
+# which gives us, using the orthogonality of the matrix $\boldsymbol{V}$,
 
 # $$
 # \tilde{y}_{\mathrm{OLS}}=\boldsymbol{U}\boldsymbol{U}^T\boldsymbol{y}=\sum_{i=0}^{p-1}\boldsymbol{u}_i\boldsymbol{u}^T_i\boldsymbol{y},
@@ -1928,8 +1928,10 @@ print(C-X)
 # It means that the ordinary least square model (with the optimal
 # parameters) $\boldsymbol{\tilde{y}}$, corresponds to an orthogonal
 # transformation of the output (or target) vector $\boldsymbol{y}$ by the
-# vectors of the matrix $\boldsymbol{U}$. Note that the summation ends at $p-1$,
-# that is $\boldsymbol{\tilde{y}}\ne \boldsymbol{y}$.
+# vectors of the matrix $\boldsymbol{U}$. **Note that the summation ends at**
+# $p-1$, that is $\boldsymbol{\tilde{y}}\ne \boldsymbol{y}$. We can thus not use the
+# orthogonality relation for the matrix $\boldsymbol{U}$. This can already be
+# when we multiply the matrices $\boldsymbol{\Sigma}^T\boldsymbol{U}^T$.
 
 # ## Further properties (important for our analyses later)
 # 

doc/pub/week35/html/._week35-bs055.html

@@ -395,7 +395,7 @@ $$
 \tilde{y}_{\mathrm{OLS}}=\boldsymbol{U}\boldsymbol{\Sigma}\boldsymbol{V}^T\left(\boldsymbol{V}\tilde{\boldsymbol{\Sigma}}^{2}(\boldsymbol{V}^T\right)^{-1}\boldsymbol{V}\boldsymbol{\Sigma}^T\boldsymbol{U}^T\boldsymbol{y},
 $$
 
-<p>which gives us, using the orthogonality of the matrices \( \boldsymbol{U} \) and \( \boldsymbol{V} \),</p>
+<p>which gives us, using the orthogonality of the matrix \( \boldsymbol{V} \),</p>
 
 $$
 \tilde{y}_{\mathrm{OLS}}=\boldsymbol{U}\boldsymbol{U}^T\boldsymbol{y}=\sum_{i=0}^{p-1}\boldsymbol{u}_i\boldsymbol{u}^T_i\boldsymbol{y},
@@ -404,8 +404,10 @@ $$
 <p>It means that the ordinary least square model (with the optimal
 parameters) \( \boldsymbol{\tilde{y}} \), corresponds to an orthogonal
 transformation of the output (or target) vector \( \boldsymbol{y} \) by the
-vectors of the matrix \( \boldsymbol{U} \). Note that the summation ends at \( p-1 \),
-that is \( \boldsymbol{\tilde{y}}\ne \boldsymbol{y} \).
+vectors of the matrix \( \boldsymbol{U} \). <b>Note that the summation ends at</b>
+\( p-1 \), that is \( \boldsymbol{\tilde{y}}\ne \boldsymbol{y} \). We can thus not use the
+orthogonality relation for the matrix \( \boldsymbol{U} \). This can already be
+when we multiply the matrices \( \boldsymbol{\Sigma}^T\boldsymbol{U}^T \).
 </p>
 
 <p>

doc/pub/week35/html/week35-reveal.html

@@ -2860,7 +2860,7 @@ $$
 $$
 <p>&nbsp;<br>
 
-<p>which gives us, using the orthogonality of the matrices \( \boldsymbol{U} \) and \( \boldsymbol{V} \),</p>
+<p>which gives us, using the orthogonality of the matrix \( \boldsymbol{V} \),</p>
 
 <p>&nbsp;<br>
 $$
@@ -2871,8 +2871,10 @@ $$
 <p>It means that the ordinary least square model (with the optimal
 parameters) \( \boldsymbol{\tilde{y}} \), corresponds to an orthogonal
 transformation of the output (or target) vector \( \boldsymbol{y} \) by the
-vectors of the matrix \( \boldsymbol{U} \). Note that the summation ends at \( p-1 \),
-that is \( \boldsymbol{\tilde{y}}\ne \boldsymbol{y} \).
+vectors of the matrix \( \boldsymbol{U} \). <b>Note that the summation ends at</b>
+\( p-1 \), that is \( \boldsymbol{\tilde{y}}\ne \boldsymbol{y} \). We can thus not use the
+orthogonality relation for the matrix \( \boldsymbol{U} \). This can already be
+when we multiply the matrices \( \boldsymbol{\Sigma}^T\boldsymbol{U}^T \).
 </p>
 </section>
 

doc/pub/week35/html/week35-solarized.html

@@ -2750,7 +2750,7 @@ $$
 \tilde{y}_{\mathrm{OLS}}=\boldsymbol{U}\boldsymbol{\Sigma}\boldsymbol{V}^T\left(\boldsymbol{V}\tilde{\boldsymbol{\Sigma}}^{2}(\boldsymbol{V}^T\right)^{-1}\boldsymbol{V}\boldsymbol{\Sigma}^T\boldsymbol{U}^T\boldsymbol{y},
 $$
 
-<p>which gives us, using the orthogonality of the matrices \( \boldsymbol{U} \) and \( \boldsymbol{V} \),</p>
+<p>which gives us, using the orthogonality of the matrix \( \boldsymbol{V} \),</p>
 
 $$
 \tilde{y}_{\mathrm{OLS}}=\boldsymbol{U}\boldsymbol{U}^T\boldsymbol{y}=\sum_{i=0}^{p-1}\boldsymbol{u}_i\boldsymbol{u}^T_i\boldsymbol{y},
@@ -2759,8 +2759,10 @@ $$
 <p>It means that the ordinary least square model (with the optimal
 parameters) \( \boldsymbol{\tilde{y}} \), corresponds to an orthogonal
 transformation of the output (or target) vector \( \boldsymbol{y} \) by the
-vectors of the matrix \( \boldsymbol{U} \). Note that the summation ends at \( p-1 \),
-that is \( \boldsymbol{\tilde{y}}\ne \boldsymbol{y} \).
+vectors of the matrix \( \boldsymbol{U} \). <b>Note that the summation ends at</b>
+\( p-1 \), that is \( \boldsymbol{\tilde{y}}\ne \boldsymbol{y} \). We can thus not use the
+orthogonality relation for the matrix \( \boldsymbol{U} \). This can already be
+when we multiply the matrices \( \boldsymbol{\Sigma}^T\boldsymbol{U}^T \).
 </p>
 
 <!-- !split --><br><br><br><br><br><br><br><br><br><br>

doc/pub/week35/html/week35.html

@@ -2827,7 +2827,7 @@ $$
 \tilde{y}_{\mathrm{OLS}}=\boldsymbol{U}\boldsymbol{\Sigma}\boldsymbol{V}^T\left(\boldsymbol{V}\tilde{\boldsymbol{\Sigma}}^{2}(\boldsymbol{V}^T\right)^{-1}\boldsymbol{V}\boldsymbol{\Sigma}^T\boldsymbol{U}^T\boldsymbol{y},
 $$
 
-<p>which gives us, using the orthogonality of the matrices \( \boldsymbol{U} \) and \( \boldsymbol{V} \),</p>
+<p>which gives us, using the orthogonality of the matrix \( \boldsymbol{V} \),</p>
 
 $$
 \tilde{y}_{\mathrm{OLS}}=\boldsymbol{U}\boldsymbol{U}^T\boldsymbol{y}=\sum_{i=0}^{p-1}\boldsymbol{u}_i\boldsymbol{u}^T_i\boldsymbol{y},
@@ -2836,8 +2836,10 @@ $$
 <p>It means that the ordinary least square model (with the optimal
 parameters) \( \boldsymbol{\tilde{y}} \), corresponds to an orthogonal
 transformation of the output (or target) vector \( \boldsymbol{y} \) by the
-vectors of the matrix \( \boldsymbol{U} \). Note that the summation ends at \( p-1 \),
-that is \( \boldsymbol{\tilde{y}}\ne \boldsymbol{y} \).
+vectors of the matrix \( \boldsymbol{U} \). <b>Note that the summation ends at</b>
+\( p-1 \), that is \( \boldsymbol{\tilde{y}}\ne \boldsymbol{y} \). We can thus not use the
+orthogonality relation for the matrix \( \boldsymbol{U} \). This can already be
+when we multiply the matrices \( \boldsymbol{\Sigma}^T\boldsymbol{U}^T \).
 </p>
 
 <!-- !split --><br><br><br><br><br><br><br><br><br><br>

doc/src/week35/week35.do.txt

@@ -2050,7 +2050,7 @@ and using our SVD decomposition of $\bm{X}$ we have
 \tilde{y}_{\mathrm{OLS}}=\bm{U}\bm{\Sigma}\bm{V}^T\left(\bm{V}\tilde{\bm{\Sigma}}^{2}(\bm{V}^T\right)^{-1}\bm{V}\bm{\Sigma}^T\bm{U}^T\bm{y},
 \]
 !et
-which gives us, using the orthogonality of the matrices $\bm{U}$ and $\bm{V}$,
+which gives us, using the orthogonality of the matrix $\bm{V}$,
 
 !bt
 \[
@@ -2061,8 +2061,10 @@ which gives us, using the orthogonality of the matrices $\bm{U}$ and $\bm{V}$,
 It means that the ordinary least square model (with the optimal
 parameters) $\bm{\tilde{y}}$, corresponds to an orthogonal
 transformation of the output (or target) vector $\bm{y}$ by the
-vectors of the matrix $\bm{U}$. Note that the summation ends at $p-1$,
-that is $\bm{\tilde{y}}\ne \bm{y}$.
+vectors of the matrix $\bm{U}$. _Note that the summation ends at_
+$p-1$, that is $\bm{\tilde{y}}\ne \bm{y}$. We can thus not use the
+orthogonality relation for the matrix $\bm{U}$. This can already be
+when we multiply the matrices $\bm{\Sigma}^T\bm{U}^T$.
 
 !split
 ===== Further properties (important for our analyses later) =====