update

doc/pub/week35/html/week35-reveal.html

@@ -2767,11 +2767,11 @@ $$
 <p>&nbsp;<br>
 
 <p>
-meaning that every squared non-singular value of \( \boldsymbol{X} \) divided by$n$,
-the number of samples, are the eigenvalues of the covariance
+meaning that every squared non-singular value of \( \boldsymbol{X} \) divided by \( n \) (
+the number of samples) are the eigenvalues of the covariance
 matrix. Every singular value of \( \boldsymbol{X} \) is thus a positive square
 root of an eigenvalue of \( \boldsymbol{X}^T\boldsymbol{X} \). If the matrix \( \boldsymbol{X} \) is
-self-adjoint, the the sinular values of \( \boldsymbol{X} \) are equal to the
+self-adjoint, the singular values of \( \boldsymbol{X} \) are equal to the
 absolute value of the eigenvalues of \( \boldsymbol{X} \).
 </section>
 
@@ -2782,31 +2782,33 @@ absolute value of the eigenvalues of \( \boldsymbol{X} \).
 <p>
 For \( \boldsymbol{X}\boldsymbol{X}^T \) we found
 
-$$
-$\boldsymbol{X}\boldsymbol{X}^T$=\boldsymbol{U}\boldsymbol{\Sigma}\boldsymbol{V}^T\boldsymbol{V}\boldsymbol{\Sigma}^T\boldsymbol{U}^T=\boldsymbol{U}\boldsymbol{\Sigma}^T\boldsymbol{\Sigma}\boldsymbol{U}^T. 
 <p>&nbsp;<br>
 $$
+\boldsymbol{X}\boldsymbol{X}^T=\boldsymbol{U}\boldsymbol{\Sigma}\boldsymbol{V}^T\boldsymbol{V}\boldsymbol{\Sigma}^T\boldsymbol{U}^T=\boldsymbol{U}\boldsymbol{\Sigma}^T\boldsymbol{\Sigma}\boldsymbol{U}^T. 
+$$
+<p>&nbsp;<br>
 
 Since the matrices here have dimension \( n\times n \), we have
-$$
-<p>&nbsp;<br>
-\boldsymbol{\Sigma}\boldsymbol{\Sigma}^T = \begin{bmatrix} \tilde{\boldsymbol{\Sigma}} \\ \boldsymbol{0}\\ \end{bmatrix}\begin{bmatrix} \tilde{\boldsymbol{\Sigma}} 0 \boldsymbol{0}\\ \end{bmatrix}=\begin{bmatrix} \tilde{\boldsymbol{\Sigma}} & \boldsymbol{0} \\ \boldsymbol{0} & \boldsymbol{0}\\ \end{bmatrix}, 
 <p>&nbsp;<br>
 $$
+\boldsymbol{\Sigma}\boldsymbol{\Sigma}^T = \begin{bmatrix} \tilde{\boldsymbol{\Sigma}} \\ \boldsymbol{0}\\ \end{bmatrix}\begin{bmatrix} \tilde{\boldsymbol{\Sigma}}  \boldsymbol{0}\\ \end{bmatrix}=\begin{bmatrix} \tilde{\boldsymbol{\Sigma}} & \boldsymbol{0} \\ \boldsymbol{0} & \boldsymbol{0}\\ \end{bmatrix}, 
+$$
+<p>&nbsp;<br>
 
 leading to
-$$
-<p>&nbsp;<br>
-$\boldsymbol{X}\boldsymbol{X}^T$=\boldsymbol{U}\begin{bmatrix} \tilde{\boldsymbol{\Sigma}} & \boldsymbol{0} \\ \boldsymbol{0} & \boldsymbol{0}\\ \end{bmatrix}\boldsymbol{U}^T. 
 <p>&nbsp;<br>
 $$
+\boldsymbol{X}\boldsymbol{X}^T=\boldsymbol{U}\begin{bmatrix} \tilde{\boldsymbol{\Sigma}} & \boldsymbol{0} \\ \boldsymbol{0} & \boldsymbol{0}\\ \end{bmatrix}\boldsymbol{U}^T. 
+$$
+<p>&nbsp;<br>
 
 <p>
 Multiplying with \( \boldsymbol{U} \) from the right gives us the eigenvalue problem
+<p>&nbsp;<br>
+$$
+(\boldsymbol{X}\boldsymbol{X}^T)\boldsymbol{U}=\boldsymbol{U}\begin{bmatrix} \tilde{\boldsymbol{\Sigma}} & \boldsymbol{0} \\ \boldsymbol{0} & \boldsymbol{0}\\ \end{bmatrix}. 
 $$
 <p>&nbsp;<br>
-$\boldsymbol{X}\boldsymbol{X}^T$\boldsymbol{U}=\boldsymbol{U}\begin{bmatrix} \tilde{\boldsymbol{\Sigma}} & \boldsymbol{0} \\ \boldsymbol{0} & \boldsymbol{0}\\ \end{bmatrix}. 
-$$
 
 <p>
 It means that the eigenvalues of \( \boldsymbol{X}\boldsymbol{X}^T \) are again given by
@@ -2950,7 +2952,7 @@ Using our insights about the SVD of the design matrix \( \boldsymbol{X} \)
 We have already analyzed the OLS solutions in terms of the eigenvectors (the columns) of the right singular value matrix \( \boldsymbol{U} \) as
 <p>&nbsp;<br>
 $$
-\boldsymbol{X}\boldsymbol{\beta} = =\boldsymbol{U}\boldsymbol{U}^T\boldsymbol{y}.
+\tilde{\boldsymbol{y}}_{\mathrm{OLS}}=\boldsymbol{X}\boldsymbol{\beta}  =\boldsymbol{U}\boldsymbol{U}^T\boldsymbol{y}.
 $$
 <p>&nbsp;<br>
 
@@ -2959,12 +2961,12 @@ For Ridge regression this becomes
 
 <p>&nbsp;<br>
 $$
-\boldsymbol{X}\boldsymbol{\beta}^{\mathrm{Ridge}} = \boldsymbol{U\Sigma V^T}\left(\boldsymbol{V}\boldsymbol{\Sigma}^2\boldsymbol{V}^T+\lambda\boldsymbol{I} \right)^{-1}(\boldsymbol{U\Sigma V^T})^T\boldsymbol{y}=\sum_{j=0}^{p-1}\boldsymbol{u}_j\boldsymbol{u}_j^T\frac{\sigma_j^2}{\sigma_j^2+\lambda}\boldsymbol{y},
+\tilde{\boldsymbol{y}}_{\mathrm{Ridge}}=\boldsymbol{X}\boldsymbol{\beta}_{\mathrm{Ridge}} = \boldsymbol{U\Sigma V^T}\left(\boldsymbol{V}\boldsymbol{\Sigma}^2\boldsymbol{V}^T+\lambda\boldsymbol{I} \right)^{-1}(\boldsymbol{U\Sigma V^T})^T\boldsymbol{y}=\sum_{j=0}^{p-1}\boldsymbol{u}_j\boldsymbol{u}_j^T\frac{\sigma_j^2}{\sigma_j^2+\lambda}\boldsymbol{y},
 $$
 <p>&nbsp;<br>
 
 <p>
-with the vectors \( \boldsymbol{u}_j \) being the columns of \( \boldsymbol{U} \).
+with the vectors \( \boldsymbol{u}_j \) being the columns of \( \boldsymbol{U} \) from the SVD of the matrix \( \boldsymbol{X} \).
 </section>
 
 

doc/pub/week35/html/week35-solarized.html

@@ -2753,11 +2753,11 @@ $$
 $$
 
 <p>
-meaning that every squared non-singular value of \( \boldsymbol{X} \) divided by$n$,
-the number of samples, are the eigenvalues of the covariance
+meaning that every squared non-singular value of \( \boldsymbol{X} \) divided by \( n \) (
+the number of samples) are the eigenvalues of the covariance
 matrix. Every singular value of \( \boldsymbol{X} \) is thus a positive square
 root of an eigenvalue of \( \boldsymbol{X}^T\boldsymbol{X} \). If the matrix \( \boldsymbol{X} \) is
-self-adjoint, the the sinular values of \( \boldsymbol{X} \) are equal to the
+self-adjoint, the singular values of \( \boldsymbol{X} \) are equal to the
 absolute value of the eigenvalues of \( \boldsymbol{X} \).
 
 <p>
@@ -2769,23 +2769,23 @@ absolute value of the eigenvalues of \( \boldsymbol{X} \).
 For \( \boldsymbol{X}\boldsymbol{X}^T \) we found
 
 $$
-$\boldsymbol{X}\boldsymbol{X}^T$=\boldsymbol{U}\boldsymbol{\Sigma}\boldsymbol{V}^T\boldsymbol{V}\boldsymbol{\Sigma}^T\boldsymbol{U}^T=\boldsymbol{U}\boldsymbol{\Sigma}^T\boldsymbol{\Sigma}\boldsymbol{U}^T. 
+\boldsymbol{X}\boldsymbol{X}^T=\boldsymbol{U}\boldsymbol{\Sigma}\boldsymbol{V}^T\boldsymbol{V}\boldsymbol{\Sigma}^T\boldsymbol{U}^T=\boldsymbol{U}\boldsymbol{\Sigma}^T\boldsymbol{\Sigma}\boldsymbol{U}^T. 
 $$
 
 Since the matrices here have dimension \( n\times n \), we have
 $$
-\boldsymbol{\Sigma}\boldsymbol{\Sigma}^T = \begin{bmatrix} \tilde{\boldsymbol{\Sigma}} \\ \boldsymbol{0}\\ \end{bmatrix}\begin{bmatrix} \tilde{\boldsymbol{\Sigma}} 0 \boldsymbol{0}\\ \end{bmatrix}=\begin{bmatrix} \tilde{\boldsymbol{\Sigma}} & \boldsymbol{0} \\ \boldsymbol{0} & \boldsymbol{0}\\ \end{bmatrix}, 
+\boldsymbol{\Sigma}\boldsymbol{\Sigma}^T = \begin{bmatrix} \tilde{\boldsymbol{\Sigma}} \\ \boldsymbol{0}\\ \end{bmatrix}\begin{bmatrix} \tilde{\boldsymbol{\Sigma}}  \boldsymbol{0}\\ \end{bmatrix}=\begin{bmatrix} \tilde{\boldsymbol{\Sigma}} & \boldsymbol{0} \\ \boldsymbol{0} & \boldsymbol{0}\\ \end{bmatrix}, 
 $$
 
 leading to
 $$
-$\boldsymbol{X}\boldsymbol{X}^T$=\boldsymbol{U}\begin{bmatrix} \tilde{\boldsymbol{\Sigma}} & \boldsymbol{0} \\ \boldsymbol{0} & \boldsymbol{0}\\ \end{bmatrix}\boldsymbol{U}^T. 
+\boldsymbol{X}\boldsymbol{X}^T=\boldsymbol{U}\begin{bmatrix} \tilde{\boldsymbol{\Sigma}} & \boldsymbol{0} \\ \boldsymbol{0} & \boldsymbol{0}\\ \end{bmatrix}\boldsymbol{U}^T. 
 $$
 
 <p>
 Multiplying with \( \boldsymbol{U} \) from the right gives us the eigenvalue problem
 $$
-$\boldsymbol{X}\boldsymbol{X}^T$\boldsymbol{U}=\boldsymbol{U}\begin{bmatrix} \tilde{\boldsymbol{\Sigma}} & \boldsymbol{0} \\ \boldsymbol{0} & \boldsymbol{0}\\ \end{bmatrix}. 
+(\boldsymbol{X}\boldsymbol{X}^T)\boldsymbol{U}=\boldsymbol{U}\begin{bmatrix} \tilde{\boldsymbol{\Sigma}} & \boldsymbol{0} \\ \boldsymbol{0} & \boldsymbol{0}\\ \end{bmatrix}. 
 $$
 
 <p>
@@ -2907,18 +2907,18 @@ even reduce the variance of the optimal parameters \( \boldsymbol{\beta} \). The
 Using our insights about the SVD of the design matrix \( \boldsymbol{X} \) 
 We have already analyzed the OLS solutions in terms of the eigenvectors (the columns) of the right singular value matrix \( \boldsymbol{U} \) as
 $$
-\boldsymbol{X}\boldsymbol{\beta} = =\boldsymbol{U}\boldsymbol{U}^T\boldsymbol{y}.
+\tilde{\boldsymbol{y}}_{\mathrm{OLS}}=\boldsymbol{X}\boldsymbol{\beta}  =\boldsymbol{U}\boldsymbol{U}^T\boldsymbol{y}.
 $$
 
 <p>
 For Ridge regression this becomes
 
 $$
-\boldsymbol{X}\boldsymbol{\beta}^{\mathrm{Ridge}} = \boldsymbol{U\Sigma V^T}\left(\boldsymbol{V}\boldsymbol{\Sigma}^2\boldsymbol{V}^T+\lambda\boldsymbol{I} \right)^{-1}(\boldsymbol{U\Sigma V^T})^T\boldsymbol{y}=\sum_{j=0}^{p-1}\boldsymbol{u}_j\boldsymbol{u}_j^T\frac{\sigma_j^2}{\sigma_j^2+\lambda}\boldsymbol{y},
+\tilde{\boldsymbol{y}}_{\mathrm{Ridge}}=\boldsymbol{X}\boldsymbol{\beta}_{\mathrm{Ridge}} = \boldsymbol{U\Sigma V^T}\left(\boldsymbol{V}\boldsymbol{\Sigma}^2\boldsymbol{V}^T+\lambda\boldsymbol{I} \right)^{-1}(\boldsymbol{U\Sigma V^T})^T\boldsymbol{y}=\sum_{j=0}^{p-1}\boldsymbol{u}_j\boldsymbol{u}_j^T\frac{\sigma_j^2}{\sigma_j^2+\lambda}\boldsymbol{y},
 $$
 
 <p>
-with the vectors \( \boldsymbol{u}_j \) being the columns of \( \boldsymbol{U} \).
+with the vectors \( \boldsymbol{u}_j \) being the columns of \( \boldsymbol{U} \) from the SVD of the matrix \( \boldsymbol{X} \).
 
 <p>
 <!-- !split --><br><br><br><br><br><br><br><br><br><br>

doc/pub/week35/html/week35.html

@@ -2758,11 +2758,11 @@ $$
 $$
 
 <p>
-meaning that every squared non-singular value of \( \boldsymbol{X} \) divided by$n$,
-the number of samples, are the eigenvalues of the covariance
+meaning that every squared non-singular value of \( \boldsymbol{X} \) divided by \( n \) (
+the number of samples) are the eigenvalues of the covariance
 matrix. Every singular value of \( \boldsymbol{X} \) is thus a positive square
 root of an eigenvalue of \( \boldsymbol{X}^T\boldsymbol{X} \). If the matrix \( \boldsymbol{X} \) is
-self-adjoint, the the sinular values of \( \boldsymbol{X} \) are equal to the
+self-adjoint, the singular values of \( \boldsymbol{X} \) are equal to the
 absolute value of the eigenvalues of \( \boldsymbol{X} \).
 
 <p>
@@ -2774,23 +2774,23 @@ absolute value of the eigenvalues of \( \boldsymbol{X} \).
 For \( \boldsymbol{X}\boldsymbol{X}^T \) we found
 
 $$
-$\boldsymbol{X}\boldsymbol{X}^T$=\boldsymbol{U}\boldsymbol{\Sigma}\boldsymbol{V}^T\boldsymbol{V}\boldsymbol{\Sigma}^T\boldsymbol{U}^T=\boldsymbol{U}\boldsymbol{\Sigma}^T\boldsymbol{\Sigma}\boldsymbol{U}^T. 
+\boldsymbol{X}\boldsymbol{X}^T=\boldsymbol{U}\boldsymbol{\Sigma}\boldsymbol{V}^T\boldsymbol{V}\boldsymbol{\Sigma}^T\boldsymbol{U}^T=\boldsymbol{U}\boldsymbol{\Sigma}^T\boldsymbol{\Sigma}\boldsymbol{U}^T. 
 $$
 
 Since the matrices here have dimension \( n\times n \), we have
 $$
-\boldsymbol{\Sigma}\boldsymbol{\Sigma}^T = \begin{bmatrix} \tilde{\boldsymbol{\Sigma}} \\ \boldsymbol{0}\\ \end{bmatrix}\begin{bmatrix} \tilde{\boldsymbol{\Sigma}} 0 \boldsymbol{0}\\ \end{bmatrix}=\begin{bmatrix} \tilde{\boldsymbol{\Sigma}} & \boldsymbol{0} \\ \boldsymbol{0} & \boldsymbol{0}\\ \end{bmatrix}, 
+\boldsymbol{\Sigma}\boldsymbol{\Sigma}^T = \begin{bmatrix} \tilde{\boldsymbol{\Sigma}} \\ \boldsymbol{0}\\ \end{bmatrix}\begin{bmatrix} \tilde{\boldsymbol{\Sigma}}  \boldsymbol{0}\\ \end{bmatrix}=\begin{bmatrix} \tilde{\boldsymbol{\Sigma}} & \boldsymbol{0} \\ \boldsymbol{0} & \boldsymbol{0}\\ \end{bmatrix}, 
 $$
 
 leading to
 $$
-$\boldsymbol{X}\boldsymbol{X}^T$=\boldsymbol{U}\begin{bmatrix} \tilde{\boldsymbol{\Sigma}} & \boldsymbol{0} \\ \boldsymbol{0} & \boldsymbol{0}\\ \end{bmatrix}\boldsymbol{U}^T. 
+\boldsymbol{X}\boldsymbol{X}^T=\boldsymbol{U}\begin{bmatrix} \tilde{\boldsymbol{\Sigma}} & \boldsymbol{0} \\ \boldsymbol{0} & \boldsymbol{0}\\ \end{bmatrix}\boldsymbol{U}^T. 
 $$
 
 <p>
 Multiplying with \( \boldsymbol{U} \) from the right gives us the eigenvalue problem
 $$
-$\boldsymbol{X}\boldsymbol{X}^T$\boldsymbol{U}=\boldsymbol{U}\begin{bmatrix} \tilde{\boldsymbol{\Sigma}} & \boldsymbol{0} \\ \boldsymbol{0} & \boldsymbol{0}\\ \end{bmatrix}. 
+(\boldsymbol{X}\boldsymbol{X}^T)\boldsymbol{U}=\boldsymbol{U}\begin{bmatrix} \tilde{\boldsymbol{\Sigma}} & \boldsymbol{0} \\ \boldsymbol{0} & \boldsymbol{0}\\ \end{bmatrix}. 
 $$
 
 <p>
@@ -2912,18 +2912,18 @@ even reduce the variance of the optimal parameters \( \boldsymbol{\beta} \). The
 Using our insights about the SVD of the design matrix \( \boldsymbol{X} \) 
 We have already analyzed the OLS solutions in terms of the eigenvectors (the columns) of the right singular value matrix \( \boldsymbol{U} \) as
 $$
-\boldsymbol{X}\boldsymbol{\beta} = =\boldsymbol{U}\boldsymbol{U}^T\boldsymbol{y}.
+\tilde{\boldsymbol{y}}_{\mathrm{OLS}}=\boldsymbol{X}\boldsymbol{\beta}  =\boldsymbol{U}\boldsymbol{U}^T\boldsymbol{y}.
 $$
 
 <p>
 For Ridge regression this becomes
 
 $$
-\boldsymbol{X}\boldsymbol{\beta}^{\mathrm{Ridge}} = \boldsymbol{U\Sigma V^T}\left(\boldsymbol{V}\boldsymbol{\Sigma}^2\boldsymbol{V}^T+\lambda\boldsymbol{I} \right)^{-1}(\boldsymbol{U\Sigma V^T})^T\boldsymbol{y}=\sum_{j=0}^{p-1}\boldsymbol{u}_j\boldsymbol{u}_j^T\frac{\sigma_j^2}{\sigma_j^2+\lambda}\boldsymbol{y},
+\tilde{\boldsymbol{y}}_{\mathrm{Ridge}}=\boldsymbol{X}\boldsymbol{\beta}_{\mathrm{Ridge}} = \boldsymbol{U\Sigma V^T}\left(\boldsymbol{V}\boldsymbol{\Sigma}^2\boldsymbol{V}^T+\lambda\boldsymbol{I} \right)^{-1}(\boldsymbol{U\Sigma V^T})^T\boldsymbol{y}=\sum_{j=0}^{p-1}\boldsymbol{u}_j\boldsymbol{u}_j^T\frac{\sigma_j^2}{\sigma_j^2+\lambda}\boldsymbol{y},
 $$
 
 <p>
-with the vectors \( \boldsymbol{u}_j \) being the columns of \( \boldsymbol{U} \).
+with the vectors \( \boldsymbol{u}_j \) being the columns of \( \boldsymbol{U} \) from the SVD of the matrix \( \boldsymbol{X} \).
 
 <p>
 <!-- !split --><br><br><br><br><br><br><br><br><br><br>

doc/pub/week35/ipynb/week35.ipynb

@@ -3580,11 +3580,11 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "meaning that every squared non-singular value of $\\boldsymbol{X}$ divided by$n$,\n",
-    "the number of samples, are the eigenvalues of the covariance\n",
+    "meaning that every squared non-singular value of $\\boldsymbol{X}$ divided by $n$ (\n",
+    "the number of samples) are the eigenvalues of the covariance\n",
     "matrix. Every singular value of $\\boldsymbol{X}$ is thus a positive square\n",
     "root of an eigenvalue of $\\boldsymbol{X}^T\\boldsymbol{X}$. If the matrix $\\boldsymbol{X}$ is\n",
-    "self-adjoint, the the sinular values of $\\boldsymbol{X}$ are equal to the\n",
+    "self-adjoint, the singular values of $\\boldsymbol{X}$ are equal to the\n",
     "absolute value of the eigenvalues of $\\boldsymbol{X}$.\n",
     "\n",
     "## And finally  $\\boldsymbol{X}\\boldsymbol{X}^T$\n",
@@ -3597,7 +3597,7 @@
    "metadata": {},
    "source": [
     "$$\n",
-    "$\\boldsymbol{X}\\boldsymbol{X}^T$=\\boldsymbol{U}\\boldsymbol{\\Sigma}\\boldsymbol{V}^T\\boldsymbol{V}\\boldsymbol{\\Sigma}^T\\boldsymbol{U}^T=\\boldsymbol{U}\\boldsymbol{\\Sigma}^T\\boldsymbol{\\Sigma}\\boldsymbol{U}^T.\n",
+    "\\boldsymbol{X}\\boldsymbol{X}^T=\\boldsymbol{U}\\boldsymbol{\\Sigma}\\boldsymbol{V}^T\\boldsymbol{V}\\boldsymbol{\\Sigma}^T\\boldsymbol{U}^T=\\boldsymbol{U}\\boldsymbol{\\Sigma}^T\\boldsymbol{\\Sigma}\\boldsymbol{U}^T.\n",
     "$$"
    ]
   },
@@ -3613,7 +3613,7 @@
    "metadata": {},
    "source": [
     "$$\n",
-    "\\boldsymbol{\\Sigma}\\boldsymbol{\\Sigma}^T = \\begin{bmatrix} \\tilde{\\boldsymbol{\\Sigma}} \\\\ \\boldsymbol{0}\\\\ \\end{bmatrix}\\begin{bmatrix} \\tilde{\\boldsymbol{\\Sigma}} 0 \\boldsymbol{0}\\\\ \\end{bmatrix}=\\begin{bmatrix} \\tilde{\\boldsymbol{\\Sigma}} & \\boldsymbol{0} \\\\ \\boldsymbol{0} & \\boldsymbol{0}\\\\ \\end{bmatrix},\n",
+    "\\boldsymbol{\\Sigma}\\boldsymbol{\\Sigma}^T = \\begin{bmatrix} \\tilde{\\boldsymbol{\\Sigma}} \\\\ \\boldsymbol{0}\\\\ \\end{bmatrix}\\begin{bmatrix} \\tilde{\\boldsymbol{\\Sigma}}  \\boldsymbol{0}\\\\ \\end{bmatrix}=\\begin{bmatrix} \\tilde{\\boldsymbol{\\Sigma}} & \\boldsymbol{0} \\\\ \\boldsymbol{0} & \\boldsymbol{0}\\\\ \\end{bmatrix},\n",
     "$$"
    ]
   },
@@ -3629,7 +3629,7 @@
    "metadata": {},
    "source": [
     "$$\n",
-    "$\\boldsymbol{X}\\boldsymbol{X}^T$=\\boldsymbol{U}\\begin{bmatrix} \\tilde{\\boldsymbol{\\Sigma}} & \\boldsymbol{0} \\\\ \\boldsymbol{0} & \\boldsymbol{0}\\\\ \\end{bmatrix}\\boldsymbol{U}^T.\n",
+    "\\boldsymbol{X}\\boldsymbol{X}^T=\\boldsymbol{U}\\begin{bmatrix} \\tilde{\\boldsymbol{\\Sigma}} & \\boldsymbol{0} \\\\ \\boldsymbol{0} & \\boldsymbol{0}\\\\ \\end{bmatrix}\\boldsymbol{U}^T.\n",
     "$$"
    ]
   },
@@ -3645,7 +3645,7 @@
    "metadata": {},
    "source": [
     "$$\n",
-    "$\\boldsymbol{X}\\boldsymbol{X}^T$\\boldsymbol{U}=\\boldsymbol{U}\\begin{bmatrix} \\tilde{\\boldsymbol{\\Sigma}} & \\boldsymbol{0} \\\\ \\boldsymbol{0} & \\boldsymbol{0}\\\\ \\end{bmatrix}.\n",
+    "(\\boldsymbol{X}\\boldsymbol{X}^T)\\boldsymbol{U}=\\boldsymbol{U}\\begin{bmatrix} \\tilde{\\boldsymbol{\\Sigma}} & \\boldsymbol{0} \\\\ \\boldsymbol{0} & \\boldsymbol{0}\\\\ \\end{bmatrix}.\n",
     "$$"
    ]
   },
@@ -3877,7 +3877,7 @@
    "metadata": {},
    "source": [
     "$$\n",
-    "\\boldsymbol{X}\\boldsymbol{\\beta} = =\\boldsymbol{U}\\boldsymbol{U}^T\\boldsymbol{y}.\n",
+    "\\tilde{\\boldsymbol{y}}_{\\mathrm{OLS}}=\\boldsymbol{X}\\boldsymbol{\\beta}  =\\boldsymbol{U}\\boldsymbol{U}^T\\boldsymbol{y}.\n",
     "$$"
    ]
   },
@@ -3893,7 +3893,7 @@
    "metadata": {},
    "source": [
     "$$\n",
-    "\\boldsymbol{X}\\boldsymbol{\\beta}^{\\mathrm{Ridge}} = \\boldsymbol{U\\Sigma V^T}\\left(\\boldsymbol{V}\\boldsymbol{\\Sigma}^2\\boldsymbol{V}^T+\\lambda\\boldsymbol{I} \\right)^{-1}(\\boldsymbol{U\\Sigma V^T})^T\\boldsymbol{y}=\\sum_{j=0}^{p-1}\\boldsymbol{u}_j\\boldsymbol{u}_j^T\\frac{\\sigma_j^2}{\\sigma_j^2+\\lambda}\\boldsymbol{y},\n",
+    "\\tilde{\\boldsymbol{y}}_{\\mathrm{Ridge}}=\\boldsymbol{X}\\boldsymbol{\\beta}_{\\mathrm{Ridge}} = \\boldsymbol{U\\Sigma V^T}\\left(\\boldsymbol{V}\\boldsymbol{\\Sigma}^2\\boldsymbol{V}^T+\\lambda\\boldsymbol{I} \\right)^{-1}(\\boldsymbol{U\\Sigma V^T})^T\\boldsymbol{y}=\\sum_{j=0}^{p-1}\\boldsymbol{u}_j\\boldsymbol{u}_j^T\\frac{\\sigma_j^2}{\\sigma_j^2+\\lambda}\\boldsymbol{y},\n",
     "$$"
    ]
   },
@@ -3901,7 +3901,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "with the vectors $\\boldsymbol{u}_j$ being the columns of $\\boldsymbol{U}$. \n",
+    "with the vectors $\\boldsymbol{u}_j$ being the columns of $\\boldsymbol{U}$ from the SVD of the matrix $\\boldsymbol{X}$. \n",
     "\n",
     "## Interpreting the Ridge results\n",
     "\n",

doc/src/week35/week35.do.txt

@@ -2244,11 +2244,11 @@ Bessel's correction) we have
 \]
 !et
 
-meaning that every squared non-singular value of $\bm{X}$ divided by$n$,
-the number of samples, are the eigenvalues of the covariance
+meaning that every squared non-singular value of $\bm{X}$ divided by $n$ (
+the number of samples) are the eigenvalues of the covariance
 matrix. Every singular value of $\bm{X}$ is thus a positive square
 root of an eigenvalue of $\bm{X}^T\bm{X}$. If the matrix $\bm{X}$ is
-self-adjoint, the the sinular values of $\bm{X}$ are equal to the
+self-adjoint, the singular values of $\bm{X}$ are equal to the
 absolute value of the eigenvalues of $\bm{X}$.
 
 !split
@@ -2258,26 +2258,26 @@ For $\bm{X}\bm{X}^T$ we found
 
 !bt
 \[
-$\bm{X}\bm{X}^T$=\bm{U}\bm{\Sigma}\bm{V}^T\bm{V}\bm{\Sigma}^T\bm{U}^T=\bm{U}\bm{\Sigma}^T\bm{\Sigma}\bm{U}^T. 
+\bm{X}\bm{X}^T=\bm{U}\bm{\Sigma}\bm{V}^T\bm{V}\bm{\Sigma}^T\bm{U}^T=\bm{U}\bm{\Sigma}^T\bm{\Sigma}\bm{U}^T. 
 \]
 !et
 Since the matrices here have dimension $n\times n$, we have
 !bt
 \[
-\bm{\Sigma}\bm{\Sigma}^T = \begin{bmatrix} \tilde{\bm{\Sigma}} \\ \bm{0}\\ \end{bmatrix}\begin{bmatrix} \tilde{\bm{\Sigma}} 0 \bm{0}\\ \end{bmatrix}=\begin{bmatrix} \tilde{\bm{\Sigma}} & \bm{0} \\ \bm{0} & \bm{0}\\ \end{bmatrix}, 
+\bm{\Sigma}\bm{\Sigma}^T = \begin{bmatrix} \tilde{\bm{\Sigma}} \\ \bm{0}\\ \end{bmatrix}\begin{bmatrix} \tilde{\bm{\Sigma}}  \bm{0}\\ \end{bmatrix}=\begin{bmatrix} \tilde{\bm{\Sigma}} & \bm{0} \\ \bm{0} & \bm{0}\\ \end{bmatrix}, 
 \]
 !et
 leading to
 !bt
 \[
-$\bm{X}\bm{X}^T$=\bm{U}\begin{bmatrix} \tilde{\bm{\Sigma}} & \bm{0} \\ \bm{0} & \bm{0}\\ \end{bmatrix}\bm{U}^T. 
+\bm{X}\bm{X}^T=\bm{U}\begin{bmatrix} \tilde{\bm{\Sigma}} & \bm{0} \\ \bm{0} & \bm{0}\\ \end{bmatrix}\bm{U}^T. 
 \]
 !et
 
 Multiplying with $\bm{U}$ from the right gives us the eigenvalue problem
 !bt
 \[
-$\bm{X}\bm{X}^T$\bm{U}=\bm{U}\begin{bmatrix} \tilde{\bm{\Sigma}} & \bm{0} \\ \bm{0} & \bm{0}\\ \end{bmatrix}. 
+(\bm{X}\bm{X}^T)\bm{U}=\bm{U}\begin{bmatrix} \tilde{\bm{\Sigma}} & \bm{0} \\ \bm{0} & \bm{0}\\ \end{bmatrix}. 
 \]
 !et
 
@@ -2404,7 +2404,7 @@ Using our insights about the SVD of the design matrix $\bm{X}$
 We have already analyzed the OLS solutions in terms of the eigenvectors (the columns) of the right singular value matrix $\bm{U}$ as
 !bt
 \[
-\bm{X}\bm{\beta} = =\bm{U}\bm{U}^T\bm{y}.
+\tilde{\bm{y}}_{\mathrm{OLS}}=\bm{X}\bm{\beta}  =\bm{U}\bm{U}^T\bm{y}.
 \]
 !et
 
@@ -2413,11 +2413,11 @@ For Ridge regression this becomes
 
 !bt
 \[
-\bm{X}\bm{\beta}^{\mathrm{Ridge}} = \bm{U\Sigma V^T}\left(\bm{V}\bm{\Sigma}^2\bm{V}^T+\lambda\bm{I} \right)^{-1}(\bm{U\Sigma V^T})^T\bm{y}=\sum_{j=0}^{p-1}\bm{u}_j\bm{u}_j^T\frac{\sigma_j^2}{\sigma_j^2+\lambda}\bm{y},
+\tilde{\bm{y}}_{\mathrm{Ridge}}=\bm{X}\bm{\beta}_{\mathrm{Ridge}} = \bm{U\Sigma V^T}\left(\bm{V}\bm{\Sigma}^2\bm{V}^T+\lambda\bm{I} \right)^{-1}(\bm{U\Sigma V^T})^T\bm{y}=\sum_{j=0}^{p-1}\bm{u}_j\bm{u}_j^T\frac{\sigma_j^2}{\sigma_j^2+\lambda}\bm{y},
 \]
 !et
 
-with the vectors $\bm{u}_j$ being the columns of $\bm{U}$. 
+with the vectors $\bm{u}_j$ being the columns of $\bm{U}$ from the SVD of the matrix $\bm{X}$. 
 
 !split
 ===== Interpreting the Ridge results =====