+This means the vectors \( \boldsymbol{v}_i \) of the orthogonal matrix \( \boldsymbol{V} \)
+are the eigenvectors of the matrix \( \boldsymbol{X}^T\boldsymbol{X} \) with eigenvalues
+given by the singular values squared, that is
-This means the vectors \( \boldsymbol{v}_i \) of the orthogonal matrix \( \boldsymbol{V} \) are the eigenvectors of the matrix \( \boldsymbol{X}^T\boldsymbol{X} \)
-with eigenvalues given by the singular values squared, that is
+In other words, each non-zero singular value of \( \boldsymbol{X} \) is a positive
+square root of an eigenvalue of \( \boldsymbol{X}^T\boldsymbol{X} \). It means also that
+the columns of \( \boldsymbol{V} \) are the eigenvectors of
+\( \boldsymbol{X}^T\boldsymbol{X} \). Since we have ordered the singular values of
+\( \boldsymbol{X} \) in a descending order, it means that the column vectors
+\( \boldsymbol{v}_i \) are hierarchically ordered by how much correlation they
+encode from the columns of \( \boldsymbol{X} \).
+
+
+Note that these are also the eigenvectors and eigenvalues of the
+Hessian matrix.
+
+
+If we now recall the definition of the covariance matrix (not using
+Bessel's correction) we have
+
+
+meaning that every squared non-singular value of \( \boldsymbol{X} \) divided by$n$,
+the number of samples, are the eigenvalues of the covariance
+matrix. Every singular value of \( \boldsymbol{X} \) is thus a positive square
+root of an eigenvalue of \( \boldsymbol{X}^T\boldsymbol{X} \). If the matrix \( \boldsymbol{X} \) is
+self-adjoint, the the sinular values of \( \boldsymbol{X} \) are equal to the
+absolute value of the eigenvalues of \( \boldsymbol{X} \).
+
+For \( \boldsymbol{X}\boldsymbol{X}^T \) we found
+
+$$
+$\boldsymbol{X}\boldsymbol{X}^T$=\boldsymbol{U}\boldsymbol{\Sigma}\boldsymbol{V}^T\boldsymbol{V}\boldsymbol{\Sigma}^T\boldsymbol{U}^T=\boldsymbol{U}\boldsymbol{\Sigma}^T\boldsymbol{\Sigma}\boldsymbol{U}^T.
+
+Multiplying with \( \boldsymbol{U} \) from the right gives us the eigenvalue problem
+$$
+
+It means that the eigenvalues of \( \boldsymbol{X}\boldsymbol{X}^T \) are again given by the non-zero singular values plus now a series of zeros. However, when performing the matrix-matrix multiplications. The column vectors of \( \boldsymbol{U} \) are the eigenvectors of \( \boldsymbol{X}\boldsymbol{X}^T \) and measure how much correlations are contained in the rows of \( \m{X} \).
+
+
+Since we will mainly be interested in the correlations among features of our data, the quantity of interest for us are the non-zero singular values and the column vectors of \( \boldsymbol{V} \).
+
+
+
+What does it mean?
+
+
$$
\left(\boldsymbol{X}^T\boldsymbol{X}\right)\boldsymbol{v}_i=\boldsymbol{v}_i\sigma_i^2.
$$
+
+
+$$
+\boldsymbol{C}[\boldsymbol{X}]=\frac{1}{n}\boldsymbol{X}^T\boldsymbol{X},
+$$
+
+
+And finally \( \boldsymbol{X}\boldsymbol{X}^T \)
+
+
+$$
+
+Since the matrices here have dimension \( n\times n \), we have
+$$
+
+\boldsymbol{\Sigma}\boldsymbol{\Sigma}^T = \begin{bmatrix} \tilde{\boldsymbol{\Sigma}} \\ \boldsymbol{0}\\ \end{bmatrix}\begin{bmatrix} \tilde{\boldsymbol{\Sigma}} 0 \boldsymbol{0}\\ \end{bmatrix}=\begin{bmatrix} \tilde{\boldsymbol{\Sigma}} & \boldsymbol{0} \\ \boldsymbol{0} & \boldsymbol{0}\\ \end{bmatrix},
+
+$$
+
+leading to
+$$
+
+$\boldsymbol{X}\boldsymbol{X}^T$=\boldsymbol{U}\begin{bmatrix} \tilde{\boldsymbol{\Sigma}} & \boldsymbol{0} \\ \boldsymbol{0} & \boldsymbol{0}\\ \end{bmatrix}\boldsymbol{U}^T.
+
+$$
+
+
+$\boldsymbol{X}\boldsymbol{X}^T$\boldsymbol{U}=\boldsymbol{U}\begin{bmatrix} \tilde{\boldsymbol{\Sigma}} & \boldsymbol{0} \\ \boldsymbol{0} & \boldsymbol{0}\\ \end{bmatrix}.
+$$
+
+
+
+
+
+This means the vectors \( \boldsymbol{v}_i \) of the orthogonal matrix \( \boldsymbol{V} \) +are the eigenvectors of the matrix \( \boldsymbol{X}^T\boldsymbol{X} \) with eigenvalues +given by the singular values squared, that is + $$ \left(\boldsymbol{X}^T\boldsymbol{X}\right)\boldsymbol{v}_i=\boldsymbol{v}_i\sigma_i^2. $$ +
+In other words, each non-zero singular value of \( \boldsymbol{X} \) is a positive +square root of an eigenvalue of \( \boldsymbol{X}^T\boldsymbol{X} \). It means also that +the columns of \( \boldsymbol{V} \) are the eigenvectors of +\( \boldsymbol{X}^T\boldsymbol{X} \). Since we have ordered the singular values of +\( \boldsymbol{X} \) in a descending order, it means that the column vectors +\( \boldsymbol{v}_i \) are hierarchically ordered by how much correlation they +encode from the columns of \( \boldsymbol{X} \). + +
+Note that these are also the eigenvectors and eigenvalues of the +Hessian matrix. + +
+If we now recall the definition of the covariance matrix (not using +Bessel's correction) we have + +$$ +\boldsymbol{C}[\boldsymbol{X}]=\frac{1}{n}\boldsymbol{X}^T\boldsymbol{X}, +$$ + +
+meaning that every squared non-singular value of \( \boldsymbol{X} \) divided by$n$, +the number of samples, are the eigenvalues of the covariance +matrix. Every singular value of \( \boldsymbol{X} \) is thus a positive square +root of an eigenvalue of \( \boldsymbol{X}^T\boldsymbol{X} \). If the matrix \( \boldsymbol{X} \) is +self-adjoint, the the sinular values of \( \boldsymbol{X} \) are equal to the +absolute value of the eigenvalues of \( \boldsymbol{X} \). + +
+
+
+
+For \( \boldsymbol{X}\boldsymbol{X}^T \) we found + +$$ +$\boldsymbol{X}\boldsymbol{X}^T$=\boldsymbol{U}\boldsymbol{\Sigma}\boldsymbol{V}^T\boldsymbol{V}\boldsymbol{\Sigma}^T\boldsymbol{U}^T=\boldsymbol{U}\boldsymbol{\Sigma}^T\boldsymbol{\Sigma}\boldsymbol{U}^T. +$$ + +Since the matrices here have dimension \( n\times n \), we have +$$ +\boldsymbol{\Sigma}\boldsymbol{\Sigma}^T = \begin{bmatrix} \tilde{\boldsymbol{\Sigma}} \\ \boldsymbol{0}\\ \end{bmatrix}\begin{bmatrix} \tilde{\boldsymbol{\Sigma}} 0 \boldsymbol{0}\\ \end{bmatrix}=\begin{bmatrix} \tilde{\boldsymbol{\Sigma}} & \boldsymbol{0} \\ \boldsymbol{0} & \boldsymbol{0}\\ \end{bmatrix}, +$$ + +leading to +$$ +$\boldsymbol{X}\boldsymbol{X}^T$=\boldsymbol{U}\begin{bmatrix} \tilde{\boldsymbol{\Sigma}} & \boldsymbol{0} \\ \boldsymbol{0} & \boldsymbol{0}\\ \end{bmatrix}\boldsymbol{U}^T. +$$ + +
+Multiplying with \( \boldsymbol{U} \) from the right gives us the eigenvalue problem +$$ +$\boldsymbol{X}\boldsymbol{X}^T$\boldsymbol{U}=\boldsymbol{U}\begin{bmatrix} \tilde{\boldsymbol{\Sigma}} & \boldsymbol{0} \\ \boldsymbol{0} & \boldsymbol{0}\\ \end{bmatrix}. +$$ + +
+It means that the eigenvalues of \( \boldsymbol{X}\boldsymbol{X}^T \) are again given by the non-zero singular values plus now a series of zeros. However, when performing the matrix-matrix multiplications. The column vectors of \( \boldsymbol{U} \) are the eigenvectors of \( \boldsymbol{X}\boldsymbol{X}^T \) and measure how much correlations are contained in the rows of \( \m{X} \). + +
+Since we will mainly be interested in the correlations among features of our data, the quantity of interest for us are the non-zero singular values and the column vectors of \( \boldsymbol{V} \). +
diff --git a/doc/pub/week35/html/week35.html b/doc/pub/week35/html/week35.html
index b261f20e9..3efd21e35 100644
--- a/doc/pub/week35/html/week35.html
+++ b/doc/pub/week35/html/week35.html
@@ -255,6 +255,11 @@ div { text-align: justify; text-justify: inter-word; }
None,
'rewriting-the-covariance-and-or-correlation-matrix'),
('Linking with the SVD', 2, None, 'linking-with-the-svd'),
+ ('What does it mean?', 2, None, 'what-does-it-mean'),
+ ('And finally $\\boldsymbol{X}\\boldsymbol{X}^T$',
+ 2,
+ None,
+ 'and-finally-boldsymbol-x-boldsymbol-x-t'),
('Ridge and LASSO Regression',
2,
None,
@@ -2723,12 +2728,83 @@ $$
\left(\boldsymbol{X}^T\boldsymbol{X}\right)\boldsymbol{V}=\boldsymbol{V}\tilde{\boldsymbol{\Sigma}}^2.
$$
-This means the vectors \( \boldsymbol{v}_i \) of the orthogonal matrix \( \boldsymbol{V} \) are the eigenvectors of the matrix \( \boldsymbol{X}^T\boldsymbol{X} \)
-with eigenvalues given by the singular values squared, that is
+
+
+
+
+This means the vectors \( \boldsymbol{v}_i \) of the orthogonal matrix \( \boldsymbol{V} \) +are the eigenvectors of the matrix \( \boldsymbol{X}^T\boldsymbol{X} \) with eigenvalues +given by the singular values squared, that is + $$ \left(\boldsymbol{X}^T\boldsymbol{X}\right)\boldsymbol{v}_i=\boldsymbol{v}_i\sigma_i^2. $$ +
+In other words, each non-zero singular value of \( \boldsymbol{X} \) is a positive +square root of an eigenvalue of \( \boldsymbol{X}^T\boldsymbol{X} \). It means also that +the columns of \( \boldsymbol{V} \) are the eigenvectors of +\( \boldsymbol{X}^T\boldsymbol{X} \). Since we have ordered the singular values of +\( \boldsymbol{X} \) in a descending order, it means that the column vectors +\( \boldsymbol{v}_i \) are hierarchically ordered by how much correlation they +encode from the columns of \( \boldsymbol{X} \). + +
+Note that these are also the eigenvectors and eigenvalues of the +Hessian matrix. + +
+If we now recall the definition of the covariance matrix (not using +Bessel's correction) we have + +$$ +\boldsymbol{C}[\boldsymbol{X}]=\frac{1}{n}\boldsymbol{X}^T\boldsymbol{X}, +$$ + +
+meaning that every squared non-singular value of \( \boldsymbol{X} \) divided by$n$, +the number of samples, are the eigenvalues of the covariance +matrix. Every singular value of \( \boldsymbol{X} \) is thus a positive square +root of an eigenvalue of \( \boldsymbol{X}^T\boldsymbol{X} \). If the matrix \( \boldsymbol{X} \) is +self-adjoint, the the sinular values of \( \boldsymbol{X} \) are equal to the +absolute value of the eigenvalues of \( \boldsymbol{X} \). + +
+
+
+
+For \( \boldsymbol{X}\boldsymbol{X}^T \) we found + +$$ +$\boldsymbol{X}\boldsymbol{X}^T$=\boldsymbol{U}\boldsymbol{\Sigma}\boldsymbol{V}^T\boldsymbol{V}\boldsymbol{\Sigma}^T\boldsymbol{U}^T=\boldsymbol{U}\boldsymbol{\Sigma}^T\boldsymbol{\Sigma}\boldsymbol{U}^T. +$$ + +Since the matrices here have dimension \( n\times n \), we have +$$ +\boldsymbol{\Sigma}\boldsymbol{\Sigma}^T = \begin{bmatrix} \tilde{\boldsymbol{\Sigma}} \\ \boldsymbol{0}\\ \end{bmatrix}\begin{bmatrix} \tilde{\boldsymbol{\Sigma}} 0 \boldsymbol{0}\\ \end{bmatrix}=\begin{bmatrix} \tilde{\boldsymbol{\Sigma}} & \boldsymbol{0} \\ \boldsymbol{0} & \boldsymbol{0}\\ \end{bmatrix}, +$$ + +leading to +$$ +$\boldsymbol{X}\boldsymbol{X}^T$=\boldsymbol{U}\begin{bmatrix} \tilde{\boldsymbol{\Sigma}} & \boldsymbol{0} \\ \boldsymbol{0} & \boldsymbol{0}\\ \end{bmatrix}\boldsymbol{U}^T. +$$ + +
+Multiplying with \( \boldsymbol{U} \) from the right gives us the eigenvalue problem +$$ +$\boldsymbol{X}\boldsymbol{X}^T$\boldsymbol{U}=\boldsymbol{U}\begin{bmatrix} \tilde{\boldsymbol{\Sigma}} & \boldsymbol{0} \\ \boldsymbol{0} & \boldsymbol{0}\\ \end{bmatrix}. +$$ + +
+It means that the eigenvalues of \( \boldsymbol{X}\boldsymbol{X}^T \) are again given by the non-zero singular values plus now a series of zeros. However, when performing the matrix-matrix multiplications. The column vectors of \( \boldsymbol{U} \) are the eigenvectors of \( \boldsymbol{X}\boldsymbol{X}^T \) and measure how much correlations are contained in the rows of \( \m{X} \). + +
+Since we will mainly be interested in the correlations among features of our data, the quantity of interest for us are the non-zero singular values and the column vectors of \( \boldsymbol{V} \). +
diff --git a/doc/pub/week35/ipynb/ipynb-week35-src.tar.gz b/doc/pub/week35/ipynb/ipynb-week35-src.tar.gz
index 0978c9ee2..76b4c6ce1 100644
Binary files a/doc/pub/week35/ipynb/ipynb-week35-src.tar.gz and b/doc/pub/week35/ipynb/ipynb-week35-src.tar.gz differ
diff --git a/doc/pub/week35/ipynb/week35.ipynb b/doc/pub/week35/ipynb/week35.ipynb
index b78df7984..86a83a788 100644
--- a/doc/pub/week35/ipynb/week35.ipynb
+++ b/doc/pub/week35/ipynb/week35.ipynb
@@ -3531,8 +3531,11 @@
"cell_type": "markdown",
"metadata": {},
"source": [
- "This means the vectors $\\boldsymbol{v}_i$ of the orthogonal matrix $\\boldsymbol{V}$ are the eigenvectors of the matrix $\\boldsymbol{X}^T\\boldsymbol{X}$\n",
- "with eigenvalues given by the singular values squared, that is"
+ "## What does it mean?\n",
+ "\n",
+ "This means the vectors $\\boldsymbol{v}_i$ of the orthogonal matrix $\\boldsymbol{V}$\n",
+ "are the eigenvectors of the matrix $\\boldsymbol{X}^T\\boldsymbol{X}$ with eigenvalues\n",
+ "given by the singular values squared, that is"
]
},
{
@@ -3548,6 +3551,113 @@
"cell_type": "markdown",
"metadata": {},
"source": [
+ "In other words, each non-zero singular value of $\\boldsymbol{X}$ is a positive\n",
+ "square root of an eigenvalue of $\\boldsymbol{X}^T\\boldsymbol{X}$. It means also that\n",
+ "the columns of $\\boldsymbol{V}$ are the eigenvectors of\n",
+ "$\\boldsymbol{X}^T\\boldsymbol{X}$. Since we have ordered the singular values of\n",
+ "$\\boldsymbol{X}$ in a descending order, it means that the column vectors\n",
+ "$\\boldsymbol{v}_i$ are hierarchically ordered by how much correlation they\n",
+ "encode from the columns of $\\boldsymbol{X}$. \n",
+ "\n",
+ "\n",
+ "Note that these are also the eigenvectors and eigenvalues of the\n",
+ "Hessian matrix.\n",
+ "\n",
+ "If we now recall the definition of the covariance matrix (not using\n",
+ "Bessel's correction) we have"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "$$\n",
+ "\\boldsymbol{C}[\\boldsymbol{X}]=\\frac{1}{n}\\boldsymbol{X}^T\\boldsymbol{X},\n",
+ "$$"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "meaning that every squared non-singular value of $\\boldsymbol{X}$ divided by$n$,\n",
+ "the number of samples, are the eigenvalues of the covariance\n",
+ "matrix. Every singular value of $\\boldsymbol{X}$ is thus a positive square\n",
+ "root of an eigenvalue of $\\boldsymbol{X}^T\\boldsymbol{X}$. If the matrix $\\boldsymbol{X}$ is\n",
+ "self-adjoint, the the sinular values of $\\boldsymbol{X}$ are equal to the\n",
+ "absolute value of the eigenvalues of $\\boldsymbol{X}$.\n",
+ "\n",
+ "## And finally $\\boldsymbol{X}\\boldsymbol{X}^T$\n",
+ "\n",
+ "For $\\boldsymbol{X}\\boldsymbol{X}^T$ we found"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "$$\n",
+ "$\\boldsymbol{X}\\boldsymbol{X}^T$=\\boldsymbol{U}\\boldsymbol{\\Sigma}\\boldsymbol{V}^T\\boldsymbol{V}\\boldsymbol{\\Sigma}^T\\boldsymbol{U}^T=\\boldsymbol{U}\\boldsymbol{\\Sigma}^T\\boldsymbol{\\Sigma}\\boldsymbol{U}^T.\n",
+ "$$"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Since the matrices here have dimension $n\\times n$, we have"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "$$\n",
+ "\\boldsymbol{\\Sigma}\\boldsymbol{\\Sigma}^T = \\begin{bmatrix} \\tilde{\\boldsymbol{\\Sigma}} \\\\ \\boldsymbol{0}\\\\ \\end{bmatrix}\\begin{bmatrix} \\tilde{\\boldsymbol{\\Sigma}} 0 \\boldsymbol{0}\\\\ \\end{bmatrix}=\\begin{bmatrix} \\tilde{\\boldsymbol{\\Sigma}} & \\boldsymbol{0} \\\\ \\boldsymbol{0} & \\boldsymbol{0}\\\\ \\end{bmatrix},\n",
+ "$$"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "leading to"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "$$\n",
+ "$\\boldsymbol{X}\\boldsymbol{X}^T$=\\boldsymbol{U}\\begin{bmatrix} \\tilde{\\boldsymbol{\\Sigma}} & \\boldsymbol{0} \\\\ \\boldsymbol{0} & \\boldsymbol{0}\\\\ \\end{bmatrix}\\boldsymbol{U}^T.\n",
+ "$$"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Multiplying with $\\boldsymbol{U}$ from the right gives us the eigenvalue problem"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "$$\n",
+ "$\\boldsymbol{X}\\boldsymbol{X}^T$\\boldsymbol{U}=\\boldsymbol{U}\\begin{bmatrix} \\tilde{\\boldsymbol{\\Sigma}} & \\boldsymbol{0} \\\\ \\boldsymbol{0} & \\boldsymbol{0}\\\\ \\end{bmatrix}.\n",
+ "$$"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "It means that the eigenvalues of $\\boldsymbol{X}\\boldsymbol{X}^T$ are again given by the non-zero singular values plus now a series of zeros. However, when performing the matrix-matrix multiplications. The column vectors of $\\boldsymbol{U}$ are the eigenvectors of $\\boldsymbol{X}\\boldsymbol{X}^T$ and measure how much correlations are contained in the rows of $\\m{X}$.\n",
+ "\n",
+ "Since we will mainly be interested in the correlations among features of our data, the quantity of interest for us are the non-zero singular values and the column vectors of $\\boldsymbol{V}$.\n",
+ "\n",
+ "\n",
"## Ridge and LASSO Regression\n",
"\n",
"Let us remind ourselves about the expression for the standard Mean Squared Error (MSE) which we used to define our cost function and the equations for the ordinary least squares (OLS) method, that is \n",
diff --git a/doc/src/week35/week35.do.txt b/doc/src/week35/week35.do.txt
index 9216eccec..0038d9465 100644
--- a/doc/src/week35/week35.do.txt
+++ b/doc/src/week35/week35.do.txt
@@ -2208,17 +2208,82 @@ Multiplying from the right with $\bm{V}$ (using the orthogonality of $\bm{V}$) w
\left(\bm{X}^T\bm{X}\right)\bm{V}=\bm{V}\tilde{\bm{\Sigma}}^2.
\]
!et
-This means the vectors $\bm{v}_i$ of the orthogonal matrix $\bm{V}$ are the eigenvectors of the matrix $\bm{X}^T\bm{X}$
-with eigenvalues given by the singular values squared, that is
+
+!split
+===== What does it mean? =====
+
+This means the vectors $\bm{v}_i$ of the orthogonal matrix $\bm{V}$
+are the eigenvectors of the matrix $\bm{X}^T\bm{X}$ with eigenvalues
+given by the singular values squared, that is
+
!bt
\[
\left(\bm{X}^T\bm{X}\right)\bm{v}_i=\bm{v}_i\sigma_i^2.
\]
!et
+In other words, each non-zero singular value of $\bm{X}$ is a positive
+square root of an eigenvalue of $\bm{X}^T\bm{X}$. It means also that
+the columns of $\bm{V}$ are the eigenvectors of
+$\bm{X}^T\bm{X}$. Since we have ordered the singular values of
+$\bm{X}$ in a descending order, it means that the column vectors
+$\bm{v}_i$ are hierarchically ordered by how much correlation they
+encode from the columns of $\bm{X}$.
+Note that these are also the eigenvectors and eigenvalues of the
+Hessian matrix.
+If we now recall the definition of the covariance matrix (not using
+Bessel's correction) we have
+
+
+!bt
+\[
+\bm{C}[\bm{X}]=\frac{1}{n}\bm{X}^T\bm{X},
+\]
+!et
+
+meaning that every squared non-singular value of $\bm{X}$ divided by$n$,
+the number of samples, are the eigenvalues of the covariance
+matrix. Every singular value of $\bm{X}$ is thus a positive square
+root of an eigenvalue of $\bm{X}^T\bm{X}$. If the matrix $\bm{X}$ is
+self-adjoint, the the sinular values of $\bm{X}$ are equal to the
+absolute value of the eigenvalues of $\bm{X}$.
+
+!split
+===== And finally $\bm{X}\bm{X}^T$ =====
+
+For $\bm{X}\bm{X}^T$ we found
+
+!bt
+\[
+$\bm{X}\bm{X}^T$=\bm{U}\bm{\Sigma}\bm{V}^T\bm{V}\bm{\Sigma}^T\bm{U}^T=\bm{U}\bm{\Sigma}^T\bm{\Sigma}\bm{U}^T.
+\]
+!et
+Since the matrices here have dimension $n\times n$, we have
+!bt
+\[
+\bm{\Sigma}\bm{\Sigma}^T = \begin{bmatrix} \tilde{\bm{\Sigma}} \\ \bm{0}\\ \end{bmatrix}\begin{bmatrix} \tilde{\bm{\Sigma}} 0 \bm{0}\\ \end{bmatrix}=\begin{bmatrix} \tilde{\bm{\Sigma}} & \bm{0} \\ \bm{0} & \bm{0}\\ \end{bmatrix},
+\]
+!et
+leading to
+!bt
+\[
+$\bm{X}\bm{X}^T$=\bm{U}\begin{bmatrix} \tilde{\bm{\Sigma}} & \bm{0} \\ \bm{0} & \bm{0}\\ \end{bmatrix}\bm{U}^T.
+\]
+!et
+
+Multiplying with $\bm{U}$ from the right gives us the eigenvalue problem
+!bt
+\[
+$\bm{X}\bm{X}^T$\bm{U}=\bm{U}\begin{bmatrix} \tilde{\bm{\Sigma}} & \bm{0} \\ \bm{0} & \bm{0}\\ \end{bmatrix}.
+\]
+!et
+
+It means that the eigenvalues of $\bm{X}\bm{X}^T$ are again given by the non-zero singular values plus now a series of zeros. However, when performing the matrix-matrix multiplications. The column vectors of $\bm{U}$ are the eigenvectors of $\bm{X}\bm{X}^T$ and measure how much correlations are contained in the rows of $\m{X}$.
+
+Since we will mainly be interested in the correlations among features of our data, the quantity of interest for us are the non-zero singular values and the column vectors of $\bm{V}$.
!split