-We define $\boldsymbol{\Sigma}^T\boldsymbol{\sigma}=\boldsymbol{D}^2, where \( \boldsymbol{D} \) is a diagonal matrix containing only the singular values squared. It has dimensionality \( p \times p \).
+We define \( \boldsymbol{\Sigma}^T\boldsymbol{\Sigma}=\tilde{\boldsymbol{\Sigma}}^2 \) which is a diagonal matrix containing only the singular values squared. It has dimensionality \( p \times p \).
This means, using the orthogonality of \( \boldsymbol{V} \), that we get
$$
-\boldsymbol{X}^T\boldsymbol{X}=\boldsymbol{D}^2.
+\boldsymbol{X}^T\boldsymbol{X}=\tilde{\boldsymbol{\Sigma}}^2.
$$
@@ -2162,7 +2162,7 @@ and using our SVD decomposition of \( \boldsymbol{X} \) we have
$$
-\tilde{y}_{\mathrm{OLS}}=\boldsymbol{U}\boldsymbol{\Sigma}\boldsymbol{V}^T\boldsymbol{D}^{-2}\boldsymbol{V}\boldsymbol{\Sigma}^T\boldsymbol{U}^T\boldsymbol{y},
+\tilde{y}_{\mathrm{OLS}}=\boldsymbol{U}\boldsymbol{\Sigma}\boldsymbol{V}^T\tilde{\boldsymbol{\Sigma}}^{-2}\boldsymbol{V}\boldsymbol{\Sigma}^T\boldsymbol{U}^T\boldsymbol{y},
$$
@@ -2182,7 +2182,7 @@ $$
$$
-that belong to \( i>p-1 \), give all zeros when we perform the multiplications. This means the sum above has non-zero elements only up to \( i=p-1 \). This corresponds also to the number of singular values (these are all non-zero).
+that belong to \( i>p-1 \), result in only zeros when we perform the multiplications. This means that the sum above has non-zero elements only up to \( i=p-1 \). This corresponds also to the number of singular values (these are all non-zero).
It means that the ordinary least square model (with the optimal parameters) \( \boldsymbol{\tilde{y}} \), corresponds to an orthogonal transformation of the output (or target) vector \( \boldsymbol{y} \) by the vectors of the matrix \( \boldsymbol{U} \). diff --git a/doc/pub/week35/html/week35-solarized.html b/doc/pub/week35/html/week35-solarized.html index 9e956bfbe..62835ff47 100644 --- a/doc/pub/week35/html/week35-solarized.html +++ b/doc/pub/week35/html/week35-solarized.html @@ -2199,13 +2199,13 @@ $$ \boldsymbol{X}^T\boldsymbol{X}=\boldsymbol{V}\boldsymbol{\Sigma}^T\boldsymbol{\Sigma}\boldsymbol{V}^T. $$ -We define $\boldsymbol{\Sigma}^T\boldsymbol{\sigma}=\boldsymbol{D}^2, where \( \boldsymbol{D} \) is a diagonal matrix containing only the singular values squared. It has dimensionality \( p \times p \). +We define \( \boldsymbol{\Sigma}^T\boldsymbol{\Sigma}=\tilde{\boldsymbol{\Sigma}}^2 \) which is a diagonal matrix containing only the singular values squared. It has dimensionality \( p \times p \).
This means, using the orthogonality of \( \boldsymbol{V} \), that we get $$ -\boldsymbol{X}^T\boldsymbol{X}=\boldsymbol{D}^2. +\boldsymbol{X}^T\boldsymbol{X}=\tilde{\boldsymbol{\Sigma}}^2. $$
@@ -2218,7 +2218,7 @@ $$ and using our SVD decomposition of \( \boldsymbol{X} \) we have $$ -\tilde{y}_{\mathrm{OLS}}=\boldsymbol{U}\boldsymbol{\Sigma}\boldsymbol{V}^T\boldsymbol{D}^{-2}\boldsymbol{V}\boldsymbol{\Sigma}^T\boldsymbol{U}^T\boldsymbol{y}, +\tilde{y}_{\mathrm{OLS}}=\boldsymbol{U}\boldsymbol{\Sigma}\boldsymbol{V}^T\tilde{\boldsymbol{\Sigma}}^{-2}\boldsymbol{V}\boldsymbol{\Sigma}^T\boldsymbol{U}^T\boldsymbol{y}, $$ which gives us, using the orthogonality of the matrices \( \boldsymbol{U} \) and \( \boldsymbol{V} \), @@ -2233,7 +2233,7 @@ $$ \boldsymbol{U}=[\boldsymbol{u}_0,\boldsymbol{u}_1,\dots,\boldsymbol{u}_{n-1}], $$ -that belong to \( i>p-1 \), give all zeros when we perform the multiplications. This means the sum above has non-zero elements only up to \( i=p-1 \). This corresponds also to the number of singular values (these are all non-zero). +that belong to \( i>p-1 \), result in only zeros when we perform the multiplications. This means that the sum above has non-zero elements only up to \( i=p-1 \). This corresponds also to the number of singular values (these are all non-zero).
It means that the ordinary least square model (with the optimal parameters) \( \boldsymbol{\tilde{y}} \), corresponds to an orthogonal transformation of the output (or target) vector \( \boldsymbol{y} \) by the vectors of the matrix \( \boldsymbol{U} \). diff --git a/doc/pub/week35/html/week35.html b/doc/pub/week35/html/week35.html index 6644f495b..6cc72bd9a 100644 --- a/doc/pub/week35/html/week35.html +++ b/doc/pub/week35/html/week35.html @@ -2204,13 +2204,13 @@ $$ \boldsymbol{X}^T\boldsymbol{X}=\boldsymbol{V}\boldsymbol{\Sigma}^T\boldsymbol{\Sigma}\boldsymbol{V}^T. $$ -We define $\boldsymbol{\Sigma}^T\boldsymbol{\sigma}=\boldsymbol{D}^2, where \( \boldsymbol{D} \) is a diagonal matrix containing only the singular values squared. It has dimensionality \( p \times p \). +We define \( \boldsymbol{\Sigma}^T\boldsymbol{\Sigma}=\tilde{\boldsymbol{\Sigma}}^2 \) which is a diagonal matrix containing only the singular values squared. It has dimensionality \( p \times p \).
This means, using the orthogonality of \( \boldsymbol{V} \), that we get $$ -\boldsymbol{X}^T\boldsymbol{X}=\boldsymbol{D}^2. +\boldsymbol{X}^T\boldsymbol{X}=\tilde{\boldsymbol{\Sigma}}^2. $$
@@ -2223,7 +2223,7 @@ $$ and using our SVD decomposition of \( \boldsymbol{X} \) we have $$ -\tilde{y}_{\mathrm{OLS}}=\boldsymbol{U}\boldsymbol{\Sigma}\boldsymbol{V}^T\boldsymbol{D}^{-2}\boldsymbol{V}\boldsymbol{\Sigma}^T\boldsymbol{U}^T\boldsymbol{y}, +\tilde{y}_{\mathrm{OLS}}=\boldsymbol{U}\boldsymbol{\Sigma}\boldsymbol{V}^T\tilde{\boldsymbol{\Sigma}}^{-2}\boldsymbol{V}\boldsymbol{\Sigma}^T\boldsymbol{U}^T\boldsymbol{y}, $$ which gives us, using the orthogonality of the matrices \( \boldsymbol{U} \) and \( \boldsymbol{V} \), @@ -2238,7 +2238,7 @@ $$ \boldsymbol{U}=[\boldsymbol{u}_0,\boldsymbol{u}_1,\dots,\boldsymbol{u}_{n-1}], $$ -that belong to \( i>p-1 \), give all zeros when we perform the multiplications. This means the sum above has non-zero elements only up to \( i=p-1 \). This corresponds also to the number of singular values (these are all non-zero). +that belong to \( i>p-1 \), result in only zeros when we perform the multiplications. This means that the sum above has non-zero elements only up to \( i=p-1 \). This corresponds also to the number of singular values (these are all non-zero).
It means that the ordinary least square model (with the optimal parameters) \( \boldsymbol{\tilde{y}} \), corresponds to an orthogonal transformation of the output (or target) vector \( \boldsymbol{y} \) by the vectors of the matrix \( \boldsymbol{U} \). diff --git a/doc/pub/week35/ipynb/ipynb-week35-src.tar.gz b/doc/pub/week35/ipynb/ipynb-week35-src.tar.gz index e4a7ec63d..1f3502f6d 100644 Binary files a/doc/pub/week35/ipynb/ipynb-week35-src.tar.gz and b/doc/pub/week35/ipynb/ipynb-week35-src.tar.gz differ diff --git a/doc/pub/week35/ipynb/week35.ipynb b/doc/pub/week35/ipynb/week35.ipynb index f80ae50ae..fb01baf4e 100644 --- a/doc/pub/week35/ipynb/week35.ipynb +++ b/doc/pub/week35/ipynb/week35.ipynb @@ -2665,7 +2665,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "We define $\\boldsymbol{\\Sigma}^T\\boldsymbol{\\sigma}=\\boldsymbol{D}^2, where $\\boldsymbol{D}$ is a diagonal matrix containing only the singular values squared. It has dimensionality $p \\times p$.\n", + "We define $\\boldsymbol{\\Sigma}^T\\boldsymbol{\\Sigma}=\\tilde{\\boldsymbol{\\Sigma}}^2$ which is a diagonal matrix containing only the singular values squared. It has dimensionality $p \\times p$.\n", "\n", "This means, using the orthogonality of $\\boldsymbol{V}$, that we get" ] @@ -2675,7 +2675,7 @@ "metadata": {}, "source": [ "$$\n", - "\\boldsymbol{X}^T\\boldsymbol{X}=\\boldsymbol{D}^2.\n", + "\\boldsymbol{X}^T\\boldsymbol{X}=\\tilde{\\boldsymbol{\\Sigma}}^2.\n", "$$" ] }, @@ -2707,7 +2707,7 @@ "metadata": {}, "source": [ "$$\n", - "\\tilde{y}_{\\mathrm{OLS}}=\\boldsymbol{U}\\boldsymbol{\\Sigma}\\boldsymbol{V}^T\\boldsymbol{D}^{-2}\\boldsymbol{V}\\boldsymbol{\\Sigma}^T\\boldsymbol{U}^T\\boldsymbol{y},\n", + "\\tilde{y}_{\\mathrm{OLS}}=\\boldsymbol{U}\\boldsymbol{\\Sigma}\\boldsymbol{V}^T\\tilde{\\boldsymbol{\\Sigma}}^{-2}\\boldsymbol{V}\\boldsymbol{\\Sigma}^T\\boldsymbol{U}^T\\boldsymbol{y},\n", "$$" ] }, @@ -2747,7 +2747,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "that belong to $i>p-1$, give all zeros when we perform the multiplications. This means the sum above has non-zero elements only up to $i=p-1$. This corresponds also to the number of singular values (these are all non-zero).\n", + "that belong to $i>p-1$, result in only zeros when we perform the multiplications. This means that the sum above has non-zero elements only up to $i=p-1$. This corresponds also to the number of singular values (these are all non-zero).\n", "\n", "It means that the ordinary least square model (with the optimal parameters) $\\boldsymbol{\\tilde{y}}$, corresponds to an orthogonal transformation of the output (or target) vector $\\boldsymbol{y}$ by the vectors of the matrix $\\boldsymbol{U}$.\n", "\n", diff --git a/doc/src/week35/week35.do.txt b/doc/src/week35/week35.do.txt index b41d1e28e..27a8ce41b 100644 --- a/doc/src/week35/week35.do.txt +++ b/doc/src/week35/week35.do.txt @@ -1695,13 +1695,13 @@ and using the orthogonality of the matrix $\bm{U}$ we have \bm{X}^T\bm{X}=\bm{V}\bm{\Sigma}^T\bm{\Sigma}\bm{V}^T. \] !et -We define $\bm{\Sigma}^T\bm{\sigma}=\bm{D}^2, where $\bm{D}$ is a diagonal matrix containing only the singular values squared. It has dimensionality $p \times p$. +We define $\bm{\Sigma}^T\bm{\Sigma}=\tilde{\bm{\Sigma}}^2$ which is a diagonal matrix containing only the singular values squared. It has dimensionality $p \times p$. This means, using the orthogonality of $\bm{V}$, that we get !bt \[ -\bm{X}^T\bm{X}=\bm{D}^2. +\bm{X}^T\bm{X}=\tilde{\bm{\Sigma}}^2. \] !et @@ -1716,7 +1716,7 @@ and using our SVD decomposition of $\bm{X}$ we have !bt \[ -\tilde{y}_{\mathrm{OLS}}=\bm{U}\bm{\Sigma}\bm{V}^T\bm{D}^{-2}\bm{V}\bm{\Sigma}^T\bm{U}^T\bm{y}, +\tilde{y}_{\mathrm{OLS}}=\bm{U}\bm{\Sigma}\bm{V}^T\tilde{\bm{\Sigma}}^{-2}\bm{V}\bm{\Sigma}^T\bm{U}^T\bm{y}, \] !et which gives us, using the orthogonality of the matrices $\bm{U}$ and $\bm{V}$, @@ -1733,7 +1733,7 @@ Note here that when we perform the multiplication of the various matrices, the \bm{U}=[\bm{u}_0,\bm{u}_1,\dots,\bm{u}_{n-1}], \] !et -that belong to $i>p-1$, give all zeros when we perform the multiplications. This means the sum above has non-zero elements only up to $i=p-1$. This corresponds also to the number of singular values (these are all non-zero). +that belong to $i>p-1$, result in only zeros when we perform the multiplications. This means that the sum above has non-zero elements only up to $i=p-1$. This corresponds also to the number of singular values (these are all non-zero). It means that the ordinary least square model (with the optimal parameters) $\bm{\tilde{y}}$, corresponds to an orthogonal transformation of the output (or target) vector $\bm{y}$ by the vectors of the matrix $\bm{U}$.