We can rewrite the covariance matrix in a more compact form in terms of the design/feature matrix \( \boldsymbol{X} \) as $$ -\boldsymbol{C}[\boldsymbol{x}] = \frac{1}{n}\boldsymbol{X}\boldsymbol{X}^T= \mathbb{E}[\boldsymbol{X}\boldsymbol{X}^T]. +\boldsymbol{C}[\boldsymbol{x}] = \frac{1}{n}\boldsymbol{X}^T\boldsymbol{X}= \mathbb{E}[\boldsymbol{X}^T\boldsymbol{X}]. $$
@@ -326,7 +326,7 @@ $$
If we then compute the expectation value $$ -\mathbb{E}[\boldsymbol{X}\boldsymbol{X}^T] = \frac{1}{n}\boldsymbol{X}\boldsymbol{X}^T=\begin{bmatrix} +\mathbb{E}[\boldsymbol{X}^T\boldsymbol{X}] = \frac{1}{n}\boldsymbol{X}^T\boldsymbol{X}=\begin{bmatrix} x_{00}^2+x_{01}^2 & x_{00}x_{10}+x_{01}x_{11}\\ x_{10}x_{00}+x_{11}x_{01} & x_{10}^2+x_{11}^2\\ \end{bmatrix}, diff --git a/doc/pub/week43/html/._week43-bs060.html b/doc/pub/week43/html/._week43-bs060.html index 21246e91b..61c87c892 100644 --- a/doc/pub/week43/html/._week43-bs060.html +++ b/doc/pub/week43/html/._week43-bs060.html @@ -309,30 +309,30 @@ MathJax.Hub.Config({
We have that the covariance matrix (the correlation matrix involves a simple rescaling) is given as $$ -\boldsymbol{C}[\boldsymbol{x}] = \frac{1}{n}\boldsymbol{X}\boldsymbol{X}^T= \mathbb{E}[\boldsymbol{X}\boldsymbol{X}^T]. +\boldsymbol{C}[\boldsymbol{x}] = \frac{1}{n}\boldsymbol{X}^T\boldsymbol{X}= \mathbb{E}[\boldsymbol{X}^T\boldsymbol{X}]. $$ Let us now assume that we can perform a series of orthogonal transformations where we employ some orthogonal matrices \( \boldsymbol{S} \). These matrices are defined as \( \boldsymbol{S}\in {\mathbb{R}}^{p\times p} \) and obey the orthogonality requirements \( \boldsymbol{S}\boldsymbol{S}^T=\boldsymbol{S}^T\boldsymbol{S}=\boldsymbol{I} \). The matrix can be written out in terms of the column vectors \( \boldsymbol{s}_i \) as \( \boldsymbol{S}=[\boldsymbol{s}_0,\boldsymbol{s}_1,\dots,\boldsymbol{s}_{p-1}] \) and \( \boldsymbol{s}_i \in {\mathbb{R}}^{p} \).
-Assume also that there is a transformation \( \boldsymbol{S}\boldsymbol{C}[\boldsymbol{x}]\boldsymbol{S}^T=\boldsymbol{C}[\boldsymbol{y}] \) such that the new matrix \( \boldsymbol{C}[\boldsymbol{y}] \) is diagonal with elements \( [\lambda_0,\lambda_1,\lambda_2,\dots,\lambda_{p-1}] \). +Assume also that there is a transformation \( \boldsymbol{S}^T\boldsymbol{C}[\boldsymbol{x}]\boldsymbol{S}=\boldsymbol{C}[\boldsymbol{y}] \) such that the new matrix \( \boldsymbol{C}[\boldsymbol{y}] \) is diagonal with elements \( [\lambda_0,\lambda_1,\lambda_2,\dots,\lambda_{p-1}] \).
That is we have $$ -\boldsymbol{C}[\boldsymbol{y}] = \mathbb{E}[\boldsymbol{S}\boldsymbol{X}\boldsymbol{X}^T\boldsymbol{S}^T]=\boldsymbol{S}\boldsymbol{C}[\boldsymbol{x}]\boldsymbol{S}^T, +\boldsymbol{C}[\boldsymbol{y}] = \mathbb{E}[\boldsymbol{S}^T\boldsymbol{X}^T\boldsymbol{X}T\boldsymbol{S}]=\boldsymbol{S}^T\boldsymbol{C}[\boldsymbol{x}]\boldsymbol{S}, $$ -since the matrix \( \boldsymbol{S} \) is not a data dependent matrix. Multiplying with \( \boldsymbol{S}^T \) from the left we have +since the matrix \( \boldsymbol{S} \) is not a data dependent matrix. Multiplying with \( \boldsymbol{S} \) from the left we have $$ -\boldsymbol{S}^T\boldsymbol{C}[\boldsymbol{y}] = \boldsymbol{C}[\boldsymbol{x}]\boldsymbol{S}^T, +\boldsymbol{S}\boldsymbol{C}[\boldsymbol{y}] = \boldsymbol{C}[\boldsymbol{x}]\boldsymbol{S}, $$ and since \( \boldsymbol{C}[\boldsymbol{y}] \) is diagonal we have for a given eigenvalue \( i \) of the covariance matrix that $$ -\boldsymbol{S}^T_i\lambda_i = \boldsymbol{C}[\boldsymbol{x}]\boldsymbol{S}^T_i. +\boldsymbol{S}_i\lambda_i = \boldsymbol{C}[\boldsymbol{x}]\boldsymbol{S}_i. $$
diff --git a/doc/pub/week43/html/week43-reveal.html b/doc/pub/week43/html/week43-reveal.html index 6446cfe85..0b372e64c 100644 --- a/doc/pub/week43/html/week43-reveal.html +++ b/doc/pub/week43/html/week43-reveal.html @@ -3302,7 +3302,7 @@ matrix without them. We can rewrite the covariance matrix in a more compact form in terms of the design/feature matrix \( \boldsymbol{X} \) as
$$
-\boldsymbol{C}[\boldsymbol{x}] = \frac{1}{n}\boldsymbol{X}\boldsymbol{X}^T= \mathbb{E}[\boldsymbol{X}\boldsymbol{X}^T].
+\boldsymbol{C}[\boldsymbol{x}] = \frac{1}{n}\boldsymbol{X}^T\boldsymbol{X}= \mathbb{E}[\boldsymbol{X}^T\boldsymbol{X}].
$$
@@ -3323,7 +3323,7 @@ $$
If we then compute the expectation value
$$
-\mathbb{E}[\boldsymbol{X}\boldsymbol{X}^T] = \frac{1}{n}\boldsymbol{X}\boldsymbol{X}^T=\begin{bmatrix}
+\mathbb{E}[\boldsymbol{X}^T\boldsymbol{X}] = \frac{1}{n}\boldsymbol{X}^T\boldsymbol{X}=\begin{bmatrix}
x_{00}^2+x_{01}^2 & x_{00}x_{10}+x_{01}x_{11}\\
x_{10}x_{00}+x_{11}x_{01} & x_{10}^2+x_{11}^2\\
\end{bmatrix},
@@ -3355,7 +3355,7 @@ It is easy to generalize this to a matrix \( \boldsymbol{X}\in {\mathbb{R}}^{n\t
We have that the covariance matrix (the correlation matrix involves a simple rescaling) is given as
$$
-\boldsymbol{C}[\boldsymbol{x}] = \frac{1}{n}\boldsymbol{X}\boldsymbol{X}^T= \mathbb{E}[\boldsymbol{X}\boldsymbol{X}^T].
+\boldsymbol{C}[\boldsymbol{x}] = \frac{1}{n}\boldsymbol{X}^T\boldsymbol{X}= \mathbb{E}[\boldsymbol{X}^T\boldsymbol{X}].
$$
@@ -3363,20 +3363,20 @@ Let us now assume that we can perform a series of orthogonal transformations whe
These matrices are defined as \( \boldsymbol{S}\in {\mathbb{R}}^{p\times p} \) and obey the orthogonality requirements \( \boldsymbol{S}\boldsymbol{S}^T=\boldsymbol{S}^T\boldsymbol{S}=\boldsymbol{I} \). The matrix can be written out in terms of the column vectors \( \boldsymbol{s}_i \) as \( \boldsymbol{S}=[\boldsymbol{s}_0,\boldsymbol{s}_1,\dots,\boldsymbol{s}_{p-1}] \) and \( \boldsymbol{s}_i \in {\mathbb{R}}^{p} \).
-Assume also that there is a transformation \( \boldsymbol{S}\boldsymbol{C}[\boldsymbol{x}]\boldsymbol{S}^T=\boldsymbol{C}[\boldsymbol{y}] \) such that the new matrix \( \boldsymbol{C}[\boldsymbol{y}] \) is diagonal with elements \( [\lambda_0,\lambda_1,\lambda_2,\dots,\lambda_{p-1}] \). +Assume also that there is a transformation \( \boldsymbol{S}^T\boldsymbol{C}[\boldsymbol{x}]\boldsymbol{S}=\boldsymbol{C}[\boldsymbol{y}] \) such that the new matrix \( \boldsymbol{C}[\boldsymbol{y}] \) is diagonal with elements \( [\lambda_0,\lambda_1,\lambda_2,\dots,\lambda_{p-1}] \).
That is we have
$$
-\boldsymbol{C}[\boldsymbol{y}] = \mathbb{E}[\boldsymbol{S}\boldsymbol{X}\boldsymbol{X}^T\boldsymbol{S}^T]=\boldsymbol{S}\boldsymbol{C}[\boldsymbol{x}]\boldsymbol{S}^T,
+\boldsymbol{C}[\boldsymbol{y}] = \mathbb{E}[\boldsymbol{S}^T\boldsymbol{X}^T\boldsymbol{X}T\boldsymbol{S}]=\boldsymbol{S}^T\boldsymbol{C}[\boldsymbol{x}]\boldsymbol{S},
$$
-since the matrix \( \boldsymbol{S} \) is not a data dependent matrix. Multiplying with \( \boldsymbol{S}^T \) from the left we have
+since the matrix \( \boldsymbol{S} \) is not a data dependent matrix. Multiplying with \( \boldsymbol{S} \) from the left we have
$$
-\boldsymbol{S}^T\boldsymbol{C}[\boldsymbol{y}] = \boldsymbol{C}[\boldsymbol{x}]\boldsymbol{S}^T,
+\boldsymbol{S}\boldsymbol{C}[\boldsymbol{y}] = \boldsymbol{C}[\boldsymbol{x}]\boldsymbol{S},
$$
@@ -3384,7 +3384,7 @@ and since \( \boldsymbol{C}[\boldsymbol{y}] \) is diagonal we have for a given e
$$
-\boldsymbol{S}^T_i\lambda_i = \boldsymbol{C}[\boldsymbol{x}]\boldsymbol{S}^T_i.
+\boldsymbol{S}_i\lambda_i = \boldsymbol{C}[\boldsymbol{x}]\boldsymbol{S}_i.
$$
diff --git a/doc/pub/week43/html/week43-solarized.html b/doc/pub/week43/html/week43-solarized.html
index e471b2059..898b21c21 100644
--- a/doc/pub/week43/html/week43-solarized.html
+++ b/doc/pub/week43/html/week43-solarized.html
@@ -3215,7 +3215,7 @@ matrix without them.
We can rewrite the covariance matrix in a more compact form in terms of the design/feature matrix \( \boldsymbol{X} \) as $$ -\boldsymbol{C}[\boldsymbol{x}] = \frac{1}{n}\boldsymbol{X}\boldsymbol{X}^T= \mathbb{E}[\boldsymbol{X}\boldsymbol{X}^T]. +\boldsymbol{C}[\boldsymbol{x}] = \frac{1}{n}\boldsymbol{X}^T\boldsymbol{X}= \mathbb{E}[\boldsymbol{X}^T\boldsymbol{X}]. $$
@@ -3232,7 +3232,7 @@ $$
If we then compute the expectation value $$ -\mathbb{E}[\boldsymbol{X}\boldsymbol{X}^T] = \frac{1}{n}\boldsymbol{X}\boldsymbol{X}^T=\begin{bmatrix} +\mathbb{E}[\boldsymbol{X}^T\boldsymbol{X}] = \frac{1}{n}\boldsymbol{X}^T\boldsymbol{X}=\begin{bmatrix} x_{00}^2+x_{01}^2 & x_{00}x_{10}+x_{01}x_{11}\\ x_{10}x_{00}+x_{11}x_{01} & x_{10}^2+x_{11}^2\\ \end{bmatrix}, @@ -3258,30 +3258,30 @@ It is easy to generalize this to a matrix \( \boldsymbol{X}\in {\mathbb{R}}^{n\t
We have that the covariance matrix (the correlation matrix involves a simple rescaling) is given as $$ -\boldsymbol{C}[\boldsymbol{x}] = \frac{1}{n}\boldsymbol{X}\boldsymbol{X}^T= \mathbb{E}[\boldsymbol{X}\boldsymbol{X}^T]. +\boldsymbol{C}[\boldsymbol{x}] = \frac{1}{n}\boldsymbol{X}^T\boldsymbol{X}= \mathbb{E}[\boldsymbol{X}^T\boldsymbol{X}]. $$ Let us now assume that we can perform a series of orthogonal transformations where we employ some orthogonal matrices \( \boldsymbol{S} \). These matrices are defined as \( \boldsymbol{S}\in {\mathbb{R}}^{p\times p} \) and obey the orthogonality requirements \( \boldsymbol{S}\boldsymbol{S}^T=\boldsymbol{S}^T\boldsymbol{S}=\boldsymbol{I} \). The matrix can be written out in terms of the column vectors \( \boldsymbol{s}_i \) as \( \boldsymbol{S}=[\boldsymbol{s}_0,\boldsymbol{s}_1,\dots,\boldsymbol{s}_{p-1}] \) and \( \boldsymbol{s}_i \in {\mathbb{R}}^{p} \).
-Assume also that there is a transformation \( \boldsymbol{S}\boldsymbol{C}[\boldsymbol{x}]\boldsymbol{S}^T=\boldsymbol{C}[\boldsymbol{y}] \) such that the new matrix \( \boldsymbol{C}[\boldsymbol{y}] \) is diagonal with elements \( [\lambda_0,\lambda_1,\lambda_2,\dots,\lambda_{p-1}] \). +Assume also that there is a transformation \( \boldsymbol{S}^T\boldsymbol{C}[\boldsymbol{x}]\boldsymbol{S}=\boldsymbol{C}[\boldsymbol{y}] \) such that the new matrix \( \boldsymbol{C}[\boldsymbol{y}] \) is diagonal with elements \( [\lambda_0,\lambda_1,\lambda_2,\dots,\lambda_{p-1}] \).
That is we have $$ -\boldsymbol{C}[\boldsymbol{y}] = \mathbb{E}[\boldsymbol{S}\boldsymbol{X}\boldsymbol{X}^T\boldsymbol{S}^T]=\boldsymbol{S}\boldsymbol{C}[\boldsymbol{x}]\boldsymbol{S}^T, +\boldsymbol{C}[\boldsymbol{y}] = \mathbb{E}[\boldsymbol{S}^T\boldsymbol{X}^T\boldsymbol{X}T\boldsymbol{S}]=\boldsymbol{S}^T\boldsymbol{C}[\boldsymbol{x}]\boldsymbol{S}, $$ -since the matrix \( \boldsymbol{S} \) is not a data dependent matrix. Multiplying with \( \boldsymbol{S}^T \) from the left we have +since the matrix \( \boldsymbol{S} \) is not a data dependent matrix. Multiplying with \( \boldsymbol{S} \) from the left we have $$ -\boldsymbol{S}^T\boldsymbol{C}[\boldsymbol{y}] = \boldsymbol{C}[\boldsymbol{x}]\boldsymbol{S}^T, +\boldsymbol{S}\boldsymbol{C}[\boldsymbol{y}] = \boldsymbol{C}[\boldsymbol{x}]\boldsymbol{S}, $$ and since \( \boldsymbol{C}[\boldsymbol{y}] \) is diagonal we have for a given eigenvalue \( i \) of the covariance matrix that $$ -\boldsymbol{S}^T_i\lambda_i = \boldsymbol{C}[\boldsymbol{x}]\boldsymbol{S}^T_i. +\boldsymbol{S}_i\lambda_i = \boldsymbol{C}[\boldsymbol{x}]\boldsymbol{S}_i. $$
diff --git a/doc/pub/week43/html/week43.html b/doc/pub/week43/html/week43.html index 178171102..7091c4c39 100644 --- a/doc/pub/week43/html/week43.html +++ b/doc/pub/week43/html/week43.html @@ -3220,7 +3220,7 @@ matrix without them.
We can rewrite the covariance matrix in a more compact form in terms of the design/feature matrix \( \boldsymbol{X} \) as $$ -\boldsymbol{C}[\boldsymbol{x}] = \frac{1}{n}\boldsymbol{X}\boldsymbol{X}^T= \mathbb{E}[\boldsymbol{X}\boldsymbol{X}^T]. +\boldsymbol{C}[\boldsymbol{x}] = \frac{1}{n}\boldsymbol{X}^T\boldsymbol{X}= \mathbb{E}[\boldsymbol{X}^T\boldsymbol{X}]. $$
@@ -3237,7 +3237,7 @@ $$
If we then compute the expectation value $$ -\mathbb{E}[\boldsymbol{X}\boldsymbol{X}^T] = \frac{1}{n}\boldsymbol{X}\boldsymbol{X}^T=\begin{bmatrix} +\mathbb{E}[\boldsymbol{X}^T\boldsymbol{X}] = \frac{1}{n}\boldsymbol{X}^T\boldsymbol{X}=\begin{bmatrix} x_{00}^2+x_{01}^2 & x_{00}x_{10}+x_{01}x_{11}\\ x_{10}x_{00}+x_{11}x_{01} & x_{10}^2+x_{11}^2\\ \end{bmatrix}, @@ -3263,30 +3263,30 @@ It is easy to generalize this to a matrix \( \boldsymbol{X}\in {\mathbb{R}}^{n\t
We have that the covariance matrix (the correlation matrix involves a simple rescaling) is given as $$ -\boldsymbol{C}[\boldsymbol{x}] = \frac{1}{n}\boldsymbol{X}\boldsymbol{X}^T= \mathbb{E}[\boldsymbol{X}\boldsymbol{X}^T]. +\boldsymbol{C}[\boldsymbol{x}] = \frac{1}{n}\boldsymbol{X}^T\boldsymbol{X}= \mathbb{E}[\boldsymbol{X}^T\boldsymbol{X}]. $$ Let us now assume that we can perform a series of orthogonal transformations where we employ some orthogonal matrices \( \boldsymbol{S} \). These matrices are defined as \( \boldsymbol{S}\in {\mathbb{R}}^{p\times p} \) and obey the orthogonality requirements \( \boldsymbol{S}\boldsymbol{S}^T=\boldsymbol{S}^T\boldsymbol{S}=\boldsymbol{I} \). The matrix can be written out in terms of the column vectors \( \boldsymbol{s}_i \) as \( \boldsymbol{S}=[\boldsymbol{s}_0,\boldsymbol{s}_1,\dots,\boldsymbol{s}_{p-1}] \) and \( \boldsymbol{s}_i \in {\mathbb{R}}^{p} \).
-Assume also that there is a transformation \( \boldsymbol{S}\boldsymbol{C}[\boldsymbol{x}]\boldsymbol{S}^T=\boldsymbol{C}[\boldsymbol{y}] \) such that the new matrix \( \boldsymbol{C}[\boldsymbol{y}] \) is diagonal with elements \( [\lambda_0,\lambda_1,\lambda_2,\dots,\lambda_{p-1}] \). +Assume also that there is a transformation \( \boldsymbol{S}^T\boldsymbol{C}[\boldsymbol{x}]\boldsymbol{S}=\boldsymbol{C}[\boldsymbol{y}] \) such that the new matrix \( \boldsymbol{C}[\boldsymbol{y}] \) is diagonal with elements \( [\lambda_0,\lambda_1,\lambda_2,\dots,\lambda_{p-1}] \).
That is we have $$ -\boldsymbol{C}[\boldsymbol{y}] = \mathbb{E}[\boldsymbol{S}\boldsymbol{X}\boldsymbol{X}^T\boldsymbol{S}^T]=\boldsymbol{S}\boldsymbol{C}[\boldsymbol{x}]\boldsymbol{S}^T, +\boldsymbol{C}[\boldsymbol{y}] = \mathbb{E}[\boldsymbol{S}^T\boldsymbol{X}^T\boldsymbol{X}T\boldsymbol{S}]=\boldsymbol{S}^T\boldsymbol{C}[\boldsymbol{x}]\boldsymbol{S}, $$ -since the matrix \( \boldsymbol{S} \) is not a data dependent matrix. Multiplying with \( \boldsymbol{S}^T \) from the left we have +since the matrix \( \boldsymbol{S} \) is not a data dependent matrix. Multiplying with \( \boldsymbol{S} \) from the left we have $$ -\boldsymbol{S}^T\boldsymbol{C}[\boldsymbol{y}] = \boldsymbol{C}[\boldsymbol{x}]\boldsymbol{S}^T, +\boldsymbol{S}\boldsymbol{C}[\boldsymbol{y}] = \boldsymbol{C}[\boldsymbol{x}]\boldsymbol{S}, $$ and since \( \boldsymbol{C}[\boldsymbol{y}] \) is diagonal we have for a given eigenvalue \( i \) of the covariance matrix that $$ -\boldsymbol{S}^T_i\lambda_i = \boldsymbol{C}[\boldsymbol{x}]\boldsymbol{S}^T_i. +\boldsymbol{S}_i\lambda_i = \boldsymbol{C}[\boldsymbol{x}]\boldsymbol{S}_i. $$
diff --git a/doc/pub/week43/ipynb/ipynb-week43-src.tar.gz b/doc/pub/week43/ipynb/ipynb-week43-src.tar.gz index d23970d5d..7d7ff64ea 100644 Binary files a/doc/pub/week43/ipynb/ipynb-week43-src.tar.gz and b/doc/pub/week43/ipynb/ipynb-week43-src.tar.gz differ diff --git a/doc/pub/week43/ipynb/week43.ipynb b/doc/pub/week43/ipynb/week43.ipynb index ee29c9421..39efd2885 100644 --- a/doc/pub/week43/ipynb/week43.ipynb +++ b/doc/pub/week43/ipynb/week43.ipynb @@ -3538,7 +3538,7 @@ "metadata": {}, "source": [ "$$\n", - "\\boldsymbol{C}[\\boldsymbol{x}] = \\frac{1}{n}\\boldsymbol{X}\\boldsymbol{X}^T= \\mathbb{E}[\\boldsymbol{X}\\boldsymbol{X}^T].\n", + "\\boldsymbol{C}[\\boldsymbol{x}] = \\frac{1}{n}\\boldsymbol{X}^T\\boldsymbol{X}= \\mathbb{E}[\\boldsymbol{X}^T\\boldsymbol{X}].\n", "$$" ] }, @@ -3575,7 +3575,7 @@ "metadata": {}, "source": [ "$$\n", - "\\mathbb{E}[\\boldsymbol{X}\\boldsymbol{X}^T] = \\frac{1}{n}\\boldsymbol{X}\\boldsymbol{X}^T=\\begin{bmatrix}\n", + "\\mathbb{E}[\\boldsymbol{X}^T\\boldsymbol{X}] = \\frac{1}{n}\\boldsymbol{X}^T\\boldsymbol{X}=\\begin{bmatrix}\n", "x_{00}^2+x_{01}^2 & x_{00}x_{10}+x_{01}x_{11}\\\\\n", "x_{10}x_{00}+x_{11}x_{01} & x_{10}^2+x_{11}^2\\\\\n", "\\end{bmatrix},\n", @@ -3619,7 +3619,7 @@ "metadata": {}, "source": [ "$$\n", - "\\boldsymbol{C}[\\boldsymbol{x}] = \\frac{1}{n}\\boldsymbol{X}\\boldsymbol{X}^T= \\mathbb{E}[\\boldsymbol{X}\\boldsymbol{X}^T].\n", + "\\boldsymbol{C}[\\boldsymbol{x}] = \\frac{1}{n}\\boldsymbol{X}^T\\boldsymbol{X}= \\mathbb{E}[\\boldsymbol{X}^T\\boldsymbol{X}].\n", "$$" ] }, @@ -3630,7 +3630,7 @@ "Let us now assume that we can perform a series of orthogonal transformations where we employ some orthogonal matrices $\\boldsymbol{S}$.\n", "These matrices are defined as $\\boldsymbol{S}\\in {\\mathbb{R}}^{p\\times p}$ and obey the orthogonality requirements $\\boldsymbol{S}\\boldsymbol{S}^T=\\boldsymbol{S}^T\\boldsymbol{S}=\\boldsymbol{I}$. The matrix can be written out in terms of the column vectors $\\boldsymbol{s}_i$ as $\\boldsymbol{S}=[\\boldsymbol{s}_0,\\boldsymbol{s}_1,\\dots,\\boldsymbol{s}_{p-1}]$ and $\\boldsymbol{s}_i \\in {\\mathbb{R}}^{p}$.\n", "\n", - "Assume also that there is a transformation $\\boldsymbol{S}\\boldsymbol{C}[\\boldsymbol{x}]\\boldsymbol{S}^T=\\boldsymbol{C}[\\boldsymbol{y}]$ such that the new matrix $\\boldsymbol{C}[\\boldsymbol{y}]$ is diagonal with elements $[\\lambda_0,\\lambda_1,\\lambda_2,\\dots,\\lambda_{p-1}]$. \n", + "Assume also that there is a transformation $\\boldsymbol{S}^T\\boldsymbol{C}[\\boldsymbol{x}]\\boldsymbol{S}=\\boldsymbol{C}[\\boldsymbol{y}]$ such that the new matrix $\\boldsymbol{C}[\\boldsymbol{y}]$ is diagonal with elements $[\\lambda_0,\\lambda_1,\\lambda_2,\\dots,\\lambda_{p-1}]$. \n", "\n", "That is we have" ] @@ -3640,7 +3640,7 @@ "metadata": {}, "source": [ "$$\n", - "\\boldsymbol{C}[\\boldsymbol{y}] = \\mathbb{E}[\\boldsymbol{S}\\boldsymbol{X}\\boldsymbol{X}^T\\boldsymbol{S}^T]=\\boldsymbol{S}\\boldsymbol{C}[\\boldsymbol{x}]\\boldsymbol{S}^T,\n", + "\\boldsymbol{C}[\\boldsymbol{y}] = \\mathbb{E}[\\boldsymbol{S}^T\\boldsymbol{X}^T\\boldsymbol{X}T\\boldsymbol{S}]=\\boldsymbol{S}^T\\boldsymbol{C}[\\boldsymbol{x}]\\boldsymbol{S},\n", "$$" ] }, @@ -3648,7 +3648,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "since the matrix $\\boldsymbol{S}$ is not a data dependent matrix. Multiplying with $\\boldsymbol{S}^T$ from the left we have" + "since the matrix $\\boldsymbol{S}$ is not a data dependent matrix. Multiplying with $\\boldsymbol{S}$ from the left we have" ] }, { @@ -3656,7 +3656,7 @@ "metadata": {}, "source": [ "$$\n", - "\\boldsymbol{S}^T\\boldsymbol{C}[\\boldsymbol{y}] = \\boldsymbol{C}[\\boldsymbol{x}]\\boldsymbol{S}^T,\n", + "\\boldsymbol{S}\\boldsymbol{C}[\\boldsymbol{y}] = \\boldsymbol{C}[\\boldsymbol{x}]\\boldsymbol{S},\n", "$$" ] }, @@ -3672,7 +3672,7 @@ "metadata": {}, "source": [ "$$\n", - "\\boldsymbol{S}^T_i\\lambda_i = \\boldsymbol{C}[\\boldsymbol{x}]\\boldsymbol{S}^T_i.\n", + "\\boldsymbol{S}_i\\lambda_i = \\boldsymbol{C}[\\boldsymbol{x}]\\boldsymbol{S}_i.\n", "$$" ] }, diff --git a/doc/src/week43/week43.do.txt b/doc/src/week43/week43.do.txt index 346881ab7..8021bab9b 100644 --- a/doc/src/week43/week43.do.txt +++ b/doc/src/week43/week43.do.txt @@ -2720,7 +2720,7 @@ matrix without them. We can rewrite the covariance matrix in a more compact form in terms of the design/feature matrix $\bm{X}$ as !bt \[ -\bm{C}[\bm{x}] = \frac{1}{n}\bm{X}\bm{X}^T= \mathbb{E}[\bm{X}\bm{X}^T]. +\bm{C}[\bm{x}] = \frac{1}{n}\bm{X}^T\bm{X}= \mathbb{E}[\bm{X}^T\bm{X}]. \] !et @@ -2739,7 +2739,7 @@ x_{10} & x_{11}\\ If we then compute the expectation value !bt \[ -\mathbb{E}[\bm{X}\bm{X}^T] = \frac{1}{n}\bm{X}\bm{X}^T=\begin{bmatrix} +\mathbb{E}[\bm{X}^T\bm{X}] = \frac{1}{n}\bm{X}^T\bm{X}=\begin{bmatrix} x_{00}^2+x_{01}^2 & x_{00}x_{10}+x_{01}x_{11}\\ x_{10}x_{00}+x_{11}x_{01} & x_{10}^2+x_{11}^2\\ \end{bmatrix}, @@ -2764,31 +2764,31 @@ It is easy to generalize this to a matrix $\bm{X}\in {\mathbb{R}}^{n\times p}$. We have that the covariance matrix (the correlation matrix involves a simple rescaling) is given as !bt \[ -\bm{C}[\bm{x}] = \frac{1}{n}\bm{X}\bm{X}^T= \mathbb{E}[\bm{X}\bm{X}^T]. +\bm{C}[\bm{x}] = \frac{1}{n}\bm{X}^T\bm{X}= \mathbb{E}[\bm{X}^T\bm{X}]. \] !et Let us now assume that we can perform a series of orthogonal transformations where we employ some orthogonal matrices $\bm{S}$. These matrices are defined as $\bm{S}\in {\mathbb{R}}^{p\times p}$ and obey the orthogonality requirements $\bm{S}\bm{S}^T=\bm{S}^T\bm{S}=\bm{I}$. The matrix can be written out in terms of the column vectors $\bm{s}_i$ as $\bm{S}=[\bm{s}_0,\bm{s}_1,\dots,\bm{s}_{p-1}]$ and $\bm{s}_i \in {\mathbb{R}}^{p}$. -Assume also that there is a transformation $\bm{S}\bm{C}[\bm{x}]\bm{S}^T=\bm{C}[\bm{y}]$ such that the new matrix $\bm{C}[\bm{y}]$ is diagonal with elements $[\lambda_0,\lambda_1,\lambda_2,\dots,\lambda_{p-1}]$. +Assume also that there is a transformation $\bm{S}^T\bm{C}[\bm{x}]\bm{S}=\bm{C}[\bm{y}]$ such that the new matrix $\bm{C}[\bm{y}]$ is diagonal with elements $[\lambda_0,\lambda_1,\lambda_2,\dots,\lambda_{p-1}]$. That is we have !bt \[ -\bm{C}[\bm{y}] = \mathbb{E}[\bm{S}\bm{X}\bm{X}^T\bm{S}^T]=\bm{S}\bm{C}[\bm{x}]\bm{S}^T, +\bm{C}[\bm{y}] = \mathbb{E}[\bm{S}^T\bm{X}^T\bm{X}T\bm{S}]=\bm{S}^T\bm{C}[\bm{x}]\bm{S}, \] !et -since the matrix $\bm{S}$ is not a data dependent matrix. Multiplying with $\bm{S}^T$ from the left we have +since the matrix $\bm{S}$ is not a data dependent matrix. Multiplying with $\bm{S}$ from the left we have !bt \[ -\bm{S}^T\bm{C}[\bm{y}] = \bm{C}[\bm{x}]\bm{S}^T, +\bm{S}\bm{C}[\bm{y}] = \bm{C}[\bm{x}]\bm{S}, \] !et and since $\bm{C}[\bm{y}]$ is diagonal we have for a given eigenvalue $i$ of the covariance matrix that !bt \[ -\bm{S}^T_i\lambda_i = \bm{C}[\bm{x}]\bm{S}^T_i. +\bm{S}_i\lambda_i = \bm{C}[\bm{x}]\bm{S}_i. \] !et