lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

corrected labels in dimred

Browse Source
This commit is contained in:
mhjensen
2019-12-26 19:20:42 +01:00
parent 54fc7da531
commit 1c0911a3d3
8 changed files with 84 additions and 101 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/pub/DimRed/html/._DimRed-bs018.html
+14 -17
View File
@@ -210,8 +210,8 @@ $$
\end{bmatrix}
$$
<p>
We will generate \( N = 1000 \) points \( X = \{ x_1, \ldots, x_N \} \) from
Note that the mean refers to each column of data.
We will generate \( n = 1000 \) points \( X = \{ x_1, \ldots, x_N \} \) from
this distribution, and store them in the \( 1000 \times 2 \) matrix \( \boldsymbol{X} \).
<p>
@@ -220,13 +220,13 @@ The following Python code aids in setting up the data
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span>N <span style="color: #666666">=</span> <span style="color: #666666">1000</span>
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span>n <span style="color: #666666">=</span> <span style="color: #666666">1000</span>
mean <span style="color: #666666">=</span> (<span style="color: #666666">-1</span>, <span style="color: #666666">2</span>)
cov <span style="color: #666666">=</span> [[<span style="color: #666666">4</span>, <span style="color: #666666">2</span>], [<span style="color: #666666">2</span>, <span style="color: #666666">2</span>]]
X <span style="color: #666666">=</span> np<span style="color: #666666">.</span>random<span style="color: #666666">.</span>multivariate_normal(mean, cov, N)
X <span style="color: #666666">=</span> np<span style="color: #666666">.</span>random<span style="color: #666666">.</span>multivariate_normal(mean, cov, n)
</pre></div>
<p>
Make a small Python code which plots the data.
Make thereafter a small Python code which plots the data. Note that the function <b>multivariate</b> returns also the covariance discussed above and that it is defined by dividing by \( n-1 \) instead of \( n \).
<p>
Now we are going to implement the PCA algorithm. We will break it down into sub-steps and across multiple cells.
@@ -234,17 +234,17 @@ Now we are going to implement the PCA algorithm. We will break it down into sub-
<h3 id="___sec18" class="anchor">Compute the sample mean and center the data </h3>
<p>
The first step of PCA is to compute the sample mean of the data and use it to center the data. Recall the sample mean is
The first step of PCA is to compute the sample mean of the data and use it to center the data. Recall that the sample mean is
$$
\mu_N = \frac{1}{N} \sum_{i=1}^N x_i
\mu_n = \frac{1}{n} \sum_{i=1}^n x_i
$$
and the mean-centered data \( \bar{X} = \{ \bar{x}_1, \ldots, \bar{x}_N \} \) takes the form
and the mean-centered data \( \bar{X} = \{ \bar{x}_1, \ldots, \bar{x}_n \} \) takes the form
$$
\bar{x}_i = x_i - \mu_N
\bar{x}_i = x_i - \mu_n.
$$
When you are done with these steps, print out \( \mu_N \) to verify it is
When you are done with these steps, print out \( \mu_n \) to verify it is
close to \( \mu \) and plot your mean centered data to verify it is
centered at the origin! Compare your code with the functionality from <b>Scikit-Learn</b> discussed above.
@@ -254,7 +254,7 @@ centered at the origin! Compare your code with the functionality from <b>Scikit-
Now we are going to use the mean centered data to compute the sample covariance of the data. Recall it is given by:
$$
\begin{equation*}
\Sigma_N = \frac{1}{N-1} \sum_{i=1}^N \bar{x}_i^T \bar{x}_i = \frac{1}{N-1} \sum_{i=1}^N (x_i - \mu_N)^T (x_i - \mu_N)
\Sigma_n = \frac{1}{n-1} \sum_{i=1}^n \bar{x}_i^T \bar{x}_i = \frac{1}{n-1} \sum_{i=1}^n (x_i - \mu_n)^T (x_i - \mu_n)
\end{equation*}
$$
@@ -265,9 +265,9 @@ Compare the computed covariance with the answer given above.
<p>
Now we are ready to solve for the principal components! To do so we
diagonalize the sample covariance matrix \( \Sigma_N \). We can use the
diagonalize the sample covariance matrix \( \Sigma_n \). We can use the
function <b>np.linalg.eig</b> to do so. It will return the eigenvalues and
eigenvectors of \( \Sigma_N \). Once you have these, carry out the
eigenvectors of \( \Sigma_n \). Once you have these, carry out the
following tasks:
<ul>
@@ -279,7 +279,7 @@ following tasks:
$$
\begin{equation*}
x_i \approx \tilde{x}_i := \mu_N + \langle x_i, v_0 \rangle v_0
x_i \approx \tilde{x}_i := \mu_n + \langle x_i, v_0 \rangle v_0
\end{equation*}
$$
@@ -293,9 +293,6 @@ Have the input be the data and have the output be the principal components and t
<p>
Finally, try out your own PCA function with other data sets.
<p>
After this we ask ourselves how do we prove the link between the maximum variance and the feature reduction.
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
doc/pub/DimRed/html/DimRed-reveal.html
+14 -17
View File
@@ -999,8 +999,8 @@ $$
$$
<p>&nbsp;<br>
<p>
We will generate \( N = 1000 \) points \( X = \{ x_1, \ldots, x_N \} \) from
Note that the mean refers to each column of data.
We will generate \( n = 1000 \) points \( X = \{ x_1, \ldots, x_N \} \) from
this distribution, and store them in the \( 1000 \times 2 \) matrix \( \boldsymbol{X} \).
<p>
@@ -1009,13 +1009,13 @@ The following Python code aids in setting up the data
<p>
<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
<div class="highlight" style="background: #eeeedd"><pre style="font-size: 80%; line-height: 125%"><span></span>N = <span style="color: #B452CD">1000</span>
<div class="highlight" style="background: #eeeedd"><pre style="font-size: 80%; line-height: 125%"><span></span>n = <span style="color: #B452CD">1000</span>
mean = (-<span style="color: #B452CD">1</span>, <span style="color: #B452CD">2</span>)
cov = [[<span style="color: #B452CD">4</span>, <span style="color: #B452CD">2</span>], [<span style="color: #B452CD">2</span>, <span style="color: #B452CD">2</span>]]
X = np.random.multivariate_normal(mean, cov, N)
X = np.random.multivariate_normal(mean, cov, n)
</pre></div>
<p>
Make a small Python code which plots the data.
Make thereafter a small Python code which plots the data. Note that the function <b>multivariate</b> returns also the covariance discussed above and that it is defined by dividing by \( n-1 \) instead of \( n \).
<p>
Now we are going to implement the PCA algorithm. We will break it down into sub-steps and across multiple cells.
@@ -1023,21 +1023,21 @@ Now we are going to implement the PCA algorithm. We will break it down into sub-
<h3 id="___sec18">Compute the sample mean and center the data </h3>
<p>
The first step of PCA is to compute the sample mean of the data and use it to center the data. Recall the sample mean is
The first step of PCA is to compute the sample mean of the data and use it to center the data. Recall that the sample mean is
<p>&nbsp;<br>
$$
\mu_N = \frac{1}{N} \sum_{i=1}^N x_i
\mu_n = \frac{1}{n} \sum_{i=1}^n x_i
$$
<p>&nbsp;<br>
and the mean-centered data \( \bar{X} = \{ \bar{x}_1, \ldots, \bar{x}_N \} \) takes the form
and the mean-centered data \( \bar{X} = \{ \bar{x}_1, \ldots, \bar{x}_n \} \) takes the form
<p>&nbsp;<br>
$$
\bar{x}_i = x_i - \mu_N
\bar{x}_i = x_i - \mu_n.
$$
<p>&nbsp;<br>
When you are done with these steps, print out \( \mu_N \) to verify it is
When you are done with these steps, print out \( \mu_n \) to verify it is
close to \( \mu \) and plot your mean centered data to verify it is
centered at the origin! Compare your code with the functionality from <b>Scikit-Learn</b> discussed above.
@@ -1048,7 +1048,7 @@ Now we are going to use the mean centered data to compute the sample covariance
<p>&nbsp;<br>
$$
\begin{equation*}
\Sigma_N = \frac{1}{N-1} \sum_{i=1}^N \bar{x}_i^T \bar{x}_i = \frac{1}{N-1} \sum_{i=1}^N (x_i - \mu_N)^T (x_i - \mu_N)
\Sigma_n = \frac{1}{n-1} \sum_{i=1}^n \bar{x}_i^T \bar{x}_i = \frac{1}{n-1} \sum_{i=1}^n (x_i - \mu_n)^T (x_i - \mu_n)
\end{equation*}
$$
<p>&nbsp;<br>
@@ -1060,9 +1060,9 @@ Compare the computed covariance with the answer given above.
<p>
Now we are ready to solve for the principal components! To do so we
diagonalize the sample covariance matrix \( \Sigma_N \). We can use the
diagonalize the sample covariance matrix \( \Sigma_n \). We can use the
function <b>np.linalg.eig</b> to do so. It will return the eigenvalues and
eigenvectors of \( \Sigma_N \). Once you have these, carry out the
eigenvectors of \( \Sigma_n \). Once you have these, carry out the
following tasks:
<ul>
@@ -1074,7 +1074,7 @@ following tasks:
<p>&nbsp;<br>
$$
\begin{equation*}
x_i \approx \tilde{x}_i := \mu_N + \langle x_i, v_0 \rangle v_0
x_i \approx \tilde{x}_i := \mu_n + \langle x_i, v_0 \rangle v_0
\end{equation*}
$$
<p>&nbsp;<br>
@@ -1088,9 +1088,6 @@ Have the input be the data and have the output be the principal components and t
<p>
Finally, try out your own PCA function with other data sets.
<p>
After this we ask ourselves how do we prove the link between the maximum variance and the feature reduction.
</section>
doc/pub/DimRed/html/DimRed-solarized.html
+14 -17
View File
@@ -974,8 +974,8 @@ $$
\end{bmatrix}
$$
<p>
We will generate \( N = 1000 \) points \( X = \{ x_1, \ldots, x_N \} \) from
Note that the mean refers to each column of data.
We will generate \( n = 1000 \) points \( X = \{ x_1, \ldots, x_N \} \) from
this distribution, and store them in the \( 1000 \times 2 \) matrix \( \boldsymbol{X} \).
<p>
@@ -984,13 +984,13 @@ The following Python code aids in setting up the data
<p>
<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
<div class="highlight" style="background: #eeeedd"><pre style="line-height: 125%"><span></span>N = <span style="color: #B452CD">1000</span>
<div class="highlight" style="background: #eeeedd"><pre style="line-height: 125%"><span></span>n = <span style="color: #B452CD">1000</span>
mean = (-<span style="color: #B452CD">1</span>, <span style="color: #B452CD">2</span>)
cov = [[<span style="color: #B452CD">4</span>, <span style="color: #B452CD">2</span>], [<span style="color: #B452CD">2</span>, <span style="color: #B452CD">2</span>]]
X = np.random.multivariate_normal(mean, cov, N)
X = np.random.multivariate_normal(mean, cov, n)
</pre></div>
<p>
Make a small Python code which plots the data.
Make thereafter a small Python code which plots the data. Note that the function <b>multivariate</b> returns also the covariance discussed above and that it is defined by dividing by \( n-1 \) instead of \( n \).
<p>
Now we are going to implement the PCA algorithm. We will break it down into sub-steps and across multiple cells.
@@ -998,17 +998,17 @@ Now we are going to implement the PCA algorithm. We will break it down into sub-
<h3 id="___sec18">Compute the sample mean and center the data </h3>
<p>
The first step of PCA is to compute the sample mean of the data and use it to center the data. Recall the sample mean is
The first step of PCA is to compute the sample mean of the data and use it to center the data. Recall that the sample mean is
$$
\mu_N = \frac{1}{N} \sum_{i=1}^N x_i
\mu_n = \frac{1}{n} \sum_{i=1}^n x_i
$$
and the mean-centered data \( \bar{X} = \{ \bar{x}_1, \ldots, \bar{x}_N \} \) takes the form
and the mean-centered data \( \bar{X} = \{ \bar{x}_1, \ldots, \bar{x}_n \} \) takes the form
$$
\bar{x}_i = x_i - \mu_N
\bar{x}_i = x_i - \mu_n.
$$
When you are done with these steps, print out \( \mu_N \) to verify it is
When you are done with these steps, print out \( \mu_n \) to verify it is
close to \( \mu \) and plot your mean centered data to verify it is
centered at the origin! Compare your code with the functionality from <b>Scikit-Learn</b> discussed above.
@@ -1018,7 +1018,7 @@ centered at the origin! Compare your code with the functionality from <b>Scikit-
Now we are going to use the mean centered data to compute the sample covariance of the data. Recall it is given by:
$$
\begin{equation*}
\Sigma_N = \frac{1}{N-1} \sum_{i=1}^N \bar{x}_i^T \bar{x}_i = \frac{1}{N-1} \sum_{i=1}^N (x_i - \mu_N)^T (x_i - \mu_N)
\Sigma_n = \frac{1}{n-1} \sum_{i=1}^n \bar{x}_i^T \bar{x}_i = \frac{1}{n-1} \sum_{i=1}^n (x_i - \mu_n)^T (x_i - \mu_n)
\end{equation*}
$$
@@ -1029,9 +1029,9 @@ Compare the computed covariance with the answer given above.
<p>
Now we are ready to solve for the principal components! To do so we
diagonalize the sample covariance matrix \( \Sigma_N \). We can use the
diagonalize the sample covariance matrix \( \Sigma_n \). We can use the
function <b>np.linalg.eig</b> to do so. It will return the eigenvalues and
eigenvectors of \( \Sigma_N \). Once you have these, carry out the
eigenvectors of \( \Sigma_n \). Once you have these, carry out the
following tasks:
<ul>
@@ -1043,7 +1043,7 @@ following tasks:
$$
\begin{equation*}
x_i \approx \tilde{x}_i := \mu_N + \langle x_i, v_0 \rangle v_0
x_i \approx \tilde{x}_i := \mu_n + \langle x_i, v_0 \rangle v_0
\end{equation*}
$$
@@ -1057,9 +1057,6 @@ Have the input be the data and have the output be the principal components and t
<p>
Finally, try out your own PCA function with other data sets.
<p>
After this we ask ourselves how do we prove the link between the maximum variance and the feature reduction.
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
doc/pub/DimRed/html/DimRed.html
+14 -17
View File
@@ -979,8 +979,8 @@ $$
\end{bmatrix}
$$
<p>
We will generate \( N = 1000 \) points \( X = \{ x_1, \ldots, x_N \} \) from
Note that the mean refers to each column of data.
We will generate \( n = 1000 \) points \( X = \{ x_1, \ldots, x_N \} \) from
this distribution, and store them in the \( 1000 \times 2 \) matrix \( \boldsymbol{X} \).
<p>
@@ -989,13 +989,13 @@ The following Python code aids in setting up the data
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span>N <span style="color: #666666">=</span> <span style="color: #666666">1000</span>
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span>n <span style="color: #666666">=</span> <span style="color: #666666">1000</span>
mean <span style="color: #666666">=</span> (<span style="color: #666666">-1</span>, <span style="color: #666666">2</span>)
cov <span style="color: #666666">=</span> [[<span style="color: #666666">4</span>, <span style="color: #666666">2</span>], [<span style="color: #666666">2</span>, <span style="color: #666666">2</span>]]
X <span style="color: #666666">=</span> np<span style="color: #666666">.</span>random<span style="color: #666666">.</span>multivariate_normal(mean, cov, N)
X <span style="color: #666666">=</span> np<span style="color: #666666">.</span>random<span style="color: #666666">.</span>multivariate_normal(mean, cov, n)
</pre></div>
<p>
Make a small Python code which plots the data.
Make thereafter a small Python code which plots the data. Note that the function <b>multivariate</b> returns also the covariance discussed above and that it is defined by dividing by \( n-1 \) instead of \( n \).
<p>
Now we are going to implement the PCA algorithm. We will break it down into sub-steps and across multiple cells.
@@ -1003,17 +1003,17 @@ Now we are going to implement the PCA algorithm. We will break it down into sub-
<h3 id="___sec18">Compute the sample mean and center the data </h3>
<p>
The first step of PCA is to compute the sample mean of the data and use it to center the data. Recall the sample mean is
The first step of PCA is to compute the sample mean of the data and use it to center the data. Recall that the sample mean is
$$
\mu_N = \frac{1}{N} \sum_{i=1}^N x_i
\mu_n = \frac{1}{n} \sum_{i=1}^n x_i
$$
and the mean-centered data \( \bar{X} = \{ \bar{x}_1, \ldots, \bar{x}_N \} \) takes the form
and the mean-centered data \( \bar{X} = \{ \bar{x}_1, \ldots, \bar{x}_n \} \) takes the form
$$
\bar{x}_i = x_i - \mu_N
\bar{x}_i = x_i - \mu_n.
$$
When you are done with these steps, print out \( \mu_N \) to verify it is
When you are done with these steps, print out \( \mu_n \) to verify it is
close to \( \mu \) and plot your mean centered data to verify it is
centered at the origin! Compare your code with the functionality from <b>Scikit-Learn</b> discussed above.
@@ -1023,7 +1023,7 @@ centered at the origin! Compare your code with the functionality from <b>Scikit-
Now we are going to use the mean centered data to compute the sample covariance of the data. Recall it is given by:
$$
\begin{equation*}
\Sigma_N = \frac{1}{N-1} \sum_{i=1}^N \bar{x}_i^T \bar{x}_i = \frac{1}{N-1} \sum_{i=1}^N (x_i - \mu_N)^T (x_i - \mu_N)
\Sigma_n = \frac{1}{n-1} \sum_{i=1}^n \bar{x}_i^T \bar{x}_i = \frac{1}{n-1} \sum_{i=1}^n (x_i - \mu_n)^T (x_i - \mu_n)
\end{equation*}
$$
@@ -1034,9 +1034,9 @@ Compare the computed covariance with the answer given above.
<p>
Now we are ready to solve for the principal components! To do so we
diagonalize the sample covariance matrix \( \Sigma_N \). We can use the
diagonalize the sample covariance matrix \( \Sigma_n \). We can use the
function <b>np.linalg.eig</b> to do so. It will return the eigenvalues and
eigenvectors of \( \Sigma_N \). Once you have these, carry out the
eigenvectors of \( \Sigma_n \). Once you have these, carry out the
following tasks:
<ul>
@@ -1048,7 +1048,7 @@ following tasks:
$$
\begin{equation*}
x_i \approx \tilde{x}_i := \mu_N + \langle x_i, v_0 \rangle v_0
x_i \approx \tilde{x}_i := \mu_n + \langle x_i, v_0 \rangle v_0
\end{equation*}
$$
@@ -1062,9 +1062,6 @@ Have the input be the data and have the output be the principal components and t
<p>
Finally, try out your own PCA function with other data sets.
<p>
After this we ask ourselves how do we prove the link between the maximum variance and the feature reduction.
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
doc/pub/DimRed/ipynb/DimRed.ipynb
+14 -16
View File
@@ -1070,7 +1070,8 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"We will generate $N = 1000$ points $X = \\{ x_1, \\ldots, x_N \\}$ from\n",
"Note that the mean refers to each column of data. \n",
"We will generate $n = 1000$ points $X = \\{ x_1, \\ldots, x_N \\}$ from\n",
"this distribution, and store them in the $1000 \\times 2$ matrix $\\boldsymbol{X}$.\n",
"\n",
"The following Python code aids in setting up the data"
@@ -1084,23 +1085,23 @@
},
"outputs": [],
"source": [
"N = 1000\n",
"n = 1000\n",
"mean = (-1, 2)\n",
"cov = [[4, 2], [2, 2]]\n",
"X = np.random.multivariate_normal(mean, cov, N)"
"X = np.random.multivariate_normal(mean, cov, n)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Make a small Python code which plots the data.\n",
"Make thereafter a small Python code which plots the data. Note that the function **multivariate** returns also the covariance discussed above and that it is defined by dividing by $n-1$ instead of $n$.\n",
"\n",
"Now we are going to implement the PCA algorithm. We will break it down into sub-steps and across multiple cells.\n",
"\n",
"### Compute the sample mean and center the data\n",
"\n",
"The first step of PCA is to compute the sample mean of the data and use it to center the data. Recall the sample mean is"
"The first step of PCA is to compute the sample mean of the data and use it to center the data. Recall that the sample mean is"
]
},
{
@@ -1108,7 +1109,7 @@
"metadata": {},
"source": [
"$$\n",
"\\mu_N = \\frac{1}{N} \\sum_{i=1}^N x_i\n",
"\\mu_n = \\frac{1}{n} \\sum_{i=1}^n x_i\n",
"$$"
]
},
@@ -1116,7 +1117,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"and the mean-centered data $\\bar{X} = \\{ \\bar{x}_1, \\ldots, \\bar{x}_N \\}$ takes the form"
"and the mean-centered data $\\bar{X} = \\{ \\bar{x}_1, \\ldots, \\bar{x}_n \\}$ takes the form"
]
},
{
@@ -1124,7 +1125,7 @@
"metadata": {},
"source": [
"$$\n",
"\\bar{x}_i = x_i - \\mu_N\n",
"\\bar{x}_i = x_i - \\mu_n.\n",
"$$"
]
},
@@ -1132,7 +1133,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"When you are done with these steps, print out $\\mu_N$ to verify it is\n",
"When you are done with these steps, print out $\\mu_n$ to verify it is\n",
"close to $\\mu$ and plot your mean centered data to verify it is\n",
"centered at the origin! Compare your code with the functionality from **Scikit-Learn** discussed above.\n",
"\n",
@@ -1147,7 +1148,7 @@
"metadata": {},
"source": [
"$$\n",
"\\Sigma_N = \\frac{1}{N-1} \\sum_{i=1}^N \\bar{x}_i^T \\bar{x}_i = \\frac{1}{N-1} \\sum_{i=1}^N (x_i - \\mu_N)^T (x_i - \\mu_N)\n",
"\\Sigma_n = \\frac{1}{n-1} \\sum_{i=1}^n \\bar{x}_i^T \\bar{x}_i = \\frac{1}{n-1} \\sum_{i=1}^n (x_i - \\mu_n)^T (x_i - \\mu_n)\n",
"$$"
]
},
@@ -1162,9 +1163,9 @@
"### Diagonalize the sample covariance matrix to obtain the principal components\n",
"\n",
"Now we are ready to solve for the principal components! To do so we\n",
"diagonalize the sample covariance matrix $\\Sigma_N$. We can use the\n",
"diagonalize the sample covariance matrix $\\Sigma_n$. We can use the\n",
"function **np.linalg.eig** to do so. It will return the eigenvalues and\n",
"eigenvectors of $\\Sigma_N$. Once you have these, carry out the\n",
"eigenvectors of $\\Sigma_n$. Once you have these, carry out the\n",
"following tasks:\n",
"\n",
"* Compute the percentage of the total variance captured by the first principal component\n",
@@ -1181,7 +1182,7 @@
"metadata": {},
"source": [
"$$\n",
"x_i \\approx \\tilde{x}_i := \\mu_N + \\langle x_i, v_0 \\rangle v_0\n",
"x_i \\approx \\tilde{x}_i := \\mu_n + \\langle x_i, v_0 \\rangle v_0\n",
"$$"
]
},
@@ -1198,9 +1199,6 @@
"Finally, try out your own PCA function with other data sets.\n",
"\n",
"\n",
"\n",
"After this we ask ourselves how do we prove the link between the maximum variance and the feature reduction.\n",
"\n",
"## Classical PCA Theorem\n",
"\n",
"We assume now that we have a design matrix $\\boldsymbol{X}$ which has been centered as discussed above. For the sake of simplicity we skip the overline symbol. The matrix is defined in terms of the various column vectors $[\\boldsymbol{x}_0,\\boldsymbol{x}_1,\\dots, \\boldsymbol{x}_{p-1}]$\n",
doc/pub/DimRed/ipynb/ipynb-DimRed-src.tar.gz
BIN
View File
Binary file not shown.
doc/pub/DimRed/pdf/DimRed-minted.pdf
BIN
View File
Binary file not shown.
doc/src/DimRed/DimRed.do.txt
+14 -17
View File
@@ -752,38 +752,38 @@ drawn from a multivariate normal distribution with the following mean and covari
\end{bmatrix}
\]
!et
We will generate $N = 1000$ points $X = \{ x_1, \ldots, x_N \}$ from
Note that the mean refers to each column of data.
We will generate $n = 1000$ points $X = \{ x_1, \ldots, x_N \}$ from
this distribution, and store them in the $1000 \times 2$ matrix $\bm{X}$.
The following Python code aids in setting up the data
!bc pycod
N = 1000
n = 1000
mean = (-1, 2)
cov = [[4, 2], [2, 2]]
X = np.random.multivariate_normal(mean, cov, N)
X = np.random.multivariate_normal(mean, cov, n)
!ec
Make a small Python code which plots the data.
Make thereafter a small Python code which plots the data. Note that the function _multivariate_ returns also the covariance discussed above and that it is defined by dividing by $n-1$ instead of $n$.
Now we are going to implement the PCA algorithm. We will break it down into sub-steps and across multiple cells.
=== Compute the sample mean and center the data ===
The first step of PCA is to compute the sample mean of the data and use it to center the data. Recall the sample mean is
The first step of PCA is to compute the sample mean of the data and use it to center the data. Recall that the sample mean is
!bt
\[
\mu_N = \frac{1}{N} \sum_{i=1}^N x_i
\mu_n = \frac{1}{n} \sum_{i=1}^n x_i
\]
!et
and the mean-centered data $\bar{X} = \{ \bar{x}_1, \ldots, \bar{x}_N \}$ takes the form
and the mean-centered data $\bar{X} = \{ \bar{x}_1, \ldots, \bar{x}_n \}$ takes the form
!bt
\[
\bar{x}_i = x_i - \mu_N
\bar{x}_i = x_i - \mu_n.
\]
!et
When you are done with these steps, print out $\mu_N$ to verify it is
When you are done with these steps, print out $\mu_n$ to verify it is
close to $\mu$ and plot your mean centered data to verify it is
centered at the origin! Compare your code with the functionality from _Scikit-Learn_ discussed above.
@@ -793,7 +793,7 @@ centered at the origin! Compare your code with the functionality from _Scikit-Le
Now we are going to use the mean centered data to compute the sample covariance of the data. Recall it is given by:
!bt
\begin{equation*}
\Sigma_N = \frac{1}{N-1} \sum_{i=1}^N \bar{x}_i^T \bar{x}_i = \frac{1}{N-1} \sum_{i=1}^N (x_i - \mu_N)^T (x_i - \mu_N)
\Sigma_n = \frac{1}{n-1} \sum_{i=1}^n \bar{x}_i^T \bar{x}_i = \frac{1}{n-1} \sum_{i=1}^n (x_i - \mu_n)^T (x_i - \mu_n)
\end{equation*}
!et
where the data points $x_i \in \mathbb{R}^p$ (here in this example $p = 2$) are column vectors and $x^T$ is the transpose of $x$.
@@ -803,9 +803,9 @@ Compare the computed covariance with the answer given above.
=== Diagonalize the sample covariance matrix to obtain the principal components ===
Now we are ready to solve for the principal components! To do so we
diagonalize the sample covariance matrix $\Sigma_N$. We can use the
diagonalize the sample covariance matrix $\Sigma_n$. We can use the
function _np.linalg.eig_ to do so. It will return the eigenvalues and
eigenvectors of $\Sigma_N$. Once you have these, carry out the
eigenvectors of $\Sigma_n$. Once you have these, carry out the
following tasks:
* Compute the percentage of the total variance captured by the first principal component
@@ -814,7 +814,7 @@ following tasks:
* Approximate the data as
!bt
\begin{equation*}
x_i \approx \tilde{x}_i := \mu_N + \langle x_i, v_0 \rangle v_0
x_i \approx \tilde{x}_i := \mu_n + \langle x_i, v_0 \rangle v_0
\end{equation*}
!et
where $v_0$ is the first principal component. What do you observe?
@@ -826,9 +826,6 @@ Have the input be the data and have the output be the principal components and t
Finally, try out your own PCA function with other data sets.
After this we ask ourselves how do we prove the link between the maximum variance and the feature reduction.
!split
===== Classical PCA Theorem =====