lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

some update of pca

Browse Source
This commit is contained in:
mhjensen
2020-01-01 10:51:38 +01:00
parent 18a95d96d8
commit 6b703367e3
11 changed files with 150 additions and 127 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/pub/DimRed/html/._DimRed-bs000.html
+2 -2
View File
@@ -224,7 +224,7 @@ MathJax.Hub.Config({
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>Dec 31, 2019</h4></center> <!-- date -->
<center><h4>Jan 1, 2020</h4></center> <!-- date -->
<br>
<p>
@@ -266,7 +266,7 @@ MathJax.Hub.Config({
<center style="font-size:80%">
<!-- copyright --> &copy; 1999-2019, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
<!-- copyright --> &copy; 1999-2020, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
</center>
doc/pub/DimRed/html/._DimRed-bs018.html
+18 -16
View File
@@ -290,7 +290,16 @@ $$
$$
where the data points \( x_i \in \mathbb{R}^p \) (here in this example \( p = 2 \)) are column vectors and \( x^T \) is the transpose of \( x \).
Compare the computed covariance with the answer given above.
We can write our own code or simply use either the functionaly of <b>numpy</b> or that of <b>pandas</b>, as follows
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #008000; font-weight: bold">print</span>(np<span style="color: #666666">.</span>cov(X<span style="color: #666666">.</span>T))
<span style="color: #008000; font-weight: bold">print</span>(df<span style="color: #666666">.</span>cov())
<span style="color: #008000; font-weight: bold">print</span>(np<span style="color: #666666">.</span>cov(X_centered<span style="color: #666666">.</span>T))
</pre></div>
<p>
Depending on the number of points \( n \), we will get results that are close to the covariance values defined above.
<h3 id="___sec20" class="anchor">Diagonalize the sample covariance matrix to obtain the principal components </h3>
@@ -298,14 +307,14 @@ Compare the computed covariance with the answer given above.
Now we are ready to solve for the principal components! To do so we
diagonalize the sample covariance matrix \( \Sigma_n \). We can use the
function <b>np.linalg.eig</b> to do so. It will return the eigenvalues and
eigenvectors of \( \Sigma_n \). Once you have these, carry out the
eigenvectors of \( \Sigma_n \). Once we have these we can perform the
following tasks:
<ul>
<li> Compute the percentage of the total variance captured by the first principal component</li>
<li> Plot the mean centered data and lines along the first and second principal components</li>
<li> Project the mean centered data onto the first and second principal components, and plot the projected data. What do you observe?</li>
<li> Approximate the data as</li>
<li> We compute the percentage of the total variance captured by the first principal component</li>
<li> We plot the mean centered data and lines along the first and second principal components</li>
<li> Then we project the mean centered data onto the first and second principal components, and plot the projected data.</li>
<li> Finally, we approximate the data as</li>
</ul>
$$
@@ -317,15 +326,11 @@ $$
where \( v_0 \) is the first principal component. What do you observe?
<p>
Finally, collect all these steps and write your own PCA function and
compare this with the functionality included in <b>Scikit-Learn</b>. Have
the input be the data and have the output be the principal components
and their associated eigenvalues, sorted in descending order. Can you
think of a way to make it more efficient than the algorithm outlined
above?
Collecting all these steps we can write our own PCA function and
compare this with the functionality included in <b>Scikit-Learn</b>.
<p>
The code here outlines some of the elements you could include in this analysis.
The code here outlines some of the elements we could include in the analysis. Feel free to extend upon this.
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
@@ -345,9 +350,6 @@ X2Dsl <span style="color: #666666">=</span> pca<span style="color: #666666">.</s
<span style="color: #008000; font-weight: bold">print</span>(pca<span style="color: #666666">.</span>components_<span style="color: #666666">.</span>T[:, <span style="color: #666666">0</span>])
</pre></div>
<p>
Finally, try out your own PCA function with other data sets.
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
doc/pub/DimRed/html/DimRed-bs.html
+2 -2
View File
@@ -224,7 +224,7 @@ MathJax.Hub.Config({
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>Dec 31, 2019</h4></center> <!-- date -->
<center><h4>Jan 1, 2020</h4></center> <!-- date -->
<br>
<p>
@@ -266,7 +266,7 @@ MathJax.Hub.Config({
<center style="font-size:80%">
<!-- copyright --> &copy; 1999-2019, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
<!-- copyright --> &copy; 1999-2020, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
</center>
doc/pub/DimRed/html/DimRed-reveal.html
+20 -17
View File
@@ -148,12 +148,12 @@ MathJax.Hub.Config({
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>&nbsp;<br>
<center><h4>Dec 31, 2019</h4></center> <!-- date -->
<center><h4>Jan 1, 2020</h4></center> <!-- date -->
<br>
<p>
<center style="font-size:80%">
<!-- copyright --> &copy; 1999-2019, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
<!-- copyright --> &copy; 1999-2020, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
</center>
</section>
@@ -1102,7 +1102,16 @@ $$
<p>&nbsp;<br>
where the data points \( x_i \in \mathbb{R}^p \) (here in this example \( p = 2 \)) are column vectors and \( x^T \) is the transpose of \( x \).
Compare the computed covariance with the answer given above.
We can write our own code or simply use either the functionaly of <b>numpy</b> or that of <b>pandas</b>, as follows
<p>
<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
<div class="highlight" style="background: #eeeedd"><pre style="font-size: 80%; line-height: 125%"><span></span><span style="color: #8B008B; font-weight: bold">print</span>(np.cov(X.T))
<span style="color: #8B008B; font-weight: bold">print</span>(df.cov())
<span style="color: #8B008B; font-weight: bold">print</span>(np.cov(X_centered.T))
</pre></div>
<p>
Depending on the number of points \( n \), we will get results that are close to the covariance values defined above.
<h3 id="___sec20">Diagonalize the sample covariance matrix to obtain the principal components </h3>
@@ -1110,14 +1119,14 @@ Compare the computed covariance with the answer given above.
Now we are ready to solve for the principal components! To do so we
diagonalize the sample covariance matrix \( \Sigma_n \). We can use the
function <b>np.linalg.eig</b> to do so. It will return the eigenvalues and
eigenvectors of \( \Sigma_n \). Once you have these, carry out the
eigenvectors of \( \Sigma_n \). Once we have these we can perform the
following tasks:
<ul>
<p><li> Compute the percentage of the total variance captured by the first principal component</li>
<p><li> Plot the mean centered data and lines along the first and second principal components</li>
<p><li> Project the mean centered data onto the first and second principal components, and plot the projected data. What do you observe?</li>
<p><li> Approximate the data as</li>
<p><li> We compute the percentage of the total variance captured by the first principal component</li>
<p><li> We plot the mean centered data and lines along the first and second principal components</li>
<p><li> Then we project the mean centered data onto the first and second principal components, and plot the projected data.</li>
<p><li> Finally, we approximate the data as</li>
</ul>
<p>&nbsp;<br>
$$
@@ -1130,15 +1139,11 @@ $$
where \( v_0 \) is the first principal component. What do you observe?
<p>
Finally, collect all these steps and write your own PCA function and
compare this with the functionality included in <b>Scikit-Learn</b>. Have
the input be the data and have the output be the principal components
and their associated eigenvalues, sorted in descending order. Can you
think of a way to make it more efficient than the algorithm outlined
above?
Collecting all these steps we can write our own PCA function and
compare this with the functionality included in <b>Scikit-Learn</b>.
<p>
The code here outlines some of the elements you could include in this analysis.
The code here outlines some of the elements we could include in the analysis. Feel free to extend upon this.
<p>
<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
@@ -1158,8 +1163,6 @@ X2Dsl = pca.fit_transform(X)
<span style="color: #8B008B; font-weight: bold">print</span>(pca.components_.T[:, <span style="color: #B452CD">0</span>])
</pre></div>
<p>
Finally, try out your own PCA function with other data sets.
</section>
doc/pub/DimRed/html/DimRed-solarized.html
+20 -18
View File
@@ -179,7 +179,7 @@ MathJax.Hub.Config({
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>Dec 31, 2019</h4></center> <!-- date -->
<center><h4>Jan 1, 2020</h4></center> <!-- date -->
<br>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
@@ -1076,7 +1076,16 @@ $$
$$
where the data points \( x_i \in \mathbb{R}^p \) (here in this example \( p = 2 \)) are column vectors and \( x^T \) is the transpose of \( x \).
Compare the computed covariance with the answer given above.
We can write our own code or simply use either the functionaly of <b>numpy</b> or that of <b>pandas</b>, as follows
<p>
<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
<div class="highlight" style="background: #eeeedd"><pre style="line-height: 125%"><span></span><span style="color: #8B008B; font-weight: bold">print</span>(np.cov(X.T))
<span style="color: #8B008B; font-weight: bold">print</span>(df.cov())
<span style="color: #8B008B; font-weight: bold">print</span>(np.cov(X_centered.T))
</pre></div>
<p>
Depending on the number of points \( n \), we will get results that are close to the covariance values defined above.
<h3 id="___sec20">Diagonalize the sample covariance matrix to obtain the principal components </h3>
@@ -1084,14 +1093,14 @@ Compare the computed covariance with the answer given above.
Now we are ready to solve for the principal components! To do so we
diagonalize the sample covariance matrix \( \Sigma_n \). We can use the
function <b>np.linalg.eig</b> to do so. It will return the eigenvalues and
eigenvectors of \( \Sigma_n \). Once you have these, carry out the
eigenvectors of \( \Sigma_n \). Once we have these we can perform the
following tasks:
<ul>
<li> Compute the percentage of the total variance captured by the first principal component</li>
<li> Plot the mean centered data and lines along the first and second principal components</li>
<li> Project the mean centered data onto the first and second principal components, and plot the projected data. What do you observe?</li>
<li> Approximate the data as</li>
<li> We compute the percentage of the total variance captured by the first principal component</li>
<li> We plot the mean centered data and lines along the first and second principal components</li>
<li> Then we project the mean centered data onto the first and second principal components, and plot the projected data.</li>
<li> Finally, we approximate the data as</li>
</ul>
$$
@@ -1103,15 +1112,11 @@ $$
where \( v_0 \) is the first principal component. What do you observe?
<p>
Finally, collect all these steps and write your own PCA function and
compare this with the functionality included in <b>Scikit-Learn</b>. Have
the input be the data and have the output be the principal components
and their associated eigenvalues, sorted in descending order. Can you
think of a way to make it more efficient than the algorithm outlined
above?
Collecting all these steps we can write our own PCA function and
compare this with the functionality included in <b>Scikit-Learn</b>.
<p>
The code here outlines some of the elements you could include in this analysis.
The code here outlines some of the elements we could include in the analysis. Feel free to extend upon this.
<p>
<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
@@ -1131,9 +1136,6 @@ X2Dsl = pca.fit_transform(X)
<span style="color: #8B008B; font-weight: bold">print</span>(pca.components_.T[:, <span style="color: #B452CD">0</span>])
</pre></div>
<p>
Finally, try out your own PCA function with other data sets.
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
@@ -1538,7 +1540,7 @@ Here are some of the most popular:
<center style="font-size:80%">
<!-- copyright --> &copy; 1999-2019, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
<!-- copyright --> &copy; 1999-2020, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
</center>
doc/pub/DimRed/html/DimRed.html
+20 -18
View File
@@ -184,7 +184,7 @@ MathJax.Hub.Config({
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>Dec 31, 2019</h4></center> <!-- date -->
<center><h4>Jan 1, 2020</h4></center> <!-- date -->
<br>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
@@ -1081,7 +1081,16 @@ $$
$$
where the data points \( x_i \in \mathbb{R}^p \) (here in this example \( p = 2 \)) are column vectors and \( x^T \) is the transpose of \( x \).
Compare the computed covariance with the answer given above.
We can write our own code or simply use either the functionaly of <b>numpy</b> or that of <b>pandas</b>, as follows
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #008000; font-weight: bold">print</span>(np<span style="color: #666666">.</span>cov(X<span style="color: #666666">.</span>T))
<span style="color: #008000; font-weight: bold">print</span>(df<span style="color: #666666">.</span>cov())
<span style="color: #008000; font-weight: bold">print</span>(np<span style="color: #666666">.</span>cov(X_centered<span style="color: #666666">.</span>T))
</pre></div>
<p>
Depending on the number of points \( n \), we will get results that are close to the covariance values defined above.
<h3 id="___sec20">Diagonalize the sample covariance matrix to obtain the principal components </h3>
@@ -1089,14 +1098,14 @@ Compare the computed covariance with the answer given above.
Now we are ready to solve for the principal components! To do so we
diagonalize the sample covariance matrix \( \Sigma_n \). We can use the
function <b>np.linalg.eig</b> to do so. It will return the eigenvalues and
eigenvectors of \( \Sigma_n \). Once you have these, carry out the
eigenvectors of \( \Sigma_n \). Once we have these we can perform the
following tasks:
<ul>
<li> Compute the percentage of the total variance captured by the first principal component</li>
<li> Plot the mean centered data and lines along the first and second principal components</li>
<li> Project the mean centered data onto the first and second principal components, and plot the projected data. What do you observe?</li>
<li> Approximate the data as</li>
<li> We compute the percentage of the total variance captured by the first principal component</li>
<li> We plot the mean centered data and lines along the first and second principal components</li>
<li> Then we project the mean centered data onto the first and second principal components, and plot the projected data.</li>
<li> Finally, we approximate the data as</li>
</ul>
$$
@@ -1108,15 +1117,11 @@ $$
where \( v_0 \) is the first principal component. What do you observe?
<p>
Finally, collect all these steps and write your own PCA function and
compare this with the functionality included in <b>Scikit-Learn</b>. Have
the input be the data and have the output be the principal components
and their associated eigenvalues, sorted in descending order. Can you
think of a way to make it more efficient than the algorithm outlined
above?
Collecting all these steps we can write our own PCA function and
compare this with the functionality included in <b>Scikit-Learn</b>.
<p>
The code here outlines some of the elements you could include in this analysis.
The code here outlines some of the elements we could include in the analysis. Feel free to extend upon this.
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
@@ -1136,9 +1141,6 @@ X2Dsl <span style="color: #666666">=</span> pca<span style="color: #666666">.</s
<span style="color: #008000; font-weight: bold">print</span>(pca<span style="color: #666666">.</span>components_<span style="color: #666666">.</span>T[:, <span style="color: #666666">0</span>])
</pre></div>
<p>
Finally, try out your own PCA function with other data sets.
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
@@ -1543,7 +1545,7 @@ Here are some of the most popular:
<center style="font-size:80%">
<!-- copyright --> &copy; 1999-2019, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
<!-- copyright --> &copy; 1999-2020, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
</center>
doc/pub/DimRed/ipynb/DimRed.ipynb
+40 -29
View File
@@ -10,9 +10,9 @@
"<!-- Author: --> \n",
"**Morten Hjorth-Jensen**, Department of Physics, University of Oslo and Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University\n",
"\n",
"Date: **Dec 31, 2019**\n",
"Date: **Jan 1, 2020**\n",
"\n",
"Copyright 1999-2019, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license\n",
"Copyright 1999-2020, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license\n",
"\n",
"\n",
"\n",
@@ -1212,7 +1212,27 @@
"metadata": {},
"source": [
"where the data points $x_i \\in \\mathbb{R}^p$ (here in this example $p = 2$) are column vectors and $x^T$ is the transpose of $x$.\n",
"Compare the computed covariance with the answer given above.\n",
"We can write our own code or simply use either the functionaly of **numpy** or that of **pandas**, as follows"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"print(np.cov(X.T))\n",
"print(df.cov())\n",
"print(np.cov(X_centered.T))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Depending on the number of points $n$, we will get results that are close to the covariance values defined above.\n",
"\n",
"\n",
"### Diagonalize the sample covariance matrix to obtain the principal components\n",
@@ -1220,16 +1240,16 @@
"Now we are ready to solve for the principal components! To do so we\n",
"diagonalize the sample covariance matrix $\\Sigma_n$. We can use the\n",
"function **np.linalg.eig** to do so. It will return the eigenvalues and\n",
"eigenvectors of $\\Sigma_n$. Once you have these, carry out the\n",
"eigenvectors of $\\Sigma_n$. Once we have these we can perform the \n",
"following tasks:\n",
"\n",
"* Compute the percentage of the total variance captured by the first principal component\n",
"* We compute the percentage of the total variance captured by the first principal component\n",
"\n",
"* Plot the mean centered data and lines along the first and second principal components\n",
"* We plot the mean centered data and lines along the first and second principal components\n",
"\n",
"* Project the mean centered data onto the first and second principal components, and plot the projected data. What do you observe?\n",
"* Then we project the mean centered data onto the first and second principal components, and plot the projected data. \n",
"\n",
"* Approximate the data as"
"* Finally, we approximate the data as"
]
},
{
@@ -1247,19 +1267,15 @@
"source": [
"where $v_0$ is the first principal component. What do you observe?\n",
"\n",
"Finally, collect all these steps and write your own PCA function and\n",
"compare this with the functionality included in **Scikit-Learn**. Have\n",
"the input be the data and have the output be the principal components\n",
"and their associated eigenvalues, sorted in descending order. Can you\n",
"think of a way to make it more efficient than the algorithm outlined\n",
"above?\n",
"Collecting all these steps we can write our own PCA function and\n",
"compare this with the functionality included in **Scikit-Learn**. \n",
"\n",
"The code here outlines some of the elements you could include in this analysis."
"The code here outlines some of the elements we could include in the analysis. Feel free to extend upon this."
]
},
{
"cell_type": "code",
"execution_count": 13,
"execution_count": 14,
"metadata": {
"collapsed": false
},
@@ -1286,11 +1302,6 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Finally, try out your own PCA function with other data sets.\n",
"\n",
"\n",
"\n",
"\n",
"## Classical PCA Theorem\n",
"\n",
"We assume now that we have a design matrix $\\boldsymbol{X}$ which has been\n",
@@ -1575,7 +1586,7 @@
},
{
"cell_type": "code",
"execution_count": 14,
"execution_count": 15,
"metadata": {
"collapsed": false
},
@@ -1622,7 +1633,7 @@
},
{
"cell_type": "code",
"execution_count": 15,
"execution_count": 16,
"metadata": {
"collapsed": false
},
@@ -1646,7 +1657,7 @@
},
{
"cell_type": "code",
"execution_count": 16,
"execution_count": 17,
"metadata": {
"collapsed": false
},
@@ -1670,7 +1681,7 @@
},
{
"cell_type": "code",
"execution_count": 17,
"execution_count": 18,
"metadata": {
"collapsed": false
},
@@ -1694,7 +1705,7 @@
},
{
"cell_type": "code",
"execution_count": 18,
"execution_count": 19,
"metadata": {
"collapsed": false
},
@@ -1748,7 +1759,7 @@
},
{
"cell_type": "code",
"execution_count": 19,
"execution_count": 20,
"metadata": {
"collapsed": false
},
@@ -1771,7 +1782,7 @@
},
{
"cell_type": "code",
"execution_count": 20,
"execution_count": 21,
"metadata": {
"collapsed": false
},
@@ -1818,7 +1829,7 @@
},
{
"cell_type": "code",
"execution_count": 21,
"execution_count": 22,
"metadata": {
"collapsed": false
},
doc/pub/DimRed/ipynb/ipynb-DimRed-src.tar.gz
BIN
View File
Binary file not shown.
doc/pub/DimRed/pdf/DimRed-minted.pdf
BIN
View File
Binary file not shown.
doc/src/DimRed/DimRed.do.txt
+15 -19
View File
@@ -839,7 +839,13 @@ Now we are going to use the mean centered data to compute the sample covariance
\end{equation*}
!et
where the data points $x_i \in \mathbb{R}^p$ (here in this example $p = 2$) are column vectors and $x^T$ is the transpose of $x$.
Compare the computed covariance with the answer given above.
We can write our own code or simply use either the functionaly of _numpy_ or that of _pandas_, as follows
!bc pycod
print(np.cov(X.T))
print(df.cov())
print(np.cov(X_centered.T))
!ec
Depending on the number of points $n$, we will get results that are close to the covariance values defined above.
=== Diagonalize the sample covariance matrix to obtain the principal components ===
@@ -847,13 +853,13 @@ Compare the computed covariance with the answer given above.
Now we are ready to solve for the principal components! To do so we
diagonalize the sample covariance matrix $\Sigma_n$. We can use the
function _np.linalg.eig_ to do so. It will return the eigenvalues and
eigenvectors of $\Sigma_n$. Once you have these, carry out the
eigenvectors of $\Sigma_n$. Once we have these we can perform the
following tasks:
* Compute the percentage of the total variance captured by the first principal component
* Plot the mean centered data and lines along the first and second principal components
* Project the mean centered data onto the first and second principal components, and plot the projected data. What do you observe?
* Approximate the data as
* We compute the percentage of the total variance captured by the first principal component
* We plot the mean centered data and lines along the first and second principal components
* Then we project the mean centered data onto the first and second principal components, and plot the projected data.
* Finally, we approximate the data as
!bt
\begin{equation*}
x_i \approx \tilde{x}_i := \mu_n + \langle x_i, v_0 \rangle v_0
@@ -861,14 +867,10 @@ x_i \approx \tilde{x}_i := \mu_n + \langle x_i, v_0 \rangle v_0
!et
where $v_0$ is the first principal component. What do you observe?
Finally, collect all these steps and write your own PCA function and
compare this with the functionality included in _Scikit-Learn_. Have
the input be the data and have the output be the principal components
and their associated eigenvalues, sorted in descending order. Can you
think of a way to make it more efficient than the algorithm outlined
above?
Collecting all these steps we can write our own PCA function and
compare this with the functionality included in _Scikit-Learn_.
The code here outlines some of the elements you could include in this analysis.
The code here outlines some of the elements we could include in the analysis. Feel free to extend upon this.
!bc pycod
#Now we do an SVD
U, s, V = np.linalg.svd(X_centered)
@@ -890,12 +892,6 @@ print(pca.components_.T[:, 0])
!ec
Finally, try out your own PCA function with other data sets.
!split
===== Classical PCA Theorem =====
doc/src/DimRed/Programs/PCAsimple.py
+13 -6
View File
@@ -1,30 +1,35 @@
import numpy as np
import pandas as pd
from IPython.display import display
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import MinMaxScaler, StandardScaler
n = 100
n = 10000
mean = (-1, 2)
cov = [[4, 2], [2, 2]]
print(cov)
X = np.random.multivariate_normal(mean, cov, n)
print(X)
print(np.cov(X.T))
df = pd.DataFrame(X)
# Pandas does the centering for us
df = df -df.mean()
display(df)
correlation_matrix = df.cov()
print(correlation_matrix)
# we center it ourselves
X_centered = X - X.mean(axis=0)
# test that we get the same as Pandas
print(X_centered-df)
print(np.cov(X_centered.T))
#print("test that we get the same as Pandas")
#print(X_centered-X_train_scaled)
#Now we do an SVD
U, s, V = np.linalg.svd(X_centered)
c1 = V.T[:, 0]
c2 = V.T[:, 1]
W2 = V.T[:, :2]
X2D = X_centered.dot(W2)
print(X2D)
#thereafter we do a PCA with Scikit-learn
from sklearn.decomposition import PCA
pca = PCA(n_components = 2)
@@ -34,3 +39,5 @@ print(X2D-X2Dsl)
print(pca.components_.T[:, 0])