lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

added more to pca code example

Browse Source
This commit is contained in:
mhjensen
2019-12-31 18:11:32 +01:00
parent 994e26c835
commit 18a95d96d8
10 changed files with 107 additions and 42 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/pub/DimRed/html/._DimRed-bs000.html
+1 -1
View File
@@ -224,7 +224,7 @@ MathJax.Hub.Config({
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>Dec 30, 2019</h4></center> <!-- date -->
<center><h4>Dec 31, 2019</h4></center> <!-- date -->
<br>
<p>
doc/pub/DimRed/html/._DimRed-bs018.html
+17 -6
View File
@@ -221,8 +221,8 @@ We will generate \( n = 1000 \) points \( X = \{ x_1, \ldots, x_N \} \) from
this distribution, and store them in the \( 1000 \times 2 \) matrix \( \boldsymbol{X} \).
<p>
The following Python code aids in setting up the data
The following Python code aids in setting up the data and writing out the design matrix.
Note that the function <b>multivariate</b> returns also the covariance discussed above and that it is defined by dividing by \( n-1 \) instead of \( n \).
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
@@ -237,7 +237,7 @@ X <span style="color: #666666">=</span> np<span style="color: #666666">.</span>r
<span style="color: #008000; font-weight: bold">print</span>(X)
</pre></div>
<p>
Make thereafter a small Python code which writes out the data. Note that the function <b>multivariate</b> returns also the covariance discussed above and that it is defined by dividing by \( n-1 \) instead of \( n \).
Try to add to this code your own calculation of the covariance matrix.
<p>
Now we are going to implement the PCA algorithm. We will break it down into various substeps.
@@ -258,7 +258,7 @@ $$
When you are done with these steps, print out \( \mu_n \) to verify it is
close to \( \mu \) and plot your mean centered data to verify it is
centered at the origin! Compare your code with the functionality from <b>Scikit-Learn</b> discussed above.
The following code elements perform these operations using <b>pandas</b> or your own functionality for doing so.
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
@@ -272,6 +272,12 @@ X_centered <span style="color: #666666">=</span> X <span style="color: #666666">
<span style="color: #408080; font-style: italic"># test that we get the same as Pandas</span>
<span style="color: #008000; font-weight: bold">print</span>(X_centered<span style="color: #666666">-</span>df)
</pre></div>
<p>
Alternatively, you could also have used the functions we discussed earlier for scaling the data set.
That is, we could have used the <b>StandardScaler</b> function in <b>Scikit-Learn</b>, a function which
ensures that for each feature/predictor we study the mean value is
zero and the variance is one (every column in the design/feature
matrix).
<h3 id="___sec19" class="anchor">Compute the sample covariance </h3>
@@ -312,9 +318,14 @@ where \( v_0 \) is the first principal component. What do you observe?
<p>
Finally, collect all these steps and write your own PCA function and
compare this with the functionality included in <b>Scikit-Learn</b>.
Have the input be the data and have the output be the principal components and their associated eigenvalues, sorted in descending order. Can you think of a way to make it more efficient than the algorithm outlined above?
compare this with the functionality included in <b>Scikit-Learn</b>. Have
the input be the data and have the output be the principal components
and their associated eigenvalues, sorted in descending order. Can you
think of a way to make it more efficient than the algorithm outlined
above?
<p>
The code here outlines some of the elements you could include in this analysis.
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
doc/pub/DimRed/html/DimRed-bs.html
+1 -1
View File
@@ -224,7 +224,7 @@ MathJax.Hub.Config({
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>Dec 30, 2019</h4></center> <!-- date -->
<center><h4>Dec 31, 2019</h4></center> <!-- date -->
<br>
<p>
doc/pub/DimRed/html/DimRed-reveal.html
+18 -7
View File
@@ -148,7 +148,7 @@ MathJax.Hub.Config({
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>&nbsp;<br>
<center><h4>Dec 30, 2019</h4></center> <!-- date -->
<center><h4>Dec 31, 2019</h4></center> <!-- date -->
<br>
<p>
@@ -1027,8 +1027,8 @@ We will generate \( n = 1000 \) points \( X = \{ x_1, \ldots, x_N \} \) from
this distribution, and store them in the \( 1000 \times 2 \) matrix \( \boldsymbol{X} \).
<p>
The following Python code aids in setting up the data
The following Python code aids in setting up the data and writing out the design matrix.
Note that the function <b>multivariate</b> returns also the covariance discussed above and that it is defined by dividing by \( n-1 \) instead of \( n \).
<p>
<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
@@ -1043,7 +1043,7 @@ X = np.random.multivariate_normal(mean, cov, n)
<span style="color: #8B008B; font-weight: bold">print</span>(X)
</pre></div>
<p>
Make thereafter a small Python code which writes out the data. Note that the function <b>multivariate</b> returns also the covariance discussed above and that it is defined by dividing by \( n-1 \) instead of \( n \).
Try to add to this code your own calculation of the covariance matrix.
<p>
Now we are going to implement the PCA algorithm. We will break it down into various substeps.
@@ -1068,7 +1068,7 @@ $$
When you are done with these steps, print out \( \mu_n \) to verify it is
close to \( \mu \) and plot your mean centered data to verify it is
centered at the origin! Compare your code with the functionality from <b>Scikit-Learn</b> discussed above.
The following code elements perform these operations using <b>pandas</b> or your own functionality for doing so.
<p>
<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
@@ -1082,6 +1082,12 @@ X_centered = X - X.mean(axis=<span style="color: #B452CD">0</span>)
<span style="color: #228B22"># test that we get the same as Pandas</span>
<span style="color: #8B008B; font-weight: bold">print</span>(X_centered-df)
</pre></div>
<p>
Alternatively, you could also have used the functions we discussed earlier for scaling the data set.
That is, we could have used the <b>StandardScaler</b> function in <b>Scikit-Learn</b>, a function which
ensures that for each feature/predictor we study the mean value is
zero and the variance is one (every column in the design/feature
matrix).
<h3 id="___sec19">Compute the sample covariance </h3>
@@ -1125,9 +1131,14 @@ where \( v_0 \) is the first principal component. What do you observe?
<p>
Finally, collect all these steps and write your own PCA function and
compare this with the functionality included in <b>Scikit-Learn</b>.
Have the input be the data and have the output be the principal components and their associated eigenvalues, sorted in descending order. Can you think of a way to make it more efficient than the algorithm outlined above?
compare this with the functionality included in <b>Scikit-Learn</b>. Have
the input be the data and have the output be the principal components
and their associated eigenvalues, sorted in descending order. Can you
think of a way to make it more efficient than the algorithm outlined
above?
<p>
The code here outlines some of the elements you could include in this analysis.
<p>
<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
doc/pub/DimRed/html/DimRed-solarized.html
+18 -7
View File
@@ -179,7 +179,7 @@ MathJax.Hub.Config({
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>Dec 30, 2019</h4></center> <!-- date -->
<center><h4>Dec 31, 2019</h4></center> <!-- date -->
<br>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
@@ -1007,8 +1007,8 @@ We will generate \( n = 1000 \) points \( X = \{ x_1, \ldots, x_N \} \) from
this distribution, and store them in the \( 1000 \times 2 \) matrix \( \boldsymbol{X} \).
<p>
The following Python code aids in setting up the data
The following Python code aids in setting up the data and writing out the design matrix.
Note that the function <b>multivariate</b> returns also the covariance discussed above and that it is defined by dividing by \( n-1 \) instead of \( n \).
<p>
<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
@@ -1023,7 +1023,7 @@ X = np.random.multivariate_normal(mean, cov, n)
<span style="color: #8B008B; font-weight: bold">print</span>(X)
</pre></div>
<p>
Make thereafter a small Python code which writes out the data. Note that the function <b>multivariate</b> returns also the covariance discussed above and that it is defined by dividing by \( n-1 \) instead of \( n \).
Try to add to this code your own calculation of the covariance matrix.
<p>
Now we are going to implement the PCA algorithm. We will break it down into various substeps.
@@ -1044,7 +1044,7 @@ $$
When you are done with these steps, print out \( \mu_n \) to verify it is
close to \( \mu \) and plot your mean centered data to verify it is
centered at the origin! Compare your code with the functionality from <b>Scikit-Learn</b> discussed above.
The following code elements perform these operations using <b>pandas</b> or your own functionality for doing so.
<p>
<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
@@ -1058,6 +1058,12 @@ X_centered = X - X.mean(axis=<span style="color: #B452CD">0</span>)
<span style="color: #228B22"># test that we get the same as Pandas</span>
<span style="color: #8B008B; font-weight: bold">print</span>(X_centered-df)
</pre></div>
<p>
Alternatively, you could also have used the functions we discussed earlier for scaling the data set.
That is, we could have used the <b>StandardScaler</b> function in <b>Scikit-Learn</b>, a function which
ensures that for each feature/predictor we study the mean value is
zero and the variance is one (every column in the design/feature
matrix).
<h3 id="___sec19">Compute the sample covariance </h3>
@@ -1098,9 +1104,14 @@ where \( v_0 \) is the first principal component. What do you observe?
<p>
Finally, collect all these steps and write your own PCA function and
compare this with the functionality included in <b>Scikit-Learn</b>.
Have the input be the data and have the output be the principal components and their associated eigenvalues, sorted in descending order. Can you think of a way to make it more efficient than the algorithm outlined above?
compare this with the functionality included in <b>Scikit-Learn</b>. Have
the input be the data and have the output be the principal components
and their associated eigenvalues, sorted in descending order. Can you
think of a way to make it more efficient than the algorithm outlined
above?
<p>
The code here outlines some of the elements you could include in this analysis.
<p>
<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
doc/pub/DimRed/html/DimRed.html
+18 -7
View File
@@ -184,7 +184,7 @@ MathJax.Hub.Config({
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>Dec 30, 2019</h4></center> <!-- date -->
<center><h4>Dec 31, 2019</h4></center> <!-- date -->
<br>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
@@ -1012,8 +1012,8 @@ We will generate \( n = 1000 \) points \( X = \{ x_1, \ldots, x_N \} \) from
this distribution, and store them in the \( 1000 \times 2 \) matrix \( \boldsymbol{X} \).
<p>
The following Python code aids in setting up the data
The following Python code aids in setting up the data and writing out the design matrix.
Note that the function <b>multivariate</b> returns also the covariance discussed above and that it is defined by dividing by \( n-1 \) instead of \( n \).
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
@@ -1028,7 +1028,7 @@ X <span style="color: #666666">=</span> np<span style="color: #666666">.</span>r
<span style="color: #008000; font-weight: bold">print</span>(X)
</pre></div>
<p>
Make thereafter a small Python code which writes out the data. Note that the function <b>multivariate</b> returns also the covariance discussed above and that it is defined by dividing by \( n-1 \) instead of \( n \).
Try to add to this code your own calculation of the covariance matrix.
<p>
Now we are going to implement the PCA algorithm. We will break it down into various substeps.
@@ -1049,7 +1049,7 @@ $$
When you are done with these steps, print out \( \mu_n \) to verify it is
close to \( \mu \) and plot your mean centered data to verify it is
centered at the origin! Compare your code with the functionality from <b>Scikit-Learn</b> discussed above.
The following code elements perform these operations using <b>pandas</b> or your own functionality for doing so.
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
@@ -1063,6 +1063,12 @@ X_centered <span style="color: #666666">=</span> X <span style="color: #666666">
<span style="color: #408080; font-style: italic"># test that we get the same as Pandas</span>
<span style="color: #008000; font-weight: bold">print</span>(X_centered<span style="color: #666666">-</span>df)
</pre></div>
<p>
Alternatively, you could also have used the functions we discussed earlier for scaling the data set.
That is, we could have used the <b>StandardScaler</b> function in <b>Scikit-Learn</b>, a function which
ensures that for each feature/predictor we study the mean value is
zero and the variance is one (every column in the design/feature
matrix).
<h3 id="___sec19">Compute the sample covariance </h3>
@@ -1103,9 +1109,14 @@ where \( v_0 \) is the first principal component. What do you observe?
<p>
Finally, collect all these steps and write your own PCA function and
compare this with the functionality included in <b>Scikit-Learn</b>.
Have the input be the data and have the output be the principal components and their associated eigenvalues, sorted in descending order. Can you think of a way to make it more efficient than the algorithm outlined above?
compare this with the functionality included in <b>Scikit-Learn</b>. Have
the input be the data and have the output be the principal components
and their associated eigenvalues, sorted in descending order. Can you
think of a way to make it more efficient than the algorithm outlined
above?
<p>
The code here outlines some of the elements you could include in this analysis.
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
doc/pub/DimRed/ipynb/DimRed.ipynb
+19 -6
View File
@@ -10,7 +10,7 @@
"<!-- Author: --> \n",
"**Morten Hjorth-Jensen**, Department of Physics, University of Oslo and Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University\n",
"\n",
"Date: **Dec 30, 2019**\n",
"Date: **Dec 31, 2019**\n",
"\n",
"Copyright 1999-2019, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license\n",
"\n",
@@ -1093,7 +1093,8 @@
"We will generate $n = 1000$ points $X = \\{ x_1, \\ldots, x_N \\}$ from\n",
"this distribution, and store them in the $1000 \\times 2$ matrix $\\boldsymbol{X}$.\n",
"\n",
"The following Python code aids in setting up the data"
"The following Python code aids in setting up the data and writing out the design matrix.\n",
"Note that the function **multivariate** returns also the covariance discussed above and that it is defined by dividing by $n-1$ instead of $n$."
]
},
{
@@ -1119,7 +1120,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Make thereafter a small Python code which writes out the data. Note that the function **multivariate** returns also the covariance discussed above and that it is defined by dividing by $n-1$ instead of $n$.\n",
"Try to add to this code your own calculation of the covariance matrix.\n",
"\n",
"Now we are going to implement the PCA algorithm. We will break it down into various substeps.\n",
"\n",
@@ -1159,7 +1160,8 @@
"source": [
"When you are done with these steps, print out $\\mu_n$ to verify it is\n",
"close to $\\mu$ and plot your mean centered data to verify it is\n",
"centered at the origin! Compare your code with the functionality from **Scikit-Learn** discussed above."
"centered at the origin! Compare your code with the functionality from **Scikit-Learn** discussed above.\n",
"The following code elements perform these operations using **pandas** or your own functionality for doing so."
]
},
{
@@ -1185,6 +1187,12 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Alternatively, you could also have used the functions we discussed earlier for scaling the data set. \n",
"That is, we could have used the **StandardScaler** function in **Scikit-Learn**, a function which \n",
"ensures that for each feature/predictor we study the mean value is\n",
"zero and the variance is one (every column in the design/feature\n",
"matrix).\n",
"\n",
"### Compute the sample covariance\n",
"\n",
"Now we are going to use the mean centered data to compute the sample covariance of the data. Recall it is given by:"
@@ -1240,8 +1248,13 @@
"where $v_0$ is the first principal component. What do you observe?\n",
"\n",
"Finally, collect all these steps and write your own PCA function and\n",
"compare this with the functionality included in **Scikit-Learn**.\n",
"Have the input be the data and have the output be the principal components and their associated eigenvalues, sorted in descending order. Can you think of a way to make it more efficient than the algorithm outlined above?"
"compare this with the functionality included in **Scikit-Learn**. Have\n",
"the input be the data and have the output be the principal components\n",
"and their associated eigenvalues, sorted in descending order. Can you\n",
"think of a way to make it more efficient than the algorithm outlined\n",
"above?\n",
"\n",
"The code here outlines some of the elements you could include in this analysis."
]
},
{
doc/pub/DimRed/ipynb/ipynb-DimRed-src.tar.gz
BIN
View File
Binary file not shown.
doc/pub/DimRed/pdf/DimRed-minted.pdf
BIN
View File
Binary file not shown.
doc/src/DimRed/DimRed.do.txt
+15 -7
View File
@@ -776,8 +776,8 @@ Note that the mean refers to each column of data.
We will generate $n = 1000$ points $X = \{ x_1, \ldots, x_N \}$ from
this distribution, and store them in the $1000 \times 2$ matrix $\bm{X}$.
The following Python code aids in setting up the data
The following Python code aids in setting up the data and writing out the design matrix.
Note that the function _multivariate_ returns also the covariance discussed above and that it is defined by dividing by $n-1$ instead of $n$.
!bc pycod
import numpy as np
import pandas as pd
@@ -791,7 +791,7 @@ print(X)
!ec
Make thereafter a small Python code which writes out the data. Note that the function _multivariate_ returns also the covariance discussed above and that it is defined by dividing by $n-1$ instead of $n$.
Try to add to this code your own calculation of the covariance matrix.
Now we are going to implement the PCA algorithm. We will break it down into various substeps.
@@ -812,7 +812,7 @@ and the mean-centered data $\bar{X} = \{ \bar{x}_1, \ldots, \bar{x}_n \}$ takes
When you are done with these steps, print out $\mu_n$ to verify it is
close to $\mu$ and plot your mean centered data to verify it is
centered at the origin! Compare your code with the functionality from _Scikit-Learn_ discussed above.
The following code elements perform these operations using _pandas_ or your own functionality for doing so.
!bc pycod
df = pd.DataFrame(X)
# Pandas does the centering for us
@@ -824,7 +824,11 @@ X_centered = X - X.mean(axis=0)
# test that we get the same as Pandas
print(X_centered-df)
!ec
Alternatively, you could also have used the functions we discussed earlier for scaling the data set.
That is, we could have used the _StandardScaler_ function in _Scikit-Learn_, a function which
ensures that for each feature/predictor we study the mean value is
zero and the variance is one (every column in the design/feature
matrix).
=== Compute the sample covariance ===
@@ -858,9 +862,13 @@ x_i \approx \tilde{x}_i := \mu_n + \langle x_i, v_0 \rangle v_0
where $v_0$ is the first principal component. What do you observe?
Finally, collect all these steps and write your own PCA function and
compare this with the functionality included in _Scikit-Learn_.
Have the input be the data and have the output be the principal components and their associated eigenvalues, sorted in descending order. Can you think of a way to make it more efficient than the algorithm outlined above?
compare this with the functionality included in _Scikit-Learn_. Have
the input be the data and have the output be the principal components
and their associated eigenvalues, sorted in descending order. Can you
think of a way to make it more efficient than the algorithm outlined
above?
The code here outlines some of the elements you could include in this analysis.
!bc pycod
#Now we do an SVD
U, s, V = np.linalg.svd(X_centered)