diff --git a/doc/pub/DimRed/html/._DimRed-bs000.html b/doc/pub/DimRed/html/._DimRed-bs000.html index f4b4f0e52..59e9e22ca 100644 --- a/doc/pub/DimRed/html/._DimRed-bs000.html +++ b/doc/pub/DimRed/html/._DimRed-bs000.html @@ -224,7 +224,7 @@ MathJax.Hub.Config({
[2] Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University

-

Dec 31, 2019

+

Jan 1, 2020


@@ -266,7 +266,7 @@ MathJax.Hub.Config({

- © 1999-2019, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license + © 1999-2020, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
diff --git a/doc/pub/DimRed/html/._DimRed-bs018.html b/doc/pub/DimRed/html/._DimRed-bs018.html index e7381a724..4686a77d7 100644 --- a/doc/pub/DimRed/html/._DimRed-bs018.html +++ b/doc/pub/DimRed/html/._DimRed-bs018.html @@ -290,7 +290,16 @@ $$ $$ where the data points \( x_i \in \mathbb{R}^p \) (here in this example \( p = 2 \)) are column vectors and \( x^T \) is the transpose of \( x \). -Compare the computed covariance with the answer given above. +We can write our own code or simply use either the functionaly of numpy or that of pandas, as follows +

+ + +

print(np.cov(X.T))
+print(df.cov())
+print(np.cov(X_centered.T))
+
+

+Depending on the number of points \( n \), we will get results that are close to the covariance values defined above.

Diagonalize the sample covariance matrix to obtain the principal components

@@ -298,14 +307,14 @@ Compare the computed covariance with the answer given above. Now we are ready to solve for the principal components! To do so we diagonalize the sample covariance matrix \( \Sigma_n \). We can use the function np.linalg.eig to do so. It will return the eigenvalues and -eigenvectors of \( \Sigma_n \). Once you have these, carry out the +eigenvectors of \( \Sigma_n \). Once we have these we can perform the following tasks: $$ @@ -317,15 +326,11 @@ $$ where \( v_0 \) is the first principal component. What do you observe?

-Finally, collect all these steps and write your own PCA function and -compare this with the functionality included in Scikit-Learn. Have -the input be the data and have the output be the principal components -and their associated eigenvalues, sorted in descending order. Can you -think of a way to make it more efficient than the algorithm outlined -above? +Collecting all these steps we can write our own PCA function and +compare this with the functionality included in Scikit-Learn.

-The code here outlines some of the elements you could include in this analysis. +The code here outlines some of the elements we could include in the analysis. Feel free to extend upon this.

@@ -345,9 +350,6 @@ X2Dsl = pca.print(pca.components_.T[:, 0]) -

-Finally, try out your own PCA function with other data sets. -

diff --git a/doc/pub/DimRed/html/DimRed-bs.html b/doc/pub/DimRed/html/DimRed-bs.html index f4b4f0e52..59e9e22ca 100644 --- a/doc/pub/DimRed/html/DimRed-bs.html +++ b/doc/pub/DimRed/html/DimRed-bs.html @@ -224,7 +224,7 @@ MathJax.Hub.Config({

[2] Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University

-

Dec 31, 2019

+

Jan 1, 2020


@@ -266,7 +266,7 @@ MathJax.Hub.Config({

- © 1999-2019, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license + © 1999-2020, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
diff --git a/doc/pub/DimRed/html/DimRed-reveal.html b/doc/pub/DimRed/html/DimRed-reveal.html index 4c8a61a41..dff77c4d9 100644 --- a/doc/pub/DimRed/html/DimRed-reveal.html +++ b/doc/pub/DimRed/html/DimRed-reveal.html @@ -148,12 +148,12 @@ MathJax.Hub.Config({
[2] Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University

 
-

Dec 31, 2019

+

Jan 1, 2020


- © 1999-2019, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license + © 1999-2020, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
@@ -1102,7 +1102,16 @@ $$

 
where the data points \( x_i \in \mathbb{R}^p \) (here in this example \( p = 2 \)) are column vectors and \( x^T \) is the transpose of \( x \). -Compare the computed covariance with the answer given above. +We can write our own code or simply use either the functionaly of numpy or that of pandas, as follows +

+ + +

print(np.cov(X.T))
+print(df.cov())
+print(np.cov(X_centered.T))
+
+

+Depending on the number of points \( n \), we will get results that are close to the covariance values defined above.

Diagonalize the sample covariance matrix to obtain the principal components

@@ -1110,14 +1119,14 @@ Compare the computed covariance with the answer given above. Now we are ready to solve for the principal components! To do so we diagonalize the sample covariance matrix \( \Sigma_n \). We can use the function np.linalg.eig to do so. It will return the eigenvalues and -eigenvectors of \( \Sigma_n \). Once you have these, carry out the +eigenvectors of \( \Sigma_n \). Once we have these we can perform the following tasks:

 
$$ @@ -1130,15 +1139,11 @@ $$ where \( v_0 \) is the first principal component. What do you observe?

-Finally, collect all these steps and write your own PCA function and -compare this with the functionality included in Scikit-Learn. Have -the input be the data and have the output be the principal components -and their associated eigenvalues, sorted in descending order. Can you -think of a way to make it more efficient than the algorithm outlined -above? +Collecting all these steps we can write our own PCA function and +compare this with the functionality included in Scikit-Learn.

-The code here outlines some of the elements you could include in this analysis. +The code here outlines some of the elements we could include in the analysis. Feel free to extend upon this.

@@ -1158,8 +1163,6 @@ X2Dsl = pca.fit_transform(X) print(pca.components_.T[:, 0]) -

-Finally, try out your own PCA function with other data sets. diff --git a/doc/pub/DimRed/html/DimRed-solarized.html b/doc/pub/DimRed/html/DimRed-solarized.html index b46a53c3d..ee3a4cc6f 100644 --- a/doc/pub/DimRed/html/DimRed-solarized.html +++ b/doc/pub/DimRed/html/DimRed-solarized.html @@ -179,7 +179,7 @@ MathJax.Hub.Config({

[2] Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University

-

Dec 31, 2019

+

Jan 1, 2020












@@ -1076,7 +1076,16 @@ $$ $$ where the data points \( x_i \in \mathbb{R}^p \) (here in this example \( p = 2 \)) are column vectors and \( x^T \) is the transpose of \( x \). -Compare the computed covariance with the answer given above. +We can write our own code or simply use either the functionaly of numpy or that of pandas, as follows +

+ + +

print(np.cov(X.T))
+print(df.cov())
+print(np.cov(X_centered.T))
+
+

+Depending on the number of points \( n \), we will get results that are close to the covariance values defined above.

Diagonalize the sample covariance matrix to obtain the principal components

@@ -1084,14 +1093,14 @@ Compare the computed covariance with the answer given above. Now we are ready to solve for the principal components! To do so we diagonalize the sample covariance matrix \( \Sigma_n \). We can use the function np.linalg.eig to do so. It will return the eigenvalues and -eigenvectors of \( \Sigma_n \). Once you have these, carry out the +eigenvectors of \( \Sigma_n \). Once we have these we can perform the following tasks: $$ @@ -1103,15 +1112,11 @@ $$ where \( v_0 \) is the first principal component. What do you observe?

-Finally, collect all these steps and write your own PCA function and -compare this with the functionality included in Scikit-Learn. Have -the input be the data and have the output be the principal components -and their associated eigenvalues, sorted in descending order. Can you -think of a way to make it more efficient than the algorithm outlined -above? +Collecting all these steps we can write our own PCA function and +compare this with the functionality included in Scikit-Learn.

-The code here outlines some of the elements you could include in this analysis. +The code here outlines some of the elements we could include in the analysis. Feel free to extend upon this.

@@ -1131,9 +1136,6 @@ X2Dsl = pca.fit_transform(X) print(pca.components_.T[:, 0]) -

-Finally, try out your own PCA function with other data sets. -











@@ -1538,7 +1540,7 @@ Here are some of the most popular:

- © 1999-2019, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license + © 1999-2020, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
diff --git a/doc/pub/DimRed/html/DimRed.html b/doc/pub/DimRed/html/DimRed.html index 3d025d9fe..2e10a51a3 100644 --- a/doc/pub/DimRed/html/DimRed.html +++ b/doc/pub/DimRed/html/DimRed.html @@ -184,7 +184,7 @@ MathJax.Hub.Config({
[2] Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University

-

Dec 31, 2019

+

Jan 1, 2020












@@ -1081,7 +1081,16 @@ $$ $$ where the data points \( x_i \in \mathbb{R}^p \) (here in this example \( p = 2 \)) are column vectors and \( x^T \) is the transpose of \( x \). -Compare the computed covariance with the answer given above. +We can write our own code or simply use either the functionaly of numpy or that of pandas, as follows +

+ + +

print(np.cov(X.T))
+print(df.cov())
+print(np.cov(X_centered.T))
+
+

+Depending on the number of points \( n \), we will get results that are close to the covariance values defined above.

Diagonalize the sample covariance matrix to obtain the principal components

@@ -1089,14 +1098,14 @@ Compare the computed covariance with the answer given above. Now we are ready to solve for the principal components! To do so we diagonalize the sample covariance matrix \( \Sigma_n \). We can use the function np.linalg.eig to do so. It will return the eigenvalues and -eigenvectors of \( \Sigma_n \). Once you have these, carry out the +eigenvectors of \( \Sigma_n \). Once we have these we can perform the following tasks: $$ @@ -1108,15 +1117,11 @@ $$ where \( v_0 \) is the first principal component. What do you observe?

-Finally, collect all these steps and write your own PCA function and -compare this with the functionality included in Scikit-Learn. Have -the input be the data and have the output be the principal components -and their associated eigenvalues, sorted in descending order. Can you -think of a way to make it more efficient than the algorithm outlined -above? +Collecting all these steps we can write our own PCA function and +compare this with the functionality included in Scikit-Learn.

-The code here outlines some of the elements you could include in this analysis. +The code here outlines some of the elements we could include in the analysis. Feel free to extend upon this.

@@ -1136,9 +1141,6 @@ X2Dsl = pca.print(pca.components_.T[:, 0]) -

-Finally, try out your own PCA function with other data sets. -











@@ -1543,7 +1545,7 @@ Here are some of the most popular:

- © 1999-2019, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license + © 1999-2020, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license
diff --git a/doc/pub/DimRed/ipynb/DimRed.ipynb b/doc/pub/DimRed/ipynb/DimRed.ipynb index 5707585db..c6bca2820 100644 --- a/doc/pub/DimRed/ipynb/DimRed.ipynb +++ b/doc/pub/DimRed/ipynb/DimRed.ipynb @@ -10,9 +10,9 @@ " \n", "**Morten Hjorth-Jensen**, Department of Physics, University of Oslo and Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University\n", "\n", - "Date: **Dec 31, 2019**\n", + "Date: **Jan 1, 2020**\n", "\n", - "Copyright 1999-2019, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license\n", + "Copyright 1999-2020, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license\n", "\n", "\n", "\n", @@ -1212,7 +1212,27 @@ "metadata": {}, "source": [ "where the data points $x_i \\in \\mathbb{R}^p$ (here in this example $p = 2$) are column vectors and $x^T$ is the transpose of $x$.\n", - "Compare the computed covariance with the answer given above.\n", + "We can write our own code or simply use either the functionaly of **numpy** or that of **pandas**, as follows" + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": { + "collapsed": false + }, + "outputs": [], + "source": [ + "print(np.cov(X.T))\n", + "print(df.cov())\n", + "print(np.cov(X_centered.T))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Depending on the number of points $n$, we will get results that are close to the covariance values defined above.\n", "\n", "\n", "### Diagonalize the sample covariance matrix to obtain the principal components\n", @@ -1220,16 +1240,16 @@ "Now we are ready to solve for the principal components! To do so we\n", "diagonalize the sample covariance matrix $\\Sigma_n$. We can use the\n", "function **np.linalg.eig** to do so. It will return the eigenvalues and\n", - "eigenvectors of $\\Sigma_n$. Once you have these, carry out the\n", + "eigenvectors of $\\Sigma_n$. Once we have these we can perform the \n", "following tasks:\n", "\n", - "* Compute the percentage of the total variance captured by the first principal component\n", + "* We compute the percentage of the total variance captured by the first principal component\n", "\n", - "* Plot the mean centered data and lines along the first and second principal components\n", + "* We plot the mean centered data and lines along the first and second principal components\n", "\n", - "* Project the mean centered data onto the first and second principal components, and plot the projected data. What do you observe?\n", + "* Then we project the mean centered data onto the first and second principal components, and plot the projected data. \n", "\n", - "* Approximate the data as" + "* Finally, we approximate the data as" ] }, { @@ -1247,19 +1267,15 @@ "source": [ "where $v_0$ is the first principal component. What do you observe?\n", "\n", - "Finally, collect all these steps and write your own PCA function and\n", - "compare this with the functionality included in **Scikit-Learn**. Have\n", - "the input be the data and have the output be the principal components\n", - "and their associated eigenvalues, sorted in descending order. Can you\n", - "think of a way to make it more efficient than the algorithm outlined\n", - "above?\n", + "Collecting all these steps we can write our own PCA function and\n", + "compare this with the functionality included in **Scikit-Learn**. \n", "\n", - "The code here outlines some of the elements you could include in this analysis." + "The code here outlines some of the elements we could include in the analysis. Feel free to extend upon this." ] }, { "cell_type": "code", - "execution_count": 13, + "execution_count": 14, "metadata": { "collapsed": false }, @@ -1286,11 +1302,6 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "Finally, try out your own PCA function with other data sets.\n", - "\n", - "\n", - "\n", - "\n", "## Classical PCA Theorem\n", "\n", "We assume now that we have a design matrix $\\boldsymbol{X}$ which has been\n", @@ -1575,7 +1586,7 @@ }, { "cell_type": "code", - "execution_count": 14, + "execution_count": 15, "metadata": { "collapsed": false }, @@ -1622,7 +1633,7 @@ }, { "cell_type": "code", - "execution_count": 15, + "execution_count": 16, "metadata": { "collapsed": false }, @@ -1646,7 +1657,7 @@ }, { "cell_type": "code", - "execution_count": 16, + "execution_count": 17, "metadata": { "collapsed": false }, @@ -1670,7 +1681,7 @@ }, { "cell_type": "code", - "execution_count": 17, + "execution_count": 18, "metadata": { "collapsed": false }, @@ -1694,7 +1705,7 @@ }, { "cell_type": "code", - "execution_count": 18, + "execution_count": 19, "metadata": { "collapsed": false }, @@ -1748,7 +1759,7 @@ }, { "cell_type": "code", - "execution_count": 19, + "execution_count": 20, "metadata": { "collapsed": false }, @@ -1771,7 +1782,7 @@ }, { "cell_type": "code", - "execution_count": 20, + "execution_count": 21, "metadata": { "collapsed": false }, @@ -1818,7 +1829,7 @@ }, { "cell_type": "code", - "execution_count": 21, + "execution_count": 22, "metadata": { "collapsed": false }, diff --git a/doc/pub/DimRed/ipynb/ipynb-DimRed-src.tar.gz b/doc/pub/DimRed/ipynb/ipynb-DimRed-src.tar.gz index ed525379a..352fb2e04 100644 Binary files a/doc/pub/DimRed/ipynb/ipynb-DimRed-src.tar.gz and b/doc/pub/DimRed/ipynb/ipynb-DimRed-src.tar.gz differ diff --git a/doc/pub/DimRed/pdf/DimRed-minted.pdf b/doc/pub/DimRed/pdf/DimRed-minted.pdf index 243dbec69..02e5a8671 100644 Binary files a/doc/pub/DimRed/pdf/DimRed-minted.pdf and b/doc/pub/DimRed/pdf/DimRed-minted.pdf differ diff --git a/doc/src/DimRed/DimRed.do.txt b/doc/src/DimRed/DimRed.do.txt index 0d9e62d79..c4536868a 100644 --- a/doc/src/DimRed/DimRed.do.txt +++ b/doc/src/DimRed/DimRed.do.txt @@ -839,7 +839,13 @@ Now we are going to use the mean centered data to compute the sample covariance \end{equation*} !et where the data points $x_i \in \mathbb{R}^p$ (here in this example $p = 2$) are column vectors and $x^T$ is the transpose of $x$. -Compare the computed covariance with the answer given above. +We can write our own code or simply use either the functionaly of _numpy_ or that of _pandas_, as follows +!bc pycod +print(np.cov(X.T)) +print(df.cov()) +print(np.cov(X_centered.T)) +!ec +Depending on the number of points $n$, we will get results that are close to the covariance values defined above. === Diagonalize the sample covariance matrix to obtain the principal components === @@ -847,13 +853,13 @@ Compare the computed covariance with the answer given above. Now we are ready to solve for the principal components! To do so we diagonalize the sample covariance matrix $\Sigma_n$. We can use the function _np.linalg.eig_ to do so. It will return the eigenvalues and -eigenvectors of $\Sigma_n$. Once you have these, carry out the +eigenvectors of $\Sigma_n$. Once we have these we can perform the following tasks: -* Compute the percentage of the total variance captured by the first principal component -* Plot the mean centered data and lines along the first and second principal components -* Project the mean centered data onto the first and second principal components, and plot the projected data. What do you observe? -* Approximate the data as +* We compute the percentage of the total variance captured by the first principal component +* We plot the mean centered data and lines along the first and second principal components +* Then we project the mean centered data onto the first and second principal components, and plot the projected data. +* Finally, we approximate the data as !bt \begin{equation*} x_i \approx \tilde{x}_i := \mu_n + \langle x_i, v_0 \rangle v_0 @@ -861,14 +867,10 @@ x_i \approx \tilde{x}_i := \mu_n + \langle x_i, v_0 \rangle v_0 !et where $v_0$ is the first principal component. What do you observe? -Finally, collect all these steps and write your own PCA function and -compare this with the functionality included in _Scikit-Learn_. Have -the input be the data and have the output be the principal components -and their associated eigenvalues, sorted in descending order. Can you -think of a way to make it more efficient than the algorithm outlined -above? +Collecting all these steps we can write our own PCA function and +compare this with the functionality included in _Scikit-Learn_. -The code here outlines some of the elements you could include in this analysis. +The code here outlines some of the elements we could include in the analysis. Feel free to extend upon this. !bc pycod #Now we do an SVD U, s, V = np.linalg.svd(X_centered) @@ -890,12 +892,6 @@ print(pca.components_.T[:, 0]) !ec - -Finally, try out your own PCA function with other data sets. - - - - !split ===== Classical PCA Theorem ===== diff --git a/doc/src/DimRed/Programs/PCAsimple.py b/doc/src/DimRed/Programs/PCAsimple.py index bd168f6df..bfbccb78e 100644 --- a/doc/src/DimRed/Programs/PCAsimple.py +++ b/doc/src/DimRed/Programs/PCAsimple.py @@ -1,30 +1,35 @@ import numpy as np import pandas as pd from IPython.display import display +import matplotlib.pyplot as plt +from sklearn.model_selection import train_test_split +from sklearn.preprocessing import MinMaxScaler, StandardScaler -n = 100 +n = 10000 mean = (-1, 2) cov = [[4, 2], [2, 2]] +print(cov) X = np.random.multivariate_normal(mean, cov, n) -print(X) +print(np.cov(X.T)) df = pd.DataFrame(X) # Pandas does the centering for us df = df -df.mean() -display(df) +correlation_matrix = df.cov() +print(correlation_matrix) # we center it ourselves X_centered = X - X.mean(axis=0) -# test that we get the same as Pandas -print(X_centered-df) +print(np.cov(X_centered.T)) +#print("test that we get the same as Pandas") +#print(X_centered-X_train_scaled) #Now we do an SVD U, s, V = np.linalg.svd(X_centered) c1 = V.T[:, 0] c2 = V.T[:, 1] W2 = V.T[:, :2] X2D = X_centered.dot(W2) -print(X2D) #thereafter we do a PCA with Scikit-learn from sklearn.decomposition import PCA pca = PCA(n_components = 2) @@ -34,3 +39,5 @@ print(X2D-X2Dsl) print(pca.components_.T[:, 0]) + +