lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

more pca

Browse Source
This commit is contained in:
mhjensen
2020-01-02 10:45:16 +01:00
parent 31a7c6e760
commit ac368dd0c1
9 changed files with 177 additions and 109 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/pub/DimRed/html/._DimRed-bs018.html
+27 -14
View File
@@ -229,7 +229,7 @@ Note that the function <b>multivariate</b> returns also the covariance discussed
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">numpy</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">np</span>
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">pandas</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">pd</span>
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">IPython.display</span> <span style="color: #008000; font-weight: bold">import</span> display
n <span style="color: #666666">=</span> <span style="color: #666666">100</span>
n <span style="color: #666666">=</span> <span style="color: #666666">10000</span>
mean <span style="color: #666666">=</span> (<span style="color: #666666">-1</span>, <span style="color: #666666">2</span>)
cov <span style="color: #666666">=</span> [[<span style="color: #666666">4</span>, <span style="color: #666666">2</span>], [<span style="color: #666666">2</span>, <span style="color: #666666">2</span>]]
X <span style="color: #666666">=</span> np<span style="color: #666666">.</span>random<span style="color: #666666">.</span>multivariate_normal(mean, cov, n)
@@ -313,14 +313,15 @@ plt<span style="color: #666666">.</span>show()
</pre></div>
<p>
Depending on the number of points \( n \), we will get results that are close to the covariance values defined above.
The plot shows how the data are clustered around a line with slope close to one. Is this expected?
<h3 id="___sec20" class="anchor">Diagonalize the sample covariance matrix to obtain the principal components </h3>
<p>
Now we are ready to solve for the principal components! To do so we
diagonalize the sample covariance matrix \( \Sigma_n \). We can use the
diagonalize the sample covariance matrix \( \Sigma \). We can use the
function <b>np.linalg.eig</b> to do so. It will return the eigenvalues and
eigenvectors of \( \Sigma_n \). Once we have these we can perform the
eigenvectors of \( \Sigma \). Once we have these we can perform the
following tasks:
<ul>
@@ -343,26 +344,38 @@ Collecting all these steps we can write our own PCA function and
compare this with the functionality included in <b>Scikit-Learn</b>.
<p>
The code here outlines some of the elements we could include in the analysis. Feel free to extend upon this.
The code here outlines some of the elements we could include in the
analysis. Feel free to extend upon this in order to address the above
questions.
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #408080; font-style: italic">#Now we do an SVD</span>
U, s, V <span style="color: #666666">=</span> np<span style="color: #666666">.</span>linalg<span style="color: #666666">.</span>svd(X_centered)
c1 <span style="color: #666666">=</span> V<span style="color: #666666">.</span>T[:, <span style="color: #666666">0</span>]
c2 <span style="color: #666666">=</span> V<span style="color: #666666">.</span>T[:, <span style="color: #666666">1</span>]
W2 <span style="color: #666666">=</span> V<span style="color: #666666">.</span>T[:, :<span style="color: #666666">2</span>]
X2D <span style="color: #666666">=</span> X_centered<span style="color: #666666">.</span>dot(W2)
<span style="color: #008000; font-weight: bold">print</span>(X2D)
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #408080; font-style: italic"># diagonalize and obtain eigenvalues, not necessarily sorted</span>
EigValues, EigVectors <span style="color: #666666">=</span> np<span style="color: #666666">.</span>linalg<span style="color: #666666">.</span>eig(Cov)
<span style="color: #408080; font-style: italic"># sort eigenvectors and eigenvalues</span>
<span style="color: #408080; font-style: italic">#permute = EigValues.argsort()</span>
<span style="color: #408080; font-style: italic">#EigValues = EigValues[permute]</span>
<span style="color: #408080; font-style: italic">#EigVectors = EigVectors[:,permute]</span>
<span style="color: #008000; font-weight: bold">print</span>(<span style="color: #BA2121">&quot;Eigenvalues of Covariance matrix&quot;</span>)
<span style="color: #008000; font-weight: bold">for</span> i <span style="color: #AA22FF; font-weight: bold">in</span> <span style="color: #008000">range</span>(<span style="color: #666666">2</span>):
<span style="color: #008000; font-weight: bold">print</span>(EigValues[i])
FirstEigvector <span style="color: #666666">=</span> EigVectors[:,<span style="color: #666666">0</span>]
SecondEigvector <span style="color: #666666">=</span> EigVectors[:,<span style="color: #666666">1</span>]
<span style="color: #008000; font-weight: bold">print</span>(<span style="color: #BA2121">&quot;First eigenvector&quot;</span>)
<span style="color: #008000; font-weight: bold">print</span>(FirstEigvector)
<span style="color: #008000; font-weight: bold">print</span>(<span style="color: #BA2121">&quot;Second eigenvector&quot;</span>)
<span style="color: #008000; font-weight: bold">print</span>(SecondEigvector)
<span style="color: #408080; font-style: italic">#thereafter we do a PCA with Scikit-learn</span>
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.decomposition</span> <span style="color: #008000; font-weight: bold">import</span> PCA
pca <span style="color: #666666">=</span> PCA(n_components <span style="color: #666666">=</span> <span style="color: #666666">2</span>)
X2Dsl <span style="color: #666666">=</span> pca<span style="color: #666666">.</span>fit_transform(X)
<span style="color: #008000; font-weight: bold">print</span>(<span style="color: #BA2121">&quot;Check that we get the same&quot;</span>)
<span style="color: #008000; font-weight: bold">print</span>(X2D<span style="color: #666666">-</span>X2Dsl)
<span style="color: #008000; font-weight: bold">print</span>(<span style="color: #BA2121">&quot;Eigenvector of largest eigenvalue&quot;</span>)
<span style="color: #008000; font-weight: bold">print</span>(pca<span style="color: #666666">.</span>components_<span style="color: #666666">.</span>T[:, <span style="color: #666666">0</span>])
</pre></div>
<p>
This code does not contain all the above elements, but it shows how we can use <b>Scikit-Learn</b> to extract the eigenvector which corresponds to the largest eigenvalue. Try to, based on the above, to address the questions above.
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
doc/pub/DimRed/html/DimRed-reveal.html
+26 -14
View File
@@ -1035,7 +1035,7 @@ Note that the function <b>multivariate</b> returns also the covariance discussed
<div class="highlight" style="background: #eeeedd"><pre style="font-size: 80%; line-height: 125%"><span></span><span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">numpy</span> <span style="color: #8B008B; font-weight: bold">as</span> <span style="color: #008b45; text-decoration: underline">np</span>
<span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">pandas</span> <span style="color: #8B008B; font-weight: bold">as</span> <span style="color: #008b45; text-decoration: underline">pd</span>
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">IPython.display</span> <span style="color: #8B008B; font-weight: bold">import</span> display
n = <span style="color: #B452CD">100</span>
n = <span style="color: #B452CD">10000</span>
mean = (-<span style="color: #B452CD">1</span>, <span style="color: #B452CD">2</span>)
cov = [[<span style="color: #B452CD">4</span>, <span style="color: #B452CD">2</span>], [<span style="color: #B452CD">2</span>, <span style="color: #B452CD">2</span>]]
X = np.random.multivariate_normal(mean, cov, n)
@@ -1125,14 +1125,15 @@ plt.show()
</pre></div>
<p>
Depending on the number of points \( n \), we will get results that are close to the covariance values defined above.
The plot shows how the data are clustered around a line with slope close to one. Is this expected?
<h3 id="___sec20">Diagonalize the sample covariance matrix to obtain the principal components </h3>
<p>
Now we are ready to solve for the principal components! To do so we
diagonalize the sample covariance matrix \( \Sigma_n \). We can use the
diagonalize the sample covariance matrix \( \Sigma \). We can use the
function <b>np.linalg.eig</b> to do so. It will return the eigenvalues and
eigenvectors of \( \Sigma_n \). Once we have these we can perform the
eigenvectors of \( \Sigma \). Once we have these we can perform the
following tasks:
<ul>
@@ -1156,26 +1157,37 @@ Collecting all these steps we can write our own PCA function and
compare this with the functionality included in <b>Scikit-Learn</b>.
<p>
The code here outlines some of the elements we could include in the analysis. Feel free to extend upon this.
The code here outlines some of the elements we could include in the
analysis. Feel free to extend upon this in order to address the above
questions.
<p>
<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
<div class="highlight" style="background: #eeeedd"><pre style="font-size: 80%; line-height: 125%"><span></span><span style="color: #228B22">#Now we do an SVD</span>
U, s, V = np.linalg.svd(X_centered)
c1 = V.T[:, <span style="color: #B452CD">0</span>]
c2 = V.T[:, <span style="color: #B452CD">1</span>]
W2 = V.T[:, :<span style="color: #B452CD">2</span>]
X2D = X_centered.dot(W2)
<span style="color: #8B008B; font-weight: bold">print</span>(X2D)
<div class="highlight" style="background: #eeeedd"><pre style="font-size: 80%; line-height: 125%"><span></span><span style="color: #228B22"># diagonalize and obtain eigenvalues, not necessarily sorted</span>
EigValues, EigVectors = np.linalg.eig(Cov)
<span style="color: #228B22"># sort eigenvectors and eigenvalues</span>
<span style="color: #228B22">#permute = EigValues.argsort()</span>
<span style="color: #228B22">#EigValues = EigValues[permute]</span>
<span style="color: #228B22">#EigVectors = EigVectors[:,permute]</span>
<span style="color: #8B008B; font-weight: bold">print</span>(<span style="color: #CD5555">&quot;Eigenvalues of Covariance matrix&quot;</span>)
<span style="color: #8B008B; font-weight: bold">for</span> i <span style="color: #8B008B">in</span> <span style="color: #658b00">range</span>(<span style="color: #B452CD">2</span>):
<span style="color: #8B008B; font-weight: bold">print</span>(EigValues[i])
FirstEigvector = EigVectors[:,<span style="color: #B452CD">0</span>]
SecondEigvector = EigVectors[:,<span style="color: #B452CD">1</span>]
<span style="color: #8B008B; font-weight: bold">print</span>(<span style="color: #CD5555">&quot;First eigenvector&quot;</span>)
<span style="color: #8B008B; font-weight: bold">print</span>(FirstEigvector)
<span style="color: #8B008B; font-weight: bold">print</span>(<span style="color: #CD5555">&quot;Second eigenvector&quot;</span>)
<span style="color: #8B008B; font-weight: bold">print</span>(SecondEigvector)
<span style="color: #228B22">#thereafter we do a PCA with Scikit-learn</span>
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">sklearn.decomposition</span> <span style="color: #8B008B; font-weight: bold">import</span> PCA
pca = PCA(n_components = <span style="color: #B452CD">2</span>)
X2Dsl = pca.fit_transform(X)
<span style="color: #8B008B; font-weight: bold">print</span>(<span style="color: #CD5555">&quot;Check that we get the same&quot;</span>)
<span style="color: #8B008B; font-weight: bold">print</span>(X2D-X2Dsl)
<span style="color: #8B008B; font-weight: bold">print</span>(<span style="color: #CD5555">&quot;Eigenvector of largest eigenvalue&quot;</span>)
<span style="color: #8B008B; font-weight: bold">print</span>(pca.components_.T[:, <span style="color: #B452CD">0</span>])
</pre></div>
<p>
This code does not contain all the above elements, but it shows how we can use <b>Scikit-Learn</b> to extract the eigenvector which corresponds to the largest eigenvalue. Try to, based on the above, to address the questions above.
</section>
doc/pub/DimRed/html/DimRed-solarized.html
+27 -14
View File
@@ -1015,7 +1015,7 @@ Note that the function <b>multivariate</b> returns also the covariance discussed
<div class="highlight" style="background: #eeeedd"><pre style="line-height: 125%"><span></span><span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">numpy</span> <span style="color: #8B008B; font-weight: bold">as</span> <span style="color: #008b45; text-decoration: underline">np</span>
<span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">pandas</span> <span style="color: #8B008B; font-weight: bold">as</span> <span style="color: #008b45; text-decoration: underline">pd</span>
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">IPython.display</span> <span style="color: #8B008B; font-weight: bold">import</span> display
n = <span style="color: #B452CD">100</span>
n = <span style="color: #B452CD">10000</span>
mean = (-<span style="color: #B452CD">1</span>, <span style="color: #B452CD">2</span>)
cov = [[<span style="color: #B452CD">4</span>, <span style="color: #B452CD">2</span>], [<span style="color: #B452CD">2</span>, <span style="color: #B452CD">2</span>]]
X = np.random.multivariate_normal(mean, cov, n)
@@ -1099,14 +1099,15 @@ plt.show()
</pre></div>
<p>
Depending on the number of points \( n \), we will get results that are close to the covariance values defined above.
The plot shows how the data are clustered around a line with slope close to one. Is this expected?
<h3 id="___sec20">Diagonalize the sample covariance matrix to obtain the principal components </h3>
<p>
Now we are ready to solve for the principal components! To do so we
diagonalize the sample covariance matrix \( \Sigma_n \). We can use the
diagonalize the sample covariance matrix \( \Sigma \). We can use the
function <b>np.linalg.eig</b> to do so. It will return the eigenvalues and
eigenvectors of \( \Sigma_n \). Once we have these we can perform the
eigenvectors of \( \Sigma \). Once we have these we can perform the
following tasks:
<ul>
@@ -1129,26 +1130,38 @@ Collecting all these steps we can write our own PCA function and
compare this with the functionality included in <b>Scikit-Learn</b>.
<p>
The code here outlines some of the elements we could include in the analysis. Feel free to extend upon this.
The code here outlines some of the elements we could include in the
analysis. Feel free to extend upon this in order to address the above
questions.
<p>
<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
<div class="highlight" style="background: #eeeedd"><pre style="line-height: 125%"><span></span><span style="color: #228B22">#Now we do an SVD</span>
U, s, V = np.linalg.svd(X_centered)
c1 = V.T[:, <span style="color: #B452CD">0</span>]
c2 = V.T[:, <span style="color: #B452CD">1</span>]
W2 = V.T[:, :<span style="color: #B452CD">2</span>]
X2D = X_centered.dot(W2)
<span style="color: #8B008B; font-weight: bold">print</span>(X2D)
<div class="highlight" style="background: #eeeedd"><pre style="line-height: 125%"><span></span><span style="color: #228B22"># diagonalize and obtain eigenvalues, not necessarily sorted</span>
EigValues, EigVectors = np.linalg.eig(Cov)
<span style="color: #228B22"># sort eigenvectors and eigenvalues</span>
<span style="color: #228B22">#permute = EigValues.argsort()</span>
<span style="color: #228B22">#EigValues = EigValues[permute]</span>
<span style="color: #228B22">#EigVectors = EigVectors[:,permute]</span>
<span style="color: #8B008B; font-weight: bold">print</span>(<span style="color: #CD5555">&quot;Eigenvalues of Covariance matrix&quot;</span>)
<span style="color: #8B008B; font-weight: bold">for</span> i <span style="color: #8B008B">in</span> <span style="color: #658b00">range</span>(<span style="color: #B452CD">2</span>):
<span style="color: #8B008B; font-weight: bold">print</span>(EigValues[i])
FirstEigvector = EigVectors[:,<span style="color: #B452CD">0</span>]
SecondEigvector = EigVectors[:,<span style="color: #B452CD">1</span>]
<span style="color: #8B008B; font-weight: bold">print</span>(<span style="color: #CD5555">&quot;First eigenvector&quot;</span>)
<span style="color: #8B008B; font-weight: bold">print</span>(FirstEigvector)
<span style="color: #8B008B; font-weight: bold">print</span>(<span style="color: #CD5555">&quot;Second eigenvector&quot;</span>)
<span style="color: #8B008B; font-weight: bold">print</span>(SecondEigvector)
<span style="color: #228B22">#thereafter we do a PCA with Scikit-learn</span>
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">sklearn.decomposition</span> <span style="color: #8B008B; font-weight: bold">import</span> PCA
pca = PCA(n_components = <span style="color: #B452CD">2</span>)
X2Dsl = pca.fit_transform(X)
<span style="color: #8B008B; font-weight: bold">print</span>(<span style="color: #CD5555">&quot;Check that we get the same&quot;</span>)
<span style="color: #8B008B; font-weight: bold">print</span>(X2D-X2Dsl)
<span style="color: #8B008B; font-weight: bold">print</span>(<span style="color: #CD5555">&quot;Eigenvector of largest eigenvalue&quot;</span>)
<span style="color: #8B008B; font-weight: bold">print</span>(pca.components_.T[:, <span style="color: #B452CD">0</span>])
</pre></div>
<p>
This code does not contain all the above elements, but it shows how we can use <b>Scikit-Learn</b> to extract the eigenvector which corresponds to the largest eigenvalue. Try to, based on the above, to address the questions above.
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
doc/pub/DimRed/html/DimRed.html
+27 -14
View File
@@ -1020,7 +1020,7 @@ Note that the function <b>multivariate</b> returns also the covariance discussed
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">numpy</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">np</span>
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">pandas</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">pd</span>
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">IPython.display</span> <span style="color: #008000; font-weight: bold">import</span> display
n <span style="color: #666666">=</span> <span style="color: #666666">100</span>
n <span style="color: #666666">=</span> <span style="color: #666666">10000</span>
mean <span style="color: #666666">=</span> (<span style="color: #666666">-1</span>, <span style="color: #666666">2</span>)
cov <span style="color: #666666">=</span> [[<span style="color: #666666">4</span>, <span style="color: #666666">2</span>], [<span style="color: #666666">2</span>, <span style="color: #666666">2</span>]]
X <span style="color: #666666">=</span> np<span style="color: #666666">.</span>random<span style="color: #666666">.</span>multivariate_normal(mean, cov, n)
@@ -1104,14 +1104,15 @@ plt<span style="color: #666666">.</span>show()
</pre></div>
<p>
Depending on the number of points \( n \), we will get results that are close to the covariance values defined above.
The plot shows how the data are clustered around a line with slope close to one. Is this expected?
<h3 id="___sec20">Diagonalize the sample covariance matrix to obtain the principal components </h3>
<p>
Now we are ready to solve for the principal components! To do so we
diagonalize the sample covariance matrix \( \Sigma_n \). We can use the
diagonalize the sample covariance matrix \( \Sigma \). We can use the
function <b>np.linalg.eig</b> to do so. It will return the eigenvalues and
eigenvectors of \( \Sigma_n \). Once we have these we can perform the
eigenvectors of \( \Sigma \). Once we have these we can perform the
following tasks:
<ul>
@@ -1134,26 +1135,38 @@ Collecting all these steps we can write our own PCA function and
compare this with the functionality included in <b>Scikit-Learn</b>.
<p>
The code here outlines some of the elements we could include in the analysis. Feel free to extend upon this.
The code here outlines some of the elements we could include in the
analysis. Feel free to extend upon this in order to address the above
questions.
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #408080; font-style: italic">#Now we do an SVD</span>
U, s, V <span style="color: #666666">=</span> np<span style="color: #666666">.</span>linalg<span style="color: #666666">.</span>svd(X_centered)
c1 <span style="color: #666666">=</span> V<span style="color: #666666">.</span>T[:, <span style="color: #666666">0</span>]
c2 <span style="color: #666666">=</span> V<span style="color: #666666">.</span>T[:, <span style="color: #666666">1</span>]
W2 <span style="color: #666666">=</span> V<span style="color: #666666">.</span>T[:, :<span style="color: #666666">2</span>]
X2D <span style="color: #666666">=</span> X_centered<span style="color: #666666">.</span>dot(W2)
<span style="color: #008000; font-weight: bold">print</span>(X2D)
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #408080; font-style: italic"># diagonalize and obtain eigenvalues, not necessarily sorted</span>
EigValues, EigVectors <span style="color: #666666">=</span> np<span style="color: #666666">.</span>linalg<span style="color: #666666">.</span>eig(Cov)
<span style="color: #408080; font-style: italic"># sort eigenvectors and eigenvalues</span>
<span style="color: #408080; font-style: italic">#permute = EigValues.argsort()</span>
<span style="color: #408080; font-style: italic">#EigValues = EigValues[permute]</span>
<span style="color: #408080; font-style: italic">#EigVectors = EigVectors[:,permute]</span>
<span style="color: #008000; font-weight: bold">print</span>(<span style="color: #BA2121">&quot;Eigenvalues of Covariance matrix&quot;</span>)
<span style="color: #008000; font-weight: bold">for</span> i <span style="color: #AA22FF; font-weight: bold">in</span> <span style="color: #008000">range</span>(<span style="color: #666666">2</span>):
<span style="color: #008000; font-weight: bold">print</span>(EigValues[i])
FirstEigvector <span style="color: #666666">=</span> EigVectors[:,<span style="color: #666666">0</span>]
SecondEigvector <span style="color: #666666">=</span> EigVectors[:,<span style="color: #666666">1</span>]
<span style="color: #008000; font-weight: bold">print</span>(<span style="color: #BA2121">&quot;First eigenvector&quot;</span>)
<span style="color: #008000; font-weight: bold">print</span>(FirstEigvector)
<span style="color: #008000; font-weight: bold">print</span>(<span style="color: #BA2121">&quot;Second eigenvector&quot;</span>)
<span style="color: #008000; font-weight: bold">print</span>(SecondEigvector)
<span style="color: #408080; font-style: italic">#thereafter we do a PCA with Scikit-learn</span>
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.decomposition</span> <span style="color: #008000; font-weight: bold">import</span> PCA
pca <span style="color: #666666">=</span> PCA(n_components <span style="color: #666666">=</span> <span style="color: #666666">2</span>)
X2Dsl <span style="color: #666666">=</span> pca<span style="color: #666666">.</span>fit_transform(X)
<span style="color: #008000; font-weight: bold">print</span>(<span style="color: #BA2121">&quot;Check that we get the same&quot;</span>)
<span style="color: #008000; font-weight: bold">print</span>(X2D<span style="color: #666666">-</span>X2Dsl)
<span style="color: #008000; font-weight: bold">print</span>(<span style="color: #BA2121">&quot;Eigenvector of largest eigenvalue&quot;</span>)
<span style="color: #008000; font-weight: bold">print</span>(pca<span style="color: #666666">.</span>components_<span style="color: #666666">.</span>T[:, <span style="color: #666666">0</span>])
</pre></div>
<p>
This code does not contain all the above elements, but it shows how we can use <b>Scikit-Learn</b> to extract the eigenvector which corresponds to the largest eigenvalue. Try to, based on the above, to address the questions above.
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
doc/pub/DimRed/ipynb/DimRed.ipynb
+25 -15
View File
@@ -1108,7 +1108,7 @@
"import numpy as np\n",
"import pandas as pd\n",
"from IPython.display import display\n",
"n = 100\n",
"n = 10000\n",
"mean = (-1, 2)\n",
"cov = [[4, 2], [2, 2]]\n",
"X = np.random.multivariate_normal(mean, cov, n)"
@@ -1258,14 +1258,14 @@
"metadata": {},
"source": [
"Depending on the number of points $n$, we will get results that are close to the covariance values defined above.\n",
"\n",
"The plot shows how the data are clustered around a line with slope close to one. Is this expected?\n",
"\n",
"### Diagonalize the sample covariance matrix to obtain the principal components\n",
"\n",
"Now we are ready to solve for the principal components! To do so we\n",
"diagonalize the sample covariance matrix $\\Sigma_n$. We can use the\n",
"diagonalize the sample covariance matrix $\\Sigma$. We can use the\n",
"function **np.linalg.eig** to do so. It will return the eigenvalues and\n",
"eigenvectors of $\\Sigma_n$. Once we have these we can perform the \n",
"eigenvectors of $\\Sigma$. Once we have these we can perform the \n",
"following tasks:\n",
"\n",
"* We compute the percentage of the total variance captured by the first principal component\n",
@@ -1295,7 +1295,9 @@
"Collecting all these steps we can write our own PCA function and\n",
"compare this with the functionality included in **Scikit-Learn**. \n",
"\n",
"The code here outlines some of the elements we could include in the analysis. Feel free to extend upon this."
"The code here outlines some of the elements we could include in the\n",
"analysis. Feel free to extend upon this in order to address the above\n",
"questions."
]
},
{
@@ -1306,20 +1308,26 @@
},
"outputs": [],
"source": [
"#Now we do an SVD\n",
"U, s, V = np.linalg.svd(X_centered)\n",
"c1 = V.T[:, 0]\n",
"c2 = V.T[:, 1]\n",
"W2 = V.T[:, :2]\n",
"X2D = X_centered.dot(W2)\n",
"print(X2D)\n",
"# diagonalize and obtain eigenvalues, not necessarily sorted\n",
"EigValues, EigVectors = np.linalg.eig(Cov)\n",
"# sort eigenvectors and eigenvalues\n",
"#permute = EigValues.argsort()\n",
"#EigValues = EigValues[permute]\n",
"#EigVectors = EigVectors[:,permute]\n",
"print(\"Eigenvalues of Covariance matrix\")\n",
"for i in range(2):\n",
" print(EigValues[i])\n",
"FirstEigvector = EigVectors[:,0]\n",
"SecondEigvector = EigVectors[:,1]\n",
"print(\"First eigenvector\")\n",
"print(FirstEigvector)\n",
"print(\"Second eigenvector\")\n",
"print(SecondEigvector)\n",
"#thereafter we do a PCA with Scikit-learn\n",
"from sklearn.decomposition import PCA\n",
"pca = PCA(n_components = 2)\n",
"X2Dsl = pca.fit_transform(X)\n",
"print(\"Check that we get the same\")\n",
"print(X2D-X2Dsl)\n",
"\n",
"print(\"Eigenvector of largest eigenvalue\")\n",
"print(pca.components_.T[:, 0])"
]
},
@@ -1327,6 +1335,8 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"This code does not contain all the above elements, but it shows how we can use **Scikit-Learn** to extract the eigenvector which corresponds to the largest eigenvalue. Try to, based on the above, to address the questions above. \n",
"\n",
"## Classical PCA Theorem\n",
"\n",
"We assume now that we have a design matrix $\\boldsymbol{X}$ which has been\n",
doc/pub/DimRed/ipynb/ipynb-DimRed-src.tar.gz
BIN
View File
Binary file not shown.
doc/pub/DimRed/pdf/DimRed-minted.pdf
BIN
View File
Binary file not shown.
doc/src/DimRed/DimRed.do.txt
+25 -17
View File
@@ -782,7 +782,7 @@ Note that the function _multivariate_ returns also the covariance discussed abov
import numpy as np
import pandas as pd
from IPython.display import display
n = 100
n = 10000
mean = (-1, 2)
cov = [[4, 2], [2, 2]]
X = np.random.multivariate_normal(mean, cov, n)
@@ -858,14 +858,14 @@ plt.show()
!ec
Depending on the number of points $n$, we will get results that are close to the covariance values defined above.
The plot shows how the data are clustered around a line with slope close to one. Is this expected?
=== Diagonalize the sample covariance matrix to obtain the principal components ===
Now we are ready to solve for the principal components! To do so we
diagonalize the sample covariance matrix $\Sigma_n$. We can use the
diagonalize the sample covariance matrix $\Sigma$. We can use the
function _np.linalg.eig_ to do so. It will return the eigenvalues and
eigenvectors of $\Sigma_n$. Once we have these we can perform the
eigenvectors of $\Sigma$. Once we have these we can perform the
following tasks:
* We compute the percentage of the total variance captured by the first principal component
@@ -883,27 +883,35 @@ where $v_0$ is the first principal component.
Collecting all these steps we can write our own PCA function and
compare this with the functionality included in _Scikit-Learn_.
The code here outlines some of the elements we could include in the analysis. Feel free to extend upon this.
The code here outlines some of the elements we could include in the
analysis. Feel free to extend upon this in order to address the above
questions.
!bc pycod
#Now we do an SVD
U, s, V = np.linalg.svd(X_centered)
c1 = V.T[:, 0]
c2 = V.T[:, 1]
W2 = V.T[:, :2]
X2D = X_centered.dot(W2)
print(X2D)
# diagonalize and obtain eigenvalues, not necessarily sorted
EigValues, EigVectors = np.linalg.eig(Cov)
# sort eigenvectors and eigenvalues
#permute = EigValues.argsort()
#EigValues = EigValues[permute]
#EigVectors = EigVectors[:,permute]
print("Eigenvalues of Covariance matrix")
for i in range(2):
print(EigValues[i])
FirstEigvector = EigVectors[:,0]
SecondEigvector = EigVectors[:,1]
print("First eigenvector")
print(FirstEigvector)
print("Second eigenvector")
print(SecondEigvector)
#thereafter we do a PCA with Scikit-learn
from sklearn.decomposition import PCA
pca = PCA(n_components = 2)
X2Dsl = pca.fit_transform(X)
print("Check that we get the same")
print(X2D-X2Dsl)
print("Eigenvector of largest eigenvalue")
print(pca.components_.T[:, 0])
!ec
This code does not contain all the above elements, but it shows how we can use _Scikit-Learn_ to extract the eigenvector which corresponds to the largest eigenvalue. Try to, based on the above, to address the questions above.
!split
===== Classical PCA Theorem =====
doc/src/DimRed/Programs/PCAsimple.py
+20 -21
View File
@@ -24,13 +24,9 @@ print(np.cov(X_centered.T))
x = X_centered[:,[0]]
y = X_centered[:,[1]]
Cov = np.zeros((2,2))
cov_xy = np.sum(x.T@y)/(n-1.0)
cov_xx = np.sum(x.T@x)/(n-1.0)
cov_yy = np.sum(y.T@y)/(n-1.0)
Cov[0,0]= cov_xx
Cov[1,1]= cov_yy
Cov[0,1]= cov_xy
Cov[0,1] = np.sum(x.T@y)/(n-1.0)
Cov[0,0] = np.sum(x.T@x)/(n-1.0)
Cov[1,1] = np.sum(y.T@y)/(n-1.0)
Cov[1,0]= Cov[0,1]
print("Centered covariance using own code")
print(Cov)
@@ -39,24 +35,27 @@ plt.plot(x, y, 'x')
plt.axis('equal')
plt.show()
"""
#Now we do an SVD
U, s, V = np.linalg.svd(X_centered)
c1 = V.T[:, 0]
c2 = V.T[:, 1]
W2 = V.T[:, :2]
X2D = X_centered.dot(W2)
# diagonalize and obtain eigenvalues, not necessarily sorted
EigValues, EigVectors = np.linalg.eig(Cov)
# sort eigenvectors and eigenvalues
#permute = EigValues.argsort()
#EigValues = EigValues[permute]
#EigVectors = EigVectors[:,permute]
print("Eigenvalues of Covariance matrix")
for i in range(2):
print(EigValues[i])
FirstEigvector = EigVectors[:,0]
SecondEigvector = EigVectors[:,1]
print("First eigenvector")
print(FirstEigvector)
print("Second eigenvector")
print(SecondEigvector)
#thereafter we do a PCA with Scikit-learn
from sklearn.decomposition import PCA
pca = PCA(n_components = 2)
X2Dsl = pca.fit_transform(X)
print("Check that we get the same")
print(X2D-X2Dsl)
print("Eigenvector of largest eigenvalue")
print(pca.components_.T[:, 0])
"""