lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

more pca

Browse Source
This commit is contained in:
mhjensen
2020-01-02 10:45:16 +01:00
parent 31a7c6e760
commit ac368dd0c1
9 changed files with 177 additions and 109 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/pub/DimRed/html/DimRed.html
+27 -14
View File
@@ -1020,7 +1020,7 @@ Note that the function <b>multivariate</b> returns also the covariance discussed
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">numpy</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">np</span>
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">pandas</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">pd</span>
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">IPython.display</span> <span style="color: #008000; font-weight: bold">import</span> display
n <span style="color: #666666">=</span> <span style="color: #666666">100</span>
n <span style="color: #666666">=</span> <span style="color: #666666">10000</span>
mean <span style="color: #666666">=</span> (<span style="color: #666666">-1</span>, <span style="color: #666666">2</span>)
cov <span style="color: #666666">=</span> [[<span style="color: #666666">4</span>, <span style="color: #666666">2</span>], [<span style="color: #666666">2</span>, <span style="color: #666666">2</span>]]
X <span style="color: #666666">=</span> np<span style="color: #666666">.</span>random<span style="color: #666666">.</span>multivariate_normal(mean, cov, n)
@@ -1104,14 +1104,15 @@ plt<span style="color: #666666">.</span>show()
</pre></div>
<p>
Depending on the number of points \( n \), we will get results that are close to the covariance values defined above.
The plot shows how the data are clustered around a line with slope close to one. Is this expected?
<h3 id="___sec20">Diagonalize the sample covariance matrix to obtain the principal components </h3>
<p>
Now we are ready to solve for the principal components! To do so we
diagonalize the sample covariance matrix \( \Sigma_n \). We can use the
diagonalize the sample covariance matrix \( \Sigma \). We can use the
function <b>np.linalg.eig</b> to do so. It will return the eigenvalues and
eigenvectors of \( \Sigma_n \). Once we have these we can perform the
eigenvectors of \( \Sigma \). Once we have these we can perform the
following tasks:
<ul>
@@ -1134,26 +1135,38 @@ Collecting all these steps we can write our own PCA function and
compare this with the functionality included in <b>Scikit-Learn</b>.
<p>
The code here outlines some of the elements we could include in the analysis. Feel free to extend upon this.
The code here outlines some of the elements we could include in the
analysis. Feel free to extend upon this in order to address the above
questions.
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #408080; font-style: italic">#Now we do an SVD</span>
U, s, V <span style="color: #666666">=</span> np<span style="color: #666666">.</span>linalg<span style="color: #666666">.</span>svd(X_centered)
c1 <span style="color: #666666">=</span> V<span style="color: #666666">.</span>T[:, <span style="color: #666666">0</span>]
c2 <span style="color: #666666">=</span> V<span style="color: #666666">.</span>T[:, <span style="color: #666666">1</span>]
W2 <span style="color: #666666">=</span> V<span style="color: #666666">.</span>T[:, :<span style="color: #666666">2</span>]
X2D <span style="color: #666666">=</span> X_centered<span style="color: #666666">.</span>dot(W2)
<span style="color: #008000; font-weight: bold">print</span>(X2D)
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #408080; font-style: italic"># diagonalize and obtain eigenvalues, not necessarily sorted</span>
EigValues, EigVectors <span style="color: #666666">=</span> np<span style="color: #666666">.</span>linalg<span style="color: #666666">.</span>eig(Cov)
<span style="color: #408080; font-style: italic"># sort eigenvectors and eigenvalues</span>
<span style="color: #408080; font-style: italic">#permute = EigValues.argsort()</span>
<span style="color: #408080; font-style: italic">#EigValues = EigValues[permute]</span>
<span style="color: #408080; font-style: italic">#EigVectors = EigVectors[:,permute]</span>
<span style="color: #008000; font-weight: bold">print</span>(<span style="color: #BA2121">&quot;Eigenvalues of Covariance matrix&quot;</span>)
<span style="color: #008000; font-weight: bold">for</span> i <span style="color: #AA22FF; font-weight: bold">in</span> <span style="color: #008000">range</span>(<span style="color: #666666">2</span>):
<span style="color: #008000; font-weight: bold">print</span>(EigValues[i])
FirstEigvector <span style="color: #666666">=</span> EigVectors[:,<span style="color: #666666">0</span>]
SecondEigvector <span style="color: #666666">=</span> EigVectors[:,<span style="color: #666666">1</span>]
<span style="color: #008000; font-weight: bold">print</span>(<span style="color: #BA2121">&quot;First eigenvector&quot;</span>)
<span style="color: #008000; font-weight: bold">print</span>(FirstEigvector)
<span style="color: #008000; font-weight: bold">print</span>(<span style="color: #BA2121">&quot;Second eigenvector&quot;</span>)
<span style="color: #008000; font-weight: bold">print</span>(SecondEigvector)
<span style="color: #408080; font-style: italic">#thereafter we do a PCA with Scikit-learn</span>
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.decomposition</span> <span style="color: #008000; font-weight: bold">import</span> PCA
pca <span style="color: #666666">=</span> PCA(n_components <span style="color: #666666">=</span> <span style="color: #666666">2</span>)
X2Dsl <span style="color: #666666">=</span> pca<span style="color: #666666">.</span>fit_transform(X)
<span style="color: #008000; font-weight: bold">print</span>(<span style="color: #BA2121">&quot;Check that we get the same&quot;</span>)
<span style="color: #008000; font-weight: bold">print</span>(X2D<span style="color: #666666">-</span>X2Dsl)
<span style="color: #008000; font-weight: bold">print</span>(<span style="color: #BA2121">&quot;Eigenvector of largest eigenvalue&quot;</span>)
<span style="color: #008000; font-weight: bold">print</span>(pca<span style="color: #666666">.</span>components_<span style="color: #666666">.</span>T[:, <span style="color: #666666">0</span>])
</pre></div>
<p>
This code does not contain all the above elements, but it shows how we can use <b>Scikit-Learn</b> to extract the eigenvector which corresponds to the largest eigenvalue. Try to, based on the above, to address the questions above.
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>