added code to pca

2019-12-30 15:13:28 +01:00
parent c55d506848
commit 994e26c835
10 changed files with 284 additions and 30 deletions
@@ -224,7 +224,7 @@ MathJax.Hub.Config({
 <center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
 <br>
 <p>
-<center><h4>Dec 29, 2019</h4></center> <!-- date -->
+<center><h4>Dec 30, 2019</h4></center> <!-- date -->
 <br>
 <p>

@@ -226,13 +226,18 @@ The following Python code aids in setting up the data
 <p>

 <!-- code=python (!bc pycod) typeset with pygments style "default" -->
-<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span>n <span style="color: #666666">=</span> <span style="color: #666666">1000</span>
+<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">numpy</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">np</span>
+<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">pandas</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">pd</span>
+<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">IPython.display</span> <span style="color: #008000; font-weight: bold">import</span> display
+n <span style="color: #666666">=</span> <span style="color: #666666">100</span>
 mean <span style="color: #666666">=</span> (<span style="color: #666666">-1</span>, <span style="color: #666666">2</span>)
 cov <span style="color: #666666">=</span> [[<span style="color: #666666">4</span>, <span style="color: #666666">2</span>], [<span style="color: #666666">2</span>, <span style="color: #666666">2</span>]]
 X <span style="color: #666666">=</span> np<span style="color: #666666">.</span>random<span style="color: #666666">.</span>multivariate_normal(mean, cov, n)
+<span style="color: #408080; font-style: italic"># Print the X-matrix</span>
+<span style="color: #008000; font-weight: bold">print</span>(X)
 </pre></div>
 <p>
-Make thereafter a small Python code which plots the data. Note that the function <b>multivariate</b> returns also the covariance discussed above and that it is defined by dividing by \( n-1 \) instead of \( n \).
+Make thereafter a small Python code which writes out the data. Note that the function <b>multivariate</b> returns also the covariance discussed above and that it is defined by dividing by \( n-1 \) instead of \( n \).

 <p>
 Now we are going to implement the PCA algorithm. We will break it down into various substeps.
@@ -254,6 +259,20 @@ When you are done with these steps, print out \( \mu_n \) to verify it is
 close to \( \mu \) and plot your mean centered data to verify it is
 centered at the origin! Compare your code with the functionality from <b>Scikit-Learn</b> discussed above.

+<p>
+
+<!-- code=python (!bc pycod) typeset with pygments style "default" -->
+<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span>df <span style="color: #666666">=</span> pd<span style="color: #666666">.</span>DataFrame(X)
+<span style="color: #408080; font-style: italic"># Pandas does the centering for us</span>
+df <span style="color: #666666">=</span> df <span style="color: #666666">-</span>df<span style="color: #666666">.</span>mean()
+display(df)
+
+<span style="color: #408080; font-style: italic"># we center it ourselves</span>
+X_centered <span style="color: #666666">=</span> X <span style="color: #666666">-</span> X<span style="color: #666666">.</span>mean(axis<span style="color: #666666">=0</span>)
+<span style="color: #408080; font-style: italic"># test that we get the same as Pandas</span>
+<span style="color: #008000; font-weight: bold">print</span>(X_centered<span style="color: #666666">-</span>df)
+</pre></div>
+
 <h3 id="___sec19" class="anchor">Compute the sample covariance </h3>

 <p>
@@ -296,6 +315,25 @@ Finally, collect all these steps and write your own PCA function and
 compare this with the functionality included in <b>Scikit-Learn</b>.
 Have the input be the data and have the output be the principal components and their associated eigenvalues, sorted in descending order. Can you think of a way to make it more efficient than the algorithm outlined above?

+<p>
+
+<!-- code=python (!bc pycod) typeset with pygments style "default" -->
+<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #408080; font-style: italic">#Now we do an SVD</span>
+U, s, V <span style="color: #666666">=</span> np<span style="color: #666666">.</span>linalg<span style="color: #666666">.</span>svd(X_centered)
+c1 <span style="color: #666666">=</span> V<span style="color: #666666">.</span>T[:, <span style="color: #666666">0</span>]
+c2 <span style="color: #666666">=</span> V<span style="color: #666666">.</span>T[:, <span style="color: #666666">1</span>]
+W2 <span style="color: #666666">=</span> V<span style="color: #666666">.</span>T[:, :<span style="color: #666666">2</span>]
+X2D <span style="color: #666666">=</span> X_centered<span style="color: #666666">.</span>dot(W2)
+<span style="color: #008000; font-weight: bold">print</span>(X2D)
+<span style="color: #408080; font-style: italic">#thereafter we do a PCA with Scikit-learn</span>
+<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.decomposition</span> <span style="color: #008000; font-weight: bold">import</span> PCA
+pca <span style="color: #666666">=</span> PCA(n_components <span style="color: #666666">=</span> <span style="color: #666666">2</span>)
+X2Dsl <span style="color: #666666">=</span> pca<span style="color: #666666">.</span>fit_transform(X)
+<span style="color: #008000; font-weight: bold">print</span>(<span style="color: #BA2121">&quot;Check that we get the same&quot;</span>)
+<span style="color: #008000; font-weight: bold">print</span>(X2D<span style="color: #666666">-</span>X2Dsl)
+
+<span style="color: #008000; font-weight: bold">print</span>(pca<span style="color: #666666">.</span>components_<span style="color: #666666">.</span>T[:, <span style="color: #666666">0</span>])
+</pre></div>
 <p>
 Finally, try out your own PCA function with other data sets.

@@ -224,7 +224,7 @@ MathJax.Hub.Config({
 <center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
 <br>
 <p>
-<center><h4>Dec 29, 2019</h4></center> <!-- date -->
+<center><h4>Dec 30, 2019</h4></center> <!-- date -->
 <br>
 <p>

@@ -148,7 +148,7 @@ MathJax.Hub.Config({
 <center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
 <br>
 <p>&nbsp;<br>
-<center><h4>Dec 29, 2019</h4></center> <!-- date -->
+<center><h4>Dec 30, 2019</h4></center> <!-- date -->
 <br>
 <p>

@@ -1032,13 +1032,18 @@ The following Python code aids in setting up the data
 <p>

 <!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
-<div class="highlight" style="background: #eeeedd"><pre style="font-size: 80%; line-height: 125%"><span></span>n = <span style="color: #B452CD">1000</span>
+<div class="highlight" style="background: #eeeedd"><pre style="font-size: 80%; line-height: 125%"><span></span><span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">numpy</span> <span style="color: #8B008B; font-weight: bold">as</span> <span style="color: #008b45; text-decoration: underline">np</span>
+<span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">pandas</span> <span style="color: #8B008B; font-weight: bold">as</span> <span style="color: #008b45; text-decoration: underline">pd</span>
+<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">IPython.display</span> <span style="color: #8B008B; font-weight: bold">import</span> display
+n = <span style="color: #B452CD">100</span>
 mean = (-<span style="color: #B452CD">1</span>, <span style="color: #B452CD">2</span>)
 cov = [[<span style="color: #B452CD">4</span>, <span style="color: #B452CD">2</span>], [<span style="color: #B452CD">2</span>, <span style="color: #B452CD">2</span>]]
 X = np.random.multivariate_normal(mean, cov, n)
+<span style="color: #228B22"># Print the X-matrix</span>
+<span style="color: #8B008B; font-weight: bold">print</span>(X)
 </pre></div>
 <p>
-Make thereafter a small Python code which plots the data. Note that the function <b>multivariate</b> returns also the covariance discussed above and that it is defined by dividing by \( n-1 \) instead of \( n \).
+Make thereafter a small Python code which writes out the data. Note that the function <b>multivariate</b> returns also the covariance discussed above and that it is defined by dividing by \( n-1 \) instead of \( n \).

 <p>
 Now we are going to implement the PCA algorithm. We will break it down into various substeps.
@@ -1064,6 +1069,20 @@ When you are done with these steps, print out \( \mu_n \) to verify it is
 close to \( \mu \) and plot your mean centered data to verify it is
 centered at the origin! Compare your code with the functionality from <b>Scikit-Learn</b> discussed above.

+<p>
+
+<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
+<div class="highlight" style="background: #eeeedd"><pre style="font-size: 80%; line-height: 125%"><span></span>df = pd.DataFrame(X)
+<span style="color: #228B22"># Pandas does the centering for us</span>
+df = df -df.mean()
+display(df)
+
+<span style="color: #228B22"># we center it ourselves</span>
+X_centered = X - X.mean(axis=<span style="color: #B452CD">0</span>)
+<span style="color: #228B22"># test that we get the same as Pandas</span>
+<span style="color: #8B008B; font-weight: bold">print</span>(X_centered-df)
+</pre></div>
+
 <h3 id="___sec19">Compute the sample covariance </h3>

 <p>
@@ -1109,6 +1128,25 @@ Finally, collect all these steps and write your own PCA function and
 compare this with the functionality included in <b>Scikit-Learn</b>.
 Have the input be the data and have the output be the principal components and their associated eigenvalues, sorted in descending order. Can you think of a way to make it more efficient than the algorithm outlined above?

+<p>
+
+<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
+<div class="highlight" style="background: #eeeedd"><pre style="font-size: 80%; line-height: 125%"><span></span><span style="color: #228B22">#Now we do an SVD</span>
+U, s, V = np.linalg.svd(X_centered)
+c1 = V.T[:, <span style="color: #B452CD">0</span>]
+c2 = V.T[:, <span style="color: #B452CD">1</span>]
+W2 = V.T[:, :<span style="color: #B452CD">2</span>]
+X2D = X_centered.dot(W2)
+<span style="color: #8B008B; font-weight: bold">print</span>(X2D)
+<span style="color: #228B22">#thereafter we do a PCA with Scikit-learn</span>
+<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">sklearn.decomposition</span> <span style="color: #8B008B; font-weight: bold">import</span> PCA
+pca = PCA(n_components = <span style="color: #B452CD">2</span>)
+X2Dsl = pca.fit_transform(X)
+<span style="color: #8B008B; font-weight: bold">print</span>(<span style="color: #CD5555">&quot;Check that we get the same&quot;</span>)
+<span style="color: #8B008B; font-weight: bold">print</span>(X2D-X2Dsl)
+
+<span style="color: #8B008B; font-weight: bold">print</span>(pca.components_.T[:, <span style="color: #B452CD">0</span>])
+</pre></div>
 <p>
 Finally, try out your own PCA function with other data sets.
 </section>
@@ -179,7 +179,7 @@ MathJax.Hub.Config({
 <center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
 <br>
 <p>
-<center><h4>Dec 29, 2019</h4></center> <!-- date -->
+<center><h4>Dec 30, 2019</h4></center> <!-- date -->
 <br>
 <p>
 <!-- !split --><br><br><br><br><br><br><br><br><br><br>
@@ -1012,13 +1012,18 @@ The following Python code aids in setting up the data
 <p>

 <!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
-<div class="highlight" style="background: #eeeedd"><pre style="line-height: 125%"><span></span>n = <span style="color: #B452CD">1000</span>
+<div class="highlight" style="background: #eeeedd"><pre style="line-height: 125%"><span></span><span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">numpy</span> <span style="color: #8B008B; font-weight: bold">as</span> <span style="color: #008b45; text-decoration: underline">np</span>
+<span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">pandas</span> <span style="color: #8B008B; font-weight: bold">as</span> <span style="color: #008b45; text-decoration: underline">pd</span>
+<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">IPython.display</span> <span style="color: #8B008B; font-weight: bold">import</span> display
+n = <span style="color: #B452CD">100</span>
 mean = (-<span style="color: #B452CD">1</span>, <span style="color: #B452CD">2</span>)
 cov = [[<span style="color: #B452CD">4</span>, <span style="color: #B452CD">2</span>], [<span style="color: #B452CD">2</span>, <span style="color: #B452CD">2</span>]]
 X = np.random.multivariate_normal(mean, cov, n)
+<span style="color: #228B22"># Print the X-matrix</span>
+<span style="color: #8B008B; font-weight: bold">print</span>(X)
 </pre></div>
 <p>
-Make thereafter a small Python code which plots the data. Note that the function <b>multivariate</b> returns also the covariance discussed above and that it is defined by dividing by \( n-1 \) instead of \( n \).
+Make thereafter a small Python code which writes out the data. Note that the function <b>multivariate</b> returns also the covariance discussed above and that it is defined by dividing by \( n-1 \) instead of \( n \).

 <p>
 Now we are going to implement the PCA algorithm. We will break it down into various substeps.
@@ -1040,6 +1045,20 @@ When you are done with these steps, print out \( \mu_n \) to verify it is
 close to \( \mu \) and plot your mean centered data to verify it is
 centered at the origin! Compare your code with the functionality from <b>Scikit-Learn</b> discussed above.

+<p>
+
+<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
+<div class="highlight" style="background: #eeeedd"><pre style="line-height: 125%"><span></span>df = pd.DataFrame(X)
+<span style="color: #228B22"># Pandas does the centering for us</span>
+df = df -df.mean()
+display(df)
+
+<span style="color: #228B22"># we center it ourselves</span>
+X_centered = X - X.mean(axis=<span style="color: #B452CD">0</span>)
+<span style="color: #228B22"># test that we get the same as Pandas</span>
+<span style="color: #8B008B; font-weight: bold">print</span>(X_centered-df)
+</pre></div>
+
 <h3 id="___sec19">Compute the sample covariance </h3>

 <p>
@@ -1082,6 +1101,25 @@ Finally, collect all these steps and write your own PCA function and
 compare this with the functionality included in <b>Scikit-Learn</b>.
 Have the input be the data and have the output be the principal components and their associated eigenvalues, sorted in descending order. Can you think of a way to make it more efficient than the algorithm outlined above?

+<p>
+
+<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
+<div class="highlight" style="background: #eeeedd"><pre style="line-height: 125%"><span></span><span style="color: #228B22">#Now we do an SVD</span>
+U, s, V = np.linalg.svd(X_centered)
+c1 = V.T[:, <span style="color: #B452CD">0</span>]
+c2 = V.T[:, <span style="color: #B452CD">1</span>]
+W2 = V.T[:, :<span style="color: #B452CD">2</span>]
+X2D = X_centered.dot(W2)
+<span style="color: #8B008B; font-weight: bold">print</span>(X2D)
+<span style="color: #228B22">#thereafter we do a PCA with Scikit-learn</span>
+<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">sklearn.decomposition</span> <span style="color: #8B008B; font-weight: bold">import</span> PCA
+pca = PCA(n_components = <span style="color: #B452CD">2</span>)
+X2Dsl = pca.fit_transform(X)
+<span style="color: #8B008B; font-weight: bold">print</span>(<span style="color: #CD5555">&quot;Check that we get the same&quot;</span>)
+<span style="color: #8B008B; font-weight: bold">print</span>(X2D-X2Dsl)
+
+<span style="color: #8B008B; font-weight: bold">print</span>(pca.components_.T[:, <span style="color: #B452CD">0</span>])
+</pre></div>
 <p>
 Finally, try out your own PCA function with other data sets.

@@ -184,7 +184,7 @@ MathJax.Hub.Config({
 <center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
 <br>
 <p>
-<center><h4>Dec 29, 2019</h4></center> <!-- date -->
+<center><h4>Dec 30, 2019</h4></center> <!-- date -->
 <br>
 <p>
 <!-- !split --><br><br><br><br><br><br><br><br><br><br>
@@ -1017,13 +1017,18 @@ The following Python code aids in setting up the data
 <p>

 <!-- code=python (!bc pycod) typeset with pygments style "default" -->
-<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span>n <span style="color: #666666">=</span> <span style="color: #666666">1000</span>
+<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">numpy</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">np</span>
+<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">pandas</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">pd</span>
+<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">IPython.display</span> <span style="color: #008000; font-weight: bold">import</span> display
+n <span style="color: #666666">=</span> <span style="color: #666666">100</span>
 mean <span style="color: #666666">=</span> (<span style="color: #666666">-1</span>, <span style="color: #666666">2</span>)
 cov <span style="color: #666666">=</span> [[<span style="color: #666666">4</span>, <span style="color: #666666">2</span>], [<span style="color: #666666">2</span>, <span style="color: #666666">2</span>]]
 X <span style="color: #666666">=</span> np<span style="color: #666666">.</span>random<span style="color: #666666">.</span>multivariate_normal(mean, cov, n)
+<span style="color: #408080; font-style: italic"># Print the X-matrix</span>
+<span style="color: #008000; font-weight: bold">print</span>(X)
 </pre></div>
 <p>
-Make thereafter a small Python code which plots the data. Note that the function <b>multivariate</b> returns also the covariance discussed above and that it is defined by dividing by \( n-1 \) instead of \( n \).
+Make thereafter a small Python code which writes out the data. Note that the function <b>multivariate</b> returns also the covariance discussed above and that it is defined by dividing by \( n-1 \) instead of \( n \).

 <p>
 Now we are going to implement the PCA algorithm. We will break it down into various substeps.
@@ -1045,6 +1050,20 @@ When you are done with these steps, print out \( \mu_n \) to verify it is
 close to \( \mu \) and plot your mean centered data to verify it is
 centered at the origin! Compare your code with the functionality from <b>Scikit-Learn</b> discussed above.

+<p>
+
+<!-- code=python (!bc pycod) typeset with pygments style "default" -->
+<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span>df <span style="color: #666666">=</span> pd<span style="color: #666666">.</span>DataFrame(X)
+<span style="color: #408080; font-style: italic"># Pandas does the centering for us</span>
+df <span style="color: #666666">=</span> df <span style="color: #666666">-</span>df<span style="color: #666666">.</span>mean()
+display(df)
+
+<span style="color: #408080; font-style: italic"># we center it ourselves</span>
+X_centered <span style="color: #666666">=</span> X <span style="color: #666666">-</span> X<span style="color: #666666">.</span>mean(axis<span style="color: #666666">=0</span>)
+<span style="color: #408080; font-style: italic"># test that we get the same as Pandas</span>
+<span style="color: #008000; font-weight: bold">print</span>(X_centered<span style="color: #666666">-</span>df)
+</pre></div>
+
 <h3 id="___sec19">Compute the sample covariance </h3>

 <p>
@@ -1087,6 +1106,25 @@ Finally, collect all these steps and write your own PCA function and
 compare this with the functionality included in <b>Scikit-Learn</b>.
 Have the input be the data and have the output be the principal components and their associated eigenvalues, sorted in descending order. Can you think of a way to make it more efficient than the algorithm outlined above?

+<p>
+
+<!-- code=python (!bc pycod) typeset with pygments style "default" -->
+<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #408080; font-style: italic">#Now we do an SVD</span>
+U, s, V <span style="color: #666666">=</span> np<span style="color: #666666">.</span>linalg<span style="color: #666666">.</span>svd(X_centered)
+c1 <span style="color: #666666">=</span> V<span style="color: #666666">.</span>T[:, <span style="color: #666666">0</span>]
+c2 <span style="color: #666666">=</span> V<span style="color: #666666">.</span>T[:, <span style="color: #666666">1</span>]
+W2 <span style="color: #666666">=</span> V<span style="color: #666666">.</span>T[:, :<span style="color: #666666">2</span>]
+X2D <span style="color: #666666">=</span> X_centered<span style="color: #666666">.</span>dot(W2)
+<span style="color: #008000; font-weight: bold">print</span>(X2D)
+<span style="color: #408080; font-style: italic">#thereafter we do a PCA with Scikit-learn</span>
+<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.decomposition</span> <span style="color: #008000; font-weight: bold">import</span> PCA
+pca <span style="color: #666666">=</span> PCA(n_components <span style="color: #666666">=</span> <span style="color: #666666">2</span>)
+X2Dsl <span style="color: #666666">=</span> pca<span style="color: #666666">.</span>fit_transform(X)
+<span style="color: #008000; font-weight: bold">print</span>(<span style="color: #BA2121">&quot;Check that we get the same&quot;</span>)
+<span style="color: #008000; font-weight: bold">print</span>(X2D<span style="color: #666666">-</span>X2Dsl)
+
+<span style="color: #008000; font-weight: bold">print</span>(pca<span style="color: #666666">.</span>components_<span style="color: #666666">.</span>T[:, <span style="color: #666666">0</span>])
+</pre></div>
 <p>
 Finally, try out your own PCA function with other data sets.

@@ -10,7 +10,7 @@
    "<!-- Author: -->  \n",
    "**Morten Hjorth-Jensen**, Department of Physics, University of Oslo and Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University\n",
    "\n",
-    "Date: **Dec 29, 2019**\n",
+    "Date: **Dec 30, 2019**\n",
    "\n",
    "Copyright 1999-2019, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license\n",
    "\n",
@@ -1104,17 +1104,22 @@
   },
   "outputs": [],
   "source": [
-    "n = 1000\n",
+    "import numpy as np\n",
+    "import pandas as pd\n",
+    "from IPython.display import display\n",
+    "n = 100\n",
    "mean = (-1, 2)\n",
    "cov = [[4, 2], [2, 2]]\n",
-    "X = np.random.multivariate_normal(mean, cov, n)"
+    "X = np.random.multivariate_normal(mean, cov, n)\n",
+    "# Print the X-matrix\n",
+    "print(X)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
-    "Make thereafter a small Python code which plots the data. Note that the function **multivariate** returns also the covariance discussed above and that it is defined by dividing by $n-1$ instead of $n$.\n",
+    "Make thereafter a small Python code which writes out the data. Note that the function **multivariate** returns also the covariance discussed above and that it is defined by dividing by $n-1$ instead of $n$.\n",
    "\n",
    "Now we are going to implement the PCA algorithm. We will break it down into various substeps.\n",
    "\n",
@@ -1154,9 +1159,32 @@
   "source": [
    "When you are done with these steps, print out $\\mu_n$ to verify it is\n",
    "close to $\\mu$ and plot your mean centered data to verify it is\n",
-    "centered at the origin! Compare your code with the functionality from **Scikit-Learn** discussed above.\n",
-    "\n",
+    "centered at the origin! Compare your code with the functionality from **Scikit-Learn** discussed above."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "df = pd.DataFrame(X)\n",
+    "# Pandas does the centering for us\n",
+    "df = df -df.mean()\n",
+    "display(df)\n",
    "\n",
+    "# we center it ourselves\n",
+    "X_centered = X - X.mean(axis=0)\n",
+    "# test that we get the same as Pandas\n",
+    "print(X_centered-df)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
    "### Compute the sample covariance\n",
    "\n",
    "Now we are going to use the mean centered data to compute the sample covariance of the data. Recall it is given by:"
@@ -1213,11 +1241,43 @@
    "\n",
    "Finally, collect all these steps and write your own PCA function and\n",
    "compare this with the functionality included in **Scikit-Learn**.\n",
-    "Have the input be the data and have the output be the principal components and their associated eigenvalues, sorted in descending order. Can you think of a way to make it more efficient than the algorithm outlined above?\n",
+    "Have the input be the data and have the output be the principal components and their associated eigenvalues, sorted in descending order. Can you think of a way to make it more efficient than the algorithm outlined above?"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 13,
+   "metadata": {
+    "collapsed": false
+   },
+   "outputs": [],
+   "source": [
+    "#Now we do an SVD\n",
+    "U, s, V = np.linalg.svd(X_centered)\n",
+    "c1 = V.T[:, 0]\n",
+    "c2 = V.T[:, 1]\n",
+    "W2 = V.T[:, :2]\n",
+    "X2D = X_centered.dot(W2)\n",
+    "print(X2D)\n",
+    "#thereafter we do a PCA with Scikit-learn\n",
+    "from sklearn.decomposition import PCA\n",
+    "pca = PCA(n_components = 2)\n",
+    "X2Dsl = pca.fit_transform(X)\n",
+    "print(\"Check that we get the same\")\n",
+    "print(X2D-X2Dsl)\n",
    "\n",
+    "print(pca.components_.T[:, 0])"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
    "Finally, try out your own PCA function with other data sets.\n",
    "\n",
    "\n",
+    "\n",
+    "\n",
    "## Classical PCA Theorem\n",
    "\n",
    "We assume now that we have a design matrix $\\boldsymbol{X}$ which has been\n",
@@ -1502,7 +1562,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 12,
+   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
@@ -1549,7 +1609,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 13,
+   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
@@ -1573,7 +1633,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 14,
+   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
@@ -1597,7 +1657,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 15,
+   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
@@ -1621,7 +1681,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 16,
+   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
@@ -1675,7 +1735,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 17,
+   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
@@ -1698,7 +1758,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 18,
+   "execution_count": 20,
   "metadata": {
    "collapsed": false
   },
@@ -1745,7 +1805,7 @@
  },
  {
   "cell_type": "code",
-   "execution_count": 19,
+   "execution_count": 21,
   "metadata": {
    "collapsed": false
   },
@@ -779,13 +779,19 @@ this distribution, and store them in the $1000 \times 2$ matrix $\bm{X}$.
 The following Python code aids in setting up the data

 !bc pycod 
-n = 1000
+import numpy as np
+import pandas as pd
+from IPython.display import display
+n = 100
 mean = (-1, 2)
 cov = [[4, 2], [2, 2]]
 X = np.random.multivariate_normal(mean, cov, n)
+# Print the X-matrix
+print(X)
 !ec

-Make thereafter a small Python code which plots the data. Note that the function _multivariate_ returns also the covariance discussed above and that it is defined by dividing by $n-1$ instead of $n$.
+
+Make thereafter a small Python code which writes out the data. Note that the function _multivariate_ returns also the covariance discussed above and that it is defined by dividing by $n-1$ instead of $n$.

 Now we are going to implement the PCA algorithm. We will break it down into various substeps.

@@ -807,6 +813,18 @@ When you are done with these steps, print out $\mu_n$ to verify it is
 close to $\mu$ and plot your mean centered data to verify it is
 centered at the origin! Compare your code with the functionality from _Scikit-Learn_ discussed above.

+!bc pycod
+df = pd.DataFrame(X)
+# Pandas does the centering for us
+df = df -df.mean()
+display(df)
+
+# we center it ourselves
+X_centered = X - X.mean(axis=0)
+# test that we get the same as Pandas
+print(X_centered-df)
+!ec
+

 === Compute the sample covariance ===

@@ -843,9 +861,33 @@ Finally, collect all these steps and write your own PCA function and
 compare this with the functionality included in _Scikit-Learn_.
 Have the input be the data and have the output be the principal components and their associated eigenvalues, sorted in descending order. Can you think of a way to make it more efficient than the algorithm outlined above?

+!bc pycod
+#Now we do an SVD
+U, s, V = np.linalg.svd(X_centered)
+c1 = V.T[:, 0]
+c2 = V.T[:, 1]
+W2 = V.T[:, :2]
+X2D = X_centered.dot(W2)
+print(X2D)
+#thereafter we do a PCA with Scikit-learn
+from sklearn.decomposition import PCA
+pca = PCA(n_components = 2)
+X2Dsl = pca.fit_transform(X)
+print("Check that we get the same")
+print(X2D-X2Dsl)
+
+print(pca.components_.T[:, 0])
+
+
+!ec
+
+
+
 Finally, try out your own PCA function with other data sets.


+
+
 !split
 ===== Classical PCA Theorem   =====