lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

added pca theorem proof

Browse Source
This commit is contained in:
mhjensen
2019-10-22 15:54:52 +02:00
parent 7fbad90972
commit 5aeb822fd8
36 changed files with 319 additions and 77 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/pub/DimRed/html/._DimRed-bs000.html
+2 -2
View File
@@ -90,7 +90,7 @@ Automatically generated HTML file from DocOnce source
('Towards the PCA theorem', 2, None, '___sec15'),
('The Algorithm before the Theorem', 2, None, '___sec16'),
('Classical PCA Theorem', 2, None, '___sec17'),
('Prof of the PCA Theorem', 2, None, '___sec18'),
('Proof of the PCA Theorem', 2, None, '___sec18'),
('Getting started with PCA', 2, None, '___sec19'),
('Principal Component Analysis', 2, None, '___sec20'),
('PCA and scikit-learn', 2, None, '___sec21'),
@@ -155,7 +155,7 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._DimRed-bs016.html#___sec15" style="font-size: 80%;">Towards the PCA theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs017.html#___sec16" style="font-size: 80%;">The Algorithm before the Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs018.html#___sec17" style="font-size: 80%;">Classical PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Prof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Proof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs020.html#___sec19" style="font-size: 80%;">Getting started with PCA</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs021.html#___sec20" style="font-size: 80%;">Principal Component Analysis</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs022.html#___sec21" style="font-size: 80%;">PCA and scikit-learn</a></li>
doc/pub/DimRed/html/._DimRed-bs001.html
+2 -2
View File
@@ -90,7 +90,7 @@ Automatically generated HTML file from DocOnce source
('Towards the PCA theorem', 2, None, '___sec15'),
('The Algorithm before the Theorem', 2, None, '___sec16'),
('Classical PCA Theorem', 2, None, '___sec17'),
('Prof of the PCA Theorem', 2, None, '___sec18'),
('Proof of the PCA Theorem', 2, None, '___sec18'),
('Getting started with PCA', 2, None, '___sec19'),
('Principal Component Analysis', 2, None, '___sec20'),
('PCA and scikit-learn', 2, None, '___sec21'),
@@ -155,7 +155,7 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._DimRed-bs016.html#___sec15" style="font-size: 80%;">Towards the PCA theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs017.html#___sec16" style="font-size: 80%;">The Algorithm before the Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs018.html#___sec17" style="font-size: 80%;">Classical PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Prof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Proof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs020.html#___sec19" style="font-size: 80%;">Getting started with PCA</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs021.html#___sec20" style="font-size: 80%;">Principal Component Analysis</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs022.html#___sec21" style="font-size: 80%;">PCA and scikit-learn</a></li>
doc/pub/DimRed/html/._DimRed-bs002.html
+2 -2
View File
@@ -90,7 +90,7 @@ Automatically generated HTML file from DocOnce source
('Towards the PCA theorem', 2, None, '___sec15'),
('The Algorithm before the Theorem', 2, None, '___sec16'),
('Classical PCA Theorem', 2, None, '___sec17'),
('Prof of the PCA Theorem', 2, None, '___sec18'),
('Proof of the PCA Theorem', 2, None, '___sec18'),
('Getting started with PCA', 2, None, '___sec19'),
('Principal Component Analysis', 2, None, '___sec20'),
('PCA and scikit-learn', 2, None, '___sec21'),
@@ -155,7 +155,7 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._DimRed-bs016.html#___sec15" style="font-size: 80%;">Towards the PCA theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs017.html#___sec16" style="font-size: 80%;">The Algorithm before the Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs018.html#___sec17" style="font-size: 80%;">Classical PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Prof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Proof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs020.html#___sec19" style="font-size: 80%;">Getting started with PCA</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs021.html#___sec20" style="font-size: 80%;">Principal Component Analysis</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs022.html#___sec21" style="font-size: 80%;">PCA and scikit-learn</a></li>
doc/pub/DimRed/html/._DimRed-bs003.html
+2 -2
View File
@@ -90,7 +90,7 @@ Automatically generated HTML file from DocOnce source
('Towards the PCA theorem', 2, None, '___sec15'),
('The Algorithm before the Theorem', 2, None, '___sec16'),
('Classical PCA Theorem', 2, None, '___sec17'),
('Prof of the PCA Theorem', 2, None, '___sec18'),
('Proof of the PCA Theorem', 2, None, '___sec18'),
('Getting started with PCA', 2, None, '___sec19'),
('Principal Component Analysis', 2, None, '___sec20'),
('PCA and scikit-learn', 2, None, '___sec21'),
@@ -155,7 +155,7 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._DimRed-bs016.html#___sec15" style="font-size: 80%;">Towards the PCA theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs017.html#___sec16" style="font-size: 80%;">The Algorithm before the Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs018.html#___sec17" style="font-size: 80%;">Classical PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Prof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Proof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs020.html#___sec19" style="font-size: 80%;">Getting started with PCA</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs021.html#___sec20" style="font-size: 80%;">Principal Component Analysis</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs022.html#___sec21" style="font-size: 80%;">PCA and scikit-learn</a></li>
doc/pub/DimRed/html/._DimRed-bs004.html
+2 -2
View File
@@ -90,7 +90,7 @@ Automatically generated HTML file from DocOnce source
('Towards the PCA theorem', 2, None, '___sec15'),
('The Algorithm before the Theorem', 2, None, '___sec16'),
('Classical PCA Theorem', 2, None, '___sec17'),
('Prof of the PCA Theorem', 2, None, '___sec18'),
('Proof of the PCA Theorem', 2, None, '___sec18'),
('Getting started with PCA', 2, None, '___sec19'),
('Principal Component Analysis', 2, None, '___sec20'),
('PCA and scikit-learn', 2, None, '___sec21'),
@@ -155,7 +155,7 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._DimRed-bs016.html#___sec15" style="font-size: 80%;">Towards the PCA theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs017.html#___sec16" style="font-size: 80%;">The Algorithm before the Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs018.html#___sec17" style="font-size: 80%;">Classical PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Prof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Proof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs020.html#___sec19" style="font-size: 80%;">Getting started with PCA</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs021.html#___sec20" style="font-size: 80%;">Principal Component Analysis</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs022.html#___sec21" style="font-size: 80%;">PCA and scikit-learn</a></li>
doc/pub/DimRed/html/._DimRed-bs005.html
+2 -2
View File
@@ -90,7 +90,7 @@ Automatically generated HTML file from DocOnce source
('Towards the PCA theorem', 2, None, '___sec15'),
('The Algorithm before the Theorem', 2, None, '___sec16'),
('Classical PCA Theorem', 2, None, '___sec17'),
('Prof of the PCA Theorem', 2, None, '___sec18'),
('Proof of the PCA Theorem', 2, None, '___sec18'),
('Getting started with PCA', 2, None, '___sec19'),
('Principal Component Analysis', 2, None, '___sec20'),
('PCA and scikit-learn', 2, None, '___sec21'),
@@ -155,7 +155,7 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._DimRed-bs016.html#___sec15" style="font-size: 80%;">Towards the PCA theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs017.html#___sec16" style="font-size: 80%;">The Algorithm before the Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs018.html#___sec17" style="font-size: 80%;">Classical PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Prof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Proof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs020.html#___sec19" style="font-size: 80%;">Getting started with PCA</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs021.html#___sec20" style="font-size: 80%;">Principal Component Analysis</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs022.html#___sec21" style="font-size: 80%;">PCA and scikit-learn</a></li>
doc/pub/DimRed/html/._DimRed-bs006.html
+2 -2
View File
@@ -90,7 +90,7 @@ Automatically generated HTML file from DocOnce source
('Towards the PCA theorem', 2, None, '___sec15'),
('The Algorithm before the Theorem', 2, None, '___sec16'),
('Classical PCA Theorem', 2, None, '___sec17'),
('Prof of the PCA Theorem', 2, None, '___sec18'),
('Proof of the PCA Theorem', 2, None, '___sec18'),
('Getting started with PCA', 2, None, '___sec19'),
('Principal Component Analysis', 2, None, '___sec20'),
('PCA and scikit-learn', 2, None, '___sec21'),
@@ -155,7 +155,7 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._DimRed-bs016.html#___sec15" style="font-size: 80%;">Towards the PCA theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs017.html#___sec16" style="font-size: 80%;">The Algorithm before the Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs018.html#___sec17" style="font-size: 80%;">Classical PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Prof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Proof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs020.html#___sec19" style="font-size: 80%;">Getting started with PCA</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs021.html#___sec20" style="font-size: 80%;">Principal Component Analysis</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs022.html#___sec21" style="font-size: 80%;">PCA and scikit-learn</a></li>
doc/pub/DimRed/html/._DimRed-bs007.html
+2 -2
View File
@@ -90,7 +90,7 @@ Automatically generated HTML file from DocOnce source
('Towards the PCA theorem', 2, None, '___sec15'),
('The Algorithm before the Theorem', 2, None, '___sec16'),
('Classical PCA Theorem', 2, None, '___sec17'),
('Prof of the PCA Theorem', 2, None, '___sec18'),
('Proof of the PCA Theorem', 2, None, '___sec18'),
('Getting started with PCA', 2, None, '___sec19'),
('Principal Component Analysis', 2, None, '___sec20'),
('PCA and scikit-learn', 2, None, '___sec21'),
@@ -155,7 +155,7 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._DimRed-bs016.html#___sec15" style="font-size: 80%;">Towards the PCA theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs017.html#___sec16" style="font-size: 80%;">The Algorithm before the Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs018.html#___sec17" style="font-size: 80%;">Classical PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Prof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Proof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs020.html#___sec19" style="font-size: 80%;">Getting started with PCA</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs021.html#___sec20" style="font-size: 80%;">Principal Component Analysis</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs022.html#___sec21" style="font-size: 80%;">PCA and scikit-learn</a></li>
doc/pub/DimRed/html/._DimRed-bs008.html
+2 -2
View File
@@ -90,7 +90,7 @@ Automatically generated HTML file from DocOnce source
('Towards the PCA theorem', 2, None, '___sec15'),
('The Algorithm before the Theorem', 2, None, '___sec16'),
('Classical PCA Theorem', 2, None, '___sec17'),
('Prof of the PCA Theorem', 2, None, '___sec18'),
('Proof of the PCA Theorem', 2, None, '___sec18'),
('Getting started with PCA', 2, None, '___sec19'),
('Principal Component Analysis', 2, None, '___sec20'),
('PCA and scikit-learn', 2, None, '___sec21'),
@@ -155,7 +155,7 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._DimRed-bs016.html#___sec15" style="font-size: 80%;">Towards the PCA theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs017.html#___sec16" style="font-size: 80%;">The Algorithm before the Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs018.html#___sec17" style="font-size: 80%;">Classical PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Prof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Proof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs020.html#___sec19" style="font-size: 80%;">Getting started with PCA</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs021.html#___sec20" style="font-size: 80%;">Principal Component Analysis</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs022.html#___sec21" style="font-size: 80%;">PCA and scikit-learn</a></li>
doc/pub/DimRed/html/._DimRed-bs009.html
+2 -2
View File
@@ -90,7 +90,7 @@ Automatically generated HTML file from DocOnce source
('Towards the PCA theorem', 2, None, '___sec15'),
('The Algorithm before the Theorem', 2, None, '___sec16'),
('Classical PCA Theorem', 2, None, '___sec17'),
('Prof of the PCA Theorem', 2, None, '___sec18'),
('Proof of the PCA Theorem', 2, None, '___sec18'),
('Getting started with PCA', 2, None, '___sec19'),
('Principal Component Analysis', 2, None, '___sec20'),
('PCA and scikit-learn', 2, None, '___sec21'),
@@ -155,7 +155,7 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._DimRed-bs016.html#___sec15" style="font-size: 80%;">Towards the PCA theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs017.html#___sec16" style="font-size: 80%;">The Algorithm before the Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs018.html#___sec17" style="font-size: 80%;">Classical PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Prof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Proof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs020.html#___sec19" style="font-size: 80%;">Getting started with PCA</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs021.html#___sec20" style="font-size: 80%;">Principal Component Analysis</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs022.html#___sec21" style="font-size: 80%;">PCA and scikit-learn</a></li>
doc/pub/DimRed/html/._DimRed-bs010.html
+2 -2
View File
@@ -90,7 +90,7 @@ Automatically generated HTML file from DocOnce source
('Towards the PCA theorem', 2, None, '___sec15'),
('The Algorithm before the Theorem', 2, None, '___sec16'),
('Classical PCA Theorem', 2, None, '___sec17'),
('Prof of the PCA Theorem', 2, None, '___sec18'),
('Proof of the PCA Theorem', 2, None, '___sec18'),
('Getting started with PCA', 2, None, '___sec19'),
('Principal Component Analysis', 2, None, '___sec20'),
('PCA and scikit-learn', 2, None, '___sec21'),
@@ -155,7 +155,7 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._DimRed-bs016.html#___sec15" style="font-size: 80%;">Towards the PCA theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs017.html#___sec16" style="font-size: 80%;">The Algorithm before the Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs018.html#___sec17" style="font-size: 80%;">Classical PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Prof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Proof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs020.html#___sec19" style="font-size: 80%;">Getting started with PCA</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs021.html#___sec20" style="font-size: 80%;">Principal Component Analysis</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs022.html#___sec21" style="font-size: 80%;">PCA and scikit-learn</a></li>
doc/pub/DimRed/html/._DimRed-bs011.html
+2 -2
View File
@@ -90,7 +90,7 @@ Automatically generated HTML file from DocOnce source
('Towards the PCA theorem', 2, None, '___sec15'),
('The Algorithm before the Theorem', 2, None, '___sec16'),
('Classical PCA Theorem', 2, None, '___sec17'),
('Prof of the PCA Theorem', 2, None, '___sec18'),
('Proof of the PCA Theorem', 2, None, '___sec18'),
('Getting started with PCA', 2, None, '___sec19'),
('Principal Component Analysis', 2, None, '___sec20'),
('PCA and scikit-learn', 2, None, '___sec21'),
@@ -155,7 +155,7 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._DimRed-bs016.html#___sec15" style="font-size: 80%;">Towards the PCA theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs017.html#___sec16" style="font-size: 80%;">The Algorithm before the Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs018.html#___sec17" style="font-size: 80%;">Classical PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Prof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Proof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs020.html#___sec19" style="font-size: 80%;">Getting started with PCA</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs021.html#___sec20" style="font-size: 80%;">Principal Component Analysis</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs022.html#___sec21" style="font-size: 80%;">PCA and scikit-learn</a></li>
doc/pub/DimRed/html/._DimRed-bs012.html
+2 -2
View File
@@ -90,7 +90,7 @@ Automatically generated HTML file from DocOnce source
('Towards the PCA theorem', 2, None, '___sec15'),
('The Algorithm before the Theorem', 2, None, '___sec16'),
('Classical PCA Theorem', 2, None, '___sec17'),
('Prof of the PCA Theorem', 2, None, '___sec18'),
('Proof of the PCA Theorem', 2, None, '___sec18'),
('Getting started with PCA', 2, None, '___sec19'),
('Principal Component Analysis', 2, None, '___sec20'),
('PCA and scikit-learn', 2, None, '___sec21'),
@@ -155,7 +155,7 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._DimRed-bs016.html#___sec15" style="font-size: 80%;">Towards the PCA theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs017.html#___sec16" style="font-size: 80%;">The Algorithm before the Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs018.html#___sec17" style="font-size: 80%;">Classical PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Prof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Proof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs020.html#___sec19" style="font-size: 80%;">Getting started with PCA</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs021.html#___sec20" style="font-size: 80%;">Principal Component Analysis</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs022.html#___sec21" style="font-size: 80%;">PCA and scikit-learn</a></li>
doc/pub/DimRed/html/._DimRed-bs013.html
+2 -2
View File
@@ -90,7 +90,7 @@ Automatically generated HTML file from DocOnce source
('Towards the PCA theorem', 2, None, '___sec15'),
('The Algorithm before the Theorem', 2, None, '___sec16'),
('Classical PCA Theorem', 2, None, '___sec17'),
('Prof of the PCA Theorem', 2, None, '___sec18'),
('Proof of the PCA Theorem', 2, None, '___sec18'),
('Getting started with PCA', 2, None, '___sec19'),
('Principal Component Analysis', 2, None, '___sec20'),
('PCA and scikit-learn', 2, None, '___sec21'),
@@ -155,7 +155,7 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._DimRed-bs016.html#___sec15" style="font-size: 80%;">Towards the PCA theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs017.html#___sec16" style="font-size: 80%;">The Algorithm before the Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs018.html#___sec17" style="font-size: 80%;">Classical PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Prof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Proof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs020.html#___sec19" style="font-size: 80%;">Getting started with PCA</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs021.html#___sec20" style="font-size: 80%;">Principal Component Analysis</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs022.html#___sec21" style="font-size: 80%;">PCA and scikit-learn</a></li>
doc/pub/DimRed/html/._DimRed-bs014.html
+2 -2
View File
@@ -90,7 +90,7 @@ Automatically generated HTML file from DocOnce source
('Towards the PCA theorem', 2, None, '___sec15'),
('The Algorithm before the Theorem', 2, None, '___sec16'),
('Classical PCA Theorem', 2, None, '___sec17'),
('Prof of the PCA Theorem', 2, None, '___sec18'),
('Proof of the PCA Theorem', 2, None, '___sec18'),
('Getting started with PCA', 2, None, '___sec19'),
('Principal Component Analysis', 2, None, '___sec20'),
('PCA and scikit-learn', 2, None, '___sec21'),
@@ -155,7 +155,7 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._DimRed-bs016.html#___sec15" style="font-size: 80%;">Towards the PCA theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs017.html#___sec16" style="font-size: 80%;">The Algorithm before the Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs018.html#___sec17" style="font-size: 80%;">Classical PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Prof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Proof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs020.html#___sec19" style="font-size: 80%;">Getting started with PCA</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs021.html#___sec20" style="font-size: 80%;">Principal Component Analysis</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs022.html#___sec21" style="font-size: 80%;">PCA and scikit-learn</a></li>
doc/pub/DimRed/html/._DimRed-bs015.html
+2 -2
View File
@@ -90,7 +90,7 @@ Automatically generated HTML file from DocOnce source
('Towards the PCA theorem', 2, None, '___sec15'),
('The Algorithm before the Theorem', 2, None, '___sec16'),
('Classical PCA Theorem', 2, None, '___sec17'),
('Prof of the PCA Theorem', 2, None, '___sec18'),
('Proof of the PCA Theorem', 2, None, '___sec18'),
('Getting started with PCA', 2, None, '___sec19'),
('Principal Component Analysis', 2, None, '___sec20'),
('PCA and scikit-learn', 2, None, '___sec21'),
@@ -155,7 +155,7 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._DimRed-bs016.html#___sec15" style="font-size: 80%;">Towards the PCA theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs017.html#___sec16" style="font-size: 80%;">The Algorithm before the Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs018.html#___sec17" style="font-size: 80%;">Classical PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Prof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Proof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs020.html#___sec19" style="font-size: 80%;">Getting started with PCA</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs021.html#___sec20" style="font-size: 80%;">Principal Component Analysis</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs022.html#___sec21" style="font-size: 80%;">PCA and scikit-learn</a></li>
doc/pub/DimRed/html/._DimRed-bs016.html
+2 -2
View File
@@ -90,7 +90,7 @@ Automatically generated HTML file from DocOnce source
('Towards the PCA theorem', 2, None, '___sec15'),
('The Algorithm before the Theorem', 2, None, '___sec16'),
('Classical PCA Theorem', 2, None, '___sec17'),
('Prof of the PCA Theorem', 2, None, '___sec18'),
('Proof of the PCA Theorem', 2, None, '___sec18'),
('Getting started with PCA', 2, None, '___sec19'),
('Principal Component Analysis', 2, None, '___sec20'),
('PCA and scikit-learn', 2, None, '___sec21'),
@@ -155,7 +155,7 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="#___sec15" style="font-size: 80%;">Towards the PCA theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs017.html#___sec16" style="font-size: 80%;">The Algorithm before the Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs018.html#___sec17" style="font-size: 80%;">Classical PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Prof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Proof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs020.html#___sec19" style="font-size: 80%;">Getting started with PCA</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs021.html#___sec20" style="font-size: 80%;">Principal Component Analysis</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs022.html#___sec21" style="font-size: 80%;">PCA and scikit-learn</a></li>
doc/pub/DimRed/html/._DimRed-bs017.html
+3 -3
View File
@@ -90,7 +90,7 @@ Automatically generated HTML file from DocOnce source
('Towards the PCA theorem', 2, None, '___sec15'),
('The Algorithm before the Theorem', 2, None, '___sec16'),
('Classical PCA Theorem', 2, None, '___sec17'),
('Prof of the PCA Theorem', 2, None, '___sec18'),
('Proof of the PCA Theorem', 2, None, '___sec18'),
('Getting started with PCA', 2, None, '___sec19'),
('Principal Component Analysis', 2, None, '___sec20'),
('PCA and scikit-learn', 2, None, '___sec21'),
@@ -155,7 +155,7 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._DimRed-bs016.html#___sec15" style="font-size: 80%;">Towards the PCA theorem</a></li>
<!-- navigation toc: --> <li><a href="#___sec16" style="font-size: 80%;">The Algorithm before the Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs018.html#___sec17" style="font-size: 80%;">Classical PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Prof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Proof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs020.html#___sec19" style="font-size: 80%;">Getting started with PCA</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs021.html#___sec20" style="font-size: 80%;">Principal Component Analysis</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs022.html#___sec21" style="font-size: 80%;">PCA and scikit-learn</a></li>
@@ -203,7 +203,7 @@ $$
<ul>
<li> Center the data by subtracting the mean value for each column. This leads to a new matrix \( \boldsymbol{X}\rightarrow \overline{\boldsymbol{X}} \).</li>
<li> Compute then the covariance/correlation matrix $\mathbb{E}[\overline{\boldsymbol{X}}\overline{\boldsymbol{X}}^T].</li>
<li> Compute then the covariance/correlation matrix \( \mathbb{E}[\overline{\boldsymbol{X}}\overline{\boldsymbol{X}}^T] \).</li>
<li> Find the eigenpairs of \( \boldsymbol{C} \) with eigenvalues \( [\lambda_0,\lambda_1,\dots,\lambda_{p-1}] \) and eigenvectors \( [\boldsymbol{s}_0,\boldsymbol{s}_1,\dots,\boldsymbol{s}_{p-1}] \).</li>
<li> Order the eigenvalue (and the eigenvectors accordingly) in order of decreasing eigenvalues.</li>
<li> Keep only those \( l \) eigenvalues larger than a selected threshold value, discarding thus \( p-l \) features since we expect small variations in the data here.</li>
doc/pub/DimRed/html/._DimRed-bs018.html
+18 -2
View File
@@ -90,7 +90,7 @@ Automatically generated HTML file from DocOnce source
('Towards the PCA theorem', 2, None, '___sec15'),
('The Algorithm before the Theorem', 2, None, '___sec16'),
('Classical PCA Theorem', 2, None, '___sec17'),
('Prof of the PCA Theorem', 2, None, '___sec18'),
('Proof of the PCA Theorem', 2, None, '___sec18'),
('Getting started with PCA', 2, None, '___sec19'),
('Principal Component Analysis', 2, None, '___sec20'),
('PCA and scikit-learn', 2, None, '___sec21'),
@@ -155,7 +155,7 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._DimRed-bs016.html#___sec15" style="font-size: 80%;">Towards the PCA theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs017.html#___sec16" style="font-size: 80%;">The Algorithm before the Theorem</a></li>
<!-- navigation toc: --> <li><a href="#___sec17" style="font-size: 80%;">Classical PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Prof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Proof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs020.html#___sec19" style="font-size: 80%;">Getting started with PCA</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs021.html#___sec20" style="font-size: 80%;">Principal Component Analysis</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs022.html#___sec21" style="font-size: 80%;">PCA and scikit-learn</a></li>
@@ -182,6 +182,22 @@ MathJax.Hub.Config({
<h2 id="___sec17" class="anchor">Classical PCA Theorem </h2>
<p>
We assume now that we have a design matrix \( \boldsymbol{X} \) which has been centered as discussed above. For the sake of simplicity we skip the overline symbol. The matrix is defined in terms of the various column vectors \( [\boldsymbol{x}_0,\boldsymbol{x}_1,\dots, \boldsymbol{x}_{p-1}] \)
each with dimension \( \boldsymbol{x}\in {\mathbb{R}}^{n} \).
<p>
We assume also that we have an orthogonal transformation \( \boldsymbol{W}\in {\mathbb{R}}^{p\times p} \). We define the reconstruction error (which is similar to the mean squared error we have seen before) as
$$
J(\boldsymbol{W},\boldsymbol{Z}) = \frac{1}{p}\sum_i (\boldsymbol{x}_i - \overline{\boldsymbol{x}_i})^2,
$$
with \( \overline{\boldsymbol{x}_i} = \boldsymbol{W}\boldsymbol{z}_i \), where \( \boldsymbol{z}_i \) is a row vector with dimension \( {\mathbb{R}}^{n} \) of the matrix
\( \boldsymbol{Z}\in {\mathbb{R}}^{p\times n} \).
<p>
The PCA theorem states that minimizing the above reconstruction error corresponds to setting \( \boldsymbol{W}=\boldsymbol{S} \), the orthogonal matrix which diagonalizes the empirical covariance(correlation) matrix. The optimal low-dimensional encoding of the data is then given by a set of vectors \( l \) $\boldsymbol{z}_i$, with \( l < < p \), defined by the orthogonal projection of the data onto the columns spanned by they eigenvectors of the covariance(correlations matrix).
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
doc/pub/DimRed/html/._DimRed-bs019.html
+21 -3
View File
@@ -90,7 +90,7 @@ Automatically generated HTML file from DocOnce source
('Towards the PCA theorem', 2, None, '___sec15'),
('The Algorithm before the Theorem', 2, None, '___sec16'),
('Classical PCA Theorem', 2, None, '___sec17'),
('Prof of the PCA Theorem', 2, None, '___sec18'),
('Proof of the PCA Theorem', 2, None, '___sec18'),
('Getting started with PCA', 2, None, '___sec19'),
('Principal Component Analysis', 2, None, '___sec20'),
('PCA and scikit-learn', 2, None, '___sec21'),
@@ -155,7 +155,7 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._DimRed-bs016.html#___sec15" style="font-size: 80%;">Towards the PCA theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs017.html#___sec16" style="font-size: 80%;">The Algorithm before the Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs018.html#___sec17" style="font-size: 80%;">Classical PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="#___sec18" style="font-size: 80%;">Prof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="#___sec18" style="font-size: 80%;">Proof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs020.html#___sec19" style="font-size: 80%;">Getting started with PCA</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs021.html#___sec20" style="font-size: 80%;">Principal Component Analysis</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs022.html#___sec21" style="font-size: 80%;">PCA and scikit-learn</a></li>
@@ -180,7 +180,25 @@ MathJax.Hub.Config({
<a name="part0019"></a>
<!-- !split -->
<h2 id="___sec18" class="anchor">Prof of the PCA Theorem </h2>
<h2 id="___sec18" class="anchor">Proof of the PCA Theorem </h2>
<p>
To show the PCA theorem let us start with the assumption that there is a vector \( \boldsymbol{w}_0 \) which corresponds to a solution which minimized the reconstruction error \( J \). This is an orthogonal vector. It means that we now approximate the reconstruction error in terms of \( \boldsymbol{w}_0 \) and \( \boldsymbol{z}_0 \) as
$$
J(\boldsymbol{w}_0,\boldsymbol{z}_0)= \frac{1}{p}\sum_i (\boldsymbol{x}_i - z_{i0}\boldsymbol{w}_0)^2=\frac{1}{p}\sum_i (\boldsymbol{x}_^T\boldsymbol{x}_i - 2z_{i0}\boldsymbol{w}_0^T\boldsymbol{x}_i+z_{i0}^2\boldsymbol{w}_0^T\boldsymbol{w}_0),
$$
which we can rewrite due to the orthogonality of \( \boldsymbol{w}_i \) as
$$
J(\boldsymbol{w}_0,\boldsymbol{z}_0)=\frac{1}{p}\sum_i (\boldsymbol{x}_^T\boldsymbol{x}_i - 2z_{i0}\boldsymbol{w}_0^T\boldsymbol{x}_i+z_{i0}^2).
$$
Minimizing \( J \) with respect to the unknown parameters \( z_{0i} \) we obtain that
$$
z_{i0}=\boldsymbol{w}_0^T\boldsymbol{x}_i,
$$
where the vectors on the rhs are known.
<p>
<p>
doc/pub/DimRed/html/._DimRed-bs020.html
+2 -2
View File
@@ -90,7 +90,7 @@ Automatically generated HTML file from DocOnce source
('Towards the PCA theorem', 2, None, '___sec15'),
('The Algorithm before the Theorem', 2, None, '___sec16'),
('Classical PCA Theorem', 2, None, '___sec17'),
('Prof of the PCA Theorem', 2, None, '___sec18'),
('Proof of the PCA Theorem', 2, None, '___sec18'),
('Getting started with PCA', 2, None, '___sec19'),
('Principal Component Analysis', 2, None, '___sec20'),
('PCA and scikit-learn', 2, None, '___sec21'),
@@ -155,7 +155,7 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._DimRed-bs016.html#___sec15" style="font-size: 80%;">Towards the PCA theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs017.html#___sec16" style="font-size: 80%;">The Algorithm before the Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs018.html#___sec17" style="font-size: 80%;">Classical PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Prof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Proof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="#___sec19" style="font-size: 80%;">Getting started with PCA</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs021.html#___sec20" style="font-size: 80%;">Principal Component Analysis</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs022.html#___sec21" style="font-size: 80%;">PCA and scikit-learn</a></li>
doc/pub/DimRed/html/._DimRed-bs021.html
+2 -2
View File
@@ -90,7 +90,7 @@ Automatically generated HTML file from DocOnce source
('Towards the PCA theorem', 2, None, '___sec15'),
('The Algorithm before the Theorem', 2, None, '___sec16'),
('Classical PCA Theorem', 2, None, '___sec17'),
('Prof of the PCA Theorem', 2, None, '___sec18'),
('Proof of the PCA Theorem', 2, None, '___sec18'),
('Getting started with PCA', 2, None, '___sec19'),
('Principal Component Analysis', 2, None, '___sec20'),
('PCA and scikit-learn', 2, None, '___sec21'),
@@ -155,7 +155,7 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._DimRed-bs016.html#___sec15" style="font-size: 80%;">Towards the PCA theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs017.html#___sec16" style="font-size: 80%;">The Algorithm before the Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs018.html#___sec17" style="font-size: 80%;">Classical PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Prof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Proof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs020.html#___sec19" style="font-size: 80%;">Getting started with PCA</a></li>
<!-- navigation toc: --> <li><a href="#___sec20" style="font-size: 80%;">Principal Component Analysis</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs022.html#___sec21" style="font-size: 80%;">PCA and scikit-learn</a></li>
doc/pub/DimRed/html/._DimRed-bs022.html
+2 -2
View File
@@ -90,7 +90,7 @@ Automatically generated HTML file from DocOnce source
('Towards the PCA theorem', 2, None, '___sec15'),
('The Algorithm before the Theorem', 2, None, '___sec16'),
('Classical PCA Theorem', 2, None, '___sec17'),
('Prof of the PCA Theorem', 2, None, '___sec18'),
('Proof of the PCA Theorem', 2, None, '___sec18'),
('Getting started with PCA', 2, None, '___sec19'),
('Principal Component Analysis', 2, None, '___sec20'),
('PCA and scikit-learn', 2, None, '___sec21'),
@@ -155,7 +155,7 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._DimRed-bs016.html#___sec15" style="font-size: 80%;">Towards the PCA theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs017.html#___sec16" style="font-size: 80%;">The Algorithm before the Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs018.html#___sec17" style="font-size: 80%;">Classical PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Prof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Proof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs020.html#___sec19" style="font-size: 80%;">Getting started with PCA</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs021.html#___sec20" style="font-size: 80%;">Principal Component Analysis</a></li>
<!-- navigation toc: --> <li><a href="#___sec21" style="font-size: 80%;">PCA and scikit-learn</a></li>
doc/pub/DimRed/html/._DimRed-bs023.html
+2 -2
View File
@@ -90,7 +90,7 @@ Automatically generated HTML file from DocOnce source
('Towards the PCA theorem', 2, None, '___sec15'),
('The Algorithm before the Theorem', 2, None, '___sec16'),
('Classical PCA Theorem', 2, None, '___sec17'),
('Prof of the PCA Theorem', 2, None, '___sec18'),
('Proof of the PCA Theorem', 2, None, '___sec18'),
('Getting started with PCA', 2, None, '___sec19'),
('Principal Component Analysis', 2, None, '___sec20'),
('PCA and scikit-learn', 2, None, '___sec21'),
@@ -155,7 +155,7 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._DimRed-bs016.html#___sec15" style="font-size: 80%;">Towards the PCA theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs017.html#___sec16" style="font-size: 80%;">The Algorithm before the Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs018.html#___sec17" style="font-size: 80%;">Classical PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Prof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Proof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs020.html#___sec19" style="font-size: 80%;">Getting started with PCA</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs021.html#___sec20" style="font-size: 80%;">Principal Component Analysis</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs022.html#___sec21" style="font-size: 80%;">PCA and scikit-learn</a></li>
doc/pub/DimRed/html/._DimRed-bs024.html
+2 -2
View File
@@ -90,7 +90,7 @@ Automatically generated HTML file from DocOnce source
('Towards the PCA theorem', 2, None, '___sec15'),
('The Algorithm before the Theorem', 2, None, '___sec16'),
('Classical PCA Theorem', 2, None, '___sec17'),
('Prof of the PCA Theorem', 2, None, '___sec18'),
('Proof of the PCA Theorem', 2, None, '___sec18'),
('Getting started with PCA', 2, None, '___sec19'),
('Principal Component Analysis', 2, None, '___sec20'),
('PCA and scikit-learn', 2, None, '___sec21'),
@@ -155,7 +155,7 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._DimRed-bs016.html#___sec15" style="font-size: 80%;">Towards the PCA theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs017.html#___sec16" style="font-size: 80%;">The Algorithm before the Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs018.html#___sec17" style="font-size: 80%;">Classical PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Prof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Proof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs020.html#___sec19" style="font-size: 80%;">Getting started with PCA</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs021.html#___sec20" style="font-size: 80%;">Principal Component Analysis</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs022.html#___sec21" style="font-size: 80%;">PCA and scikit-learn</a></li>
doc/pub/DimRed/html/._DimRed-bs025.html
+2 -2
View File
@@ -90,7 +90,7 @@ Automatically generated HTML file from DocOnce source
('Towards the PCA theorem', 2, None, '___sec15'),
('The Algorithm before the Theorem', 2, None, '___sec16'),
('Classical PCA Theorem', 2, None, '___sec17'),
('Prof of the PCA Theorem', 2, None, '___sec18'),
('Proof of the PCA Theorem', 2, None, '___sec18'),
('Getting started with PCA', 2, None, '___sec19'),
('Principal Component Analysis', 2, None, '___sec20'),
('PCA and scikit-learn', 2, None, '___sec21'),
@@ -155,7 +155,7 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._DimRed-bs016.html#___sec15" style="font-size: 80%;">Towards the PCA theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs017.html#___sec16" style="font-size: 80%;">The Algorithm before the Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs018.html#___sec17" style="font-size: 80%;">Classical PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Prof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Proof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs020.html#___sec19" style="font-size: 80%;">Getting started with PCA</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs021.html#___sec20" style="font-size: 80%;">Principal Component Analysis</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs022.html#___sec21" style="font-size: 80%;">PCA and scikit-learn</a></li>
doc/pub/DimRed/html/._DimRed-bs026.html
+2 -2
View File
@@ -90,7 +90,7 @@ Automatically generated HTML file from DocOnce source
('Towards the PCA theorem', 2, None, '___sec15'),
('The Algorithm before the Theorem', 2, None, '___sec16'),
('Classical PCA Theorem', 2, None, '___sec17'),
('Prof of the PCA Theorem', 2, None, '___sec18'),
('Proof of the PCA Theorem', 2, None, '___sec18'),
('Getting started with PCA', 2, None, '___sec19'),
('Principal Component Analysis', 2, None, '___sec20'),
('PCA and scikit-learn', 2, None, '___sec21'),
@@ -155,7 +155,7 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._DimRed-bs016.html#___sec15" style="font-size: 80%;">Towards the PCA theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs017.html#___sec16" style="font-size: 80%;">The Algorithm before the Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs018.html#___sec17" style="font-size: 80%;">Classical PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Prof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Proof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs020.html#___sec19" style="font-size: 80%;">Getting started with PCA</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs021.html#___sec20" style="font-size: 80%;">Principal Component Analysis</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs022.html#___sec21" style="font-size: 80%;">PCA and scikit-learn</a></li>
doc/pub/DimRed/html/._DimRed-bs027.html
+2 -2
View File
@@ -90,7 +90,7 @@ Automatically generated HTML file from DocOnce source
('Towards the PCA theorem', 2, None, '___sec15'),
('The Algorithm before the Theorem', 2, None, '___sec16'),
('Classical PCA Theorem', 2, None, '___sec17'),
('Prof of the PCA Theorem', 2, None, '___sec18'),
('Proof of the PCA Theorem', 2, None, '___sec18'),
('Getting started with PCA', 2, None, '___sec19'),
('Principal Component Analysis', 2, None, '___sec20'),
('PCA and scikit-learn', 2, None, '___sec21'),
@@ -155,7 +155,7 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._DimRed-bs016.html#___sec15" style="font-size: 80%;">Towards the PCA theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs017.html#___sec16" style="font-size: 80%;">The Algorithm before the Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs018.html#___sec17" style="font-size: 80%;">Classical PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Prof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Proof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs020.html#___sec19" style="font-size: 80%;">Getting started with PCA</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs021.html#___sec20" style="font-size: 80%;">Principal Component Analysis</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs022.html#___sec21" style="font-size: 80%;">PCA and scikit-learn</a></li>
doc/pub/DimRed/html/DimRed-bs.html
+2 -2
View File
@@ -90,7 +90,7 @@ Automatically generated HTML file from DocOnce source
('Towards the PCA theorem', 2, None, '___sec15'),
('The Algorithm before the Theorem', 2, None, '___sec16'),
('Classical PCA Theorem', 2, None, '___sec17'),
('Prof of the PCA Theorem', 2, None, '___sec18'),
('Proof of the PCA Theorem', 2, None, '___sec18'),
('Getting started with PCA', 2, None, '___sec19'),
('Principal Component Analysis', 2, None, '___sec20'),
('PCA and scikit-learn', 2, None, '___sec21'),
@@ -155,7 +155,7 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._DimRed-bs016.html#___sec15" style="font-size: 80%;">Towards the PCA theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs017.html#___sec16" style="font-size: 80%;">The Algorithm before the Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs018.html#___sec17" style="font-size: 80%;">Classical PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Prof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs019.html#___sec18" style="font-size: 80%;">Proof of the PCA Theorem</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs020.html#___sec19" style="font-size: 80%;">Getting started with PCA</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs021.html#___sec20" style="font-size: 80%;">Principal Component Analysis</a></li>
<!-- navigation toc: --> <li><a href="._DimRed-bs022.html#___sec21" style="font-size: 80%;">PCA and scikit-learn</a></li>
doc/pub/DimRed/html/DimRed-reveal.html
+44 -2
View File
@@ -977,7 +977,7 @@ $$
<ul>
<p><li> Center the data by subtracting the mean value for each column. This leads to a new matrix \( \boldsymbol{X}\rightarrow \overline{\boldsymbol{X}} \).</li>
<p><li> Compute then the covariance/correlation matrix $\mathbb{E}[\overline{\boldsymbol{X}}\overline{\boldsymbol{X}}^T].</li>
<p><li> Compute then the covariance/correlation matrix \( \mathbb{E}[\overline{\boldsymbol{X}}\overline{\boldsymbol{X}}^T] \).</li>
<p><li> Find the eigenpairs of \( \boldsymbol{C} \) with eigenvalues \( [\lambda_0,\lambda_1,\dots,\lambda_{p-1}] \) and eigenvectors \( [\boldsymbol{s}_0,\boldsymbol{s}_1,\dots,\boldsymbol{s}_{p-1}] \).</li>
<p><li> Order the eigenvalue (and the eigenvectors accordingly) in order of decreasing eigenvalues.</li>
<p><li> Keep only those \( l \) eigenvalues larger than a selected threshold value, discarding thus \( p-l \) features since we expect small variations in the data here.</li>
@@ -990,11 +990,53 @@ After this we ask ourselves how do we prove the link between the maximum varianc
<section>
<h2 id="___sec17">Classical PCA Theorem </h2>
<p>
We assume now that we have a design matrix \( \boldsymbol{X} \) which has been centered as discussed above. For the sake of simplicity we skip the overline symbol. The matrix is defined in terms of the various column vectors \( [\boldsymbol{x}_0,\boldsymbol{x}_1,\dots, \boldsymbol{x}_{p-1}] \)
each with dimension \( \boldsymbol{x}\in {\mathbb{R}}^{n} \).
<p>
We assume also that we have an orthogonal transformation \( \boldsymbol{W}\in {\mathbb{R}}^{p\times p} \). We define the reconstruction error (which is similar to the mean squared error we have seen before) as
<p>&nbsp;<br>
$$
J(\boldsymbol{W},\boldsymbol{Z}) = \frac{1}{p}\sum_i (\boldsymbol{x}_i - \overline{\boldsymbol{x}_i})^2,
$$
<p>&nbsp;<br>
with \( \overline{\boldsymbol{x}_i} = \boldsymbol{W}\boldsymbol{z}_i \), where \( \boldsymbol{z}_i \) is a row vector with dimension \( {\mathbb{R}}^{n} \) of the matrix
\( \boldsymbol{Z}\in {\mathbb{R}}^{p\times n} \).
<p>
The PCA theorem states that minimizing the above reconstruction error corresponds to setting \( \boldsymbol{W}=\boldsymbol{S} \), the orthogonal matrix which diagonalizes the empirical covariance(correlation) matrix. The optimal low-dimensional encoding of the data is then given by a set of vectors \( l \) $\boldsymbol{z}_i$, with \( l < < p \), defined by the orthogonal projection of the data onto the columns spanned by they eigenvectors of the covariance(correlations matrix).
</section>
<section>
<h2 id="___sec18">Prof of the PCA Theorem </h2>
<h2 id="___sec18">Proof of the PCA Theorem </h2>
<p>
To show the PCA theorem let us start with the assumption that there is a vector \( \boldsymbol{w}_0 \) which corresponds to a solution which minimized the reconstruction error \( J \). This is an orthogonal vector. It means that we now approximate the reconstruction error in terms of \( \boldsymbol{w}_0 \) and \( \boldsymbol{z}_0 \) as
<p>&nbsp;<br>
$$
J(\boldsymbol{w}_0,\boldsymbol{z}_0)= \frac{1}{p}\sum_i (\boldsymbol{x}_i - z_{i0}\boldsymbol{w}_0)^2=\frac{1}{p}\sum_i (\boldsymbol{x}_^T\boldsymbol{x}_i - 2z_{i0}\boldsymbol{w}_0^T\boldsymbol{x}_i+z_{i0}^2\boldsymbol{w}_0^T\boldsymbol{w}_0),
$$
<p>&nbsp;<br>
which we can rewrite due to the orthogonality of \( \boldsymbol{w}_i \) as
<p>&nbsp;<br>
$$
J(\boldsymbol{w}_0,\boldsymbol{z}_0)=\frac{1}{p}\sum_i (\boldsymbol{x}_^T\boldsymbol{x}_i - 2z_{i0}\boldsymbol{w}_0^T\boldsymbol{x}_i+z_{i0}^2).
$$
<p>&nbsp;<br>
Minimizing \( J \) with respect to the unknown parameters \( z_{0i} \) we obtain that
<p>&nbsp;<br>
$$
z_{i0}=\boldsymbol{w}_0^T\boldsymbol{x}_i,
$$
<p>&nbsp;<br>
where the vectors on the rhs are known.
</section>
doc/pub/DimRed/html/DimRed-solarized.html
+37 -3
View File
@@ -110,7 +110,7 @@ div { text-align: justify; text-justify: inter-word; }
('Towards the PCA theorem', 2, None, '___sec15'),
('The Algorithm before the Theorem', 2, None, '___sec16'),
('Classical PCA Theorem', 2, None, '___sec17'),
('Prof of the PCA Theorem', 2, None, '___sec18'),
('Proof of the PCA Theorem', 2, None, '___sec18'),
('Getting started with PCA', 2, None, '___sec19'),
('Principal Component Analysis', 2, None, '___sec20'),
('PCA and scikit-learn', 2, None, '___sec21'),
@@ -942,7 +942,7 @@ $$
<ul>
<li> Center the data by subtracting the mean value for each column. This leads to a new matrix \( \boldsymbol{X}\rightarrow \overline{\boldsymbol{X}} \).</li>
<li> Compute then the covariance/correlation matrix $\mathbb{E}[\overline{\boldsymbol{X}}\overline{\boldsymbol{X}}^T].</li>
<li> Compute then the covariance/correlation matrix \( \mathbb{E}[\overline{\boldsymbol{X}}\overline{\boldsymbol{X}}^T] \).</li>
<li> Find the eigenpairs of \( \boldsymbol{C} \) with eigenvalues \( [\lambda_0,\lambda_1,\dots,\lambda_{p-1}] \) and eigenvectors \( [\boldsymbol{s}_0,\boldsymbol{s}_1,\dots,\boldsymbol{s}_{p-1}] \).</li>
<li> Order the eigenvalue (and the eigenvectors accordingly) in order of decreasing eigenvalues.</li>
<li> Keep only those \( l \) eigenvalues larger than a selected threshold value, discarding thus \( p-l \) features since we expect small variations in the data here.</li>
@@ -955,10 +955,44 @@ After this we ask ourselves how do we prove the link between the maximum varianc
<h2 id="___sec17">Classical PCA Theorem </h2>
<p>
We assume now that we have a design matrix \( \boldsymbol{X} \) which has been centered as discussed above. For the sake of simplicity we skip the overline symbol. The matrix is defined in terms of the various column vectors \( [\boldsymbol{x}_0,\boldsymbol{x}_1,\dots, \boldsymbol{x}_{p-1}] \)
each with dimension \( \boldsymbol{x}\in {\mathbb{R}}^{n} \).
<p>
We assume also that we have an orthogonal transformation \( \boldsymbol{W}\in {\mathbb{R}}^{p\times p} \). We define the reconstruction error (which is similar to the mean squared error we have seen before) as
$$
J(\boldsymbol{W},\boldsymbol{Z}) = \frac{1}{p}\sum_i (\boldsymbol{x}_i - \overline{\boldsymbol{x}_i})^2,
$$
with \( \overline{\boldsymbol{x}_i} = \boldsymbol{W}\boldsymbol{z}_i \), where \( \boldsymbol{z}_i \) is a row vector with dimension \( {\mathbb{R}}^{n} \) of the matrix
\( \boldsymbol{Z}\in {\mathbb{R}}^{p\times n} \).
<p>
The PCA theorem states that minimizing the above reconstruction error corresponds to setting \( \boldsymbol{W}=\boldsymbol{S} \), the orthogonal matrix which diagonalizes the empirical covariance(correlation) matrix. The optimal low-dimensional encoding of the data is then given by a set of vectors \( l \) $\boldsymbol{z}_i$, with \( l < < p \), defined by the orthogonal projection of the data onto the columns spanned by they eigenvectors of the covariance(correlations matrix).
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec18">Prof of the PCA Theorem </h2>
<h2 id="___sec18">Proof of the PCA Theorem </h2>
<p>
To show the PCA theorem let us start with the assumption that there is a vector \( \boldsymbol{w}_0 \) which corresponds to a solution which minimized the reconstruction error \( J \). This is an orthogonal vector. It means that we now approximate the reconstruction error in terms of \( \boldsymbol{w}_0 \) and \( \boldsymbol{z}_0 \) as
$$
J(\boldsymbol{w}_0,\boldsymbol{z}_0)= \frac{1}{p}\sum_i (\boldsymbol{x}_i - z_{i0}\boldsymbol{w}_0)^2=\frac{1}{p}\sum_i (\boldsymbol{x}_^T\boldsymbol{x}_i - 2z_{i0}\boldsymbol{w}_0^T\boldsymbol{x}_i+z_{i0}^2\boldsymbol{w}_0^T\boldsymbol{w}_0),
$$
which we can rewrite due to the orthogonality of \( \boldsymbol{w}_i \) as
$$
J(\boldsymbol{w}_0,\boldsymbol{z}_0)=\frac{1}{p}\sum_i (\boldsymbol{x}_^T\boldsymbol{x}_i - 2z_{i0}\boldsymbol{w}_0^T\boldsymbol{x}_i+z_{i0}^2).
$$
Minimizing \( J \) with respect to the unknown parameters \( z_{0i} \) we obtain that
$$
z_{i0}=\boldsymbol{w}_0^T\boldsymbol{x}_i,
$$
where the vectors on the rhs are known.
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
doc/pub/DimRed/html/DimRed.html
+37 -3
View File
@@ -115,7 +115,7 @@ div { text-align: justify; text-justify: inter-word; }
('Towards the PCA theorem', 2, None, '___sec15'),
('The Algorithm before the Theorem', 2, None, '___sec16'),
('Classical PCA Theorem', 2, None, '___sec17'),
('Prof of the PCA Theorem', 2, None, '___sec18'),
('Proof of the PCA Theorem', 2, None, '___sec18'),
('Getting started with PCA', 2, None, '___sec19'),
('Principal Component Analysis', 2, None, '___sec20'),
('PCA and scikit-learn', 2, None, '___sec21'),
@@ -947,7 +947,7 @@ $$
<ul>
<li> Center the data by subtracting the mean value for each column. This leads to a new matrix \( \boldsymbol{X}\rightarrow \overline{\boldsymbol{X}} \).</li>
<li> Compute then the covariance/correlation matrix $\mathbb{E}[\overline{\boldsymbol{X}}\overline{\boldsymbol{X}}^T].</li>
<li> Compute then the covariance/correlation matrix \( \mathbb{E}[\overline{\boldsymbol{X}}\overline{\boldsymbol{X}}^T] \).</li>
<li> Find the eigenpairs of \( \boldsymbol{C} \) with eigenvalues \( [\lambda_0,\lambda_1,\dots,\lambda_{p-1}] \) and eigenvectors \( [\boldsymbol{s}_0,\boldsymbol{s}_1,\dots,\boldsymbol{s}_{p-1}] \).</li>
<li> Order the eigenvalue (and the eigenvectors accordingly) in order of decreasing eigenvalues.</li>
<li> Keep only those \( l \) eigenvalues larger than a selected threshold value, discarding thus \( p-l \) features since we expect small variations in the data here.</li>
@@ -960,10 +960,44 @@ After this we ask ourselves how do we prove the link between the maximum varianc
<h2 id="___sec17">Classical PCA Theorem </h2>
<p>
We assume now that we have a design matrix \( \boldsymbol{X} \) which has been centered as discussed above. For the sake of simplicity we skip the overline symbol. The matrix is defined in terms of the various column vectors \( [\boldsymbol{x}_0,\boldsymbol{x}_1,\dots, \boldsymbol{x}_{p-1}] \)
each with dimension \( \boldsymbol{x}\in {\mathbb{R}}^{n} \).
<p>
We assume also that we have an orthogonal transformation \( \boldsymbol{W}\in {\mathbb{R}}^{p\times p} \). We define the reconstruction error (which is similar to the mean squared error we have seen before) as
$$
J(\boldsymbol{W},\boldsymbol{Z}) = \frac{1}{p}\sum_i (\boldsymbol{x}_i - \overline{\boldsymbol{x}_i})^2,
$$
with \( \overline{\boldsymbol{x}_i} = \boldsymbol{W}\boldsymbol{z}_i \), where \( \boldsymbol{z}_i \) is a row vector with dimension \( {\mathbb{R}}^{n} \) of the matrix
\( \boldsymbol{Z}\in {\mathbb{R}}^{p\times n} \).
<p>
The PCA theorem states that minimizing the above reconstruction error corresponds to setting \( \boldsymbol{W}=\boldsymbol{S} \), the orthogonal matrix which diagonalizes the empirical covariance(correlation) matrix. The optimal low-dimensional encoding of the data is then given by a set of vectors \( l \) $\boldsymbol{z}_i$, with \( l < < p \), defined by the orthogonal projection of the data onto the columns spanned by they eigenvectors of the covariance(correlations matrix).
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec18">Prof of the PCA Theorem </h2>
<h2 id="___sec18">Proof of the PCA Theorem </h2>
<p>
To show the PCA theorem let us start with the assumption that there is a vector \( \boldsymbol{w}_0 \) which corresponds to a solution which minimized the reconstruction error \( J \). This is an orthogonal vector. It means that we now approximate the reconstruction error in terms of \( \boldsymbol{w}_0 \) and \( \boldsymbol{z}_0 \) as
$$
J(\boldsymbol{w}_0,\boldsymbol{z}_0)= \frac{1}{p}\sum_i (\boldsymbol{x}_i - z_{i0}\boldsymbol{w}_0)^2=\frac{1}{p}\sum_i (\boldsymbol{x}_^T\boldsymbol{x}_i - 2z_{i0}\boldsymbol{w}_0^T\boldsymbol{x}_i+z_{i0}^2\boldsymbol{w}_0^T\boldsymbol{w}_0),
$$
which we can rewrite due to the orthogonality of \( \boldsymbol{w}_i \) as
$$
J(\boldsymbol{w}_0,\boldsymbol{z}_0)=\frac{1}{p}\sum_i (\boldsymbol{x}_^T\boldsymbol{x}_i - 2z_{i0}\boldsymbol{w}_0^T\boldsymbol{x}_i+z_{i0}^2).
$$
Minimizing \( J \) with respect to the unknown parameters \( z_{0i} \) we obtain that
$$
z_{i0}=\boldsymbol{w}_0^T\boldsymbol{x}_i,
$$
where the vectors on the rhs are known.
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
doc/pub/DimRed/ipynb/DimRed.ipynb
+72 -3
View File
@@ -1041,7 +1041,7 @@
"source": [
"* Center the data by subtracting the mean value for each column. This leads to a new matrix $\\boldsymbol{X}\\rightarrow \\overline{\\boldsymbol{X}}$.\n",
"\n",
"* Compute then the covariance/correlation matrix $\\mathbb{E}[\\overline{\\boldsymbol{X}}\\overline{\\boldsymbol{X}}^T].\n",
"* Compute then the covariance/correlation matrix $\\mathbb{E}[\\overline{\\boldsymbol{X}}\\overline{\\boldsymbol{X}}^T]$.\n",
"\n",
"* Find the eigenpairs of $\\boldsymbol{C}$ with eigenvalues $[\\lambda_0,\\lambda_1,\\dots,\\lambda_{p-1}]$ and eigenvectors $[\\boldsymbol{s}_0,\\boldsymbol{s}_1,\\dots,\\boldsymbol{s}_{p-1}]$.\n",
"\n",
@@ -1053,14 +1053,83 @@
"\n",
"## Classical PCA Theorem\n",
"\n",
"We assume now that we have a design matrix $\\boldsymbol{X}$ which has been centered as discussed above. For the sake of simplicity we skip the overline symbol. The matrix is defined in terms of the various column vectors $[\\boldsymbol{x}_0,\\boldsymbol{x}_1,\\dots, \\boldsymbol{x}_{p-1}]$\n",
"each with dimension $\\boldsymbol{x}\\in {\\mathbb{R}}^{n}$.\n",
"\n",
"We assume also that we have an orthogonal transformation $\\boldsymbol{W}\\in {\\mathbb{R}}^{p\\times p}$. We define the reconstruction error (which is similar to the mean squared error we have seen before) as"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"$$\n",
"J(\\boldsymbol{W},\\boldsymbol{Z}) = \\frac{1}{p}\\sum_i (\\boldsymbol{x}_i - \\overline{\\boldsymbol{x}_i})^2,\n",
"$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"with $\\overline{\\boldsymbol{x}_i} = \\boldsymbol{W}\\boldsymbol{z}_i$, where $\\boldsymbol{z}_i$ is a row vector with dimension ${\\mathbb{R}}^{n}$ of the matrix\n",
"$\\boldsymbol{Z}\\in {\\mathbb{R}}^{p\\times n}$. \n",
"\n",
"## Prof of the PCA Theorem\n",
"\n",
"The PCA theorem states that minimizing the above reconstruction error corresponds to setting $\\boldsymbol{W}=\\boldsymbol{S}$, the orthogonal matrix which diagonalizes the empirical covariance(correlation) matrix. The optimal low-dimensional encoding of the data is then given by a set of vectors $l$ $\\boldsymbol{z}_i$, with $l << p$, defined by the orthogonal projection of the data onto the columns spanned by they eigenvectors of the covariance(correlations matrix).\n",
"\n",
"\n",
"\n",
"## Proof of the PCA Theorem\n",
"\n",
"To show the PCA theorem let us start with the assumption that there is a vector $\\boldsymbol{w}_0$ which corresponds to a solution which minimized the reconstruction error $J$. This is an orthogonal vector. It means that we now approximate the reconstruction error in terms of $\\boldsymbol{w}_0$ and $\\boldsymbol{z}_0$ as"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"$$\n",
"J(\\boldsymbol{w}_0,\\boldsymbol{z}_0)= \\frac{1}{p}\\sum_i (\\boldsymbol{x}_i - z_{i0}\\boldsymbol{w}_0)^2=\\frac{1}{p}\\sum_i (\\boldsymbol{x}_^T\\boldsymbol{x}_i - 2z_{i0}\\boldsymbol{w}_0^T\\boldsymbol{x}_i+z_{i0}^2\\boldsymbol{w}_0^T\\boldsymbol{w}_0),\n",
"$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"which we can rewrite due to the orthogonality of $\\boldsymbol{w}_i$ as"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"$$\n",
"J(\\boldsymbol{w}_0,\\boldsymbol{z}_0)=\\frac{1}{p}\\sum_i (\\boldsymbol{x}_^T\\boldsymbol{x}_i - 2z_{i0}\\boldsymbol{w}_0^T\\boldsymbol{x}_i+z_{i0}^2).\n",
"$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Minimizing $J$ with respect to the unknown parameters $z_{0i}$ we obtain that"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"$$\n",
"z_{i0}=\\boldsymbol{w}_0^T\\boldsymbol{x}_i,\n",
"$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"where the vectors on the rhs are known. \n",
"\n",
"## Getting started with PCA"
]
doc/pub/DimRed/ipynb/ipynb-DimRed-src.tar.gz
BIN
View File
Binary file not shown.
doc/pub/DimRed/pdf/DimRed-minted.pdf
BIN
View File
Binary file not shown.
doc/src/DimRed/DimRed.do.txt
+35 -6
View File
@@ -734,7 +734,7 @@ x_{n-1,0} & x_{n-1,1} & x_{n-1,2}& \dots & \dots x_{n-1,p-1}\\
\]
!et
* Center the data by subtracting the mean value for each column. This leads to a new matrix $\bm{X}\rightarrow \overline{\bm{X}}$.
* Compute then the covariance/correlation matrix $\mathbb{E}[\overline{\bm{X}}\overline{\bm{X}}^T].
* Compute then the covariance/correlation matrix $\mathbb{E}[\overline{\bm{X}}\overline{\bm{X}}^T]$.
* Find the eigenpairs of $\bm{C}$ with eigenvalues $[\lambda_0,\lambda_1,\dots,\lambda_{p-1}]$ and eigenvectors $[\bm{s}_0,\bm{s}_1,\dots,\bm{s}_{p-1}]$.
* Order the eigenvalue (and the eigenvectors accordingly) in order of decreasing eigenvalues.
* Keep only those $l$ eigenvalues larger than a selected threshold value, discarding thus $p-l$ features since we expect small variations in the data here.
@@ -744,15 +744,44 @@ After this we ask ourselves how do we prove the link between the maximum varianc
!split
===== Classical PCA Theorem =====
We assume now that we have a design matrix $\bm{X}$ which has been centered as discussed above. For the sake of simplicity we skip the overline symbol. The matrix is defined in terms of the various column vectors $[\bm{x}_0,\bm{x}_1,\dots, \bm{x}_{p-1}]$
each with dimension $\bm{x}\in {\mathbb{R}}^{n}$.
We assume also that we have an orthogonal transformation $\bm{W}\in {\mathbb{R}}^{p\times p}$. We define the reconstruction error (which is similar to the mean squared error we have seen before) as
!bt
\[
J(\bm{W},\bm{Z}) = \frac{1}{p}\sum_i (\bm{x}_i - \overline{\bm{x}_i})^2,
\]
!et
with $\overline{\bm{x}_i} = \bm{W}\bm{z}_i$, where $\bm{z}_i$ is a row vector with dimension ${\mathbb{R}}^{n}$ of the matrix
$\bm{Z}\in {\mathbb{R}}^{p\times n}$.
The PCA theorem states that minimizing the above reconstruction error corresponds to setting $\bm{W}=\bm{S}$, the orthogonal matrix which diagonalizes the empirical covariance(correlation) matrix. The optimal low-dimensional encoding of the data is then given by a set of vectors $l$ $\bm{z}_i$, with $l << p$, defined by the orthogonal projection of the data onto the columns spanned by they eigenvectors of the covariance(correlations matrix).
!split
===== Prof of the PCA Theorem =====
===== Proof of the PCA Theorem =====
To show the PCA theorem let us start with the assumption that there is a vector $\bm{w}_0$ which corresponds to a solution which minimized the reconstruction error $J$. This is an orthogonal vector. It means that we now approximate the reconstruction error in terms of $\bm{w}_0$ and $\bm{z}_0$ as
!bt
\[
J(\bm{w}_0,\bm{z}_0)= \frac{1}{p}\sum_i (\bm{x}_i - z_{i0}\bm{w}_0)^2=\frac{1}{p}\sum_i (\bm{x}_^T\bm{x}_i - 2z_{i0}\bm{w}_0^T\bm{x}_i+z_{i0}^2\bm{w}_0^T\bm{w}_0),
\]
!et
which we can rewrite due to the orthogonality of $\bm{w}_i$ as
!bt
\[
J(\bm{w}_0,\bm{z}_0)=\frac{1}{p}\sum_i (\bm{x}_^T\bm{x}_i - 2z_{i0}\bm{w}_0^T\bm{x}_i+z_{i0}^2).
\]
!et
Minimizing $J$ with respect to the unknown parameters $z_{0i}$ we obtain that
!bt
\[
z_{i0}=\bm{w}_0^T\bm{x}_i,
\]
!et
where the vectors on the rhs are known.
!split
===== Getting started with PCA =====