From 4c4f496202619b962435130ec437a408088dffbd Mon Sep 17 00:00:00 2001 From: mhjensen Date: Thu, 24 Oct 2019 06:24:37 +0200 Subject: [PATCH] more cleaning up --- doc/pub/DimRed/html/._DimRed-bs018.html | 6 ++--- doc/pub/DimRed/html/._DimRed-bs019.html | 4 ++-- doc/pub/DimRed/html/._DimRed-bs020.html | 12 +++++----- doc/pub/DimRed/html/DimRed-reveal.html | 22 +++++++++---------- doc/pub/DimRed/html/DimRed-solarized.html | 22 +++++++++---------- doc/pub/DimRed/html/DimRed.html | 22 +++++++++---------- doc/pub/DimRed/ipynb/DimRed.ipynb | 22 +++++++++---------- doc/pub/DimRed/ipynb/ipynb-DimRed-src.tar.gz | Bin 190 -> 191 bytes doc/pub/DimRed/pdf/DimRed-minted.pdf | Bin 253634 -> 253692 bytes doc/src/DimRed/DimRed.do.txt | 22 +++++++++---------- 10 files changed, 66 insertions(+), 66 deletions(-) diff --git a/doc/pub/DimRed/html/._DimRed-bs018.html b/doc/pub/DimRed/html/._DimRed-bs018.html index c172b2cd4..dcd6f46d7 100644 --- a/doc/pub/DimRed/html/._DimRed-bs018.html +++ b/doc/pub/DimRed/html/._DimRed-bs018.html @@ -193,14 +193,14 @@ each with dimension \( \boldsymbol{x}\in {\mathbb{R}}^{n} \).

We assume also that we have an orthogonal transformation \( \boldsymbol{W}\in {\mathbb{R}}^{p\times p} \). We define the reconstruction error (which is similar to the mean squared error we have seen before) as $$ -J(\boldsymbol{W},\boldsymbol{Z}) = \frac{1}{p}\sum_i (\boldsymbol{x}_i - \overline{\boldsymbol{x}_i})^2, +J(\boldsymbol{W},\boldsymbol{Z}) = \frac{1}{n}\sum_i (\boldsymbol{x}_i - \overline{\boldsymbol{x}_i})^2, $$ with \( \overline{\boldsymbol{x}_i} = \boldsymbol{W}\boldsymbol{z}_i \), where \( \boldsymbol{z}_i \) is a row vector with dimension \( {\mathbb{R}}^{n} \) of the matrix -\( \boldsymbol{Z}\in {\mathbb{R}}^{p\times n} \). When doing PCA we want to reduce this dimensionality. +\( \boldsymbol{Z}\in{\mathbb{R}}^{p\times n} \). When doing PCA we want to reduce this dimensionality.

-The PCA theorem states that minimizing the above reconstruction error corresponds to setting \( \boldsymbol{W}=\boldsymbol{S} \), the orthogonal matrix which diagonalizes the empirical covariance(correlation) matrix. The optimal low-dimensional encoding of the data is then given by a set of vectors \( l \) $\boldsymbol{z}_i$, with \( l < < p \), defined by the orthogonal projection of the data onto the columns spanned by the eigenvectors of the covariance(correlations matrix). +The PCA theorem states that minimizing the above reconstruction error corresponds to setting \( \boldsymbol{W}=\boldsymbol{S} \), the orthogonal matrix which diagonalizes the empirical covariance(correlation) matrix. The optimal low-dimensional encoding of the data is then given by a set of vectors \( \boldsymbol{z}_i \) with at most \( l \) vectors, with \( l < < p \), defined by the orthogonal projection of the data onto the columns spanned by the eigenvectors of the covariance(correlations matrix).

diff --git a/doc/pub/DimRed/html/._DimRed-bs019.html b/doc/pub/DimRed/html/._DimRed-bs019.html index e4746e5e3..363c14d79 100644 --- a/doc/pub/DimRed/html/._DimRed-bs019.html +++ b/doc/pub/DimRed/html/._DimRed-bs019.html @@ -189,12 +189,12 @@ MathJax.Hub.Config({

To show the PCA theorem let us start with the assumption that there is one vector \( \boldsymbol{w}_0 \) which corresponds to a solution which minimized the reconstruction error \( J \). This is an orthogonal vector. It means that we now approximate the reconstruction error in terms of \( \boldsymbol{w}_0 \) and \( \boldsymbol{z}_0 \) as $$ -J(\boldsymbol{w}_0,\boldsymbol{z}_0)= \frac{1}{p}\sum_i (\boldsymbol{x}_i - z_{i0}\boldsymbol{w}_0)^2=\frac{1}{p}\sum_i (\boldsymbol{x}_i^T\boldsymbol{x}_i - 2z_{i0}\boldsymbol{w}_0^T\boldsymbol{x}_i+z_{i0}^2\boldsymbol{w}_0^T\boldsymbol{w}_0), +J(\boldsymbol{w}_0,\boldsymbol{z}_0)= \frac{1}{n}\sum_i (\boldsymbol{x}_i - z_{i0}\boldsymbol{w}_0)^2=\frac{1}{n}\sum_i (\boldsymbol{x}_i^T\boldsymbol{x}_i - 2z_{i0}\boldsymbol{w}_0^T\boldsymbol{x}_i+z_{i0}^2\boldsymbol{w}_0^T\boldsymbol{w}_0), $$ which we can rewrite due to the orthogonality of \( \boldsymbol{w}_i \) as $$ -J(\boldsymbol{w}_0,\boldsymbol{z}_0)=\frac{1}{p}\sum_i (\boldsymbol{x}_i^T\boldsymbol{x}_i - 2z_{i0}\boldsymbol{w}_0^T\boldsymbol{x}_i+z_{i0}^2). +J(\boldsymbol{w}_0,\boldsymbol{z}_0)=\frac{1}{n}\sum_i (\boldsymbol{x}_i^T\boldsymbol{x}_i - 2z_{i0}\boldsymbol{w}_0^T\boldsymbol{x}_i+z_{i0}^2). $$ Minimizing \( J \) with respect to the unknown parameters \( z_{0i} \) we obtain that diff --git a/doc/pub/DimRed/html/._DimRed-bs020.html b/doc/pub/DimRed/html/._DimRed-bs020.html index ec98cd767..1b841aa38 100644 --- a/doc/pub/DimRed/html/._DimRed-bs020.html +++ b/doc/pub/DimRed/html/._DimRed-bs020.html @@ -187,15 +187,15 @@ MathJax.Hub.Config({

PCA Proof continued

-We have now found the unknown parameters \( z_{i0} \). These correspond to the project coordinates and we can write +We have now found the unknown parameters \( z_{i0} \). These correspond to the projected coordinates and we can write $$ -J(\boldsymbol{w}_0)= \frac{1}{p}\sum_i (\boldsymbol{x}_i^T\boldsymbol{x}_i - z_{i0}^2)=\mathrm{const}-\frac{1}{p}\sum_i z_{i0}^2. +J(\boldsymbol{w}_0)= \frac{1}{p}\sum_i (\boldsymbol{x}_i^T\boldsymbol{x}_i - z_{i0}^2)=\mathrm{const}-\frac{1}{n}\sum_i z_{i0}^2. $$

-We can show that the variance of the project coordinates defined by \( \boldsymbol{w}_0^T\boldsymbol{x}_i \) are given by +We can show that the variance of the projected coordinates defined by \( \boldsymbol{w}_0^T\boldsymbol{x}_i \) are given by $$ -\mathrm{var}[\boldsymbol{w}_0^T\boldsymbol{x}_i] = \frac{1}{p}\sum_i z_{i0}^2, +\mathrm{var}[\boldsymbol{w}_0^T\boldsymbol{x}_i] = \frac{1}{n}\sum_i z_{i0}^2, $$ since the expectation value of @@ -208,12 +208,12 @@ where we have used the fact that our data are centered.

Recalling our definition of the covariance as $$ -\boldsymbol{C}[\boldsymbol{x}] = \frac{1}{n}\boldsymbol{X}\boldsymbol{X}^T= \mathbb{E}[\boldsymbol{X}\boldsymbol{X}^T], +\boldsymbol{C}[\boldsymbol{x}] = \frac{1}{n}\boldsymbol{X}\boldsymbol{X}^T=\mathbb{E}[\boldsymbol{X}\boldsymbol{X}^T], $$ we have thus that $$ -\mathrm{var}[\boldsymbol{w}_0^T\boldsymbol{x}_i] = \frac{1}{p}\sum_i z_{i0}^2=\boldsymbol{w}_0^T\boldsymbol{C}[\boldsymbol{x}]\boldsymbol{w}_0. +\mathrm{var}[\boldsymbol{w}_0^T\boldsymbol{x}_i] = \frac{1}{n}\sum_i z_{i0}^2=\boldsymbol{w}_0^T\boldsymbol{C}[\boldsymbol{x}]\boldsymbol{w}_0. $$

diff --git a/doc/pub/DimRed/html/DimRed-reveal.html b/doc/pub/DimRed/html/DimRed-reveal.html index fb83652a5..74eb2f961 100644 --- a/doc/pub/DimRed/html/DimRed-reveal.html +++ b/doc/pub/DimRed/html/DimRed-reveal.html @@ -999,15 +999,15 @@ each with dimension \( \boldsymbol{x}\in {\mathbb{R}}^{n} \). We assume also that we have an orthogonal transformation \( \boldsymbol{W}\in {\mathbb{R}}^{p\times p} \). We define the reconstruction error (which is similar to the mean squared error we have seen before) as

 
$$ -J(\boldsymbol{W},\boldsymbol{Z}) = \frac{1}{p}\sum_i (\boldsymbol{x}_i - \overline{\boldsymbol{x}_i})^2, +J(\boldsymbol{W},\boldsymbol{Z}) = \frac{1}{n}\sum_i (\boldsymbol{x}_i - \overline{\boldsymbol{x}_i})^2, $$

 
with \( \overline{\boldsymbol{x}_i} = \boldsymbol{W}\boldsymbol{z}_i \), where \( \boldsymbol{z}_i \) is a row vector with dimension \( {\mathbb{R}}^{n} \) of the matrix -\( \boldsymbol{Z}\in {\mathbb{R}}^{p\times n} \). When doing PCA we want to reduce this dimensionality. +\( \boldsymbol{Z}\in{\mathbb{R}}^{p\times n} \). When doing PCA we want to reduce this dimensionality.

-The PCA theorem states that minimizing the above reconstruction error corresponds to setting \( \boldsymbol{W}=\boldsymbol{S} \), the orthogonal matrix which diagonalizes the empirical covariance(correlation) matrix. The optimal low-dimensional encoding of the data is then given by a set of vectors \( l \) $\boldsymbol{z}_i$, with \( l < < p \), defined by the orthogonal projection of the data onto the columns spanned by the eigenvectors of the covariance(correlations matrix). +The PCA theorem states that minimizing the above reconstruction error corresponds to setting \( \boldsymbol{W}=\boldsymbol{S} \), the orthogonal matrix which diagonalizes the empirical covariance(correlation) matrix. The optimal low-dimensional encoding of the data is then given by a set of vectors \( \boldsymbol{z}_i \) with at most \( l \) vectors, with \( l < < p \), defined by the orthogonal projection of the data onto the columns spanned by the eigenvectors of the covariance(correlations matrix). @@ -1018,14 +1018,14 @@ The PCA theorem states that minimizing the above reconstruction error correspond To show the PCA theorem let us start with the assumption that there is one vector \( \boldsymbol{w}_0 \) which corresponds to a solution which minimized the reconstruction error \( J \). This is an orthogonal vector. It means that we now approximate the reconstruction error in terms of \( \boldsymbol{w}_0 \) and \( \boldsymbol{z}_0 \) as

 
$$ -J(\boldsymbol{w}_0,\boldsymbol{z}_0)= \frac{1}{p}\sum_i (\boldsymbol{x}_i - z_{i0}\boldsymbol{w}_0)^2=\frac{1}{p}\sum_i (\boldsymbol{x}_i^T\boldsymbol{x}_i - 2z_{i0}\boldsymbol{w}_0^T\boldsymbol{x}_i+z_{i0}^2\boldsymbol{w}_0^T\boldsymbol{w}_0), +J(\boldsymbol{w}_0,\boldsymbol{z}_0)= \frac{1}{n}\sum_i (\boldsymbol{x}_i - z_{i0}\boldsymbol{w}_0)^2=\frac{1}{n}\sum_i (\boldsymbol{x}_i^T\boldsymbol{x}_i - 2z_{i0}\boldsymbol{w}_0^T\boldsymbol{x}_i+z_{i0}^2\boldsymbol{w}_0^T\boldsymbol{w}_0), $$

 
which we can rewrite due to the orthogonality of \( \boldsymbol{w}_i \) as

 
$$ -J(\boldsymbol{w}_0,\boldsymbol{z}_0)=\frac{1}{p}\sum_i (\boldsymbol{x}_i^T\boldsymbol{x}_i - 2z_{i0}\boldsymbol{w}_0^T\boldsymbol{x}_i+z_{i0}^2). +J(\boldsymbol{w}_0,\boldsymbol{z}_0)=\frac{1}{n}\sum_i (\boldsymbol{x}_i^T\boldsymbol{x}_i - 2z_{i0}\boldsymbol{w}_0^T\boldsymbol{x}_i+z_{i0}^2). $$

 
@@ -1044,18 +1044,18 @@ where the vectors on the rhs are known.

PCA Proof continued

-We have now found the unknown parameters \( z_{i0} \). These correspond to the project coordinates and we can write +We have now found the unknown parameters \( z_{i0} \). These correspond to the projected coordinates and we can write

 
$$ -J(\boldsymbol{w}_0)= \frac{1}{p}\sum_i (\boldsymbol{x}_i^T\boldsymbol{x}_i - z_{i0}^2)=\mathrm{const}-\frac{1}{p}\sum_i z_{i0}^2. +J(\boldsymbol{w}_0)= \frac{1}{p}\sum_i (\boldsymbol{x}_i^T\boldsymbol{x}_i - z_{i0}^2)=\mathrm{const}-\frac{1}{n}\sum_i z_{i0}^2. $$

 

-We can show that the variance of the project coordinates defined by \( \boldsymbol{w}_0^T\boldsymbol{x}_i \) are given by +We can show that the variance of the projected coordinates defined by \( \boldsymbol{w}_0^T\boldsymbol{x}_i \) are given by

 
$$ -\mathrm{var}[\boldsymbol{w}_0^T\boldsymbol{x}_i] = \frac{1}{p}\sum_i z_{i0}^2, +\mathrm{var}[\boldsymbol{w}_0^T\boldsymbol{x}_i] = \frac{1}{n}\sum_i z_{i0}^2, $$

 
@@ -1072,14 +1072,14 @@ where we have used the fact that our data are centered. Recalling our definition of the covariance as

 
$$ -\boldsymbol{C}[\boldsymbol{x}] = \frac{1}{n}\boldsymbol{X}\boldsymbol{X}^T= \mathbb{E}[\boldsymbol{X}\boldsymbol{X}^T], +\boldsymbol{C}[\boldsymbol{x}] = \frac{1}{n}\boldsymbol{X}\boldsymbol{X}^T=\mathbb{E}[\boldsymbol{X}\boldsymbol{X}^T], $$

 
we have thus that

 
$$ -\mathrm{var}[\boldsymbol{w}_0^T\boldsymbol{x}_i] = \frac{1}{p}\sum_i z_{i0}^2=\boldsymbol{w}_0^T\boldsymbol{C}[\boldsymbol{x}]\boldsymbol{w}_0. +\mathrm{var}[\boldsymbol{w}_0^T\boldsymbol{x}_i] = \frac{1}{n}\sum_i z_{i0}^2=\boldsymbol{w}_0^T\boldsymbol{C}[\boldsymbol{x}]\boldsymbol{w}_0. $$

 
diff --git a/doc/pub/DimRed/html/DimRed-solarized.html b/doc/pub/DimRed/html/DimRed-solarized.html index d48c7a463..249cb656a 100644 --- a/doc/pub/DimRed/html/DimRed-solarized.html +++ b/doc/pub/DimRed/html/DimRed-solarized.html @@ -964,14 +964,14 @@ each with dimension \( \boldsymbol{x}\in {\mathbb{R}}^{n} \).

We assume also that we have an orthogonal transformation \( \boldsymbol{W}\in {\mathbb{R}}^{p\times p} \). We define the reconstruction error (which is similar to the mean squared error we have seen before) as $$ -J(\boldsymbol{W},\boldsymbol{Z}) = \frac{1}{p}\sum_i (\boldsymbol{x}_i - \overline{\boldsymbol{x}_i})^2, +J(\boldsymbol{W},\boldsymbol{Z}) = \frac{1}{n}\sum_i (\boldsymbol{x}_i - \overline{\boldsymbol{x}_i})^2, $$ with \( \overline{\boldsymbol{x}_i} = \boldsymbol{W}\boldsymbol{z}_i \), where \( \boldsymbol{z}_i \) is a row vector with dimension \( {\mathbb{R}}^{n} \) of the matrix -\( \boldsymbol{Z}\in {\mathbb{R}}^{p\times n} \). When doing PCA we want to reduce this dimensionality. +\( \boldsymbol{Z}\in{\mathbb{R}}^{p\times n} \). When doing PCA we want to reduce this dimensionality.

-The PCA theorem states that minimizing the above reconstruction error corresponds to setting \( \boldsymbol{W}=\boldsymbol{S} \), the orthogonal matrix which diagonalizes the empirical covariance(correlation) matrix. The optimal low-dimensional encoding of the data is then given by a set of vectors \( l \) $\boldsymbol{z}_i$, with \( l < < p \), defined by the orthogonal projection of the data onto the columns spanned by the eigenvectors of the covariance(correlations matrix). +The PCA theorem states that minimizing the above reconstruction error corresponds to setting \( \boldsymbol{W}=\boldsymbol{S} \), the orthogonal matrix which diagonalizes the empirical covariance(correlation) matrix. The optimal low-dimensional encoding of the data is then given by a set of vectors \( \boldsymbol{z}_i \) with at most \( l \) vectors, with \( l < < p \), defined by the orthogonal projection of the data onto the columns spanned by the eigenvectors of the covariance(correlations matrix).











@@ -981,12 +981,12 @@ The PCA theorem states that minimizing the above reconstruction error correspond

To show the PCA theorem let us start with the assumption that there is one vector \( \boldsymbol{w}_0 \) which corresponds to a solution which minimized the reconstruction error \( J \). This is an orthogonal vector. It means that we now approximate the reconstruction error in terms of \( \boldsymbol{w}_0 \) and \( \boldsymbol{z}_0 \) as $$ -J(\boldsymbol{w}_0,\boldsymbol{z}_0)= \frac{1}{p}\sum_i (\boldsymbol{x}_i - z_{i0}\boldsymbol{w}_0)^2=\frac{1}{p}\sum_i (\boldsymbol{x}_i^T\boldsymbol{x}_i - 2z_{i0}\boldsymbol{w}_0^T\boldsymbol{x}_i+z_{i0}^2\boldsymbol{w}_0^T\boldsymbol{w}_0), +J(\boldsymbol{w}_0,\boldsymbol{z}_0)= \frac{1}{n}\sum_i (\boldsymbol{x}_i - z_{i0}\boldsymbol{w}_0)^2=\frac{1}{n}\sum_i (\boldsymbol{x}_i^T\boldsymbol{x}_i - 2z_{i0}\boldsymbol{w}_0^T\boldsymbol{x}_i+z_{i0}^2\boldsymbol{w}_0^T\boldsymbol{w}_0), $$ which we can rewrite due to the orthogonality of \( \boldsymbol{w}_i \) as $$ -J(\boldsymbol{w}_0,\boldsymbol{z}_0)=\frac{1}{p}\sum_i (\boldsymbol{x}_i^T\boldsymbol{x}_i - 2z_{i0}\boldsymbol{w}_0^T\boldsymbol{x}_i+z_{i0}^2). +J(\boldsymbol{w}_0,\boldsymbol{z}_0)=\frac{1}{n}\sum_i (\boldsymbol{x}_i^T\boldsymbol{x}_i - 2z_{i0}\boldsymbol{w}_0^T\boldsymbol{x}_i+z_{i0}^2). $$ Minimizing \( J \) with respect to the unknown parameters \( z_{0i} \) we obtain that @@ -1002,15 +1002,15 @@ where the vectors on the rhs are known.

PCA Proof continued

-We have now found the unknown parameters \( z_{i0} \). These correspond to the project coordinates and we can write +We have now found the unknown parameters \( z_{i0} \). These correspond to the projected coordinates and we can write $$ -J(\boldsymbol{w}_0)= \frac{1}{p}\sum_i (\boldsymbol{x}_i^T\boldsymbol{x}_i - z_{i0}^2)=\mathrm{const}-\frac{1}{p}\sum_i z_{i0}^2. +J(\boldsymbol{w}_0)= \frac{1}{p}\sum_i (\boldsymbol{x}_i^T\boldsymbol{x}_i - z_{i0}^2)=\mathrm{const}-\frac{1}{n}\sum_i z_{i0}^2. $$

-We can show that the variance of the project coordinates defined by \( \boldsymbol{w}_0^T\boldsymbol{x}_i \) are given by +We can show that the variance of the projected coordinates defined by \( \boldsymbol{w}_0^T\boldsymbol{x}_i \) are given by $$ -\mathrm{var}[\boldsymbol{w}_0^T\boldsymbol{x}_i] = \frac{1}{p}\sum_i z_{i0}^2, +\mathrm{var}[\boldsymbol{w}_0^T\boldsymbol{x}_i] = \frac{1}{n}\sum_i z_{i0}^2, $$ since the expectation value of @@ -1023,12 +1023,12 @@ where we have used the fact that our data are centered.

Recalling our definition of the covariance as $$ -\boldsymbol{C}[\boldsymbol{x}] = \frac{1}{n}\boldsymbol{X}\boldsymbol{X}^T= \mathbb{E}[\boldsymbol{X}\boldsymbol{X}^T], +\boldsymbol{C}[\boldsymbol{x}] = \frac{1}{n}\boldsymbol{X}\boldsymbol{X}^T=\mathbb{E}[\boldsymbol{X}\boldsymbol{X}^T], $$ we have thus that $$ -\mathrm{var}[\boldsymbol{w}_0^T\boldsymbol{x}_i] = \frac{1}{p}\sum_i z_{i0}^2=\boldsymbol{w}_0^T\boldsymbol{C}[\boldsymbol{x}]\boldsymbol{w}_0. +\mathrm{var}[\boldsymbol{w}_0^T\boldsymbol{x}_i] = \frac{1}{n}\sum_i z_{i0}^2=\boldsymbol{w}_0^T\boldsymbol{C}[\boldsymbol{x}]\boldsymbol{w}_0. $$

diff --git a/doc/pub/DimRed/html/DimRed.html b/doc/pub/DimRed/html/DimRed.html index effe9e5c9..b13959662 100644 --- a/doc/pub/DimRed/html/DimRed.html +++ b/doc/pub/DimRed/html/DimRed.html @@ -969,14 +969,14 @@ each with dimension \( \boldsymbol{x}\in {\mathbb{R}}^{n} \).

We assume also that we have an orthogonal transformation \( \boldsymbol{W}\in {\mathbb{R}}^{p\times p} \). We define the reconstruction error (which is similar to the mean squared error we have seen before) as $$ -J(\boldsymbol{W},\boldsymbol{Z}) = \frac{1}{p}\sum_i (\boldsymbol{x}_i - \overline{\boldsymbol{x}_i})^2, +J(\boldsymbol{W},\boldsymbol{Z}) = \frac{1}{n}\sum_i (\boldsymbol{x}_i - \overline{\boldsymbol{x}_i})^2, $$ with \( \overline{\boldsymbol{x}_i} = \boldsymbol{W}\boldsymbol{z}_i \), where \( \boldsymbol{z}_i \) is a row vector with dimension \( {\mathbb{R}}^{n} \) of the matrix -\( \boldsymbol{Z}\in {\mathbb{R}}^{p\times n} \). When doing PCA we want to reduce this dimensionality. +\( \boldsymbol{Z}\in{\mathbb{R}}^{p\times n} \). When doing PCA we want to reduce this dimensionality.

-The PCA theorem states that minimizing the above reconstruction error corresponds to setting \( \boldsymbol{W}=\boldsymbol{S} \), the orthogonal matrix which diagonalizes the empirical covariance(correlation) matrix. The optimal low-dimensional encoding of the data is then given by a set of vectors \( l \) $\boldsymbol{z}_i$, with \( l < < p \), defined by the orthogonal projection of the data onto the columns spanned by the eigenvectors of the covariance(correlations matrix). +The PCA theorem states that minimizing the above reconstruction error corresponds to setting \( \boldsymbol{W}=\boldsymbol{S} \), the orthogonal matrix which diagonalizes the empirical covariance(correlation) matrix. The optimal low-dimensional encoding of the data is then given by a set of vectors \( \boldsymbol{z}_i \) with at most \( l \) vectors, with \( l < < p \), defined by the orthogonal projection of the data onto the columns spanned by the eigenvectors of the covariance(correlations matrix).











@@ -986,12 +986,12 @@ The PCA theorem states that minimizing the above reconstruction error correspond

To show the PCA theorem let us start with the assumption that there is one vector \( \boldsymbol{w}_0 \) which corresponds to a solution which minimized the reconstruction error \( J \). This is an orthogonal vector. It means that we now approximate the reconstruction error in terms of \( \boldsymbol{w}_0 \) and \( \boldsymbol{z}_0 \) as $$ -J(\boldsymbol{w}_0,\boldsymbol{z}_0)= \frac{1}{p}\sum_i (\boldsymbol{x}_i - z_{i0}\boldsymbol{w}_0)^2=\frac{1}{p}\sum_i (\boldsymbol{x}_i^T\boldsymbol{x}_i - 2z_{i0}\boldsymbol{w}_0^T\boldsymbol{x}_i+z_{i0}^2\boldsymbol{w}_0^T\boldsymbol{w}_0), +J(\boldsymbol{w}_0,\boldsymbol{z}_0)= \frac{1}{n}\sum_i (\boldsymbol{x}_i - z_{i0}\boldsymbol{w}_0)^2=\frac{1}{n}\sum_i (\boldsymbol{x}_i^T\boldsymbol{x}_i - 2z_{i0}\boldsymbol{w}_0^T\boldsymbol{x}_i+z_{i0}^2\boldsymbol{w}_0^T\boldsymbol{w}_0), $$ which we can rewrite due to the orthogonality of \( \boldsymbol{w}_i \) as $$ -J(\boldsymbol{w}_0,\boldsymbol{z}_0)=\frac{1}{p}\sum_i (\boldsymbol{x}_i^T\boldsymbol{x}_i - 2z_{i0}\boldsymbol{w}_0^T\boldsymbol{x}_i+z_{i0}^2). +J(\boldsymbol{w}_0,\boldsymbol{z}_0)=\frac{1}{n}\sum_i (\boldsymbol{x}_i^T\boldsymbol{x}_i - 2z_{i0}\boldsymbol{w}_0^T\boldsymbol{x}_i+z_{i0}^2). $$ Minimizing \( J \) with respect to the unknown parameters \( z_{0i} \) we obtain that @@ -1007,15 +1007,15 @@ where the vectors on the rhs are known.

PCA Proof continued

-We have now found the unknown parameters \( z_{i0} \). These correspond to the project coordinates and we can write +We have now found the unknown parameters \( z_{i0} \). These correspond to the projected coordinates and we can write $$ -J(\boldsymbol{w}_0)= \frac{1}{p}\sum_i (\boldsymbol{x}_i^T\boldsymbol{x}_i - z_{i0}^2)=\mathrm{const}-\frac{1}{p}\sum_i z_{i0}^2. +J(\boldsymbol{w}_0)= \frac{1}{p}\sum_i (\boldsymbol{x}_i^T\boldsymbol{x}_i - z_{i0}^2)=\mathrm{const}-\frac{1}{n}\sum_i z_{i0}^2. $$

-We can show that the variance of the project coordinates defined by \( \boldsymbol{w}_0^T\boldsymbol{x}_i \) are given by +We can show that the variance of the projected coordinates defined by \( \boldsymbol{w}_0^T\boldsymbol{x}_i \) are given by $$ -\mathrm{var}[\boldsymbol{w}_0^T\boldsymbol{x}_i] = \frac{1}{p}\sum_i z_{i0}^2, +\mathrm{var}[\boldsymbol{w}_0^T\boldsymbol{x}_i] = \frac{1}{n}\sum_i z_{i0}^2, $$ since the expectation value of @@ -1028,12 +1028,12 @@ where we have used the fact that our data are centered.

Recalling our definition of the covariance as $$ -\boldsymbol{C}[\boldsymbol{x}] = \frac{1}{n}\boldsymbol{X}\boldsymbol{X}^T= \mathbb{E}[\boldsymbol{X}\boldsymbol{X}^T], +\boldsymbol{C}[\boldsymbol{x}] = \frac{1}{n}\boldsymbol{X}\boldsymbol{X}^T=\mathbb{E}[\boldsymbol{X}\boldsymbol{X}^T], $$ we have thus that $$ -\mathrm{var}[\boldsymbol{w}_0^T\boldsymbol{x}_i] = \frac{1}{p}\sum_i z_{i0}^2=\boldsymbol{w}_0^T\boldsymbol{C}[\boldsymbol{x}]\boldsymbol{w}_0. +\mathrm{var}[\boldsymbol{w}_0^T\boldsymbol{x}_i] = \frac{1}{n}\sum_i z_{i0}^2=\boldsymbol{w}_0^T\boldsymbol{C}[\boldsymbol{x}]\boldsymbol{w}_0. $$

diff --git a/doc/pub/DimRed/ipynb/DimRed.ipynb b/doc/pub/DimRed/ipynb/DimRed.ipynb index 24a0aba51..c9ff60e5b 100644 --- a/doc/pub/DimRed/ipynb/DimRed.ipynb +++ b/doc/pub/DimRed/ipynb/DimRed.ipynb @@ -1064,7 +1064,7 @@ "metadata": {}, "source": [ "$$\n", - "J(\\boldsymbol{W},\\boldsymbol{Z}) = \\frac{1}{p}\\sum_i (\\boldsymbol{x}_i - \\overline{\\boldsymbol{x}_i})^2,\n", + "J(\\boldsymbol{W},\\boldsymbol{Z}) = \\frac{1}{n}\\sum_i (\\boldsymbol{x}_i - \\overline{\\boldsymbol{x}_i})^2,\n", "$$" ] }, @@ -1073,9 +1073,9 @@ "metadata": {}, "source": [ "with $\\overline{\\boldsymbol{x}_i} = \\boldsymbol{W}\\boldsymbol{z}_i$, where $\\boldsymbol{z}_i$ is a row vector with dimension ${\\mathbb{R}}^{n}$ of the matrix\n", - "$\\boldsymbol{Z}\\in {\\mathbb{R}}^{p\\times n}$. When doing PCA we want to reduce this dimensionality. \n", + "$\\boldsymbol{Z}\\in{\\mathbb{R}}^{p\\times n}$. When doing PCA we want to reduce this dimensionality. \n", "\n", - "The PCA theorem states that minimizing the above reconstruction error corresponds to setting $\\boldsymbol{W}=\\boldsymbol{S}$, the orthogonal matrix which diagonalizes the empirical covariance(correlation) matrix. The optimal low-dimensional encoding of the data is then given by a set of vectors $l$ $\\boldsymbol{z}_i$, with $l << p$, defined by the orthogonal projection of the data onto the columns spanned by the eigenvectors of the covariance(correlations matrix).\n", + "The PCA theorem states that minimizing the above reconstruction error corresponds to setting $\\boldsymbol{W}=\\boldsymbol{S}$, the orthogonal matrix which diagonalizes the empirical covariance(correlation) matrix. The optimal low-dimensional encoding of the data is then given by a set of vectors $\\boldsymbol{z}_i$ with at most $l$ vectors, with $l << p$, defined by the orthogonal projection of the data onto the columns spanned by the eigenvectors of the covariance(correlations matrix).\n", "\n", "\n", "\n", @@ -1089,7 +1089,7 @@ "metadata": {}, "source": [ "$$\n", - "J(\\boldsymbol{w}_0,\\boldsymbol{z}_0)= \\frac{1}{p}\\sum_i (\\boldsymbol{x}_i - z_{i0}\\boldsymbol{w}_0)^2=\\frac{1}{p}\\sum_i (\\boldsymbol{x}_i^T\\boldsymbol{x}_i - 2z_{i0}\\boldsymbol{w}_0^T\\boldsymbol{x}_i+z_{i0}^2\\boldsymbol{w}_0^T\\boldsymbol{w}_0),\n", + "J(\\boldsymbol{w}_0,\\boldsymbol{z}_0)= \\frac{1}{n}\\sum_i (\\boldsymbol{x}_i - z_{i0}\\boldsymbol{w}_0)^2=\\frac{1}{n}\\sum_i (\\boldsymbol{x}_i^T\\boldsymbol{x}_i - 2z_{i0}\\boldsymbol{w}_0^T\\boldsymbol{x}_i+z_{i0}^2\\boldsymbol{w}_0^T\\boldsymbol{w}_0),\n", "$$" ] }, @@ -1105,7 +1105,7 @@ "metadata": {}, "source": [ "$$\n", - "J(\\boldsymbol{w}_0,\\boldsymbol{z}_0)=\\frac{1}{p}\\sum_i (\\boldsymbol{x}_i^T\\boldsymbol{x}_i - 2z_{i0}\\boldsymbol{w}_0^T\\boldsymbol{x}_i+z_{i0}^2).\n", + "J(\\boldsymbol{w}_0,\\boldsymbol{z}_0)=\\frac{1}{n}\\sum_i (\\boldsymbol{x}_i^T\\boldsymbol{x}_i - 2z_{i0}\\boldsymbol{w}_0^T\\boldsymbol{x}_i+z_{i0}^2).\n", "$$" ] }, @@ -1134,7 +1134,7 @@ "\n", "## PCA Proof continued\n", "\n", - "We have now found the unknown parameters $z_{i0}$. These correspond to the project coordinates and we can write" + "We have now found the unknown parameters $z_{i0}$. These correspond to the projected coordinates and we can write" ] }, { @@ -1142,7 +1142,7 @@ "metadata": {}, "source": [ "$$\n", - "J(\\boldsymbol{w}_0)= \\frac{1}{p}\\sum_i (\\boldsymbol{x}_i^T\\boldsymbol{x}_i - z_{i0}^2)=\\mathrm{const}-\\frac{1}{p}\\sum_i z_{i0}^2.\n", + "J(\\boldsymbol{w}_0)= \\frac{1}{p}\\sum_i (\\boldsymbol{x}_i^T\\boldsymbol{x}_i - z_{i0}^2)=\\mathrm{const}-\\frac{1}{n}\\sum_i z_{i0}^2.\n", "$$" ] }, @@ -1150,7 +1150,7 @@ "cell_type": "markdown", "metadata": {}, "source": [ - "We can show that the variance of the project coordinates defined by $\\boldsymbol{w}_0^T\\boldsymbol{x}_i$ are given by" + "We can show that the variance of the projected coordinates defined by $\\boldsymbol{w}_0^T\\boldsymbol{x}_i$ are given by" ] }, { @@ -1158,7 +1158,7 @@ "metadata": {}, "source": [ "$$\n", - "\\mathrm{var}[\\boldsymbol{w}_0^T\\boldsymbol{x}_i] = \\frac{1}{p}\\sum_i z_{i0}^2,\n", + "\\mathrm{var}[\\boldsymbol{w}_0^T\\boldsymbol{x}_i] = \\frac{1}{n}\\sum_i z_{i0}^2,\n", "$$" ] }, @@ -1192,7 +1192,7 @@ "metadata": {}, "source": [ "$$\n", - "\\boldsymbol{C}[\\boldsymbol{x}] = \\frac{1}{n}\\boldsymbol{X}\\boldsymbol{X}^T= \\mathbb{E}[\\boldsymbol{X}\\boldsymbol{X}^T],\n", + "\\boldsymbol{C}[\\boldsymbol{x}] = \\frac{1}{n}\\boldsymbol{X}\\boldsymbol{X}^T=\\mathbb{E}[\\boldsymbol{X}\\boldsymbol{X}^T],\n", "$$" ] }, @@ -1208,7 +1208,7 @@ "metadata": {}, "source": [ "$$\n", - "\\mathrm{var}[\\boldsymbol{w}_0^T\\boldsymbol{x}_i] = \\frac{1}{p}\\sum_i z_{i0}^2=\\boldsymbol{w}_0^T\\boldsymbol{C}[\\boldsymbol{x}]\\boldsymbol{w}_0.\n", + "\\mathrm{var}[\\boldsymbol{w}_0^T\\boldsymbol{x}_i] = \\frac{1}{n}\\sum_i z_{i0}^2=\\boldsymbol{w}_0^T\\boldsymbol{C}[\\boldsymbol{x}]\\boldsymbol{w}_0.\n", "$$" ] }, diff --git a/doc/pub/DimRed/ipynb/ipynb-DimRed-src.tar.gz b/doc/pub/DimRed/ipynb/ipynb-DimRed-src.tar.gz index e8237b747cae75a688a55bea30f7718d61da2cbf..9a7f9668cd90a41bdb946716ced2a938e8027a66 100644 GIT binary patch delta 157 zcmV;O0Al~X0lxtoABzY8b|koUDSKyg{;@Bt$yTjlc zqd^-lD`PH@tG1LPQQe>`5JtMG1y7D0xJf%5H?WBG?H;&cyhds+H@XSAPY?OoDVepO7 zppBQ6F&D^HTS}3rZqOA7Bi+;jD{sBB61X0K3!%I+nqTA2nv>}5I!n@S(D?o5q_Uvp+`=|f{Bx$j|azP@XaYxKbq)c)=yZh_YjRq-#f)3A&{g5YONF0quA76jaq>KASy7>D|I(z>0?&i~1 zk}quymT|E#I%7iW#oc=GQ}T1lv)f|LNycr$xPv53n!*MwP3+ zX*>E-tX56CuIl^k2N=rfWmVI&Ijr5hrRTa_wIwW9bW7Y7NIjEXX(lX*4zu1arIgd< zwMKc%j?HuuC~}ESxma?`WNNN|(<5go*K=WnVR@!Vns3W;58}Z7n|k|!TKxvL(sU(G z{OK!@dorJ&OGwF5DZ{z8G~lEFLgq}1EF3h|f+8)T6gJrw?LF$5Br0lZ74+!JEu2@m zrmp>_ExS##UDL-=pP$g*>uNX_mWsJDxCRMxl4Vu6Lx)jDp%z4zpb>R{=M=#+W?_6V z&z=SZrP06Hjjnh%>Pop(;@(!f{kB?F&Imuyx6zXE`@3o%Or*vIVOK2}vwz#c#y&Vx^_DjUU!$B6HLpY`^R%kh!qd2Ufe=a~e z<0C=UZz#U5t6f=l5MLt-GdauLdQ*i3DLiK|$IwCe4mRkZ(oyCAiKrYSw_ZqQxW{jG zWZxPr2w>i%7aRRm4cqjfT3L)ye2m@7g$Zd6u^Zce+pZZL(gkjKHwJEKW8k*l4gH9k zo=w|tn){}9`31+fMcut^+MUA(XB-doz%tnSOul2C0(ueJTu>^vze}mtn6ZdD#KesE zCoEEp776|%vv7wHVU@~$Kn0eIi5OTR0!sXnR0be_1agjHSmKNk@#b%LH^1JDWka4Z4V{#d zV3>Td+THy0OS)LYM|5-v)$gP0x`S}wO5yF|`NajFy|%M;`e^aUjk&qDb%k7QC(7X&M=f`}Nw2FppW z&o2XgOp0*$N{dNSJW?PB_m&poHoyk~_WKM4j#CY*UXMd!8k}w?<6sUp+|QS1vJto7 z8|iGoYs5}z@-ZS2cyJ0aE+W#&VmyU^DjeJ^Sm6`M1`>)UEQ8NThlRWrv*Y;~^-Xkw z-y;>&MS6M!j(>tAp|~r~;C+HQWvK1&?*kgAZUmn2iw95GzmDYTL+VdKKp8H2K5y&O zU}&-%#lfKDOix&9pqMVZVdG+fIc74zk}CI=+>bG6!WlO&1o!j{B9jZ4Q4g4Z5uk`l zUT-KNt7P3kvBr~o`iqzX$kUIL?CsixW8->HQ#AC&5fh&Nk!m1mrii-O4kfFV$ua5U z@hYTErj>^ecMg%{+zSJZ-qRNDkFGCzM`gJW4!A;VSJi-SK7uU7)X^wv6a+dNk4mU! zhWAHc3f>XiIZ@?Hn}Xwbgtlsb+P3V7Gi&NKN~FAI;MtXZkCL46Z>gCPDxCE7`4A^G z7@x|+G3P>@FoLDI2?u^eOqyV(1@Kd2J6s}|Su^g0i zpc?F4#Rf2h^p;TW_Eqa?ER}-BJt3S8>=~j9cMe-ud3ZRfxEBYFU2AWDP%uHW@2efn z#*?j^?Q!W$6*ENe=9$-Gypn1hH8xkir5XV9A2}C=UGcKl#$)2Uipl}Q4r4-?%Q=4)@dn-F!?d=OY3Ne$0m*5!!T*+%Vvjhq^iZ~uow@N8p_^$3 znGT#9k>c=ow=e3sAhD88N&!#5%aNC!UXI~i0nAJHV;KKQxCx7YJ$kG@_E@|Z8?F;z zEO{(?aGd1!1h)r_-=S_Krxi0jK0P7-6v`V66U>Nl>+cfpkn(oXTqoV(CP590s9_PM zCJ74OZqawK;aE)^-50HY`^9B=o6f$onV4hr7t)vq9fnuqvY&4@Y1|l4z8~$(Uq# z68AEr*2ENdyP+oeM^7IdT|-U{e|e)3=5Q$h?V3`YUt#ZmZ@b<@4(!WeT%6BApfM1t zM=%M)fM4|<92O?Id5KfHOm=`QfpSCo}L;(ekmuHdXawfBpjQJQ@?I;Ic zwOEX&TJj8HJuHs|i5du!=MYubs!5%_mGD;u>PkxMypm>1$#Y9RiTi@WOBe_mkav(q zdj|J4%p~xC()l{>EC2uDzGm8>pNGtjI1egKBW0ATni_&|zh_7TBbb{~Xav8{bT8Xe zoEB3tT1*BWEMykYV%{bNoF_kTlQISPC&UZb_NH0G9nUvdd`l}p$QE|JANA`d)5AM16t z&@u%jW`U~HsH&0if2yhj6}qrL>GPLs1521YQpms&_sqk+DJaV#X7rTsGuiwpW%EBu z+kZ^Fv!}c?M(!h~V! zi~~M@fkBLCb0m|`)DulT!LDFS(0$VB1PIfLf!1Tj2 zW0^UH8mJk;+zbK#TIzG#bG z*_UnfMkN z&uAlO>dA7I41CvtTHN>>&Z6}PKiX>eUm6Q3bCyB^TnhY2gimcY=mV+ue~2stnUnuk zC;~V$lMp^Be_C5_+sGAu_pjh1g;uki`!Gd;wu{}i!4}wd4eSEhJkSzjGb@o^in5dZ z`kpfvi9?Mpwvz(si!~e`&dj-e=Q1v`+pNg`epUG2zg=H_^F5;(rKFUSW!LXA$~n=> zWJ+qnxXiAX*_-^wZ~r=*b1m~9`wj*)@7}rRMc2;Ye{pE+&$eF9-d?}{=6kBMl9Y-m zyCNwV>{evdE)J858)5{!T*KNw&kW1!*<8Q^ch!vP`~m;?nYNoTlRvtLcipyKy5Vr= z3AXKTNTc(|)=k`3eYLKKy5G37Ff%@jKL{6s2*Kl%eDWWZnIfg;20su)ah}a75v6dl zXgrk~e^D`FXbB5jg5tCcxKxBI5pPXqb1h7MeOGThUyH8q>&^Xa4y<)MuDd<=u*r^{ zozj>hrlfAod(;&O?|r=(KrwU9Ijm}@x_&vMdDB)yy}=hEuUb#>i1MyIb=A7Jk9{-L z32`bz7|Z5N65JG-f9c@q!*A7e{Blb8YFqk^?3pg4W?-SyS)S3#Oga3UnlQ~|%5WsCLIo3}-_gqN?gg7DI0 zkCt?uNm5W`ZI%6W^>6P^V*!PwWGM(0+HHC74$jB?Yhv)=imKO{C(L@EhVrPMEUZKYUbb9ke5bcASSU<6(>PzOJ^dcwIl z9W5n^gy%@7MUgN@I||HvN|+ER3$!_6h9%B4FNlCqg1-ZgHxrMaT4Kci*bwc{0z9y5 zH$#GB!3Z5WOp#h2v=-bOtC!eugw_Q_3mF;m zF}>0+saJDCudfnlUpQe}WEO^KP^JW^+#xfBJZUkONsqBG*xZFshfl*@HH49;e3(g< zSAA2p3qNC{@R;NP@^J&4DiQJ_NYmUyK2!O!{*lyW8f$#y%hOZt&q_%JUc)2?Vq`1*ZwV;FU9*)BH_JfDi)ky!BhkKf8YT%FOP!|h^c@qL1HM>@D4U$P=C0$umPx14Ncd& zSukmtSF5cj0}3a+gXOX_#{E|~%Eu{SpSN5&okquOO`&bX*5_AUB*O#M9TOuxXKdzt+WUOv z+n~X|qDX_r~ zZWf(8HNuNcXCMdhXRS5~bwSRF`JXK_3y;*_Cd(ef<;5WLA!#|N_u{#s4#-K#Gr=XH zkRClh(uIY9W2541Tf|i(h58L@el9|so5PV)(T+jrVIq}b@Ti6=e+1yih$@L>1(8%e z70HgP;>@Y}xl_AhbA0NzN$}K4pLRV?ZRHF;2*s5Oh_?5xU4LpfGu+|Bb`wmz8d6ea zfD{;qIWiU{j7O72^?w!&M;>&d^I`-k;A?JhNB$BI3M*-BW0t3LMfoLiMe#Wr5#>b% z>&aM-{6`gQGKfbWfBnBUxnp21UJT}H59X;AMu70No})RHEWJ9@1otfgE;*5L51b3? z<1h|Z>u!VbS`-*N`}&VKmU}=FbP7l?)7=bJ<0^Vw!~OIaVOJryhlv~8SFQ)!IJfO~ zTlT_E>!xki%_kSw#i-RBck!{?46lg}f2h3f`>uCG*I$N#%L0TR z$NxH_(Sb(UOd&hY(8VdL?;{(ajx zU|#to$m4qF=+@PTy>P}A)Z~b}tY<`$YR6;tMZ8Hxw=e}@jV2}(ZP#`YEpwY&RV(+f z8S48p-6wb7f5Hs60VrxdF=If!+tm_>AoIR?Xdt8e=-zlvYya}je*k>yJ)by(d`2W= zKCLC-B=(=JdOc}h`}T^R@HThU(3qx#OFhz`us1|t*j9!3)U@sw0vhb+;a^}`Z09XKN zwD4``S?+@1=+yF=EFqWhow}y=skvhqAO}5&m-# z0v+zUf7{M>L03m@O=u#*wVXRR%7SYT686p$Y$|{(Oxz&n9}|82{e_aTT8q|x|2`oQ zyz3Cp3Q6$=!Zaj>%_eDxgwzMSv}%-wxXYbgOO~ZNpnI_~4bTdiLBtvsz(%e6APWTBxV%{Yi22@a}bL-iK0;yy;c4A%OQ?@XZ`tpyWFq4bQ9J4wJxT z-TUluXdXrw^4LI9Vn_ZzI&7!if7~S-08k+c^Q<@!F#siZ6b8n%Z8K|1z5cZ?BU#VuUmK8UMrMf63T&*h%1w$P%nT8Q4Ic>LdWAVfs`rftOIQ zffGZfp(G8`NJVcW_&FB&*?(f85ST&xkTWn7N_Gp*EM+Sg%yy>o)nR3XNe5VY3Qu01QXDuB&zgOraYYLt6~y zKSKI;zWQ2QFlo_>sK><)y4LyvtDvdAUrWcLp3xwF*rdsL_#z;IW#sfMmRz= zLNziuLNGT%LNh*H3NK7$ZfA68AT%^NAd?V2D1RN&IZu>P6vpxMyqBwtqKF{sAe#%T z3JM}P!lJIy=yj-pi*YlKFlDLN&8_RG(a8Xp$4j%R{;@r!EOo%;4mEH8^f5B z<$or8`#ltnlJ-Isa7N^`$foHK$*Gam1z&a$YN@g!k)mISLaJxzo7T%3S9}FGh)1YG zY=OM2Gw)UYDEj>xp$VGd6tv3vdVJn-H~}Z26WV0MJ6>&v4$kmh&O(>0)b7=8I1Oi@ z4|-&?<6iBBb8sFmKmqz?izS~j02g5phJWA^jKXEv>nfi&3?op4D{vLA!A%&Gz5VQy zufq)(hY4;^Q{o*`CFw3HN1B8wxW_kd*?pL1NW0?$cnC8v3v)0J3-Czx;g28wShn`b z(-Wz-=;^8S{=27V(z1DaE`2L_T9nq8JiSmh`qR@(Wy}8O$rOHnEkEV0Z68XZ*h1?m zQ&|1zDJcASUC^!Y e>s9$Ke+%{>PmNA(mzmxJAPG1(3MC~)Peuw7s3Eif delta 6387 zcmVaJJ3D3-&vvW=>~t(t3IEZ<%$y)1Y4*KFxN{gSG*Y&P)$IR;=< zwI158w_nQjy6rY~bGv(wq0GLl8~bbzYqziMb5pIm3d@!KigpF5XR=F`@Z5@yS+7?@ zi0Sf5k-Sx(%ybecaz#zCSg|~R5vjgRk1R=0;0lfKgtb^0A}$rF)>3%{<);l48(#18tpSy#6C zKpJ-K>K#rTTY~~!zkRve*`_^&t}chVZQL|u_f8?q&#@_oGH&T93`4uK-Bw+dJkmT& zM-&T{>qKOk?fA{>!5npemSL5e*3EK~IWz0wZP~xEFWdVLXaClIqN6cyw=KPRuzfe} zG5{*%7g*xx2u~s_SGxs1!@{dS$1^Q*gmdvw%nKR>b)_HeYk(@N~PFVCKJiI z-IfhR(55{H0o76vBImagqT!Cr{?S#N`!%UPan^x}SyCv)ZBXQYs#xIY5>_y4Bj4`I zzOUCM9bcxF-+%j!ZQN|DwyXB1Er?{sOU3gfmrmrD*Eu&DA{XlW`@L&7qlWPZ+;On2 z(XP5_P!jt`kj?a%7{Y=m^FilfmA=9-Uud63g{{r^RTVgB+J%I zN45g%c86R!f!1Vy)D3ln37UXnb{@0ZOy>*5bdu-JY!O)2_kC|iiDmi9_|?SHlIxXX zYWds^S0jK?do-HjPSkvHd-=$e^cKD6UlKw?5cw?ySF_y z_THGP6KiLr(PSHDcgV4Ym~StYNg- zLMR64L}`W8n#jT-lvsR*YiEl|pD!azll$jBgkQE4i!Y^%aIF>7+42i|X#2?gBPIuj z8;N2rNHG(CL|4Qt0P`ljlF?tK-=+iAGN%~D#@H=?O_&h+5WA_p>s#-TCUE237`V}< zz@0lcbR!!3Y`bCG-nNa&FEqX@oBnm%?F~LS;dszPl40vJ`5x;O=!GkNMyV`+l2R`* zV;*&gi5cG=u}C>uB>YEY;SLdDr7ZkGxD>+VEXcyMfsveX`he1Xn3!ssX^z7n6Ql_p*ZUgq*-hDcknGmjR~}6FzulLe2hmkGPVMr|+^>x6 zsKOX?+npN@l8()*a5a}^UsEa3Rkn&U2q=oC3HhoF!3auC9k%%{5=e9+b8$FEl}2n( zBzZQ1=$En85P)*#dz=m8KSks~30Cll98@BI0w#WKb%w8+42x=u%*v>c=9T6ewsSn7Y z1j%uNP%%c>UOz^>KqXm@w7?l^k-~PXqZ-huB6ON5dHHdi$wM+&F9;M0-nN+Z@u?_( zRI8BW#E3h1LLaBpS^&q*)Ede?PL(3Ge1!Ocx=`~VJO%PA7D=U~Ek)6a$si;~z`<(L z>&x>pjWC_Vkb%ys#N-Dm%qrVSu+;+3*iM}}WME#pcqCVvQ6atEH(bI`r9Y;fx z-AD$55;HwftwCa%^hS-d73P@A048POO1Yn6*n}laKNH?l&j?M^F^M*aU4Wjw_R6%^_Dbin+-`Moo4Xtt6?BXPWZP_iRLn#^yTRg#}$lE z7U7svA&zUFq=gO#{BR~b0Ej9v6O2ex>FF`-FQ@_V_s9yMJ;k4rY@swA>kpMz(t&Hp z=PK5K5W-nPwZE%7$79J98h4Cv=IJv67v>yxru1-lQZgqF8M{)>pfEvydpFd3I~!Z> z%xn)UW2!_N)_ylc{a`J!nTj5ylP3h1kipRN7NabSg`yn6dbe^)k+M4JdY?H;|HidAG}37MW6 z=Yd|92T#&RJjsP|>qg{Oaq8L8&JM+WPg)VoE|;nC>1gtgA)=9g6L^j#a(BabLqxO* zv1-{L_=Cs_k;ch;fQ(r}G5dMG>h$}vHQNu(u8?%dhRks`5OI=52orfQq9 zst<%G=AUOOK{R*7|A?VZ;1G>nhvOb16+xbGnC!H^NaNa^z+;4UR1v|m@cD^xYY& z(kfx$2uc%zjM6eCdg-5*5t$KQdKzi(Q2B9jmW8svr#T;C7rHujTn(DFdo@&!em7SM5v!Abl;g)j{EVfk^)FfgK8NVx#r)j3_oQ_y@ zDG|LZke@(Er4%eUz$!P~)~66sekx9P!Dk4m!V*%&vk+49QbJ0dBc#+tgcOT0QZsGX zUu~QKU=lEYSVRe=sVQ*tGlMj+g1Iq;R`8@`a)Cj`$toEmt3+_fTx0=R<%|+1MLz~P zWt1WXVB^nXl;c#F7-jzdX_R`Z>l+Jb7b|@6tZyAH060P;$B~NP)8xf5laCXa#esnT zU-(23_=M8zk!EL0tx~8l3zVHkm93D~K33U*5+5ml`%jk!moS5*8H>4S5pGVR6|tm` zQTJr`$JBk1w)NAs{j_v_X~@fu`Du)G2d-1*>T|^*9`|-PMb6@Y6473ljfc$-T|KzB zoBJvx;6`+PfT)Mxh|ud8=;IW|4&{IRIOgh!)&D5A(S6eC2oO`Kg%DgRF6R0<@A005 zSI_-_lH$yT-&4@)5hGHc0K~M$#yR>*)O6V=0Wm*ShIpFuKmKM=J;%);#R?J0dHNZ0 zUi8{2e($LMxRj@!CFQA0Ql7Xh<>}{3dFqmsr=BI{iOW)+eww-?E_Q)x7#}Iy6oKfk0u-bAH_n5B1v<2JEz|)xD1yEeUO6x3rDo{v zC;~S(ld*Uxf7M!BZ{xTXe$TJaql~V(;+>kJK-^vy8Cef^IWo0?Z z%&+e`yvU+VUnWVhn3sg2k4T=|cP>-k!=T~}}KXLDe!9k_1y+`}dV zJA=}gBBrEn&3n`p2=8OP=s{bY!gyTJxA=p|E8GAn9#Pu0n_R%})y9n< zyQZ%b@>GWC0_4La!EKTKx1NlT$%rsK$mf?re|ixKF{0tbXc=f86GAqGnOGG?Vk^+A z&PFt_8$}_S{7+6~2BeP~)}Dak^wsy*SHE2aaWcV43AV2a4y&nh6oBB31DoA z_8)>LxL}`PSujG`-eHQ=2B2OF$~$Sve_R)YqPf4^lEYFLS+cpvl2zK0Bfu^oV8o{U zyJ*&&((Icw;?n`11;8W-R{~rP@EqbE2gF@ioMmOcy$kUUpZdG1cLX-v;nc%S%Dn2D zYP0a~+E8qX{lo|59+Vsh95Y27~CQgeM0sgk%t8VkRCFVm9E^e_(k; zQ8|K)iI71T43q%6D2$0jJwXvEIf`0eh9WXjYow4l7ijQ+o?ufLcagB_Y8Uu!n+F^E ze7w8ChfgV|V;tfK?gkMrcmXIy!BVq1&EKU22;r|#)6jM(GBjU)iO8f@7l=$fM`RJR zz$$gY)DyjrA}J(@fVY4@ygUv-e;_9b%u9g)RK5s+!crUrCFMl`WL_13j2OnQ})h4K~e+WK~gA#lWiLZ9ANSeFknzW-CKMByr}x7-MCqYG^?E_0+?}< zjv{VJ3a+gXOX_#{KAHb1F@&>}iy+JToQE8wAt?t@5x&33L1Z%TPdkV>e{_(EtiT>2 z0SG)^Dy2A?982kStCd9Ra+IwpxQ*C)ebq%WJW$;+G17C!X3izOFHyezmvUV2lY$Q_ zPJ1n+eh4^`Pd(n%AyADgG%S}*{xHn&e7CKak#~#yLlw$DpH84N?>aYFR(<75z_7sr zVjyN5u;Y?ApqGS0`8s!9e*$-+f7gp@wQ4rE(*<2p(d===<|nYh05^-)oq9-QAPM$o zt+okuLB5H(pRF+qkJR5Kiyp(}#US&Pv>eoXk&bm*s57w+$f4HD1eb(Df^;&}mF85@ z=obQxjq>iWNid|L0DvkL_!c3~&Ed$YXvZM*Fp)|zcvL|Z0`Ox*f0abCf=H^q6v>XO z;>@Y}xl=2b7mcXiC&5!IecJUng}NCn4RBPdfM|Q~8roCCOn-+DyKOM>s!vH3w#K{@ zQpYmnv6rNbuM-gU>jgyhvVb_s7*F(GjIgKW26yBy>b-=eG`2F!i+V5SYpqz#seq3B z>#>HDkvwwi(`Cy4f5GUELArP`q^muor&buj!m+laNVXl#sg&u>nNGNG32@1YoHPUk z_0bpyt984@s4WVNpk4h9j^!Sf1f9YX%(ORs)ws$Y*Kq$jjI*nd;lsp@Z7$cn?Vj77 zyDfiVr**Sw*3D-Z-o>cj9JlhZ&DA!frHBV$Vhw?Q4r#&%nXb+w0jdNB)8C1H1?rR@^Iv6${#p~a8 zjf3ZvPlY_Xw~lUIJ?(`vuA?SL9CDu#$+$NYGBwP>h&QR|Dy9Ii0cZ-P+qGRp%iLyI z)yh3=`}#g%e`I8XD7-aH&W76ZV=24BM&@pPG&P4`JEn zlsKN+q+lxlu-n)t*OEo5a1tD^>b|$yMbcrXP1~*Kf6PsWVmRF=n12R(?F`QKZQ1Arp#2Cn-ibQY~^ zzzAjF8%_ez>gsKU0eQUfU#o)i?L6rtXG1TZb&)wdWm<`%&W?#r+Ne$Uh8td6*v<$epxsHNY!elwN8c z@{%TxOx~1Ey%dT~X@0jR_E(S{o{%-v!=Tm?(I1XnUygLoWYSv=WzSM=K~vZSo~X*h ze|yHb|1v(;$x@VG*5LaRy?;^MOn(S4Ht$2JN8a=%*%rWiFL-ATE>QA;N5k{#*TW=m zS$95r?3;%XhCDWql-QB~ti$%({mX5#1ppkd5`fY$eX5zjODNdD zi6PTak_Ks{qPG$J<=?U3Wfy24at3BX&2GV&rECR**X-PsIkA!rL&Ha!)8rY;d2G@%Qicy~%IF2a z!2bbO9IwWMIAMo4VF8CYVFI@}VFSN$0x>d|`O*U+0x>eT9Mc1S2~s&jFfv9sMma(- zHAOWyL`60^F*!j(F+nv&LN+uuG(MK(DxIYB}(K{Z7} zHZ(RgL_S>#FHB`_XLM*FG&M0Gld*Uxe#27vRv6#-@#MoHaSy-9S8Z9g=Ecgl*cGvqqG`~Fap4r} zX7L8h!c8z`uXl|ksE}E;4>JfQQWYG8<4_9)$U`;r${<2H?4xi9jzR<97{;6|e>dmb z@27B#R0#)wGa{!&HcdxJPF_|UeA!{Bp~{Lxihdyqsg9vnpfHtfvaG1q!mR zf>-&Y==W=cCTNCJ&?+0~^LcG>0!~5~w9AJ7d9?#NIm379g>KnwhgW;xG@O9}=#wqy zy?PeTK|c(_5S*8-Rr!<)a1k!Se`UA=lW_Pjo-=N>VG(_(bCqjZZv_)DHAKdCz$asT+C}e)N{^ z^1EpN0eCo32MT3wWOH