lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

update and correcting typos

Browse Source
This commit is contained in:
mhjensen
2019-10-21 13:13:16 +02:00
parent 48e5ebfd92
commit 46aa974be9
27 changed files with 4130 additions and 130 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/src/DimRed/DimRed.do.txt
+24 -22
View File
@@ -332,40 +332,42 @@ applications.
===== Basic ideas of the Principal Component Analysis (PCA) =====
We have a data set defined by a design/feature matrix $\bm{X}$ (see below for its definition)
* So each data point is determined by $p$ extrinsic (measurement) variables
* Each data point is determined by $p$ extrinsic (measurement) variables
* We may want to ask the following question: Are there fewer intrinsic variables (say $d << p$) that still approximately describe the data?
* If so, these intrinsic variables may tell us something important and finding these intrinsic variables is what dimension reduction methods do
* If so, these intrinsic variables may tell us something important and finding these intrinsic variables is what dimension reduction methods do.
!split
===== Introducing the Covariance and Correlation functions =====
Before we discuss the PCA theorem, we need to remind ourselves about the definition of the covariance and the correlation function.
Suppose we have defined two vectors
$\hat{x}$ and $\hat{y}$ with $n$ elements each. The covariance matrix $\bm{C}$ is defined as
!bt
\[
\bm{C}[\bm{x},\bm{y}] = \begin{bmatrix} cov[\bm{x},\bm{x}] & cov[\bm{x},\bm{y}] \\
cov[\bm{y},\bm{x}] & cov[\bm{y},\bm{y}] \\
\bm{C}[\bm{x},\bm{y}] = \begin{bmatrix} \mathrm{cov}[\bm{x},\bm{x}] & \mathrm{cov}[\bm{x},\bm{y}] \\
\mathrm{cov}[\bm{y},\bm{x}] & \mathrm{cov}[\bm{y},\bm{y}] \\
\end{bmatrix},
\]
!et
where for example
!bt
\[
cov[\bm{x},\bm{y}] =\frac{1}{n} \sum_{i=0}^{n-1}(x_i- \overline{x})(y_i- \overline{y}).
\mathrm{cov}[\bm{x},\bm{y}] =\frac{1}{n} \sum_{i=0}^{n-1}(x_i- \overline{x})(y_i- \overline{y}).
\]
!et
With this definition and recalling that the variance is
With this definition and recalling that the variance is defined as
!bt
\[
var[\bm{x}]=\frac{1}{n} \sum_{i=0}^{n-1}(x_i- \overline{x})^2,
\mathrm{var}[\bm{x}]=\frac{1}{n} \sum_{i=0}^{n-1}(x_i- \overline{x})^2,
\]
!et
we can rewrite the covariance matrix in this case as
we can rewrite the covariance matrix as
!bt
\[
\bm{C}[\bm{x},\bm{y}] = \begin{bmatrix} var[\bm{x}] & cov[\bm{x},\bm{y}] \\
cov[\bm{x},\bm{y}] & var[\bm{y}] \\
\bm{C}[\bm{x},\bm{y}] = \begin{bmatrix} \mathrm{var}[\bm{x}] & \mathrm{cov}[\bm{x},\bm{y}] \\
\mathrm{cov}[\bm{x},\bm{y}] & \mathrm{var}[\bm{y}] \\
\end{bmatrix},
\]
!et
@@ -378,19 +380,19 @@ correlation function
!bt
\[
corr[\bm{x},\bm{y}]=\frac{cov[\bm{x},\bm{y}]}{\sqrt{var[\bm{x}]\var[\bm{y}]}}.
\mathrm{corr}[\bm{x},\bm{y}]=\frac{\mathrm{cov}[\bm{x},\bm{y}]}{\sqrt{\mathrm{var}[\bm{x}] \mathrm{var}[\bm{y}]}}.
\]
!et
The correlation function is then given by values $corr[\bm{x},\bm{y}]
The correlation function is then given by values $\mathrm{corr}[\bm{x},\bm{y}]
\in [-1,1]$. This avoids eventual problems with too large values. We
can then define the correlation matrix for the two vectors $\bm{x}$
and $\bm{y}$ as
!bt
\[
\bm{K}[\bm{x},\bm{y}] = \begin{bmatrix} 1 & corr[\bm{x},\bm{y}] \\
corr[\bm{y},\bm{x}] & 1 \\
\bm{K}[\bm{x},\bm{y}] = \begin{bmatrix} 1 & \mathrm{corr}[\bm{x},\bm{y}] \\
\mathrm{corr}[\bm{y},\bm{x}] & 1 \\
\end{bmatrix},
\]
!et
@@ -432,12 +434,12 @@ With these definitions, we can now rewrite our $2\times 2$ correaltion/covarianc
!bt
\[
\bm{C}[\bm{x}] = \begin{bmatrix}
var[\bm{x}_0] & cov[\bm{x}_0,\bm{x}_1] & cov[\bm{x}_0,\bm{x}_2] & \dots & \dots & cov[\bm{x}_0,\bm{x}_{p-1}]\\
cov[\bm{x}_1,\bm{x}_0] & var[\bm{x}_1] & cov[\bm{x}_1,\bm{x}_2] & \dots & \dots & cov[\bm{x}_1,\bm{x}_{p-1}]\\
cov[\bm{x}_2,\bm{x}_0] & cov[\bm{x}_2,\bm{x}_1] & var[\bm{x}_2] & \dots & \dots & cov[\bm{x}_2,\bm{x}_{p-1}]\\
\mathrm{var}[\bm{x}_0] & \mathrm{cov}[\bm{x}_0,\bm{x}_1] & \mathrm{cov}[\bm{x}_0,\bm{x}_2] & \dots & \dots & \mathrm{cov}[\bm{x}_0,\bm{x}_{p-1}]\\
\mathrm{cov}[\bm{x}_1,\bm{x}_0] & \mathrm{var}[\bm{x}_1] & \mathrm{cov}[\bm{x}_1,\bm{x}_2] & \dots & \dots & \mathrm{cov}[\bm{x}_1,\bm{x}_{p-1}]\\
\mathrm{cov}[\bm{x}_2,\bm{x}_0] & \mathrm{cov}[\bm{x}_2,\bm{x}_1] & \mathrm{var}[\bm{x}_2] & \dots & \dots & \mathrm{cov}[\bm{x}_2,\bm{x}_{p-1}]\\
\dots & \dots & \dots & \dots & \dots & \dots \\
\dots & \dots & \dots & \dots & \dots & \dots \\
cov[\bm{x}_{p-1},\bm{x}_0] & cov[\bm{x}_{p-1},\bm{x}_1] & cov[\bm{x}_{p-1},\bm{x}_{2}] & \dots & \dots & var[\bm{x}_{p-1}]\\
\mathrm{cov}[\bm{x}_{p-1},\bm{x}_0] & \mathrm{cov}[\bm{x}_{p-1},\bm{x}_1] & \mathrm{cov}[\bm{x}_{p-1},\bm{x}_{2}] & \dots & \dots & \mathrm{var}[\bm{x}_{p-1}]\\
\end{bmatrix},
\]
!et
@@ -445,12 +447,12 @@ and the correlation matrix
!bt
\[
\bm{K}[\bm{x}] = \begin{bmatrix}
1 & corr[\bm{x}_0,\bm{x}_1] & corr[\bm{x}_0,\bm{x}_2] & \dots & \dots & corr[\bm{x}_0,\bm{x}_{p-1}]\\
corr[\bm{x}_1,\bm{x}_0] & 1 & corr[\bm{x}_1,\bm{x}_2] & \dots & \dots & corr[\bm{x}_1,\bm{x}_{p-1}]\\
corr[\bm{x}_2,\bm{x}_0] & corr[\bm{x}_2,\bm{x}_1] & 1 & \dots & \dots & corr[\bm{x}_2,\bm{x}_{p-1}]\\
1 & \mathrm{corr}[\bm{x}_0,\bm{x}_1] & \mathrm{corr}[\bm{x}_0,\bm{x}_2] & \dots & \dots & \mathrm{corr}[\bm{x}_0,\bm{x}_{p-1}]\\
\mathrm{corr}[\bm{x}_1,\bm{x}_0] & 1 & \mathrm{corr}[\bm{x}_1,\bm{x}_2] & \dots & \dots & \mathrm{corr}[\bm{x}_1,\bm{x}_{p-1}]\\
\mathrm{corr}[\bm{x}_2,\bm{x}_0] & \mathrm{corr}[\bm{x}_2,\bm{x}_1] & 1 & \dots & \dots & \mathrm{corr}[\bm{x}_2,\bm{x}_{p-1}]\\
\dots & \dots & \dots & \dots & \dots & \dots \\
\dots & \dots & \dots & \dots & \dots & \dots \\
corr[\bm{x}_{p-1},\bm{x}_0] & corr[\bm{x}_{p-1},\bm{x}_1] & corr[\bm{x}_{p-1},\bm{x}_{2}] & \dots & \dots & 1\\
\mathrm{corr}[\bm{x}_{p-1},\bm{x}_0] & \mathrm{corr}[\bm{x}_{p-1},\bm{x}_1] & \mathrm{corr}[\bm{x}_{p-1},\bm{x}_{2}] & \dots & \dots & 1\\
\end{bmatrix},
\]
!et