lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

more material

Browse Source
This commit is contained in:
mhjensen
2019-10-20 21:55:17 +02:00
parent c6c476bd42
commit cbaec5ba67
9 changed files with 228 additions and 27 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/pub/DimRed/html/._DimRed-bs008.html
+8
View File
@@ -163,6 +163,14 @@ MathJax.Hub.Config({
<h2 id="___sec7" class="anchor">Basic ideas of the Principal Component Analysis (PCA) </h2>
<p>
We have a data set defined by a design/feature matrix \( \boldsymbol{X} \) (see below for its definition)
<ul>
<li> So each data point is determined by \( p \) extrinsic (measurement) variables</li>
<li> We may want to ask the following question: Are there fewer intrinsic variables (say \( d < < p \)) that still approximately describe the data?</li>
<li> If so, these intrinsic variables may tell us something important and finding these intrinsic variables is what dimension reduction methods do</li>
</ul>
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
doc/pub/DimRed/html/._DimRed-bs009.html
+12 -2
View File
@@ -189,12 +189,22 @@ $$
$$
<p>
The covariance takes values between zero and infinity and may thus lead to problems with loss of numerical precision for particularly large values. It is common to scale the covariance matrix by introducing instead the correlation matrix defined via the so-called correlation function
The covariance takes values between zero and infinity and may thus
lead to problems with loss of numerical precision for particularly
large values. It is common to scale the covariance matrix by
introducing instead the correlation matrix defined via the so-called
correlation function
$$
corr[\boldsymbol{x},\boldsymbol{y}]=\frac{cov[\boldsymbol{x},\boldsymbol{y}]}{\sqrt{var[\boldsymbol{x}]\var\boldsymbol{y}]}}.
$$
The correlation function is then given by values \( corr[\boldsymbol{x},\boldsymbol{y}] \in [-1,1] \). This avoids eventual problems with too large values. We can then define the correlation matrix for the two vectors \( \boldsymbol{x} \) and \( \boldsymbol{y} \) as
<p>
The correlation function is then given by values \( corr[\boldsymbol{x},\boldsymbol{y}]
\in [-1,1] \). This avoids eventual problems with too large values. We
can then define the correlation matrix for the two vectors \( \boldsymbol{x} \)
and \( \boldsymbol{y} \) as
$$
\boldsymbol{K}[\boldsymbol{x},\boldsymbol{y}] = \begin{bmatrix} 1 & corr[\boldsymbol{x},\boldsymbol{y}] \\
corr[\boldsymbol{y},\boldsymbol{x}] & 1 \\
doc/pub/DimRed/html/DimRed-reveal.html
+43 -5
View File
@@ -509,6 +509,15 @@ applications.
<section>
<h2 id="___sec7">Basic ideas of the Principal Component Analysis (PCA) </h2>
<p>
We have a data set defined by a design/feature matrix \( \boldsymbol{X} \) (see below for its definition)
<ul>
<p><li> So each data point is determined by \( p \) extrinsic (measurement) variables</li>
<p><li> We may want to ask the following question: Are there fewer intrinsic variables (say \( d < < p \)) that still approximately describe the data?</li>
<p><li> If so, these intrinsic variables may tell us something important and finding these intrinsic variables is what dimension reduction methods do</li>
</ul>
</section>
@@ -550,14 +559,24 @@ $$
<p>&nbsp;<br>
<p>
The covariance takes values between zero and infinity and may thus lead to problems with loss of numerical precision for particularly large values. It is common to scale the covariance matrix by introducing instead the correlation matrix defined via the so-called correlation function
The covariance takes values between zero and infinity and may thus
lead to problems with loss of numerical precision for particularly
large values. It is common to scale the covariance matrix by
introducing instead the correlation matrix defined via the so-called
correlation function
<p>&nbsp;<br>
$$
corr[\boldsymbol{x},\boldsymbol{y}]=\frac{cov[\boldsymbol{x},\boldsymbol{y}]}{\sqrt{var[\boldsymbol{x}]\var\boldsymbol{y}]}}.
$$
<p>&nbsp;<br>
The correlation function is then given by values \( corr[\boldsymbol{x},\boldsymbol{y}] \in [-1,1] \). This avoids eventual problems with too large values. We can then define the correlation matrix for the two vectors \( \boldsymbol{x} \) and \( \boldsymbol{y} \) as
<p>
The correlation function is then given by values \( corr[\boldsymbol{x},\boldsymbol{y}]
\in [-1,1] \). This avoids eventual problems with too large values. We
can then define the correlation matrix for the two vectors \( \boldsymbol{x} \)
and \( \boldsymbol{y} \) as
<p>&nbsp;<br>
$$
\boldsymbol{K}[\boldsymbol{x},\boldsymbol{y}] = \begin{bmatrix} 1 & corr[\boldsymbol{x},\boldsymbol{y}] \\
@@ -609,9 +628,28 @@ $$
With these definitions, we can now rewrite our \( 2\times 2 \) correaltion/covariance matrix in terms of a moe general design/feature matrix \( \boldsymbol{X}\in {\mathbb{R}}^{n\times p} \). This leads to a \( p\times p \) covariance matrix for the vectors \( \boldsymbol{x}_i \) with \( i =0,1,\dots,p-1 \)
<p>&nbsp;<br>
$$
\boldsymbol{C}[\boldsymbol{x}] = \begin{bmatrix} var[\boldsymbol{x}_0] & cov[\boldsymbol{x}_0,\boldsymbol{x}_1] \\
cov[\boldsymbol{x},\boldsymbol{y}] & var[\boldsymbol{x}_{p-1}] \\
\end{bmatrix},
\boldsymbol{C}[\boldsymbol{x}] = \begin{bmatrix}
var[\boldsymbol{x}_0] & cov[\boldsymbol{x}_0,\boldsymbol{x}_1] & cov[\boldsymbol{x}_0,\boldsymbol{x}_2] & \dots & \dots & cov[\boldsymbol{x}_0,\boldsymbol{x}_{p-1}]\\
cov[\boldsymbol{x}_1,\boldsymbol{x}_0] & var[\boldsymbol{x}_1] & cov[\boldsymbol{x}_1,\boldsymbol{x}_2] & \dots & \dots & cov[\boldsymbol{x}_1,\boldsymbol{x}_{p-1}]\\
cov[\boldsymbol{x}_2,\boldsymbol{x}_0] & cov[\boldsymbol{x}_2,\boldsymbol{x}_1] & var[\boldsymbol{x}_2] & \dots & \dots & cov[\boldsymbol{x}_2,\boldsymbol{x}_{p-1}]\\
\dots & \dots & \dots & \dots & \dots & \dots \\
\dots & \dots & \dots & \dots & \dots & \dots \\
cov[\boldsymbol{x}_{p-1},\boldsymbol{x}_0] & cov[\boldsymbol{x}_{p-1},\boldsymbol{x}_1] & cov[\boldsymbol{x}_{p-1},\boldsymbol{x}_{2}] & \dots & \dots & var[\boldsymbol{x}_{p-1}]\\
\end{bmatrix},
$$
<p>&nbsp;<br>
and the correlation matrix
<p>&nbsp;<br>
$$
\boldsymbol{K}[\boldsymbol{x}] = \begin{bmatrix}
1 & corr[\boldsymbol{x}_0,\boldsymbol{x}_1] & corr[\boldsymbol{x}_0,\boldsymbol{x}_2] & \dots & \dots & corr[\boldsymbol{x}_0,\boldsymbol{x}_{p-1}]\\
corr[\boldsymbol{x}_1,\boldsymbol{x}_0] & 1 & corr[\boldsymbol{x}_1,\boldsymbol{x}_2] & \dots & \dots & corr[\boldsymbol{x}_1,\boldsymbol{x}_{p-1}]\\
corr[\boldsymbol{x}_2,\boldsymbol{x}_0] & corr[\boldsymbol{x}_2,\boldsymbol{x}_1] & 1 & \dots & \dots & corr[\boldsymbol{x}_2,\boldsymbol{x}_{p-1}]\\
\dots & \dots & \dots & \dots & \dots & \dots \\
\dots & \dots & \dots & \dots & \dots & \dots \\
corr[\boldsymbol{x}_{p-1},\boldsymbol{x}_0] & corr[\boldsymbol{x}_{p-1},\boldsymbol{x}_1] & corr[\boldsymbol{x}_{p-1},\boldsymbol{x}_{2}] & \dots & \dots & 1\\
\end{bmatrix},
$$
<p>&nbsp;<br>
doc/pub/DimRed/html/DimRed-solarized.html
+40 -5
View File
@@ -508,6 +508,14 @@ applications.
<h2 id="___sec7">Basic ideas of the Principal Component Analysis (PCA) </h2>
<p>
We have a data set defined by a design/feature matrix \( \boldsymbol{X} \) (see below for its definition)
<ul>
<li> So each data point is determined by \( p \) extrinsic (measurement) variables</li>
<li> We may want to ask the following question: Are there fewer intrinsic variables (say \( d < < p \)) that still approximately describe the data?</li>
<li> If so, these intrinsic variables may tell us something important and finding these intrinsic variables is what dimension reduction methods do</li>
</ul>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec8">Introducing the Covariance and Correlation functions </h2>
@@ -539,12 +547,22 @@ $$
$$
<p>
The covariance takes values between zero and infinity and may thus lead to problems with loss of numerical precision for particularly large values. It is common to scale the covariance matrix by introducing instead the correlation matrix defined via the so-called correlation function
The covariance takes values between zero and infinity and may thus
lead to problems with loss of numerical precision for particularly
large values. It is common to scale the covariance matrix by
introducing instead the correlation matrix defined via the so-called
correlation function
$$
corr[\boldsymbol{x},\boldsymbol{y}]=\frac{cov[\boldsymbol{x},\boldsymbol{y}]}{\sqrt{var[\boldsymbol{x}]\var\boldsymbol{y}]}}.
$$
The correlation function is then given by values \( corr[\boldsymbol{x},\boldsymbol{y}] \in [-1,1] \). This avoids eventual problems with too large values. We can then define the correlation matrix for the two vectors \( \boldsymbol{x} \) and \( \boldsymbol{y} \) as
<p>
The correlation function is then given by values \( corr[\boldsymbol{x},\boldsymbol{y}]
\in [-1,1] \). This avoids eventual problems with too large values. We
can then define the correlation matrix for the two vectors \( \boldsymbol{x} \)
and \( \boldsymbol{y} \) as
$$
\boldsymbol{K}[\boldsymbol{x},\boldsymbol{y}] = \begin{bmatrix} 1 & corr[\boldsymbol{x},\boldsymbol{y}] \\
corr[\boldsymbol{y},\boldsymbol{x}] & 1 \\
@@ -587,9 +605,26 @@ $$
<p>
With these definitions, we can now rewrite our \( 2\times 2 \) correaltion/covariance matrix in terms of a moe general design/feature matrix \( \boldsymbol{X}\in {\mathbb{R}}^{n\times p} \). This leads to a \( p\times p \) covariance matrix for the vectors \( \boldsymbol{x}_i \) with \( i =0,1,\dots,p-1 \)
$$
\boldsymbol{C}[\boldsymbol{x}] = \begin{bmatrix} var[\boldsymbol{x}_0] & cov[\boldsymbol{x}_0,\boldsymbol{x}_1] \\
cov[\boldsymbol{x},\boldsymbol{y}] & var[\boldsymbol{x}_{p-1}] \\
\end{bmatrix},
\boldsymbol{C}[\boldsymbol{x}] = \begin{bmatrix}
var[\boldsymbol{x}_0] & cov[\boldsymbol{x}_0,\boldsymbol{x}_1] & cov[\boldsymbol{x}_0,\boldsymbol{x}_2] & \dots & \dots & cov[\boldsymbol{x}_0,\boldsymbol{x}_{p-1}]\\
cov[\boldsymbol{x}_1,\boldsymbol{x}_0] & var[\boldsymbol{x}_1] & cov[\boldsymbol{x}_1,\boldsymbol{x}_2] & \dots & \dots & cov[\boldsymbol{x}_1,\boldsymbol{x}_{p-1}]\\
cov[\boldsymbol{x}_2,\boldsymbol{x}_0] & cov[\boldsymbol{x}_2,\boldsymbol{x}_1] & var[\boldsymbol{x}_2] & \dots & \dots & cov[\boldsymbol{x}_2,\boldsymbol{x}_{p-1}]\\
\dots & \dots & \dots & \dots & \dots & \dots \\
\dots & \dots & \dots & \dots & \dots & \dots \\
cov[\boldsymbol{x}_{p-1},\boldsymbol{x}_0] & cov[\boldsymbol{x}_{p-1},\boldsymbol{x}_1] & cov[\boldsymbol{x}_{p-1},\boldsymbol{x}_{2}] & \dots & \dots & var[\boldsymbol{x}_{p-1}]\\
\end{bmatrix},
$$
and the correlation matrix
$$
\boldsymbol{K}[\boldsymbol{x}] = \begin{bmatrix}
1 & corr[\boldsymbol{x}_0,\boldsymbol{x}_1] & corr[\boldsymbol{x}_0,\boldsymbol{x}_2] & \dots & \dots & corr[\boldsymbol{x}_0,\boldsymbol{x}_{p-1}]\\
corr[\boldsymbol{x}_1,\boldsymbol{x}_0] & 1 & corr[\boldsymbol{x}_1,\boldsymbol{x}_2] & \dots & \dots & corr[\boldsymbol{x}_1,\boldsymbol{x}_{p-1}]\\
corr[\boldsymbol{x}_2,\boldsymbol{x}_0] & corr[\boldsymbol{x}_2,\boldsymbol{x}_1] & 1 & \dots & \dots & corr[\boldsymbol{x}_2,\boldsymbol{x}_{p-1}]\\
\dots & \dots & \dots & \dots & \dots & \dots \\
\dots & \dots & \dots & \dots & \dots & \dots \\
corr[\boldsymbol{x}_{p-1},\boldsymbol{x}_0] & corr[\boldsymbol{x}_{p-1},\boldsymbol{x}_1] & corr[\boldsymbol{x}_{p-1},\boldsymbol{x}_{2}] & \dots & \dots & 1\\
\end{bmatrix},
$$
<p>
doc/pub/DimRed/html/DimRed.html
+40 -5
View File
@@ -513,6 +513,14 @@ applications.
<h2 id="___sec7">Basic ideas of the Principal Component Analysis (PCA) </h2>
<p>
We have a data set defined by a design/feature matrix \( \boldsymbol{X} \) (see below for its definition)
<ul>
<li> So each data point is determined by \( p \) extrinsic (measurement) variables</li>
<li> We may want to ask the following question: Are there fewer intrinsic variables (say \( d < < p \)) that still approximately describe the data?</li>
<li> If so, these intrinsic variables may tell us something important and finding these intrinsic variables is what dimension reduction methods do</li>
</ul>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec8">Introducing the Covariance and Correlation functions </h2>
@@ -544,12 +552,22 @@ $$
$$
<p>
The covariance takes values between zero and infinity and may thus lead to problems with loss of numerical precision for particularly large values. It is common to scale the covariance matrix by introducing instead the correlation matrix defined via the so-called correlation function
The covariance takes values between zero and infinity and may thus
lead to problems with loss of numerical precision for particularly
large values. It is common to scale the covariance matrix by
introducing instead the correlation matrix defined via the so-called
correlation function
$$
corr[\boldsymbol{x},\boldsymbol{y}]=\frac{cov[\boldsymbol{x},\boldsymbol{y}]}{\sqrt{var[\boldsymbol{x}]\var\boldsymbol{y}]}}.
$$
The correlation function is then given by values \( corr[\boldsymbol{x},\boldsymbol{y}] \in [-1,1] \). This avoids eventual problems with too large values. We can then define the correlation matrix for the two vectors \( \boldsymbol{x} \) and \( \boldsymbol{y} \) as
<p>
The correlation function is then given by values \( corr[\boldsymbol{x},\boldsymbol{y}]
\in [-1,1] \). This avoids eventual problems with too large values. We
can then define the correlation matrix for the two vectors \( \boldsymbol{x} \)
and \( \boldsymbol{y} \) as
$$
\boldsymbol{K}[\boldsymbol{x},\boldsymbol{y}] = \begin{bmatrix} 1 & corr[\boldsymbol{x},\boldsymbol{y}] \\
corr[\boldsymbol{y},\boldsymbol{x}] & 1 \\
@@ -592,9 +610,26 @@ $$
<p>
With these definitions, we can now rewrite our \( 2\times 2 \) correaltion/covariance matrix in terms of a moe general design/feature matrix \( \boldsymbol{X}\in {\mathbb{R}}^{n\times p} \). This leads to a \( p\times p \) covariance matrix for the vectors \( \boldsymbol{x}_i \) with \( i =0,1,\dots,p-1 \)
$$
\boldsymbol{C}[\boldsymbol{x}] = \begin{bmatrix} var[\boldsymbol{x}_0] & cov[\boldsymbol{x}_0,\boldsymbol{x}_1] \\
cov[\boldsymbol{x},\boldsymbol{y}] & var[\boldsymbol{x}_{p-1}] \\
\end{bmatrix},
\boldsymbol{C}[\boldsymbol{x}] = \begin{bmatrix}
var[\boldsymbol{x}_0] & cov[\boldsymbol{x}_0,\boldsymbol{x}_1] & cov[\boldsymbol{x}_0,\boldsymbol{x}_2] & \dots & \dots & cov[\boldsymbol{x}_0,\boldsymbol{x}_{p-1}]\\
cov[\boldsymbol{x}_1,\boldsymbol{x}_0] & var[\boldsymbol{x}_1] & cov[\boldsymbol{x}_1,\boldsymbol{x}_2] & \dots & \dots & cov[\boldsymbol{x}_1,\boldsymbol{x}_{p-1}]\\
cov[\boldsymbol{x}_2,\boldsymbol{x}_0] & cov[\boldsymbol{x}_2,\boldsymbol{x}_1] & var[\boldsymbol{x}_2] & \dots & \dots & cov[\boldsymbol{x}_2,\boldsymbol{x}_{p-1}]\\
\dots & \dots & \dots & \dots & \dots & \dots \\
\dots & \dots & \dots & \dots & \dots & \dots \\
cov[\boldsymbol{x}_{p-1},\boldsymbol{x}_0] & cov[\boldsymbol{x}_{p-1},\boldsymbol{x}_1] & cov[\boldsymbol{x}_{p-1},\boldsymbol{x}_{2}] & \dots & \dots & var[\boldsymbol{x}_{p-1}]\\
\end{bmatrix},
$$
and the correlation matrix
$$
\boldsymbol{K}[\boldsymbol{x}] = \begin{bmatrix}
1 & corr[\boldsymbol{x}_0,\boldsymbol{x}_1] & corr[\boldsymbol{x}_0,\boldsymbol{x}_2] & \dots & \dots & corr[\boldsymbol{x}_0,\boldsymbol{x}_{p-1}]\\
corr[\boldsymbol{x}_1,\boldsymbol{x}_0] & 1 & corr[\boldsymbol{x}_1,\boldsymbol{x}_2] & \dots & \dots & corr[\boldsymbol{x}_1,\boldsymbol{x}_{p-1}]\\
corr[\boldsymbol{x}_2,\boldsymbol{x}_0] & corr[\boldsymbol{x}_2,\boldsymbol{x}_1] & 1 & \dots & \dots & corr[\boldsymbol{x}_2,\boldsymbol{x}_{p-1}]\\
\dots & \dots & \dots & \dots & \dots & \dots \\
\dots & \dots & \dots & \dots & \dots & \dots \\
corr[\boldsymbol{x}_{p-1},\boldsymbol{x}_0] & corr[\boldsymbol{x}_{p-1},\boldsymbol{x}_1] & corr[\boldsymbol{x}_{p-1},\boldsymbol{x}_{2}] & \dots & \dots & 1\\
\end{bmatrix},
$$
<p>
doc/pub/DimRed/ipynb/DimRed.ipynb
+46 -5
View File
@@ -402,6 +402,12 @@
"\n",
"## Basic ideas of the Principal Component Analysis (PCA)\n",
"\n",
"We have a data set defined by a design/feature matrix $\\boldsymbol{X}$ (see below for its definition) \n",
"* So each data point is determined by $p$ extrinsic (measurement) variables\n",
"\n",
"* We may want to ask the following question: Are there fewer intrinsic variables (say $d << p$) that still approximately describe the data?\n",
"\n",
"* If so, these intrinsic variables may tell us something important and finding these intrinsic variables is what dimension reduction methods do\n",
"\n",
"## Introducing the Covariance and Correlation functions\n",
"\n",
@@ -474,7 +480,11 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"The covariance takes values between zero and infinity and may thus lead to problems with loss of numerical precision for particularly large values. It is common to scale the covariance matrix by introducing instead the correlation matrix defined via the so-called correlation function"
"The covariance takes values between zero and infinity and may thus\n",
"lead to problems with loss of numerical precision for particularly\n",
"large values. It is common to scale the covariance matrix by\n",
"introducing instead the correlation matrix defined via the so-called\n",
"correlation function"
]
},
{
@@ -490,7 +500,10 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"The correlation function is then given by values $corr[\\boldsymbol{x},\\boldsymbol{y}] \\in [-1,1]$. This avoids eventual problems with too large values. We can then define the correlation matrix for the two vectors $\\boldsymbol{x}$ and $\\boldsymbol{y}$ as"
"The correlation function is then given by values $corr[\\boldsymbol{x},\\boldsymbol{y}]\n",
"\\in [-1,1]$. This avoids eventual problems with too large values. We\n",
"can then define the correlation matrix for the two vectors $\\boldsymbol{x}$\n",
"and $\\boldsymbol{y}$ as"
]
},
{
@@ -577,9 +590,37 @@
"metadata": {},
"source": [
"$$\n",
"\\boldsymbol{C}[\\boldsymbol{x}] = \\begin{bmatrix} var[\\boldsymbol{x}_0] & cov[\\boldsymbol{x}_0,\\boldsymbol{x}_1] \\\\\n",
" cov[\\boldsymbol{x},\\boldsymbol{y}] & var[\\boldsymbol{x}_{p-1}] \\\\\n",
" \\end{bmatrix},\n",
"\\boldsymbol{C}[\\boldsymbol{x}] = \\begin{bmatrix}\n",
"var[\\boldsymbol{x}_0] & cov[\\boldsymbol{x}_0,\\boldsymbol{x}_1] & cov[\\boldsymbol{x}_0,\\boldsymbol{x}_2] & \\dots & \\dots & cov[\\boldsymbol{x}_0,\\boldsymbol{x}_{p-1}]\\\\\n",
"cov[\\boldsymbol{x}_1,\\boldsymbol{x}_0] & var[\\boldsymbol{x}_1] & cov[\\boldsymbol{x}_1,\\boldsymbol{x}_2] & \\dots & \\dots & cov[\\boldsymbol{x}_1,\\boldsymbol{x}_{p-1}]\\\\\n",
"cov[\\boldsymbol{x}_2,\\boldsymbol{x}_0] & cov[\\boldsymbol{x}_2,\\boldsymbol{x}_1] & var[\\boldsymbol{x}_2] & \\dots & \\dots & cov[\\boldsymbol{x}_2,\\boldsymbol{x}_{p-1}]\\\\\n",
"\\dots & \\dots & \\dots & \\dots & \\dots & \\dots \\\\\n",
"\\dots & \\dots & \\dots & \\dots & \\dots & \\dots \\\\\n",
"cov[\\boldsymbol{x}_{p-1},\\boldsymbol{x}_0] & cov[\\boldsymbol{x}_{p-1},\\boldsymbol{x}_1] & cov[\\boldsymbol{x}_{p-1},\\boldsymbol{x}_{2}] & \\dots & \\dots & var[\\boldsymbol{x}_{p-1}]\\\\\n",
"\\end{bmatrix},\n",
"$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"and the correlation matrix"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"$$\n",
"\\boldsymbol{K}[\\boldsymbol{x}] = \\begin{bmatrix}\n",
"1 & corr[\\boldsymbol{x}_0,\\boldsymbol{x}_1] & corr[\\boldsymbol{x}_0,\\boldsymbol{x}_2] & \\dots & \\dots & corr[\\boldsymbol{x}_0,\\boldsymbol{x}_{p-1}]\\\\\n",
"corr[\\boldsymbol{x}_1,\\boldsymbol{x}_0] & 1 & corr[\\boldsymbol{x}_1,\\boldsymbol{x}_2] & \\dots & \\dots & corr[\\boldsymbol{x}_1,\\boldsymbol{x}_{p-1}]\\\\\n",
"corr[\\boldsymbol{x}_2,\\boldsymbol{x}_0] & corr[\\boldsymbol{x}_2,\\boldsymbol{x}_1] & 1 & \\dots & \\dots & corr[\\boldsymbol{x}_2,\\boldsymbol{x}_{p-1}]\\\\\n",
"\\dots & \\dots & \\dots & \\dots & \\dots & \\dots \\\\\n",
"\\dots & \\dots & \\dots & \\dots & \\dots & \\dots \\\\\n",
"corr[\\boldsymbol{x}_{p-1},\\boldsymbol{x}_0] & corr[\\boldsymbol{x}_{p-1},\\boldsymbol{x}_1] & corr[\\boldsymbol{x}_{p-1},\\boldsymbol{x}_{2}] & \\dots & \\dots & 1\\\\\n",
"\\end{bmatrix},\n",
"$$"
]
},
doc/pub/DimRed/ipynb/ipynb-DimRed-src.tar.gz
BIN
View File
Binary file not shown.
doc/pub/DimRed/pdf/DimRed-minted.pdf
BIN
View File
Binary file not shown.
doc/src/DimRed/DimRed.do.txt
+39 -5
View File
@@ -331,6 +331,11 @@ applications.
!split
===== Basic ideas of the Principal Component Analysis (PCA) =====
We have a data set defined by a design/feature matrix $\bm{X}$ (see below for its definition)
* So each data point is determined by $p$ extrinsic (measurement) variables
* We may want to ask the following question: Are there fewer intrinsic variables (say $d << p$) that still approximately describe the data?
* If so, these intrinsic variables may tell us something important and finding these intrinsic variables is what dimension reduction methods do
!split
===== Introducing the Covariance and Correlation functions =====
@@ -365,13 +370,23 @@ we can rewrite the covariance matrix in this case as
\]
!et
The covariance takes values between zero and infinity and may thus lead to problems with loss of numerical precision for particularly large values. It is common to scale the covariance matrix by introducing instead the correlation matrix defined via the so-called correlation function
The covariance takes values between zero and infinity and may thus
lead to problems with loss of numerical precision for particularly
large values. It is common to scale the covariance matrix by
introducing instead the correlation matrix defined via the so-called
correlation function
!bt
\[
corr[\bm{x},\bm{y}]=\frac{cov[\bm{x},\bm{y}]}{\sqrt{var[\bm{x}]\var\bm{y}]}}.
\]
!et
The correlation function is then given by values $corr[\bm{x},\bm{y}] \in [-1,1]$. This avoids eventual problems with too large values. We can then define the correlation matrix for the two vectors $\bm{x}$ and $\bm{y}$ as
The correlation function is then given by values $corr[\bm{x},\bm{y}]
\in [-1,1]$. This avoids eventual problems with too large values. We
can then define the correlation matrix for the two vectors $\bm{x}$
and $\bm{y}$ as
!bt
\[
\bm{K}[\bm{x},\bm{y}] = \begin{bmatrix} 1 & corr[\bm{x},\bm{y}] \\
@@ -416,11 +431,30 @@ with a given vector
With these definitions, we can now rewrite our $2\times 2$ correaltion/covariance matrix in terms of a moe general design/feature matrix $\bm{X}\in {\mathbb{R}}^{n\times p}$. This leads to a $p\times p$ covariance matrix for the vectors $\bm{x}_i$ with $i =0,1,\dots,p-1$
!bt
\[
\bm{C}[\bm{x}] = \begin{bmatrix} var[\bm{x}_0] & cov[\bm{x}_0,\bm{x}_1] \\
cov[\bm{x},\bm{y}] & var[\bm{x}_{p-1}] \\
\end{bmatrix},
\bm{C}[\bm{x}] = \begin{bmatrix}
var[\bm{x}_0] & cov[\bm{x}_0,\bm{x}_1] & cov[\bm{x}_0,\bm{x}_2] & \dots & \dots & cov[\bm{x}_0,\bm{x}_{p-1}]\\
cov[\bm{x}_1,\bm{x}_0] & var[\bm{x}_1] & cov[\bm{x}_1,\bm{x}_2] & \dots & \dots & cov[\bm{x}_1,\bm{x}_{p-1}]\\
cov[\bm{x}_2,\bm{x}_0] & cov[\bm{x}_2,\bm{x}_1] & var[\bm{x}_2] & \dots & \dots & cov[\bm{x}_2,\bm{x}_{p-1}]\\
\dots & \dots & \dots & \dots & \dots & \dots \\
\dots & \dots & \dots & \dots & \dots & \dots \\
cov[\bm{x}_{p-1},\bm{x}_0] & cov[\bm{x}_{p-1},\bm{x}_1] & cov[\bm{x}_{p-1},\bm{x}_{2}] & \dots & \dots & var[\bm{x}_{p-1}]\\
\end{bmatrix},
\]
!et
and the correlation matrix
!bt
\[
\bm{K}[\bm{x}] = \begin{bmatrix}
1 & corr[\bm{x}_0,\bm{x}_1] & corr[\bm{x}_0,\bm{x}_2] & \dots & \dots & corr[\bm{x}_0,\bm{x}_{p-1}]\\
corr[\bm{x}_1,\bm{x}_0] & 1 & corr[\bm{x}_1,\bm{x}_2] & \dots & \dots & corr[\bm{x}_1,\bm{x}_{p-1}]\\
corr[\bm{x}_2,\bm{x}_0] & corr[\bm{x}_2,\bm{x}_1] & 1 & \dots & \dots & corr[\bm{x}_2,\bm{x}_{p-1}]\\
\dots & \dots & \dots & \dots & \dots & \dots \\
\dots & \dots & \dots & \dots & \dots & \dots \\
corr[\bm{x}_{p-1},\bm{x}_0] & corr[\bm{x}_{p-1},\bm{x}_1] & corr[\bm{x}_{p-1},\bm{x}_{2}] & \dots & \dots & 1\\
\end{bmatrix},
\]
!et