lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

typo in svm

Browse Source
This commit is contained in:
mhjensen
2019-11-22 06:54:36 +01:00
parent 89f488b991
commit a99959483f
11 changed files with 114 additions and 58 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/pub/svm/html/._svm-bs000.html
+1 -1
View File
@@ -174,7 +174,7 @@ MathJax.Hub.Config({
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>Nov 18, 2019</h4></center> <!-- date -->
<center><h4>Nov 22, 2019</h4></center> <!-- date -->
<br>
<p>
doc/pub/svm/html/._svm-bs020.html
+5 -2
View File
@@ -166,7 +166,7 @@ $$
<p>
With our new basis, the equations we solved earlier are basically the same, that is we have now (without the slack option for simplicity)
$$
{\cal L}=\sum_i\lambda_i-\frac{1}{2}\sum_{ij}^n\lambda_i\lambda_jy_iy_j\boldsymbol{x}_i^T\boldsymbol{z}_j,
{\cal L}=\sum_i\lambda_i-\frac{1}{2}\sum_{ij}^n\lambda_i\lambda_jy_iy_j\boldsymbol{z}_i^T\boldsymbol{z}_j,
$$
subject to the constraints \( \lambda_i\geq 0 \), \( \sum_i\lambda_iy_i=0 \), and for the support vectors
@@ -189,7 +189,10 @@ $$
We note that this is nothing but the dot product of the two original
vectors \( (\boldsymbol{x}_i^T\boldsymbol{x}_j)^2 \). Instead of thus computing the
product in the Lagrangian of \( \boldsymbol{z}_i^T\boldsymbol{z}_j \) we simply compute
the dot product \( (\boldsymbol{x}_i^T\boldsymbol{x}_j)^2 \). This leads to the so-called
the dot product \( (\boldsymbol{x}_i^T\boldsymbol{x}_j)^2 \).
<p>
This leads to the so-called
kernel trick and the result leads to the same as if we went through
the trouble of performing the transformation
\( (\phi(\boldsymbol{x}_i)^T\phi(\boldsymbol{x}_j) \) during the SVM calculations.
doc/pub/svm/html/._svm-bs022.html
+14 -7
View File
@@ -170,18 +170,25 @@ There are several popular kernels being used. These are
and many other ones.
<p>
An important theorem for us is <a href="https://en.wikipedia.org/wiki/Mercer%27s_theorem" target="_self">Mercer's theorem</a>.
The theorem states that if a kernel function \( K \) is symmetric, continuous and leads to a positive semi-definite matrix \( \boldsymbol{P} \) then
there exists a function \( \phi \) that maps \( \boldsymbol{x}_i \) and \( \boldsymbol{x}_j \) into another space
(possibly with much higher dimensions) such that
An important theorem for us is <a href="https://en.wikipedia.org/wiki/Mercer%27s_theorem" target="_self">Mercer's
theorem</a>. The
theorem states that if a kernel function \( K \) is symmetric, continuous
and leads to a positive semi-definite matrix \( \boldsymbol{P} \) then there
exists a function \( \phi \) that maps \( \boldsymbol{x}_i \) and \( \boldsymbol{x}_j \) into
another space (possibly with much higher dimensions) such that
$$
K(\boldsymbol{x}_i,\boldsymbol{x}_j)=\phi(\boldsymbol{x}_i)^T\phi(\boldsymbol{x}_j).
$$
<p>
So you can use \( K \) as a kernel since you know \( \phi \) exists, even if
you don&#8217;t know what \( \phi \) is.
Note that some frequently used kernels (such as the Sigmoid kernel) don&#8217;t respect all of Mercer&#8217;s conditions, yet they generally work
well in practice.
you don&#8217;t know what \( \phi \) is.
<p>
Note that some frequently used kernels (such as the Sigmoid kernel)
don&#8217;t respect all of Mercer&#8217;s conditions, yet they generally work well
in practice.
<p>
<p>
doc/pub/svm/html/svm-bs.html
+1 -1
View File
@@ -174,7 +174,7 @@ MathJax.Hub.Config({
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>Nov 18, 2019</h4></center> <!-- date -->
<center><h4>Nov 22, 2019</h4></center> <!-- date -->
<br>
<p>
doc/pub/svm/html/svm-reveal.html
+20 -10
View File
@@ -148,7 +148,7 @@ MathJax.Hub.Config({
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>&nbsp;<br>
<center><h4>Nov 18, 2019</h4></center> <!-- date -->
<center><h4>Nov 22, 2019</h4></center> <!-- date -->
<br>
<p>
@@ -988,7 +988,7 @@ $$
With our new basis, the equations we solved earlier are basically the same, that is we have now (without the slack option for simplicity)
<p>&nbsp;<br>
$$
{\cal L}=\sum_i\lambda_i-\frac{1}{2}\sum_{ij}^n\lambda_i\lambda_jy_iy_j\boldsymbol{x}_i^T\boldsymbol{z}_j,
{\cal L}=\sum_i\lambda_i-\frac{1}{2}\sum_{ij}^n\lambda_i\lambda_jy_iy_j\boldsymbol{z}_i^T\boldsymbol{z}_j,
$$
<p>&nbsp;<br>
@@ -1018,7 +1018,10 @@ $$
We note that this is nothing but the dot product of the two original
vectors \( (\boldsymbol{x}_i^T\boldsymbol{x}_j)^2 \). Instead of thus computing the
product in the Lagrangian of \( \boldsymbol{z}_i^T\boldsymbol{z}_j \) we simply compute
the dot product \( (\boldsymbol{x}_i^T\boldsymbol{x}_j)^2 \). This leads to the so-called
the dot product \( (\boldsymbol{x}_i^T\boldsymbol{x}_j)^2 \).
<p>
This leads to the so-called
kernel trick and the result leads to the same as if we went through
the trouble of performing the transformation
\( (\phi(\boldsymbol{x}_i)^T\phi(\boldsymbol{x}_j) \) during the SVM calculations.
@@ -1084,20 +1087,27 @@ There are several popular kernels being used. These are
and many other ones.
<p>
An important theorem for us is <a href="https://en.wikipedia.org/wiki/Mercer%27s_theorem" target="_blank">Mercer's theorem</a>.
The theorem states that if a kernel function \( K \) is symmetric, continuous and leads to a positive semi-definite matrix \( \boldsymbol{P} \) then
there exists a function \( \phi \) that maps \( \boldsymbol{x}_i \) and \( \boldsymbol{x}_j \) into another space
(possibly with much higher dimensions) such that
An important theorem for us is <a href="https://en.wikipedia.org/wiki/Mercer%27s_theorem" target="_blank">Mercer's
theorem</a>. The
theorem states that if a kernel function \( K \) is symmetric, continuous
and leads to a positive semi-definite matrix \( \boldsymbol{P} \) then there
exists a function \( \phi \) that maps \( \boldsymbol{x}_i \) and \( \boldsymbol{x}_j \) into
another space (possibly with much higher dimensions) such that
<p>&nbsp;<br>
$$
K(\boldsymbol{x}_i,\boldsymbol{x}_j)=\phi(\boldsymbol{x}_i)^T\phi(\boldsymbol{x}_j).
$$
<p>&nbsp;<br>
<p>
So you can use \( K \) as a kernel since you know \( \phi \) exists, even if
you don&#8217;t know what \( \phi \) is.
Note that some frequently used kernels (such as the Sigmoid kernel) don&#8217;t respect all of Mercer&#8217;s conditions, yet they generally work
well in practice.
you don&#8217;t know what \( \phi \) is.
<p>
Note that some frequently used kernels (such as the Sigmoid kernel)
don&#8217;t respect all of Mercer&#8217;s conditions, yet they generally work well
in practice.
</section>
doc/pub/svm/html/svm-solarized.html
+20 -10
View File
@@ -110,7 +110,7 @@ MathJax.Hub.Config({
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>Nov 18, 2019</h4></center> <!-- date -->
<center><h4>Nov 22, 2019</h4></center> <!-- date -->
<br>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
@@ -831,7 +831,7 @@ $$
<p>
With our new basis, the equations we solved earlier are basically the same, that is we have now (without the slack option for simplicity)
$$
{\cal L}=\sum_i\lambda_i-\frac{1}{2}\sum_{ij}^n\lambda_i\lambda_jy_iy_j\boldsymbol{x}_i^T\boldsymbol{z}_j,
{\cal L}=\sum_i\lambda_i-\frac{1}{2}\sum_{ij}^n\lambda_i\lambda_jy_iy_j\boldsymbol{z}_i^T\boldsymbol{z}_j,
$$
subject to the constraints \( \lambda_i\geq 0 \), \( \sum_i\lambda_iy_i=0 \), and for the support vectors
@@ -854,7 +854,10 @@ $$
We note that this is nothing but the dot product of the two original
vectors \( (\boldsymbol{x}_i^T\boldsymbol{x}_j)^2 \). Instead of thus computing the
product in the Lagrangian of \( \boldsymbol{z}_i^T\boldsymbol{z}_j \) we simply compute
the dot product \( (\boldsymbol{x}_i^T\boldsymbol{x}_j)^2 \). This leads to the so-called
the dot product \( (\boldsymbol{x}_i^T\boldsymbol{x}_j)^2 \).
<p>
This leads to the so-called
kernel trick and the result leads to the same as if we went through
the trouble of performing the transformation
\( (\phi(\boldsymbol{x}_i)^T\phi(\boldsymbol{x}_j) \) during the SVM calculations.
@@ -913,18 +916,25 @@ There are several popular kernels being used. These are
and many other ones.
<p>
An important theorem for us is <a href="https://en.wikipedia.org/wiki/Mercer%27s_theorem" target="_blank">Mercer's theorem</a>.
The theorem states that if a kernel function \( K \) is symmetric, continuous and leads to a positive semi-definite matrix \( \boldsymbol{P} \) then
there exists a function \( \phi \) that maps \( \boldsymbol{x}_i \) and \( \boldsymbol{x}_j \) into another space
(possibly with much higher dimensions) such that
An important theorem for us is <a href="https://en.wikipedia.org/wiki/Mercer%27s_theorem" target="_blank">Mercer's
theorem</a>. The
theorem states that if a kernel function \( K \) is symmetric, continuous
and leads to a positive semi-definite matrix \( \boldsymbol{P} \) then there
exists a function \( \phi \) that maps \( \boldsymbol{x}_i \) and \( \boldsymbol{x}_j \) into
another space (possibly with much higher dimensions) such that
$$
K(\boldsymbol{x}_i,\boldsymbol{x}_j)=\phi(\boldsymbol{x}_i)^T\phi(\boldsymbol{x}_j).
$$
<p>
So you can use \( K \) as a kernel since you know \( \phi \) exists, even if
you don&#8217;t know what \( \phi \) is.
Note that some frequently used kernels (such as the Sigmoid kernel) don&#8217;t respect all of Mercer&#8217;s conditions, yet they generally work
well in practice.
you don&#8217;t know what \( \phi \) is.
<p>
Note that some frequently used kernels (such as the Sigmoid kernel)
don&#8217;t respect all of Mercer&#8217;s conditions, yet they generally work well
in practice.
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
doc/pub/svm/html/svm.html
+20 -10
View File
@@ -115,7 +115,7 @@ MathJax.Hub.Config({
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>Nov 18, 2019</h4></center> <!-- date -->
<center><h4>Nov 22, 2019</h4></center> <!-- date -->
<br>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
@@ -836,7 +836,7 @@ $$
<p>
With our new basis, the equations we solved earlier are basically the same, that is we have now (without the slack option for simplicity)
$$
{\cal L}=\sum_i\lambda_i-\frac{1}{2}\sum_{ij}^n\lambda_i\lambda_jy_iy_j\boldsymbol{x}_i^T\boldsymbol{z}_j,
{\cal L}=\sum_i\lambda_i-\frac{1}{2}\sum_{ij}^n\lambda_i\lambda_jy_iy_j\boldsymbol{z}_i^T\boldsymbol{z}_j,
$$
subject to the constraints \( \lambda_i\geq 0 \), \( \sum_i\lambda_iy_i=0 \), and for the support vectors
@@ -859,7 +859,10 @@ $$
We note that this is nothing but the dot product of the two original
vectors \( (\boldsymbol{x}_i^T\boldsymbol{x}_j)^2 \). Instead of thus computing the
product in the Lagrangian of \( \boldsymbol{z}_i^T\boldsymbol{z}_j \) we simply compute
the dot product \( (\boldsymbol{x}_i^T\boldsymbol{x}_j)^2 \). This leads to the so-called
the dot product \( (\boldsymbol{x}_i^T\boldsymbol{x}_j)^2 \).
<p>
This leads to the so-called
kernel trick and the result leads to the same as if we went through
the trouble of performing the transformation
\( (\phi(\boldsymbol{x}_i)^T\phi(\boldsymbol{x}_j) \) during the SVM calculations.
@@ -918,18 +921,25 @@ There are several popular kernels being used. These are
and many other ones.
<p>
An important theorem for us is <a href="https://en.wikipedia.org/wiki/Mercer%27s_theorem" target="_blank">Mercer's theorem</a>.
The theorem states that if a kernel function \( K \) is symmetric, continuous and leads to a positive semi-definite matrix \( \boldsymbol{P} \) then
there exists a function \( \phi \) that maps \( \boldsymbol{x}_i \) and \( \boldsymbol{x}_j \) into another space
(possibly with much higher dimensions) such that
An important theorem for us is <a href="https://en.wikipedia.org/wiki/Mercer%27s_theorem" target="_blank">Mercer's
theorem</a>. The
theorem states that if a kernel function \( K \) is symmetric, continuous
and leads to a positive semi-definite matrix \( \boldsymbol{P} \) then there
exists a function \( \phi \) that maps \( \boldsymbol{x}_i \) and \( \boldsymbol{x}_j \) into
another space (possibly with much higher dimensions) such that
$$
K(\boldsymbol{x}_i,\boldsymbol{x}_j)=\phi(\boldsymbol{x}_i)^T\phi(\boldsymbol{x}_j).
$$
<p>
So you can use \( K \) as a kernel since you know \( \phi \) exists, even if
you don&#8217;t know what \( \phi \) is.
Note that some frequently used kernels (such as the Sigmoid kernel) don&#8217;t respect all of Mercer&#8217;s conditions, yet they generally work
well in practice.
you don&#8217;t know what \( \phi \) is.
<p>
Note that some frequently used kernels (such as the Sigmoid kernel)
don&#8217;t respect all of Mercer&#8217;s conditions, yet they generally work well
in practice.
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
doc/pub/svm/ipynb/ipynb-svm-src.tar.gz
BIN
View File
Binary file not shown.
doc/pub/svm/ipynb/svm.ipynb
+16 -9
View File
@@ -10,7 +10,7 @@
"<!-- Author: --> \n",
"**Morten Hjorth-Jensen**, Department of Physics, University of Oslo and Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University\n",
"\n",
"Date: **Nov 18, 2019**\n",
"Date: **Nov 22, 2019**\n",
"\n",
"Copyright 1999-2019, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license\n",
"\n",
@@ -1267,7 +1267,7 @@
"metadata": {},
"source": [
"$$\n",
"{\\cal L}=\\sum_i\\lambda_i-\\frac{1}{2}\\sum_{ij}^n\\lambda_i\\lambda_jy_iy_j\\boldsymbol{x}_i^T\\boldsymbol{z}_j,\n",
"{\\cal L}=\\sum_i\\lambda_i-\\frac{1}{2}\\sum_{ij}^n\\lambda_i\\lambda_jy_iy_j\\boldsymbol{z}_i^T\\boldsymbol{z}_j,\n",
"$$"
]
},
@@ -1327,7 +1327,10 @@
"We note that this is nothing but the dot product of the two original\n",
"vectors $(\\boldsymbol{x}_i^T\\boldsymbol{x}_j)^2$. Instead of thus computing the\n",
"product in the Lagrangian of $\\boldsymbol{z}_i^T\\boldsymbol{z}_j$ we simply compute\n",
"the dot product $(\\boldsymbol{x}_i^T\\boldsymbol{x}_j)^2$. This leads to the so-called\n",
"the dot product $(\\boldsymbol{x}_i^T\\boldsymbol{x}_j)^2$.\n",
"\n",
"\n",
"This leads to the so-called\n",
"kernel trick and the result leads to the same as if we went through\n",
"the trouble of performing the transformation\n",
"$(\\phi(\\boldsymbol{x}_i)^T\\phi(\\boldsymbol{x}_j)$ during the SVM calculations.\n",
@@ -1412,10 +1415,12 @@
"\n",
"and many other ones.\n",
"\n",
"An important theorem for us is [Mercer's theorem](https://en.wikipedia.org/wiki/Mercer%27s_theorem). \n",
"The theorem states that if a kernel function $K$ is symmetric, continuous and leads to a positive semi-definite matrix $\\boldsymbol{P}$ then\n",
"there exists a function $\\phi$ that maps $\\boldsymbol{x}_i$ and $\\boldsymbol{x}_j$ into another space\n",
"(possibly with much higher dimensions) such that"
"An important theorem for us is [Mercer's\n",
"theorem](https://en.wikipedia.org/wiki/Mercer%27s_theorem). The\n",
"theorem states that if a kernel function $K$ is symmetric, continuous\n",
"and leads to a positive semi-definite matrix $\\boldsymbol{P}$ then there\n",
"exists a function $\\phi$ that maps $\\boldsymbol{x}_i$ and $\\boldsymbol{x}_j$ into\n",
"another space (possibly with much higher dimensions) such that"
]
},
{
@@ -1433,8 +1438,10 @@
"source": [
"So you can use $K$ as a kernel since you know $\\phi$ exists, even if\n",
"you don’t know what $\\phi$ is. \n",
"Note that some frequently used kernels (such as the Sigmoid kernel) don’t respect all of Mercer’s conditions, yet they generally work\n",
"well in practice.\n",
"\n",
"Note that some frequently used kernels (such as the Sigmoid kernel)\n",
"don’t respect all of Mercer’s conditions, yet they generally work well\n",
"in practice.\n",
"\n",
"\n",
"## The moons example"
doc/pub/svm/pdf/svm-minted.pdf
BIN
View File
Binary file not shown.
doc/src/SupportVMachines/svm.do.txt
+17 -8
View File
@@ -708,7 +708,7 @@ z = \phi(x_i) =\left(x_i^2, y_i^2, \sqrt{2}x_iy_i\right).
With our new basis, the equations we solved earlier are basically the same, that is we have now (without the slack option for simplicity)
!bt
\[
{\cal L}=\sum_i\lambda_i-\frac{1}{2}\sum_{ij}^n\lambda_i\lambda_jy_iy_j\bm{x}_i^T\bm{z}_j,
{\cal L}=\sum_i\lambda_i-\frac{1}{2}\sum_{ij}^n\lambda_i\lambda_jy_iy_j\bm{z}_i^T\bm{z}_j,
\]
!et
subject to the constraints $\lambda_i\geq 0$, $\sum_i\lambda_iy_i=0$, and for the support vectors
@@ -734,7 +734,10 @@ K(\bm{x}_i,\bm{x}_j)=[x_i^2, y_i^2, \sqrt{2}x_iy_i]^T\begin{bmatrix} x_j^2 \\ y_
We note that this is nothing but the dot product of the two original
vectors $(\bm{x}_i^T\bm{x}_j)^2$. Instead of thus computing the
product in the Lagrangian of $\bm{z}_i^T\bm{z}_j$ we simply compute
the dot product $(\bm{x}_i^T\bm{x}_j)^2$. This leads to the so-called
the dot product $(\bm{x}_i^T\bm{x}_j)^2$.
This leads to the so-called
kernel trick and the result leads to the same as if we went through
the trouble of performing the transformation
$(\phi(\bm{x}_i)^T\phi(\bm{x}_j)$ during the SVM calculations.
@@ -785,19 +788,25 @@ o Gaussian Radial Basis Function: $K(\bm{x},\bm{y})=\exp{\left(-\gamma\vert\vert
o Tanh: $K(\bm{x},\bm{y})=\tanh{(\bm{x}^T\bm{y}+\gamma)}$,
and many other ones.
An important theorem for us is "Mercer's theorem":"https://en.wikipedia.org/wiki/Mercer%27s_theorem".
The theorem states that if a kernel function $K$ is symmetric, continuous and leads to a positive semi-definite matrix $\bm{P}$ then
there exists a function $\phi$ that maps $\bm{x}_i$ and $\bm{x}_j$ into another space
(possibly with much higher dimensions) such that
An important theorem for us is "Mercer's
theorem":"https://en.wikipedia.org/wiki/Mercer%27s_theorem". The
theorem states that if a kernel function $K$ is symmetric, continuous
and leads to a positive semi-definite matrix $\bm{P}$ then there
exists a function $\phi$ that maps $\bm{x}_i$ and $\bm{x}_j$ into
another space (possibly with much higher dimensions) such that
!bt
\[
K(\bm{x}_i,\bm{x}_j)=\phi(\bm{x}_i)^T\phi(\bm{x}_j).
\]
!et
So you can use $K$ as a kernel since you know $\phi$ exists, even if
you don’t know what $\phi$ is.
Note that some frequently used kernels (such as the Sigmoid kernel) don’t respect all of Mercer’s conditions, yet they generally work
well in practice.
Note that some frequently used kernels (such as the Sigmoid kernel)
don’t respect all of Mercer’s conditions, yet they generally work well
in practice.
!split