diff --git a/doc/pub/week47/html/week47-bs.html b/doc/pub/week47/html/week47-bs.html
index 527cec9d1..211f2600c 100644
--- a/doc/pub/week47/html/week47-bs.html
+++ b/doc/pub/week47/html/week47-bs.html
@@ -89,10 +89,10 @@ doconce format html week47.do.txt --html_style=bootstrap --pygments_html_style=d
None,
'how-do-we-solve-these-problems'),
('A simple example', 2, None, 'a-simple-example'),
- ('Back to the more realistic cases',
+ ('Support Vector Machines and Regression',
2,
None,
- 'back-to-the-more-realistic-cases'),
+ 'support-vector-machines-and-regression'),
('Summary of course', 2, None, 'summary-of-course'),
('What? Me worry? No final exam in this course!',
2,
@@ -289,7 +289,7 @@ MathJax.Hub.Config({
Mathematical optimization of convex functions
How do we solve these problems?
A simple example
- Back to the more realistic cases
+ Support Vector Machines and Regression
Summary of course
What? Me worry? No final exam in this course!
What is the link between Artificial Intelligence and Machine Learning and some general Remarks
diff --git a/doc/pub/week47/html/week47-reveal.html b/doc/pub/week47/html/week47-reveal.html
index 7ac2ab4cb..1e3141286 100644
--- a/doc/pub/week47/html/week47-reveal.html
+++ b/doc/pub/week47/html/week47-reveal.html
@@ -567,7 +567,7 @@ the two classes (if they are well separated from the beginning).
$$
-y_i(\boldsymbol{w}^T\boldsymbol{x}_i+b) \geq M \hspace{0.1cm}\forall i=1,2,\dots, p.
+y_i(\boldsymbol{w}^T\boldsymbol{x}_i+b) \geq M \hspace{0.1cm}\forall i=1,2,\dots, n.
$$
@@ -583,16 +583,16 @@ $$
or just
$$
-y_i(\boldsymbol{w}^T\boldsymbol{x}_i+b) \geq M\vert \vert \boldsymbol{w}\vert\vert \hspace{0.1cm}\forall i.
+y_i(\boldsymbol{w}^T\boldsymbol{x}_i+b) \geq M\vert \vert \boldsymbol{w}\vert\vert \hspace{0.1cm}\forall i=1,2,\dots,n.
$$
If we scale the equation so that \( \vert \vert \boldsymbol{w}\vert\vert = 1/M \), we have to find the minimum of
-\( \boldsymbol{w}^T\boldsymbol{w}=\vert \vert \boldsymbol{w}\vert\vert \) (the norm) subject to the condition
+\( \boldsymbol{w}^T\boldsymbol{w}=\vert \vert \boldsymbol{w}\vert\vert_2^2 \) (the norm) subject to the condition
$$
-y_i(\boldsymbol{w}^T\boldsymbol{x}_i+b) \geq 1 \hspace{0.1cm}\forall i.
+y_i(\boldsymbol{w}^T\boldsymbol{x}_i+b) \geq 1 \hspace{0.1cm}\forall i=1,2,\dots,n.
$$
@@ -791,8 +791,8 @@ y_ny_1\boldsymbol{x}_n^T\boldsymbol{x}_1 & y_ny_2\boldsymbol{x}_n^T\boldsymbol{x
$$
-
subject to \( \boldsymbol{y}^T\boldsymbol{\lambda}=0 \). Here we defined the vectors \( \boldsymbol{\lambda} =[\lambda_1,\lambda_2,\dots,\lambda_n] \) and
-\( \boldsymbol{y}=[y_1,y_2,\dots,y_n] \).
+
subject to \( \boldsymbol{y}^T\boldsymbol{\lambda}=0 \). Here we defined the vectors \( \boldsymbol{\lambda}^T =[\lambda_1,\lambda_2,\dots,\lambda_n] \) and
+\( \boldsymbol{y}^T=[y_1,y_2,\dots,y_n] \).
@@ -1545,26 +1545,9 @@ sol[’primal obj
-Back to the more realistic cases
+Support Vector Machines and Regression
-We are now ready to return to our setup of the optmization problem for a more realistic case. Introducing the slack parameter \( C \) we have
-
-$$
-\frac{1}{2} \boldsymbol{\lambda}^T\begin{bmatrix} y_1y_1K(\boldsymbol{x}_1,\boldsymbol{x}_1) & y_1y_2K(\boldsymbol{x}_1,\boldsymbol{x}_2) & \dots & \dots & y_1y_nK(\boldsymbol{x}_1,\boldsymbol{x}_n) \\
-y_2y_1K(\boldsymbol{x}_2,\boldsymbol{x}_1) & y_2y_2K(\boldsymbol{x}_2,\boldsymbol{x}_2) & \dots & \dots & y_1y_nK(\boldsymbol{x}_2,\boldsymbol{x}_n) \\
-\dots & \dots & \dots & \dots & \dots \\
-\dots & \dots & \dots & \dots & \dots \\
-y_ny_1K(\boldsymbol{x}_n,\boldsymbol{x}_1) & y_ny_2K(\boldsymbol{x}_n\boldsymbol{x}_2) & \dots & \dots & y_ny_nK(\boldsymbol{x}_n,\boldsymbol{x}_n) \\
-\end{bmatrix}\boldsymbol{\lambda}-\mathbb{I}\boldsymbol{\lambda},
-$$
-
-
-
subject to \( \boldsymbol{y}^T\boldsymbol{\lambda}=0 \). Here we defined the vectors \( \boldsymbol{\lambda} =[\lambda_1,\lambda_2,\dots,\lambda_n] \) and
-\( \boldsymbol{y}=[y_1,y_2,\dots,y_n] \).
-With the slack constants this leads to the additional constraint \( 0\leq \lambda_i \leq C \).
-
-
-code will be added
+Material may be added if of interest. See Bishop chapter 7.1 for a discussion.
diff --git a/doc/pub/week47/html/week47-solarized.html b/doc/pub/week47/html/week47-solarized.html
index f13d51e70..408c50854 100644
--- a/doc/pub/week47/html/week47-solarized.html
+++ b/doc/pub/week47/html/week47-solarized.html
@@ -116,10 +116,10 @@ div.toc p,a {
None,
'how-do-we-solve-these-problems'),
('A simple example', 2, None, 'a-simple-example'),
- ('Back to the more realistic cases',
+ ('Support Vector Machines and Regression',
2,
None,
- 'back-to-the-more-realistic-cases'),
+ 'support-vector-machines-and-regression'),
('Summary of course', 2, None, 'summary-of-course'),
('What? Me worry? No final exam in this course!',
2,
@@ -632,7 +632,7 @@ the two classes (if they are well separated from the beginning).
$$
-y_i(\boldsymbol{w}^T\boldsymbol{x}_i+b) \geq M \hspace{0.1cm}\forall i=1,2,\dots, p.
+y_i(\boldsymbol{w}^T\boldsymbol{x}_i+b) \geq M \hspace{0.1cm}\forall i=1,2,\dots, n.
$$
All points are thus at a signed distance from the decision boundary defined by the line \( L \). The parameters \( b \) and \( w_1 \) and \( w_2 \) define this line.
@@ -644,14 +644,14 @@ $$
or just
$$
-y_i(\boldsymbol{w}^T\boldsymbol{x}_i+b) \geq M\vert \vert \boldsymbol{w}\vert\vert \hspace{0.1cm}\forall i.
+y_i(\boldsymbol{w}^T\boldsymbol{x}_i+b) \geq M\vert \vert \boldsymbol{w}\vert\vert \hspace{0.1cm}\forall i=1,2,\dots,n.
$$
If we scale the equation so that \( \vert \vert \boldsymbol{w}\vert\vert = 1/M \), we have to find the minimum of
-\( \boldsymbol{w}^T\boldsymbol{w}=\vert \vert \boldsymbol{w}\vert\vert \) (the norm) subject to the condition
+\( \boldsymbol{w}^T\boldsymbol{w}=\vert \vert \boldsymbol{w}\vert\vert_2^2 \) (the norm) subject to the condition
$$
-y_i(\boldsymbol{w}^T\boldsymbol{x}_i+b) \geq 1 \hspace{0.1cm}\forall i.
+y_i(\boldsymbol{w}^T\boldsymbol{x}_i+b) \geq 1 \hspace{0.1cm}\forall i=1,2,\dots,n.
$$
We have thus defined our margin as the invers of the norm of
@@ -807,8 +807,8 @@ y_ny_1\boldsymbol{x}_n^T\boldsymbol{x}_1 & y_ny_2\boldsymbol{x}_n^T\boldsymbol{x
\end{bmatrix}\boldsymbol{\lambda}-\mathbb{1}\boldsymbol{\lambda},
$$
-
subject to \( \boldsymbol{y}^T\boldsymbol{\lambda}=0 \). Here we defined the vectors \( \boldsymbol{\lambda} =[\lambda_1,\lambda_2,\dots,\lambda_n] \) and
-\( \boldsymbol{y}=[y_1,y_2,\dots,y_n] \).
+
subject to \( \boldsymbol{y}^T\boldsymbol{\lambda}=0 \). Here we defined the vectors \( \boldsymbol{\lambda}^T =[\lambda_1,\lambda_2,\dots,\lambda_n] \) and
+\( \boldsymbol{y}^T=[y_1,y_2,\dots,y_n] \).
@@ -1488,24 +1488,9 @@ sol[’primal obj
-Back to the more realistic cases
+Support Vector Machines and Regression
-We are now ready to return to our setup of the optmization problem for a more realistic case. Introducing the slack parameter \( C \) we have
-$$
-\frac{1}{2} \boldsymbol{\lambda}^T\begin{bmatrix} y_1y_1K(\boldsymbol{x}_1,\boldsymbol{x}_1) & y_1y_2K(\boldsymbol{x}_1,\boldsymbol{x}_2) & \dots & \dots & y_1y_nK(\boldsymbol{x}_1,\boldsymbol{x}_n) \\
-y_2y_1K(\boldsymbol{x}_2,\boldsymbol{x}_1) & y_2y_2K(\boldsymbol{x}_2,\boldsymbol{x}_2) & \dots & \dots & y_1y_nK(\boldsymbol{x}_2,\boldsymbol{x}_n) \\
-\dots & \dots & \dots & \dots & \dots \\
-\dots & \dots & \dots & \dots & \dots \\
-y_ny_1K(\boldsymbol{x}_n,\boldsymbol{x}_1) & y_ny_2K(\boldsymbol{x}_n\boldsymbol{x}_2) & \dots & \dots & y_ny_nK(\boldsymbol{x}_n,\boldsymbol{x}_n) \\
-\end{bmatrix}\boldsymbol{\lambda}-\mathbb{I}\boldsymbol{\lambda},
-$$
-
-subject to \( \boldsymbol{y}^T\boldsymbol{\lambda}=0 \). Here we defined the vectors \( \boldsymbol{\lambda} =[\lambda_1,\lambda_2,\dots,\lambda_n] \) and
-\( \boldsymbol{y}=[y_1,y_2,\dots,y_n] \).
-With the slack constants this leads to the additional constraint \( 0\leq \lambda_i \leq C \).
-
-
-code will be added
+Material may be added if of interest. See Bishop chapter 7.1 for a discussion.
Summary of course
diff --git a/doc/pub/week47/html/week47.html b/doc/pub/week47/html/week47.html
index d39d65e2c..408daae09 100644
--- a/doc/pub/week47/html/week47.html
+++ b/doc/pub/week47/html/week47.html
@@ -193,10 +193,10 @@ div.toc p,a {
None,
'how-do-we-solve-these-problems'),
('A simple example', 2, None, 'a-simple-example'),
- ('Back to the more realistic cases',
+ ('Support Vector Machines and Regression',
2,
None,
- 'back-to-the-more-realistic-cases'),
+ 'support-vector-machines-and-regression'),
('Summary of course', 2, None, 'summary-of-course'),
('What? Me worry? No final exam in this course!',
2,
@@ -709,7 +709,7 @@ the two classes (if they are well separated from the beginning).
$$
-y_i(\boldsymbol{w}^T\boldsymbol{x}_i+b) \geq M \hspace{0.1cm}\forall i=1,2,\dots, p.
+y_i(\boldsymbol{w}^T\boldsymbol{x}_i+b) \geq M \hspace{0.1cm}\forall i=1,2,\dots, n.
$$
All points are thus at a signed distance from the decision boundary defined by the line \( L \). The parameters \( b \) and \( w_1 \) and \( w_2 \) define this line.
@@ -721,14 +721,14 @@ $$
or just
$$
-y_i(\boldsymbol{w}^T\boldsymbol{x}_i+b) \geq M\vert \vert \boldsymbol{w}\vert\vert \hspace{0.1cm}\forall i.
+y_i(\boldsymbol{w}^T\boldsymbol{x}_i+b) \geq M\vert \vert \boldsymbol{w}\vert\vert \hspace{0.1cm}\forall i=1,2,\dots,n.
$$
If we scale the equation so that \( \vert \vert \boldsymbol{w}\vert\vert = 1/M \), we have to find the minimum of
-\( \boldsymbol{w}^T\boldsymbol{w}=\vert \vert \boldsymbol{w}\vert\vert \) (the norm) subject to the condition
+\( \boldsymbol{w}^T\boldsymbol{w}=\vert \vert \boldsymbol{w}\vert\vert_2^2 \) (the norm) subject to the condition
$$
-y_i(\boldsymbol{w}^T\boldsymbol{x}_i+b) \geq 1 \hspace{0.1cm}\forall i.
+y_i(\boldsymbol{w}^T\boldsymbol{x}_i+b) \geq 1 \hspace{0.1cm}\forall i=1,2,\dots,n.
$$
We have thus defined our margin as the invers of the norm of
@@ -884,8 +884,8 @@ y_ny_1\boldsymbol{x}_n^T\boldsymbol{x}_1 & y_ny_2\boldsymbol{x}_n^T\boldsymbol{x
\end{bmatrix}\boldsymbol{\lambda}-\mathbb{1}\boldsymbol{\lambda},
$$
-
subject to \( \boldsymbol{y}^T\boldsymbol{\lambda}=0 \). Here we defined the vectors \( \boldsymbol{\lambda} =[\lambda_1,\lambda_2,\dots,\lambda_n] \) and
-\( \boldsymbol{y}=[y_1,y_2,\dots,y_n] \).
+
subject to \( \boldsymbol{y}^T\boldsymbol{\lambda}=0 \). Here we defined the vectors \( \boldsymbol{\lambda}^T =[\lambda_1,\lambda_2,\dots,\lambda_n] \) and
+\( \boldsymbol{y}^T=[y_1,y_2,\dots,y_n] \).
@@ -1565,24 +1565,9 @@ sol[’primal objective’]
-Back to the more realistic cases
+Support Vector Machines and Regression
-We are now ready to return to our setup of the optmization problem for a more realistic case. Introducing the slack parameter \( C \) we have
-$$
-\frac{1}{2} \boldsymbol{\lambda}^T\begin{bmatrix} y_1y_1K(\boldsymbol{x}_1,\boldsymbol{x}_1) & y_1y_2K(\boldsymbol{x}_1,\boldsymbol{x}_2) & \dots & \dots & y_1y_nK(\boldsymbol{x}_1,\boldsymbol{x}_n) \\
-y_2y_1K(\boldsymbol{x}_2,\boldsymbol{x}_1) & y_2y_2K(\boldsymbol{x}_2,\boldsymbol{x}_2) & \dots & \dots & y_1y_nK(\boldsymbol{x}_2,\boldsymbol{x}_n) \\
-\dots & \dots & \dots & \dots & \dots \\
-\dots & \dots & \dots & \dots & \dots \\
-y_ny_1K(\boldsymbol{x}_n,\boldsymbol{x}_1) & y_ny_2K(\boldsymbol{x}_n\boldsymbol{x}_2) & \dots & \dots & y_ny_nK(\boldsymbol{x}_n,\boldsymbol{x}_n) \\
-\end{bmatrix}\boldsymbol{\lambda}-\mathbb{I}\boldsymbol{\lambda},
-$$
-
-subject to \( \boldsymbol{y}^T\boldsymbol{\lambda}=0 \). Here we defined the vectors \( \boldsymbol{\lambda} =[\lambda_1,\lambda_2,\dots,\lambda_n] \) and
-\( \boldsymbol{y}=[y_1,y_2,\dots,y_n] \).
-With the slack constants this leads to the additional constraint \( 0\leq \lambda_i \leq C \).
-
-
-code will be added
+Material may be added if of interest. See Bishop chapter 7.1 for a discussion.
Summary of course
diff --git a/doc/pub/week47/ipynb/ipynb-week47-src.tar.gz b/doc/pub/week47/ipynb/ipynb-week47-src.tar.gz
index 664058921..ed7421f30 100644
Binary files a/doc/pub/week47/ipynb/ipynb-week47-src.tar.gz and b/doc/pub/week47/ipynb/ipynb-week47-src.tar.gz differ
diff --git a/doc/pub/week47/ipynb/week47.ipynb b/doc/pub/week47/ipynb/week47.ipynb
index 0969fa0b1..dba1bdab4 100644
--- a/doc/pub/week47/ipynb/week47.ipynb
+++ b/doc/pub/week47/ipynb/week47.ipynb
@@ -2,7 +2,7 @@
"cells": [
{
"cell_type": "markdown",
- "id": "4cc34bad",
+ "id": "d297b5a6",
"metadata": {
"editable": true
},
@@ -14,7 +14,7 @@
},
{
"cell_type": "markdown",
- "id": "8579d326",
+ "id": "c3e26ae4",
"metadata": {
"editable": true
},
@@ -29,7 +29,7 @@
},
{
"cell_type": "markdown",
- "id": "a1de5e67",
+ "id": "3696edde",
"metadata": {
"editable": true
},
@@ -52,7 +52,7 @@
},
{
"cell_type": "markdown",
- "id": "9a1ee036",
+ "id": "3bce99d8",
"metadata": {
"editable": true
},
@@ -86,7 +86,7 @@
},
{
"cell_type": "markdown",
- "id": "706de241",
+ "id": "775d98d8",
"metadata": {
"editable": true
},
@@ -108,7 +108,7 @@
{
"cell_type": "code",
"execution_count": 1,
- "id": "a244fa48",
+ "id": "7d9df056",
"metadata": {
"collapsed": false,
"editable": true
@@ -187,7 +187,7 @@
},
{
"cell_type": "markdown",
- "id": "007f9d3f",
+ "id": "c64f1a9a",
"metadata": {
"editable": true
},
@@ -207,7 +207,7 @@
},
{
"cell_type": "markdown",
- "id": "59d6aee7",
+ "id": "674b12a3",
"metadata": {
"editable": true
},
@@ -219,7 +219,7 @@
},
{
"cell_type": "markdown",
- "id": "73aaa39a",
+ "id": "43f0a20b",
"metadata": {
"editable": true
},
@@ -232,7 +232,7 @@
},
{
"cell_type": "markdown",
- "id": "90533217",
+ "id": "ce09877b",
"metadata": {
"editable": true
},
@@ -244,7 +244,7 @@
},
{
"cell_type": "markdown",
- "id": "c811e175",
+ "id": "d9a14fd7",
"metadata": {
"editable": true
},
@@ -254,7 +254,7 @@
},
{
"cell_type": "markdown",
- "id": "6084370d",
+ "id": "ce869655",
"metadata": {
"editable": true
},
@@ -267,7 +267,7 @@
},
{
"cell_type": "markdown",
- "id": "4127686f",
+ "id": "c703f2d0",
"metadata": {
"editable": true
},
@@ -279,7 +279,7 @@
},
{
"cell_type": "markdown",
- "id": "ee2e985c",
+ "id": "9ce7dd30",
"metadata": {
"editable": true
},
@@ -291,7 +291,7 @@
},
{
"cell_type": "markdown",
- "id": "d6e38fb6",
+ "id": "b455795b",
"metadata": {
"editable": true
},
@@ -303,7 +303,7 @@
},
{
"cell_type": "markdown",
- "id": "d2ca014b",
+ "id": "010b9238",
"metadata": {
"editable": true
},
@@ -313,7 +313,7 @@
},
{
"cell_type": "markdown",
- "id": "8abf8383",
+ "id": "b999e39f",
"metadata": {
"editable": true
},
@@ -325,7 +325,7 @@
},
{
"cell_type": "markdown",
- "id": "48755c0e",
+ "id": "42e6b71e",
"metadata": {
"editable": true
},
@@ -336,7 +336,7 @@
},
{
"cell_type": "markdown",
- "id": "96b14dd6",
+ "id": "f8c7506b",
"metadata": {
"editable": true
},
@@ -348,7 +348,7 @@
},
{
"cell_type": "markdown",
- "id": "bf75f7e1",
+ "id": "902b6664",
"metadata": {
"editable": true
},
@@ -361,7 +361,7 @@
},
{
"cell_type": "markdown",
- "id": "b9a76b13",
+ "id": "54f2aab2",
"metadata": {
"editable": true
},
@@ -373,7 +373,7 @@
},
{
"cell_type": "markdown",
- "id": "c1c92394",
+ "id": "d2b39bc3",
"metadata": {
"editable": true
},
@@ -383,7 +383,7 @@
},
{
"cell_type": "markdown",
- "id": "ece075ba",
+ "id": "d8e438d6",
"metadata": {
"editable": true
},
@@ -412,7 +412,7 @@
},
{
"cell_type": "markdown",
- "id": "a36c4ce3",
+ "id": "284ea218",
"metadata": {
"editable": true
},
@@ -424,7 +424,7 @@
},
{
"cell_type": "markdown",
- "id": "9d9fdae9",
+ "id": "4d4e8fdc",
"metadata": {
"editable": true
},
@@ -436,7 +436,7 @@
},
{
"cell_type": "markdown",
- "id": "cb45022e",
+ "id": "5cfd3056",
"metadata": {
"editable": true
},
@@ -451,7 +451,7 @@
},
{
"cell_type": "markdown",
- "id": "3b678990",
+ "id": "6f935fa6",
"metadata": {
"editable": true
},
@@ -463,7 +463,7 @@
},
{
"cell_type": "markdown",
- "id": "55567107",
+ "id": "206d9255",
"metadata": {
"editable": true
},
@@ -477,7 +477,7 @@
},
{
"cell_type": "markdown",
- "id": "2688d372",
+ "id": "5f7c7666",
"metadata": {
"editable": true
},
@@ -489,7 +489,7 @@
},
{
"cell_type": "markdown",
- "id": "1c53912a",
+ "id": "2d213860",
"metadata": {
"editable": true
},
@@ -499,7 +499,7 @@
},
{
"cell_type": "markdown",
- "id": "3f4bf847",
+ "id": "276c7e93",
"metadata": {
"editable": true
},
@@ -511,7 +511,7 @@
},
{
"cell_type": "markdown",
- "id": "3b3876db",
+ "id": "264b761f",
"metadata": {
"editable": true
},
@@ -521,7 +521,7 @@
},
{
"cell_type": "markdown",
- "id": "1e239842",
+ "id": "10dd0353",
"metadata": {
"editable": true
},
@@ -533,7 +533,7 @@
},
{
"cell_type": "markdown",
- "id": "011ed1d2",
+ "id": "69606ee8",
"metadata": {
"editable": true
},
@@ -545,7 +545,7 @@
},
{
"cell_type": "markdown",
- "id": "4618893d",
+ "id": "3c6f4034",
"metadata": {
"editable": true
},
@@ -557,7 +557,7 @@
},
{
"cell_type": "markdown",
- "id": "9d515930",
+ "id": "45ebb87b",
"metadata": {
"editable": true
},
@@ -567,7 +567,7 @@
},
{
"cell_type": "markdown",
- "id": "d963d9ae",
+ "id": "dcddfbd3",
"metadata": {
"editable": true
},
@@ -579,7 +579,7 @@
},
{
"cell_type": "markdown",
- "id": "d92faaa6",
+ "id": "52da5b57",
"metadata": {
"editable": true
},
@@ -589,7 +589,7 @@
},
{
"cell_type": "markdown",
- "id": "8d521484",
+ "id": "4281f447",
"metadata": {
"editable": true
},
@@ -612,7 +612,7 @@
},
{
"cell_type": "markdown",
- "id": "5d5f78cb",
+ "id": "dd89f6d6",
"metadata": {
"editable": true
},
@@ -628,19 +628,19 @@
},
{
"cell_type": "markdown",
- "id": "2b51b0c3",
+ "id": "894e43d0",
"metadata": {
"editable": true
},
"source": [
"$$\n",
- "y_i(\\boldsymbol{w}^T\\boldsymbol{x}_i+b) \\geq M \\hspace{0.1cm}\\forall i=1,2,\\dots, p.\n",
+ "y_i(\\boldsymbol{w}^T\\boldsymbol{x}_i+b) \\geq M \\hspace{0.1cm}\\forall i=1,2,\\dots, n.\n",
"$$"
]
},
{
"cell_type": "markdown",
- "id": "dbdab28c",
+ "id": "ab462cfa",
"metadata": {
"editable": true
},
@@ -652,7 +652,7 @@
},
{
"cell_type": "markdown",
- "id": "af1c3ae8",
+ "id": "f66b13c2",
"metadata": {
"editable": true
},
@@ -664,7 +664,7 @@
},
{
"cell_type": "markdown",
- "id": "751ee766",
+ "id": "b09478e1",
"metadata": {
"editable": true
},
@@ -674,42 +674,42 @@
},
{
"cell_type": "markdown",
- "id": "15798c2e",
+ "id": "e63e1ed8",
"metadata": {
"editable": true
},
"source": [
"$$\n",
- "y_i(\\boldsymbol{w}^T\\boldsymbol{x}_i+b) \\geq M\\vert \\vert \\boldsymbol{w}\\vert\\vert \\hspace{0.1cm}\\forall i.\n",
+ "y_i(\\boldsymbol{w}^T\\boldsymbol{x}_i+b) \\geq M\\vert \\vert \\boldsymbol{w}\\vert\\vert \\hspace{0.1cm}\\forall i=1,2,\\dots,n.\n",
"$$"
]
},
{
"cell_type": "markdown",
- "id": "ab9bbfa4",
+ "id": "09d0ab60",
"metadata": {
"editable": true
},
"source": [
"If we scale the equation so that $\\vert \\vert \\boldsymbol{w}\\vert\\vert = 1/M$, we have to find the minimum of \n",
- "$\\boldsymbol{w}^T\\boldsymbol{w}=\\vert \\vert \\boldsymbol{w}\\vert\\vert$ (the norm) subject to the condition"
+ "$\\boldsymbol{w}^T\\boldsymbol{w}=\\vert \\vert \\boldsymbol{w}\\vert\\vert_2^2$ (the norm) subject to the condition"
]
},
{
"cell_type": "markdown",
- "id": "cc5d8b13",
+ "id": "815f9576",
"metadata": {
"editable": true
},
"source": [
"$$\n",
- "y_i(\\boldsymbol{w}^T\\boldsymbol{x}_i+b) \\geq 1 \\hspace{0.1cm}\\forall i.\n",
+ "y_i(\\boldsymbol{w}^T\\boldsymbol{x}_i+b) \\geq 1 \\hspace{0.1cm}\\forall i=1,2,\\dots,n.\n",
"$$"
]
},
{
"cell_type": "markdown",
- "id": "7f69fb14",
+ "id": "81dff301",
"metadata": {
"editable": true
},
@@ -722,7 +722,7 @@
},
{
"cell_type": "markdown",
- "id": "dd1321d9",
+ "id": "6a5bcc89",
"metadata": {
"editable": true
},
@@ -735,7 +735,7 @@
},
{
"cell_type": "markdown",
- "id": "689ed44a",
+ "id": "2545b335",
"metadata": {
"editable": true
},
@@ -747,7 +747,7 @@
},
{
"cell_type": "markdown",
- "id": "35df1f77",
+ "id": "7d5bf787",
"metadata": {
"editable": true
},
@@ -757,7 +757,7 @@
},
{
"cell_type": "markdown",
- "id": "4a7bbc5f",
+ "id": "761895bf",
"metadata": {
"editable": true
},
@@ -769,7 +769,7 @@
},
{
"cell_type": "markdown",
- "id": "85d7a8b6",
+ "id": "15ff9911",
"metadata": {
"editable": true
},
@@ -779,7 +779,7 @@
},
{
"cell_type": "markdown",
- "id": "4fe49eac",
+ "id": "e5666256",
"metadata": {
"editable": true
},
@@ -791,7 +791,7 @@
},
{
"cell_type": "markdown",
- "id": "67c642cb",
+ "id": "9a3d325a",
"metadata": {
"editable": true
},
@@ -808,7 +808,7 @@
},
{
"cell_type": "markdown",
- "id": "48d25961",
+ "id": "2cda88e0",
"metadata": {
"editable": true
},
@@ -820,7 +820,7 @@
},
{
"cell_type": "markdown",
- "id": "7bd7b578",
+ "id": "70285535",
"metadata": {
"editable": true
},
@@ -830,7 +830,7 @@
},
{
"cell_type": "markdown",
- "id": "afc29bad",
+ "id": "3d155c0c",
"metadata": {
"editable": true
},
@@ -842,7 +842,7 @@
},
{
"cell_type": "markdown",
- "id": "0543d207",
+ "id": "65cfa308",
"metadata": {
"editable": true
},
@@ -852,7 +852,7 @@
},
{
"cell_type": "markdown",
- "id": "8ac4e5fa",
+ "id": "204f80ec",
"metadata": {
"editable": true
},
@@ -864,7 +864,7 @@
},
{
"cell_type": "markdown",
- "id": "7dac42c7",
+ "id": "89675f53",
"metadata": {
"editable": true
},
@@ -877,7 +877,7 @@
},
{
"cell_type": "markdown",
- "id": "abb322a1",
+ "id": "6ebe98f7",
"metadata": {
"editable": true
},
@@ -889,7 +889,7 @@
},
{
"cell_type": "markdown",
- "id": "7f0f11ba",
+ "id": "cb58d54b",
"metadata": {
"editable": true
},
@@ -901,7 +901,7 @@
},
{
"cell_type": "markdown",
- "id": "0ac75fc9",
+ "id": "ac15bc71",
"metadata": {
"editable": true
},
@@ -911,7 +911,7 @@
},
{
"cell_type": "markdown",
- "id": "92428367",
+ "id": "c0f9ca25",
"metadata": {
"editable": true
},
@@ -925,7 +925,7 @@
},
{
"cell_type": "markdown",
- "id": "78ae892d",
+ "id": "5734050e",
"metadata": {
"editable": true
},
@@ -935,7 +935,7 @@
},
{
"cell_type": "markdown",
- "id": "bd74c1ff",
+ "id": "3deaaab1",
"metadata": {
"editable": true
},
@@ -947,7 +947,7 @@
},
{
"cell_type": "markdown",
- "id": "ad60aeb2",
+ "id": "c1905f47",
"metadata": {
"editable": true
},
@@ -957,7 +957,7 @@
},
{
"cell_type": "markdown",
- "id": "38368495",
+ "id": "4e0423a8",
"metadata": {
"editable": true
},
@@ -969,7 +969,7 @@
},
{
"cell_type": "markdown",
- "id": "c6c520b7",
+ "id": "e0dec665",
"metadata": {
"editable": true
},
@@ -979,7 +979,7 @@
},
{
"cell_type": "markdown",
- "id": "9749dfca",
+ "id": "e2d57eef",
"metadata": {
"editable": true
},
@@ -991,7 +991,7 @@
},
{
"cell_type": "markdown",
- "id": "7985f058",
+ "id": "77bc96c6",
"metadata": {
"editable": true
},
@@ -1005,7 +1005,7 @@
},
{
"cell_type": "markdown",
- "id": "55fa560e",
+ "id": "38775bb9",
"metadata": {
"editable": true
},
@@ -1017,7 +1017,7 @@
},
{
"cell_type": "markdown",
- "id": "35b9dc6e",
+ "id": "2862aca0",
"metadata": {
"editable": true
},
@@ -1029,7 +1029,7 @@
},
{
"cell_type": "markdown",
- "id": "96ca3b5d",
+ "id": "b356a9e2",
"metadata": {
"editable": true
},
@@ -1041,7 +1041,7 @@
},
{
"cell_type": "markdown",
- "id": "dcf1e5fe",
+ "id": "673fbcec",
"metadata": {
"editable": true
},
@@ -1053,7 +1053,7 @@
},
{
"cell_type": "markdown",
- "id": "197180b0",
+ "id": "fce13dac",
"metadata": {
"editable": true
},
@@ -1065,7 +1065,7 @@
},
{
"cell_type": "markdown",
- "id": "e5601abd",
+ "id": "5cef7ece",
"metadata": {
"editable": true
},
@@ -1075,7 +1075,7 @@
},
{
"cell_type": "markdown",
- "id": "0ded669a",
+ "id": "116189a3",
"metadata": {
"editable": true
},
@@ -1087,7 +1087,7 @@
},
{
"cell_type": "markdown",
- "id": "df3ef2de",
+ "id": "192e3d83",
"metadata": {
"editable": true
},
@@ -1097,7 +1097,7 @@
},
{
"cell_type": "markdown",
- "id": "0a18588a",
+ "id": "0367b41a",
"metadata": {
"editable": true
},
@@ -1109,7 +1109,7 @@
},
{
"cell_type": "markdown",
- "id": "fd303bd1",
+ "id": "73055eee",
"metadata": {
"editable": true
},
@@ -1120,7 +1120,7 @@
},
{
"cell_type": "markdown",
- "id": "a55ffcfc",
+ "id": "1e9a8e7d",
"metadata": {
"editable": true
},
@@ -1132,7 +1132,7 @@
},
{
"cell_type": "markdown",
- "id": "0214d5d7",
+ "id": "fec4bd3f",
"metadata": {
"editable": true
},
@@ -1146,7 +1146,7 @@
},
{
"cell_type": "markdown",
- "id": "b74e82e3",
+ "id": "83d140dc",
"metadata": {
"editable": true
},
@@ -1158,7 +1158,7 @@
},
{
"cell_type": "markdown",
- "id": "aabee1ef",
+ "id": "1e8bd1a9",
"metadata": {
"editable": true
},
@@ -1170,7 +1170,7 @@
},
{
"cell_type": "markdown",
- "id": "c34316ad",
+ "id": "f9020497",
"metadata": {
"editable": true
},
@@ -1180,7 +1180,7 @@
},
{
"cell_type": "markdown",
- "id": "093dce59",
+ "id": "a3777e9e",
"metadata": {
"editable": true
},
@@ -1197,18 +1197,18 @@
},
{
"cell_type": "markdown",
- "id": "aa1a49f2",
+ "id": "f4b5422e",
"metadata": {
"editable": true
},
"source": [
- "subject to $\\boldsymbol{y}^T\\boldsymbol{\\lambda}=0$. Here we defined the vectors $\\boldsymbol{\\lambda} =[\\lambda_1,\\lambda_2,\\dots,\\lambda_n]$ and \n",
- "$\\boldsymbol{y}=[y_1,y_2,\\dots,y_n]$."
+ "subject to $\\boldsymbol{y}^T\\boldsymbol{\\lambda}=0$. Here we defined the vectors $\\boldsymbol{\\lambda}^T =[\\lambda_1,\\lambda_2,\\dots,\\lambda_n]$ and \n",
+ "$\\boldsymbol{y}^T=[y_1,y_2,\\dots,y_n]$."
]
},
{
"cell_type": "markdown",
- "id": "74f7ad97",
+ "id": "5434dedf",
"metadata": {
"editable": true
},
@@ -1221,7 +1221,7 @@
},
{
"cell_type": "markdown",
- "id": "29ab6c37",
+ "id": "59d9227f",
"metadata": {
"editable": true
},
@@ -1233,7 +1233,7 @@
},
{
"cell_type": "markdown",
- "id": "422b1e2c",
+ "id": "e3205aa2",
"metadata": {
"editable": true
},
@@ -1243,7 +1243,7 @@
},
{
"cell_type": "markdown",
- "id": "bb88d9b6",
+ "id": "b9773b95",
"metadata": {
"editable": true
},
@@ -1255,7 +1255,7 @@
},
{
"cell_type": "markdown",
- "id": "b8db4fcb",
+ "id": "472ad856",
"metadata": {
"editable": true
},
@@ -1265,7 +1265,7 @@
},
{
"cell_type": "markdown",
- "id": "eacb5e84",
+ "id": "fbce138f",
"metadata": {
"editable": true
},
@@ -1277,7 +1277,7 @@
},
{
"cell_type": "markdown",
- "id": "6293d8f3",
+ "id": "cee42ab7",
"metadata": {
"editable": true
},
@@ -1287,7 +1287,7 @@
},
{
"cell_type": "markdown",
- "id": "6d20462b",
+ "id": "5ae1628c",
"metadata": {
"editable": true
},
@@ -1299,7 +1299,7 @@
},
{
"cell_type": "markdown",
- "id": "26e9a13f",
+ "id": "dbc046c4",
"metadata": {
"editable": true
},
@@ -1309,7 +1309,7 @@
},
{
"cell_type": "markdown",
- "id": "90aef52e",
+ "id": "62b7934c",
"metadata": {
"editable": true
},
@@ -1321,7 +1321,7 @@
},
{
"cell_type": "markdown",
- "id": "c53e0879",
+ "id": "cd3f6f0a",
"metadata": {
"editable": true
},
@@ -1331,7 +1331,7 @@
},
{
"cell_type": "markdown",
- "id": "1c962cdd",
+ "id": "372433e3",
"metadata": {
"editable": true
},
@@ -1351,7 +1351,7 @@
},
{
"cell_type": "markdown",
- "id": "37fb436c",
+ "id": "3b0cb06e",
"metadata": {
"editable": true
},
@@ -1363,7 +1363,7 @@
},
{
"cell_type": "markdown",
- "id": "d54252a2",
+ "id": "fd1d9050",
"metadata": {
"editable": true
},
@@ -1373,7 +1373,7 @@
},
{
"cell_type": "markdown",
- "id": "84beaae2",
+ "id": "0b22eabb",
"metadata": {
"editable": true
},
@@ -1385,7 +1385,7 @@
},
{
"cell_type": "markdown",
- "id": "fa603cae",
+ "id": "f38b37d2",
"metadata": {
"editable": true
},
@@ -1401,7 +1401,7 @@
},
{
"cell_type": "markdown",
- "id": "66a6c715",
+ "id": "4560c89b",
"metadata": {
"editable": true
},
@@ -1413,7 +1413,7 @@
},
{
"cell_type": "markdown",
- "id": "6d3f3f1e",
+ "id": "8255a80f",
"metadata": {
"editable": true
},
@@ -1425,7 +1425,7 @@
},
{
"cell_type": "markdown",
- "id": "82c185cb",
+ "id": "8dbf6a88",
"metadata": {
"editable": true
},
@@ -1435,7 +1435,7 @@
},
{
"cell_type": "markdown",
- "id": "bce2b968",
+ "id": "dc353570",
"metadata": {
"editable": true
},
@@ -1447,7 +1447,7 @@
},
{
"cell_type": "markdown",
- "id": "a970e099",
+ "id": "6565e7ac",
"metadata": {
"editable": true
},
@@ -1459,7 +1459,7 @@
},
{
"cell_type": "markdown",
- "id": "237be57f",
+ "id": "05e47e0a",
"metadata": {
"editable": true
},
@@ -1471,7 +1471,7 @@
},
{
"cell_type": "markdown",
- "id": "a18e6161",
+ "id": "d55883da",
"metadata": {
"editable": true
},
@@ -1481,7 +1481,7 @@
},
{
"cell_type": "markdown",
- "id": "bfc79b5d",
+ "id": "39f7b542",
"metadata": {
"editable": true
},
@@ -1493,7 +1493,7 @@
},
{
"cell_type": "markdown",
- "id": "ff9be50a",
+ "id": "141d2a34",
"metadata": {
"editable": true
},
@@ -1503,7 +1503,7 @@
},
{
"cell_type": "markdown",
- "id": "c36b4d3b",
+ "id": "787755c1",
"metadata": {
"editable": true
},
@@ -1515,7 +1515,7 @@
},
{
"cell_type": "markdown",
- "id": "1f47d2fe",
+ "id": "ddd7223d",
"metadata": {
"editable": true
},
@@ -1525,7 +1525,7 @@
},
{
"cell_type": "markdown",
- "id": "de02dfeb",
+ "id": "de1e5717",
"metadata": {
"editable": true
},
@@ -1537,7 +1537,7 @@
},
{
"cell_type": "markdown",
- "id": "f2f7f11f",
+ "id": "0d5ad78d",
"metadata": {
"editable": true
},
@@ -1548,7 +1548,7 @@
},
{
"cell_type": "markdown",
- "id": "51c9b081",
+ "id": "c112457b",
"metadata": {
"editable": true
},
@@ -1560,7 +1560,7 @@
},
{
"cell_type": "markdown",
- "id": "21385c25",
+ "id": "363c1a47",
"metadata": {
"editable": true
},
@@ -1572,7 +1572,7 @@
},
{
"cell_type": "markdown",
- "id": "128fe7b5",
+ "id": "06a0a3ed",
"metadata": {
"editable": true
},
@@ -1582,7 +1582,7 @@
},
{
"cell_type": "markdown",
- "id": "0e6bd855",
+ "id": "98083d0f",
"metadata": {
"editable": true
},
@@ -1594,7 +1594,7 @@
},
{
"cell_type": "markdown",
- "id": "42648dca",
+ "id": "6e494443",
"metadata": {
"editable": true
},
@@ -1620,7 +1620,7 @@
{
"cell_type": "code",
"execution_count": 2,
- "id": "d489c6c3",
+ "id": "42524afe",
"metadata": {
"collapsed": false,
"editable": true
@@ -1677,7 +1677,7 @@
},
{
"cell_type": "markdown",
- "id": "5ca6d314",
+ "id": "8a04c098",
"metadata": {
"editable": true
},
@@ -1689,7 +1689,7 @@
},
{
"cell_type": "markdown",
- "id": "617b0921",
+ "id": "8a837902",
"metadata": {
"editable": true
},
@@ -1701,7 +1701,7 @@
},
{
"cell_type": "markdown",
- "id": "1c83283f",
+ "id": "5ff37c0d",
"metadata": {
"editable": true
},
@@ -1711,7 +1711,7 @@
},
{
"cell_type": "markdown",
- "id": "947e5e41",
+ "id": "59243c73",
"metadata": {
"editable": true
},
@@ -1723,7 +1723,7 @@
},
{
"cell_type": "markdown",
- "id": "d6a9f842",
+ "id": "aa20b129",
"metadata": {
"editable": true
},
@@ -1733,7 +1733,7 @@
},
{
"cell_type": "markdown",
- "id": "872e689a",
+ "id": "3b67b193",
"metadata": {
"editable": true
},
@@ -1745,7 +1745,7 @@
},
{
"cell_type": "markdown",
- "id": "cebc00e8",
+ "id": "28823267",
"metadata": {
"editable": true
},
@@ -1756,7 +1756,7 @@
},
{
"cell_type": "markdown",
- "id": "7eec7f44",
+ "id": "c90b67ca",
"metadata": {
"editable": true
},
@@ -1768,7 +1768,7 @@
},
{
"cell_type": "markdown",
- "id": "672c83dc",
+ "id": "e4fcdba6",
"metadata": {
"editable": true
},
@@ -1778,7 +1778,7 @@
},
{
"cell_type": "markdown",
- "id": "91c46fce",
+ "id": "715664f9",
"metadata": {
"editable": true
},
@@ -1790,7 +1790,7 @@
},
{
"cell_type": "markdown",
- "id": "b55056e0",
+ "id": "7b853f11",
"metadata": {
"editable": true
},
@@ -1808,7 +1808,7 @@
},
{
"cell_type": "markdown",
- "id": "5ddac4ae",
+ "id": "803a45ce",
"metadata": {
"editable": true
},
@@ -1819,7 +1819,7 @@
},
{
"cell_type": "markdown",
- "id": "22fa1dc5",
+ "id": "61310ff9",
"metadata": {
"editable": true
},
@@ -1831,7 +1831,7 @@
},
{
"cell_type": "markdown",
- "id": "8a644118",
+ "id": "66491438",
"metadata": {
"editable": true
},
@@ -1841,7 +1841,7 @@
},
{
"cell_type": "markdown",
- "id": "0dd071ac",
+ "id": "e8d2d136",
"metadata": {
"editable": true
},
@@ -1858,7 +1858,7 @@
},
{
"cell_type": "markdown",
- "id": "91e247a6",
+ "id": "f7952451",
"metadata": {
"editable": true
},
@@ -1872,7 +1872,7 @@
},
{
"cell_type": "markdown",
- "id": "3d1a70c1",
+ "id": "b19b5113",
"metadata": {
"editable": true
},
@@ -1887,7 +1887,7 @@
},
{
"cell_type": "markdown",
- "id": "80dba55b",
+ "id": "4caf5a80",
"metadata": {
"editable": true
},
@@ -1899,7 +1899,7 @@
},
{
"cell_type": "markdown",
- "id": "54a00058",
+ "id": "633e3356",
"metadata": {
"editable": true
},
@@ -1927,7 +1927,7 @@
},
{
"cell_type": "markdown",
- "id": "19ee429a",
+ "id": "76ac96ea",
"metadata": {
"editable": true
},
@@ -1939,7 +1939,7 @@
},
{
"cell_type": "markdown",
- "id": "672c4ae8",
+ "id": "74510d78",
"metadata": {
"editable": true
},
@@ -1954,7 +1954,7 @@
},
{
"cell_type": "markdown",
- "id": "65e42723",
+ "id": "faac8dc8",
"metadata": {
"editable": true
},
@@ -1965,7 +1965,7 @@
{
"cell_type": "code",
"execution_count": 3,
- "id": "ac13df75",
+ "id": "91ea69e9",
"metadata": {
"collapsed": false,
"editable": true
@@ -2164,7 +2164,7 @@
},
{
"cell_type": "markdown",
- "id": "a9dfad7f",
+ "id": "d0b7f9cf",
"metadata": {
"editable": true
},
@@ -2176,7 +2176,7 @@
},
{
"cell_type": "markdown",
- "id": "d5f21bf1",
+ "id": "9463bbe9",
"metadata": {
"editable": true
},
@@ -2191,7 +2191,7 @@
},
{
"cell_type": "markdown",
- "id": "dec92019",
+ "id": "7b64fd8d",
"metadata": {
"editable": true
},
@@ -2208,7 +2208,7 @@
},
{
"cell_type": "markdown",
- "id": "9fae1ff2",
+ "id": "8b168bf3",
"metadata": {
"editable": true
},
@@ -2229,7 +2229,7 @@
{
"cell_type": "code",
"execution_count": 4,
- "id": "407db5be",
+ "id": "fe97004b",
"metadata": {
"collapsed": false,
"editable": true
@@ -2242,7 +2242,7 @@
},
{
"cell_type": "markdown",
- "id": "0559db68",
+ "id": "9ebe52b7",
"metadata": {
"editable": true
},
@@ -2252,7 +2252,7 @@
},
{
"cell_type": "markdown",
- "id": "31fb5879",
+ "id": "ea4caa80",
"metadata": {
"editable": true
},
@@ -2264,7 +2264,7 @@
},
{
"cell_type": "markdown",
- "id": "70875994",
+ "id": "447ad01e",
"metadata": {
"editable": true
},
@@ -2279,7 +2279,7 @@
},
{
"cell_type": "markdown",
- "id": "6e356c92",
+ "id": "9917fc9c",
"metadata": {
"editable": true
},
@@ -2289,7 +2289,7 @@
},
{
"cell_type": "markdown",
- "id": "e3665fc9",
+ "id": "4ec0b02c",
"metadata": {
"editable": true
},
@@ -2308,7 +2308,7 @@
},
{
"cell_type": "markdown",
- "id": "ee5d3e53",
+ "id": "23f12ebb",
"metadata": {
"editable": true
},
@@ -2318,7 +2318,7 @@
},
{
"cell_type": "markdown",
- "id": "d4fc6762",
+ "id": "e53e01bb",
"metadata": {
"editable": true
},
@@ -2330,7 +2330,7 @@
},
{
"cell_type": "markdown",
- "id": "5aed4a56",
+ "id": "953de6d5",
"metadata": {
"editable": true
},
@@ -2340,7 +2340,7 @@
},
{
"cell_type": "markdown",
- "id": "d3097c98",
+ "id": "1addd2bf",
"metadata": {
"editable": true
},
@@ -2352,7 +2352,7 @@
},
{
"cell_type": "markdown",
- "id": "5e9ac7a0",
+ "id": "818d1345",
"metadata": {
"editable": true
},
@@ -2362,7 +2362,7 @@
},
{
"cell_type": "markdown",
- "id": "0b7eb4cd",
+ "id": "7da978de",
"metadata": {
"editable": true
},
@@ -2374,7 +2374,7 @@
},
{
"cell_type": "markdown",
- "id": "b0c0def1",
+ "id": "d4d1d665",
"metadata": {
"editable": true
},
@@ -2385,7 +2385,7 @@
},
{
"cell_type": "markdown",
- "id": "b59f20fc",
+ "id": "ebf7b0d8",
"metadata": {
"editable": true
},
@@ -2397,7 +2397,7 @@
},
{
"cell_type": "markdown",
- "id": "d5eed580",
+ "id": "aa66eb96",
"metadata": {
"editable": true
},
@@ -2409,7 +2409,7 @@
{
"cell_type": "code",
"execution_count": 5,
- "id": "af36849d",
+ "id": "bcb69ce9",
"metadata": {
"collapsed": false,
"editable": true
@@ -2433,50 +2433,19 @@
},
{
"cell_type": "markdown",
- "id": "ae234a3d",
+ "id": "60a3d131",
"metadata": {
"editable": true
},
"source": [
- "## Back to the more realistic cases\n",
+ "## Support Vector Machines and Regression\n",
"\n",
- "We are now ready to return to our setup of the optmization problem for a more realistic case. Introducing the **slack** parameter $C$ we have"
+ "Material may be added if of interest. See Bishop chapter 7.1 for a discussion."
]
},
{
"cell_type": "markdown",
- "id": "41639135",
- "metadata": {
- "editable": true
- },
- "source": [
- "$$\n",
- "\\frac{1}{2} \\boldsymbol{\\lambda}^T\\begin{bmatrix} y_1y_1K(\\boldsymbol{x}_1,\\boldsymbol{x}_1) & y_1y_2K(\\boldsymbol{x}_1,\\boldsymbol{x}_2) & \\dots & \\dots & y_1y_nK(\\boldsymbol{x}_1,\\boldsymbol{x}_n) \\\\\n",
- "y_2y_1K(\\boldsymbol{x}_2,\\boldsymbol{x}_1) & y_2y_2K(\\boldsymbol{x}_2,\\boldsymbol{x}_2) & \\dots & \\dots & y_1y_nK(\\boldsymbol{x}_2,\\boldsymbol{x}_n) \\\\\n",
- "\\dots & \\dots & \\dots & \\dots & \\dots \\\\\n",
- "\\dots & \\dots & \\dots & \\dots & \\dots \\\\\n",
- "y_ny_1K(\\boldsymbol{x}_n,\\boldsymbol{x}_1) & y_ny_2K(\\boldsymbol{x}_n\\boldsymbol{x}_2) & \\dots & \\dots & y_ny_nK(\\boldsymbol{x}_n,\\boldsymbol{x}_n) \\\\\n",
- "\\end{bmatrix}\\boldsymbol{\\lambda}-\\mathbb{I}\\boldsymbol{\\lambda},\n",
- "$$"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "32fd6400",
- "metadata": {
- "editable": true
- },
- "source": [
- "subject to $\\boldsymbol{y}^T\\boldsymbol{\\lambda}=0$. Here we defined the vectors $\\boldsymbol{\\lambda} =[\\lambda_1,\\lambda_2,\\dots,\\lambda_n]$ and \n",
- "$\\boldsymbol{y}=[y_1,y_2,\\dots,y_n]$. \n",
- "With the slack constants this leads to the additional constraint $0\\leq \\lambda_i \\leq C$.\n",
- "\n",
- "**code will be added**"
- ]
- },
- {
- "cell_type": "markdown",
- "id": "74d5d1dd",
+ "id": "09a979c4",
"metadata": {
"editable": true
},
@@ -2486,7 +2455,7 @@
},
{
"cell_type": "markdown",
- "id": "25c2688b",
+ "id": "e93c0928",
"metadata": {
"editable": true
},
@@ -2501,7 +2470,7 @@
},
{
"cell_type": "markdown",
- "id": "9477fb34",
+ "id": "906a462a",
"metadata": {
"editable": true
},
@@ -2517,7 +2486,7 @@
},
{
"cell_type": "markdown",
- "id": "20d9a817",
+ "id": "ce4e7093",
"metadata": {
"editable": true
},
@@ -2542,7 +2511,7 @@
},
{
"cell_type": "markdown",
- "id": "47038b09",
+ "id": "030684fc",
"metadata": {
"editable": true
},
@@ -2571,7 +2540,7 @@
},
{
"cell_type": "markdown",
- "id": "47679fe4",
+ "id": "8e1d2370",
"metadata": {
"editable": true
},
@@ -2587,7 +2556,7 @@
},
{
"cell_type": "markdown",
- "id": "006416fc",
+ "id": "5ceec6b5",
"metadata": {
"editable": true
},
@@ -2612,7 +2581,7 @@
},
{
"cell_type": "markdown",
- "id": "9b7e17cf",
+ "id": "5336ae51",
"metadata": {
"editable": true
},
@@ -2659,7 +2628,7 @@
},
{
"cell_type": "markdown",
- "id": "019587a2",
+ "id": "69d7807b",
"metadata": {
"editable": true
},
@@ -2695,7 +2664,7 @@
},
{
"cell_type": "markdown",
- "id": "3746e15c",
+ "id": "a5788fcf",
"metadata": {
"editable": true
},
@@ -2718,7 +2687,7 @@
},
{
"cell_type": "markdown",
- "id": "bdf84776",
+ "id": "fda2577a",
"metadata": {
"editable": true
},
@@ -2741,7 +2710,7 @@
},
{
"cell_type": "markdown",
- "id": "38966515",
+ "id": "aa0f65b6",
"metadata": {
"editable": true
},
@@ -2761,7 +2730,7 @@
},
{
"cell_type": "markdown",
- "id": "d6397d83",
+ "id": "a387c41a",
"metadata": {
"editable": true
},
@@ -2777,7 +2746,7 @@
},
{
"cell_type": "markdown",
- "id": "9b9e5831",
+ "id": "b4412a32",
"metadata": {
"editable": true
},
@@ -2805,7 +2774,7 @@
},
{
"cell_type": "markdown",
- "id": "9e1c05ba",
+ "id": "09fe89d1",
"metadata": {
"editable": true
},
@@ -2827,7 +2796,7 @@
},
{
"cell_type": "markdown",
- "id": "d5a44aa7",
+ "id": "a1625399",
"metadata": {
"editable": true
},
@@ -2852,7 +2821,7 @@
},
{
"cell_type": "markdown",
- "id": "b50a77ca",
+ "id": "a60af950",
"metadata": {
"editable": true
},
@@ -2870,7 +2839,7 @@
},
{
"cell_type": "markdown",
- "id": "ac817b17",
+ "id": "717ff832",
"metadata": {
"editable": true
},
@@ -2898,7 +2867,7 @@
},
{
"cell_type": "markdown",
- "id": "ff3859f2",
+ "id": "69120211",
"metadata": {
"editable": true
},
@@ -2912,7 +2881,7 @@
},
{
"cell_type": "markdown",
- "id": "54becfd9",
+ "id": "cf743a91",
"metadata": {
"editable": true
},
@@ -2938,7 +2907,7 @@
},
{
"cell_type": "markdown",
- "id": "d716cbf5",
+ "id": "2b1c821d",
"metadata": {
"editable": true
},
@@ -2967,7 +2936,7 @@
},
{
"cell_type": "markdown",
- "id": "c3fdf6bc",
+ "id": "d07a08c1",
"metadata": {
"editable": true
},
@@ -2984,7 +2953,7 @@
},
{
"cell_type": "markdown",
- "id": "fcfe2c42",
+ "id": "21760d88",
"metadata": {
"editable": true
},
@@ -3006,7 +2975,7 @@
},
{
"cell_type": "markdown",
- "id": "08107735",
+ "id": "a215b485",
"metadata": {
"editable": true
},
@@ -3024,7 +2993,7 @@
},
{
"cell_type": "markdown",
- "id": "cb0979f6",
+ "id": "e0e5b2f2",
"metadata": {
"editable": true
},
@@ -3047,7 +3016,7 @@
},
{
"cell_type": "markdown",
- "id": "fde0921c",
+ "id": "883ff967",
"metadata": {
"editable": true
},
@@ -3066,7 +3035,7 @@
},
{
"cell_type": "markdown",
- "id": "321a4de5",
+ "id": "a8c66672",
"metadata": {
"editable": true
},
@@ -3082,7 +3051,7 @@
},
{
"cell_type": "markdown",
- "id": "e5213ac1",
+ "id": "38867651",
"metadata": {
"editable": true
},
@@ -3097,7 +3066,7 @@
},
{
"cell_type": "markdown",
- "id": "83919fb4",
+ "id": "31d4a91e",
"metadata": {
"editable": true
},
@@ -3121,7 +3090,7 @@
},
{
"cell_type": "markdown",
- "id": "7f6ebb56",
+ "id": "d8eb36e7",
"metadata": {
"editable": true
},
@@ -3133,7 +3102,7 @@
},
{
"cell_type": "markdown",
- "id": "bc772bc0",
+ "id": "03f66dfa",
"metadata": {
"editable": true
},
@@ -3151,7 +3120,7 @@
},
{
"cell_type": "markdown",
- "id": "941c527b",
+ "id": "15a58ad9",
"metadata": {
"editable": true
},
@@ -3161,7 +3130,7 @@
},
{
"cell_type": "markdown",
- "id": "1ee5cfa8",
+ "id": "8fee60a6",
"metadata": {
"editable": true
},
@@ -3179,7 +3148,7 @@
},
{
"cell_type": "markdown",
- "id": "d8d11e38",
+ "id": "bd6b9ab1",
"metadata": {
"editable": true
},
@@ -3189,7 +3158,7 @@
},
{
"cell_type": "markdown",
- "id": "1f4310f2",
+ "id": "2f1d5ba5",
"metadata": {
"editable": true
},
@@ -3208,7 +3177,7 @@
},
{
"cell_type": "markdown",
- "id": "81055485",
+ "id": "cda96c2d",
"metadata": {
"editable": true
},
@@ -3220,7 +3189,7 @@
},
{
"cell_type": "markdown",
- "id": "47cf1bb7",
+ "id": "17566561",
"metadata": {
"editable": true
},
@@ -3234,7 +3203,7 @@
},
{
"cell_type": "markdown",
- "id": "fafde12c",
+ "id": "161d5743",
"metadata": {
"editable": true
},
@@ -3249,7 +3218,7 @@
},
{
"cell_type": "markdown",
- "id": "ebc67a04",
+ "id": "707e8fcb",
"metadata": {
"editable": true
},
@@ -3267,7 +3236,7 @@
},
{
"cell_type": "markdown",
- "id": "a85c0132",
+ "id": "d1fff888",
"metadata": {
"editable": true
},
@@ -3281,7 +3250,7 @@
},
{
"cell_type": "markdown",
- "id": "122db74b",
+ "id": "ae35a3c3",
"metadata": {
"editable": true
},
@@ -3299,7 +3268,7 @@
},
{
"cell_type": "markdown",
- "id": "17fbd221",
+ "id": "de8a68fe",
"metadata": {
"editable": true
},
@@ -3323,7 +3292,7 @@
},
{
"cell_type": "markdown",
- "id": "1f27d3da",
+ "id": "6fce707b",
"metadata": {
"editable": true
},
@@ -3361,7 +3330,7 @@
},
{
"cell_type": "markdown",
- "id": "f58c00db",
+ "id": "1b2d107e",
"metadata": {
"editable": true
},
@@ -3384,7 +3353,7 @@
},
{
"cell_type": "markdown",
- "id": "21ac0978",
+ "id": "649355d5",
"metadata": {
"editable": true
},
@@ -3420,7 +3389,7 @@
},
{
"cell_type": "markdown",
- "id": "1cb06f52",
+ "id": "1bd0b6d2",
"metadata": {
"editable": true
},
@@ -3441,7 +3410,7 @@
},
{
"cell_type": "markdown",
- "id": "29cb1748",
+ "id": "d3d56d06",
"metadata": {
"editable": true
},
@@ -3463,7 +3432,7 @@
},
{
"cell_type": "markdown",
- "id": "d4cf1abd",
+ "id": "281ec76c",
"metadata": {
"editable": true
},
@@ -3483,7 +3452,7 @@
},
{
"cell_type": "markdown",
- "id": "60a028e5",
+ "id": "16eb46ae",
"metadata": {
"editable": true
},
@@ -3498,7 +3467,7 @@
},
{
"cell_type": "markdown",
- "id": "0b3b3776",
+ "id": "d6880678",
"metadata": {
"editable": true
},
@@ -3516,7 +3485,7 @@
},
{
"cell_type": "markdown",
- "id": "f933b11b",
+ "id": "f85f760a",
"metadata": {
"editable": true
},
@@ -3546,7 +3515,7 @@
},
{
"cell_type": "markdown",
- "id": "49558fe5",
+ "id": "469842c0",
"metadata": {
"editable": true
},
@@ -3575,7 +3544,7 @@
},
{
"cell_type": "markdown",
- "id": "68bc325e",
+ "id": "66388087",
"metadata": {
"editable": true
},
@@ -3606,7 +3575,7 @@
},
{
"cell_type": "markdown",
- "id": "42068b4c",
+ "id": "e26957e8",
"metadata": {
"editable": true
},
@@ -3629,7 +3598,7 @@
},
{
"cell_type": "markdown",
- "id": "0adb028d",
+ "id": "31886d45",
"metadata": {
"editable": true
},
@@ -3647,7 +3616,7 @@
},
{
"cell_type": "markdown",
- "id": "ad0660f2",
+ "id": "36603525",
"metadata": {
"editable": true
},
@@ -3670,7 +3639,7 @@
},
{
"cell_type": "markdown",
- "id": "90602dfa",
+ "id": "63bf1e9a",
"metadata": {
"editable": true
},
@@ -3691,7 +3660,7 @@
},
{
"cell_type": "markdown",
- "id": "3028d0ca",
+ "id": "c5b98e9b",
"metadata": {
"editable": true
},
@@ -3707,7 +3676,7 @@
},
{
"cell_type": "markdown",
- "id": "0f5f1146",
+ "id": "518e3444",
"metadata": {
"editable": true
},
diff --git a/doc/src/week47/week47.do.txt b/doc/src/week47/week47.do.txt
index 39d85acbf..2137f86c7 100644
--- a/doc/src/week47/week47.do.txt
+++ b/doc/src/week47/week47.do.txt
@@ -326,7 +326,7 @@ $\vert\vert \bm{w}\vert\vert =1$ subject to the condition
!bt
\[
-y_i(\bm{w}^T\bm{x}_i+b) \geq M \hspace{0.1cm}\forall i=1,2,\dots, p.
+y_i(\bm{w}^T\bm{x}_i+b) \geq M \hspace{0.1cm}\forall i=1,2,\dots, n.
\]
!et
All points are thus at a signed distance from the decision boundary defined by the line $L$. The parameters $b$ and $w_1$ and $w_2$ define this line.
@@ -340,14 +340,14 @@ We seek thus the largest value $M$ defined by
or just
!bt
\[
-y_i(\bm{w}^T\bm{x}_i+b) \geq M\vert \vert \bm{w}\vert\vert \hspace{0.1cm}\forall i.
+y_i(\bm{w}^T\bm{x}_i+b) \geq M\vert \vert \bm{w}\vert\vert \hspace{0.1cm}\forall i=1,2,\dots,n.
\]
!et
If we scale the equation so that $\vert \vert \bm{w}\vert\vert = 1/M$, we have to find the minimum of
-$\bm{w}^T\bm{w}=\vert \vert \bm{w}\vert\vert$ (the norm) subject to the condition
+$\bm{w}^T\bm{w}=\vert \vert \bm{w}\vert\vert_2^2$ (the norm) subject to the condition
!bt
\[
-y_i(\bm{w}^T\bm{x}_i+b) \geq 1 \hspace{0.1cm}\forall i.
+y_i(\bm{w}^T\bm{x}_i+b) \geq 1 \hspace{0.1cm}\forall i=1,2,\dots,n.
\]
!et
@@ -515,8 +515,8 @@ y_ny_1\bm{x}_n^T\bm{x}_1 & y_ny_2\bm{x}_n^T\bm{x}_2 & \dots & \dots & y_ny_n\bm{
\end{bmatrix}\bm{\lambda}-\mathbb{1}\bm{\lambda},
\]
!et
-subject to $\bm{y}^T\bm{\lambda}=0$. Here we defined the vectors $\bm{\lambda} =[\lambda_1,\lambda_2,\dots,\lambda_n]$ and
-$\bm{y}=[y_1,y_2,\dots,y_n]$.
+subject to $\bm{y}^T\bm{\lambda}=0$. Here we defined the vectors $\bm{\lambda}^T =[\lambda_1,\lambda_2,\dots,\lambda_n]$ and
+$\bm{y}^T=[y_1,y_2,\dots,y_n]$.
!split
@@ -1136,25 +1136,11 @@ sol[’x’]
sol[’primal objective’]
!ec
+
!split
-===== Back to the more realistic cases =====
+===== Support Vector Machines and Regression =====
-We are now ready to return to our setup of the optmization problem for a more realistic case. Introducing the _slack_ parameter $C$ we have
-!bt
-\[
-\frac{1}{2} \bm{\lambda}^T\begin{bmatrix} y_1y_1K(\bm{x}_1,\bm{x}_1) & y_1y_2K(\bm{x}_1,\bm{x}_2) & \dots & \dots & y_1y_nK(\bm{x}_1,\bm{x}_n) \\
-y_2y_1K(\bm{x}_2,\bm{x}_1) & y_2y_2K(\bm{x}_2,\bm{x}_2) & \dots & \dots & y_1y_nK(\bm{x}_2,\bm{x}_n) \\
-\dots & \dots & \dots & \dots & \dots \\
-\dots & \dots & \dots & \dots & \dots \\
-y_ny_1K(\bm{x}_n,\bm{x}_1) & y_ny_2K(\bm{x}_n\bm{x}_2) & \dots & \dots & y_ny_nK(\bm{x}_n,\bm{x}_n) \\
-\end{bmatrix}\bm{\lambda}-\mathbb{I}\bm{\lambda},
-\]
-!et
-subject to $\bm{y}^T\bm{\lambda}=0$. Here we defined the vectors $\bm{\lambda} =[\lambda_1,\lambda_2,\dots,\lambda_n]$ and
-$\bm{y}=[y_1,y_2,\dots,y_n]$.
-With the slack constants this leads to the additional constraint $0\leq \lambda_i \leq C$.
-
-_code will be added_
+Material may be added if of interest. See Bishop chapter 7.1 for a discussion.
!split
===== Summary of course =====