diff --git a/doc/pub/week47/html/week47-bs.html b/doc/pub/week47/html/week47-bs.html index ea9e55a0f..527cec9d1 100644 --- a/doc/pub/week47/html/week47-bs.html +++ b/doc/pub/week47/html/week47-bs.html @@ -54,7 +54,6 @@ doconce format html week47.do.txt --html_style=bootstrap --pygments_html_style=d None, 'first-attempt-at-a-minimization-approach'), ('Solving the equations', 2, None, 'solving-the-equations'), - ('Code Example', 2, None, 'code-example'), ('Problems with the Simpler Approach', 2, None, @@ -273,75 +272,74 @@ MathJax.Hub.Config({
+
+
For figures, see handwritten notes for Thursday November 25.
-
as the function that determines the line \( L \) that separates two classes (our two features), see the figure here.
+as the function that determines the line \( L \) that separates two classes (our two features), see the figures in the handwritten notes for Thursday November 25. +. +
Any point defined by \( \boldsymbol{x}_i \) and \( \boldsymbol{x}_2 \) on the line \( L \) will satisfy \( \boldsymbol{w}^T(\boldsymbol{x}_1-\boldsymbol{x}_2)=0 \).
@@ -531,39 +535,13 @@ $$The equations we discussed above can be coded rather easily (the framework is similar to what we developed for logistic -regression). We are going to set up a simple case with two classes only and we want to find a line which separates them the best possible way. +regression).
- -
-
-There are however problems with this approach, although it looks pretty straightforward to implement. When running the above code, we see that we can easily end up with many diffeent lines which separate the two classes.
@@ -743,6 +721,7 @@ $$In order to solve the above problem, we define the following Lagrangian function to be minimized
$$
diff --git a/doc/pub/week47/html/week47-solarized.html b/doc/pub/week47/html/week47-solarized.html
index 19684e17b..96a055e4b 100644
--- a/doc/pub/week47/html/week47-solarized.html
+++ b/doc/pub/week47/html/week47-solarized.html
@@ -81,7 +81,6 @@ div.toc p,a {
None,
'first-attempt-at-a-minimization-approach'),
('Solving the equations', 2, None, 'solving-the-equations'),
- ('Code Example', 2, None, 'code-example'),
('Problems with the Simpler Approach',
2,
None,
@@ -476,6 +475,7 @@ $$
\boldsymbol{x}^T\boldsymbol{w}+b=0.
$$
+
For figures, see handwritten notes for Thursday November 25.
as the function that determines the line \( L \) that separates two classes (our two features), see the figure here.
+as the function that determines the line \( L \) that separates two classes (our two features), see the figures in the handwritten notes for Thursday November 25. +. +
Any point defined by \( \boldsymbol{x}_i \) and \( \boldsymbol{x}_2 \) on the line \( L \) will satisfy \( \boldsymbol{w}^T(\boldsymbol{x}_1-\boldsymbol{x}_2)=0 \).
@@ -600,39 +602,13 @@ $$where \( \eta \) is our by now well-known learning rate.
The equations we discussed above can be coded rather easily (the framework is similar to what we developed for logistic -regression). We are going to set up a simple case with two classes only and we want to find a line which separates them the best possible way. +regression).
- -
-
-There are however problems with this approach, although it looks pretty straightforward to implement. When running the above code, we see that we can easily end up with many diffeent lines which separate the two classes.
@@ -777,6 +753,7 @@ $$In order to solve the above problem, we define the following Lagrangian function to be minimized
$$ {\cal L}(\lambda,b,\boldsymbol{w})=\frac{1}{2}\boldsymbol{w}^T\boldsymbol{w}-\sum_{i=1}^n\lambda_i\left[y_i(\boldsymbol{w}^T\boldsymbol{x}_i+b)-1\right], diff --git a/doc/pub/week47/html/week47.html b/doc/pub/week47/html/week47.html index 79418794d..7369ceef8 100644 --- a/doc/pub/week47/html/week47.html +++ b/doc/pub/week47/html/week47.html @@ -158,7 +158,6 @@ div.toc p,a { None, 'first-attempt-at-a-minimization-approach'), ('Solving the equations', 2, None, 'solving-the-equations'), - ('Code Example', 2, None, 'code-example'), ('Problems with the Simpler Approach', 2, None, @@ -553,6 +552,7 @@ $$ \boldsymbol{x}^T\boldsymbol{w}+b=0. $$ +For figures, see handwritten notes for Thursday November 25.
as the function that determines the line \( L \) that separates two classes (our two features), see the figure here.
+as the function that determines the line \( L \) that separates two classes (our two features), see the figures in the handwritten notes for Thursday November 25. +. +
Any point defined by \( \boldsymbol{x}_i \) and \( \boldsymbol{x}_2 \) on the line \( L \) will satisfy \( \boldsymbol{w}^T(\boldsymbol{x}_1-\boldsymbol{x}_2)=0 \).
@@ -677,39 +679,13 @@ $$where \( \eta \) is our by now well-known learning rate.
The equations we discussed above can be coded rather easily (the framework is similar to what we developed for logistic -regression). We are going to set up a simple case with two classes only and we want to find a line which separates them the best possible way. +regression).
- -
-
-There are however problems with this approach, although it looks pretty straightforward to implement. When running the above code, we see that we can easily end up with many diffeent lines which separate the two classes.
@@ -854,6 +830,7 @@ $$In order to solve the above problem, we define the following Lagrangian function to be minimized
$$ {\cal L}(\lambda,b,\boldsymbol{w})=\frac{1}{2}\boldsymbol{w}^T\boldsymbol{w}-\sum_{i=1}^n\lambda_i\left[y_i(\boldsymbol{w}^T\boldsymbol{x}_i+b)-1\right], diff --git a/doc/pub/week47/ipynb/ipynb-week47-src.tar.gz b/doc/pub/week47/ipynb/ipynb-week47-src.tar.gz index f5c93d0b9..9554402be 100644 Binary files a/doc/pub/week47/ipynb/ipynb-week47-src.tar.gz and b/doc/pub/week47/ipynb/ipynb-week47-src.tar.gz differ diff --git a/doc/pub/week47/ipynb/week47.ipynb b/doc/pub/week47/ipynb/week47.ipynb index ac896dc25..514d3775d 100644 --- a/doc/pub/week47/ipynb/week47.ipynb +++ b/doc/pub/week47/ipynb/week47.ipynb @@ -2,7 +2,7 @@ "cells": [ { "cell_type": "markdown", - "id": "11eedad1", + "id": "441a43d3", "metadata": { "editable": true }, @@ -14,7 +14,7 @@ }, { "cell_type": "markdown", - "id": "d229ce51", + "id": "9b7b8016", "metadata": { "editable": true }, @@ -29,7 +29,7 @@ }, { "cell_type": "markdown", - "id": "05269c87", + "id": "8f765cce", "metadata": { "editable": true }, @@ -52,7 +52,7 @@ }, { "cell_type": "markdown", - "id": "835c09a2", + "id": "d4cefcfd", "metadata": { "editable": true }, @@ -86,7 +86,7 @@ }, { "cell_type": "markdown", - "id": "873bd806", + "id": "e4693764", "metadata": { "editable": true }, @@ -108,7 +108,7 @@ { "cell_type": "code", "execution_count": 1, - "id": "70acd877", + "id": "cd2ae27b", "metadata": { "collapsed": false, "editable": true @@ -187,7 +187,7 @@ }, { "cell_type": "markdown", - "id": "037c53af", + "id": "224d0cee", "metadata": { "editable": true }, @@ -207,7 +207,7 @@ }, { "cell_type": "markdown", - "id": "6937fa1a", + "id": "c383589e", "metadata": { "editable": true }, @@ -219,7 +219,7 @@ }, { "cell_type": "markdown", - "id": "5a04f85f", + "id": "3c8ea551", "metadata": { "editable": true }, @@ -232,7 +232,7 @@ }, { "cell_type": "markdown", - "id": "6b532ca6", + "id": "4a7aa762", "metadata": { "editable": true }, @@ -244,7 +244,17 @@ }, { "cell_type": "markdown", - "id": "7fd0bcd8", + "id": "b6eec1b9", + "metadata": { + "editable": true + }, + "source": [ + "For figures, see [handwritten notes](https://github.com/CompPhysics/MachineLearning/tree/master/doc/HandWrittenNotes/2021) for Thursday November 25." + ] + }, + { + "cell_type": "markdown", + "id": "7ddfc2a4", "metadata": { "editable": true }, @@ -257,7 +267,7 @@ }, { "cell_type": "markdown", - "id": "60502389", + "id": "c1a7c663", "metadata": { "editable": true }, @@ -269,7 +279,7 @@ }, { "cell_type": "markdown", - "id": "cbe32b34", + "id": "6e56a6bd", "metadata": { "editable": true }, @@ -281,7 +291,7 @@ }, { "cell_type": "markdown", - "id": "12186d6e", + "id": "5a75b5b8", "metadata": { "editable": true }, @@ -293,7 +303,7 @@ }, { "cell_type": "markdown", - "id": "fcb8d56c", + "id": "3136f844", "metadata": { "editable": true }, @@ -303,7 +313,7 @@ }, { "cell_type": "markdown", - "id": "adcc6c78", + "id": "d99fe2c5", "metadata": { "editable": true }, @@ -315,7 +325,7 @@ }, { "cell_type": "markdown", - "id": "ed8a91a8", + "id": "bfa14a88", "metadata": { "editable": true }, @@ -326,7 +336,7 @@ }, { "cell_type": "markdown", - "id": "d8ee2f04", + "id": "9eeec720", "metadata": { "editable": true }, @@ -338,7 +348,7 @@ }, { "cell_type": "markdown", - "id": "8a5282be", + "id": "7681b46e", "metadata": { "editable": true }, @@ -351,7 +361,7 @@ }, { "cell_type": "markdown", - "id": "6864075d", + "id": "bf112257", "metadata": { "editable": true }, @@ -363,7 +373,7 @@ }, { "cell_type": "markdown", - "id": "b8d30ca8", + "id": "df7a0279", "metadata": { "editable": true }, @@ -373,7 +383,7 @@ }, { "cell_type": "markdown", - "id": "a2d08a2d", + "id": "48d64b2e", "metadata": { "editable": true }, @@ -402,7 +412,7 @@ }, { "cell_type": "markdown", - "id": "66f55738", + "id": "657dd547", "metadata": { "editable": true }, @@ -414,7 +424,7 @@ }, { "cell_type": "markdown", - "id": "325b47fb", + "id": "635493a3", "metadata": { "editable": true }, @@ -426,12 +436,13 @@ }, { "cell_type": "markdown", - "id": "7d7fbf12", + "id": "942246f0", "metadata": { "editable": true }, "source": [ - "as the function that determines the line $L$ that separates two classes (our two features), see the figure here. \n", + "as the function that determines the line $L$ that separates two classes (our two features), see the figures in the [handwritten notes](https://github.com/CompPhysics/MachineLearning/tree/master/doc/HandWrittenNotes/2021) for Thursday November 25.\n", + ". \n", "\n", "Any point defined by $\\boldsymbol{x}_i$ and $\\boldsymbol{x}_2$ on the line $L$ will satisfy $\\boldsymbol{w}^T(\\boldsymbol{x}_1-\\boldsymbol{x}_2)=0$. \n", "\n", @@ -440,7 +451,7 @@ }, { "cell_type": "markdown", - "id": "91066108", + "id": "e531a14e", "metadata": { "editable": true }, @@ -452,7 +463,7 @@ }, { "cell_type": "markdown", - "id": "3c5f028b", + "id": "6e49cdbc", "metadata": { "editable": true }, @@ -466,7 +477,7 @@ }, { "cell_type": "markdown", - "id": "01aea9cd", + "id": "a191b445", "metadata": { "editable": true }, @@ -478,7 +489,7 @@ }, { "cell_type": "markdown", - "id": "def8f1fc", + "id": "56fe3b52", "metadata": { "editable": true }, @@ -488,7 +499,7 @@ }, { "cell_type": "markdown", - "id": "034d7e72", + "id": "ef48a94c", "metadata": { "editable": true }, @@ -500,7 +511,7 @@ }, { "cell_type": "markdown", - "id": "a18e2f52", + "id": "f6fbcbae", "metadata": { "editable": true }, @@ -510,7 +521,7 @@ }, { "cell_type": "markdown", - "id": "27e7a61a", + "id": "b29775a4", "metadata": { "editable": true }, @@ -522,7 +533,7 @@ }, { "cell_type": "markdown", - "id": "e8188a27", + "id": "61d25a02", "metadata": { "editable": true }, @@ -534,7 +545,7 @@ }, { "cell_type": "markdown", - "id": "e7df215c", + "id": "88ad70f0", "metadata": { "editable": true }, @@ -546,7 +557,7 @@ }, { "cell_type": "markdown", - "id": "aef531b4", + "id": "8a9805b1", "metadata": { "editable": true }, @@ -556,7 +567,7 @@ }, { "cell_type": "markdown", - "id": "b73a9613", + "id": "38184889", "metadata": { "editable": true }, @@ -568,7 +579,7 @@ }, { "cell_type": "markdown", - "id": "93b93705", + "id": "68e207ee", "metadata": { "editable": true }, @@ -578,27 +589,17 @@ }, { "cell_type": "markdown", - "id": "2acb6d5c", - "metadata": { - "editable": true - }, - "source": [ - "## Code Example\n", - "\n", - "The equations we discussed above can be coded rather easily (the\n", - "framework is similar to what we developed for logistic\n", - "regression). We are going to set up a simple case with two classes only and we want to find a line which separates them the best possible way." - ] - }, - { - "cell_type": "markdown", - "id": "42451c9c", + "id": "fe1446d6", "metadata": { "editable": true }, "source": [ "## Problems with the Simpler Approach\n", "\n", + "The equations we discussed above can be coded rather easily (the\n", + "framework is similar to what we developed for logistic\n", + "regression). \n", + "\n", "There are however problems with this approach, although it looks\n", "pretty straightforward to implement. When running the above code, we see that we can easily end up with many diffeent lines which separate the two classes.\n", "\n", @@ -611,7 +612,7 @@ }, { "cell_type": "markdown", - "id": "875c0666", + "id": "14060d30", "metadata": { "editable": true }, @@ -627,7 +628,7 @@ }, { "cell_type": "markdown", - "id": "1815e5a6", + "id": "0fcf7b2d", "metadata": { "editable": true }, @@ -639,7 +640,7 @@ }, { "cell_type": "markdown", - "id": "e9e0cf87", + "id": "c6809e9b", "metadata": { "editable": true }, @@ -651,7 +652,7 @@ }, { "cell_type": "markdown", - "id": "7d4147d6", + "id": "915107ff", "metadata": { "editable": true }, @@ -663,7 +664,7 @@ }, { "cell_type": "markdown", - "id": "aa538edd", + "id": "148b8fbc", "metadata": { "editable": true }, @@ -673,7 +674,7 @@ }, { "cell_type": "markdown", - "id": "515cabfa", + "id": "20fe90e0", "metadata": { "editable": true }, @@ -685,7 +686,7 @@ }, { "cell_type": "markdown", - "id": "98ad7d18", + "id": "1f7f5ad4", "metadata": { "editable": true }, @@ -696,7 +697,7 @@ }, { "cell_type": "markdown", - "id": "1e1f5365", + "id": "405e59ec", "metadata": { "editable": true }, @@ -708,7 +709,7 @@ }, { "cell_type": "markdown", - "id": "d8b4a59e", + "id": "7c051028", "metadata": { "editable": true }, @@ -721,7 +722,7 @@ }, { "cell_type": "markdown", - "id": "110cb7ee", + "id": "07fbb08f", "metadata": { "editable": true }, @@ -734,7 +735,7 @@ }, { "cell_type": "markdown", - "id": "b769d022", + "id": "ca5e55e8", "metadata": { "editable": true }, @@ -746,7 +747,7 @@ }, { "cell_type": "markdown", - "id": "514d49d9", + "id": "24b89676", "metadata": { "editable": true }, @@ -756,7 +757,7 @@ }, { "cell_type": "markdown", - "id": "ce2e8468", + "id": "4b4db4f7", "metadata": { "editable": true }, @@ -768,7 +769,7 @@ }, { "cell_type": "markdown", - "id": "faae74f3", + "id": "2399f9c4", "metadata": { "editable": true }, @@ -778,7 +779,7 @@ }, { "cell_type": "markdown", - "id": "596f292e", + "id": "b7f7666e", "metadata": { "editable": true }, @@ -790,7 +791,7 @@ }, { "cell_type": "markdown", - "id": "50946c25", + "id": "76ae2593", "metadata": { "editable": true }, @@ -807,7 +808,7 @@ }, { "cell_type": "markdown", - "id": "c9803b27", + "id": "a415d18a", "metadata": { "editable": true }, @@ -819,7 +820,7 @@ }, { "cell_type": "markdown", - "id": "61dd7d63", + "id": "a7311b9b", "metadata": { "editable": true }, @@ -829,7 +830,7 @@ }, { "cell_type": "markdown", - "id": "f8371778", + "id": "c3370b3e", "metadata": { "editable": true }, @@ -841,7 +842,7 @@ }, { "cell_type": "markdown", - "id": "c9eaf0be", + "id": "ef69a59d", "metadata": { "editable": true }, @@ -851,7 +852,7 @@ }, { "cell_type": "markdown", - "id": "b829b68d", + "id": "32b0d248", "metadata": { "editable": true }, @@ -863,7 +864,7 @@ }, { "cell_type": "markdown", - "id": "ce089ead", + "id": "efce8b1d", "metadata": { "editable": true }, @@ -876,7 +877,7 @@ }, { "cell_type": "markdown", - "id": "ad85648b", + "id": "94c464d3", "metadata": { "editable": true }, @@ -888,7 +889,7 @@ }, { "cell_type": "markdown", - "id": "b2823efd", + "id": "330899ae", "metadata": { "editable": true }, @@ -900,7 +901,7 @@ }, { "cell_type": "markdown", - "id": "61eb6a55", + "id": "72b6b3c5", "metadata": { "editable": true }, @@ -910,7 +911,7 @@ }, { "cell_type": "markdown", - "id": "e3b50194", + "id": "e409858a", "metadata": { "editable": true }, @@ -924,7 +925,7 @@ }, { "cell_type": "markdown", - "id": "10fe759e", + "id": "6b3c4204", "metadata": { "editable": true }, @@ -934,7 +935,7 @@ }, { "cell_type": "markdown", - "id": "fa95693e", + "id": "94db4a8c", "metadata": { "editable": true }, @@ -946,7 +947,7 @@ }, { "cell_type": "markdown", - "id": "33dc27e7", + "id": "1799a58d", "metadata": { "editable": true }, @@ -956,7 +957,7 @@ }, { "cell_type": "markdown", - "id": "8a97e646", + "id": "04ee0d2d", "metadata": { "editable": true }, @@ -968,7 +969,7 @@ }, { "cell_type": "markdown", - "id": "9833d58a", + "id": "947f9c04", "metadata": { "editable": true }, @@ -978,7 +979,7 @@ }, { "cell_type": "markdown", - "id": "56748148", + "id": "ecf773f9", "metadata": { "editable": true }, @@ -990,7 +991,7 @@ }, { "cell_type": "markdown", - "id": "5aafb764", + "id": "bc8f11e0", "metadata": { "editable": true }, @@ -1004,7 +1005,7 @@ }, { "cell_type": "markdown", - "id": "82cd5c73", + "id": "1f6db8fe", "metadata": { "editable": true }, @@ -1016,18 +1017,19 @@ }, { "cell_type": "markdown", - "id": "35dc5cd9", + "id": "7ae6fca4", "metadata": { "editable": true }, "source": [ "## Setting up the Problem\n", + "\n", "In order to solve the above problem, we define the following Lagrangian function to be minimized" ] }, { "cell_type": "markdown", - "id": "2770972c", + "id": "7c8906c8", "metadata": { "editable": true }, @@ -1039,7 +1041,7 @@ }, { "cell_type": "markdown", - "id": "7d7c478c", + "id": "5b5adbd5", "metadata": { "editable": true }, @@ -1051,7 +1053,7 @@ }, { "cell_type": "markdown", - "id": "394e17a9", + "id": "f2c09c3f", "metadata": { "editable": true }, @@ -1063,7 +1065,7 @@ }, { "cell_type": "markdown", - "id": "6d085f58", + "id": "a3db7c50", "metadata": { "editable": true }, @@ -1073,7 +1075,7 @@ }, { "cell_type": "markdown", - "id": "5512ef2d", + "id": "ec5c7315", "metadata": { "editable": true }, @@ -1085,7 +1087,7 @@ }, { "cell_type": "markdown", - "id": "022d5960", + "id": "54afc174", "metadata": { "editable": true }, @@ -1095,7 +1097,7 @@ }, { "cell_type": "markdown", - "id": "3e32ab66", + "id": "42358128", "metadata": { "editable": true }, @@ -1107,7 +1109,7 @@ }, { "cell_type": "markdown", - "id": "1e4819da", + "id": "a1629247", "metadata": { "editable": true }, @@ -1118,7 +1120,7 @@ }, { "cell_type": "markdown", - "id": "d747b0ba", + "id": "4b650985", "metadata": { "editable": true }, @@ -1130,7 +1132,7 @@ }, { "cell_type": "markdown", - "id": "a6a4d86a", + "id": "3ca241de", "metadata": { "editable": true }, @@ -1144,7 +1146,7 @@ }, { "cell_type": "markdown", - "id": "28f83c49", + "id": "f8db86f9", "metadata": { "editable": true }, @@ -1156,7 +1158,7 @@ }, { "cell_type": "markdown", - "id": "582c0490", + "id": "7aabb47c", "metadata": { "editable": true }, @@ -1168,7 +1170,7 @@ }, { "cell_type": "markdown", - "id": "d91f1fba", + "id": "c8d61881", "metadata": { "editable": true }, @@ -1178,7 +1180,7 @@ }, { "cell_type": "markdown", - "id": "d8489e22", + "id": "11b7dc90", "metadata": { "editable": true }, @@ -1195,7 +1197,7 @@ }, { "cell_type": "markdown", - "id": "15cf6b01", + "id": "031eb330", "metadata": { "editable": true }, @@ -1206,7 +1208,7 @@ }, { "cell_type": "markdown", - "id": "7ae516c8", + "id": "36d7d3e9", "metadata": { "editable": true }, @@ -1219,7 +1221,7 @@ }, { "cell_type": "markdown", - "id": "a4475f57", + "id": "672614da", "metadata": { "editable": true }, @@ -1231,7 +1233,7 @@ }, { "cell_type": "markdown", - "id": "db157b5b", + "id": "70e17308", "metadata": { "editable": true }, @@ -1241,7 +1243,7 @@ }, { "cell_type": "markdown", - "id": "62e61a7b", + "id": "2aba9362", "metadata": { "editable": true }, @@ -1253,7 +1255,7 @@ }, { "cell_type": "markdown", - "id": "cc75cb92", + "id": "2dc26905", "metadata": { "editable": true }, @@ -1263,7 +1265,7 @@ }, { "cell_type": "markdown", - "id": "b28d44ef", + "id": "f4b79beb", "metadata": { "editable": true }, @@ -1275,7 +1277,7 @@ }, { "cell_type": "markdown", - "id": "5d9a5a73", + "id": "b52766fa", "metadata": { "editable": true }, @@ -1285,7 +1287,7 @@ }, { "cell_type": "markdown", - "id": "7014e7a3", + "id": "4cbe5c72", "metadata": { "editable": true }, @@ -1297,7 +1299,7 @@ }, { "cell_type": "markdown", - "id": "974813b7", + "id": "3fa36d56", "metadata": { "editable": true }, @@ -1307,7 +1309,7 @@ }, { "cell_type": "markdown", - "id": "d1691443", + "id": "b2f2f797", "metadata": { "editable": true }, @@ -1319,7 +1321,7 @@ }, { "cell_type": "markdown", - "id": "630ea9df", + "id": "924792c0", "metadata": { "editable": true }, @@ -1329,7 +1331,7 @@ }, { "cell_type": "markdown", - "id": "873d9437", + "id": "97545484", "metadata": { "editable": true }, @@ -1349,7 +1351,7 @@ }, { "cell_type": "markdown", - "id": "7479ef27", + "id": "d3b1f4d6", "metadata": { "editable": true }, @@ -1361,7 +1363,7 @@ }, { "cell_type": "markdown", - "id": "9d24f832", + "id": "0f6c0fd4", "metadata": { "editable": true }, @@ -1371,7 +1373,7 @@ }, { "cell_type": "markdown", - "id": "f54f1925", + "id": "7106ab0c", "metadata": { "editable": true }, @@ -1383,7 +1385,7 @@ }, { "cell_type": "markdown", - "id": "1ab5f910", + "id": "4be246c4", "metadata": { "editable": true }, @@ -1399,7 +1401,7 @@ }, { "cell_type": "markdown", - "id": "a8cfa315", + "id": "1a10e09c", "metadata": { "editable": true }, @@ -1411,7 +1413,7 @@ }, { "cell_type": "markdown", - "id": "928c273c", + "id": "5a6591cb", "metadata": { "editable": true }, @@ -1423,7 +1425,7 @@ }, { "cell_type": "markdown", - "id": "89c3a2a8", + "id": "0b6a60bb", "metadata": { "editable": true }, @@ -1433,7 +1435,7 @@ }, { "cell_type": "markdown", - "id": "3444a4f1", + "id": "9002c900", "metadata": { "editable": true }, @@ -1445,7 +1447,7 @@ }, { "cell_type": "markdown", - "id": "1c648831", + "id": "efaeee24", "metadata": { "editable": true }, @@ -1457,7 +1459,7 @@ }, { "cell_type": "markdown", - "id": "9a53ac4a", + "id": "94a0a96a", "metadata": { "editable": true }, @@ -1469,7 +1471,7 @@ }, { "cell_type": "markdown", - "id": "d93cd756", + "id": "35da28cd", "metadata": { "editable": true }, @@ -1479,7 +1481,7 @@ }, { "cell_type": "markdown", - "id": "41e268fc", + "id": "3e302d5a", "metadata": { "editable": true }, @@ -1491,7 +1493,7 @@ }, { "cell_type": "markdown", - "id": "fcbaafba", + "id": "efe155f7", "metadata": { "editable": true }, @@ -1501,7 +1503,7 @@ }, { "cell_type": "markdown", - "id": "3128acc6", + "id": "ff1f016f", "metadata": { "editable": true }, @@ -1513,7 +1515,7 @@ }, { "cell_type": "markdown", - "id": "68eee6ce", + "id": "59cab632", "metadata": { "editable": true }, @@ -1523,7 +1525,7 @@ }, { "cell_type": "markdown", - "id": "f68dee1a", + "id": "86e2728b", "metadata": { "editable": true }, @@ -1535,7 +1537,7 @@ }, { "cell_type": "markdown", - "id": "6aadb5c2", + "id": "fa9dd640", "metadata": { "editable": true }, @@ -1546,7 +1548,7 @@ }, { "cell_type": "markdown", - "id": "a8c28ca0", + "id": "8fb8d57e", "metadata": { "editable": true }, @@ -1558,7 +1560,7 @@ }, { "cell_type": "markdown", - "id": "b5a714d4", + "id": "e2726fd1", "metadata": { "editable": true }, @@ -1570,7 +1572,7 @@ }, { "cell_type": "markdown", - "id": "54ab9331", + "id": "1619157b", "metadata": { "editable": true }, @@ -1580,7 +1582,7 @@ }, { "cell_type": "markdown", - "id": "5b16a8d8", + "id": "3f7a7063", "metadata": { "editable": true }, @@ -1592,7 +1594,7 @@ }, { "cell_type": "markdown", - "id": "7e663d27", + "id": "035b199e", "metadata": { "editable": true }, @@ -1618,7 +1620,7 @@ { "cell_type": "code", "execution_count": 2, - "id": "c37b6b57", + "id": "2564e935", "metadata": { "collapsed": false, "editable": true @@ -1675,7 +1677,7 @@ }, { "cell_type": "markdown", - "id": "c02cfa09", + "id": "a0f7aa27", "metadata": { "editable": true }, @@ -1687,7 +1689,7 @@ }, { "cell_type": "markdown", - "id": "23ce8aa9", + "id": "1aa783a3", "metadata": { "editable": true }, @@ -1699,7 +1701,7 @@ }, { "cell_type": "markdown", - "id": "56d1e78f", + "id": "d803e017", "metadata": { "editable": true }, @@ -1709,7 +1711,7 @@ }, { "cell_type": "markdown", - "id": "bcaa6bfb", + "id": "aabaaf68", "metadata": { "editable": true }, @@ -1721,7 +1723,7 @@ }, { "cell_type": "markdown", - "id": "c6889b36", + "id": "4e674340", "metadata": { "editable": true }, @@ -1731,7 +1733,7 @@ }, { "cell_type": "markdown", - "id": "bbb242b8", + "id": "08423b9f", "metadata": { "editable": true }, @@ -1743,7 +1745,7 @@ }, { "cell_type": "markdown", - "id": "54c69269", + "id": "cfe3d3fa", "metadata": { "editable": true }, @@ -1754,7 +1756,7 @@ }, { "cell_type": "markdown", - "id": "198117f5", + "id": "0654390f", "metadata": { "editable": true }, @@ -1766,7 +1768,7 @@ }, { "cell_type": "markdown", - "id": "bb6a092d", + "id": "cfc69ba5", "metadata": { "editable": true }, @@ -1776,7 +1778,7 @@ }, { "cell_type": "markdown", - "id": "7c7963db", + "id": "4f953fd1", "metadata": { "editable": true }, @@ -1788,7 +1790,7 @@ }, { "cell_type": "markdown", - "id": "0bb4529c", + "id": "be4b01d0", "metadata": { "editable": true }, @@ -1806,7 +1808,7 @@ }, { "cell_type": "markdown", - "id": "d0860da4", + "id": "9d221f7a", "metadata": { "editable": true }, @@ -1817,7 +1819,7 @@ }, { "cell_type": "markdown", - "id": "02cbb83e", + "id": "f70d38be", "metadata": { "editable": true }, @@ -1829,7 +1831,7 @@ }, { "cell_type": "markdown", - "id": "21ab8e0a", + "id": "9501dd3a", "metadata": { "editable": true }, @@ -1839,7 +1841,7 @@ }, { "cell_type": "markdown", - "id": "0ccde505", + "id": "4c89e3d6", "metadata": { "editable": true }, @@ -1856,7 +1858,7 @@ }, { "cell_type": "markdown", - "id": "d325c73a", + "id": "f455aee7", "metadata": { "editable": true }, @@ -1870,7 +1872,7 @@ }, { "cell_type": "markdown", - "id": "14abe1e0", + "id": "57e5ae96", "metadata": { "editable": true }, @@ -1885,7 +1887,7 @@ }, { "cell_type": "markdown", - "id": "4aa701ba", + "id": "0b64948c", "metadata": { "editable": true }, @@ -1897,7 +1899,7 @@ }, { "cell_type": "markdown", - "id": "de58ef40", + "id": "cd1ff8c0", "metadata": { "editable": true }, @@ -1925,7 +1927,7 @@ }, { "cell_type": "markdown", - "id": "aa95c803", + "id": "d1cd71e4", "metadata": { "editable": true }, @@ -1937,7 +1939,7 @@ }, { "cell_type": "markdown", - "id": "e44cbcef", + "id": "f944a50f", "metadata": { "editable": true }, @@ -1952,7 +1954,7 @@ }, { "cell_type": "markdown", - "id": "7d15bfb0", + "id": "03656fab", "metadata": { "editable": true }, @@ -1963,7 +1965,7 @@ { "cell_type": "code", "execution_count": 3, - "id": "4516246b", + "id": "51a487e3", "metadata": { "collapsed": false, "editable": true @@ -2162,7 +2164,7 @@ }, { "cell_type": "markdown", - "id": "70711864", + "id": "b806a2e1", "metadata": { "editable": true }, @@ -2174,7 +2176,7 @@ }, { "cell_type": "markdown", - "id": "8effd67e", + "id": "7804736e", "metadata": { "editable": true }, @@ -2189,7 +2191,7 @@ }, { "cell_type": "markdown", - "id": "d02e2818", + "id": "d0f71fc0", "metadata": { "editable": true }, @@ -2206,7 +2208,7 @@ }, { "cell_type": "markdown", - "id": "76c05a44", + "id": "0cf90126", "metadata": { "editable": true }, @@ -2227,7 +2229,7 @@ { "cell_type": "code", "execution_count": 4, - "id": "80889c0c", + "id": "ae4eb7cf", "metadata": { "collapsed": false, "editable": true @@ -2240,7 +2242,7 @@ }, { "cell_type": "markdown", - "id": "cf7d92a4", + "id": "0b0262af", "metadata": { "editable": true }, @@ -2250,7 +2252,7 @@ }, { "cell_type": "markdown", - "id": "2ef9611b", + "id": "9a3d1bb1", "metadata": { "editable": true }, @@ -2262,7 +2264,7 @@ }, { "cell_type": "markdown", - "id": "3a00902a", + "id": "dfda25e9", "metadata": { "editable": true }, @@ -2277,7 +2279,7 @@ }, { "cell_type": "markdown", - "id": "1cb22c82", + "id": "fcefe1f8", "metadata": { "editable": true }, @@ -2287,7 +2289,7 @@ }, { "cell_type": "markdown", - "id": "ba290e45", + "id": "20377892", "metadata": { "editable": true }, @@ -2306,7 +2308,7 @@ }, { "cell_type": "markdown", - "id": "9b9e61f8", + "id": "23078a4a", "metadata": { "editable": true }, @@ -2316,7 +2318,7 @@ }, { "cell_type": "markdown", - "id": "bf906cdd", + "id": "9b02e601", "metadata": { "editable": true }, @@ -2328,7 +2330,7 @@ }, { "cell_type": "markdown", - "id": "de6f4dda", + "id": "cfd8fd0c", "metadata": { "editable": true }, @@ -2338,7 +2340,7 @@ }, { "cell_type": "markdown", - "id": "5a0cf8d8", + "id": "5cbdb0ee", "metadata": { "editable": true }, @@ -2350,7 +2352,7 @@ }, { "cell_type": "markdown", - "id": "e4d78d0d", + "id": "67492dc9", "metadata": { "editable": true }, @@ -2360,7 +2362,7 @@ }, { "cell_type": "markdown", - "id": "dcc18a57", + "id": "36a5ac5b", "metadata": { "editable": true }, @@ -2372,7 +2374,7 @@ }, { "cell_type": "markdown", - "id": "17ef78ff", + "id": "86207d6c", "metadata": { "editable": true }, @@ -2383,7 +2385,7 @@ }, { "cell_type": "markdown", - "id": "2a994afc", + "id": "4a1859e1", "metadata": { "editable": true }, @@ -2395,7 +2397,7 @@ }, { "cell_type": "markdown", - "id": "b59ed115", + "id": "a1ec9520", "metadata": { "editable": true }, @@ -2407,7 +2409,7 @@ { "cell_type": "code", "execution_count": 5, - "id": "2b8129eb", + "id": "991724bc", "metadata": { "collapsed": false, "editable": true @@ -2431,7 +2433,7 @@ }, { "cell_type": "markdown", - "id": "d41b30f0", + "id": "91fee137", "metadata": { "editable": true }, @@ -2443,7 +2445,7 @@ }, { "cell_type": "markdown", - "id": "71194f96", + "id": "56617f67", "metadata": { "editable": true }, @@ -2460,7 +2462,7 @@ }, { "cell_type": "markdown", - "id": "13cb5251", + "id": "d07335c7", "metadata": { "editable": true }, @@ -2474,7 +2476,7 @@ }, { "cell_type": "markdown", - "id": "2b6d7d6e", + "id": "eeae8340", "metadata": { "editable": true }, @@ -2484,7 +2486,7 @@ }, { "cell_type": "markdown", - "id": "8931b59e", + "id": "4b276877", "metadata": { "editable": true }, @@ -2499,7 +2501,7 @@ }, { "cell_type": "markdown", - "id": "03727abd", + "id": "0ee162a3", "metadata": { "editable": true }, @@ -2515,7 +2517,7 @@ }, { "cell_type": "markdown", - "id": "f72ea138", + "id": "9f4d8dd7", "metadata": { "editable": true }, @@ -2540,7 +2542,7 @@ }, { "cell_type": "markdown", - "id": "e57c523f", + "id": "7ab47dbb", "metadata": { "editable": true }, @@ -2569,7 +2571,7 @@ }, { "cell_type": "markdown", - "id": "7490d694", + "id": "dba69bc1", "metadata": { "editable": true }, @@ -2585,7 +2587,7 @@ }, { "cell_type": "markdown", - "id": "4f7bb20b", + "id": "fb548338", "metadata": { "editable": true }, @@ -2610,7 +2612,7 @@ }, { "cell_type": "markdown", - "id": "b83826b7", + "id": "9a1e615b", "metadata": { "editable": true }, @@ -2657,7 +2659,7 @@ }, { "cell_type": "markdown", - "id": "134d371b", + "id": "94260cc2", "metadata": { "editable": true }, @@ -2693,7 +2695,7 @@ }, { "cell_type": "markdown", - "id": "e131c86a", + "id": "e6a30863", "metadata": { "editable": true }, @@ -2716,7 +2718,7 @@ }, { "cell_type": "markdown", - "id": "2f7b0955", + "id": "cca199f2", "metadata": { "editable": true }, @@ -2739,7 +2741,7 @@ }, { "cell_type": "markdown", - "id": "49fa4b7b", + "id": "f23bfec8", "metadata": { "editable": true }, @@ -2759,7 +2761,7 @@ }, { "cell_type": "markdown", - "id": "7d5c5a1c", + "id": "9f04f253", "metadata": { "editable": true }, @@ -2775,7 +2777,7 @@ }, { "cell_type": "markdown", - "id": "d556fdb2", + "id": "1fe7e57e", "metadata": { "editable": true }, @@ -2803,7 +2805,7 @@ }, { "cell_type": "markdown", - "id": "a4ba04de", + "id": "6f00ca9d", "metadata": { "editable": true }, @@ -2825,7 +2827,7 @@ }, { "cell_type": "markdown", - "id": "82576d56", + "id": "02275c60", "metadata": { "editable": true }, @@ -2850,7 +2852,7 @@ }, { "cell_type": "markdown", - "id": "3957ee87", + "id": "6bcb8e9f", "metadata": { "editable": true }, @@ -2868,7 +2870,7 @@ }, { "cell_type": "markdown", - "id": "8f43da22", + "id": "f397cbc9", "metadata": { "editable": true }, @@ -2896,7 +2898,7 @@ }, { "cell_type": "markdown", - "id": "e08a1bc0", + "id": "e27568c5", "metadata": { "editable": true }, @@ -2910,7 +2912,7 @@ }, { "cell_type": "markdown", - "id": "cf1b3d0d", + "id": "7c5dca33", "metadata": { "editable": true }, @@ -2936,7 +2938,7 @@ }, { "cell_type": "markdown", - "id": "8dca4061", + "id": "75ea06c4", "metadata": { "editable": true }, @@ -2965,7 +2967,7 @@ }, { "cell_type": "markdown", - "id": "b9f9edee", + "id": "973910ed", "metadata": { "editable": true }, @@ -2982,7 +2984,7 @@ }, { "cell_type": "markdown", - "id": "08640fb4", + "id": "e8a9939d", "metadata": { "editable": true }, @@ -3004,7 +3006,7 @@ }, { "cell_type": "markdown", - "id": "9557e3f5", + "id": "22ee217d", "metadata": { "editable": true }, @@ -3022,7 +3024,7 @@ }, { "cell_type": "markdown", - "id": "8a30742f", + "id": "1539fe90", "metadata": { "editable": true }, @@ -3045,7 +3047,7 @@ }, { "cell_type": "markdown", - "id": "0cea4abf", + "id": "fd5ea5f8", "metadata": { "editable": true }, @@ -3064,7 +3066,7 @@ }, { "cell_type": "markdown", - "id": "f13fb9f6", + "id": "e629ac9a", "metadata": { "editable": true }, @@ -3080,7 +3082,7 @@ }, { "cell_type": "markdown", - "id": "81d6be62", + "id": "b8a18a9c", "metadata": { "editable": true }, @@ -3095,7 +3097,7 @@ }, { "cell_type": "markdown", - "id": "c9da5a3f", + "id": "79cbd92e", "metadata": { "editable": true }, @@ -3119,7 +3121,7 @@ }, { "cell_type": "markdown", - "id": "a1ea8499", + "id": "b26ea45e", "metadata": { "editable": true }, @@ -3131,7 +3133,7 @@ }, { "cell_type": "markdown", - "id": "ff723dbd", + "id": "307b4b7b", "metadata": { "editable": true }, @@ -3149,7 +3151,7 @@ }, { "cell_type": "markdown", - "id": "84fe806f", + "id": "5c066128", "metadata": { "editable": true }, @@ -3159,7 +3161,7 @@ }, { "cell_type": "markdown", - "id": "29932151", + "id": "721ef62a", "metadata": { "editable": true }, @@ -3177,7 +3179,7 @@ }, { "cell_type": "markdown", - "id": "5e6fb888", + "id": "9a832728", "metadata": { "editable": true }, @@ -3187,7 +3189,7 @@ }, { "cell_type": "markdown", - "id": "1f48a948", + "id": "ee7d0eba", "metadata": { "editable": true }, @@ -3206,7 +3208,7 @@ }, { "cell_type": "markdown", - "id": "62302009", + "id": "243fef20", "metadata": { "editable": true }, @@ -3218,7 +3220,7 @@ }, { "cell_type": "markdown", - "id": "03ad1fab", + "id": "4858119d", "metadata": { "editable": true }, @@ -3232,7 +3234,7 @@ }, { "cell_type": "markdown", - "id": "36861f13", + "id": "926ef74b", "metadata": { "editable": true }, @@ -3247,7 +3249,7 @@ }, { "cell_type": "markdown", - "id": "3ed52b2d", + "id": "a57111e6", "metadata": { "editable": true }, @@ -3265,7 +3267,7 @@ }, { "cell_type": "markdown", - "id": "c28fbd39", + "id": "8d6b130d", "metadata": { "editable": true }, @@ -3279,7 +3281,7 @@ }, { "cell_type": "markdown", - "id": "5ab7e1fd", + "id": "6b861446", "metadata": { "editable": true }, @@ -3297,7 +3299,7 @@ }, { "cell_type": "markdown", - "id": "deaa82f8", + "id": "01055af3", "metadata": { "editable": true }, @@ -3321,7 +3323,7 @@ }, { "cell_type": "markdown", - "id": "4a3b2c29", + "id": "a08e71c6", "metadata": { "editable": true }, @@ -3359,7 +3361,7 @@ }, { "cell_type": "markdown", - "id": "2b0f5287", + "id": "26ca4343", "metadata": { "editable": true }, @@ -3382,7 +3384,7 @@ }, { "cell_type": "markdown", - "id": "03fdfcaf", + "id": "d73601ed", "metadata": { "editable": true }, @@ -3418,7 +3420,7 @@ }, { "cell_type": "markdown", - "id": "230473cf", + "id": "224c0bc8", "metadata": { "editable": true }, @@ -3439,7 +3441,7 @@ }, { "cell_type": "markdown", - "id": "846d275b", + "id": "6e099fd1", "metadata": { "editable": true }, @@ -3461,7 +3463,7 @@ }, { "cell_type": "markdown", - "id": "ed7f7eed", + "id": "d6d3e71c", "metadata": { "editable": true }, @@ -3481,7 +3483,7 @@ }, { "cell_type": "markdown", - "id": "94f27695", + "id": "a5b322df", "metadata": { "editable": true }, @@ -3496,7 +3498,7 @@ }, { "cell_type": "markdown", - "id": "2ae2b5d9", + "id": "7bc64edb", "metadata": { "editable": true }, @@ -3514,7 +3516,7 @@ }, { "cell_type": "markdown", - "id": "c8170bd4", + "id": "636cbc0f", "metadata": { "editable": true }, @@ -3544,7 +3546,7 @@ }, { "cell_type": "markdown", - "id": "770c9f3b", + "id": "57c30225", "metadata": { "editable": true }, @@ -3573,7 +3575,7 @@ }, { "cell_type": "markdown", - "id": "24ab7dc9", + "id": "c51bbe72", "metadata": { "editable": true }, @@ -3604,7 +3606,7 @@ }, { "cell_type": "markdown", - "id": "daff58eb", + "id": "d8561e2a", "metadata": { "editable": true }, @@ -3627,7 +3629,7 @@ }, { "cell_type": "markdown", - "id": "49afc03f", + "id": "a2c6d822", "metadata": { "editable": true }, @@ -3645,7 +3647,7 @@ }, { "cell_type": "markdown", - "id": "6279873b", + "id": "01529986", "metadata": { "editable": true }, @@ -3668,7 +3670,7 @@ }, { "cell_type": "markdown", - "id": "10760689", + "id": "6fc3228e", "metadata": { "editable": true }, @@ -3689,7 +3691,7 @@ }, { "cell_type": "markdown", - "id": "5245b378", + "id": "a7857509", "metadata": { "editable": true }, @@ -3705,7 +3707,7 @@ }, { "cell_type": "markdown", - "id": "7bce2b05", + "id": "595ec8ca", "metadata": { "editable": true }, diff --git a/doc/src/week47/week47.do.txt b/doc/src/week47/week47.do.txt index 6ded92c4b..34554d2d0 100644 --- a/doc/src/week47/week47.do.txt +++ b/doc/src/week47/week47.do.txt @@ -162,6 +162,8 @@ We can then rewrite the above equation as \] !et +For figures, see "handwritten notes":"https://github.com/CompPhysics/MachineLearning/tree/master/doc/HandWrittenNotes/2021" for Thursday November 25. + !split ===== A $p$-dimensional space of features ===== @@ -236,7 +238,8 @@ Let us define the function f(x) = \bm{w}^T\bm{x}+b = 0, \] !et -as the function that determines the line $L$ that separates two classes (our two features), see the figure here. +as the function that determines the line $L$ that separates two classes (our two features), see the figures in the "handwritten notes":"https://github.com/CompPhysics/MachineLearning/tree/master/doc/HandWrittenNotes/2021" for Thursday November 25. +. Any point defined by $\bm{x}_i$ and $\bm{x}_2$ on the line $L$ will satisfy $\bm{w}^T(\bm{x}_1-\bm{x}_2)=0$. @@ -292,19 +295,15 @@ and where $\eta$ is our by now well-known learning rate. -!split -===== Code Example ===== -The equations we discussed above can be coded rather easily (the -framework is similar to what we developed for logistic -regression). We are going to set up a simple case with two classes only and we want to find a line which separates them the best possible way. -!bc pycod - -!ec !split ===== Problems with the Simpler Approach ===== +The equations we discussed above can be coded rather easily (the +framework is similar to what we developed for logistic +regression). + There are however problems with this approach, although it looks pretty straightforward to implement. When running the above code, we see that we can easily end up with many diffeent lines which separate the two classes. @@ -458,6 +457,7 @@ If we have a set of constraints $\phi_k$ we have the equations !split ===== Setting up the Problem ===== + In order to solve the above problem, we define the following Lagrangian function to be minimized !bt \[