lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

update week 46

Browse Source
This commit is contained in:
Morten Hjorth-Jensen
2022-11-13 10:15:13 +01:00
parent 8ac7d9e4af
commit ff85f76c97
38 changed files with 2479 additions and 2000 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/pub/week46/html/._week46-bs006.html
+60 -46
View File
@@ -37,6 +37,14 @@ doconce format html week46.do.txt --html_style=bootstrap --pygments_html_style=d
<!-- tocinfo
{'highest level': 2,
'sections': [('Overview of week 46', 2, None, 'overview-of-week-46'),
('Eventual mini-workshop on project 3, Friday November 18',
2,
None,
'eventual-mini-workshop-on-project-3-friday-november-18'),
('Workshop topics 2021 (partly online)',
2,
None,
'workshop-topics-2021-partly-online'),
('Support Vector Machines, overarching aims',
2,
None,
@@ -129,33 +137,35 @@ MathJax.Hub.Config({
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._week46-bs001.html#overview-of-week-46" style="font-size: 80%;">Overview of week 46</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs002.html#support-vector-machines-overarching-aims" style="font-size: 80%;">Support Vector Machines, overarching aims</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs003.html#hyperplanes-and-all-that" style="font-size: 80%;">Hyperplanes and all that</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs004.html#what-is-a-hyperplane" style="font-size: 80%;">What is a hyperplane?</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs005.html#a-p-dimensional-space-of-features" style="font-size: 80%;">A \( p \)-dimensional space of features</a></li>
<!-- navigation toc: --> <li><a href="#the-two-dimensional-case" style="font-size: 80%;">The two-dimensional case</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs007.html#getting-into-the-details" style="font-size: 80%;">Getting into the details</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs008.html#first-attempt-at-a-minimization-approach" style="font-size: 80%;">First attempt at a minimization approach</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs009.html#solving-the-equations" style="font-size: 80%;">Solving the equations</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs010.html#code-example" style="font-size: 80%;">Code Example</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs011.html#problems-with-the-simpler-approach" style="font-size: 80%;">Problems with the Simpler Approach</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs012.html#a-better-approach" style="font-size: 80%;">A better approach</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs013.html#a-quick-reminder-on-lagrangian-multipliers" style="font-size: 80%;">A quick Reminder on Lagrangian Multipliers</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs014.html#adding-the-multiplier" style="font-size: 80%;">Adding the Multiplier</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs015.html#setting-up-the-problem" style="font-size: 80%;">Setting up the Problem</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs022.html#the-problem-to-solve" style="font-size: 80%;">The problem to solve</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs017.html#the-last-steps" style="font-size: 80%;">The last steps</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs018.html#a-soft-classifier" style="font-size: 80%;">A soft classifier</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs019.html#soft-optmization-problem" style="font-size: 80%;">Soft optmization problem</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs020.html#kernels-and-non-linearity" style="font-size: 80%;">Kernels and non-linearity</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs021.html#the-equations" style="font-size: 80%;">The equations</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs022.html#the-problem-to-solve" style="font-size: 80%;">The problem to solve</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs023.html#different-kernels-and-mercer-s-theorem" style="font-size: 80%;">Different kernels and Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs024.html#the-moons-example" style="font-size: 80%;">The moons example</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs025.html#mathematical-optimization-of-convex-functions" style="font-size: 80%;">Mathematical optimization of convex functions</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs026.html#how-do-we-solve-these-problems" style="font-size: 80%;">How do we solve these problems?</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs027.html#a-simple-example" style="font-size: 80%;">A simple example</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs028.html#back-to-the-more-realistic-cases" style="font-size: 80%;">Back to the more realistic cases</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs002.html#eventual-mini-workshop-on-project-3-friday-november-18" style="font-size: 80%;">Eventual mini-workshop on project 3, Friday November 18</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs003.html#workshop-topics-2021-partly-online" style="font-size: 80%;">Workshop topics 2021 (partly online)</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs004.html#support-vector-machines-overarching-aims" style="font-size: 80%;">Support Vector Machines, overarching aims</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs005.html#hyperplanes-and-all-that" style="font-size: 80%;">Hyperplanes and all that</a></li>
<!-- navigation toc: --> <li><a href="#what-is-a-hyperplane" style="font-size: 80%;">What is a hyperplane?</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs007.html#a-p-dimensional-space-of-features" style="font-size: 80%;">A \( p \)-dimensional space of features</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs008.html#the-two-dimensional-case" style="font-size: 80%;">The two-dimensional case</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs009.html#getting-into-the-details" style="font-size: 80%;">Getting into the details</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs010.html#first-attempt-at-a-minimization-approach" style="font-size: 80%;">First attempt at a minimization approach</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs011.html#solving-the-equations" style="font-size: 80%;">Solving the equations</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs012.html#code-example" style="font-size: 80%;">Code Example</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs013.html#problems-with-the-simpler-approach" style="font-size: 80%;">Problems with the Simpler Approach</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs014.html#a-better-approach" style="font-size: 80%;">A better approach</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs015.html#a-quick-reminder-on-lagrangian-multipliers" style="font-size: 80%;">A quick Reminder on Lagrangian Multipliers</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs016.html#adding-the-multiplier" style="font-size: 80%;">Adding the Multiplier</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs017.html#setting-up-the-problem" style="font-size: 80%;">Setting up the Problem</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs024.html#the-problem-to-solve" style="font-size: 80%;">The problem to solve</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs019.html#the-last-steps" style="font-size: 80%;">The last steps</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs020.html#a-soft-classifier" style="font-size: 80%;">A soft classifier</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs021.html#soft-optmization-problem" style="font-size: 80%;">Soft optmization problem</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs022.html#kernels-and-non-linearity" style="font-size: 80%;">Kernels and non-linearity</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs023.html#the-equations" style="font-size: 80%;">The equations</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs024.html#the-problem-to-solve" style="font-size: 80%;">The problem to solve</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs025.html#different-kernels-and-mercer-s-theorem" style="font-size: 80%;">Different kernels and Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs026.html#the-moons-example" style="font-size: 80%;">The moons example</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs027.html#mathematical-optimization-of-convex-functions" style="font-size: 80%;">Mathematical optimization of convex functions</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs028.html#how-do-we-solve-these-problems" style="font-size: 80%;">How do we solve these problems?</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs029.html#a-simple-example" style="font-size: 80%;">A simple example</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs030.html#back-to-the-more-realistic-cases" style="font-size: 80%;">Back to the more realistic cases</a></li>
</ul>
</li>
@@ -166,31 +176,35 @@ MathJax.Hub.Config({
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<a name="part0006"></a>
<!-- !split -->
<h2 id="the-two-dimensional-case" class="anchor">The two-dimensional case </h2>
<!-- !split -->
<h2 id="what-is-a-hyperplane" class="anchor">What is a hyperplane? </h2>
<p>Let us try to develop our intuition about SVMs by limiting ourselves to a two-dimensional
plane. To separate the two classes of data points, there are many
possible lines (hyperplanes if you prefer a more strict naming)
that could be chosen. Our objective is to find a
plane that has the maximum margin, i.e the maximum distance between
data points of both classes. Maximizing the margin distance provides
some reinforcement so that future data points can be classified with
more confidence.
<p>The aim of the SVM algorithm is to find a hyperplane in a
\( p \)-dimensional space, where \( p \) is the number of features that
distinctly classifies the data points.
</p>
<p>What a linear classifier attempts to accomplish is to split the
feature space into two half spaces by placing a hyperplane between the
data points. This hyperplane will be our decision boundary. All
points on one side of the plane will belong to class one and all points
on the other side of the plane will belong to the second class two.
<p>In a \( p \)-dimensional space, a hyperplane is what we call an affine subspace of dimension of \( p-1 \).
As an example, in two dimension, a hyperplane is simply as straight line while in three dimensions it is
a two-dimensional subspace, or stated simply, a plane.
</p>
<p>Unfortunately there are many ways in which we can place a hyperplane
to divide the data. Below is an example of two candidate hyperplanes
for our data sample.
<p>In two dimensions, with the variables \( x_1 \) and \( x_2 \), the hyperplane is defined as</p>
$$
b+w_1x_1+w_2x_2=0,
$$
<p>where \( b \) is the intercept and \( w_1 \) and \( w_2 \) define the elements of a vector orthogonal to the line
\( b+w_1x_1+w_2x_2=0 \).
In two dimensions we define the vectors \( \boldsymbol{x} =[x1,x2] \) and \( \boldsymbol{w}=[w1,w2] \).
We can then rewrite the above equation as
</p>
$$
\boldsymbol{x}^T\boldsymbol{w}+b=0.
$$
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
@@ -212,7 +226,7 @@ for our data sample.
<li><a href="._week46-bs014.html">15</a></li>
<li><a href="._week46-bs015.html">16</a></li>
<li><a href="">...</a></li>
<li><a href="._week46-bs028.html">29</a></li>
<li><a href="._week46-bs030.html">31</a></li>
<li><a href="._week46-bs007.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->