lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

added dot files

Browse Source
This commit is contained in:
mhjensen
2020-11-19 05:37:19 +01:00
parent b1af9b1fcb
commit cd4d19037b
4 changed files with 949 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/pub/week47/html/._week47-bs028.html
+229
View File
@@ -0,0 +1,229 @@
<!--
Automatically generated HTML file from DocOnce source
(https://github.com/hplgit/doconce/)
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="description" content="Week 47: Support Vector Machines">
<title>Week 47: Support Vector Machines</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<!-- not necessary
<link href="https://netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
-->
<style type="text/css">
/* Add scrollbar to dropdown menus in bootstrap navigation bar */
.dropdown-menu {
height: auto;
max-height: 400px;
overflow-x: hidden;
}
/* Adds an invisible element before each target to offset for the navigation
bar */
.anchor::before {
content:"";
display:block;
height:50px; /* fixed header height for style bootstrap */
margin:-50px 0 0; /* negative fixed header height */
}
</style>
</head>
<!-- tocinfo
{'highest level': 2,
'sections': [('Overview of week 46', 2, None, '___sec0'),
('Thursday', 2, None, '___sec1'),
('Friday', 2, None, '___sec2'),
('Workshop plan Friday (first lecture)', 2, None, '___sec3'),
('Support Vector Machines, overarching aims', 2, None, '___sec4'),
('Hyperplanes and all that', 2, None, '___sec5'),
('What is a hyperplane?', 2, None, '___sec6'),
('A $p$-dimensional space of features', 2, None, '___sec7'),
('The two-dimensional case', 2, None, '___sec8'),
('Getting into the details', 2, None, '___sec9'),
('First attempt at a minimization approach', 2, None, '___sec10'),
('Solving the equations', 2, None, '___sec11'),
('Code Example', 2, None, '___sec12'),
('Problems with the Simpler Approach', 2, None, '___sec13'),
('A better approach', 2, None, '___sec14'),
('A quick Reminder on Lagrangian Multipliers',
2,
None,
'___sec15'),
('Adding the Multiplier', 2, None, '___sec16'),
('Setting up the Problem', 2, None, '___sec17'),
('The problem to solve', 2, None, '___sec18'),
('The last steps', 2, None, '___sec19'),
('A soft classifier', 2, None, '___sec20'),
('Soft optmization problem', 2, None, '___sec21'),
('Kernels and non-linearity', 2, None, '___sec22'),
('The equations', 2, None, '___sec23'),
('The problem to solve', 2, None, '___sec24'),
("Different kernels and Mercer's theorem", 2, None, '___sec25'),
('The moons example', 2, None, '___sec26'),
('Mathematical optimization of convex functions',
2,
None,
'___sec27'),
('How do we solve these problems?', 2, None, '___sec28'),
('A simple example', 2, None, '___sec29'),
('Back to the more realistic cases', 2, None, '___sec30')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "none" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-responsive-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="week47-bs.html">Week 47: Support Vector Machines</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._week47-bs001.html#___sec0" style="font-size: 80%;">Overview of week 46</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs002.html#___sec1" style="font-size: 80%;">Thursday</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs003.html#___sec2" style="font-size: 80%;">Friday</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs004.html#___sec3" style="font-size: 80%;">Workshop plan Friday (first lecture)</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs005.html#___sec4" style="font-size: 80%;">Support Vector Machines, overarching aims</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs006.html#___sec5" style="font-size: 80%;">Hyperplanes and all that</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs007.html#___sec6" style="font-size: 80%;">What is a hyperplane?</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs008.html#___sec7" style="font-size: 80%;">A \( p \)-dimensional space of features</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs009.html#___sec8" style="font-size: 80%;">The two-dimensional case</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs010.html#___sec9" style="font-size: 80%;">Getting into the details</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs011.html#___sec10" style="font-size: 80%;">First attempt at a minimization approach</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs012.html#___sec11" style="font-size: 80%;">Solving the equations</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs013.html#___sec12" style="font-size: 80%;">Code Example</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs014.html#___sec13" style="font-size: 80%;">Problems with the Simpler Approach</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs015.html#___sec14" style="font-size: 80%;">A better approach</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs016.html#___sec15" style="font-size: 80%;">A quick Reminder on Lagrangian Multipliers</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs017.html#___sec16" style="font-size: 80%;">Adding the Multiplier</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs018.html#___sec17" style="font-size: 80%;">Setting up the Problem</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs019.html#___sec18" style="font-size: 80%;">The problem to solve</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs020.html#___sec19" style="font-size: 80%;">The last steps</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs021.html#___sec20" style="font-size: 80%;">A soft classifier</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs022.html#___sec21" style="font-size: 80%;">Soft optmization problem</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs023.html#___sec22" style="font-size: 80%;">Kernels and non-linearity</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs024.html#___sec23" style="font-size: 80%;">The equations</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs025.html#___sec24" style="font-size: 80%;">The problem to solve</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs026.html#___sec25" style="font-size: 80%;">Different kernels and Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs027.html#___sec26" style="font-size: 80%;">The moons example</a></li>
<!-- navigation toc: --> <li><a href="#___sec27" style="font-size: 80%;">Mathematical optimization of convex functions</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs029.html#___sec28" style="font-size: 80%;">How do we solve these problems?</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs030.html#___sec29" style="font-size: 80%;">A simple example</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs031.html#___sec30" style="font-size: 80%;">Back to the more realistic cases</a></li>
</ul>
</li>
</ul>
</div>
</div>
</div> <!-- end of navigation bar -->
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<a name="part0028"></a>
<!-- !split -->
<h2 id="___sec27" class="anchor">Mathematical optimization of convex functions </h2>
<p>
A mathematical (quadratic) optimization problem, or just optimization problem, has the form
$$
\begin{align*}
&\mathrm{min}_{\lambda}\hspace{0.2cm} \frac{1}{2}\boldsymbol{\lambda}^T\boldsymbol{P}\boldsymbol{\lambda}+\boldsymbol{q}^T\boldsymbol{\lambda},\\ \nonumber
&\mathrm{subject\hspace{0.1cm}to} \hspace{0.2cm} \boldsymbol{G}\boldsymbol{\lambda} \preceq \boldsymbol{h} \wedge \boldsymbol{A}\boldsymbol{\lambda}=f.
\end{align*}
$$
subject to some constraints for say a selected set \( i=1,2,\dots, n \).
In our case we are optimizing with respect to the Lagrangian multipliers \( \lambda_i \), and the
vector \( \boldsymbol{\lambda}=[\lambda_1, \lambda_2,\dots, \lambda_n] \) is the optimization variable we are dealing with.
<p>
In our case we are particularly interested in a class of optimization problems called convex optmization problems.
In our discussion on gradient descent methods we discussed at length the definition of a convex function.
<p>
Convex optimization problems play a central role in applied mathematics and we recommend strongly <a href="http://web.stanford.edu/~boyd/cvxbook/" target="_self">Boyd and Vandenberghe's text on the topics</a>.
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
<li><a href="._week47-bs027.html">&laquo;</a></li>
<li><a href="._week47-bs000.html">1</a></li>
<li><a href="">...</a></li>
<li><a href="._week47-bs020.html">21</a></li>
<li><a href="._week47-bs021.html">22</a></li>
<li><a href="._week47-bs022.html">23</a></li>
<li><a href="._week47-bs023.html">24</a></li>
<li><a href="._week47-bs024.html">25</a></li>
<li><a href="._week47-bs025.html">26</a></li>
<li><a href="._week47-bs026.html">27</a></li>
<li><a href="._week47-bs027.html">28</a></li>
<li class="active"><a href="._week47-bs028.html">29</a></li>
<li><a href="._week47-bs029.html">30</a></li>
<li><a href="._week47-bs030.html">31</a></li>
<li><a href="._week47-bs031.html">32</a></li>
<li><a href="._week47-bs029.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
</div> <!-- end container -->
<!-- include javascript, jQuery *first* -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script src="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/js/bootstrap.min.js"></script>
<!-- Bootstrap footer
<footer>
<a href="http://..."><img width="250" align=right src="http://..."></a>
</footer>
-->
<center style="font-size:80%">
<!-- copyright only on the titlepage -->
</center>
</body>
</html>
doc/pub/week47/html/._week47-bs029.html
+229
View File
@@ -0,0 +1,229 @@
<!--
Automatically generated HTML file from DocOnce source
(https://github.com/hplgit/doconce/)
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="description" content="Week 47: Support Vector Machines">
<title>Week 47: Support Vector Machines</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<!-- not necessary
<link href="https://netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
-->
<style type="text/css">
/* Add scrollbar to dropdown menus in bootstrap navigation bar */
.dropdown-menu {
height: auto;
max-height: 400px;
overflow-x: hidden;
}
/* Adds an invisible element before each target to offset for the navigation
bar */
.anchor::before {
content:"";
display:block;
height:50px; /* fixed header height for style bootstrap */
margin:-50px 0 0; /* negative fixed header height */
}
</style>
</head>
<!-- tocinfo
{'highest level': 2,
'sections': [('Overview of week 46', 2, None, '___sec0'),
('Thursday', 2, None, '___sec1'),
('Friday', 2, None, '___sec2'),
('Workshop plan Friday (first lecture)', 2, None, '___sec3'),
('Support Vector Machines, overarching aims', 2, None, '___sec4'),
('Hyperplanes and all that', 2, None, '___sec5'),
('What is a hyperplane?', 2, None, '___sec6'),
('A $p$-dimensional space of features', 2, None, '___sec7'),
('The two-dimensional case', 2, None, '___sec8'),
('Getting into the details', 2, None, '___sec9'),
('First attempt at a minimization approach', 2, None, '___sec10'),
('Solving the equations', 2, None, '___sec11'),
('Code Example', 2, None, '___sec12'),
('Problems with the Simpler Approach', 2, None, '___sec13'),
('A better approach', 2, None, '___sec14'),
('A quick Reminder on Lagrangian Multipliers',
2,
None,
'___sec15'),
('Adding the Multiplier', 2, None, '___sec16'),
('Setting up the Problem', 2, None, '___sec17'),
('The problem to solve', 2, None, '___sec18'),
('The last steps', 2, None, '___sec19'),
('A soft classifier', 2, None, '___sec20'),
('Soft optmization problem', 2, None, '___sec21'),
('Kernels and non-linearity', 2, None, '___sec22'),
('The equations', 2, None, '___sec23'),
('The problem to solve', 2, None, '___sec24'),
("Different kernels and Mercer's theorem", 2, None, '___sec25'),
('The moons example', 2, None, '___sec26'),
('Mathematical optimization of convex functions',
2,
None,
'___sec27'),
('How do we solve these problems?', 2, None, '___sec28'),
('A simple example', 2, None, '___sec29'),
('Back to the more realistic cases', 2, None, '___sec30')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "none" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-responsive-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="week47-bs.html">Week 47: Support Vector Machines</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._week47-bs001.html#___sec0" style="font-size: 80%;">Overview of week 46</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs002.html#___sec1" style="font-size: 80%;">Thursday</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs003.html#___sec2" style="font-size: 80%;">Friday</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs004.html#___sec3" style="font-size: 80%;">Workshop plan Friday (first lecture)</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs005.html#___sec4" style="font-size: 80%;">Support Vector Machines, overarching aims</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs006.html#___sec5" style="font-size: 80%;">Hyperplanes and all that</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs007.html#___sec6" style="font-size: 80%;">What is a hyperplane?</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs008.html#___sec7" style="font-size: 80%;">A \( p \)-dimensional space of features</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs009.html#___sec8" style="font-size: 80%;">The two-dimensional case</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs010.html#___sec9" style="font-size: 80%;">Getting into the details</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs011.html#___sec10" style="font-size: 80%;">First attempt at a minimization approach</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs012.html#___sec11" style="font-size: 80%;">Solving the equations</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs013.html#___sec12" style="font-size: 80%;">Code Example</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs014.html#___sec13" style="font-size: 80%;">Problems with the Simpler Approach</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs015.html#___sec14" style="font-size: 80%;">A better approach</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs016.html#___sec15" style="font-size: 80%;">A quick Reminder on Lagrangian Multipliers</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs017.html#___sec16" style="font-size: 80%;">Adding the Multiplier</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs018.html#___sec17" style="font-size: 80%;">Setting up the Problem</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs019.html#___sec18" style="font-size: 80%;">The problem to solve</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs020.html#___sec19" style="font-size: 80%;">The last steps</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs021.html#___sec20" style="font-size: 80%;">A soft classifier</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs022.html#___sec21" style="font-size: 80%;">Soft optmization problem</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs023.html#___sec22" style="font-size: 80%;">Kernels and non-linearity</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs024.html#___sec23" style="font-size: 80%;">The equations</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs025.html#___sec24" style="font-size: 80%;">The problem to solve</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs026.html#___sec25" style="font-size: 80%;">Different kernels and Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs027.html#___sec26" style="font-size: 80%;">The moons example</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs028.html#___sec27" style="font-size: 80%;">Mathematical optimization of convex functions</a></li>
<!-- navigation toc: --> <li><a href="#___sec28" style="font-size: 80%;">How do we solve these problems?</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs030.html#___sec29" style="font-size: 80%;">A simple example</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs031.html#___sec30" style="font-size: 80%;">Back to the more realistic cases</a></li>
</ul>
</li>
</ul>
</div>
</div>
</div> <!-- end of navigation bar -->
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<a name="part0029"></a>
<!-- !split -->
<h2 id="___sec28" class="anchor">How do we solve these problems? </h2>
<p>
If we use Python as programming language and wish to venture beyond
<b>scikit-learn</b>, <b>tensorflow</b> and similar software which makes our
lives so much easier, we need to dive into the wonderful world of
quadratic programming. We can, if we wish, solve the minimization
problem using say standard gradient methods or conjugate gradient
methods. However, these methods tend to exhibit a rather slow
converge. So, welcome to the promised land of quadratic programming.
<p>
The functions we need are contained in the quadratic programming package <b>CVXOPT</b> and we need to import it together with <b>numpy</b> as
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">numpy</span>
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">cvxopt</span>
</pre></div>
<p>
This will make our life much easier. You don't need t write your own optimizer.
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
<li><a href="._week47-bs028.html">&laquo;</a></li>
<li><a href="._week47-bs000.html">1</a></li>
<li><a href="">...</a></li>
<li><a href="._week47-bs021.html">22</a></li>
<li><a href="._week47-bs022.html">23</a></li>
<li><a href="._week47-bs023.html">24</a></li>
<li><a href="._week47-bs024.html">25</a></li>
<li><a href="._week47-bs025.html">26</a></li>
<li><a href="._week47-bs026.html">27</a></li>
<li><a href="._week47-bs027.html">28</a></li>
<li><a href="._week47-bs028.html">29</a></li>
<li class="active"><a href="._week47-bs029.html">30</a></li>
<li><a href="._week47-bs030.html">31</a></li>
<li><a href="._week47-bs031.html">32</a></li>
<li><a href="._week47-bs030.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
</div> <!-- end container -->
<!-- include javascript, jQuery *first* -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script src="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/js/bootstrap.min.js"></script>
<!-- Bootstrap footer
<footer>
<a href="http://..."><img width="250" align=right src="http://..."></a>
</footer>
-->
<center style="font-size:80%">
<!-- copyright only on the titlepage -->
</center>
</body>
</html>
doc/pub/week47/html/._week47-bs030.html
+270
View File
@@ -0,0 +1,270 @@
<!--
Automatically generated HTML file from DocOnce source
(https://github.com/hplgit/doconce/)
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="description" content="Week 47: Support Vector Machines">
<title>Week 47: Support Vector Machines</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<!-- not necessary
<link href="https://netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
-->
<style type="text/css">
/* Add scrollbar to dropdown menus in bootstrap navigation bar */
.dropdown-menu {
height: auto;
max-height: 400px;
overflow-x: hidden;
}
/* Adds an invisible element before each target to offset for the navigation
bar */
.anchor::before {
content:"";
display:block;
height:50px; /* fixed header height for style bootstrap */
margin:-50px 0 0; /* negative fixed header height */
}
</style>
</head>
<!-- tocinfo
{'highest level': 2,
'sections': [('Overview of week 46', 2, None, '___sec0'),
('Thursday', 2, None, '___sec1'),
('Friday', 2, None, '___sec2'),
('Workshop plan Friday (first lecture)', 2, None, '___sec3'),
('Support Vector Machines, overarching aims', 2, None, '___sec4'),
('Hyperplanes and all that', 2, None, '___sec5'),
('What is a hyperplane?', 2, None, '___sec6'),
('A $p$-dimensional space of features', 2, None, '___sec7'),
('The two-dimensional case', 2, None, '___sec8'),
('Getting into the details', 2, None, '___sec9'),
('First attempt at a minimization approach', 2, None, '___sec10'),
('Solving the equations', 2, None, '___sec11'),
('Code Example', 2, None, '___sec12'),
('Problems with the Simpler Approach', 2, None, '___sec13'),
('A better approach', 2, None, '___sec14'),
('A quick Reminder on Lagrangian Multipliers',
2,
None,
'___sec15'),
('Adding the Multiplier', 2, None, '___sec16'),
('Setting up the Problem', 2, None, '___sec17'),
('The problem to solve', 2, None, '___sec18'),
('The last steps', 2, None, '___sec19'),
('A soft classifier', 2, None, '___sec20'),
('Soft optmization problem', 2, None, '___sec21'),
('Kernels and non-linearity', 2, None, '___sec22'),
('The equations', 2, None, '___sec23'),
('The problem to solve', 2, None, '___sec24'),
("Different kernels and Mercer's theorem", 2, None, '___sec25'),
('The moons example', 2, None, '___sec26'),
('Mathematical optimization of convex functions',
2,
None,
'___sec27'),
('How do we solve these problems?', 2, None, '___sec28'),
('A simple example', 2, None, '___sec29'),
('Back to the more realistic cases', 2, None, '___sec30')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "none" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-responsive-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="week47-bs.html">Week 47: Support Vector Machines</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._week47-bs001.html#___sec0" style="font-size: 80%;">Overview of week 46</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs002.html#___sec1" style="font-size: 80%;">Thursday</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs003.html#___sec2" style="font-size: 80%;">Friday</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs004.html#___sec3" style="font-size: 80%;">Workshop plan Friday (first lecture)</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs005.html#___sec4" style="font-size: 80%;">Support Vector Machines, overarching aims</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs006.html#___sec5" style="font-size: 80%;">Hyperplanes and all that</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs007.html#___sec6" style="font-size: 80%;">What is a hyperplane?</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs008.html#___sec7" style="font-size: 80%;">A \( p \)-dimensional space of features</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs009.html#___sec8" style="font-size: 80%;">The two-dimensional case</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs010.html#___sec9" style="font-size: 80%;">Getting into the details</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs011.html#___sec10" style="font-size: 80%;">First attempt at a minimization approach</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs012.html#___sec11" style="font-size: 80%;">Solving the equations</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs013.html#___sec12" style="font-size: 80%;">Code Example</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs014.html#___sec13" style="font-size: 80%;">Problems with the Simpler Approach</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs015.html#___sec14" style="font-size: 80%;">A better approach</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs016.html#___sec15" style="font-size: 80%;">A quick Reminder on Lagrangian Multipliers</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs017.html#___sec16" style="font-size: 80%;">Adding the Multiplier</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs018.html#___sec17" style="font-size: 80%;">Setting up the Problem</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs019.html#___sec18" style="font-size: 80%;">The problem to solve</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs020.html#___sec19" style="font-size: 80%;">The last steps</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs021.html#___sec20" style="font-size: 80%;">A soft classifier</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs022.html#___sec21" style="font-size: 80%;">Soft optmization problem</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs023.html#___sec22" style="font-size: 80%;">Kernels and non-linearity</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs024.html#___sec23" style="font-size: 80%;">The equations</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs025.html#___sec24" style="font-size: 80%;">The problem to solve</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs026.html#___sec25" style="font-size: 80%;">Different kernels and Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs027.html#___sec26" style="font-size: 80%;">The moons example</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs028.html#___sec27" style="font-size: 80%;">Mathematical optimization of convex functions</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs029.html#___sec28" style="font-size: 80%;">How do we solve these problems?</a></li>
<!-- navigation toc: --> <li><a href="#___sec29" style="font-size: 80%;">A simple example</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs031.html#___sec30" style="font-size: 80%;">Back to the more realistic cases</a></li>
</ul>
</li>
</ul>
</div>
</div>
</div> <!-- end of navigation bar -->
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<a name="part0030"></a>
<!-- !split -->
<h2 id="___sec29" class="anchor">A simple example </h2>
<p>
We remind ourselves about the general problem we want to solve
$$
\begin{align*}
&\mathrm{min}_{x}\hspace{0.2cm} \frac{1}{2}\boldsymbol{x}^T\boldsymbol{P}\boldsymbol{x}+\boldsymbol{q}^T\boldsymbol{x},\\ \nonumber
&\mathrm{subject\hspace{0.1cm} to} \hspace{0.2cm} \boldsymbol{G}\boldsymbol{x} \preceq \boldsymbol{h} \wedge \boldsymbol{A}\boldsymbol{x}=f.
\end{align*}
$$
<p>
Let us show how to perform the optmization using a simple case. Assume we want to optimize the following problem
$$
\begin{align*}
&\mathrm{min}_{x}\hspace{0.2cm} \frac{1}{2}x^2+5x+3y \\ \nonumber
&\mathrm{subject to} \\ \nonumber
&x, y \geq 0 \\ \nonumber
&x+3y \geq 15 \\ \nonumber
&2x+5y \leq 100 \\ \nonumber
&3x+4y \leq 80. \\ \nonumber
\end{align*}
$$
The minimization problem can be rewritten in terms of vectors and matrices as (with \( x \) and \( y \) being the unknowns)
$$
\frac{1}{2}\begin{bmatrix} x\\ y \end{bmatrix}^T \begin{bmatrix} 1 & 0\\ 0 & 0 \end{bmatrix} \begin{bmatrix} x \\ y \end{bmatrix} + \begin{bmatrix}3\\ 4 \end{bmatrix}^T \begin{bmatrix}x \\ y \end{bmatrix}.
$$
Similarly, we can now set up the inequalities (we need to change \( \geq \) to \( \leq \) by multiplying with \( -1 \) on bot sides) as the following matrix-vector equation
$$
\begin{bmatrix} -1 & 0 \\ 0 & -1 \\ -1 & -3 \\ 2 & 5 \\ 3 & 4\end{bmatrix}\begin{bmatrix} x \\ y\end{bmatrix} \preceq \begin{bmatrix}0 \\ 0\\ -15 \\ 100 \\ 80\end{bmatrix}.
$$
We have collapsed all the inequalities into a single matrix \( \boldsymbol{G} \). We see also that our matrix
$$
\boldsymbol{P} =\begin{bmatrix} 1 & 0\\ 0 & 0 \end{bmatrix}
$$
is clearly positive semi-definite (all eigenvalues larger or equal zero).
Finally, the vector \( \boldsymbol{h} \) is defined as
$$
\boldsymbol{h} = \begin{bmatrix}0 \\ 0\\ -15 \\ 100 \\ 80\end{bmatrix}.
$$
<p>
Since we don't have any equalities the matrix \( \boldsymbol{A} \) is set to zero
The following code solves the equations for us
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #408080; font-style: italic"># Import the necessary packages</span>
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">numpy</span>
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">cvxopt</span> <span style="color: #008000; font-weight: bold">import</span> matrix
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">cvxopt</span> <span style="color: #008000; font-weight: bold">import</span> solvers
P <span style="color: #666666">=</span> matrix(numpy<span style="color: #666666">.</span>diag([<span style="color: #666666">1</span>,<span style="color: #666666">0</span>]), tc<span style="color: #666666">=</span>’d’)
q <span style="color: #666666">=</span> matrix(numpy<span style="color: #666666">.</span>array([<span style="color: #666666">3</span>,<span style="color: #666666">4</span>]), tc<span style="color: #666666">=</span>’d’)
G <span style="color: #666666">=</span> matrix(numpy<span style="color: #666666">.</span>array([[<span style="color: #666666">-1</span>,<span style="color: #666666">0</span>],[<span style="color: #666666">0</span>,<span style="color: #666666">-1</span>],[<span style="color: #666666">-1</span>,<span style="color: #666666">-3</span>],[<span style="color: #666666">2</span>,<span style="color: #666666">5</span>],[<span style="color: #666666">3</span>,<span style="color: #666666">4</span>]]), tc<span style="color: #666666">=</span>’d’)
h <span style="color: #666666">=</span> matrix(numpy<span style="color: #666666">.</span>array([<span style="color: #666666">0</span>,<span style="color: #666666">0</span>,<span style="color: #666666">-15</span>,<span style="color: #666666">100</span>,<span style="color: #666666">80</span>]), tc<span style="color: #666666">=</span>’d’)
<span style="color: #408080; font-style: italic"># Construct the QP, invoke solver</span>
sol <span style="color: #666666">=</span> solvers<span style="color: #666666">.</span>qp(P,q,G,h)
<span style="color: #408080; font-style: italic"># Extract optimal value and solution</span>
sol[’x’]
sol[’primal objective’]
</pre></div>
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
<li><a href="._week47-bs029.html">&laquo;</a></li>
<li><a href="._week47-bs000.html">1</a></li>
<li><a href="">...</a></li>
<li><a href="._week47-bs022.html">23</a></li>
<li><a href="._week47-bs023.html">24</a></li>
<li><a href="._week47-bs024.html">25</a></li>
<li><a href="._week47-bs025.html">26</a></li>
<li><a href="._week47-bs026.html">27</a></li>
<li><a href="._week47-bs027.html">28</a></li>
<li><a href="._week47-bs028.html">29</a></li>
<li><a href="._week47-bs029.html">30</a></li>
<li class="active"><a href="._week47-bs030.html">31</a></li>
<li><a href="._week47-bs031.html">32</a></li>
<li><a href="._week47-bs031.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
</div> <!-- end container -->
<!-- include javascript, jQuery *first* -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script src="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/js/bootstrap.min.js"></script>
<!-- Bootstrap footer
<footer>
<a href="http://..."><img width="250" align=right src="http://..."></a>
</footer>
-->
<center style="font-size:80%">
<!-- copyright only on the titlepage -->
</center>
</body>
</html>
doc/pub/week47/html/._week47-bs031.html
+221
View File
@@ -0,0 +1,221 @@
<!--
Automatically generated HTML file from DocOnce source
(https://github.com/hplgit/doconce/)
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="description" content="Week 47: Support Vector Machines">
<title>Week 47: Support Vector Machines</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<!-- not necessary
<link href="https://netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
-->
<style type="text/css">
/* Add scrollbar to dropdown menus in bootstrap navigation bar */
.dropdown-menu {
height: auto;
max-height: 400px;
overflow-x: hidden;
}
/* Adds an invisible element before each target to offset for the navigation
bar */
.anchor::before {
content:"";
display:block;
height:50px; /* fixed header height for style bootstrap */
margin:-50px 0 0; /* negative fixed header height */
}
</style>
</head>
<!-- tocinfo
{'highest level': 2,
'sections': [('Overview of week 46', 2, None, '___sec0'),
('Thursday', 2, None, '___sec1'),
('Friday', 2, None, '___sec2'),
('Workshop plan Friday (first lecture)', 2, None, '___sec3'),
('Support Vector Machines, overarching aims', 2, None, '___sec4'),
('Hyperplanes and all that', 2, None, '___sec5'),
('What is a hyperplane?', 2, None, '___sec6'),
('A $p$-dimensional space of features', 2, None, '___sec7'),
('The two-dimensional case', 2, None, '___sec8'),
('Getting into the details', 2, None, '___sec9'),
('First attempt at a minimization approach', 2, None, '___sec10'),
('Solving the equations', 2, None, '___sec11'),
('Code Example', 2, None, '___sec12'),
('Problems with the Simpler Approach', 2, None, '___sec13'),
('A better approach', 2, None, '___sec14'),
('A quick Reminder on Lagrangian Multipliers',
2,
None,
'___sec15'),
('Adding the Multiplier', 2, None, '___sec16'),
('Setting up the Problem', 2, None, '___sec17'),
('The problem to solve', 2, None, '___sec18'),
('The last steps', 2, None, '___sec19'),
('A soft classifier', 2, None, '___sec20'),
('Soft optmization problem', 2, None, '___sec21'),
('Kernels and non-linearity', 2, None, '___sec22'),
('The equations', 2, None, '___sec23'),
('The problem to solve', 2, None, '___sec24'),
("Different kernels and Mercer's theorem", 2, None, '___sec25'),
('The moons example', 2, None, '___sec26'),
('Mathematical optimization of convex functions',
2,
None,
'___sec27'),
('How do we solve these problems?', 2, None, '___sec28'),
('A simple example', 2, None, '___sec29'),
('Back to the more realistic cases', 2, None, '___sec30')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "none" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-responsive-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="week47-bs.html">Week 47: Support Vector Machines</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._week47-bs001.html#___sec0" style="font-size: 80%;">Overview of week 46</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs002.html#___sec1" style="font-size: 80%;">Thursday</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs003.html#___sec2" style="font-size: 80%;">Friday</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs004.html#___sec3" style="font-size: 80%;">Workshop plan Friday (first lecture)</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs005.html#___sec4" style="font-size: 80%;">Support Vector Machines, overarching aims</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs006.html#___sec5" style="font-size: 80%;">Hyperplanes and all that</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs007.html#___sec6" style="font-size: 80%;">What is a hyperplane?</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs008.html#___sec7" style="font-size: 80%;">A \( p \)-dimensional space of features</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs009.html#___sec8" style="font-size: 80%;">The two-dimensional case</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs010.html#___sec9" style="font-size: 80%;">Getting into the details</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs011.html#___sec10" style="font-size: 80%;">First attempt at a minimization approach</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs012.html#___sec11" style="font-size: 80%;">Solving the equations</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs013.html#___sec12" style="font-size: 80%;">Code Example</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs014.html#___sec13" style="font-size: 80%;">Problems with the Simpler Approach</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs015.html#___sec14" style="font-size: 80%;">A better approach</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs016.html#___sec15" style="font-size: 80%;">A quick Reminder on Lagrangian Multipliers</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs017.html#___sec16" style="font-size: 80%;">Adding the Multiplier</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs018.html#___sec17" style="font-size: 80%;">Setting up the Problem</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs019.html#___sec18" style="font-size: 80%;">The problem to solve</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs020.html#___sec19" style="font-size: 80%;">The last steps</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs021.html#___sec20" style="font-size: 80%;">A soft classifier</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs022.html#___sec21" style="font-size: 80%;">Soft optmization problem</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs023.html#___sec22" style="font-size: 80%;">Kernels and non-linearity</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs024.html#___sec23" style="font-size: 80%;">The equations</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs025.html#___sec24" style="font-size: 80%;">The problem to solve</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs026.html#___sec25" style="font-size: 80%;">Different kernels and Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs027.html#___sec26" style="font-size: 80%;">The moons example</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs028.html#___sec27" style="font-size: 80%;">Mathematical optimization of convex functions</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs029.html#___sec28" style="font-size: 80%;">How do we solve these problems?</a></li>
<!-- navigation toc: --> <li><a href="._week47-bs030.html#___sec29" style="font-size: 80%;">A simple example</a></li>
<!-- navigation toc: --> <li><a href="#___sec30" style="font-size: 80%;">Back to the more realistic cases</a></li>
</ul>
</li>
</ul>
</div>
</div>
</div> <!-- end of navigation bar -->
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<a name="part0031"></a>
<!-- !split -->
<h2 id="___sec30" class="anchor">Back to the more realistic cases </h2>
<p>
We are now ready to return to our setup of the optmization problem for a more realistic case. Introducing the <b>slack</b> parameter \( C \) we have
$$
\frac{1}{2} \boldsymbol{\lambda}^T\begin{bmatrix} y_1y_1K(\boldsymbol{x}_1,\boldsymbol{x}_1) & y_1y_2K(\boldsymbol{x}_1,\boldsymbol{x}_2) & \dots & \dots & y_1y_nK(\boldsymbol{x}_1,\boldsymbol{x}_n) \\
y_2y_1K(\boldsymbol{x}_2,\boldsymbol{x}_1) & y_2y_2K(\boldsymbol{x}_2,\boldsymbol{x}_2) & \dots & \dots & y_1y_nK(\boldsymbol{x}_2,\boldsymbol{x}_n) \\
\dots & \dots & \dots & \dots & \dots \\
\dots & \dots & \dots & \dots & \dots \\
y_ny_1K(\boldsymbol{x}_n,\boldsymbol{x}_1) & y_ny_2K(\boldsymbol{x}_n\boldsymbol{x}_2) & \dots & \dots & y_ny_nK(\boldsymbol{x}_n,\boldsymbol{x}_n) \\
\end{bmatrix}\boldsymbol{\lambda}-\mathbb{I}\boldsymbol{\lambda},
$$
subject to \( \boldsymbol{y}^T\boldsymbol{\lambda}=0 \). Here we defined the vectors \( \boldsymbol{\lambda} =[\lambda_1,\lambda_2,\dots,\lambda_n] \) and
\( \boldsymbol{y}=[y_1,y_2,\dots,y_n] \).
With the slack constants this leads to the additional constraint \( 0\leq \lambda_i \leq C \).
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
<li><a href="._week47-bs030.html">&laquo;</a></li>
<li><a href="._week47-bs000.html">1</a></li>
<li><a href="">...</a></li>
<li><a href="._week47-bs023.html">24</a></li>
<li><a href="._week47-bs024.html">25</a></li>
<li><a href="._week47-bs025.html">26</a></li>
<li><a href="._week47-bs026.html">27</a></li>
<li><a href="._week47-bs027.html">28</a></li>
<li><a href="._week47-bs028.html">29</a></li>
<li><a href="._week47-bs029.html">30</a></li>
<li><a href="._week47-bs030.html">31</a></li>
<li class="active"><a href="._week47-bs031.html">32</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
</div> <!-- end container -->
<!-- include javascript, jQuery *first* -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script src="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/js/bootstrap.min.js"></script>
<!-- Bootstrap footer
<footer>
<a href="http://..."><img width="250" align=right src="http://..."></a>
</footer>
-->
<center style="font-size:80%">
<!-- copyright only on the titlepage -->
</center>
</body>
</html>