lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

writing code for sim using cvxopt

Browse Source
This commit is contained in:
mhjensen
2018-11-05 22:40:28 +01:00
parent 3fe132f4ce
commit 0fce698a63
31 changed files with 1890 additions and 31 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/pub/svm/html/._svm-bs000.html
+12 -2
View File
@@ -59,7 +59,12 @@ Automatically generated HTML file from DocOnce source
('The last steps', 2, None, '___sec13'),
('A soft classifier', 2, None, '___sec14'),
('Soft optmization problem', 2, None, '___sec15'),
('Kernels and non-linearity', 2, None, '___sec16')]}
('Kernels and non-linearity', 2, None, '___sec16'),
('The equations', 2, None, '___sec17'),
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
end of tocinfo -->
<body>
@@ -114,6 +119,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._svm-bs015.html#___sec14" style="font-size: 80%;">A soft classifier</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs016.html#___sec15" style="font-size: 80%;">Soft optmization problem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs017.html#___sec16" style="font-size: 80%;">Kernels and non-linearity</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs018.html#___sec17" style="font-size: 80%;">The equations</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs019.html#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs020.html#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs021.html#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
@@ -172,7 +182,7 @@ MathJax.Hub.Config({
<li><a href="._svm-bs008.html">9</a></li>
<li><a href="._svm-bs009.html">10</a></li>
<li><a href="">...</a></li>
<li><a href="._svm-bs017.html">18</a></li>
<li><a href="._svm-bs022.html">23</a></li>
<li><a href="._svm-bs001.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/svm/html/._svm-bs001.html
+12 -2
View File
@@ -59,7 +59,12 @@ Automatically generated HTML file from DocOnce source
('The last steps', 2, None, '___sec13'),
('A soft classifier', 2, None, '___sec14'),
('Soft optmization problem', 2, None, '___sec15'),
('Kernels and non-linearity', 2, None, '___sec16')]}
('Kernels and non-linearity', 2, None, '___sec16'),
('The equations', 2, None, '___sec17'),
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
end of tocinfo -->
<body>
@@ -114,6 +119,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._svm-bs015.html#___sec14" style="font-size: 80%;">A soft classifier</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs016.html#___sec15" style="font-size: 80%;">Soft optmization problem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs017.html#___sec16" style="font-size: 80%;">Kernels and non-linearity</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs018.html#___sec17" style="font-size: 80%;">The equations</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs019.html#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs020.html#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs021.html#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
@@ -171,7 +181,7 @@ We distinguish also between linear and non-linear approaches. The latter are the
<li><a href="._svm-bs009.html">10</a></li>
<li><a href="._svm-bs010.html">11</a></li>
<li><a href="">...</a></li>
<li><a href="._svm-bs017.html">18</a></li>
<li><a href="._svm-bs022.html">23</a></li>
<li><a href="._svm-bs002.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/svm/html/._svm-bs002.html
+12 -2
View File
@@ -59,7 +59,12 @@ Automatically generated HTML file from DocOnce source
('The last steps', 2, None, '___sec13'),
('A soft classifier', 2, None, '___sec14'),
('Soft optmization problem', 2, None, '___sec15'),
('Kernels and non-linearity', 2, None, '___sec16')]}
('Kernels and non-linearity', 2, None, '___sec16'),
('The equations', 2, None, '___sec17'),
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
end of tocinfo -->
<body>
@@ -114,6 +119,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._svm-bs015.html#___sec14" style="font-size: 80%;">A soft classifier</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs016.html#___sec15" style="font-size: 80%;">Soft optmization problem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs017.html#___sec16" style="font-size: 80%;">Kernels and non-linearity</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs018.html#___sec17" style="font-size: 80%;">The equations</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs019.html#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs020.html#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs021.html#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
@@ -162,7 +172,7 @@ circles.
<li><a href="._svm-bs010.html">11</a></li>
<li><a href="._svm-bs011.html">12</a></li>
<li><a href="">...</a></li>
<li><a href="._svm-bs017.html">18</a></li>
<li><a href="._svm-bs022.html">23</a></li>
<li><a href="._svm-bs003.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/svm/html/._svm-bs003.html
+12 -2
View File
@@ -59,7 +59,12 @@ Automatically generated HTML file from DocOnce source
('The last steps', 2, None, '___sec13'),
('A soft classifier', 2, None, '___sec14'),
('Soft optmization problem', 2, None, '___sec15'),
('Kernels and non-linearity', 2, None, '___sec16')]}
('Kernels and non-linearity', 2, None, '___sec16'),
('The equations', 2, None, '___sec17'),
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
end of tocinfo -->
<body>
@@ -114,6 +119,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._svm-bs015.html#___sec14" style="font-size: 80%;">A soft classifier</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs016.html#___sec15" style="font-size: 80%;">Soft optmization problem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs017.html#___sec16" style="font-size: 80%;">Kernels and non-linearity</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs018.html#___sec17" style="font-size: 80%;">The equations</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs019.html#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs020.html#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs021.html#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
@@ -172,7 +182,7 @@ $$
<li><a href="._svm-bs011.html">12</a></li>
<li><a href="._svm-bs012.html">13</a></li>
<li><a href="">...</a></li>
<li><a href="._svm-bs017.html">18</a></li>
<li><a href="._svm-bs022.html">23</a></li>
<li><a href="._svm-bs004.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/svm/html/._svm-bs004.html
+12 -2
View File
@@ -59,7 +59,12 @@ Automatically generated HTML file from DocOnce source
('The last steps', 2, None, '___sec13'),
('A soft classifier', 2, None, '___sec14'),
('Soft optmization problem', 2, None, '___sec15'),
('Kernels and non-linearity', 2, None, '___sec16')]}
('Kernels and non-linearity', 2, None, '___sec16'),
('The equations', 2, None, '___sec17'),
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
end of tocinfo -->
<body>
@@ -114,6 +119,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._svm-bs015.html#___sec14" style="font-size: 80%;">A soft classifier</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs016.html#___sec15" style="font-size: 80%;">Soft optmization problem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs017.html#___sec16" style="font-size: 80%;">Kernels and non-linearity</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs018.html#___sec17" style="font-size: 80%;">The equations</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs019.html#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs020.html#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs021.html#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
@@ -188,7 +198,7 @@ When we try to separate hyperplanes, if it exists, we can use it to construct a
<li><a href="._svm-bs012.html">13</a></li>
<li><a href="._svm-bs013.html">14</a></li>
<li><a href="">...</a></li>
<li><a href="._svm-bs017.html">18</a></li>
<li><a href="._svm-bs022.html">23</a></li>
<li><a href="._svm-bs005.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/svm/html/._svm-bs005.html
+12 -2
View File
@@ -59,7 +59,12 @@ Automatically generated HTML file from DocOnce source
('The last steps', 2, None, '___sec13'),
('A soft classifier', 2, None, '___sec14'),
('Soft optmization problem', 2, None, '___sec15'),
('Kernels and non-linearity', 2, None, '___sec16')]}
('Kernels and non-linearity', 2, None, '___sec16'),
('The equations', 2, None, '___sec17'),
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
end of tocinfo -->
<body>
@@ -114,6 +119,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._svm-bs015.html#___sec14" style="font-size: 80%;">A soft classifier</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs016.html#___sec15" style="font-size: 80%;">Soft optmization problem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs017.html#___sec16" style="font-size: 80%;">Kernels and non-linearity</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs018.html#___sec17" style="font-size: 80%;">The equations</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs019.html#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs020.html#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs021.html#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
@@ -174,7 +184,7 @@ for our data sample.
<li><a href="._svm-bs013.html">14</a></li>
<li><a href="._svm-bs014.html">15</a></li>
<li><a href="">...</a></li>
<li><a href="._svm-bs017.html">18</a></li>
<li><a href="._svm-bs022.html">23</a></li>
<li><a href="._svm-bs006.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/svm/html/._svm-bs006.html
+12 -2
View File
@@ -59,7 +59,12 @@ Automatically generated HTML file from DocOnce source
('The last steps', 2, None, '___sec13'),
('A soft classifier', 2, None, '___sec14'),
('Soft optmization problem', 2, None, '___sec15'),
('Kernels and non-linearity', 2, None, '___sec16')]}
('Kernels and non-linearity', 2, None, '___sec16'),
('The equations', 2, None, '___sec17'),
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
end of tocinfo -->
<body>
@@ -114,6 +119,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._svm-bs015.html#___sec14" style="font-size: 80%;">A soft classifier</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs016.html#___sec15" style="font-size: 80%;">Soft optmization problem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs017.html#___sec16" style="font-size: 80%;">Kernels and non-linearity</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs018.html#___sec17" style="font-size: 80%;">The equations</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs019.html#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs020.html#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs021.html#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
@@ -170,7 +180,7 @@ $$
<li><a href="._svm-bs014.html">15</a></li>
<li><a href="._svm-bs015.html">16</a></li>
<li><a href="">...</a></li>
<li><a href="._svm-bs017.html">18</a></li>
<li><a href="._svm-bs022.html">23</a></li>
<li><a href="._svm-bs007.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/svm/html/._svm-bs007.html
+12 -2
View File
@@ -59,7 +59,12 @@ Automatically generated HTML file from DocOnce source
('The last steps', 2, None, '___sec13'),
('A soft classifier', 2, None, '___sec14'),
('Soft optmization problem', 2, None, '___sec15'),
('Kernels and non-linearity', 2, None, '___sec16')]}
('Kernels and non-linearity', 2, None, '___sec16'),
('The equations', 2, None, '___sec17'),
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
end of tocinfo -->
<body>
@@ -114,6 +119,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._svm-bs015.html#___sec14" style="font-size: 80%;">A soft classifier</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs016.html#___sec15" style="font-size: 80%;">Soft optmization problem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs017.html#___sec16" style="font-size: 80%;">Kernels and non-linearity</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs018.html#___sec17" style="font-size: 80%;">The equations</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs019.html#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs020.html#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs021.html#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
@@ -174,7 +184,7 @@ $$
<li><a href="._svm-bs015.html">16</a></li>
<li><a href="._svm-bs016.html">17</a></li>
<li><a href="">...</a></li>
<li><a href="._svm-bs017.html">18</a></li>
<li><a href="._svm-bs022.html">23</a></li>
<li><a href="._svm-bs008.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/svm/html/._svm-bs008.html
+13 -1
View File
@@ -59,7 +59,12 @@ Automatically generated HTML file from DocOnce source
('The last steps', 2, None, '___sec13'),
('A soft classifier', 2, None, '___sec14'),
('Soft optmization problem', 2, None, '___sec15'),
('Kernels and non-linearity', 2, None, '___sec16')]}
('Kernels and non-linearity', 2, None, '___sec16'),
('The equations', 2, None, '___sec17'),
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
end of tocinfo -->
<body>
@@ -114,6 +119,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._svm-bs015.html#___sec14" style="font-size: 80%;">A soft classifier</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs016.html#___sec15" style="font-size: 80%;">Soft optmization problem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs017.html#___sec16" style="font-size: 80%;">Kernels and non-linearity</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs018.html#___sec17" style="font-size: 80%;">The equations</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs019.html#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs020.html#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs021.html#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
@@ -177,6 +187,8 @@ at all.
<li><a href="._svm-bs015.html">16</a></li>
<li><a href="._svm-bs016.html">17</a></li>
<li><a href="._svm-bs017.html">18</a></li>
<li><a href="">...</a></li>
<li><a href="._svm-bs022.html">23</a></li>
<li><a href="._svm-bs009.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/svm/html/._svm-bs009.html
+14 -1
View File
@@ -59,7 +59,12 @@ Automatically generated HTML file from DocOnce source
('The last steps', 2, None, '___sec13'),
('A soft classifier', 2, None, '___sec14'),
('Soft optmization problem', 2, None, '___sec15'),
('Kernels and non-linearity', 2, None, '___sec16')]}
('Kernels and non-linearity', 2, None, '___sec16'),
('The equations', 2, None, '___sec17'),
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
end of tocinfo -->
<body>
@@ -114,6 +119,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._svm-bs015.html#___sec14" style="font-size: 80%;">A soft classifier</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs016.html#___sec15" style="font-size: 80%;">Soft optmization problem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs017.html#___sec16" style="font-size: 80%;">Kernels and non-linearity</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs018.html#___sec17" style="font-size: 80%;">The equations</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs019.html#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs020.html#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs021.html#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
@@ -188,6 +198,9 @@ We have thus defined our margin as the invers of the norm of \( \boldsymbol{w} \
<li><a href="._svm-bs015.html">16</a></li>
<li><a href="._svm-bs016.html">17</a></li>
<li><a href="._svm-bs017.html">18</a></li>
<li><a href="._svm-bs018.html">19</a></li>
<li><a href="">...</a></li>
<li><a href="._svm-bs022.html">23</a></li>
<li><a href="._svm-bs010.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/svm/html/._svm-bs010.html
+15 -1
View File
@@ -59,7 +59,12 @@ Automatically generated HTML file from DocOnce source
('The last steps', 2, None, '___sec13'),
('A soft classifier', 2, None, '___sec14'),
('Soft optmization problem', 2, None, '___sec15'),
('Kernels and non-linearity', 2, None, '___sec16')]}
('Kernels and non-linearity', 2, None, '___sec16'),
('The equations', 2, None, '___sec17'),
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
end of tocinfo -->
<body>
@@ -114,6 +119,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._svm-bs015.html#___sec14" style="font-size: 80%;">A soft classifier</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs016.html#___sec15" style="font-size: 80%;">Soft optmization problem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs017.html#___sec16" style="font-size: 80%;">Kernels and non-linearity</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs018.html#___sec17" style="font-size: 80%;">The equations</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs019.html#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs020.html#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs021.html#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
@@ -199,6 +209,10 @@ Then \( dz \) is no longer arbitrary.
<li><a href="._svm-bs015.html">16</a></li>
<li><a href="._svm-bs016.html">17</a></li>
<li><a href="._svm-bs017.html">18</a></li>
<li><a href="._svm-bs018.html">19</a></li>
<li><a href="._svm-bs019.html">20</a></li>
<li><a href="">...</a></li>
<li><a href="._svm-bs022.html">23</a></li>
<li><a href="._svm-bs011.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/svm/html/._svm-bs011.html
+16 -1
View File
@@ -59,7 +59,12 @@ Automatically generated HTML file from DocOnce source
('The last steps', 2, None, '___sec13'),
('A soft classifier', 2, None, '___sec14'),
('Soft optmization problem', 2, None, '___sec15'),
('Kernels and non-linearity', 2, None, '___sec16')]}
('Kernels and non-linearity', 2, None, '___sec16'),
('The equations', 2, None, '___sec17'),
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
end of tocinfo -->
<body>
@@ -114,6 +119,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._svm-bs015.html#___sec14" style="font-size: 80%;">A soft classifier</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs016.html#___sec15" style="font-size: 80%;">Soft optmization problem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs017.html#___sec16" style="font-size: 80%;">Kernels and non-linearity</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs018.html#___sec17" style="font-size: 80%;">The equations</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs019.html#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs020.html#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs021.html#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
@@ -191,6 +201,11 @@ $$
<li><a href="._svm-bs015.html">16</a></li>
<li><a href="._svm-bs016.html">17</a></li>
<li><a href="._svm-bs017.html">18</a></li>
<li><a href="._svm-bs018.html">19</a></li>
<li><a href="._svm-bs019.html">20</a></li>
<li><a href="._svm-bs020.html">21</a></li>
<li><a href="">...</a></li>
<li><a href="._svm-bs022.html">23</a></li>
<li><a href="._svm-bs012.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/svm/html/._svm-bs012.html
+17 -1
View File
@@ -59,7 +59,12 @@ Automatically generated HTML file from DocOnce source
('The last steps', 2, None, '___sec13'),
('A soft classifier', 2, None, '___sec14'),
('Soft optmization problem', 2, None, '___sec15'),
('Kernels and non-linearity', 2, None, '___sec16')]}
('Kernels and non-linearity', 2, None, '___sec16'),
('The equations', 2, None, '___sec17'),
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
end of tocinfo -->
<body>
@@ -114,6 +119,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._svm-bs015.html#___sec14" style="font-size: 80%;">A soft classifier</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs016.html#___sec15" style="font-size: 80%;">Soft optmization problem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs017.html#___sec16" style="font-size: 80%;">Kernels and non-linearity</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs018.html#___sec17" style="font-size: 80%;">The equations</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs019.html#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs020.html#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs021.html#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
@@ -188,6 +198,12 @@ When \( \lambda_i > 0 \), the vectors \( \boldsymbol{x}_i \) are called support
<li><a href="._svm-bs015.html">16</a></li>
<li><a href="._svm-bs016.html">17</a></li>
<li><a href="._svm-bs017.html">18</a></li>
<li><a href="._svm-bs018.html">19</a></li>
<li><a href="._svm-bs019.html">20</a></li>
<li><a href="._svm-bs020.html">21</a></li>
<li><a href="._svm-bs021.html">22</a></li>
<li><a href="">...</a></li>
<li><a href="._svm-bs022.html">23</a></li>
<li><a href="._svm-bs013.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/svm/html/._svm-bs013.html
+16 -1
View File
@@ -59,7 +59,12 @@ Automatically generated HTML file from DocOnce source
('The last steps', 2, None, '___sec13'),
('A soft classifier', 2, None, '___sec14'),
('Soft optmization problem', 2, None, '___sec15'),
('Kernels and non-linearity', 2, None, '___sec16')]}
('Kernels and non-linearity', 2, None, '___sec16'),
('The equations', 2, None, '___sec17'),
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
end of tocinfo -->
<body>
@@ -114,6 +119,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._svm-bs015.html#___sec14" style="font-size: 80%;">A soft classifier</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs016.html#___sec15" style="font-size: 80%;">Soft optmization problem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs017.html#___sec16" style="font-size: 80%;">Kernels and non-linearity</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs018.html#___sec17" style="font-size: 80%;">The equations</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs019.html#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs020.html#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs021.html#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
@@ -170,6 +180,11 @@ subject to \( \boldsymbol{y}^T\boldsymbol{\lambda}=0 \). Here we defined the vec
<li><a href="._svm-bs015.html">16</a></li>
<li><a href="._svm-bs016.html">17</a></li>
<li><a href="._svm-bs017.html">18</a></li>
<li><a href="._svm-bs018.html">19</a></li>
<li><a href="._svm-bs019.html">20</a></li>
<li><a href="._svm-bs020.html">21</a></li>
<li><a href="._svm-bs021.html">22</a></li>
<li><a href="._svm-bs022.html">23</a></li>
<li><a href="._svm-bs014.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/svm/html/._svm-bs014.html
+16 -1
View File
@@ -59,7 +59,12 @@ Automatically generated HTML file from DocOnce source
('The last steps', 2, None, '___sec13'),
('A soft classifier', 2, None, '___sec14'),
('Soft optmization problem', 2, None, '___sec15'),
('Kernels and non-linearity', 2, None, '___sec16')]}
('Kernels and non-linearity', 2, None, '___sec16'),
('The equations', 2, None, '___sec17'),
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
end of tocinfo -->
<body>
@@ -114,6 +119,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._svm-bs015.html#___sec14" style="font-size: 80%;">A soft classifier</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs016.html#___sec15" style="font-size: 80%;">Soft optmization problem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs017.html#___sec16" style="font-size: 80%;">Kernels and non-linearity</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs018.html#___sec17" style="font-size: 80%;">The equations</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs019.html#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs020.html#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs021.html#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
@@ -179,6 +189,11 @@ Below we discuss how to find the optimal values of \( \lambda_i \). Before we pr
<li><a href="._svm-bs015.html">16</a></li>
<li><a href="._svm-bs016.html">17</a></li>
<li><a href="._svm-bs017.html">18</a></li>
<li><a href="._svm-bs018.html">19</a></li>
<li><a href="._svm-bs019.html">20</a></li>
<li><a href="._svm-bs020.html">21</a></li>
<li><a href="._svm-bs021.html">22</a></li>
<li><a href="._svm-bs022.html">23</a></li>
<li><a href="._svm-bs015.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/svm/html/._svm-bs015.html
+16 -1
View File
@@ -59,7 +59,12 @@ Automatically generated HTML file from DocOnce source
('The last steps', 2, None, '___sec13'),
('A soft classifier', 2, None, '___sec14'),
('Soft optmization problem', 2, None, '___sec15'),
('Kernels and non-linearity', 2, None, '___sec16')]}
('Kernels and non-linearity', 2, None, '___sec16'),
('The equations', 2, None, '___sec17'),
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
end of tocinfo -->
<body>
@@ -114,6 +119,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="#___sec14" style="font-size: 80%;">A soft classifier</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs016.html#___sec15" style="font-size: 80%;">Soft optmization problem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs017.html#___sec16" style="font-size: 80%;">Kernels and non-linearity</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs018.html#___sec17" style="font-size: 80%;">The equations</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs019.html#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs020.html#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs021.html#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
@@ -179,6 +189,11 @@ misclassifications.
<li class="active"><a href="._svm-bs015.html">16</a></li>
<li><a href="._svm-bs016.html">17</a></li>
<li><a href="._svm-bs017.html">18</a></li>
<li><a href="._svm-bs018.html">19</a></li>
<li><a href="._svm-bs019.html">20</a></li>
<li><a href="._svm-bs020.html">21</a></li>
<li><a href="._svm-bs021.html">22</a></li>
<li><a href="._svm-bs022.html">23</a></li>
<li><a href="._svm-bs016.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/svm/html/._svm-bs016.html
+16 -1
View File
@@ -59,7 +59,12 @@ Automatically generated HTML file from DocOnce source
('The last steps', 2, None, '___sec13'),
('A soft classifier', 2, None, '___sec14'),
('Soft optmization problem', 2, None, '___sec15'),
('Kernels and non-linearity', 2, None, '___sec16')]}
('Kernels and non-linearity', 2, None, '___sec16'),
('The equations', 2, None, '___sec17'),
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
end of tocinfo -->
<body>
@@ -114,6 +119,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._svm-bs015.html#___sec14" style="font-size: 80%;">A soft classifier</a></li>
<!-- navigation toc: --> <li><a href="#___sec15" style="font-size: 80%;">Soft optmization problem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs017.html#___sec16" style="font-size: 80%;">Kernels and non-linearity</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs018.html#___sec17" style="font-size: 80%;">The equations</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs019.html#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs020.html#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs021.html#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
@@ -197,6 +207,11 @@ $$
<li><a href="._svm-bs015.html">16</a></li>
<li class="active"><a href="._svm-bs016.html">17</a></li>
<li><a href="._svm-bs017.html">18</a></li>
<li><a href="._svm-bs018.html">19</a></li>
<li><a href="._svm-bs019.html">20</a></li>
<li><a href="._svm-bs020.html">21</a></li>
<li><a href="._svm-bs021.html">22</a></li>
<li><a href="._svm-bs022.html">23</a></li>
<li><a href="._svm-bs017.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/svm/html/._svm-bs017.html
+36 -1
View File
@@ -59,7 +59,12 @@ Automatically generated HTML file from DocOnce source
('The last steps', 2, None, '___sec13'),
('A soft classifier', 2, None, '___sec14'),
('Soft optmization problem', 2, None, '___sec15'),
('Kernels and non-linearity', 2, None, '___sec16')]}
('Kernels and non-linearity', 2, None, '___sec16'),
('The equations', 2, None, '___sec17'),
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
end of tocinfo -->
<body>
@@ -114,6 +119,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._svm-bs015.html#___sec14" style="font-size: 80%;">A soft classifier</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs016.html#___sec15" style="font-size: 80%;">Soft optmization problem</a></li>
<!-- navigation toc: --> <li><a href="#___sec16" style="font-size: 80%;">Kernels and non-linearity</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs018.html#___sec17" style="font-size: 80%;">The equations</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs019.html#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs020.html#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs021.html#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
@@ -131,6 +141,25 @@ MathJax.Hub.Config({
<h2 id="___sec16" class="anchor">Kernels and non-linearity </h2>
<p>
The cases we have studied till were all characterized by two classes
with a close to linear separability. The classifiers we have described
so far find linear boundaries in our input feature space. It is
possible to make our procedure more flexible by exploring the feature
space using other basis expansions such higher-order polynomials,
wavelets, splines etc.
<p>
If our feature space is not easy to separate, as shown in the figure
here, we can achieve a better separation by introducing more complex
basis functions. The ideal would be, as shown in the next figure, to, via a specific transformation to
obtain a separation between the classes which is almost linear.
<p>
The change of basis, from \( x\rightarrow z=\phi(x) \) leads to the same type of equations to be solved, except that
we need to introduce for example a polynomial transformation to a two-dimensional training set.
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
@@ -146,6 +175,12 @@ MathJax.Hub.Config({
<li><a href="._svm-bs015.html">16</a></li>
<li><a href="._svm-bs016.html">17</a></li>
<li class="active"><a href="._svm-bs017.html">18</a></li>
<li><a href="._svm-bs018.html">19</a></li>
<li><a href="._svm-bs019.html">20</a></li>
<li><a href="._svm-bs020.html">21</a></li>
<li><a href="._svm-bs021.html">22</a></li>
<li><a href="._svm-bs022.html">23</a></li>
<li><a href="._svm-bs018.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/svm/html/._svm-bs018.html
+207
View File
@@ -0,0 +1,207 @@
<!--
Automatically generated HTML file from DocOnce source
(https://github.com/hplgit/doconce/)
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="description" content="Data Analysis and Machine Learning: Support Vector Machines">
<title>Data Analysis and Machine Learning: Support Vector Machines</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<!-- not necessary
<link href="https://netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
-->
<style type="text/css">
/* Add scrollbar to dropdown menus in bootstrap navigation bar */
.dropdown-menu {
height: auto;
max-height: 400px;
overflow-x: hidden;
}
/* Adds an invisible element before each target to offset for the navigation
bar */
.anchor::before {
content:"";
display:block;
height:50px; /* fixed header height for style bootstrap */
margin:-50px 0 0; /* negative fixed header height */
}
</style>
</head>
<!-- tocinfo
{'highest level': 2,
'sections': [('Support Vector Machines, overarching aims', 2, None, '___sec0'),
('Hyperplanes and all that', 2, None, '___sec1'),
('What is a hyperplane?', 2, None, '___sec2'),
('A $p$-dimensional space of features', 2, None, '___sec3'),
('The two-dimensional case', 2, None, '___sec4'),
('Getting into the details', 2, None, '___sec5'),
('First attempt at a minimization approach', 2, None, '___sec6'),
('Solving the equations', 2, None, '___sec7'),
('A better approach', 2, None, '___sec8'),
('A quick reminder on Lagrangian multipliers',
2,
None,
'___sec9'),
('Adding the muliplier', 2, None, '___sec10'),
('Setting up the problem', 2, None, '___sec11'),
('The problem to solve', 2, None, '___sec12'),
('The last steps', 2, None, '___sec13'),
('A soft classifier', 2, None, '___sec14'),
('Soft optmization problem', 2, None, '___sec15'),
('Kernels and non-linearity', 2, None, '___sec16'),
('The equations', 2, None, '___sec17'),
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "none" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-responsive-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="svm-bs.html">Data Analysis and Machine Learning: Support Vector Machines</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._svm-bs001.html#___sec0" style="font-size: 80%;">Support Vector Machines, overarching aims</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs002.html#___sec1" style="font-size: 80%;">Hyperplanes and all that</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs003.html#___sec2" style="font-size: 80%;">What is a hyperplane?</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs004.html#___sec3" style="font-size: 80%;">A \( p \)-dimensional space of features</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs005.html#___sec4" style="font-size: 80%;">The two-dimensional case</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs006.html#___sec5" style="font-size: 80%;">Getting into the details</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs007.html#___sec6" style="font-size: 80%;">First attempt at a minimization approach</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs008.html#___sec7" style="font-size: 80%;">Solving the equations</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs009.html#___sec8" style="font-size: 80%;">A better approach</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs010.html#___sec9" style="font-size: 80%;">A quick reminder on Lagrangian multipliers</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs011.html#___sec10" style="font-size: 80%;">Adding the muliplier</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs012.html#___sec11" style="font-size: 80%;">Setting up the problem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs013.html#___sec12" style="font-size: 80%;">The problem to solve</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs014.html#___sec13" style="font-size: 80%;">The last steps</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs015.html#___sec14" style="font-size: 80%;">A soft classifier</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs016.html#___sec15" style="font-size: 80%;">Soft optmization problem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs017.html#___sec16" style="font-size: 80%;">Kernels and non-linearity</a></li>
<!-- navigation toc: --> <li><a href="#___sec17" style="font-size: 80%;">The equations</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs019.html#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs020.html#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs021.html#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
</ul>
</div>
</div>
</div> <!-- end of navigation bar -->
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<a name="part0018"></a>
<!-- !split -->
<h2 id="___sec17" class="anchor">The equations </h2>
<p>
Suppose we define a polynomial transformation of degree two (we continue to live in a plane with \( x_1 \) and \( x_2 \) as variables)
$$
z = \phi(x) =\left(1, x_1, x_2, x_1^2, x_2^2, x_1x_2).
$$
<p>
With our new basis, the equations we solved earlier are basically the same, that is we have now (without the slack option for simplicity)
$$
{\cal L}=\sum_i\lambda_i-\frac{1}{2}\sum_{ij}^n\lambda_i\lambda_jy_iy_j\boldsymbol{z}_i^T\boldsymbol{Z}_j,
$$
subject to the constraints \( \lambda_i\geq 0 \), \( \sum_i\lambda_iy_i=0 \), and for the support vectors
$$
y_i(\boldsymbol{w}^T\boldsymbol{z}_i+b)= 1 \hspace{0.1cm}\forall i,
$$
from which we also find \( b \).
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
<li><a href="._svm-bs017.html">&laquo;</a></li>
<li><a href="._svm-bs000.html">1</a></li>
<li><a href="">...</a></li>
<li><a href="._svm-bs010.html">11</a></li>
<li><a href="._svm-bs011.html">12</a></li>
<li><a href="._svm-bs012.html">13</a></li>
<li><a href="._svm-bs013.html">14</a></li>
<li><a href="._svm-bs014.html">15</a></li>
<li><a href="._svm-bs015.html">16</a></li>
<li><a href="._svm-bs016.html">17</a></li>
<li><a href="._svm-bs017.html">18</a></li>
<li class="active"><a href="._svm-bs018.html">19</a></li>
<li><a href="._svm-bs019.html">20</a></li>
<li><a href="._svm-bs020.html">21</a></li>
<li><a href="._svm-bs021.html">22</a></li>
<li><a href="._svm-bs022.html">23</a></li>
<li><a href="._svm-bs019.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
</div> <!-- end container -->
<!-- include javascript, jQuery *first* -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script src="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/js/bootstrap.min.js"></script>
<!-- Bootstrap footer
<footer>
<a href="http://..."><img width="250" align=right src="http://..."></a>
</footer>
-->
<center style="font-size:80%">
<!-- copyright only on the titlepage -->
</center>
</body>
</html>
doc/pub/svm/html/._svm-bs019.html
+187
View File
@@ -0,0 +1,187 @@
<!--
Automatically generated HTML file from DocOnce source
(https://github.com/hplgit/doconce/)
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="description" content="Data Analysis and Machine Learning: Support Vector Machines">
<title>Data Analysis and Machine Learning: Support Vector Machines</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<!-- not necessary
<link href="https://netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
-->
<style type="text/css">
/* Add scrollbar to dropdown menus in bootstrap navigation bar */
.dropdown-menu {
height: auto;
max-height: 400px;
overflow-x: hidden;
}
/* Adds an invisible element before each target to offset for the navigation
bar */
.anchor::before {
content:"";
display:block;
height:50px; /* fixed header height for style bootstrap */
margin:-50px 0 0; /* negative fixed header height */
}
</style>
</head>
<!-- tocinfo
{'highest level': 2,
'sections': [('Support Vector Machines, overarching aims', 2, None, '___sec0'),
('Hyperplanes and all that', 2, None, '___sec1'),
('What is a hyperplane?', 2, None, '___sec2'),
('A $p$-dimensional space of features', 2, None, '___sec3'),
('The two-dimensional case', 2, None, '___sec4'),
('Getting into the details', 2, None, '___sec5'),
('First attempt at a minimization approach', 2, None, '___sec6'),
('Solving the equations', 2, None, '___sec7'),
('A better approach', 2, None, '___sec8'),
('A quick reminder on Lagrangian multipliers',
2,
None,
'___sec9'),
('Adding the muliplier', 2, None, '___sec10'),
('Setting up the problem', 2, None, '___sec11'),
('The problem to solve', 2, None, '___sec12'),
('The last steps', 2, None, '___sec13'),
('A soft classifier', 2, None, '___sec14'),
('Soft optmization problem', 2, None, '___sec15'),
('Kernels and non-linearity', 2, None, '___sec16'),
('The equations', 2, None, '___sec17'),
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "none" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-responsive-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="svm-bs.html">Data Analysis and Machine Learning: Support Vector Machines</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._svm-bs001.html#___sec0" style="font-size: 80%;">Support Vector Machines, overarching aims</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs002.html#___sec1" style="font-size: 80%;">Hyperplanes and all that</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs003.html#___sec2" style="font-size: 80%;">What is a hyperplane?</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs004.html#___sec3" style="font-size: 80%;">A \( p \)-dimensional space of features</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs005.html#___sec4" style="font-size: 80%;">The two-dimensional case</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs006.html#___sec5" style="font-size: 80%;">Getting into the details</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs007.html#___sec6" style="font-size: 80%;">First attempt at a minimization approach</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs008.html#___sec7" style="font-size: 80%;">Solving the equations</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs009.html#___sec8" style="font-size: 80%;">A better approach</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs010.html#___sec9" style="font-size: 80%;">A quick reminder on Lagrangian multipliers</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs011.html#___sec10" style="font-size: 80%;">Adding the muliplier</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs012.html#___sec11" style="font-size: 80%;">Setting up the problem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs013.html#___sec12" style="font-size: 80%;">The problem to solve</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs014.html#___sec13" style="font-size: 80%;">The last steps</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs015.html#___sec14" style="font-size: 80%;">A soft classifier</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs016.html#___sec15" style="font-size: 80%;">Soft optmization problem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs017.html#___sec16" style="font-size: 80%;">Kernels and non-linearity</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs018.html#___sec17" style="font-size: 80%;">The equations</a></li>
<!-- navigation toc: --> <li><a href="#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs020.html#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs021.html#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
</ul>
</div>
</div>
</div> <!-- end of navigation bar -->
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<a name="part0019"></a>
<!-- !split -->
<h2 id="___sec18" class="anchor">Different kernels </h2>
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
<li><a href="._svm-bs018.html">&laquo;</a></li>
<li><a href="._svm-bs000.html">1</a></li>
<li><a href="">...</a></li>
<li><a href="._svm-bs011.html">12</a></li>
<li><a href="._svm-bs012.html">13</a></li>
<li><a href="._svm-bs013.html">14</a></li>
<li><a href="._svm-bs014.html">15</a></li>
<li><a href="._svm-bs015.html">16</a></li>
<li><a href="._svm-bs016.html">17</a></li>
<li><a href="._svm-bs017.html">18</a></li>
<li><a href="._svm-bs018.html">19</a></li>
<li class="active"><a href="._svm-bs019.html">20</a></li>
<li><a href="._svm-bs020.html">21</a></li>
<li><a href="._svm-bs021.html">22</a></li>
<li><a href="._svm-bs022.html">23</a></li>
<li><a href="._svm-bs020.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
</div> <!-- end container -->
<!-- include javascript, jQuery *first* -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script src="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/js/bootstrap.min.js"></script>
<!-- Bootstrap footer
<footer>
<a href="http://..."><img width="250" align=right src="http://..."></a>
</footer>
-->
<center style="font-size:80%">
<!-- copyright only on the titlepage -->
</center>
</body>
</html>
doc/pub/svm/html/._svm-bs020.html
+186
View File
@@ -0,0 +1,186 @@
<!--
Automatically generated HTML file from DocOnce source
(https://github.com/hplgit/doconce/)
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="description" content="Data Analysis and Machine Learning: Support Vector Machines">
<title>Data Analysis and Machine Learning: Support Vector Machines</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<!-- not necessary
<link href="https://netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
-->
<style type="text/css">
/* Add scrollbar to dropdown menus in bootstrap navigation bar */
.dropdown-menu {
height: auto;
max-height: 400px;
overflow-x: hidden;
}
/* Adds an invisible element before each target to offset for the navigation
bar */
.anchor::before {
content:"";
display:block;
height:50px; /* fixed header height for style bootstrap */
margin:-50px 0 0; /* negative fixed header height */
}
</style>
</head>
<!-- tocinfo
{'highest level': 2,
'sections': [('Support Vector Machines, overarching aims', 2, None, '___sec0'),
('Hyperplanes and all that', 2, None, '___sec1'),
('What is a hyperplane?', 2, None, '___sec2'),
('A $p$-dimensional space of features', 2, None, '___sec3'),
('The two-dimensional case', 2, None, '___sec4'),
('Getting into the details', 2, None, '___sec5'),
('First attempt at a minimization approach', 2, None, '___sec6'),
('Solving the equations', 2, None, '___sec7'),
('A better approach', 2, None, '___sec8'),
('A quick reminder on Lagrangian multipliers',
2,
None,
'___sec9'),
('Adding the muliplier', 2, None, '___sec10'),
('Setting up the problem', 2, None, '___sec11'),
('The problem to solve', 2, None, '___sec12'),
('The last steps', 2, None, '___sec13'),
('A soft classifier', 2, None, '___sec14'),
('Soft optmization problem', 2, None, '___sec15'),
('Kernels and non-linearity', 2, None, '___sec16'),
('The equations', 2, None, '___sec17'),
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "none" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-responsive-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="svm-bs.html">Data Analysis and Machine Learning: Support Vector Machines</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._svm-bs001.html#___sec0" style="font-size: 80%;">Support Vector Machines, overarching aims</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs002.html#___sec1" style="font-size: 80%;">Hyperplanes and all that</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs003.html#___sec2" style="font-size: 80%;">What is a hyperplane?</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs004.html#___sec3" style="font-size: 80%;">A \( p \)-dimensional space of features</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs005.html#___sec4" style="font-size: 80%;">The two-dimensional case</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs006.html#___sec5" style="font-size: 80%;">Getting into the details</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs007.html#___sec6" style="font-size: 80%;">First attempt at a minimization approach</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs008.html#___sec7" style="font-size: 80%;">Solving the equations</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs009.html#___sec8" style="font-size: 80%;">A better approach</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs010.html#___sec9" style="font-size: 80%;">A quick reminder on Lagrangian multipliers</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs011.html#___sec10" style="font-size: 80%;">Adding the muliplier</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs012.html#___sec11" style="font-size: 80%;">Setting up the problem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs013.html#___sec12" style="font-size: 80%;">The problem to solve</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs014.html#___sec13" style="font-size: 80%;">The last steps</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs015.html#___sec14" style="font-size: 80%;">A soft classifier</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs016.html#___sec15" style="font-size: 80%;">Soft optmization problem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs017.html#___sec16" style="font-size: 80%;">Kernels and non-linearity</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs018.html#___sec17" style="font-size: 80%;">The equations</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs019.html#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs021.html#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
</ul>
</div>
</div>
</div> <!-- end of navigation bar -->
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<a name="part0020"></a>
<!-- !split -->
<h2 id="___sec19" class="anchor">Quadratic coefficient matrix </h2>
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
<li><a href="._svm-bs019.html">&laquo;</a></li>
<li><a href="._svm-bs000.html">1</a></li>
<li><a href="">...</a></li>
<li><a href="._svm-bs012.html">13</a></li>
<li><a href="._svm-bs013.html">14</a></li>
<li><a href="._svm-bs014.html">15</a></li>
<li><a href="._svm-bs015.html">16</a></li>
<li><a href="._svm-bs016.html">17</a></li>
<li><a href="._svm-bs017.html">18</a></li>
<li><a href="._svm-bs018.html">19</a></li>
<li><a href="._svm-bs019.html">20</a></li>
<li class="active"><a href="._svm-bs020.html">21</a></li>
<li><a href="._svm-bs021.html">22</a></li>
<li><a href="._svm-bs022.html">23</a></li>
<li><a href="._svm-bs021.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
</div> <!-- end container -->
<!-- include javascript, jQuery *first* -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script src="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/js/bootstrap.min.js"></script>
<!-- Bootstrap footer
<footer>
<a href="http://..."><img width="250" align=right src="http://..."></a>
</footer>
-->
<center style="font-size:80%">
<!-- copyright only on the titlepage -->
</center>
</body>
</html>
doc/pub/svm/html/._svm-bs021.html
+185
View File
@@ -0,0 +1,185 @@
<!--
Automatically generated HTML file from DocOnce source
(https://github.com/hplgit/doconce/)
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="description" content="Data Analysis and Machine Learning: Support Vector Machines">
<title>Data Analysis and Machine Learning: Support Vector Machines</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<!-- not necessary
<link href="https://netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
-->
<style type="text/css">
/* Add scrollbar to dropdown menus in bootstrap navigation bar */
.dropdown-menu {
height: auto;
max-height: 400px;
overflow-x: hidden;
}
/* Adds an invisible element before each target to offset for the navigation
bar */
.anchor::before {
content:"";
display:block;
height:50px; /* fixed header height for style bootstrap */
margin:-50px 0 0; /* negative fixed header height */
}
</style>
</head>
<!-- tocinfo
{'highest level': 2,
'sections': [('Support Vector Machines, overarching aims', 2, None, '___sec0'),
('Hyperplanes and all that', 2, None, '___sec1'),
('What is a hyperplane?', 2, None, '___sec2'),
('A $p$-dimensional space of features', 2, None, '___sec3'),
('The two-dimensional case', 2, None, '___sec4'),
('Getting into the details', 2, None, '___sec5'),
('First attempt at a minimization approach', 2, None, '___sec6'),
('Solving the equations', 2, None, '___sec7'),
('A better approach', 2, None, '___sec8'),
('A quick reminder on Lagrangian multipliers',
2,
None,
'___sec9'),
('Adding the muliplier', 2, None, '___sec10'),
('Setting up the problem', 2, None, '___sec11'),
('The problem to solve', 2, None, '___sec12'),
('The last steps', 2, None, '___sec13'),
('A soft classifier', 2, None, '___sec14'),
('Soft optmization problem', 2, None, '___sec15'),
('Kernels and non-linearity', 2, None, '___sec16'),
('The equations', 2, None, '___sec17'),
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "none" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-responsive-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="svm-bs.html">Data Analysis and Machine Learning: Support Vector Machines</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._svm-bs001.html#___sec0" style="font-size: 80%;">Support Vector Machines, overarching aims</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs002.html#___sec1" style="font-size: 80%;">Hyperplanes and all that</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs003.html#___sec2" style="font-size: 80%;">What is a hyperplane?</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs004.html#___sec3" style="font-size: 80%;">A \( p \)-dimensional space of features</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs005.html#___sec4" style="font-size: 80%;">The two-dimensional case</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs006.html#___sec5" style="font-size: 80%;">Getting into the details</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs007.html#___sec6" style="font-size: 80%;">First attempt at a minimization approach</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs008.html#___sec7" style="font-size: 80%;">Solving the equations</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs009.html#___sec8" style="font-size: 80%;">A better approach</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs010.html#___sec9" style="font-size: 80%;">A quick reminder on Lagrangian multipliers</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs011.html#___sec10" style="font-size: 80%;">Adding the muliplier</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs012.html#___sec11" style="font-size: 80%;">Setting up the problem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs013.html#___sec12" style="font-size: 80%;">The problem to solve</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs014.html#___sec13" style="font-size: 80%;">The last steps</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs015.html#___sec14" style="font-size: 80%;">A soft classifier</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs016.html#___sec15" style="font-size: 80%;">Soft optmization problem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs017.html#___sec16" style="font-size: 80%;">Kernels and non-linearity</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs018.html#___sec17" style="font-size: 80%;">The equations</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs019.html#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs020.html#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
</ul>
</div>
</div>
</div> <!-- end of navigation bar -->
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<a name="part0021"></a>
<!-- !split -->
<h2 id="___sec20" class="anchor">Mercer's theorem </h2>
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
<li><a href="._svm-bs020.html">&laquo;</a></li>
<li><a href="._svm-bs000.html">1</a></li>
<li><a href="">...</a></li>
<li><a href="._svm-bs013.html">14</a></li>
<li><a href="._svm-bs014.html">15</a></li>
<li><a href="._svm-bs015.html">16</a></li>
<li><a href="._svm-bs016.html">17</a></li>
<li><a href="._svm-bs017.html">18</a></li>
<li><a href="._svm-bs018.html">19</a></li>
<li><a href="._svm-bs019.html">20</a></li>
<li><a href="._svm-bs020.html">21</a></li>
<li class="active"><a href="._svm-bs021.html">22</a></li>
<li><a href="._svm-bs022.html">23</a></li>
<li><a href="._svm-bs022.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
</div> <!-- end container -->
<!-- include javascript, jQuery *first* -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script src="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/js/bootstrap.min.js"></script>
<!-- Bootstrap footer
<footer>
<a href="http://..."><img width="250" align=right src="http://..."></a>
</footer>
-->
<center style="font-size:80%">
<!-- copyright only on the titlepage -->
</center>
</body>
</html>
doc/pub/svm/html/._svm-bs022.html
+234
View File
@@ -0,0 +1,234 @@
<!--
Automatically generated HTML file from DocOnce source
(https://github.com/hplgit/doconce/)
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="description" content="Data Analysis and Machine Learning: Support Vector Machines">
<title>Data Analysis and Machine Learning: Support Vector Machines</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<!-- not necessary
<link href="https://netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
-->
<style type="text/css">
/* Add scrollbar to dropdown menus in bootstrap navigation bar */
.dropdown-menu {
height: auto;
max-height: 400px;
overflow-x: hidden;
}
/* Adds an invisible element before each target to offset for the navigation
bar */
.anchor::before {
content:"";
display:block;
height:50px; /* fixed header height for style bootstrap */
margin:-50px 0 0; /* negative fixed header height */
}
</style>
</head>
<!-- tocinfo
{'highest level': 2,
'sections': [('Support Vector Machines, overarching aims', 2, None, '___sec0'),
('Hyperplanes and all that', 2, None, '___sec1'),
('What is a hyperplane?', 2, None, '___sec2'),
('A $p$-dimensional space of features', 2, None, '___sec3'),
('The two-dimensional case', 2, None, '___sec4'),
('Getting into the details', 2, None, '___sec5'),
('First attempt at a minimization approach', 2, None, '___sec6'),
('Solving the equations', 2, None, '___sec7'),
('A better approach', 2, None, '___sec8'),
('A quick reminder on Lagrangian multipliers',
2,
None,
'___sec9'),
('Adding the muliplier', 2, None, '___sec10'),
('Setting up the problem', 2, None, '___sec11'),
('The problem to solve', 2, None, '___sec12'),
('The last steps', 2, None, '___sec13'),
('A soft classifier', 2, None, '___sec14'),
('Soft optmization problem', 2, None, '___sec15'),
('Kernels and non-linearity', 2, None, '___sec16'),
('The equations', 2, None, '___sec17'),
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "none" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-responsive-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="svm-bs.html">Data Analysis and Machine Learning: Support Vector Machines</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._svm-bs001.html#___sec0" style="font-size: 80%;">Support Vector Machines, overarching aims</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs002.html#___sec1" style="font-size: 80%;">Hyperplanes and all that</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs003.html#___sec2" style="font-size: 80%;">What is a hyperplane?</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs004.html#___sec3" style="font-size: 80%;">A \( p \)-dimensional space of features</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs005.html#___sec4" style="font-size: 80%;">The two-dimensional case</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs006.html#___sec5" style="font-size: 80%;">Getting into the details</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs007.html#___sec6" style="font-size: 80%;">First attempt at a minimization approach</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs008.html#___sec7" style="font-size: 80%;">Solving the equations</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs009.html#___sec8" style="font-size: 80%;">A better approach</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs010.html#___sec9" style="font-size: 80%;">A quick reminder on Lagrangian multipliers</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs011.html#___sec10" style="font-size: 80%;">Adding the muliplier</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs012.html#___sec11" style="font-size: 80%;">Setting up the problem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs013.html#___sec12" style="font-size: 80%;">The problem to solve</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs014.html#___sec13" style="font-size: 80%;">The last steps</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs015.html#___sec14" style="font-size: 80%;">A soft classifier</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs016.html#___sec15" style="font-size: 80%;">Soft optmization problem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs017.html#___sec16" style="font-size: 80%;">Kernels and non-linearity</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs018.html#___sec17" style="font-size: 80%;">The equations</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs019.html#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs020.html#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs021.html#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
</ul>
</div>
</div>
</div> <!-- end of navigation bar -->
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<a name="part0022"></a>
<!-- !split -->
<h2 id="___sec21" class="anchor">How do we solve these problems </h2>
<p>
If we use Python as programming language and wish to venture beyond
<b>scikit-learn</b>, <b>tensorflow</b> and similar software which makes our
lives so much easier, we need to dive into the wonderful world of
quadratic programming. We can, if we wish, solve the minimization
problem using say standard gradient methods or conjugate gradient
methods. However, these methods tend to exhibit a rather slow
converge. So, welcome to the promised land of quadratic programming.
<p>
The functions we need are contained in the quadratic programming package <b>CVXOPT</b> and we need to import it
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">numpy</span>
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">cvxopt</span>
</pre></div>
<p>
Let us first set up the standard form the of quadratic programming (QP) equations by defining the problem as
$$
\mathrm{min}
$$
<p>
subject to Gx u. Note that x itself is not provided to the solver, since it is an internal
variable being optimized over. In particular, this means that the solver has no explicit knowledge
of x itself; everything is implicity defined by the supplied parameters. It is essential
that the same variable order is maintained for the relevant parameters (e.g., qi
Non-convexity implies the existence of local optima, making it difficult to find global optima.
<p>
collapsed all inequality constraints into a single G matrix of the standard form.
Since there are no equality constraints, we do not need to provide the empty A, b. Note
that even though y
2 did not appear in the original objective, we had to include it with zero
coefficients in P because the solver parameters must be defined using the full set of variables.
Even if certain variables only appear in constraints, they will still need to be expressed with
zero coefficients in the objective parameters, and vice versa.
Let us first define the above parameters in Python. CVXOPT supplies its own matrix
object; all arguments given to its solvers must be in this matrix type. There are two ways
to do this. The first is to define the matrix directly with (potentially nested) lists:
from cvxopt import matrix
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span>P <span style="color: #666666">=</span> matrix([[<span style="color: #666666">1.0</span>,<span style="color: #666666">0.0</span>],[<span style="color: #666666">0.0</span>,<span style="color: #666666">0.0</span>]])
q <span style="color: #666666">=</span> matrix([<span style="color: #666666">3.0</span>,<span style="color: #666666">4.0</span>])
G <span style="color: #666666">=</span> matrix([[<span style="color: #666666">-1.0</span>,<span style="color: #666666">0.0</span>,<span style="color: #666666">-1.0</span>,<span style="color: #666666">2.0</span>,<span style="color: #666666">3.0</span>],[<span style="color: #666666">0.0</span>,<span style="color: #666666">-1.0</span>,<span style="color: #666666">-3.0</span>,<span style="color: #666666">5.0</span>,<span style="color: #666666">4.0</span>]])
h <span style="color: #666666">=</span> matrix([<span style="color: #666666">0.0</span>,<span style="color: #666666">0.0</span>,<span style="color: #666666">-15.0</span>,<span style="color: #666666">100.0</span>,<span style="color: #666666">80.0</span>])
</pre></div>
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
<li><a href="._svm-bs021.html">&laquo;</a></li>
<li><a href="._svm-bs000.html">1</a></li>
<li><a href="">...</a></li>
<li><a href="._svm-bs014.html">15</a></li>
<li><a href="._svm-bs015.html">16</a></li>
<li><a href="._svm-bs016.html">17</a></li>
<li><a href="._svm-bs017.html">18</a></li>
<li><a href="._svm-bs018.html">19</a></li>
<li><a href="._svm-bs019.html">20</a></li>
<li><a href="._svm-bs020.html">21</a></li>
<li><a href="._svm-bs021.html">22</a></li>
<li class="active"><a href="._svm-bs022.html">23</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
</div> <!-- end container -->
<!-- include javascript, jQuery *first* -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script src="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/js/bootstrap.min.js"></script>
<!-- Bootstrap footer
<footer>
<a href="http://..."><img width="250" align=right src="http://..."></a>
</footer>
-->
<center style="font-size:80%">
<!-- copyright only on the titlepage -->
</center>
</body>
</html>
doc/pub/svm/html/svm-bs.html
+12 -2
View File
@@ -59,7 +59,12 @@ Automatically generated HTML file from DocOnce source
('The last steps', 2, None, '___sec13'),
('A soft classifier', 2, None, '___sec14'),
('Soft optmization problem', 2, None, '___sec15'),
('Kernels and non-linearity', 2, None, '___sec16')]}
('Kernels and non-linearity', 2, None, '___sec16'),
('The equations', 2, None, '___sec17'),
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
end of tocinfo -->
<body>
@@ -114,6 +119,11 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._svm-bs015.html#___sec14" style="font-size: 80%;">A soft classifier</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs016.html#___sec15" style="font-size: 80%;">Soft optmization problem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs017.html#___sec16" style="font-size: 80%;">Kernels and non-linearity</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs018.html#___sec17" style="font-size: 80%;">The equations</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs019.html#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs020.html#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs021.html#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
@@ -172,7 +182,7 @@ MathJax.Hub.Config({
<li><a href="._svm-bs008.html">9</a></li>
<li><a href="._svm-bs009.html">10</a></li>
<li><a href="">...</a></li>
<li><a href="._svm-bs017.html">18</a></li>
<li><a href="._svm-bs022.html">23</a></li>
<li><a href="._svm-bs001.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/svm/html/svm-reveal.html
+121
View File
@@ -804,6 +804,127 @@ $$
<section>
<h2 id="___sec16">Kernels and non-linearity </h2>
<p>
The cases we have studied till were all characterized by two classes
with a close to linear separability. The classifiers we have described
so far find linear boundaries in our input feature space. It is
possible to make our procedure more flexible by exploring the feature
space using other basis expansions such higher-order polynomials,
wavelets, splines etc.
<p>
If our feature space is not easy to separate, as shown in the figure
here, we can achieve a better separation by introducing more complex
basis functions. The ideal would be, as shown in the next figure, to, via a specific transformation to
obtain a separation between the classes which is almost linear.
<p>
The change of basis, from \( x\rightarrow z=\phi(x) \) leads to the same type of equations to be solved, except that
we need to introduce for example a polynomial transformation to a two-dimensional training set.
</section>
<section>
<h2 id="___sec17">The equations </h2>
<p>
Suppose we define a polynomial transformation of degree two (we continue to live in a plane with \( x_1 \) and \( x_2 \) as variables)
<p>&nbsp;<br>
$$
z = \phi(x) =\left(1, x_1, x_2, x_1^2, x_2^2, x_1x_2).
$$
<p>&nbsp;<br>
<p>
With our new basis, the equations we solved earlier are basically the same, that is we have now (without the slack option for simplicity)
<p>&nbsp;<br>
$$
{\cal L}=\sum_i\lambda_i-\frac{1}{2}\sum_{ij}^n\lambda_i\lambda_jy_iy_j\boldsymbol{z}_i^T\boldsymbol{Z}_j,
$$
<p>&nbsp;<br>
subject to the constraints \( \lambda_i\geq 0 \), \( \sum_i\lambda_iy_i=0 \), and for the support vectors
<p>&nbsp;<br>
$$
y_i(\boldsymbol{w}^T\boldsymbol{z}_i+b)= 1 \hspace{0.1cm}\forall i,
$$
<p>&nbsp;<br>
from which we also find \( b \).
</section>
<section>
<h2 id="___sec18">Different kernels </h2>
</section>
<section>
<h2 id="___sec19">Quadratic coefficient matrix </h2>
</section>
<section>
<h2 id="___sec20">Mercer's theorem </h2>
</section>
<section>
<h2 id="___sec21">How do we solve these problems </h2>
<p>
If we use Python as programming language and wish to venture beyond
<b>scikit-learn</b>, <b>tensorflow</b> and similar software which makes our
lives so much easier, we need to dive into the wonderful world of
quadratic programming. We can, if we wish, solve the minimization
problem using say standard gradient methods or conjugate gradient
methods. However, these methods tend to exhibit a rather slow
converge. So, welcome to the promised land of quadratic programming.
<p>
The functions we need are contained in the quadratic programming package <b>CVXOPT</b> and we need to import it
<p>
<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
<div class="highlight" style="background: #eeeedd"><pre style="font-size: 80%; line-height: 125%"><span></span><span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">numpy</span>
<span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">cvxopt</span>
</pre></div>
<p>
Let us first set up the standard form the of quadratic programming (QP) equations by defining the problem as
<p>&nbsp;<br>
$$
\mathrm{min}
$$
<p>&nbsp;<br>
<p>
subject to Gx u. Note that x itself is not provided to the solver, since it is an internal
variable being optimized over. In particular, this means that the solver has no explicit knowledge
of x itself; everything is implicity defined by the supplied parameters. It is essential
that the same variable order is maintained for the relevant parameters (e.g., qi
Non-convexity implies the existence of local optima, making it difficult to find global optima.
<p>
collapsed all inequality constraints into a single G matrix of the standard form.
Since there are no equality constraints, we do not need to provide the empty A, b. Note
that even though y
2 did not appear in the original objective, we had to include it with zero
coefficients in P because the solver parameters must be defined using the full set of variables.
Even if certain variables only appear in constraints, they will still need to be expressed with
zero coefficients in the objective parameters, and vice versa.
Let us first define the above parameters in Python. CVXOPT supplies its own matrix
object; all arguments given to its solvers must be in this matrix type. There are two ways
to do this. The first is to define the matrix directly with (potentially nested) lists:
from cvxopt import matrix
<p>
<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
<div class="highlight" style="background: #eeeedd"><pre style="font-size: 80%; line-height: 125%"><span></span>P = matrix([[<span style="color: #B452CD">1.0</span>,<span style="color: #B452CD">0.0</span>],[<span style="color: #B452CD">0.0</span>,<span style="color: #B452CD">0.0</span>]])
q = matrix([<span style="color: #B452CD">3.0</span>,<span style="color: #B452CD">4.0</span>])
G = matrix([[-<span style="color: #B452CD">1.0</span>,<span style="color: #B452CD">0.0</span>,-<span style="color: #B452CD">1.0</span>,<span style="color: #B452CD">2.0</span>,<span style="color: #B452CD">3.0</span>],[<span style="color: #B452CD">0.0</span>,-<span style="color: #B452CD">1.0</span>,-<span style="color: #B452CD">3.0</span>,<span style="color: #B452CD">5.0</span>,<span style="color: #B452CD">4.0</span>]])
h = matrix([<span style="color: #B452CD">0.0</span>,<span style="color: #B452CD">0.0</span>,-<span style="color: #B452CD">15.0</span>,<span style="color: #B452CD">100.0</span>,<span style="color: #B452CD">80.0</span>])
</pre></div>
</section>
doc/pub/svm/html/svm-solarized.html
+120 -1
View File
@@ -53,7 +53,12 @@ div { text-align: justify; text-justify: inter-word; }
('The last steps', 2, None, '___sec13'),
('A soft classifier', 2, None, '___sec14'),
('Soft optmization problem', 2, None, '___sec15'),
('Kernels and non-linearity', 2, None, '___sec16')]}
('Kernels and non-linearity', 2, None, '___sec16'),
('The equations', 2, None, '___sec17'),
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
end of tocinfo -->
<body>
@@ -639,6 +644,120 @@ $$
<h2 id="___sec16">Kernels and non-linearity </h2>
<p>
The cases we have studied till were all characterized by two classes
with a close to linear separability. The classifiers we have described
so far find linear boundaries in our input feature space. It is
possible to make our procedure more flexible by exploring the feature
space using other basis expansions such higher-order polynomials,
wavelets, splines etc.
<p>
If our feature space is not easy to separate, as shown in the figure
here, we can achieve a better separation by introducing more complex
basis functions. The ideal would be, as shown in the next figure, to, via a specific transformation to
obtain a separation between the classes which is almost linear.
<p>
The change of basis, from \( x\rightarrow z=\phi(x) \) leads to the same type of equations to be solved, except that
we need to introduce for example a polynomial transformation to a two-dimensional training set.
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec17">The equations </h2>
<p>
Suppose we define a polynomial transformation of degree two (we continue to live in a plane with \( x_1 \) and \( x_2 \) as variables)
$$
z = \phi(x) =\left(1, x_1, x_2, x_1^2, x_2^2, x_1x_2).
$$
<p>
With our new basis, the equations we solved earlier are basically the same, that is we have now (without the slack option for simplicity)
$$
{\cal L}=\sum_i\lambda_i-\frac{1}{2}\sum_{ij}^n\lambda_i\lambda_jy_iy_j\boldsymbol{z}_i^T\boldsymbol{Z}_j,
$$
subject to the constraints \( \lambda_i\geq 0 \), \( \sum_i\lambda_iy_i=0 \), and for the support vectors
$$
y_i(\boldsymbol{w}^T\boldsymbol{z}_i+b)= 1 \hspace{0.1cm}\forall i,
$$
from which we also find \( b \).
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec18">Different kernels </h2>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec19">Quadratic coefficient matrix </h2>
<p>
<!-- !split -->
<h2 id="___sec20">Mercer's theorem </h2>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec21">How do we solve these problems </h2>
<p>
If we use Python as programming language and wish to venture beyond
<b>scikit-learn</b>, <b>tensorflow</b> and similar software which makes our
lives so much easier, we need to dive into the wonderful world of
quadratic programming. We can, if we wish, solve the minimization
problem using say standard gradient methods or conjugate gradient
methods. However, these methods tend to exhibit a rather slow
converge. So, welcome to the promised land of quadratic programming.
<p>
The functions we need are contained in the quadratic programming package <b>CVXOPT</b> and we need to import it
<p>
<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
<div class="highlight" style="background: #eeeedd"><pre style="line-height: 125%"><span></span><span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">numpy</span>
<span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">cvxopt</span>
</pre></div>
<p>
Let us first set up the standard form the of quadratic programming (QP) equations by defining the problem as
$$
\mathrm{min}
$$
<p>
subject to Gx u. Note that x itself is not provided to the solver, since it is an internal
variable being optimized over. In particular, this means that the solver has no explicit knowledge
of x itself; everything is implicity defined by the supplied parameters. It is essential
that the same variable order is maintained for the relevant parameters (e.g., qi
Non-convexity implies the existence of local optima, making it difficult to find global optima.
<p>
collapsed all inequality constraints into a single G matrix of the standard form.
Since there are no equality constraints, we do not need to provide the empty A, b. Note
that even though y
2 did not appear in the original objective, we had to include it with zero
coefficients in P because the solver parameters must be defined using the full set of variables.
Even if certain variables only appear in constraints, they will still need to be expressed with
zero coefficients in the objective parameters, and vice versa.
Let us first define the above parameters in Python. CVXOPT supplies its own matrix
object; all arguments given to its solvers must be in this matrix type. There are two ways
to do this. The first is to define the matrix directly with (potentially nested) lists:
from cvxopt import matrix
<p>
<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
<div class="highlight" style="background: #eeeedd"><pre style="line-height: 125%"><span></span>P = matrix([[<span style="color: #B452CD">1.0</span>,<span style="color: #B452CD">0.0</span>],[<span style="color: #B452CD">0.0</span>,<span style="color: #B452CD">0.0</span>]])
q = matrix([<span style="color: #B452CD">3.0</span>,<span style="color: #B452CD">4.0</span>])
G = matrix([[-<span style="color: #B452CD">1.0</span>,<span style="color: #B452CD">0.0</span>,-<span style="color: #B452CD">1.0</span>,<span style="color: #B452CD">2.0</span>,<span style="color: #B452CD">3.0</span>],[<span style="color: #B452CD">0.0</span>,-<span style="color: #B452CD">1.0</span>,-<span style="color: #B452CD">3.0</span>,<span style="color: #B452CD">5.0</span>,<span style="color: #B452CD">4.0</span>]])
h = matrix([<span style="color: #B452CD">0.0</span>,<span style="color: #B452CD">0.0</span>,-<span style="color: #B452CD">15.0</span>,<span style="color: #B452CD">100.0</span>,<span style="color: #B452CD">80.0</span>])
</pre></div>
<p>
<!-- ------------------- end of main content --------------- -->
doc/pub/svm/html/svm.html
+120 -1
View File
@@ -58,7 +58,12 @@ div { text-align: justify; text-justify: inter-word; }
('The last steps', 2, None, '___sec13'),
('A soft classifier', 2, None, '___sec14'),
('Soft optmization problem', 2, None, '___sec15'),
('Kernels and non-linearity', 2, None, '___sec16')]}
('Kernels and non-linearity', 2, None, '___sec16'),
('The equations', 2, None, '___sec17'),
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
end of tocinfo -->
<body>
@@ -644,6 +649,120 @@ $$
<h2 id="___sec16">Kernels and non-linearity </h2>
<p>
The cases we have studied till were all characterized by two classes
with a close to linear separability. The classifiers we have described
so far find linear boundaries in our input feature space. It is
possible to make our procedure more flexible by exploring the feature
space using other basis expansions such higher-order polynomials,
wavelets, splines etc.
<p>
If our feature space is not easy to separate, as shown in the figure
here, we can achieve a better separation by introducing more complex
basis functions. The ideal would be, as shown in the next figure, to, via a specific transformation to
obtain a separation between the classes which is almost linear.
<p>
The change of basis, from \( x\rightarrow z=\phi(x) \) leads to the same type of equations to be solved, except that
we need to introduce for example a polynomial transformation to a two-dimensional training set.
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec17">The equations </h2>
<p>
Suppose we define a polynomial transformation of degree two (we continue to live in a plane with \( x_1 \) and \( x_2 \) as variables)
$$
z = \phi(x) =\left(1, x_1, x_2, x_1^2, x_2^2, x_1x_2).
$$
<p>
With our new basis, the equations we solved earlier are basically the same, that is we have now (without the slack option for simplicity)
$$
{\cal L}=\sum_i\lambda_i-\frac{1}{2}\sum_{ij}^n\lambda_i\lambda_jy_iy_j\boldsymbol{z}_i^T\boldsymbol{Z}_j,
$$
subject to the constraints \( \lambda_i\geq 0 \), \( \sum_i\lambda_iy_i=0 \), and for the support vectors
$$
y_i(\boldsymbol{w}^T\boldsymbol{z}_i+b)= 1 \hspace{0.1cm}\forall i,
$$
from which we also find \( b \).
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec18">Different kernels </h2>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec19">Quadratic coefficient matrix </h2>
<p>
<!-- !split -->
<h2 id="___sec20">Mercer's theorem </h2>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec21">How do we solve these problems </h2>
<p>
If we use Python as programming language and wish to venture beyond
<b>scikit-learn</b>, <b>tensorflow</b> and similar software which makes our
lives so much easier, we need to dive into the wonderful world of
quadratic programming. We can, if we wish, solve the minimization
problem using say standard gradient methods or conjugate gradient
methods. However, these methods tend to exhibit a rather slow
converge. So, welcome to the promised land of quadratic programming.
<p>
The functions we need are contained in the quadratic programming package <b>CVXOPT</b> and we need to import it
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">numpy</span>
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">cvxopt</span>
</pre></div>
<p>
Let us first set up the standard form the of quadratic programming (QP) equations by defining the problem as
$$
\mathrm{min}
$$
<p>
subject to Gx u. Note that x itself is not provided to the solver, since it is an internal
variable being optimized over. In particular, this means that the solver has no explicit knowledge
of x itself; everything is implicity defined by the supplied parameters. It is essential
that the same variable order is maintained for the relevant parameters (e.g., qi
Non-convexity implies the existence of local optima, making it difficult to find global optima.
<p>
collapsed all inequality constraints into a single G matrix of the standard form.
Since there are no equality constraints, we do not need to provide the empty A, b. Note
that even though y
2 did not appear in the original objective, we had to include it with zero
coefficients in P because the solver parameters must be defined using the full set of variables.
Even if certain variables only appear in constraints, they will still need to be expressed with
zero coefficients in the objective parameters, and vice versa.
Let us first define the above parameters in Python. CVXOPT supplies its own matrix
object; all arguments given to its solvers must be in this matrix type. There are two ways
to do this. The first is to define the matrix directly with (potentially nested) lists:
from cvxopt import matrix
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span>P <span style="color: #666666">=</span> matrix([[<span style="color: #666666">1.0</span>,<span style="color: #666666">0.0</span>],[<span style="color: #666666">0.0</span>,<span style="color: #666666">0.0</span>]])
q <span style="color: #666666">=</span> matrix([<span style="color: #666666">3.0</span>,<span style="color: #666666">4.0</span>])
G <span style="color: #666666">=</span> matrix([[<span style="color: #666666">-1.0</span>,<span style="color: #666666">0.0</span>,<span style="color: #666666">-1.0</span>,<span style="color: #666666">2.0</span>,<span style="color: #666666">3.0</span>],[<span style="color: #666666">0.0</span>,<span style="color: #666666">-1.0</span>,<span style="color: #666666">-3.0</span>,<span style="color: #666666">5.0</span>,<span style="color: #666666">4.0</span>]])
h <span style="color: #666666">=</span> matrix([<span style="color: #666666">0.0</span>,<span style="color: #666666">0.0</span>,<span style="color: #666666">-15.0</span>,<span style="color: #666666">100.0</span>,<span style="color: #666666">80.0</span>])
</pre></div>
<p>
<!-- ------------------- end of main content --------------- -->
doc/pub/svm/ipynb/ipynb-svm-src.tar.gz
BIN
View File
Binary file not shown.
doc/pub/svm/ipynb/svm.ipynb
+152 -1
View File
@@ -1061,7 +1061,158 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"## Kernels and non-linearity"
"## Kernels and non-linearity\n",
"\n",
"The cases we have studied till were all characterized by two classes\n",
"with a close to linear separability. The classifiers we have described\n",
"so far find linear boundaries in our input feature space. It is\n",
"possible to make our procedure more flexible by exploring the feature\n",
"space using other basis expansions such higher-order polynomials,\n",
"wavelets, splines etc.\n",
"\n",
"If our feature space is not easy to separate, as shown in the figure\n",
"here, we can achieve a better separation by introducing more complex\n",
"basis functions. The ideal would be, as shown in the next figure, to, via a specific transformation to \n",
"obtain a separation between the classes which is almost linear. \n",
"\n",
"The change of basis, from $x\\rightarrow z=\\phi(x)$ leads to the same type of equations to be solved, except that\n",
"we need to introduce for example a polynomial transformation to a two-dimensional training set.\n",
"\n",
"## The equations\n",
"\n",
"Suppose we define a polynomial transformation of degree two (we continue to live in a plane with $x_1$ and $x_2$ as variables)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"$$\n",
"z = \\phi(x) =\\left(1, x_1, x_2, x_1^2, x_2^2, x_1x_2).\n",
"$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"With our new basis, the equations we solved earlier are basically the same, that is we have now (without the slack option for simplicity)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"$$\n",
"{\\cal L}=\\sum_i\\lambda_i-\\frac{1}{2}\\sum_{ij}^n\\lambda_i\\lambda_jy_iy_j\\boldsymbol{z}_i^T\\boldsymbol{Z}_j,\n",
"$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"subject to the constraints $\\lambda_i\\geq 0$, $\\sum_i\\lambda_iy_i=0$, and for the support vectors"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"$$\n",
"y_i(\\boldsymbol{w}^T\\boldsymbol{z}_i+b)= 1 \\hspace{0.1cm}\\forall i,\n",
"$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"from which we also find $b$. \n",
"\n",
"## Different kernels\n",
"\n",
"## Quadratic coefficient matrix\n",
"\n",
"<!-- !split -->\n",
"## Mercer's theorem\n",
"\n",
"## How do we solve these problems\n",
"\n",
"If we use Python as programming language and wish to venture beyond\n",
"**scikit-learn**, **tensorflow** and similar software which makes our\n",
"lives so much easier, we need to dive into the wonderful world of\n",
"quadratic programming. We can, if we wish, solve the minimization\n",
"problem using say standard gradient methods or conjugate gradient\n",
"methods. However, these methods tend to exhibit a rather slow\n",
"converge. So, welcome to the promised land of quadratic programming.\n",
"\n",
"The functions we need are contained in the quadratic programming package **CVXOPT** and we need to import it"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"import numpy\n",
"import cvxopt"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let us first set up the standard form the of quadratic programming (QP) equations by defining the problem as"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"$$\n",
"\\mathrm{min}\n",
"$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"subject to Gx u. Note that x itself is not provided to the solver, since it is an internal\n",
"variable being optimized over. In particular, this means that the solver has no explicit knowledge\n",
"of x itself; everything is implicity defined by the supplied parameters. It is essential\n",
"that the same variable order is maintained for the relevant parameters (e.g., qi\n",
"Non-convexity implies the existence of local optima, making it difficult to find global optima.\n",
"\n",
"collapsed all inequality constraints into a single G matrix of the standard form.\n",
"Since there are no equality constraints, we do not need to provide the empty A, b. Note\n",
"that even though y\n",
"2 did not appear in the original objective, we had to include it with zero\n",
"coefficients in P because the solver parameters must be defined using the full set of variables.\n",
"Even if certain variables only appear in constraints, they will still need to be expressed with\n",
"zero coefficients in the objective parameters, and vice versa.\n",
"Let us first define the above parameters in Python. CVXOPT supplies its own matrix\n",
"object; all arguments given to its solvers must be in this matrix type. There are two ways\n",
"to do this. The first is to define the matrix directly with (potentially nested) lists:\n",
"from cvxopt import matrix"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"P = matrix([[1.0,0.0],[0.0,0.0]])\n",
"q = matrix([3.0,4.0])\n",
"G = matrix([[-1.0,0.0,-1.0,2.0,3.0],[0.0,-1.0,-3.0,5.0,4.0]])\n",
"h = matrix([0.0,0.0,-15.0,100.0,80.0])"
]
}
],
doc/pub/svm/pdf/svm-minted.pdf
BIN
View File
Binary file not shown.
doc/src/SupportVMachines/svm.do.txt
+95
View File
@@ -535,3 +535,98 @@ y_i(\bm{w}^T\bm{x}_i+b) -(1-\xi_) \geq 0 \hspace{0.1cm}\forall i.
!split
===== Kernels and non-linearity =====
The cases we have studied till were all characterized by two classes
with a close to linear separability. The classifiers we have described
so far find linear boundaries in our input feature space. It is
possible to make our procedure more flexible by exploring the feature
space using other basis expansions such higher-order polynomials,
wavelets, splines etc.
If our feature space is not easy to separate, as shown in the figure
here, we can achieve a better separation by introducing more complex
basis functions. The ideal would be, as shown in the next figure, to, via a specific transformation to
obtain a separation between the classes which is almost linear.
The change of basis, from $x\rightarrow z=\phi(x)$ leads to the same type of equations to be solved, except that
we need to introduce for example a polynomial transformation to a two-dimensional training set.
!split
===== The equations =====
Suppose we define a polynomial transformation of degree two (we continue to live in a plane with $x_1$ and $x_2$ as variables)
!bt
\[
z = \phi(x) =\left(1, x_1, x_2, x_1^2, x_2^2, x_1x_2).
\]
!et
With our new basis, the equations we solved earlier are basically the same, that is we have now (without the slack option for simplicity)
!bt
\[
{\cal L}=\sum_i\lambda_i-\frac{1}{2}\sum_{ij}^n\lambda_i\lambda_jy_iy_j\bm{z}_i^T\bm{Z}_j,
\]
!et
subject to the constraints $\lambda_i\geq 0$, $\sum_i\lambda_iy_i=0$, and for the support vectors
!bt
\[
y_i(\bm{w}^T\bm{z}_i+b)= 1 \hspace{0.1cm}\forall i,
\]
!et
from which we also find $b$.
!split
===== Different kernels =====
!split
===== Quadratic coefficient matrix =====
!split
===== Mercer's theorem =====
!split
===== How do we solve these problems =====
If we use Python as programming language and wish to venture beyond
_scikit-learn_, _tensorflow_ and similar software which makes our
lives so much easier, we need to dive into the wonderful world of
quadratic programming. We can, if we wish, solve the minimization
problem using say standard gradient methods or conjugate gradient
methods. However, these methods tend to exhibit a rather slow
converge. So, welcome to the promised land of quadratic programming.
The functions we need are contained in the quadratic programming package _CVXOPT_ and we need to import it
!bc pycod
import numpy
import cvxopt
!ec
Let us first set up the standard form the of quadratic programming (QP) equations by defining the problem as
!bt
\[
\mathrm{min}
\]
!et
subject to Gx u. Note that x itself is not provided to the solver, since it is an internal
variable being optimized over. In particular, this means that the solver has no explicit knowledge
of x itself; everything is implicity defined by the supplied parameters. It is essential
that the same variable order is maintained for the relevant parameters (e.g., qi
Non-convexity implies the existence of local optima, making it difficult to find global optima.
collapsed all inequality constraints into a single G matrix of the standard form.
Since there are no equality constraints, we do not need to provide the empty A, b. Note
that even though y
2 did not appear in the original objective, we had to include it with zero
coefficients in P because the solver parameters must be defined using the full set of variables.
Even if certain variables only appear in constraints, they will still need to be expressed with
zero coefficients in the objective parameters, and vice versa.
Let us first define the above parameters in Python. CVXOPT supplies its own matrix
object; all arguments given to its solvers must be in this matrix type. There are two ways
to do this. The first is to define the matrix directly with (potentially nested) lists:
from cvxopt import matrix
!bc pycod
P = matrix([[1.0,0.0],[0.0,0.0]])
q = matrix([3.0,4.0])
G = matrix([[-1.0,0.0,-1.0,2.0,3.0],[0.0,-1.0,-3.0,5.0,4.0]])
h = matrix([0.0,0.0,-15.0,100.0,80.0])
!ec