lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

adding more to codes for sim

Browse Source
This commit is contained in:
mhjensen
2018-11-06 11:02:04 +01:00
parent 0fce698a63
commit cb232e1001
31 changed files with 425 additions and 121 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/pub/svm/html/._svm-bs000.html
+9 -4
View File
@@ -64,7 +64,11 @@ Automatically generated HTML file from DocOnce source
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
('Mathematical optimization of convex functions',
2,
None,
'___sec21'),
('How do we solve these problems', 2, None, '___sec22')]}
end of tocinfo -->
<body>
@@ -123,7 +127,8 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._svm-bs019.html#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs020.html#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs021.html#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">Mathematical optimization of convex functions</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs023.html#___sec22" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
@@ -158,7 +163,7 @@ MathJax.Hub.Config({
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>Nov 5, 2018</h4></center> <!-- date -->
<center><h4>Nov 6, 2018</h4></center> <!-- date -->
<br>
<p>
@@ -182,7 +187,7 @@ MathJax.Hub.Config({
<li><a href="._svm-bs008.html">9</a></li>
<li><a href="._svm-bs009.html">10</a></li>
<li><a href="">...</a></li>
<li><a href="._svm-bs022.html">23</a></li>
<li><a href="._svm-bs023.html">24</a></li>
<li><a href="._svm-bs001.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/svm/html/._svm-bs001.html
+8 -3
View File
@@ -64,7 +64,11 @@ Automatically generated HTML file from DocOnce source
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
('Mathematical optimization of convex functions',
2,
None,
'___sec21'),
('How do we solve these problems', 2, None, '___sec22')]}
end of tocinfo -->
<body>
@@ -123,7 +127,8 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._svm-bs019.html#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs020.html#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs021.html#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">Mathematical optimization of convex functions</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs023.html#___sec22" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
@@ -181,7 +186,7 @@ We distinguish also between linear and non-linear approaches. The latter are the
<li><a href="._svm-bs009.html">10</a></li>
<li><a href="._svm-bs010.html">11</a></li>
<li><a href="">...</a></li>
<li><a href="._svm-bs022.html">23</a></li>
<li><a href="._svm-bs023.html">24</a></li>
<li><a href="._svm-bs002.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/svm/html/._svm-bs002.html
+8 -3
View File
@@ -64,7 +64,11 @@ Automatically generated HTML file from DocOnce source
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
('Mathematical optimization of convex functions',
2,
None,
'___sec21'),
('How do we solve these problems', 2, None, '___sec22')]}
end of tocinfo -->
<body>
@@ -123,7 +127,8 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._svm-bs019.html#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs020.html#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs021.html#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">Mathematical optimization of convex functions</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs023.html#___sec22" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
@@ -172,7 +177,7 @@ circles.
<li><a href="._svm-bs010.html">11</a></li>
<li><a href="._svm-bs011.html">12</a></li>
<li><a href="">...</a></li>
<li><a href="._svm-bs022.html">23</a></li>
<li><a href="._svm-bs023.html">24</a></li>
<li><a href="._svm-bs003.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/svm/html/._svm-bs003.html
+8 -3
View File
@@ -64,7 +64,11 @@ Automatically generated HTML file from DocOnce source
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
('Mathematical optimization of convex functions',
2,
None,
'___sec21'),
('How do we solve these problems', 2, None, '___sec22')]}
end of tocinfo -->
<body>
@@ -123,7 +127,8 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._svm-bs019.html#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs020.html#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs021.html#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">Mathematical optimization of convex functions</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs023.html#___sec22" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
@@ -182,7 +187,7 @@ $$
<li><a href="._svm-bs011.html">12</a></li>
<li><a href="._svm-bs012.html">13</a></li>
<li><a href="">...</a></li>
<li><a href="._svm-bs022.html">23</a></li>
<li><a href="._svm-bs023.html">24</a></li>
<li><a href="._svm-bs004.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/svm/html/._svm-bs004.html
+8 -3
View File
@@ -64,7 +64,11 @@ Automatically generated HTML file from DocOnce source
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
('Mathematical optimization of convex functions',
2,
None,
'___sec21'),
('How do we solve these problems', 2, None, '___sec22')]}
end of tocinfo -->
<body>
@@ -123,7 +127,8 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._svm-bs019.html#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs020.html#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs021.html#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">Mathematical optimization of convex functions</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs023.html#___sec22" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
@@ -198,7 +203,7 @@ When we try to separate hyperplanes, if it exists, we can use it to construct a
<li><a href="._svm-bs012.html">13</a></li>
<li><a href="._svm-bs013.html">14</a></li>
<li><a href="">...</a></li>
<li><a href="._svm-bs022.html">23</a></li>
<li><a href="._svm-bs023.html">24</a></li>
<li><a href="._svm-bs005.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/svm/html/._svm-bs005.html
+8 -3
View File
@@ -64,7 +64,11 @@ Automatically generated HTML file from DocOnce source
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
('Mathematical optimization of convex functions',
2,
None,
'___sec21'),
('How do we solve these problems', 2, None, '___sec22')]}
end of tocinfo -->
<body>
@@ -123,7 +127,8 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._svm-bs019.html#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs020.html#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs021.html#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">Mathematical optimization of convex functions</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs023.html#___sec22" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
@@ -184,7 +189,7 @@ for our data sample.
<li><a href="._svm-bs013.html">14</a></li>
<li><a href="._svm-bs014.html">15</a></li>
<li><a href="">...</a></li>
<li><a href="._svm-bs022.html">23</a></li>
<li><a href="._svm-bs023.html">24</a></li>
<li><a href="._svm-bs006.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/svm/html/._svm-bs006.html
+8 -3
View File
@@ -64,7 +64,11 @@ Automatically generated HTML file from DocOnce source
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
('Mathematical optimization of convex functions',
2,
None,
'___sec21'),
('How do we solve these problems', 2, None, '___sec22')]}
end of tocinfo -->
<body>
@@ -123,7 +127,8 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._svm-bs019.html#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs020.html#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs021.html#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">Mathematical optimization of convex functions</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs023.html#___sec22" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
@@ -180,7 +185,7 @@ $$
<li><a href="._svm-bs014.html">15</a></li>
<li><a href="._svm-bs015.html">16</a></li>
<li><a href="">...</a></li>
<li><a href="._svm-bs022.html">23</a></li>
<li><a href="._svm-bs023.html">24</a></li>
<li><a href="._svm-bs007.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/svm/html/._svm-bs007.html
+8 -3
View File
@@ -64,7 +64,11 @@ Automatically generated HTML file from DocOnce source
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
('Mathematical optimization of convex functions',
2,
None,
'___sec21'),
('How do we solve these problems', 2, None, '___sec22')]}
end of tocinfo -->
<body>
@@ -123,7 +127,8 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._svm-bs019.html#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs020.html#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs021.html#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">Mathematical optimization of convex functions</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs023.html#___sec22" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
@@ -184,7 +189,7 @@ $$
<li><a href="._svm-bs015.html">16</a></li>
<li><a href="._svm-bs016.html">17</a></li>
<li><a href="">...</a></li>
<li><a href="._svm-bs022.html">23</a></li>
<li><a href="._svm-bs023.html">24</a></li>
<li><a href="._svm-bs008.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/svm/html/._svm-bs008.html
+8 -3
View File
@@ -64,7 +64,11 @@ Automatically generated HTML file from DocOnce source
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
('Mathematical optimization of convex functions',
2,
None,
'___sec21'),
('How do we solve these problems', 2, None, '___sec22')]}
end of tocinfo -->
<body>
@@ -123,7 +127,8 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._svm-bs019.html#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs020.html#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs021.html#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">Mathematical optimization of convex functions</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs023.html#___sec22" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
@@ -188,7 +193,7 @@ at all.
<li><a href="._svm-bs016.html">17</a></li>
<li><a href="._svm-bs017.html">18</a></li>
<li><a href="">...</a></li>
<li><a href="._svm-bs022.html">23</a></li>
<li><a href="._svm-bs023.html">24</a></li>
<li><a href="._svm-bs009.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/svm/html/._svm-bs009.html
+8 -3
View File
@@ -64,7 +64,11 @@ Automatically generated HTML file from DocOnce source
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
('Mathematical optimization of convex functions',
2,
None,
'___sec21'),
('How do we solve these problems', 2, None, '___sec22')]}
end of tocinfo -->
<body>
@@ -123,7 +127,8 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._svm-bs019.html#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs020.html#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs021.html#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">Mathematical optimization of convex functions</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs023.html#___sec22" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
@@ -200,7 +205,7 @@ We have thus defined our margin as the invers of the norm of \( \boldsymbol{w} \
<li><a href="._svm-bs017.html">18</a></li>
<li><a href="._svm-bs018.html">19</a></li>
<li><a href="">...</a></li>
<li><a href="._svm-bs022.html">23</a></li>
<li><a href="._svm-bs023.html">24</a></li>
<li><a href="._svm-bs010.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/svm/html/._svm-bs010.html
+8 -3
View File
@@ -64,7 +64,11 @@ Automatically generated HTML file from DocOnce source
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
('Mathematical optimization of convex functions',
2,
None,
'___sec21'),
('How do we solve these problems', 2, None, '___sec22')]}
end of tocinfo -->
<body>
@@ -123,7 +127,8 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._svm-bs019.html#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs020.html#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs021.html#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">Mathematical optimization of convex functions</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs023.html#___sec22" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
@@ -212,7 +217,7 @@ Then \( dz \) is no longer arbitrary.
<li><a href="._svm-bs018.html">19</a></li>
<li><a href="._svm-bs019.html">20</a></li>
<li><a href="">...</a></li>
<li><a href="._svm-bs022.html">23</a></li>
<li><a href="._svm-bs023.html">24</a></li>
<li><a href="._svm-bs011.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/svm/html/._svm-bs011.html
+8 -3
View File
@@ -64,7 +64,11 @@ Automatically generated HTML file from DocOnce source
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
('Mathematical optimization of convex functions',
2,
None,
'___sec21'),
('How do we solve these problems', 2, None, '___sec22')]}
end of tocinfo -->
<body>
@@ -123,7 +127,8 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._svm-bs019.html#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs020.html#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs021.html#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">Mathematical optimization of convex functions</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs023.html#___sec22" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
@@ -205,7 +210,7 @@ $$
<li><a href="._svm-bs019.html">20</a></li>
<li><a href="._svm-bs020.html">21</a></li>
<li><a href="">...</a></li>
<li><a href="._svm-bs022.html">23</a></li>
<li><a href="._svm-bs023.html">24</a></li>
<li><a href="._svm-bs012.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/svm/html/._svm-bs012.html
+8 -3
View File
@@ -64,7 +64,11 @@ Automatically generated HTML file from DocOnce source
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
('Mathematical optimization of convex functions',
2,
None,
'___sec21'),
('How do we solve these problems', 2, None, '___sec22')]}
end of tocinfo -->
<body>
@@ -123,7 +127,8 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._svm-bs019.html#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs020.html#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs021.html#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">Mathematical optimization of convex functions</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs023.html#___sec22" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
@@ -203,7 +208,7 @@ When \( \lambda_i > 0 \), the vectors \( \boldsymbol{x}_i \) are called support
<li><a href="._svm-bs020.html">21</a></li>
<li><a href="._svm-bs021.html">22</a></li>
<li><a href="">...</a></li>
<li><a href="._svm-bs022.html">23</a></li>
<li><a href="._svm-bs023.html">24</a></li>
<li><a href="._svm-bs013.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/svm/html/._svm-bs013.html
+9 -2
View File
@@ -64,7 +64,11 @@ Automatically generated HTML file from DocOnce source
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
('Mathematical optimization of convex functions',
2,
None,
'___sec21'),
('How do we solve these problems', 2, None, '___sec22')]}
end of tocinfo -->
<body>
@@ -123,7 +127,8 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._svm-bs019.html#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs020.html#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs021.html#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">Mathematical optimization of convex functions</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs023.html#___sec22" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
@@ -185,6 +190,8 @@ subject to \( \boldsymbol{y}^T\boldsymbol{\lambda}=0 \). Here we defined the vec
<li><a href="._svm-bs020.html">21</a></li>
<li><a href="._svm-bs021.html">22</a></li>
<li><a href="._svm-bs022.html">23</a></li>
<li><a href="">...</a></li>
<li><a href="._svm-bs023.html">24</a></li>
<li><a href="._svm-bs014.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/svm/html/._svm-bs014.html
+8 -2
View File
@@ -64,7 +64,11 @@ Automatically generated HTML file from DocOnce source
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
('Mathematical optimization of convex functions',
2,
None,
'___sec21'),
('How do we solve these problems', 2, None, '___sec22')]}
end of tocinfo -->
<body>
@@ -123,7 +127,8 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._svm-bs019.html#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs020.html#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs021.html#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">Mathematical optimization of convex functions</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs023.html#___sec22" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
@@ -194,6 +199,7 @@ Below we discuss how to find the optimal values of \( \lambda_i \). Before we pr
<li><a href="._svm-bs020.html">21</a></li>
<li><a href="._svm-bs021.html">22</a></li>
<li><a href="._svm-bs022.html">23</a></li>
<li><a href="._svm-bs023.html">24</a></li>
<li><a href="._svm-bs015.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/svm/html/._svm-bs015.html
+8 -2
View File
@@ -64,7 +64,11 @@ Automatically generated HTML file from DocOnce source
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
('Mathematical optimization of convex functions',
2,
None,
'___sec21'),
('How do we solve these problems', 2, None, '___sec22')]}
end of tocinfo -->
<body>
@@ -123,7 +127,8 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._svm-bs019.html#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs020.html#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs021.html#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">Mathematical optimization of convex functions</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs023.html#___sec22" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
@@ -194,6 +199,7 @@ misclassifications.
<li><a href="._svm-bs020.html">21</a></li>
<li><a href="._svm-bs021.html">22</a></li>
<li><a href="._svm-bs022.html">23</a></li>
<li><a href="._svm-bs023.html">24</a></li>
<li><a href="._svm-bs016.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/svm/html/._svm-bs016.html
+8 -2
View File
@@ -64,7 +64,11 @@ Automatically generated HTML file from DocOnce source
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
('Mathematical optimization of convex functions',
2,
None,
'___sec21'),
('How do we solve these problems', 2, None, '___sec22')]}
end of tocinfo -->
<body>
@@ -123,7 +127,8 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._svm-bs019.html#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs020.html#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs021.html#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">Mathematical optimization of convex functions</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs023.html#___sec22" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
@@ -212,6 +217,7 @@ $$
<li><a href="._svm-bs020.html">21</a></li>
<li><a href="._svm-bs021.html">22</a></li>
<li><a href="._svm-bs022.html">23</a></li>
<li><a href="._svm-bs023.html">24</a></li>
<li><a href="._svm-bs017.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/svm/html/._svm-bs017.html
+8 -2
View File
@@ -64,7 +64,11 @@ Automatically generated HTML file from DocOnce source
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
('Mathematical optimization of convex functions',
2,
None,
'___sec21'),
('How do we solve these problems', 2, None, '___sec22')]}
end of tocinfo -->
<body>
@@ -123,7 +127,8 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._svm-bs019.html#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs020.html#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs021.html#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">Mathematical optimization of convex functions</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs023.html#___sec22" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
@@ -180,6 +185,7 @@ we need to introduce for example a polynomial transformation to a two-dimensiona
<li><a href="._svm-bs020.html">21</a></li>
<li><a href="._svm-bs021.html">22</a></li>
<li><a href="._svm-bs022.html">23</a></li>
<li><a href="._svm-bs023.html">24</a></li>
<li><a href="._svm-bs018.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/svm/html/._svm-bs018.html
+8 -2
View File
@@ -64,7 +64,11 @@ Automatically generated HTML file from DocOnce source
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
('Mathematical optimization of convex functions',
2,
None,
'___sec21'),
('How do we solve these problems', 2, None, '___sec22')]}
end of tocinfo -->
<body>
@@ -123,7 +127,8 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._svm-bs019.html#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs020.html#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs021.html#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">Mathematical optimization of convex functions</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs023.html#___sec22" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
@@ -180,6 +185,7 @@ from which we also find \( b \).
<li><a href="._svm-bs020.html">21</a></li>
<li><a href="._svm-bs021.html">22</a></li>
<li><a href="._svm-bs022.html">23</a></li>
<li><a href="._svm-bs023.html">24</a></li>
<li><a href="._svm-bs019.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/svm/html/._svm-bs019.html
+8 -2
View File
@@ -64,7 +64,11 @@ Automatically generated HTML file from DocOnce source
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
('Mathematical optimization of convex functions',
2,
None,
'___sec21'),
('How do we solve these problems', 2, None, '___sec22')]}
end of tocinfo -->
<body>
@@ -123,7 +127,8 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs020.html#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs021.html#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">Mathematical optimization of convex functions</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs023.html#___sec22" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
@@ -160,6 +165,7 @@ MathJax.Hub.Config({
<li><a href="._svm-bs020.html">21</a></li>
<li><a href="._svm-bs021.html">22</a></li>
<li><a href="._svm-bs022.html">23</a></li>
<li><a href="._svm-bs023.html">24</a></li>
<li><a href="._svm-bs020.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/svm/html/._svm-bs020.html
+8 -2
View File
@@ -64,7 +64,11 @@ Automatically generated HTML file from DocOnce source
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
('Mathematical optimization of convex functions',
2,
None,
'___sec21'),
('How do we solve these problems', 2, None, '___sec22')]}
end of tocinfo -->
<body>
@@ -123,7 +127,8 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._svm-bs019.html#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs021.html#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">Mathematical optimization of convex functions</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs023.html#___sec22" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
@@ -159,6 +164,7 @@ MathJax.Hub.Config({
<li class="active"><a href="._svm-bs020.html">21</a></li>
<li><a href="._svm-bs021.html">22</a></li>
<li><a href="._svm-bs022.html">23</a></li>
<li><a href="._svm-bs023.html">24</a></li>
<li><a href="._svm-bs021.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/svm/html/._svm-bs021.html
+8 -2
View File
@@ -64,7 +64,11 @@ Automatically generated HTML file from DocOnce source
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
('Mathematical optimization of convex functions',
2,
None,
'___sec21'),
('How do we solve these problems', 2, None, '___sec22')]}
end of tocinfo -->
<body>
@@ -123,7 +127,8 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._svm-bs019.html#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs020.html#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">Mathematical optimization of convex functions</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs023.html#___sec22" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
@@ -158,6 +163,7 @@ MathJax.Hub.Config({
<li><a href="._svm-bs020.html">21</a></li>
<li class="active"><a href="._svm-bs021.html">22</a></li>
<li><a href="._svm-bs022.html">23</a></li>
<li><a href="._svm-bs023.html">24</a></li>
<li><a href="._svm-bs022.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/svm/html/._svm-bs022.html
+20 -46
View File
@@ -64,7 +64,11 @@ Automatically generated HTML file from DocOnce source
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
('Mathematical optimization of convex functions',
2,
None,
'___sec21'),
('How do we solve these problems', 2, None, '___sec22')]}
end of tocinfo -->
<body>
@@ -123,7 +127,8 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._svm-bs019.html#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs020.html#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs021.html#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
<!-- navigation toc: --> <li><a href="#___sec21" style="font-size: 80%;">Mathematical optimization of convex functions</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs023.html#___sec22" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
@@ -139,60 +144,27 @@ MathJax.Hub.Config({
<a name="part0022"></a>
<!-- !split -->
<h2 id="___sec21" class="anchor">How do we solve these problems </h2>
<h2 id="___sec21" class="anchor">Mathematical optimization of convex functions </h2>
<p>
If we use Python as programming language and wish to venture beyond
<b>scikit-learn</b>, <b>tensorflow</b> and similar software which makes our
lives so much easier, we need to dive into the wonderful world of
quadratic programming. We can, if we wish, solve the minimization
problem using say standard gradient methods or conjugate gradient
methods. However, these methods tend to exhibit a rather slow
converge. So, welcome to the promised land of quadratic programming.
<p>
The functions we need are contained in the quadratic programming package <b>CVXOPT</b> and we need to import it
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">numpy</span>
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">cvxopt</span>
</pre></div>
<p>
Let us first set up the standard form the of quadratic programming (QP) equations by defining the problem as
A mathematical optimization problem, or just optimization problem, has the form
$$
\mathrm{min}
\mathrm{minimize}\hspace{0.1cm} f(x),
$$
<p>
subject to Gx u. Note that x itself is not provided to the solver, since it is an internal
variable being optimized over. In particular, this means that the solver has no explicit knowledge
of x itself; everything is implicity defined by the supplied parameters. It is essential
that the same variable order is maintained for the relevant parameters (e.g., qi
Non-convexity implies the existence of local optima, making it difficult to find global optima.
subject to some constraints \( g(\lambda_i) \leq b_i \) for say a selected set \( i=1,2,\dots, n \).
In our case we are optimizing with respect to the Lagrangian multipliers \( \lambda_i \), and the
vector \( \boldsymbol{\lambda}=[\lambda_1, \lambda_2,\dots, \lambda_n] \) is the optimization variable we are dealing with.
and \( f(x) \) is our objective function while \( g(\lambda_i) \leq b_i \) represents our constraint function.
<p>
collapsed all inequality constraints into a single G matrix of the standard form.
Since there are no equality constraints, we do not need to provide the empty A, b. Note
that even though y
2 did not appear in the original objective, we had to include it with zero
coefficients in P because the solver parameters must be defined using the full set of variables.
Even if certain variables only appear in constraints, they will still need to be expressed with
zero coefficients in the objective parameters, and vice versa.
Let us first define the above parameters in Python. CVXOPT supplies its own matrix
object; all arguments given to its solvers must be in this matrix type. There are two ways
to do this. The first is to define the matrix directly with (potentially nested) lists:
from cvxopt import matrix
<p>
In our case we are particularly interested in a class of optimization problems called convex optmization problems.
In our disussion on gradient descent methods we discussed at length the definition of a convex function.
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span>P <span style="color: #666666">=</span> matrix([[<span style="color: #666666">1.0</span>,<span style="color: #666666">0.0</span>],[<span style="color: #666666">0.0</span>,<span style="color: #666666">0.0</span>]])
q <span style="color: #666666">=</span> matrix([<span style="color: #666666">3.0</span>,<span style="color: #666666">4.0</span>])
G <span style="color: #666666">=</span> matrix([[<span style="color: #666666">-1.0</span>,<span style="color: #666666">0.0</span>,<span style="color: #666666">-1.0</span>,<span style="color: #666666">2.0</span>,<span style="color: #666666">3.0</span>],[<span style="color: #666666">0.0</span>,<span style="color: #666666">-1.0</span>,<span style="color: #666666">-3.0</span>,<span style="color: #666666">5.0</span>,<span style="color: #666666">4.0</span>]])
h <span style="color: #666666">=</span> matrix([<span style="color: #666666">0.0</span>,<span style="color: #666666">0.0</span>,<span style="color: #666666">-15.0</span>,<span style="color: #666666">100.0</span>,<span style="color: #666666">80.0</span>])
</pre></div>
<p>
Convex optimization problems play a central role in applied mathematics and we recommend strongly <a href="http://web.stanford.edu/~boyd/cvxbook/" target="_self">Boyd and Vandenberghe's text on the topics</a>.
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
@@ -208,6 +180,8 @@ h <span style="color: #666666">=</span> matrix([<span style="color: #666666">0.0
<li><a href="._svm-bs020.html">21</a></li>
<li><a href="._svm-bs021.html">22</a></li>
<li class="active"><a href="._svm-bs022.html">23</a></li>
<li><a href="._svm-bs023.html">24</a></li>
<li><a href="._svm-bs023.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/svm/html/svm-bs.html
+9 -4
View File
@@ -64,7 +64,11 @@ Automatically generated HTML file from DocOnce source
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
('Mathematical optimization of convex functions',
2,
None,
'___sec21'),
('How do we solve these problems', 2, None, '___sec22')]}
end of tocinfo -->
<body>
@@ -123,7 +127,8 @@ MathJax.Hub.Config({
<!-- navigation toc: --> <li><a href="._svm-bs019.html#___sec18" style="font-size: 80%;">Different kernels</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs020.html#___sec19" style="font-size: 80%;">Quadratic coefficient matrix</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs021.html#___sec20" style="font-size: 80%;">Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">How do we solve these problems</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs022.html#___sec21" style="font-size: 80%;">Mathematical optimization of convex functions</a></li>
<!-- navigation toc: --> <li><a href="._svm-bs023.html#___sec22" style="font-size: 80%;">How do we solve these problems</a></li>
</ul>
</li>
@@ -158,7 +163,7 @@ MathJax.Hub.Config({
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>Nov 5, 2018</h4></center> <!-- date -->
<center><h4>Nov 6, 2018</h4></center> <!-- date -->
<br>
<p>
@@ -182,7 +187,7 @@ MathJax.Hub.Config({
<li><a href="._svm-bs008.html">9</a></li>
<li><a href="._svm-bs009.html">10</a></li>
<li><a href="">...</a></li>
<li><a href="._svm-bs022.html">23</a></li>
<li><a href="._svm-bs023.html">24</a></li>
<li><a href="._svm-bs001.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
doc/pub/svm/html/svm-reveal.html
+43 -3
View File
@@ -148,7 +148,7 @@ MathJax.Hub.Config({
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>&nbsp;<br>
<center><h4>Nov 5, 2018</h4></center> <!-- date -->
<center><h4>Nov 6, 2018</h4></center> <!-- date -->
<br>
<p>
@@ -871,7 +871,32 @@ from which we also find \( b \).
<section>
<h2 id="___sec21">How do we solve these problems </h2>
<h2 id="___sec21">Mathematical optimization of convex functions </h2>
<p>
A mathematical optimization problem, or just optimization problem, has the form
<p>&nbsp;<br>
$$
\mathrm{minimize}\hspace{0.1cm} f(x),
$$
<p>&nbsp;<br>
subject to some constraints \( g(\lambda_i) \leq b_i \) for say a selected set \( i=1,2,\dots, n \).
In our case we are optimizing with respect to the Lagrangian multipliers \( \lambda_i \), and the
vector \( \boldsymbol{\lambda}=[\lambda_1, \lambda_2,\dots, \lambda_n] \) is the optimization variable we are dealing with.
and \( f(x) \) is our objective function while \( g(\lambda_i) \leq b_i \) represents our constraint function.
<p>
In our case we are particularly interested in a class of optimization problems called convex optmization problems.
In our disussion on gradient descent methods we discussed at length the definition of a convex function.
<p>
Convex optimization problems play a central role in applied mathematics and we recommend strongly <a href="http://web.stanford.edu/~boyd/cvxbook/" target="_blank">Boyd and Vandenberghe's text on the topics</a>.
</section>
<section>
<h2 id="___sec22">How do we solve these problems </h2>
<p>
If we use Python as programming language and wish to venture beyond
@@ -920,10 +945,25 @@ from cvxopt import matrix
<p>
<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
<div class="highlight" style="background: #eeeedd"><pre style="font-size: 80%; line-height: 125%"><span></span>P = matrix([[<span style="color: #B452CD">1.0</span>,<span style="color: #B452CD">0.0</span>],[<span style="color: #B452CD">0.0</span>,<span style="color: #B452CD">0.0</span>]])
<div class="highlight" style="background: #eeeedd"><pre style="font-size: 80%; line-height: 125%"><span></span><span style="color: #228B22"># Import the necessary packages</span>
<span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">numpy</span>
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">cvxopt</span> <span style="color: #8B008B; font-weight: bold">import</span> matrix
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">cvxopt</span> <span style="color: #8B008B; font-weight: bold">import</span> solvers
<span style="color: #228B22"># Define QP parameters (directly)</span>
P = matrix([[<span style="color: #B452CD">1.0</span>,<span style="color: #B452CD">0.0</span>],[<span style="color: #B452CD">0.0</span>,<span style="color: #B452CD">0.0</span>]])
q = matrix([<span style="color: #B452CD">3.0</span>,<span style="color: #B452CD">4.0</span>])
G = matrix([[-<span style="color: #B452CD">1.0</span>,<span style="color: #B452CD">0.0</span>,-<span style="color: #B452CD">1.0</span>,<span style="color: #B452CD">2.0</span>,<span style="color: #B452CD">3.0</span>],[<span style="color: #B452CD">0.0</span>,-<span style="color: #B452CD">1.0</span>,-<span style="color: #B452CD">3.0</span>,<span style="color: #B452CD">5.0</span>,<span style="color: #B452CD">4.0</span>]])
h = matrix([<span style="color: #B452CD">0.0</span>,<span style="color: #B452CD">0.0</span>,-<span style="color: #B452CD">15.0</span>,<span style="color: #B452CD">100.0</span>,<span style="color: #B452CD">80.0</span>])
<span style="color: #228B22"># Define QP parameters (with NumPy)</span>
P = matrix(numpy.diag([<span style="color: #B452CD">1</span>,<span style="color: #B452CD">0</span>]), tc=<span style="color: #a61717; background-color: #e3d2d2">’</span>d<span style="color: #a61717; background-color: #e3d2d2">’</span>)
q = matrix(numpy.array([<span style="color: #B452CD">3</span>,<span style="color: #B452CD">4</span>]), tc=<span style="color: #a61717; background-color: #e3d2d2">’</span>d<span style="color: #a61717; background-color: #e3d2d2">’</span>)
G = matrix(numpy.array([[-<span style="color: #B452CD">1</span>,<span style="color: #B452CD">0</span>],[<span style="color: #B452CD">0</span>,-<span style="color: #B452CD">1</span>],[-<span style="color: #B452CD">1</span>,-<span style="color: #B452CD">3</span>],[<span style="color: #B452CD">2</span>,<span style="color: #B452CD">5</span>],[<span style="color: #B452CD">3</span>,<span style="color: #B452CD">4</span>]]), tc=<span style="color: #a61717; background-color: #e3d2d2">’</span>d<span style="color: #a61717; background-color: #e3d2d2">’</span>)
h = matrix(numpy.array([<span style="color: #B452CD">0</span>,<span style="color: #B452CD">0</span>,-<span style="color: #B452CD">15</span>,<span style="color: #B452CD">100</span>,<span style="color: #B452CD">80</span>]), tc=<span style="color: #a61717; background-color: #e3d2d2">’</span>d<span style="color: #a61717; background-color: #e3d2d2">’</span>)
<span style="color: #228B22"># Construct the QP, invoke solver</span>
sol = solvers.qp(P,q,G,h)
<span style="color: #228B22"># Extract optimal value and solution</span>
sol[<span style="color: #a61717; background-color: #e3d2d2">’</span>x<span style="color: #a61717; background-color: #e3d2d2">’</span>] <span style="color: #228B22"># [7.13e-07, 5.00e+00]</span>
sol[<span style="color: #a61717; background-color: #e3d2d2">’</span>primal objective<span style="color: #a61717; background-color: #e3d2d2">’</span>]
</pre></div>
</section>
doc/pub/svm/html/svm-solarized.html
+46 -4
View File
@@ -58,7 +58,11 @@ div { text-align: justify; text-justify: inter-word; }
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
('Mathematical optimization of convex functions',
2,
None,
'___sec21'),
('How do we solve these problems', 2, None, '___sec22')]}
end of tocinfo -->
<body>
@@ -100,7 +104,7 @@ MathJax.Hub.Config({
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>Nov 5, 2018</h4></center> <!-- date -->
<center><h4>Nov 6, 2018</h4></center> <!-- date -->
<br>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
@@ -704,7 +708,30 @@ from which we also find \( b \).
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec21">How do we solve these problems </h2>
<h2 id="___sec21">Mathematical optimization of convex functions </h2>
<p>
A mathematical optimization problem, or just optimization problem, has the form
$$
\mathrm{minimize}\hspace{0.1cm} f(x),
$$
subject to some constraints \( g(\lambda_i) \leq b_i \) for say a selected set \( i=1,2,\dots, n \).
In our case we are optimizing with respect to the Lagrangian multipliers \( \lambda_i \), and the
vector \( \boldsymbol{\lambda}=[\lambda_1, \lambda_2,\dots, \lambda_n] \) is the optimization variable we are dealing with.
and \( f(x) \) is our objective function while \( g(\lambda_i) \leq b_i \) represents our constraint function.
<p>
In our case we are particularly interested in a class of optimization problems called convex optmization problems.
In our disussion on gradient descent methods we discussed at length the definition of a convex function.
<p>
Convex optimization problems play a central role in applied mathematics and we recommend strongly <a href="http://web.stanford.edu/~boyd/cvxbook/" target="_blank">Boyd and Vandenberghe's text on the topics</a>.
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec22">How do we solve these problems </h2>
<p>
If we use Python as programming language and wish to venture beyond
@@ -751,10 +778,25 @@ from cvxopt import matrix
<p>
<!-- code=python (!bc pycod) typeset with pygments style "perldoc" -->
<div class="highlight" style="background: #eeeedd"><pre style="line-height: 125%"><span></span>P = matrix([[<span style="color: #B452CD">1.0</span>,<span style="color: #B452CD">0.0</span>],[<span style="color: #B452CD">0.0</span>,<span style="color: #B452CD">0.0</span>]])
<div class="highlight" style="background: #eeeedd"><pre style="line-height: 125%"><span></span><span style="color: #228B22"># Import the necessary packages</span>
<span style="color: #8B008B; font-weight: bold">import</span> <span style="color: #008b45; text-decoration: underline">numpy</span>
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">cvxopt</span> <span style="color: #8B008B; font-weight: bold">import</span> matrix
<span style="color: #8B008B; font-weight: bold">from</span> <span style="color: #008b45; text-decoration: underline">cvxopt</span> <span style="color: #8B008B; font-weight: bold">import</span> solvers
<span style="color: #228B22"># Define QP parameters (directly)</span>
P = matrix([[<span style="color: #B452CD">1.0</span>,<span style="color: #B452CD">0.0</span>],[<span style="color: #B452CD">0.0</span>,<span style="color: #B452CD">0.0</span>]])
q = matrix([<span style="color: #B452CD">3.0</span>,<span style="color: #B452CD">4.0</span>])
G = matrix([[-<span style="color: #B452CD">1.0</span>,<span style="color: #B452CD">0.0</span>,-<span style="color: #B452CD">1.0</span>,<span style="color: #B452CD">2.0</span>,<span style="color: #B452CD">3.0</span>],[<span style="color: #B452CD">0.0</span>,-<span style="color: #B452CD">1.0</span>,-<span style="color: #B452CD">3.0</span>,<span style="color: #B452CD">5.0</span>,<span style="color: #B452CD">4.0</span>]])
h = matrix([<span style="color: #B452CD">0.0</span>,<span style="color: #B452CD">0.0</span>,-<span style="color: #B452CD">15.0</span>,<span style="color: #B452CD">100.0</span>,<span style="color: #B452CD">80.0</span>])
<span style="color: #228B22"># Define QP parameters (with NumPy)</span>
P = matrix(numpy.diag([<span style="color: #B452CD">1</span>,<span style="color: #B452CD">0</span>]), tc=<span style="color: #a61717; background-color: #e3d2d2">’</span>d<span style="color: #a61717; background-color: #e3d2d2">’</span>)
q = matrix(numpy.array([<span style="color: #B452CD">3</span>,<span style="color: #B452CD">4</span>]), tc=<span style="color: #a61717; background-color: #e3d2d2">’</span>d<span style="color: #a61717; background-color: #e3d2d2">’</span>)
G = matrix(numpy.array([[-<span style="color: #B452CD">1</span>,<span style="color: #B452CD">0</span>],[<span style="color: #B452CD">0</span>,-<span style="color: #B452CD">1</span>],[-<span style="color: #B452CD">1</span>,-<span style="color: #B452CD">3</span>],[<span style="color: #B452CD">2</span>,<span style="color: #B452CD">5</span>],[<span style="color: #B452CD">3</span>,<span style="color: #B452CD">4</span>]]), tc=<span style="color: #a61717; background-color: #e3d2d2">’</span>d<span style="color: #a61717; background-color: #e3d2d2">’</span>)
h = matrix(numpy.array([<span style="color: #B452CD">0</span>,<span style="color: #B452CD">0</span>,-<span style="color: #B452CD">15</span>,<span style="color: #B452CD">100</span>,<span style="color: #B452CD">80</span>]), tc=<span style="color: #a61717; background-color: #e3d2d2">’</span>d<span style="color: #a61717; background-color: #e3d2d2">’</span>)
<span style="color: #228B22"># Construct the QP, invoke solver</span>
sol = solvers.qp(P,q,G,h)
<span style="color: #228B22"># Extract optimal value and solution</span>
sol[<span style="color: #a61717; background-color: #e3d2d2">’</span>x<span style="color: #a61717; background-color: #e3d2d2">’</span>] <span style="color: #228B22"># [7.13e-07, 5.00e+00]</span>
sol[<span style="color: #a61717; background-color: #e3d2d2">’</span>primal objective<span style="color: #a61717; background-color: #e3d2d2">’</span>]
</pre></div>
<p>
doc/pub/svm/html/svm.html
+46 -4
View File
@@ -63,7 +63,11 @@ div { text-align: justify; text-justify: inter-word; }
('Different kernels', 2, None, '___sec18'),
('Quadratic coefficient matrix', 2, None, '___sec19'),
("Mercer's theorem", 2, None, '___sec20'),
('How do we solve these problems', 2, None, '___sec21')]}
('Mathematical optimization of convex functions',
2,
None,
'___sec21'),
('How do we solve these problems', 2, None, '___sec22')]}
end of tocinfo -->
<body>
@@ -105,7 +109,7 @@ MathJax.Hub.Config({
<center>[2] <b>Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University</b></center>
<br>
<p>
<center><h4>Nov 5, 2018</h4></center> <!-- date -->
<center><h4>Nov 6, 2018</h4></center> <!-- date -->
<br>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
@@ -709,7 +713,30 @@ from which we also find \( b \).
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec21">How do we solve these problems </h2>
<h2 id="___sec21">Mathematical optimization of convex functions </h2>
<p>
A mathematical optimization problem, or just optimization problem, has the form
$$
\mathrm{minimize}\hspace{0.1cm} f(x),
$$
subject to some constraints \( g(\lambda_i) \leq b_i \) for say a selected set \( i=1,2,\dots, n \).
In our case we are optimizing with respect to the Lagrangian multipliers \( \lambda_i \), and the
vector \( \boldsymbol{\lambda}=[\lambda_1, \lambda_2,\dots, \lambda_n] \) is the optimization variable we are dealing with.
and \( f(x) \) is our objective function while \( g(\lambda_i) \leq b_i \) represents our constraint function.
<p>
In our case we are particularly interested in a class of optimization problems called convex optmization problems.
In our disussion on gradient descent methods we discussed at length the definition of a convex function.
<p>
Convex optimization problems play a central role in applied mathematics and we recommend strongly <a href="http://web.stanford.edu/~boyd/cvxbook/" target="_blank">Boyd and Vandenberghe's text on the topics</a>.
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec22">How do we solve these problems </h2>
<p>
If we use Python as programming language and wish to venture beyond
@@ -756,10 +783,25 @@ from cvxopt import matrix
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span>P <span style="color: #666666">=</span> matrix([[<span style="color: #666666">1.0</span>,<span style="color: #666666">0.0</span>],[<span style="color: #666666">0.0</span>,<span style="color: #666666">0.0</span>]])
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #408080; font-style: italic"># Import the necessary packages</span>
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">numpy</span>
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">cvxopt</span> <span style="color: #008000; font-weight: bold">import</span> matrix
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">cvxopt</span> <span style="color: #008000; font-weight: bold">import</span> solvers
<span style="color: #408080; font-style: italic"># Define QP parameters (directly)</span>
P <span style="color: #666666">=</span> matrix([[<span style="color: #666666">1.0</span>,<span style="color: #666666">0.0</span>],[<span style="color: #666666">0.0</span>,<span style="color: #666666">0.0</span>]])
q <span style="color: #666666">=</span> matrix([<span style="color: #666666">3.0</span>,<span style="color: #666666">4.0</span>])
G <span style="color: #666666">=</span> matrix([[<span style="color: #666666">-1.0</span>,<span style="color: #666666">0.0</span>,<span style="color: #666666">-1.0</span>,<span style="color: #666666">2.0</span>,<span style="color: #666666">3.0</span>],[<span style="color: #666666">0.0</span>,<span style="color: #666666">-1.0</span>,<span style="color: #666666">-3.0</span>,<span style="color: #666666">5.0</span>,<span style="color: #666666">4.0</span>]])
h <span style="color: #666666">=</span> matrix([<span style="color: #666666">0.0</span>,<span style="color: #666666">0.0</span>,<span style="color: #666666">-15.0</span>,<span style="color: #666666">100.0</span>,<span style="color: #666666">80.0</span>])
<span style="color: #408080; font-style: italic"># Define QP parameters (with NumPy)</span>
P <span style="color: #666666">=</span> matrix(numpy<span style="color: #666666">.</span>diag([<span style="color: #666666">1</span>,<span style="color: #666666">0</span>]), tc<span style="color: #666666">=</span>’d’)
q <span style="color: #666666">=</span> matrix(numpy<span style="color: #666666">.</span>array([<span style="color: #666666">3</span>,<span style="color: #666666">4</span>]), tc<span style="color: #666666">=</span>’d’)
G <span style="color: #666666">=</span> matrix(numpy<span style="color: #666666">.</span>array([[<span style="color: #666666">-1</span>,<span style="color: #666666">0</span>],[<span style="color: #666666">0</span>,<span style="color: #666666">-1</span>],[<span style="color: #666666">-1</span>,<span style="color: #666666">-3</span>],[<span style="color: #666666">2</span>,<span style="color: #666666">5</span>],[<span style="color: #666666">3</span>,<span style="color: #666666">4</span>]]), tc<span style="color: #666666">=</span>’d’)
h <span style="color: #666666">=</span> matrix(numpy<span style="color: #666666">.</span>array([<span style="color: #666666">0</span>,<span style="color: #666666">0</span>,<span style="color: #666666">-15</span>,<span style="color: #666666">100</span>,<span style="color: #666666">80</span>]), tc<span style="color: #666666">=</span>’d’)
<span style="color: #408080; font-style: italic"># Construct the QP, invoke solver</span>
sol <span style="color: #666666">=</span> solvers<span style="color: #666666">.</span>qp(P,q,G,h)
<span style="color: #408080; font-style: italic"># Extract optimal value and solution</span>
sol[’x’] <span style="color: #408080; font-style: italic"># [7.13e-07, 5.00e+00]</span>
sol[’primal objective’]
</pre></div>
<p>
doc/pub/svm/ipynb/ipynb-svm-src.tar.gz
BIN
View File
Binary file not shown.
doc/pub/svm/ipynb/svm.ipynb
+47 -2
View File
@@ -10,7 +10,7 @@
"<!-- Author: --> \n",
"**Morten Hjorth-Jensen**, Department of Physics, University of Oslo and Department of Physics and Astronomy and National Superconducting Cyclotron Laboratory, Michigan State University\n",
"\n",
"Date: **Nov 5, 2018**\n",
"Date: **Nov 6, 2018**\n",
"\n",
"Copyright 1999-2018, Morten Hjorth-Jensen. Released under CC Attribution-NonCommercial 4.0 license\n",
"\n",
@@ -1137,6 +1137,36 @@
"<!-- !split -->\n",
"## Mercer's theorem\n",
"\n",
"## Mathematical optimization of convex functions\n",
"\n",
"A mathematical optimization problem, or just optimization problem, has the form"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"$$\n",
"\\mathrm{minimize}\\hspace{0.1cm} f(x),\n",
"$$"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"subject to some constraints $g(\\lambda_i) \\leq b_i$ for say a selected set $i=1,2,\\dots, n$.\n",
"In our case we are optimizing with respect to the Lagrangian multipliers $\\lambda_i$, and the\n",
"vector $\\boldsymbol{\\lambda}=[\\lambda_1, \\lambda_2,\\dots, \\lambda_n]$ is the optimization variable we are dealing with.\n",
"and $f(x)$ is our objective function while $g(\\lambda_i) \\leq b_i$ represents our constraint function.\n",
"\n",
"In our case we are particularly interested in a class of optimization problems called convex optmization problems. \n",
"In our disussion on gradient descent methods we discussed at length the definition of a convex function. \n",
"\n",
"Convex optimization problems play a central role in applied mathematics and we recommend strongly [Boyd and Vandenberghe's text on the topics](http://web.stanford.edu/~boyd/cvxbook/).\n",
"\n",
"\n",
"\n",
"## How do we solve these problems\n",
"\n",
"If we use Python as programming language and wish to venture beyond\n",
@@ -1209,10 +1239,25 @@
},
"outputs": [],
"source": [
"# Import the necessary packages\n",
"import numpy\n",
"from cvxopt import matrix\n",
"from cvxopt import solvers\n",
"# Define QP parameters (directly)\n",
"P = matrix([[1.0,0.0],[0.0,0.0]])\n",
"q = matrix([3.0,4.0])\n",
"G = matrix([[-1.0,0.0,-1.0,2.0,3.0],[0.0,-1.0,-3.0,5.0,4.0]])\n",
"h = matrix([0.0,0.0,-15.0,100.0,80.0])"
"h = matrix([0.0,0.0,-15.0,100.0,80.0])\n",
"# Define QP parameters (with NumPy)\n",
"P = matrix(numpy.diag([1,0]), tc=’d’)\n",
"q = matrix(numpy.array([3,4]), tc=’d’)\n",
"G = matrix(numpy.array([[-1,0],[0,-1],[-1,-3],[2,5],[3,4]]), tc=’d’)\n",
"h = matrix(numpy.array([0,0,-15,100,80]), tc=’d’)\n",
"# Construct the QP, invoke solver\n",
"sol = solvers.qp(P,q,G,h)\n",
"# Extract optimal value and solution\n",
"sol[’x’] # [7.13e-07, 5.00e+00]\n",
"sol[’primal objective’]"
]
}
],
doc/pub/svm/pdf/svm-minted.pdf
BIN
View File
Binary file not shown.
doc/src/SupportVMachines/svm.do.txt
+36
View File
@@ -583,6 +583,27 @@ from which we also find $b$.
!split
===== Mercer's theorem =====
!split
===== Mathematical optimization of convex functions =====
A mathematical optimization problem, or just optimization problem, has the form
!bt
\[
\mathrm{minimize}\hspace{0.1cm} f(x),
\]
!et
subject to some constraints $g(\lambda_i) \leq b_i$ for say a selected set $i=1,2,\dots, n$.
In our case we are optimizing with respect to the Lagrangian multipliers $\lambda_i$, and the
vector $\bm{\lambda}=[\lambda_1, \lambda_2,\dots, \lambda_n]$ is the optimization variable we are dealing with.
and $f(x)$ is our objective function while $g(\lambda_i) \leq b_i$ represents our constraint function.
In our case we are particularly interested in a class of optimization problems called convex optmization problems.
In our disussion on gradient descent methods we discussed at length the definition of a convex function.
Convex optimization problems play a central role in applied mathematics and we recommend strongly "Boyd and Vandenberghe's text on the topics":"http://web.stanford.edu/~boyd/cvxbook/".
!split
===== How do we solve these problems =====
@@ -625,8 +646,23 @@ object; all arguments given to its solvers must be in this matrix type. There ar
to do this. The first is to define the matrix directly with (potentially nested) lists:
from cvxopt import matrix
!bc pycod
# Import the necessary packages
import numpy
from cvxopt import matrix
from cvxopt import solvers
# Define QP parameters (directly)
P = matrix([[1.0,0.0],[0.0,0.0]])
q = matrix([3.0,4.0])
G = matrix([[-1.0,0.0,-1.0,2.0,3.0],[0.0,-1.0,-3.0,5.0,4.0]])
h = matrix([0.0,0.0,-15.0,100.0,80.0])
# Define QP parameters (with NumPy)
P = matrix(numpy.diag([1,0]), tc=’d’)
q = matrix(numpy.array([3,4]), tc=’d’)
G = matrix(numpy.array([[-1,0],[0,-1],[-1,-3],[2,5],[3,4]]), tc=’d’)
h = matrix(numpy.array([0,0,-15,100,80]), tc=’d’)
# Construct the QP, invoke solver
sol = solvers.qp(P,q,G,h)
# Extract optimal value and solution
sol[’x’] # [7.13e-07, 5.00e+00]
sol[’primal objective’]
!ec