lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

updating week 46

Browse Source
This commit is contained in:
mhjensen
2020-11-08 22:48:11 +01:00
parent 3e13bb0d64
commit 489c98a82f
34 changed files with 2834 additions and 2713 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/pub/week46/html/._week46-bs002.html
+83 -135
View File
@@ -7,9 +7,9 @@ Automatically generated HTML file from DocOnce source
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<meta name="description" content="Week 46: Support Vector Machines">
<meta name="description" content="Week 46: Gradient Boosting Summary and Support Vector Machines">
<title>Week 46: Support Vector Machines</title>
<title>Week 46: Gradient Boosting Summary and Support Vector Machines</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
@@ -41,39 +41,40 @@ Automatically generated HTML file from DocOnce source
<!-- tocinfo
{'highest level': 2,
'sections': [('Support Vector Machines, overarching aims', 2, None, '___sec0'),
('Hyperplanes and all that', 2, None, '___sec1'),
('What is a hyperplane?', 2, None, '___sec2'),
('A $p$-dimensional space of features', 2, None, '___sec3'),
('The two-dimensional case', 2, None, '___sec4'),
('Getting into the details', 2, None, '___sec5'),
('First attempt at a minimization approach', 2, None, '___sec6'),
('Solving the equations', 2, None, '___sec7'),
('Code Example', 2, None, '___sec8'),
('Problems with the Simpler Approach', 2, None, '___sec9'),
('A better approach', 2, None, '___sec10'),
'sections': [('Overview of week 46', 2, None, '___sec0'),
('Support Vector Machines, overarching aims', 2, None, '___sec1'),
('Hyperplanes and all that', 2, None, '___sec2'),
('What is a hyperplane?', 2, None, '___sec3'),
('A $p$-dimensional space of features', 2, None, '___sec4'),
('The two-dimensional case', 2, None, '___sec5'),
('Getting into the details', 2, None, '___sec6'),
('First attempt at a minimization approach', 2, None, '___sec7'),
('Solving the equations', 2, None, '___sec8'),
('Code Example', 2, None, '___sec9'),
('Problems with the Simpler Approach', 2, None, '___sec10'),
('A better approach', 2, None, '___sec11'),
('A quick Reminder on Lagrangian Multipliers',
2,
None,
'___sec11'),
('Adding the Multiplier', 2, None, '___sec12'),
('Setting up the Problem', 2, None, '___sec13'),
('The problem to solve', 2, None, '___sec14'),
('The last steps', 2, None, '___sec15'),
('A soft classifier', 2, None, '___sec16'),
('Soft optmization problem', 2, None, '___sec17'),
('Kernels and non-linearity', 2, None, '___sec18'),
('The equations', 2, None, '___sec19'),
('The problem to solve', 2, None, '___sec20'),
("Different kernels and Mercer's theorem", 2, None, '___sec21'),
('The moons example', 2, None, '___sec22'),
'___sec12'),
('Adding the Multiplier', 2, None, '___sec13'),
('Setting up the Problem', 2, None, '___sec14'),
('The problem to solve', 2, None, '___sec15'),
('The last steps', 2, None, '___sec16'),
('A soft classifier', 2, None, '___sec17'),
('Soft optmization problem', 2, None, '___sec18'),
('Kernels and non-linearity', 2, None, '___sec19'),
('The equations', 2, None, '___sec20'),
('The problem to solve', 2, None, '___sec21'),
("Different kernels and Mercer's theorem", 2, None, '___sec22'),
('The moons example', 2, None, '___sec23'),
('Mathematical optimization of convex functions',
2,
None,
'___sec23'),
('How do we solve these problems?', 2, None, '___sec24'),
('A simple example', 2, None, '___sec25'),
('Back to the more realistic cases', 2, None, '___sec26')]}
'___sec24'),
('How do we solve these problems?', 2, None, '___sec25'),
('A simple example', 2, None, '___sec26'),
('Back to the more realistic cases', 2, None, '___sec27')]}
end of tocinfo -->
<body>
@@ -103,7 +104,7 @@ MathJax.Hub.Config({
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="week46-bs.html">Week 46: Support Vector Machines</a>
<a class="navbar-brand" href="week46-bs.html">Week 46: Gradient Boosting Summary and Support Vector Machines</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
@@ -111,33 +112,34 @@ MathJax.Hub.Config({
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._week46-bs001.html#___sec0" style="font-size: 80%;">Support Vector Machines, overarching aims</a></li>
<!-- navigation toc: --> <li><a href="#___sec1" style="font-size: 80%;">Hyperplanes and all that</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs003.html#___sec2" style="font-size: 80%;">What is a hyperplane?</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs004.html#___sec3" style="font-size: 80%;">A \( p \)-dimensional space of features</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs005.html#___sec4" style="font-size: 80%;">The two-dimensional case</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs006.html#___sec5" style="font-size: 80%;">Getting into the details</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs007.html#___sec6" style="font-size: 80%;">First attempt at a minimization approach</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs008.html#___sec7" style="font-size: 80%;">Solving the equations</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs009.html#___sec8" style="font-size: 80%;">Code Example</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs010.html#___sec9" style="font-size: 80%;">Problems with the Simpler Approach</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs011.html#___sec10" style="font-size: 80%;">A better approach</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs012.html#___sec11" style="font-size: 80%;">A quick Reminder on Lagrangian Multipliers</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs013.html#___sec12" style="font-size: 80%;">Adding the Multiplier</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs014.html#___sec13" style="font-size: 80%;">Setting up the Problem</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs015.html#___sec14" style="font-size: 80%;">The problem to solve</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs016.html#___sec15" style="font-size: 80%;">The last steps</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs017.html#___sec16" style="font-size: 80%;">A soft classifier</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs018.html#___sec17" style="font-size: 80%;">Soft optmization problem</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs019.html#___sec18" style="font-size: 80%;">Kernels and non-linearity</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs020.html#___sec19" style="font-size: 80%;">The equations</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs021.html#___sec20" style="font-size: 80%;">The problem to solve</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs022.html#___sec21" style="font-size: 80%;">Different kernels and Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs023.html#___sec22" style="font-size: 80%;">The moons example</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs024.html#___sec23" style="font-size: 80%;">Mathematical optimization of convex functions</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs025.html#___sec24" style="font-size: 80%;">How do we solve these problems?</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs026.html#___sec25" style="font-size: 80%;">A simple example</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs027.html#___sec26" style="font-size: 80%;">Back to the more realistic cases</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs001.html#___sec0" style="font-size: 80%;">Overview of week 46</a></li>
<!-- navigation toc: --> <li><a href="#___sec1" style="font-size: 80%;">Support Vector Machines, overarching aims</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs003.html#___sec2" style="font-size: 80%;">Hyperplanes and all that</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs004.html#___sec3" style="font-size: 80%;">What is a hyperplane?</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs005.html#___sec4" style="font-size: 80%;">A \( p \)-dimensional space of features</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs006.html#___sec5" style="font-size: 80%;">The two-dimensional case</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs007.html#___sec6" style="font-size: 80%;">Getting into the details</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs008.html#___sec7" style="font-size: 80%;">First attempt at a minimization approach</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs009.html#___sec8" style="font-size: 80%;">Solving the equations</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs010.html#___sec9" style="font-size: 80%;">Code Example</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs011.html#___sec10" style="font-size: 80%;">Problems with the Simpler Approach</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs012.html#___sec11" style="font-size: 80%;">A better approach</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs013.html#___sec12" style="font-size: 80%;">A quick Reminder on Lagrangian Multipliers</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs014.html#___sec13" style="font-size: 80%;">Adding the Multiplier</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs015.html#___sec14" style="font-size: 80%;">Setting up the Problem</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs016.html#___sec15" style="font-size: 80%;">The problem to solve</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs017.html#___sec16" style="font-size: 80%;">The last steps</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs018.html#___sec17" style="font-size: 80%;">A soft classifier</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs019.html#___sec18" style="font-size: 80%;">Soft optmization problem</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs020.html#___sec19" style="font-size: 80%;">Kernels and non-linearity</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs021.html#___sec20" style="font-size: 80%;">The equations</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs022.html#___sec21" style="font-size: 80%;">The problem to solve</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs023.html#___sec22" style="font-size: 80%;">Different kernels and Mercer's theorem</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs024.html#___sec23" style="font-size: 80%;">The moons example</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs025.html#___sec24" style="font-size: 80%;">Mathematical optimization of convex functions</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs026.html#___sec25" style="font-size: 80%;">How do we solve these problems?</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs027.html#___sec26" style="font-size: 80%;">A simple example</a></li>
<!-- navigation toc: --> <li><a href="._week46-bs028.html#___sec27" style="font-size: 80%;">Back to the more realistic cases</a></li>
</ul>
</li>
@@ -153,90 +155,36 @@ MathJax.Hub.Config({
<a name="part0002"></a>
<!-- !split -->
<h2 id="___sec1" class="anchor">Hyperplanes and all that </h2>
<h2 id="___sec1" class="anchor">Support Vector Machines, overarching aims </h2>
<p>
The theory behind support vector machines (SVM hereafter) is based on
the mathematical description of so-called hyperplanes. Let us start
with a two-dimensional case. This will also allow us to introduce our
first SVM examples. These will be tailored to the case of two specific
classes, as displayed in the figure here based on the usage of the petal data.
A Support Vector Machine (SVM) is a very powerful and versatile
Machine Learning method, capable of performing linear or nonlinear
classification, regression, and even outlier detection. It is one of
the most popular models in Machine Learning, and anyone interested in
Machine Learning should have it in their toolbox. SVMs are
particularly well suited for classification of complex but small-sized or
medium-sized datasets.
<p>
We assume here that our data set can be well separated into two
domains, where a straight line does the job in the separating the two
classes. Here the two classes are represented by either squares or
circles.
The case with two well-separated classes only can be understood in an
intuitive way in terms of lines in a two-dimensional space separating
the two classes (see figure below).
<p>
The basic mathematics behind the SVM is however less familiar to most of us.
It relies on the definition of hyperplanes and the
definition of a <b>margin</b> which separates classes (in case of
classification problems) of variables. It is also used for regression
problems.
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn</span> <span style="color: #008000; font-weight: bold">import</span> datasets
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.svm</span> <span style="color: #008000; font-weight: bold">import</span> SVC, LinearSVC
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.linear_model</span> <span style="color: #008000; font-weight: bold">import</span> SGDClassifier
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.preprocessing</span> <span style="color: #008000; font-weight: bold">import</span> StandardScaler
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">matplotlib</span>
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">matplotlib.pyplot</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">plt</span>
plt<span style="color: #666666">.</span>rcParams[<span style="color: #BA2121">&#39;axes.labelsize&#39;</span>] <span style="color: #666666">=</span> <span style="color: #666666">14</span>
plt<span style="color: #666666">.</span>rcParams[<span style="color: #BA2121">&#39;xtick.labelsize&#39;</span>] <span style="color: #666666">=</span> <span style="color: #666666">12</span>
plt<span style="color: #666666">.</span>rcParams[<span style="color: #BA2121">&#39;ytick.labelsize&#39;</span>] <span style="color: #666666">=</span> <span style="color: #666666">12</span>
<p>
With SVMs we distinguish between hard margin and soft margins. The
latter introduces a so-called softening parameter to be discussed
below. We distinguish also between linear and non-linear
approaches. The latter are the most frequent ones since it is rather
unlikely that we can separate classes easily by say straight lines.
iris <span style="color: #666666">=</span> datasets<span style="color: #666666">.</span>load_iris()
X <span style="color: #666666">=</span> iris[<span style="color: #BA2121">&quot;data&quot;</span>][:, (<span style="color: #666666">2</span>, <span style="color: #666666">3</span>)] <span style="color: #408080; font-style: italic"># petal length, petal width</span>
y <span style="color: #666666">=</span> iris[<span style="color: #BA2121">&quot;target&quot;</span>]
setosa_or_versicolor <span style="color: #666666">=</span> (y <span style="color: #666666">==</span> <span style="color: #666666">0</span>) <span style="color: #666666">|</span> (y <span style="color: #666666">==</span> <span style="color: #666666">1</span>)
X <span style="color: #666666">=</span> X[setosa_or_versicolor]
y <span style="color: #666666">=</span> y[setosa_or_versicolor]
C <span style="color: #666666">=</span> <span style="color: #666666">5</span>
alpha <span style="color: #666666">=</span> <span style="color: #666666">1</span> <span style="color: #666666">/</span> (C <span style="color: #666666">*</span> <span style="color: #008000">len</span>(X))
lin_clf <span style="color: #666666">=</span> LinearSVC(loss<span style="color: #666666">=</span><span style="color: #BA2121">&quot;hinge&quot;</span>, C<span style="color: #666666">=</span>C, random_state<span style="color: #666666">=42</span>)
svm_clf <span style="color: #666666">=</span> SVC(kernel<span style="color: #666666">=</span><span style="color: #BA2121">&quot;linear&quot;</span>, C<span style="color: #666666">=</span>C)
sgd_clf <span style="color: #666666">=</span> SGDClassifier(loss<span style="color: #666666">=</span><span style="color: #BA2121">&quot;hinge&quot;</span>, learning_rate<span style="color: #666666">=</span><span style="color: #BA2121">&quot;constant&quot;</span>, eta0<span style="color: #666666">=0.001</span>, alpha<span style="color: #666666">=</span>alpha,
max_iter<span style="color: #666666">=100000</span>, random_state<span style="color: #666666">=42</span>)
scaler <span style="color: #666666">=</span> StandardScaler()
X_scaled <span style="color: #666666">=</span> scaler<span style="color: #666666">.</span>fit_transform(X)
lin_clf<span style="color: #666666">.</span>fit(X_scaled, y)
svm_clf<span style="color: #666666">.</span>fit(X_scaled, y)
sgd_clf<span style="color: #666666">.</span>fit(X_scaled, y)
<span style="color: #008000; font-weight: bold">print</span>(<span style="color: #BA2121">&quot;LinearSVC: &quot;</span>, lin_clf<span style="color: #666666">.</span>intercept_, lin_clf<span style="color: #666666">.</span>coef_)
<span style="color: #008000; font-weight: bold">print</span>(<span style="color: #BA2121">&quot;SVC: &quot;</span>, svm_clf<span style="color: #666666">.</span>intercept_, svm_clf<span style="color: #666666">.</span>coef_)
<span style="color: #008000; font-weight: bold">print</span>(<span style="color: #BA2121">&quot;SGDClassifier(alpha={:.5f}):&quot;</span><span style="color: #666666">.</span>format(sgd_clf<span style="color: #666666">.</span>alpha), sgd_clf<span style="color: #666666">.</span>intercept_, sgd_clf<span style="color: #666666">.</span>coef_)
<span style="color: #408080; font-style: italic"># Compute the slope and bias of each decision boundary</span>
w1 <span style="color: #666666">=</span> <span style="color: #666666">-</span>lin_clf<span style="color: #666666">.</span>coef_[<span style="color: #666666">0</span>, <span style="color: #666666">0</span>]<span style="color: #666666">/</span>lin_clf<span style="color: #666666">.</span>coef_[<span style="color: #666666">0</span>, <span style="color: #666666">1</span>]
b1 <span style="color: #666666">=</span> <span style="color: #666666">-</span>lin_clf<span style="color: #666666">.</span>intercept_[<span style="color: #666666">0</span>]<span style="color: #666666">/</span>lin_clf<span style="color: #666666">.</span>coef_[<span style="color: #666666">0</span>, <span style="color: #666666">1</span>]
w2 <span style="color: #666666">=</span> <span style="color: #666666">-</span>svm_clf<span style="color: #666666">.</span>coef_[<span style="color: #666666">0</span>, <span style="color: #666666">0</span>]<span style="color: #666666">/</span>svm_clf<span style="color: #666666">.</span>coef_[<span style="color: #666666">0</span>, <span style="color: #666666">1</span>]
b2 <span style="color: #666666">=</span> <span style="color: #666666">-</span>svm_clf<span style="color: #666666">.</span>intercept_[<span style="color: #666666">0</span>]<span style="color: #666666">/</span>svm_clf<span style="color: #666666">.</span>coef_[<span style="color: #666666">0</span>, <span style="color: #666666">1</span>]
w3 <span style="color: #666666">=</span> <span style="color: #666666">-</span>sgd_clf<span style="color: #666666">.</span>coef_[<span style="color: #666666">0</span>, <span style="color: #666666">0</span>]<span style="color: #666666">/</span>sgd_clf<span style="color: #666666">.</span>coef_[<span style="color: #666666">0</span>, <span style="color: #666666">1</span>]
b3 <span style="color: #666666">=</span> <span style="color: #666666">-</span>sgd_clf<span style="color: #666666">.</span>intercept_[<span style="color: #666666">0</span>]<span style="color: #666666">/</span>sgd_clf<span style="color: #666666">.</span>coef_[<span style="color: #666666">0</span>, <span style="color: #666666">1</span>]
<span style="color: #408080; font-style: italic"># Transform the decision boundary lines back to the original scale</span>
line1 <span style="color: #666666">=</span> scaler<span style="color: #666666">.</span>inverse_transform([[<span style="color: #666666">-10</span>, <span style="color: #666666">-10</span> <span style="color: #666666">*</span> w1 <span style="color: #666666">+</span> b1], [<span style="color: #666666">10</span>, <span style="color: #666666">10</span> <span style="color: #666666">*</span> w1 <span style="color: #666666">+</span> b1]])
line2 <span style="color: #666666">=</span> scaler<span style="color: #666666">.</span>inverse_transform([[<span style="color: #666666">-10</span>, <span style="color: #666666">-10</span> <span style="color: #666666">*</span> w2 <span style="color: #666666">+</span> b2], [<span style="color: #666666">10</span>, <span style="color: #666666">10</span> <span style="color: #666666">*</span> w2 <span style="color: #666666">+</span> b2]])
line3 <span style="color: #666666">=</span> scaler<span style="color: #666666">.</span>inverse_transform([[<span style="color: #666666">-10</span>, <span style="color: #666666">-10</span> <span style="color: #666666">*</span> w3 <span style="color: #666666">+</span> b3], [<span style="color: #666666">10</span>, <span style="color: #666666">10</span> <span style="color: #666666">*</span> w3 <span style="color: #666666">+</span> b3]])
<span style="color: #408080; font-style: italic"># Plot all three decision boundaries</span>
plt<span style="color: #666666">.</span>figure(figsize<span style="color: #666666">=</span>(<span style="color: #666666">11</span>, <span style="color: #666666">4</span>))
plt<span style="color: #666666">.</span>plot(line1[:, <span style="color: #666666">0</span>], line1[:, <span style="color: #666666">1</span>], <span style="color: #BA2121">&quot;k:&quot;</span>, label<span style="color: #666666">=</span><span style="color: #BA2121">&quot;LinearSVC&quot;</span>)
plt<span style="color: #666666">.</span>plot(line2[:, <span style="color: #666666">0</span>], line2[:, <span style="color: #666666">1</span>], <span style="color: #BA2121">&quot;b--&quot;</span>, linewidth<span style="color: #666666">=2</span>, label<span style="color: #666666">=</span><span style="color: #BA2121">&quot;SVC&quot;</span>)
plt<span style="color: #666666">.</span>plot(line3[:, <span style="color: #666666">0</span>], line3[:, <span style="color: #666666">1</span>], <span style="color: #BA2121">&quot;r-&quot;</span>, label<span style="color: #666666">=</span><span style="color: #BA2121">&quot;SGDClassifier&quot;</span>)
plt<span style="color: #666666">.</span>plot(X[:, <span style="color: #666666">0</span>][y<span style="color: #666666">==1</span>], X[:, <span style="color: #666666">1</span>][y<span style="color: #666666">==1</span>], <span style="color: #BA2121">&quot;bs&quot;</span>) <span style="color: #408080; font-style: italic"># label=&quot;Iris-Versicolor&quot;</span>
plt<span style="color: #666666">.</span>plot(X[:, <span style="color: #666666">0</span>][y<span style="color: #666666">==0</span>], X[:, <span style="color: #666666">1</span>][y<span style="color: #666666">==0</span>], <span style="color: #BA2121">&quot;yo&quot;</span>) <span style="color: #408080; font-style: italic"># label=&quot;Iris-Setosa&quot;</span>
plt<span style="color: #666666">.</span>xlabel(<span style="color: #BA2121">&quot;Petal length&quot;</span>, fontsize<span style="color: #666666">=14</span>)
plt<span style="color: #666666">.</span>ylabel(<span style="color: #BA2121">&quot;Petal width&quot;</span>, fontsize<span style="color: #666666">=14</span>)
plt<span style="color: #666666">.</span>legend(loc<span style="color: #666666">=</span><span style="color: #BA2121">&quot;upper center&quot;</span>, fontsize<span style="color: #666666">=14</span>)
plt<span style="color: #666666">.</span>axis([<span style="color: #666666">0</span>, <span style="color: #666666">5.5</span>, <span style="color: #666666">0</span>, <span style="color: #666666">2</span>])
plt<span style="color: #666666">.</span>show()
</pre></div>
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
@@ -255,7 +203,7 @@ plt<span style="color: #666666">.</span>show()
<li><a href="._week46-bs010.html">11</a></li>
<li><a href="._week46-bs011.html">12</a></li>
<li><a href="">...</a></li>
<li><a href="._week46-bs027.html">28</a></li>
<li><a href="._week46-bs028.html">29</a></li>
<li><a href="._week46-bs003.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->