lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
5c6c05efbe058f033f3f35def695f64a7c70b9ce
FYS-STK4155/doc/pub/LogReg/html/._LogReg-bs022.html
T
mhjensen 05f321f985 update on neural networks
2018-10-18 05:50:28 +02:00

239 lines
12 KiB
HTML
Raw Blame History

<!--
Automatically generated HTML file from DocOnce source
(https://github.com/hplgit/doconce/)
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="description" content="Data Analysis and Machine Learning: Logistic Regression">
<title>Data Analysis and Machine Learning: Logistic Regression</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<!-- not necessary
<link href="https://netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
-->
<style type="text/css">
/* Add scrollbar to dropdown menus in bootstrap navigation bar */
.dropdown-menu {
height: auto;
max-height: 400px;
overflow-x: hidden;
}
/* Adds an invisible element before each target to offset for the navigation
bar */
.anchor::before {
content:"";
display:block;
height:50px; /* fixed header height for style bootstrap */
margin:-50px 0 0; /* negative fixed header height */
}
</style>
</head>
<!-- tocinfo
{'highest level': 2,
'sections': [('Logistic Regression', 2, None, '___sec0'),
('Optimization and Deep learning', 2, None, '___sec1'),
('Basics', 2, None, '___sec2'),
('Linear classifier', 2, None, '___sec3'),
('Some selected properties', 2, None, '___sec4'),
('The logistic function', 2, None, '___sec5'),
('Two parameters', 2, None, '___sec6'),
('Maximum likelihood', 2, None, '___sec7'),
('The cost function rewritten', 2, None, '___sec8'),
('Minimizing the cross entropy', 2, None, '___sec9'),
('A more compact expression', 2, None, '___sec10'),
('Extending to more predictors', 2, None, '___sec11'),
('Including more classes', 2, None, '___sec12'),
('The Softmax function', 2, None, '___sec13'),
('A _scikit-learn_ example', 2, None, '___sec14'),
('A simple classification problem', 2, None, '___sec15'),
('The two-dimensional Ising model, Predicting phase transition '
'of the two-dimensional Ising model',
2,
None,
'___sec16'),
('Reading in the data', 2, None, '___sec17'),
('Logistic regression', 2, None, '___sec18'),
('Exploring the logistic regression', 2, None, '___sec19'),
('Accuracy of a classification model', 2, None, '___sec20'),
('Analyzing the results', 2, None, '___sec21')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "none" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-responsive-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="LogReg-bs.html">Data Analysis and Machine Learning: Logistic Regression</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._LogReg-bs001.html#___sec0" style="font-size: 80%;">Logistic Regression</a></li>
<!-- navigation toc: --> <li><a href="._LogReg-bs002.html#___sec1" style="font-size: 80%;">Optimization and Deep learning</a></li>
<!-- navigation toc: --> <li><a href="._LogReg-bs003.html#___sec2" style="font-size: 80%;">Basics</a></li>
<!-- navigation toc: --> <li><a href="._LogReg-bs004.html#___sec3" style="font-size: 80%;">Linear classifier</a></li>
<!-- navigation toc: --> <li><a href="._LogReg-bs005.html#___sec4" style="font-size: 80%;">Some selected properties</a></li>
<!-- navigation toc: --> <li><a href="._LogReg-bs006.html#___sec5" style="font-size: 80%;">The logistic function</a></li>
<!-- navigation toc: --> <li><a href="._LogReg-bs007.html#___sec6" style="font-size: 80%;">Two parameters</a></li>
<!-- navigation toc: --> <li><a href="._LogReg-bs008.html#___sec7" style="font-size: 80%;">Maximum likelihood</a></li>
<!-- navigation toc: --> <li><a href="._LogReg-bs009.html#___sec8" style="font-size: 80%;">The cost function rewritten</a></li>
<!-- navigation toc: --> <li><a href="._LogReg-bs010.html#___sec9" style="font-size: 80%;">Minimizing the cross entropy</a></li>
<!-- navigation toc: --> <li><a href="._LogReg-bs011.html#___sec10" style="font-size: 80%;">A more compact expression</a></li>
<!-- navigation toc: --> <li><a href="._LogReg-bs012.html#___sec11" style="font-size: 80%;">Extending to more predictors</a></li>
<!-- navigation toc: --> <li><a href="._LogReg-bs013.html#___sec12" style="font-size: 80%;">Including more classes</a></li>
<!-- navigation toc: --> <li><a href="._LogReg-bs014.html#___sec13" style="font-size: 80%;">The Softmax function</a></li>
<!-- navigation toc: --> <li><a href="._LogReg-bs015.html#___sec14" style="font-size: 80%;">A <b>scikit-learn</b> example</a></li>
<!-- navigation toc: --> <li><a href="._LogReg-bs016.html#___sec15" style="font-size: 80%;">A simple classification problem</a></li>
<!-- navigation toc: --> <li><a href="._LogReg-bs017.html#___sec16" style="font-size: 80%;">The two-dimensional Ising model, Predicting phase transition of the two-dimensional Ising model</a></li>
<!-- navigation toc: --> <li><a href="._LogReg-bs018.html#___sec17" style="font-size: 80%;">Reading in the data</a></li>
<!-- navigation toc: --> <li><a href="._LogReg-bs019.html#___sec18" style="font-size: 80%;">Logistic regression</a></li>
<!-- navigation toc: --> <li><a href="._LogReg-bs020.html#___sec19" style="font-size: 80%;">Exploring the logistic regression</a></li>
<!-- navigation toc: --> <li><a href="._LogReg-bs021.html#___sec20" style="font-size: 80%;">Accuracy of a classification model</a></li>
<!-- navigation toc: --> <li><a href="#___sec21" style="font-size: 80%;">Analyzing the results</a></li>
</ul>
</li>
</ul>
</div>
</div>
</div> <!-- end of navigation bar -->
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<a name="part0022"></a>
<!-- !split -->
<h2 id="___sec21" class="anchor">Analyzing the results </h2>
<p>
Below we show a different metric for determining the quality of our
model, namely the <b>reciever operating characteristic</b> (ROC). The ROC
curve tells us how well the model correctly classifies the different
labels. We plot the <b>true positive rate</b> (the rate of predicted
positive classes that are positive) versus the <b>false positive rate</b>
(the rate of predicted positive classes that are negative). The ROC
curve is built by computing the true positive rate and the false
positive rate for varying <b>thresholds</b>, i.e, which probability we
should acredit a certain class.
<p>
By computing the <b>area under the curve</b> (AUC) of the ROC curve we get an estimate of how well our model is performing. Pure guessing will get an AUC of \( 0.5 \). A perfect score will get an AUC of \( 1.0 \).
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span>fig <span style="color: #666666">=</span> plt<span style="color: #666666">.</span>figure(figsize<span style="color: #666666">=</span>(<span style="color: #666666">20</span>, <span style="color: #666666">14</span>))
<span style="color: #008000; font-weight: bold">for</span> (_X, _y), label <span style="color: #AA22FF; font-weight: bold">in</span> <span style="color: #008000">zip</span>(
[
(X_train, y_train),
(X_test, y_test),
(data[critical], labels[critical])
],
[<span style="color: #BA2121">&quot;Train&quot;</span>, <span style="color: #BA2121">&quot;Test&quot;</span>, <span style="color: #BA2121">&quot;Critical&quot;</span>]
):
proba <span style="color: #666666">=</span> clf<span style="color: #666666">.</span>predict_proba(_X)
fpr, tpr, _ <span style="color: #666666">=</span> skm<span style="color: #666666">.</span>roc_curve(_y, proba[:, <span style="color: #666666">1</span>])
roc_auc <span style="color: #666666">=</span> skm<span style="color: #666666">.</span>auc(fpr, tpr)
<span style="color: #008000; font-weight: bold">print</span> (<span style="color: #BA2121">&quot;LogisticRegression AUC ({0}): {1}&quot;</span><span style="color: #666666">.</span>format(label, roc_auc))
plt<span style="color: #666666">.</span>plot(fpr, tpr, label<span style="color: #666666">=</span><span style="color: #BA2121">&quot;{0} (AUC = {1})&quot;</span><span style="color: #666666">.</span>format(label, roc_auc), linewidth<span style="color: #666666">=4.0</span>)
plt<span style="color: #666666">.</span>plot([<span style="color: #666666">0</span>, <span style="color: #666666">1</span>], [<span style="color: #666666">0</span>, <span style="color: #666666">1</span>], <span style="color: #BA2121">&quot;--&quot;</span>, label<span style="color: #666666">=</span><span style="color: #BA2121">&quot;Guessing (AUC = 0.5)&quot;</span>, linewidth<span style="color: #666666">=4.0</span>)
plt<span style="color: #666666">.</span>title(<span style="color: #BA2121">r&quot;The ROC curve for LogisticRegression&quot;</span>, fontsize<span style="color: #666666">=18</span>)
plt<span style="color: #666666">.</span>xlabel(<span style="color: #BA2121">r&quot;False positive rate&quot;</span>, fontsize<span style="color: #666666">=18</span>)
plt<span style="color: #666666">.</span>ylabel(<span style="color: #BA2121">r&quot;True positive rate&quot;</span>, fontsize<span style="color: #666666">=18</span>)
plt<span style="color: #666666">.</span>axis([<span style="color: #666666">-0.01</span>, <span style="color: #666666">1.01</span>, <span style="color: #666666">-0.01</span>, <span style="color: #666666">1.01</span>])
plt<span style="color: #666666">.</span>xticks(fontsize<span style="color: #666666">=18</span>)
plt<span style="color: #666666">.</span>yticks(fontsize<span style="color: #666666">=18</span>)
plt<span style="color: #666666">.</span>legend(loc<span style="color: #666666">=</span><span style="color: #BA2121">&quot;best&quot;</span>, fontsize<span style="color: #666666">=18</span>)
plt<span style="color: #666666">.</span>show()
</pre></div>
<p>
We can see that this plot of the ROC looks very strange. This tells us
that logistic regression is quite inept at predicting the Ising model
transition and is therefore highly non-linear. The ROC curve for the
training data looks quite good, but as the testing data is so far off
we see that we are dealing with an overfit model.
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
<li><a href="._LogReg-bs021.html">&laquo;</a></li>
<li><a href="._LogReg-bs000.html">1</a></li>
<li><a href="">...</a></li>
<li><a href="._LogReg-bs014.html">15</a></li>
<li><a href="._LogReg-bs015.html">16</a></li>
<li><a href="._LogReg-bs016.html">17</a></li>
<li><a href="._LogReg-bs017.html">18</a></li>
<li><a href="._LogReg-bs018.html">19</a></li>
<li><a href="._LogReg-bs019.html">20</a></li>
<li><a href="._LogReg-bs020.html">21</a></li>
<li><a href="._LogReg-bs021.html">22</a></li>
<li class="active"><a href="._LogReg-bs022.html">23</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
</div> <!-- end container -->
<!-- include javascript, jQuery *first* -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script src="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/js/bootstrap.min.js"></script>
<!-- Bootstrap footer
<footer>
<a href="http://..."><img width="250" align=right src="http://..."></a>
</footer>
-->
<center style="font-size:80%">
<!-- copyright only on the titlepage -->
</center>
</body>
</html>
Reference in New Issue View Git Blame Copy Permalink