lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
4f4c4e69464b1b4bc304ff84c9054e5ffdb7a4ee
FYS-STK4155/doc/pub/LogReg/html/._LogReg-bs014.html
T
mhjensen 4f4c4e6946 update on log reg
2018-10-11 05:59:30 +02:00

222 lines
9.9 KiB
HTML
Raw Blame History

<!--
Automatically generated HTML file from DocOnce source
(https://github.com/hplgit/doconce/)
-->
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="DocOnce: https://github.com/hplgit/doconce/" />
<meta name="description" content="Data Analysis and Machine Learning: Logistic Regression">
<title>Data Analysis and Machine Learning: Logistic Regression</title>
<!-- Bootstrap style: bootstrap -->
<link href="https://netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css" rel="stylesheet">
<!-- not necessary
<link href="https://netdna.bootstrapcdn.com/font-awesome/4.0.3/css/font-awesome.css" rel="stylesheet">
-->
<style type="text/css">
/* Add scrollbar to dropdown menus in bootstrap navigation bar */
.dropdown-menu {
height: auto;
max-height: 400px;
overflow-x: hidden;
}
/* Adds an invisible element before each target to offset for the navigation
bar */
.anchor::before {
content:"";
display:block;
height:50px; /* fixed header height for style bootstrap */
margin:-50px 0 0; /* negative fixed header height */
}
</style>
</head>
<!-- tocinfo
{'highest level': 2,
'sections': [('Logistic Regression', 2, None, '___sec0'),
('Optimization and Deep learning', 2, None, '___sec1'),
('Basics', 2, None, '___sec2'),
('Linear classifier', 2, None, '___sec3'),
('Some selected properties', 2, None, '___sec4'),
('The logistic function', 2, None, '___sec5'),
('Two parameters', 2, None, '___sec6'),
('Maximum likelihood', 2, None, '___sec7'),
('The cost function rewritten', 2, None, '___sec8'),
('Minimizing the cross entropy', 2, None, '___sec9'),
('A more compact expression', 2, None, '___sec10'),
('Extending to more predictors', 2, None, '___sec11'),
('Including more classes', 2, None, '___sec12'),
('The Softmax function', 2, None, '___sec13'),
('A _scikit-learn_ example', 2, None, '___sec14'),
('A simple classification problem', 2, None, '___sec15'),
('The two-dimensional Ising model, Predicting phase transition '
'of the two-dimensional Ising model',
2,
None,
'___sec16'),
('Reading in the data', 2, None, '___sec17'),
('Logistic regression', 2, None, '___sec18'),
('Exploring the logistic regression', 2, None, '___sec19'),
('Accuracy of a classification model', 2, None, '___sec20'),
('Analyzing the results', 2, None, '___sec21')]}
end of tocinfo -->
<body>
<script type="text/x-mathjax-config">
MathJax.Hub.Config({
TeX: {
equationNumbers: { autoNumber: "none" },
extensions: ["AMSmath.js", "AMSsymbols.js", "autobold.js", "color.js"]
}
});
</script>
<script type="text/javascript" async
src="https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.1/MathJax.js?config=TeX-AMS-MML_HTMLorMML">
</script>
<!-- Bootstrap navigation bar -->
<div class="navbar navbar-default navbar-fixed-top">
<div class="navbar-header">
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".navbar-responsive-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="LogReg-bs.html">Data Analysis and Machine Learning: Logistic Regression</a>
</div>
<div class="navbar-collapse collapse navbar-responsive-collapse">
<ul class="nav navbar-nav navbar-right">
<li class="dropdown">
<a href="#" class="dropdown-toggle" data-toggle="dropdown">Contents <b class="caret"></b></a>
<ul class="dropdown-menu">
<!-- navigation toc: --> <li><a href="._LogReg-bs001.html#___sec0" style="font-size: 80%;">Logistic Regression</a></li>
<!-- navigation toc: --> <li><a href="._LogReg-bs002.html#___sec1" style="font-size: 80%;">Optimization and Deep learning</a></li>
<!-- navigation toc: --> <li><a href="._LogReg-bs003.html#___sec2" style="font-size: 80%;">Basics</a></li>
<!-- navigation toc: --> <li><a href="._LogReg-bs004.html#___sec3" style="font-size: 80%;">Linear classifier</a></li>
<!-- navigation toc: --> <li><a href="._LogReg-bs005.html#___sec4" style="font-size: 80%;">Some selected properties</a></li>
<!-- navigation toc: --> <li><a href="._LogReg-bs006.html#___sec5" style="font-size: 80%;">The logistic function</a></li>
<!-- navigation toc: --> <li><a href="._LogReg-bs007.html#___sec6" style="font-size: 80%;">Two parameters</a></li>
<!-- navigation toc: --> <li><a href="._LogReg-bs008.html#___sec7" style="font-size: 80%;">Maximum likelihood</a></li>
<!-- navigation toc: --> <li><a href="._LogReg-bs009.html#___sec8" style="font-size: 80%;">The cost function rewritten</a></li>
<!-- navigation toc: --> <li><a href="._LogReg-bs010.html#___sec9" style="font-size: 80%;">Minimizing the cross entropy</a></li>
<!-- navigation toc: --> <li><a href="._LogReg-bs011.html#___sec10" style="font-size: 80%;">A more compact expression</a></li>
<!-- navigation toc: --> <li><a href="._LogReg-bs012.html#___sec11" style="font-size: 80%;">Extending to more predictors</a></li>
<!-- navigation toc: --> <li><a href="._LogReg-bs013.html#___sec12" style="font-size: 80%;">Including more classes</a></li>
<!-- navigation toc: --> <li><a href="#___sec13" style="font-size: 80%;">The Softmax function</a></li>
<!-- navigation toc: --> <li><a href="._LogReg-bs015.html#___sec14" style="font-size: 80%;">A <b>scikit-learn</b> example</a></li>
<!-- navigation toc: --> <li><a href="._LogReg-bs016.html#___sec15" style="font-size: 80%;">A simple classification problem</a></li>
<!-- navigation toc: --> <li><a href="._LogReg-bs017.html#___sec16" style="font-size: 80%;">The two-dimensional Ising model, Predicting phase transition of the two-dimensional Ising model</a></li>
<!-- navigation toc: --> <li><a href="._LogReg-bs018.html#___sec17" style="font-size: 80%;">Reading in the data</a></li>
<!-- navigation toc: --> <li><a href="._LogReg-bs019.html#___sec18" style="font-size: 80%;">Logistic regression</a></li>
<!-- navigation toc: --> <li><a href="._LogReg-bs020.html#___sec19" style="font-size: 80%;">Exploring the logistic regression</a></li>
<!-- navigation toc: --> <li><a href="._LogReg-bs021.html#___sec20" style="font-size: 80%;">Accuracy of a classification model</a></li>
<!-- navigation toc: --> <li><a href="._LogReg-bs022.html#___sec21" style="font-size: 80%;">Analyzing the results</a></li>
</ul>
</li>
</ul>
</div>
</div>
</div> <!-- end of navigation bar -->
<div class="container">
<p>&nbsp;</p><p>&nbsp;</p><p>&nbsp;</p> <!-- add vertical space -->
<a name="part0014"></a>
<!-- !split -->
<h2 id="___sec13" class="anchor">The Softmax function </h2>
<p>
In our discussion of neural networks we will encounter the above again in terms of the so-called <b>Softmax</b> function.
<p>
The softmax function is used in various multiclass classification
methods, such as multinomial logistic regression (also known as
softmax regression), multiclass linear discriminant
analysis, naive Bayes classifiers, and artificial neural networks.
Specifically, in multinomial logistic regression and linear
discriminant analysis, the input to the function is the result of \( K \)
distinct linear functions, and the predicted probability for the \( k \)-th
class given a sample vector \( \hat{x} \) and a weighting vector \( \hat{\beta} \) is (with two predictors):
$$
p(C=k\vert \mathbf {x} )=\frac{\exp{(\beta_{k0}+\beta_{k1}x_1)}}{1+\sum_{l=1}^{K-1}\exp{(\beta_{l0}+\beta_{l1}x_1)}}.
$$
It is easy to extend to more predictors. The final class is
$$
p(C=K\vert \mathbf {x} )=\frac{1}{1+\sum_{l=1}^{K-1}\exp{(\beta_{l0}+\beta_{l1}x_1)}},
$$
and they sum to one. Our earlier discussions were all specialized to the case with two classes only. It is easy to see from the above that what we derived earlier is compatible with these equations.
<p>
To find the optimal parameters we would typically use a gradient descent method.
Newton's method and gradient descent methods are discussed in the material on <a href="https://compphysics.github.io/MachineLearning/doc/pub/Splines/html/Splines-bs.html" target="_self">optimization methods</a>.
<p>
<p>
<!-- navigation buttons at the bottom of the page -->
<ul class="pagination">
<li><a href="._LogReg-bs013.html">&laquo;</a></li>
<li><a href="._LogReg-bs000.html">1</a></li>
<li><a href="">...</a></li>
<li><a href="._LogReg-bs006.html">7</a></li>
<li><a href="._LogReg-bs007.html">8</a></li>
<li><a href="._LogReg-bs008.html">9</a></li>
<li><a href="._LogReg-bs009.html">10</a></li>
<li><a href="._LogReg-bs010.html">11</a></li>
<li><a href="._LogReg-bs011.html">12</a></li>
<li><a href="._LogReg-bs012.html">13</a></li>
<li><a href="._LogReg-bs013.html">14</a></li>
<li class="active"><a href="._LogReg-bs014.html">15</a></li>
<li><a href="._LogReg-bs015.html">16</a></li>
<li><a href="._LogReg-bs016.html">17</a></li>
<li><a href="._LogReg-bs017.html">18</a></li>
<li><a href="._LogReg-bs018.html">19</a></li>
<li><a href="._LogReg-bs019.html">20</a></li>
<li><a href="._LogReg-bs020.html">21</a></li>
<li><a href="._LogReg-bs021.html">22</a></li>
<li><a href="._LogReg-bs022.html">23</a></li>
<li><a href="._LogReg-bs015.html">&raquo;</a></li>
</ul>
<!-- ------------------- end of main content --------------- -->
</div> <!-- end container -->
<!-- include javascript, jQuery *first* -->
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.10.2/jquery.min.js"></script>
<script src="https://netdna.bootstrapcdn.com/bootstrap/3.0.0/js/bootstrap.min.js"></script>
<!-- Bootstrap footer
<footer>
<a href="http://..."><img width="250" align=right src="http://..."></a>
</footer>
-->
<center style="font-size:80%">
<!-- copyright only on the titlepage -->
</center>
</body>
</html>
Reference in New Issue View Git Blame Copy Permalink