lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

update of notes

Browse Source
This commit is contained in:
Morten Hjorth-Jensen
2022-12-11 22:07:05 +01:00
parent 8051880d95
commit 5473efe28f
70 changed files with 3777 additions and 1484 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/LectureNotes/_build/html/chapter4.html
+32 -26
View File
@@ -328,8 +328,8 @@ const thebe_selector_output = ".output, .cell_output"
</a>
</li>
<li class="toc-h2 nav-item toc-entry">
<a class="reference internal nav-link" href="#examples-of-likelihood-functions-used-in-logistic-regression-and-nueral-networks">
6.4. Examples of likelihood functions used in logistic regression and nueral networks
<a class="reference internal nav-link" href="#examples-of-likelihood-functions-used-in-logistic-regression-and-neural-networks">
6.4. Examples of likelihood functions used in logistic regression and neural networks
</a>
</li>
<li class="toc-h2 nav-item toc-entry">
@@ -373,8 +373,8 @@ const thebe_selector_output = ".output, .cell_output"
</a>
</li>
<li class="toc-h2 nav-item toc-entry">
<a class="reference internal nav-link" href="#examples-of-likelihood-functions-used-in-logistic-regression-and-nueral-networks">
6.4. Examples of likelihood functions used in logistic regression and nueral networks
<a class="reference internal nav-link" href="#examples-of-likelihood-functions-used-in-logistic-regression-and-neural-networks">
6.4. Examples of likelihood functions used in logistic regression and neural networks
</a>
</li>
<li class="toc-h2 nav-item toc-entry">
@@ -391,7 +391,8 @@ const thebe_selector_output = ".output, .cell_output"
<div>
<div class="tex2jax_ignore mathjax_ignore section" id="logistic-regression">
<!-- HTML file automatically generated from DocOnce source (https://github.com/doconce/doconce/)
doconce format html chapter4.do.txt --><div class="tex2jax_ignore mathjax_ignore section" id="logistic-regression">
<h1><span class="section-number">6. </span>Logistic Regression<a class="headerlink" href="#logistic-regression" title="Permalink to this headline"></a></h1>
<div class="section" id="id1">
<h2><span class="section-number">6.1. </span>Logistic Regression<a class="headerlink" href="#id1" title="Permalink to this headline"></a></h2>
@@ -647,7 +648,7 @@ the probability of a given category. This leads us to the logistic function.</p>
</tbody>
</table>
<p>100 rows × 4 columns</p>
</div></div><img alt="_images/chapter4_5_1.png" src="_images/chapter4_5_1.png" />
</div></div><img alt="_images/chapter4_8_1.png" src="_images/chapter4_8_1.png" />
</div>
</div>
<p>What we could attempt however is to plot the mean value for each group.</p>
@@ -665,7 +666,7 @@ the probability of a given category. This leads us to the logistic function.</p>
</div>
</div>
<div class="cell_output docutils container">
<img alt="_images/chapter4_7_0.png" src="_images/chapter4_7_0.png" />
<img alt="_images/chapter4_10_0.png" src="_images/chapter4_10_0.png" />
</div>
</div>
<p>We are now trying to find a function <span class="math notranslate nohighlight">\(f(y\vert x)\)</span>, that is a function which gives us an expected value for the output <span class="math notranslate nohighlight">\(y\)</span> with a given input <span class="math notranslate nohighlight">\(x\)</span>.
@@ -704,8 +705,8 @@ p(t) = \frac{1}{1+\mathrm \exp{-t}}=\frac{\exp{t}}{1+\mathrm \exp{t}}.
\]</div>
<p>Note that <span class="math notranslate nohighlight">\(1-p(t)= p(-t)\)</span>.</p>
</div>
<div class="section" id="examples-of-likelihood-functions-used-in-logistic-regression-and-nueral-networks">
<h2><span class="section-number">6.4. </span>Examples of likelihood functions used in logistic regression and nueral networks<a class="headerlink" href="#examples-of-likelihood-functions-used-in-logistic-regression-and-nueral-networks" title="Permalink to this headline"></a></h2>
<div class="section" id="examples-of-likelihood-functions-used-in-logistic-regression-and-neural-networks">
<h2><span class="section-number">6.4. </span>Examples of likelihood functions used in logistic regression and neural networks<a class="headerlink" href="#examples-of-likelihood-functions-used-in-logistic-regression-and-neural-networks" title="Permalink to this headline"></a></h2>
<p>The following code plots the logistic function, the step function and other functions we will encounter from here and on.</p>
<div class="cell docutils container">
<div class="cell_input docutils container">
@@ -767,9 +768,9 @@ p(t) = \frac{1}{1+\mathrm \exp{-t}}=\frac{\exp{t}}{1+\mathrm \exp{t}}.
</div>
</div>
<div class="cell_output docutils container">
<img alt="_images/chapter4_13_0.png" src="_images/chapter4_13_0.png" />
<img alt="_images/chapter4_13_1.png" src="_images/chapter4_13_1.png" />
<img alt="_images/chapter4_13_2.png" src="_images/chapter4_13_2.png" />
<img alt="_images/chapter4_18_0.png" src="_images/chapter4_18_0.png" />
<img alt="_images/chapter4_18_1.png" src="_images/chapter4_18_1.png" />
<img alt="_images/chapter4_18_2.png" src="_images/chapter4_18_2.png" />
</div>
</div>
<p>We assume now that we have two classes with <span class="math notranslate nohighlight">\(y_i\)</span> either <span class="math notranslate nohighlight">\(0\)</span> or <span class="math notranslate nohighlight">\(1\)</span>. Furthermore we assume also that we have only two parameters <span class="math notranslate nohighlight">\(\beta\)</span> in our fitting of the Sigmoid function, that is we define probabilities</p>
@@ -996,8 +997,8 @@ We use <strong>Pandas</strong> to compute the correlation matrix.</p>
</div>
</div>
<div class="cell_output docutils container">
<img alt="_images/chapter4_51_0.png" src="_images/chapter4_51_0.png" />
<img alt="_images/chapter4_51_1.png" src="_images/chapter4_51_1.png" />
<img alt="_images/chapter4_57_0.png" src="_images/chapter4_57_0.png" />
<img alt="_images/chapter4_57_1.png" src="_images/chapter4_57_1.png" />
</div>
</div>
<p>In the above example we note two things. In the first plot we display
@@ -1030,6 +1031,20 @@ matrix.</p>
features are of relevance and which are not. This leads us to
the classical Principal Component Analysis (PCA) theorem with
applications. This will be discussed later this semester (<a class="reference external" href="https://compphysics.github.io/MachineLearning/doc/pub/week43/html/week43-bs.html">week 43</a>).</p>
<p>Here we present a further way to present our results in terms of a so-called <strong>confusion matrix</strong>, the cumulative gain and the <strong>ROC</strong> curve.
This way of displaying our data are based upon different ways to classify our possible outcomes. Before we proceed we need some definitions.</p>
<ol class="simple">
<li><p><strong>TP</strong>: true positive or in other words, something equivalent with a proper classification</p></li>
<li><p><strong>TN</strong>: true negative, which is equivalent with a correct rejection</p></li>
<li><p><strong>FP</strong>: false positive, or in simpler words something that is equivalent with a false alarm</p></li>
<li><p><strong>FN</strong>: false negative, which is mean to be equivalent with a miss.</p></li>
</ol>
<p>The total data set is then the sum of the true positive and true negative targets or outputs, labeled by <span class="math notranslate nohighlight">\(n\)</span>.
Based on this we can then define the accuracy score as the sum of correctly predicted <strong>TP</strong> and <strong>TN</strong> cases divided by the sum of true positive and treue negative events in our data set, or as</p>
<div class="math notranslate nohighlight">
\[
\mathrm{Accuracy} = \frac{\sum_{i=0}^{n-1}I(y_i=\tilde{y}_i)}{n}.
\]</div>
<div class="cell docutils container">
<div class="cell_input docutils container">
<div class="highlight-ipython3 notranslate"><div class="highlight"><pre><span></span><span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="k">as</span> <span class="nn">plt</span>
@@ -1099,18 +1114,9 @@ Please also refer to the documentation for alternative solver options:
n_iter_i = _check_optimize_result(
</pre></div>
</div>
<div class="output traceback highlight-ipythontb notranslate"><div class="highlight"><pre><span></span><span class="gt">---------------------------------------------------------------------------</span>
<span class="ne">ModuleNotFoundError</span><span class="g g-Whitespace"> </span>Traceback (most recent call last)
<span class="nn">Input In [8],</span> in <span class="ni">&lt;cell line: 36&gt;</span><span class="nt">()</span>
<span class="g g-Whitespace"> </span><span class="mi">32</span> <span class="nb">print</span><span class="p">(</span><span class="n">accuracy</span><span class="p">)</span>
<span class="g g-Whitespace"> </span><span class="mi">33</span> <span class="nb">print</span><span class="p">(</span><span class="s2">&quot;Test set accuracy with Logistic Regression and scaled data: </span><span class="si">{:.2f}</span><span class="s2">&quot;</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">logreg</span><span class="o">.</span><span class="n">score</span><span class="p">(</span><span class="n">X_test_scaled</span><span class="p">,</span><span class="n">y_test</span><span class="p">)))</span>
<span class="ne">---&gt; </span><span class="mi">36</span> <span class="kn">import</span> <span class="nn">scikitplot</span> <span class="k">as</span> <span class="nn">skplt</span>
<span class="g g-Whitespace"> </span><span class="mi">37</span> <span class="n">y_pred</span> <span class="o">=</span> <span class="n">logreg</span><span class="o">.</span><span class="n">predict</span><span class="p">(</span><span class="n">X_test_scaled</span><span class="p">)</span>
<span class="g g-Whitespace"> </span><span class="mi">38</span> <span class="n">skplt</span><span class="o">.</span><span class="n">metrics</span><span class="o">.</span><span class="n">plot_confusion_matrix</span><span class="p">(</span><span class="n">y_test</span><span class="p">,</span> <span class="n">y_pred</span><span class="p">,</span> <span class="n">normalize</span><span class="o">=</span><span class="kc">True</span><span class="p">)</span>
<span class="ne">ModuleNotFoundError</span>: No module named &#39;scikitplot&#39;
</pre></div>
</div>
<img alt="_images/chapter4_64_2.png" src="_images/chapter4_64_2.png" />
<img alt="_images/chapter4_64_3.png" src="_images/chapter4_64_3.png" />
<img alt="_images/chapter4_64_4.png" src="_images/chapter4_64_4.png" />
</div>
</div>
</div>