lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

update

Browse Source
This commit is contained in:
Morten Hjorth-Jensen
2023-08-31 06:19:44 +02:00
parent e9bac09427
commit d05f791ff6
98 changed files with 3861 additions and 2496 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/LectureNotes/_build/html/chapter6.html
+177 -12
View File
@@ -242,6 +242,33 @@ const thebe_selector_output = ".output, .cell_output"
</a>
</li>
</ul>
<p aria-level="2" class="caption" role="heading">
<span class="caption-text">
Weekly material, notes and exercises
</span>
</p>
<ul class="nav bd-sidenav">
<li class="toctree-l1">
<a class="reference internal" href="exercisesweek34.html">
Exercises week 34
</a>
</li>
<li class="toctree-l1">
<a class="reference internal" href="week34.html">
Week 34: Introduction to the course, Logistics and Practicalities
</a>
</li>
<li class="toctree-l1">
<a class="reference internal" href="exercisesweek35.html">
Exercises week 35
</a>
</li>
<li class="toctree-l1">
<a class="reference internal" href="week35.html">
Week 35: From Ordinary Linear Regression to Ridge and Lasso Regression
</a>
</li>
</ul>
</div>
</nav> <!-- To handle the deprecated key -->
@@ -688,9 +715,9 @@ predicting the target features of query instances is as follows:</p>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>2nd degree coefficients:
zero power: 0.18790439176058887
first power: -0.014599964106338128
second power: 0.00010403373827253124
zero power: -4.653578701904388
first power: 0.17297886491529482
second power: -0.0007790285013223805
</pre></div>
</div>
<img alt="_images/chapter6_1_1.png" src="_images/chapter6_1_1.png" />
@@ -923,16 +950,102 @@ s = -\sum_{k=1}^K p_{mk}\log{p_{mk}}.
</div>
</div>
<div class="cell_output docutils container">
<div class="output traceback highlight-ipythontb notranslate"><div class="highlight"><pre><span></span><span class="gt">---------------------------------------------------------------------------</span>
<span class="ne">ModuleNotFoundError</span><span class="g g-Whitespace"> </span>Traceback (most recent call last)
<span class="nn">Input In [2],</span> in <span class="ni">&lt;cell line: 9&gt;</span><span class="nt">()</span>
<span class="g g-Whitespace"> </span><span class="mi">6</span> <span class="kn">from</span> <span class="nn">sklearn.tree</span> <span class="kn">import</span> <span class="n">export_graphviz</span>
<span class="g g-Whitespace"> </span><span class="mi">8</span> <span class="kn">from</span> <span class="nn">IPython.display</span> <span class="kn">import</span> <span class="n">Image</span>
<span class="ne">----&gt; </span><span class="mi">9</span> <span class="kn">from</span> <span class="nn">pydot</span> <span class="kn">import</span> <span class="n">graph_from_dot_data</span>
<span class="g g-Whitespace"> </span><span class="mi">10</span> <span class="kn">import</span> <span class="nn">pandas</span> <span class="k">as</span> <span class="nn">pd</span>
<span class="g g-Whitespace"> </span><span class="mi">11</span> <span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="nn">np</span>
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span> mean radius mean texture mean perimeter mean area mean smoothness \
0 17.99 10.38 122.80 1001.0 0.11840
1 20.57 17.77 132.90 1326.0 0.08474
2 19.69 21.25 130.00 1203.0 0.10960
3 11.42 20.38 77.58 386.1 0.14250
4 20.29 14.34 135.10 1297.0 0.10030
.. ... ... ... ... ...
564 21.56 22.39 142.00 1479.0 0.11100
565 20.13 28.25 131.20 1261.0 0.09780
566 16.60 28.08 108.30 858.1 0.08455
567 20.60 29.33 140.10 1265.0 0.11780
568 7.76 24.54 47.92 181.0 0.05263
<span class="ne">ModuleNotFoundError</span>: No module named &#39;pydot&#39;
mean compactness mean concavity mean concave points mean symmetry \
0 0.27760 0.30010 0.14710 0.2419
1 0.07864 0.08690 0.07017 0.1812
2 0.15990 0.19740 0.12790 0.2069
3 0.28390 0.24140 0.10520 0.2597
4 0.13280 0.19800 0.10430 0.1809
.. ... ... ... ...
564 0.11590 0.24390 0.13890 0.1726
565 0.10340 0.14400 0.09791 0.1752
566 0.10230 0.09251 0.05302 0.1590
567 0.27700 0.35140 0.15200 0.2397
568 0.04362 0.00000 0.00000 0.1587
mean fractal dimension ... worst radius worst texture \
0 0.07871 ... 25.380 17.33
1 0.05667 ... 24.990 23.41
2 0.05999 ... 23.570 25.53
3 0.09744 ... 14.910 26.50
4 0.05883 ... 22.540 16.67
.. ... ... ... ...
564 0.05623 ... 25.450 26.40
565 0.05533 ... 23.690 38.25
566 0.05648 ... 18.980 34.12
567 0.07016 ... 25.740 39.42
568 0.05884 ... 9.456 30.37
worst perimeter worst area worst smoothness worst compactness \
0 184.60 2019.0 0.16220 0.66560
1 158.80 1956.0 0.12380 0.18660
2 152.50 1709.0 0.14440 0.42450
3 98.87 567.7 0.20980 0.86630
4 152.20 1575.0 0.13740 0.20500
.. ... ... ... ...
564 166.10 2027.0 0.14100 0.21130
565 155.00 1731.0 0.11660 0.19220
566 126.70 1124.0 0.11390 0.30940
567 184.60 1821.0 0.16500 0.86810
568 59.16 268.6 0.08996 0.06444
worst concavity worst concave points worst symmetry \
0 0.7119 0.2654 0.4601
1 0.2416 0.1860 0.2750
2 0.4504 0.2430 0.3613
3 0.6869 0.2575 0.6638
4 0.4000 0.1625 0.2364
.. ... ... ...
564 0.4107 0.2216 0.2060
565 0.3215 0.1628 0.2572
566 0.3403 0.1418 0.2218
567 0.9387 0.2650 0.4087
568 0.0000 0.0000 0.2871
worst fractal dimension
0 0.11890
1 0.08902
2 0.08758
3 0.17300
4 0.07678
.. ...
564 0.07115
565 0.06637
566 0.07820
567 0.12400
568 0.07039
[569 rows x 30 columns]
malignant benign
0 1 0
1 1 0
2 1 0
3 1 0
4 1 0
.. ... ...
564 1 0
565 1 0
566 1 0
567 1 0
568 0 1
[569 rows x 2 columns]
</pre></div>
</div>
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0
</pre></div>
</div>
</div>
@@ -966,6 +1079,11 @@ s = -\sum_{k=1}^K p_{mk}\log{p_{mk}}.
</pre></div>
</div>
</div>
<div class="cell_output docutils container">
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>0
</pre></div>
</div>
</div>
</div>
</div>
<div class="section" id="other-ways-of-visualizing-the-trees">
@@ -983,6 +1101,28 @@ s = -\sum_{k=1}^K p_{mk}\log{p_{mk}}.
</pre></div>
</div>
</div>
<div class="cell_output docutils container">
<div class="output text_plain highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>[Text(0.5, 0.9166666666666666, &#39;X[2] &lt;= 2.45\ngini = 0.667\nsamples = 150\nvalue = [50, 50, 50]&#39;),
Text(0.4230769230769231, 0.75, &#39;gini = 0.0\nsamples = 50\nvalue = [50, 0, 0]&#39;),
Text(0.5769230769230769, 0.75, &#39;X[3] &lt;= 1.75\ngini = 0.5\nsamples = 100\nvalue = [0, 50, 50]&#39;),
Text(0.3076923076923077, 0.5833333333333334, &#39;X[2] &lt;= 4.95\ngini = 0.168\nsamples = 54\nvalue = [0, 49, 5]&#39;),
Text(0.15384615384615385, 0.4166666666666667, &#39;X[3] &lt;= 1.65\ngini = 0.041\nsamples = 48\nvalue = [0, 47, 1]&#39;),
Text(0.07692307692307693, 0.25, &#39;gini = 0.0\nsamples = 47\nvalue = [0, 47, 0]&#39;),
Text(0.23076923076923078, 0.25, &#39;gini = 0.0\nsamples = 1\nvalue = [0, 0, 1]&#39;),
Text(0.46153846153846156, 0.4166666666666667, &#39;X[3] &lt;= 1.55\ngini = 0.444\nsamples = 6\nvalue = [0, 2, 4]&#39;),
Text(0.38461538461538464, 0.25, &#39;gini = 0.0\nsamples = 3\nvalue = [0, 0, 3]&#39;),
Text(0.5384615384615384, 0.25, &#39;X[2] &lt;= 5.45\ngini = 0.444\nsamples = 3\nvalue = [0, 2, 1]&#39;),
Text(0.46153846153846156, 0.08333333333333333, &#39;gini = 0.0\nsamples = 2\nvalue = [0, 2, 0]&#39;),
Text(0.6153846153846154, 0.08333333333333333, &#39;gini = 0.0\nsamples = 1\nvalue = [0, 0, 1]&#39;),
Text(0.8461538461538461, 0.5833333333333334, &#39;X[2] &lt;= 4.85\ngini = 0.043\nsamples = 46\nvalue = [0, 1, 45]&#39;),
Text(0.7692307692307693, 0.4166666666666667, &#39;X[1] &lt;= 3.1\ngini = 0.444\nsamples = 3\nvalue = [0, 1, 2]&#39;),
Text(0.6923076923076923, 0.25, &#39;gini = 0.0\nsamples = 2\nvalue = [0, 0, 2]&#39;),
Text(0.8461538461538461, 0.25, &#39;gini = 0.0\nsamples = 1\nvalue = [0, 1, 0]&#39;),
Text(0.9230769230769231, 0.4166666666666667, &#39;gini = 0.0\nsamples = 43\nvalue = [0, 0, 43]&#39;)]
</pre></div>
</div>
<img alt="_images/chapter6_24_1.png" src="_images/chapter6_24_1.png" />
</div>
</div>
<p>Alternatively, the tree can also be exported in textual format with the function exporttext.
This method doesn’t require the installation of external libraries and is more compact:</p>
@@ -999,6 +1139,17 @@ This method doesn’t require the installation of external libraries and is more
</pre></div>
</div>
</div>
<div class="cell_output docutils container">
<div class="output stream highlight-myst-ansi notranslate"><div class="highlight"><pre><span></span>|--- petal width (cm) &lt;= 0.80
| |--- class: 0
|--- petal width (cm) &gt; 0.80
| |--- petal width (cm) &lt;= 1.75
| | |--- class: 1
| |--- petal width (cm) &gt; 1.75
| | |--- class: 2
</pre></div>
</div>
</div>
</div>
</div>
</div>
@@ -1163,6 +1314,20 @@ humidity and weak and strong for wind.</p>
</pre></div>
</div>
</div>
<div class="cell_output docutils container">
<div class="output traceback highlight-ipythontb notranslate"><div class="highlight"><pre><span></span><span class="gt">---------------------------------------------------------------------------</span>
<span class="ne">FileNotFoundError</span><span class="g g-Whitespace"> </span>Traceback (most recent call last)
<span class="nn">Input In [6],</span> in <span class="ni">&lt;cell line: 37&gt;</span><span class="nt">()</span>
<span class="g g-Whitespace"> </span><span class="mi">34</span> <span class="k">def</span> <span class="nf">save_fig</span><span class="p">(</span><span class="n">fig_id</span><span class="p">):</span>
<span class="g g-Whitespace"> </span><span class="mi">35</span> <span class="n">plt</span><span class="o">.</span><span class="n">savefig</span><span class="p">(</span><span class="n">image_path</span><span class="p">(</span><span class="n">fig_id</span><span class="p">)</span> <span class="o">+</span> <span class="s2">&quot;.png&quot;</span><span class="p">,</span> <span class="nb">format</span><span class="o">=</span><span class="s1">&#39;png&#39;</span><span class="p">)</span>
<span class="ne">---&gt; </span><span class="mi">37</span> <span class="n">infile</span> <span class="o">=</span> <span class="nb">open</span><span class="p">(</span><span class="n">data_path</span><span class="p">(</span><span class="s2">&quot;rideclass.csv&quot;</span><span class="p">),</span><span class="s1">&#39;r&#39;</span><span class="p">)</span>
<span class="g g-Whitespace"> </span><span class="mi">39</span> <span class="c1"># Read the experimental data with Pandas</span>
<span class="g g-Whitespace"> </span><span class="mi">40</span> <span class="kn">from</span> <span class="nn">IPython.display</span> <span class="kn">import</span> <span class="n">display</span>
<span class="ne">FileNotFoundError</span>: [Errno 2] No such file or directory: &#39;DataFiles/rideclass.csv&#39;
</pre></div>
</div>
</div>
</div>
<p>The above functions (gini, entropy and misclassification error) are
important components of the so-called CART algorithm. We will discuss