lars/FYS-STK4155
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

added zoo data

Browse Source
This commit is contained in:
mhjensen
2018-11-09 05:21:24 +01:00
parent 82f7105add
commit 47bf3582ee
9 changed files with 254 additions and 72 deletions
Download Patch File Download Diff File Expand all files Collapse all files
doc/pub/DecisionTrees/html/DecisionTrees.html
+53 -17
View File
@@ -81,11 +81,13 @@ div { text-align: justify; text-justify: inter-word; }
('A schematic procedure', 2, None, '___sec10'),
('A classification tree', 2, None, '___sec11'),
('Growing a classification tree', 2, None, '___sec12'),
('Pros and cons of trees, pros', 2, None, '___sec13'),
('Disadvantages', 2, None, '___sec14'),
('Bagging', 2, None, '___sec15'),
('Random forests', 2, None, '___sec16'),
('A simple scikit-learn example', 2, None, '___sec17')]}
('The zoo data', 2, None, '___sec13'),
('Pros and cons of trees, pros', 2, None, '___sec14'),
('Disadvantages', 2, None, '___sec15'),
('Bagging', 2, None, '___sec16'),
('Random forests', 2, None, '___sec17'),
('A simple scikit-learn example', 2, None, '___sec18'),
('Boosting', 2, None, '___sec19')]}
end of tocinfo -->
<body>
@@ -526,7 +528,7 @@ $$
is as small as possible. Here \( \overline{T} \) is
the number of terminal nodes of the tree \( T \) , \( R_m \) is the
rectangle (i.e. the subset of predictor space) corresponding to the $m$th terminal node.
rectangle (i.e. the subset of predictor space) corresponding to the \( m \)-th terminal node.
<p>
The tuning parameter \( \alpha \) controls a trade-off between the subtree&#8217;s
@@ -557,14 +559,16 @@ subtree corresponding to \( \alpha \).
<p>
<ol>
<li> Use recursive binary splitting to grow a large tree on the training data, stopping only when each terminal node has fewer than some minimum number of observations.
<ol type="a"></li>
<li> Use recursive binary splitting to grow a large tree on the training data, stopping only when each terminal node has fewer than some minimum number of observations.</li>
<li> Apply cost complexity pruning to the large tree in order to obtain a sequence of best subtrees, as a function of \( \alpha \).</li>
</ol>
<li> Use for example \( K \)-fold cross-validation to choose \( \alpha \). Divide the training observations into \( K \) folds. For each \( k=1,2,\dots,K \) we:</li>
<ul>
<li> repeat steps 1 and 2 on all but the \( k \)-th fold of the training data.</li>
<li> Then we valuate the mean squared prediction error on the data in the left-out \( k \)-th fold, as a function of \( \alpha \).</li>
<li> Finally we average the results for each value of \( alpha \), and pick \( \alpha \) to minimize the average error.</li>
</ul>
<li> Use for example \( K \)-fold cross-validation to choose \( \alpha \). Divide the training observations into \( K \) folds. For each \( k=1,2,\dots,K \) we repeat Steps 1 and 2 on all but the $k$th fold of the training data. Then we valuate the mean squared prediction error on the data in the left-out $k$th fold, as a function of \( \alpha \).</li>
<li> Then we average the results for each value of \( alpha \), and pick \( \alpha \) to minimize the average error.</li>
<li> Return the subtree from Step 2 that corresponds to the chosen value of \( \alpha \).</li>
</ol>
</div>
@@ -616,7 +620,35 @@ than is the classification error rate.
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec13">Pros and cons of trees, pros </h2>
<h2 id="___sec13">The zoo data </h2>
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
<div class="highlight" style="background: #f8f8f8"><pre style="line-height: 125%"><span></span><span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">pandas</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">pd</span>
<span style="color: #008000; font-weight: bold">import</span> <span style="color: #0000FF; font-weight: bold">numpy</span> <span style="color: #008000; font-weight: bold">as</span> <span style="color: #0000FF; font-weight: bold">np</span>
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">pprint</span> <span style="color: #008000; font-weight: bold">import</span> pprint
<span style="color: #008000; font-weight: bold">from</span> <span style="color: #0000FF; font-weight: bold">sklearn.tree</span> <span style="color: #008000; font-weight: bold">import</span> DecisionTreeClassifier
<span style="color: #408080; font-style: italic">#Import the dataset </span>
dataset <span style="color: #666666">=</span> pd<span style="color: #666666">.</span>read_csv(<span style="color: #BA2121">&#39;data/zoo.csv&#39;</span>)
<span style="color: #408080; font-style: italic">#We drop the animal names since this is not a good feature to split the data on</span>
<span style="color: #408080; font-style: italic">#dataset=dataset.drop(&#39;animal_name&#39;,axis=1)</span>
<span style="color: #408080; font-style: italic">#Split the data into a training and a testing set</span>
train_features <span style="color: #666666">=</span> dataset<span style="color: #666666">.</span>iloc[:<span style="color: #666666">80</span>,:<span style="color: #666666">-1</span>]
test_features <span style="color: #666666">=</span> dataset<span style="color: #666666">.</span>iloc[<span style="color: #666666">80</span>:,:<span style="color: #666666">-1</span>]
train_targets <span style="color: #666666">=</span> dataset<span style="color: #666666">.</span>iloc[:<span style="color: #666666">80</span>,<span style="color: #666666">-1</span>]
test_targets <span style="color: #666666">=</span> dataset<span style="color: #666666">.</span>iloc[<span style="color: #666666">80</span>:,<span style="color: #666666">-1</span>]
<span style="color: #408080; font-style: italic">#Train the model</span>
tree <span style="color: #666666">=</span> DecisionTreeClassifier(criterion <span style="color: #666666">=</span> <span style="color: #BA2121">&#39;entropy&#39;</span>)<span style="color: #666666">.</span>fit(train_features,train_targets)
<span style="color: #408080; font-style: italic">#Predict the classes of new, unseen data</span>
prediction <span style="color: #666666">=</span> tree<span style="color: #666666">.</span>predict(test_features)
<span style="color: #408080; font-style: italic">#Check the accuracy</span>
<span style="color: #008000; font-weight: bold">print</span>(<span style="color: #BA2121">&quot;The prediction accuracy is: &quot;</span>,tree<span style="color: #666666">.</span>score(test_features,test_targets)<span style="color: #666666">*100</span>,<span style="color: #BA2121">&quot;%&quot;</span>)
</pre></div>
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec14">Pros and cons of trees, pros </h2>
<ul>
<li> White box, easy to interpret model. Some people believe that decision trees more closely mirror human decision-making than do the regression and classification approaches discussed earlier (think of support vector machines)</li>
@@ -630,7 +662,7 @@ than is the classification error rate.
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec14">Disadvantages </h2>
<h2 id="___sec15">Disadvantages </h2>
<ul>
<li> Unfortunately, trees generally do not have the same level of predictive accuracy as some of the other regression and classification approaches</li>
@@ -647,7 +679,7 @@ However, by aggregating many decision trees, using methods like bagging, random
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec15">Bagging </h2>
<h2 id="___sec16">Bagging </h2>
<p>
The <b>plain</b> decision trees suffer from high
@@ -690,7 +722,7 @@ predictor, averaged over all \( B \) trees.
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec16">Random forests </h2>
<h2 id="___sec17">Random forests </h2>
<p>
Random forests provide an improvement over bagged trees by way of a
@@ -732,7 +764,7 @@ setting.
<p>
<!-- !split --><br><br><br><br><br><br><br><br><br><br>
<h2 id="___sec17">A simple scikit-learn example </h2>
<h2 id="___sec18">A simple scikit-learn example </h2>
<p>
<!-- code=python (!bc pycod) typeset with pygments style "default" -->
@@ -748,6 +780,10 @@ Random_Forest_model <span style="color: #666666">=</span> RandomForestClassifier
accuracy <span style="color: #666666">=</span> cross_validate(Random_Forest_model,X,Y,cv<span style="color: #666666">=10</span>)[<span style="color: #BA2121">&#39;test_score&#39;</span>]
</pre></div>
<p>
<!-- !split -->
<h2 id="___sec19">Boosting </h2>
More material to come here.
<!-- ------------------- end of main content --------------- -->