diff --git a/doc/pub/DecisionTrees/html/DecisionTrees-bs.html b/doc/pub/DecisionTrees/html/DecisionTrees-bs.html index 37fe3d3ad..12855f998 100644 --- a/doc/pub/DecisionTrees/html/DecisionTrees-bs.html +++ b/doc/pub/DecisionTrees/html/DecisionTrees-bs.html @@ -56,11 +56,13 @@ Automatically generated HTML file from DocOnce source ('A schematic procedure', 2, None, '___sec10'), ('A classification tree', 2, None, '___sec11'), ('Growing a classification tree', 2, None, '___sec12'), - ('Pros and cons of trees, pros', 2, None, '___sec13'), - ('Disadvantages', 2, None, '___sec14'), - ('Bagging', 2, None, '___sec15'), - ('Random forests', 2, None, '___sec16'), - ('A simple scikit-learn example', 2, None, '___sec17')]} + ('The zoo data', 2, None, '___sec13'), + ('Pros and cons of trees, pros', 2, None, '___sec14'), + ('Disadvantages', 2, None, '___sec15'), + ('Bagging', 2, None, '___sec16'), + ('Random forests', 2, None, '___sec17'), + ('A simple scikit-learn example', 2, None, '___sec18'), + ('Boosting', 2, None, '___sec19')]} end of tocinfo --> @@ -111,11 +113,13 @@ MathJax.Hub.Config({
  • A schematic procedure
  • A classification tree
  • Growing a classification tree
    • -
  • Pros and cons of trees, pros
    • -
  • Disadvantages
    • -
  • Bagging
    • -
  • Random forests
    • -
  • A simple scikit-learn example
    • +
  • The zoo data
    • +
  • Pros and cons of trees, pros
    • +
  • Disadvantages
    • +
  • Bagging
    • +
  • Random forests
    • +
  • A simple scikit-learn example
    • +
  • Boosting
    • @@ -552,7 +556,7 @@ $$ is as small as possible. Here \( \overline{T} \) is the number of terminal nodes of the tree \( T \) , \( R_m \) is the -rectangle (i.e. the subset of predictor space) corresponding to the $m$th terminal node. +rectangle (i.e. the subset of predictor space) corresponding to the \( m \)-th terminal node.

    The tuning parameter \( \alpha \) controls a trade-off between the subtree’s @@ -583,14 +587,16 @@ subtree corresponding to \( \alpha \).

      -
    1. Use recursive binary splitting to grow a large tree on the training data, stopping only when each terminal node has fewer than some minimum number of observations. - -
        +
      1. Use recursive binary splitting to grow a large tree on the training data, stopping only when each terminal node has fewer than some minimum number of observations.
      2. Apply cost complexity pruning to the large tree in order to obtain a sequence of best subtrees, as a function of \( \alpha \).
        3. -
      +
    2. Use for example \( K \)-fold cross-validation to choose \( \alpha \). Divide the training observations into \( K \) folds. For each \( k=1,2,\dots,K \) we:
      3. + + -
    3. Use for example \( K \)-fold cross-validation to choose \( \alpha \). Divide the training observations into \( K \) folds. For each \( k=1,2,\dots,K \) we repeat Steps 1 and 2 on all but the $k$th fold of the training data. Then we valuate the mean squared prediction error on the data in the left-out $k$th fold, as a function of \( \alpha \).
      4. -
    4. Then we average the results for each value of \( alpha \), and pick \( \alpha \) to minimize the average error.
    5. Return the subtree from Step 2 that corresponds to the chosen value of \( \alpha \).
    @@ -643,7 +649,35 @@ than is the classification error rate.

    -

    Pros and cons of trees, pros

    +

    The zoo data

    +

    + + +

    import pandas as pd
+import numpy as np
+from pprint import pprint
+from sklearn.tree import DecisionTreeClassifier
+
+#Import the dataset 
+dataset = pd.read_csv('data/zoo.csv')
+#We drop the animal names since this is not a good feature to split the data on
+#dataset=dataset.drop('animal_name',axis=1)
+#Split the data into a training and a testing set
+train_features = dataset.iloc[:80,:-1]
+test_features = dataset.iloc[80:,:-1]
+train_targets = dataset.iloc[:80,-1]
+test_targets = dataset.iloc[80:,-1]
+#Train the model
+tree = DecisionTreeClassifier(criterion = 'entropy').fit(train_features,train_targets)
+#Predict the classes of new, unseen data
+prediction = tree.predict(test_features)
+#Check the accuracy
+print("The prediction accuracy is: ",tree.score(test_features,test_targets)*100,"%")
+
    +

    + + +

    Pros and cons of trees, pros